Save only if dataset was compressed

[Karl Tjensvoll]

Hi

I often deal with very large datasets and save a lot of space when compressing. Sometimes I have the same do files that sometimes need to handle a new dataset (because I updated it, or added some new columns) and therefore need to compress it again. I then -save, replace- the dataset, so that I then have a compressed data set.

I often manually compress and save when I have a new dataset, but I would like to automate it. If I understood correctly the compress check if often quite quick, but the save command may often take a lot of time. Is there a way to only have STATA save the data set if it was compressed? Something like:

compress
if compress==True:
   save, replace

[Nick Cox]

Perhaps

program mycompress 
     compress 
     if c(changed) save, replace 
end

This saves even if thecompress didn't change the dataset but something earlier did. Naturally you can check for changes made only by compress

program mycompress 
     local changed = c(changed) 
     compress 
     if `changed' == 0 & c(changed) save, replace 
end

[daniel klein]

We have had discussions about what is flagged by c(changed) and what is not. It turns out, compress never changes c(changed). Therefore, neither version of mycompress will not work as intended. I suggest relying on c(width) instead.

program savecompressed
    version 11.2
    syntax [ , REPLACE ]
    if (c(changed) & ("`replace'"!="replace")) error 4
    local width = c(width)
    compress
    if (c(width)<`width') save , replace
end

Best
Daniel

[Nick Cox]

daniel klein Good catch.

[Karl Tjensvoll]

We have had discussions about what is flagged by c(changed) and what is not. It turns out, compress never changes c(changed). Therefore, neither version of mycompress will not work as intended. I suggest relying on c(width) instead.

program savecompressed
version 11.2
syntax [ , REPLACE ]
if (c(changed) & ("`replace'"!="replace")) error 4
local width = c(width)
compress
if (c(width)<`width') save , replace
end

Best
Daniel

Excellent, I was not entirely aware of these -creturn- values, and it looks like exactly what I need. I am not familiar with the start of your program, the syntax the first if statement, but I should be safe with the program:

program define savecompressed
    local width = c(width)
    compress
    if (c(width)<`width') save, replace
end