Stata Crashed and Deleted Contents of .dta File

[CooCooCachoo]

Hey guys,

I have been compiling my own dataset for hours and hours, frequently saving my .dta file (but never creating a copy of it). While I was adding a new observation, Stata (13) crashed. I thought "this sucks, but I'll just reopen the file." But when I try to open the file, I get this message "not Stata format" (even though it's a .dta file). Then I looked at file info and it is now zero bytes, meaning that Stata erased the contents of the program.

Is there any way to retrieve the data? Anything I could do?

Thanks!

[Sarah Edgington]

Did you have a log running? If so, you can use the log to (with some effort) reconstruct what you did. Otherwise you're likely out of luck unless you have a system that does frequent automatic backups.

This is one of many reasons to do as much as possible using a dofile and log anything that can't be done in a dofile. Sadly, sometimes these sort of technical problems highlight how important it is to make sure everything we do is documented and reproducible.

[CooCooCachoo]

Yep Sarah. it's definitely a lesson learned to do more in a do-file (which I hadn't done).

[Philip Jones]

If you are using a Mac, you may be lucky enough to have a version stored in Time Machine backup. You could check there.

[Sergiy Radyakin]

This looks like already the second reported issue with data lost on a Mac without a backup (see another thread here: http://www.statalist.org/forums/foru...to-fix-my-data ). I was thinking the Mac crowd was a bit more organized .

Regardless of platform: do backups!

[Clyde Schechter]

@Sergiy,

At the risk of hijacking this thread, why did you think that the Mac crowd was more organized? If there is any group I would hold that expectation of, it would be the Unix folk. Even there, I expect the overall difference would be small, if any. So I'm curious where your expectation came from.

[Sergiy Radyakin]

@Clyde
Every time I discuss platform-specific user differences the final argument seems to be http://www.cultofmac.com/106660/iq-t...-users-report/
the dismissal to it is rarely cited however http://techland.time.com/2011/08/03/that-internet-explorer-users-are-dumber-study-bogus/
But at the latest Stata User group meeting in Boston I've seen surprisingly many Macs, with lots of serious looking people behind them. Perhaps this impression played here.


Do backups!

If someone else has experienced data loss on a Mac, let StataCorp know.

[Richard Williams]

This seems like very peculiar behavior to me. If Stata crashes, then I would expect you to get the last saved version of your dta file. The only times I can remember seeing something like this was when the hard disk itself was getting corrupted. I would run diagnostic tests on it to make sure even worse problems don't lie ahead.

[Carlo Lazzaro]

CooCooCachoo (as per FAQ, please re-register with your true name and surname),
if none of the above replied advices can solve your problem, contact Stata Tech Support.

Kind regards,
Carlo

[Konrad Zdeb]

But at the latest Stata User group meeting in Boston I've seen surprisingly many Macs, with lots of serious looking people behind them. Perhaps this impression played here.

It's a nice machine to heave, but recent idea to remove Ethernet connection from MBP puts me off. In addition constant need to visualise is a necessary and unpleasant drain on resources. If my work would be only concerned with writing stuff in R then I would rather work on Mac as RStudio looks beautiful on Mac.

If someone else has experienced data loss on a Mac, let StataCorp know.

Never happened to me and I've been using different versions of Stata on Mac for years.

[Clyde Schechter]

I have never irreversibly lost data using Stata, but only because I never create a data set that I might need again without immediately backing it up. But I have had Stata clobber data sets due to its unsafe feature of allowing reference to undefined macros. For example, if attempting to build up a file by appending a bunch of data sets:

clear
tempfile working
save `working', emptyok

local to_append: dir "." files "whatever*.dta"

foreach t of local to_append {
    use "`t'", clear
    // DO SOMETHING TO THE DATA in `t'
    append using `working'
    save `qorking', replace  // TYPO IN MACRO NAME;  q NEXT TO w ON KEYBOARD
}

Assuming that there isn't coincidentally another macro qorking defined, Stata interprets `qorking' as an empty string, and it interprets -save `qorking', replace- as "go ahead and write over the file that was most recently loaded into memory. So the files captured in to_append will all get overwritten by versions transformed in whatever way. If the transformations involved in "DO SOMETHING" are irreversible calculations, or variables get -dropped- you have an unrecoverable data loss.

OK, I know I'm obsessed with this unsafe macro thing, and have put something about it on every wish list pretty much since day 1, and raised the issue in other contexts as well. And I know that it can't be just changed without breaking a lot of existing code. But I have more recently suggested ways that a new kind of safer macro could be added to Stata's repertoire while keeping the existing macros for backwards compatibility (or for those who prefer the current behavior). I hope someday Stata Corp. will do something about this.

[Phil Schumm]

save `qorking', replace // TYPO IN MACRO NAME; q NEXT TO w ON KEYBOARD

Assuming that there isn't coincidentally another macro qorking defined, Stata interprets `qorking' as an empty string, and it interprets -save `qorking', replace- as "go ahead and write over the file that was most recently loaded into memory."

Unless I'm missing something, this problem can be easily avoided by changing the line above to

save `"`qorking'"', replace

which will cause an "invalid file specification" error if the macro qorking does not exist. In fact, it is good practice to habitually enclose all such filename references in compound double quotes (or at least double quotes), to ensure that you don't get errors if the filename includes spaces and/or quote characters.

[Nick Cox]

I see Clyde's point and note Phil's useful solution. Another solution is to write your own program as a wrapper for save and make that program insist on a filename.

[Clyde Schechter]

Thanks to both Phil and Nick for their solutions to this problem. As an aside, I normally enclose filenames in quotes (see, for example -use "`t'"- in the code I posted earlier). The one exception has always been with tempfiles because their names are guaranteed not to have spaces or quote characters in them. I never realized that save "", replace would throw an exception. And, writing a wrapper for -save- that deals with this is also an excellent work-around, one that won't rely on my vigilance about quotes!

There are other contexts in which referencing undefined macros is problematic, but I have always regarded this particular situation as the most dangerous of all. I'm glad to learn that there are simple ways to safeguard against it.

[Phil Schumm]

I normally enclose filenames in quotes (see, for example -use "`t'"- in the code I posted earlier). The one exception has always been with tempfiles because their names are guaranteed not to have spaces or quote characters in them.

I don't believe this is in general true. The user can specify the location of Stata's tempfiles arbitrarily, and the resulting (full) paths can therefore have both spaces and quotes.