Advice on how to use a command that causes my Stata to crash

[Luis Mijares Castaneda]

Hello, I am trying to use the dropout() subcommand on the user created program Traj, I use this command, however, the dropout() subcommand causes my stata to crash, after a while stata says not responding, It might be because the number of variables in the var() and ind() is very large, does anyone have any advice on how to utilize the dropout(), I need a model of dropout, without causing my stata to crash, I already tried setting the memory and variables to maximum.


traj, model(cnorm) var(cogtot27_55-cogtot27_90) order(1,3,3,1) indep(age_55-age_90) min(0) max(27) obsmar(death_indicator) weight(new_weight) dropout(00000)

[Clyde Schechter]

after a while stata says not responding

I know nothing about the -traj- command. And I don't know where it comes from, so I have no way to find out. But without specific reference to that command "Stata not responding" does not necessarily mean that Stata has crashed. When Stata is engaged in a lengthy, time-consuming disk operation, or during certain other calculations that have been specifically programmed to be uninterruptible (because there is no way to exit from them and leave behind a valid data set), Stata will be "not responding" but will be continuing to work. Since you refer to having very large numbers of variables involved, this may be one of those situations.

If you are running Windows, you can open Task Manager and see what is going on with Stata. If it continues to show variation in both memory and CPU use as you watch, it is likely just working on something that is taking a long time. If Task Manager shows no variation in those, then it may well have crashed.

If you can provide information about where to find the -traj- program, then it may be possible to give a more specific answer to your question, or to suggest different ways of using it, or to suggest some other approach.

[Luis Mijares Castaneda]

I know nothing about the -traj- command. And I don't know where it comes from, so I have no way to find out. But without specific reference to that command "Stata not responding" does not necessarily mean that Stata has crashed. When Stata is engaged in a lengthy, time-consuming disk operation, or during certain other calculations that have been specifically programmed to be uninterruptible (because there is no way to exit from them and leave behind a valid data set), Stata will be "not responding" but will be continuing to work. Since you refer to having very large numbers of variables involved, this may be one of those situations.

If you are running Windows, you can open Task Manager and see what is going on with Stata. If it continues to show variation in both memory and CPU use as you watch, it is likely just working on something that is taking a long time. If Task Manager shows no variation in those, then it may well have crashed.

If you can provide information about where to find the -traj- program, then it may be possible to give a more specific answer to your question, or to suggest different ways of using it, or to suggest some other approach.

I attached the link to the user manual for the Traj command, https://www.andrew.cmu.edu/user/bjones/refpdf/ref3.pdf

[Clyde Schechter]

After a quick perusal of the document you linked to, I think that Stata's -fmm- command may do the same thing, with one limitation. For -fmm- you must specify the number of components (classes) in your command. I don't know how crucial the ability to estimate the number of classes is to your situation. That said, knowing the kind of calculations involved in fitting this kind of model, with a large data set it doesn't surprise me if there are periods where Stata is "not responding" even though computations are still proceeding well.

Further advice from other Forum members who might have greater familiarity with this specific situation would be welcome.

[Luis Mijares Castaneda]

After a quick perusal of the document you linked to, I think that Stata's -fmm- command may do the same thing, with one limitation. For -fmm- you must specify the number of components (classes) in your command. I don't know how crucial the ability to estimate the number of classes is to your situation. That said, knowing the kind of calculations involved in fitting this kind of model, with a large data set it doesn't surprise me if there are periods where Stata is "not responding" even though computations are still proceeding well.

Further advice from other Forum members who might have greater familiarity with this specific situation would be welcome.

Currently my cpu is changing but the memory isn't, would this indicate that stata is still computing? I'm not sure if stata encountered an error or it's still making the calculations.

[Clyde Schechter]

I would interpret that as still computing. How large is your data set? And how long has the computation been running?

[Luis Mijares Castaneda]

I would interpret that as still computing. How large is your data set? And how long has the computation been running?

12,000 observations, sorry I don't remember the exact number, it has been running for close to 3 hours now

[Clyde Schechter]

Three hours does not, to me, sound excessive for a problem of this complexity on a data set this size. I would, at least, let it run overnight.