Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Specification test is always refusing the model

    Hi people!

    My problem is now a big one, i am writing my thesis in which i want to explain job satisfaction and the quit intention. In every case of quit intention in a logit or in a probit model the scoregof and the linktest refuse the null hypothsis, so just telling the model is wrong specified. The dependent variable is 1 if you think very often about quitting otherwise zero, or 1 for satisfied. Even a biprobit doesn't work. For a cross section data set, that is the usual way to go and there are no other possibilities. What shall I do? Is this normal in cases of job satisfaction as dependent variable?

  • #2
    It may not be sensible to rely on these tests. After all, parametric models like logit or probit are inherently simplifications of reality, and it you have a large enough sample, you will always be able to detect the discrepancy between reality and the model. So what is your sample size? You may be making a mountain out of a molehill.

    So what to do instead? It depends on the goal of your research. If your model is intended only to develop a model that discriminates well between those who often of quitting and those who do not, then the area under the ROC curve (-lroc-, -roctab-) are the statistics to look at. Even if the model doesn't really fit the data well in detail, it may be quite good at this discrimination. If your model is intended to actually identify an accurate probability of intending to quit, then I would look at the results of -estat gof, table-. I would basically ignore the p-value there and scrutinize the actual extent of match/mismatch between the observed and expected numbers: they may very well be close enough for practical purposes.

    If you find that these more relevant tests of your models still suggest that they are unsatisfactory, then you need to look into the possibility of re-specifing some linear terms with non-linear transformations, or adding interaction terms, or even getting data on additional variables that you don't currently have. The possibilities here are quite broad and can't be nailed down in a brief post. Usually extensive exploratory data analysis, especially with graphs, is required to figure out these details.

    Comment


    • #3
      So my sample size varies a lot, between about 5000 observations and in the case of merged data sets between 1800 observations. I am mainly not interested in the exact effect, and the variable or
      the link I thought about from theoretical aspects is insignificant, but thats is also a result to be reported in a master thesis, but the model should fit , thats what makes me worry about- So if I am only interested in the rough relationship especially about the sign, a normal biprobit or whatever model I use even if it is not specified about the distribution is ok? I have doubts about that. I try first to have a suitable model in general before I start with this basis to get work and robustness in detail, I hope this way is ok.....

      Comment


      • #4
        Sebastian:
        just an aside to Clyde's superb advice.
        Assuming that your statement about the remarkable variation of your sample size (post #3) means that you have missing values, missingness might be a problem you shoud deal with.
        As a closing-out remark, despite my being sympathetic with your need of getting a clear-cut answer, your chances of getting helpful repies are conditional on posting what you typed and what Stata gave you back (as per FAQ).
        Kind regards,
        Carlo
        (Stata 18.0 SE)

        Comment

        Working...
        X