Parameter bounds in maximum likelihood estimation

[Gabriel Kreindler]

Hello everyone,

I am estimating a model where one of the parameters is a probability, hence its domain is [0,1]. I am struggling to find an effective way to "tell" the ml commands to only search within this domain.

There is a bound option, yet it is poorly documented in ml.pdf (page 14). For example, how does one specify multiple parameters in the same equation? In any case, the utility of the option is not clear given that it only applies to search (and not to the maximization).

Another suggestion (provided here) is to use a transformation that maps the real line to the desired interval. In this case, I used exp(x)/(1+exp(x)). However, this is not a neutral choice, as it "stretches" the space close to the bounds. In my case this leads to non-convergence behavior in those areas, or it does converge to what looks like a local maximum.

I've also tried to ad-hoc add large negative terms to the likelihood if the test parameter is outside the range, but I ran into several errors that suggest that this is a poor way to go ahead. One error was (check's TEST 1 failed because "Two different coefficient vectors resulted in equal log likelihood values"). Another error was that search could not find feasible values.

Any suggestions welcome. Thank you!


The model is a simple probit with a cap on the maximum probability:

cap program drop myprobit
program myprobit
    version 13
    args lnf theta1 theta2
    tempvar probs
    qui gen double `probs' = exp(`theta2')/(1+exp(`theta2'))
    quietly replace `lnf' = ln(`probs'*normal(`theta1'))     if $ML_y1==1
    quietly replace `lnf' = ln(1 - `probs'*normal(`theta1')) if $ML_y1==0
end

ml model lf myprobit (binaryyvar = depvar depvar2) ()
ml check
ml search, repeat(150)
ml maximize, difficult iterate(50)

[Joao Santos Silva]

Gabriel,

The logit trick you are using is a standard approach. If you get convergence problems with that, this suggests that the model is trying to go outside the (0,1) limit and therefore your model is possibly inappropriate. Actually, in these cases I like to estimate the model without imposing any restrictions and see what is the unrestricted estimate; if it is way out of the admissible interval that is an indication that the model does not fit the data.

All the best,

Joao

[Gabriel Kreindler]

Hello Joao,

Thank you, this is useful! Indeed, as far as I know MLE asymptotic theory does not hold if the true parameter is on the boundary. And, indeed, the behavior I described suggests that the model wants to hit the upper limit of 1.

What makes things complicated in my case is that I am bootstrapping. For the entire sample and most bootstrap samples, MLE converges nicely to an interior solution, far from the boundary. However, for only a few bootstrap runs the above behavior happens. So this makes it more complicated to choose the simpler model.

My temporary fix is to add an ad-hoc smooth penalty term for theta2 > log(999). The idea is that for all practical purposes anything larger than 0.999 is just as good as equal to 1.

cap program drop myprobit_1
program myprobit_1
    version 13
    args lnf theta1 theta2
    tempvar probs
    qui gen double `probs' = exp(`theta2')/(1+exp(`theta2'))
    quietly replace `lnf' = ln(`probs'*normal(`theta1'))     if $ML_y1==1
    quietly replace `lnf' = ln(1 - `probs'*normal(`theta1')) if $ML_y1==0

    * ad-hoc adjustment to avoid large `theta2' solutions
    quietly replace `lnf' = `lnf' - exp( `theta2'-ln(999) )/100
end

[Joao Santos Silva]

Dear Gabriel,

If the problem only occurs in some bootstrap replicas, I would estimate the model without any restriction; that is, I would not use the logit trick and just estimate probs as an unrestricted parameter. Sometimes the estimate will be above one, but that is what you expect if the true parameter is close to the upper bound. Any other approach will underestimate the variance, right?

All the best,

Joao

[Gabriel Kreindler]

Dear Joao,

This is a great idea, I will try it are report back.

Many thanks,
Gabriel