BLP estimation, iterations are not concave

[Mahmoud Ahmed]

Dear Stata Listers,

I am running BLP model on an aggregated market data, over t= 3990 markets ( interactions of weeks and regions), to estimate market shares for j products across different markets, and using demograhics to capture the consumers' heterogenity. my product cahracterstics are x1= fair trade or not, x2= organic or not, x3= product size in grams, in Addition to Price, and brand dummies.

my data Looks as follow:

input float(s x1 x2 x3 p z d1 d2 d3 d4 d5 d6 d7 x12 x22 x32 x1z x2z x3z x1s x2s x3s) int Market
.009983872 0 0 500 .007252727 .0072385 57.55 .075 .075 56.8 .6 2.25 2211.25 0 0 250000 0 0 3.61925 0 1 8500 1
.006143922 0 0 500 .00776 .007816923 57.55 .075 .075 56.8 .6 2.25 2211.25 0 0 250000 0 0 3.908461 0 1 8500 1
.0015359804 0 0 500 .00798 .008424444 57.55 .075 .075 56.8 .6 2.25 2211.25 0 0 250000 0 0 4.212222 0 1 8500 1
.0023039705 0 0 500 .006913333 .006850952 57.55 .075 .075 56.8 .6 2.25 2211.25 0 0 250000 0 0 3.425476 0 1 8500 1
.0015359804 0 0 500 .00614 .006663959 57.55 .075 .075 56.8 .6 2.25 2211.25 0 0 250000 0 0 3.331979 0 1 8500 1
.0015359804 0 1 500 .00798 .00798 57.55 .075 .075 56.8 .6 2.25 2211.25 0 1 250000 0 .00798 3.99 0 0 8500 1
.0007679902 0 0 500 .0057 .005685 57.55 .075 .075 56.8 .6 2.25 2211.25 0 0 250000 0 0 2.8425 0 1 8500 1
.0023039705 0 0 500 .00776 .008159524 57.55 .075 .075 56.8 .6 2.25 2211.25 0 0 250000 0 0 4.079762 0 1 8500 1
.003839951 0 0 500 .00798 .008551429 57.55 .075 .075 56.8 .6 2.25 2211.25 0 0 250000 0 0 4.2757144 0 1 8500 1


I started by running the model without demograhics, but I face Problems in the convergance, when I run the following code: [CODE][import delimited \\winfs-uni.top.gwdg.de\ahmed12$\Downloads\Mahmoud\data\Da ten_RK_12-16\blpdata.txt, delimiter(space) bindquote(strict) case(preserve) asdouble clear


// destring variables:

destring, replace dpcomma ignore(" ")




encode Product, gen(product)
encode Brand, gen(brand)


// variables preparation:

gen x1= Fair
gen x2= Bio
gen x3= Size
gen s= VolShare
gen p= PriceGr
gen z= PriceIV
gen d1= AgeHead
gen d2= AbiturHead
gen d3= UniHead
gen d4= AgeSR
gen d5= FemaleSR
gen d6= HHSize
gen d7= HHIncome





generate x12=x1^2
generate x22=x2^2
generate x32=x3^2
generate z2=z^2

generate x1z=x1*z
generate x2z=x2*z
generate x3z=x3*z

bysort Market: egen x1s=sum(x1)
bysort Market: egen x2s=sum(x2)
bysort Market: egen x3s=sum(x3)

replace x1s=x1s-x1
replace x2s=x2s-x2
replace x3s=x3s-x3
blp s x1 x2 x3 brand_dummy*, stochastic(x1,x2,x3,p) endog(p=z z2 x12 x22 x32 x1z x2z x3z x1s x2s x3s) markets(Market) draws(200)/CODE]

It takes too much time and does not converge in the end, I got the following message after 50 iterations:[Iteration 44: f(p) = 2769.6874 (not concave)
Iteration 45: f(p) = 2769.6883 (not concave)
Iteration 46: f(p) = 2769.7062 (not concave)
Iteration 47: f(p) = 2769.6931 (not concave)
Iteration 48: f(p) = 2769.6912 (not concave)
Iteration 49: f(p) = 2769.6888 (not concave)
Iteration 50: f(p) = 2769.6968 (not concave)
convergence not achieved
Warning: variance matrix is nonsymmetric or highly singular]

the table of results looked as follows:
-------------------------------------------------------------------------------
| Coef. Std. Err. z P>|z| [95% Conf. Interval]
--------------+----------------------------------------------------------------
Mean utility |
cons | .8228627 . . . . .
x1 | .3926909 . . . . .
x2 | .070797 . . . . .
x3 | .0017025 . . . . .
brand_dummy1 | -1.754114 . . . . .
brand_dummy2 | -2.340487 . . . . .
brand_dummy3 | -1.062552 . . . . .
brand_dummy4 | -.5074868 . . . . .
brand_dummy5 | -2.578009 . . . . .
brand_dummy6 | -1.750712 . . . . .
brand_dummy7 | -1.294738 . . . . .
brand_dummy8 | -1.666641 . . . . .
brand_dummy9 | -1.704865 . . . . .
brand_dummy10 | -2.360217 . . . . .
brand_dummy11 | -1.462816 . . . . .
brand_dummy12 | -2.44297 . . . . .
brand_dummy13 | -1.660811 . . . . .
brand_dummy14 | -1.102351 . . . . .
brand_dummy15 | -1.558638 . . . . .
brand_dummy16 | -2.056598 . . . . .
brand_dummy17 | -1.492722 . . . . .
brand_dummy18 | -2.379745 . . . . .
brand_dummy19 | -1.886506 . . . . .
brand_dummy20 | -1.404502 . . . . .
brand_dummy21 | -2.707936 . . . . .
brand_dummy22 | -2.3914 . . . . .
brand_dummy23 | -2.607298 . . . . .
brand_dummy24 | -1.730492 . . . . .
brand_dummy25 | -2.306587 . . . . .
brand_dummy26 | -1.915122 . . . . .
brand_dummy27 | -2.90919 . . . . .
brand_dummy28 | -2.119405 . . . . .
brand_dummy29 | -2.833878 . . . . .
brand_dummy30 | -2.975195 . . . . .
brand_dummy31 | -2.586929 . . . . .
brand_dummy32 | -3.476073 . . . . .
brand_dummy33 | -3.384767 . . . . .
brand_dummy34 | -1.951016 . . . . .
brand_dummy35 | -1.649924 . . . . .
brand_dummy36 | -2.453735 . . . . .
brand_dummy37 | -1.460477 . . . . .
brand_dummy38 | -1.740989 . . . . .
brand_dummy39 | -1.506543 . . . . .
brand_dummy40 | -1.173403 . . . . .
brand_dummy41 | -2.410974 . . . . .
brand_dummy42 | -1.493008 . . . . .
brand_dummy43 | -1.897421 . . . . .
brand_dummy44 | -2.184064 . . . . .
brand_dummy45 | -2.694391 . . . . .
brand_dummy46 | -3.28812 . . . . .
brand_dummy47 | -2.4508 . . . . .
brand_dummy48 | -2.371292 . . . . .
brand_dummy49 | -.9735492 . . . . .
brand_dummy50 | -2.152592 . . . . .
brand_dummy51 | -4.435938 . . . . .
brand_dummy52 | -.9688985 . . . . .
brand_dummy53 | -2.148244 . . . . .
brand_dummy54 | -2.44406 . . . . .
brand_dummy55 | -1.555192 . . . . .
brand_dummy56 | -1.239317 . . . . .
brand_dummy57 | -1.395652 . . . . .
brand_dummy58 | -2.533191 . . . . .
brand_dummy59 | -1.436539 . . . . .
brand_dummy60 | -1.040952 . . . . .
brand_dummy61 | .0719828 . . . . .
brand_dummy62 | -1.351868 . . . . .
brand_dummy63 | -1.500132 . . . . .
brand_dummy64 | -.1410059 . . . . .
brand_dummy65 | -1.754581 . . . . .
brand_dummy66 | -2.51948 . . . . .
brand_dummy67 | -1.754044 . . . . .
brand_dummy68 | -.7651347 . . . . .
brand_dummy69 | -1.884069 . . . . .
p | -1191.584 . . . . .
--------------+----------------------------------------------------------------
x1 |
SD | 1.85e-85 . . . . .
--------------+----------------------------------------------------------------
x2 |
SD | .0423504 . . . . .
--------------+----------------------------------------------------------------
x3 |
SD | 1.00e-14 . . . . .
--------------+----------------------------------------------------------------
p |
SD | 508.4741 . . . . .
-------------------------------------------------------------------------------

I know not concave means the model is misspecified, but I do not know the source of it?
also i tried including the x1, x2, x3 and brand dummies as exogenous variables only and not stochastic, only the Price is treated as stochastic, it improved the results( converged), but for my Analysis i want to include also product cahracteristics, any idea how to include them in a specification that is not problematic.

thanks in advance for your replies.

Regards,
Mahmoud Ahmed

[Weiwen Ng]

Mahmoud, it's hard to read your results table, hence it's hard to diagnose your results. It's best to paste the results in code delimiters, which you seem to be aware of, but you didn't write the close delimiter (i.e. [\CODE]).

I don't know the -blp- command (and you should assume that not everyone knows it). If SD represents the standard deviation of something like a random effects parameter, then you seem to have two standard deviations that are tending toward zero (x1 and x3). In a regular random effects model, if a random effect had an SD or variance of zero, that might suggest you don't treat that parameter as random at all. I do wonder if that's one possible source of your convergence trouble.

[Mahmoud Ahmed]

Hi Weiwen NG,

Thanks for your reply, here is another version of the data:

* Example generated by -dataex-. To install: ssc install dataex
clear
input float(s p) int Market float(x1 x2 x3 z d1 d2 d3 d4 d5 d6 d7)
 .005411694 .007935556  979 0 0 500   .0075201 60.86956  .0942029  .2246377 60.34783 .7391304 1.8913044  2494.384
.0006336074     .00638  983 0 0 500 .006746667 59.41322 .09917355  .2479339 58.80165 .7603306 1.9338843  2423.967
.0006028761     .00638 1013 0 0 500 .006540098 59.82569 .12844037 .18348624 59.14679 .7247707 2.0183487 2328.6697
.0021786492     .00998 1250 0 0 500 .009864615   56.375 .04166667      .125 56.45833 .8333333  2.458333  3005.208
.0017066738 .006054545 1349 0 0 550  .00687132 56.21429  .0952381  .2857143 55.38095 .7380952  2.047619  2574.405
.0019285093     .00698 2252 0 0 500 .007057798 59.87324 .11267605  .1549296 58.84507 .8309859  2.140845 2520.4226
 .014816865 .006595385 2588 0 0 500 .006938978 55.67059 .07058824        .2 54.71765 .7058824 2.3411765  2805.294
.0008248423     .01476 2901 0 0 250  .01456111 59.41177  .0882353  .2352941 59.76471 .7058824  2.235294 2290.4412
.0005549197     .01396 3085 0 0 250     .01396  56.2695 .10638298  .3262411 56.88652 .7943262  2.106383 2172.5178
 .010103562    .007532 3263 0 0 500 .007454115   56.175        .1        .2     53.1       .8     2.425   2556.25
end

where product cahracterstics are x1= fair trade or not, x2= organic or not, x3= product size in grams, in Addition to Price,and brand dummies, I am trying to run, the random coefficient mixed logit demand, using stata function blp.

I ran the following code:

gen x1= Fair
gen x2= Bio
gen x3= Size
gen s= VolShare
gen p= PriceGr
gen z= PriceIV
gen d1= AgeHead
gen d2= AbiturHead
gen d3= UniHead
gen d4= AgeSR
gen d5= FemaleSR
gen d6= HHSize
gen d7= HHIncome
generate x12=x1^2
generate x22=x2^2
generate x32=x3^2
generate z2=z^2

generate x1z=x1*z
generate x2z=x2*z
generate x3z=x3*z

bysort Market: egen x1s=sum(x1)
bysort Market: egen x2s=sum(x2)
bysort Market: egen x3s=sum(x3)

replace x1s=x1s-x1
replace x2s=x2s-x2
replace x3s=x3s-x3

blp s x1 x2 x3 brand_dummy*, stochastic(x1,x2,x3,p) endog(p=z z2 x12 x22 x32 x1z x2z x3z x1s x2s x3s) markets(Market) draws(200)

It converges only when I exclude the product characteristics from Stochastic ( I do not consider them random coefficients).

I know not concave means the model is misspecified, but I do not know the source of it?
also i tried including the x1, x2, x3 and brand dummies as exogenous variables only and not stochastic, only the Price is treated as stochastic, it improved the results( converged), but for my Analysis i want to include also product cahracteristics, any idea how to include them in a specification that is not problematic.

thanks in advance for your replies.

Regards,
Mahmoud Ahmed