[Clyde Schechter]

So you are differencing over markets rather than over consecutive time observations. But the same explanation holds. There is no reason to expect the results of this differenced model to resemble that of a plain regression of Price on euclideankm, scaledsubscription and rainfall. Changes in magnitudes and even signs of coefficients is to be expected here.

[Mohammed Islam]

Hi Clyde,

Apologies as I was struggling with how I would code so that I had the following regression:

Pricemarket1,t commodity i - Pricemarket2,,t commodity i = euclideankmmarket1,t commodity i - euclideankmmarket2,,t commodity i +scaledsubscriptionmarket1,t commodity i - scaledsubscriptionmarket2,,t commodity i+rainfallmarket1,t commodity i - rainfallmarket2,,t commodity i +Dummy(commodity type) + Dummy (price type)

(e.g. commodity type millet=0; sugar=1 and price type 0=producer; retail=1)

I would really appreciate this

[Clyde Schechter]

The data examples posted earlier in the thread show data from only a single market, so they aren't suitable for demonstrating or testing code. But it would be something like this:

reshape wide price euclideankm scaledsubscription rainfall, i(commodity country) j(market)
foreach v in price euclideankm scaledsubscription rainfall {
    gen delta_`v' = `v'2 - `v'1
}
regress delta_price delta_euclieankm delta_scaledsubscription delta_rainfall i.commodity i.price_type

Since this is panel data, you probably want to actually use -xtreg- rather than -regress-. Anyway, this is a start, and you can probably modify this code to suit your actual data.

[Mohammed Islam]

Unfortunately I receive the following

 reshape wide price euclideankm scaledsubscription rainfall, i(commodity count
> ry) j(market)
variable market is string; specify string option
r(109);

. reshape wide price euclideankm scaledsubscription rainfall, i(commodity count
> ry) j(marketid)
(note: j = 1 2 3 4 5 11)
values of variable marketid not unique within commodity country
    Your data are currently long.  You are performing a reshape wide.  You
    specified i(commodity country) and j(marketid).  There are observations
    within i(commodity country) with the same value of j(marketid).  In the
    long data, variables i() and j() together must uniquely identify the
    observations.

         long                                wide
        +---------------+                   +------------------+
        | i   j   a   b |                   | i   a1 a2  b1 b2 |
        |---------------| <--- reshape ---> |------------------|
        | 1   1   1   2 |                   | 1   1   3   2  4 |
        | 1   2   3   4 |                   | 2   5   7   6  8 |
        | 2   1   5   6 |                   +------------------+
        | 2   2   7   8 |
        +---------------+
    Type reshape error for a list of the problem variables.
r(9);

[Clyde Schechter]

Well, as we don't have a sample of your data that is usable for this, you may have to fiddle with the code to get it just right. Looking in a bit more detail at the data you have shown so far, these changes may do the trick:

reshape wide price euclideankm scaledsubscription rainfall, i(commodity country qdate) j(market_id)

Notes: Adding qdate to the i() option will, I think, turn i() into an identifier of observations when combined with market_id. If it doesn't, then we're back to figuring out what the actual organization of your data is! That was really the thrust of the original post #1, and it seemed we resolved it, at least well enough that you moved on to other questions, but there was never confirmation of what the final answer was. I'm assuming that commodity country qdate and market_id jointly identify unique observations in the data.

In the data I have seen there is a variable market_id that takes on numerical values (the value 1 only in the example). I assume this is the variable that corresponds to your market1 and market2 subscripts in the equation you show. If it is not actually numeric in your data, then you need to -destring market_id, replace- before the -reshape-.

[Mohammed Islam]

Unfortunately I recieve the following errors:

gen qdate = quarterly(string(quarter)+"q"+string(year), "QY")

. format qdate %tq

. reshape wide price euclideankm scaledsubscription rainfall, i(commodity count
> ry qdate) j(marketid)
(note: j = 1 2 3 4 5 6 7 8 9 10 11)
values of variable marketid not unique within commodity country qdate
    Your data are currently long.  You are performing a reshape wide.  You
    specified i(commodity country qdate) and j(marketid).  There are
    observations within i(commodity country qdate) with the same value of
    j(marketid).  In the long data, variables i() and j() together must
    uniquely identify the observations.

         long                                wide
        +---------------+                   +------------------+
        | i   j   a   b |                   | i   a1 a2  b1 b2 |
        |---------------| <--- reshape ---> |------------------|
        | 1   1   1   2 |                   | 1   1   3   2  4 |
        | 1   2   3   4 |                   | 2   5   7   6  8 |
        | 2   1   5   6 |                   +------------------+
        | 2   2   7   8 |
        +---------------+
    Type reshape error for a list of the problem variables.
r(9);

. dataex
input statement exceeds linesize limit. Try specifying fewer variables
r(1000);

If I delete the variable names I am able to run the code dataex but as before it does not format as it should and so the output is unwieldy

[Mohammed Islam]

Also even if I reduce the variables included (all the way to 1 variable) the error continues:

input statement exceeds linesize limit. Try specifying fewer variables
r(1000);

[Mohammed Islam]

I was wondering if this could shed some light on my data input

 list in 1

     +-----------------------------------------------------------------+
  1. | Country | countr~d |   market | marketid | latitude | longit~e  |
     |   Benin |        1 | Dantokpa |        1 |  6.37194 |  2.43476  |
     |-----------------------------------------------------------------|
     | closes~t | euclid~m | subscr~n | propor~n | scaled~n | commod~y |
     |     Come |    59.68 |      .37 |      .14 |    .0518 |   Millet |
     |----------+------------------------------------------------------|
     | commod~d | quarter | year | pricet~e | price | unit | currency  |
     |        1 |       1 | 2000 | Producer |   110 |   KG |      XOF  |
     |-----------------------------------------------------------------|
     |                            rainfall                             |
     |                             .832087                             |
     +-----------------------------------------------------------------+

. list in 69

     +-----------------------------------------------------------------+
 69. | Country | countr~d |   market | marketid | latitude | longit~e  |
     |   Benin |        1 | Dantokpa |        1 |  6.37194 |  2.43476  |
     |-----------------------------------------------------------------|
     | closes~t | euclid~m | subscr~n | propor~n | scaled~n | commod~y |
     |     Come |    59.68 |      .37 |      .14 |    .0518 |   Millet |
     |----------+-------------------------------------------+----------|
     | commod~d | quarter | year | pricet~e |  price | unit | currency |
     |        1 |       1 | 2000 |   Retail | 271.22 |   KG |      XOF |
     |-----------------------------------------------------------------|
     |                            rainfall                             |
     |                             .832087                             |
     +-----------------------------------------------------------------+

. list in 137

     +-----------------------------------------------------------------+
137. | Country | countr~d |   market | marketid | latitude | longit~e  |
     |   Benin |        1 | Dantokpa |        1 |  6.37194 |  2.43476  |
     |-----------------------------------------------------------------|
     | closes~t | euclid~m | subscr~n | propor~n | scaled~n | commod~y |
     |     Come |    59.68 |      .37 |      .14 |    .0518 |    Sugar |
     |----------+-------------------------------------------+----------|
     | commod~d | quarter | year | pricet~e |  price | unit | currency |
     |        2 |       1 | 2000 | Producer | 211.11 |   KG |      XOF |
     |-----------------------------------------------------------------|
     |                            rainfall                             |
     |                             .832087                             |
     +-----------------------------------------------------------------+
 
.list in 205

     +-----------------------------------------------------------------+
205. | Country | countr~d |   market | marketid | latitude | longit~e  |
     |   Benin |        1 | Dantokpa |        1 |  6.37194 |  2.43476  |
     |-----------------------------------------------------------------|
     | closes~t | euclid~m | subscr~n | propor~n | scaled~n | commod~y |
     |     Come |    59.68 |      .37 |      .14 |    .0518 |    Sugar |
     |----------+-------------------------------------------+----------|
     | commod~d | quarter | year | pricet~e |  price | unit | currency |
     |        2 |       1 | 2000 |   Retail | 428.11 |   KG |      XOF |
     |-----------------------------------------------------------------|
     |                            rainfall                             |
     |                             .832087                             |
     +-----------------------------------------------------------------+

. list in 273

     +------------------------------------------------------------------------+
273. | Country | countr~d |           market | marketid | latitude | longit~e |
     |   Benin |        1 | Malanville (CBM) |        2 |  11.8633 |  3.38424 |
     |------------------------------------------------------------------------|
     | closes~t  | euclid~m  | subscr~n  | propor~n  | scaled~n  |  commod~y  |
     |     Come  |   116.34  |      .37  |     .208  |   .07696  |    Millet  |
     |-----------+------------------------------------------------------------|
     | commod~d  | quarter  | year  | pricet~e  |  price  | unit  | currency  |
     |        1  |       1  | 2000  | Producer  | 141.22  |   KG  |      XOF  |
     |------------------------------------------------------------------------|
     |                                rainfall                                |
     |                                 1.72515                                |
     +------------------------------------------------------------------------+

. list in 341

     +------------------------------------------------------------------------+
341. | Country | countr~d |           market | marketid | latitude | longit~e |
     |   Benin |        1 | Malanville (CBM) |        2 |  11.8633 |  3.38424 |
     |------------------------------------------------------------------------|
     | closes~t  | euclid~m  | subscr~n  | propor~n  | scaled~n  |  commod~y  |
     |     Come  |   116.34  |      .37  |     .208  |   .07696  |    Millet  |
     |-----------+-----------------------------------------------+------------|
     | commod~d  | quarter  | year  | pricet~e  | price  | unit  |  currency  |
     |        1  |       1  | 2000  |   Retail  |   255  |   KG  |       XOF  |
     |------------------------------------------------------------------------|
     |                                rainfall                                |
     |                                 1.72515                                |
     +------------------------------------------------------------------------+

. list in 409

     +------------------------------------------------------------------------+
409. | Country | countr~d |           market | marketid | latitude | longit~e |
     |   Benin |        1 | Malanville (CBM) |        2 |  11.8633 |  3.38424 |
     |------------------------------------------------------------------------|
     | closes~t  | euclid~m  | subscr~n  | propor~n  | scaled~n  |  commod~y  |
     |     Come  |   116.34  |      .37  |     .208  |   .07696  |     Sugar  |
     |-----------+------------------------------------------------------------|
     | commod~d  | quarter  | year  | pricet~e  |  price  | unit  | currency  |
     |        2  |       1  | 2000  | Producer  | 271.11  |   KG  |      XOF  |
     |------------------------------------------------------------------------|
     |                                rainfall                                |
     |                                 1.72515                                |
     +------------------------------------------------------------------------+

. list in 477

     +------------------------------------------------------------------------+
477. | Country | countr~d |           market | marketid | latitude | longit~e |
     |   Benin |        1 | Malanville (CBM) |        2 |  11.8633 |  3.38424 |
     |------------------------------------------------------------------------|
     | closes~t  | euclid~m  | subscr~n  | propor~n  | scaled~n  |  commod~y  |
     |     Come  |   116.34  |      .37  |     .208  |   .07696  |     Sugar  |
     |-----------+-----------------------------------------------+------------|
     | commod~d  | quarter  | year  | pricet~e  | price  | unit  |  currency  |
     |        2  |       1  | 2000  |   Retail  |   500  |   KG  |       XOF  |
     |------------------------------------------------------------------------|
     |                                rainfall                                |
     |                                 1.72515                                |
     +------------------------------------------------------------------------+

. list in 545

     +-----------------------------------------------------------------+
545. | Country | countr~d | market | marketid  | latitude  | longit~e  |
     |   Benin |        1 |   Come |        3  |  6.40961  |   1.8855  |
     |-----------------------------------------------------------------|
     | closes~t | euclid~m | subscr~n | propor~n | scaled~n | commod~y |
     | Dantokpa |    59.68 |      .37 |     .134 |   .04958 |   Millet |
     |----------+------------------------------------------------------|
     | commod~d | quarter | year | pricet~e | price | unit | currency  |
     |        1 |       1 | 2000 | Producer |   215 |   KG |      XOF  |
     |-----------------------------------------------------------------|
     |                            rainfall                             |
     |                             .913332                             |
     +-----------------------------------------------------------------+

. list in 681

     +--------------------------------------------------------------------+
681. |      Country | countr~d |  market | marketid | latitude | longit~e |
     | Burkina Faso |        2 | Banfora |        4 |    10.64 |  -4.7588 |
     |--------------------------------------------------------------------|
     | closest~t | euclid~m | subscr~n | propor~n | scaled~n  | commod~y  |
     | Diebougou |   168.24 |      .07 |      .04 |    .0028  |   Millet  |
     |--------------------------------------------------------+-----------|
     | commod~d | quarter | year | pricet~e  | price  | unit  | currency  |
     |        1 |       1 | 2000 | Producer  | 53.53  |   KG  |      XOF  |
     |--------------------------------------------------------------------|
     |                              rainfall                              |
     |                               8.36372                              |
     +--------------------------------------------------------------------+

. list in 817

     +--------------------------------------------------------------------+
817. |      Country | countr~d |  market | marketid | latitude | longit~e |
     | Burkina Faso |        2 | Banfora |        4 |    10.64 |  -4.7588 |
     |--------------------------------------------------------------------|
     | closest~t | euclid~m | subscr~n | propor~n | scaled~n  | commod~y  |
     | Diebougou |   168.24 |      .07 |      .04 |    .0028  |    Sugar  |
     |--------------------------------------------------------+-----------|
     | commod~d | quarter | year | pricet~e |  price  | unit  | currency  |
     |        2 |       1 | 2000 | Producer | 155.29  |   KG  |      XOF  |
     |--------------------------------------------------------------------|
     |                              rainfall                              |
     |                               8.36372                              |
     +--------------------------------------------------------------------+

. list in 885

     +--------------------------------------------------------------------+
885. |      Country | countr~d |  market | marketid | latitude | longit~e |
     | Burkina Faso |        2 | Banfora |        4 |    10.64 |  -4.7588 |
     |--------------------------------------------------------------------|
     | closest~t | euclid~m | subscr~n | propor~n | scaled~n  | commod~y  |
     | Diebougou |   168.24 |      .07 |      .04 |    .0028  |    Sugar  |
     |--------------------------------------------------------+-----------|
     | commod~d | quarter | year | pricet~e |  price  | unit  | currency  |
     |        2 |       1 | 2000 |   Retail | 252.42  |   KG  |      XOF  |
     |--------------------------------------------------------------------|
     |                              rainfall                              |
     |                               8.36372                              |
     +--------------------------------------------------------------------+

[Clyde Schechter]

The message you are getting from -reshape- reflects the same phenomenon that you faced at the start of this thread with regard to -xtset-. You have to understand your data set and know which combination of variables uniquely identifies observations. You apparently figured that out because it seems you were able to ultimately -xtset- your data. So, go back to that -xtset- command. The panel variable for it was created with -egen panel_var = group(some list of variables). Take that exact list of variables, remove market_id from it, and stick what's left into i(). Also put qdate into i() and j(market_id) as is. That should enable you to do the -reshape-.

[Mohammed Islam]

Hi Clyde,

I have made some progress on the reshape though I am unable to do the regression. My code is the follows:

 clear

. *(14 variables, 2720 observations pasted into data editor)

. gen qdate = quarterly(string(quarter)+"q"+string(year), "QY")

. format qdate %tq

. egen panel_price = group(commodityid pricetype)

. reshape wide price euclideankm scaledsubscription rainfall, i(commodity count
> ry pricetype qdate) j(marketid)
(note: j = 1 2 4 5 6 7 8 9 10 11)

Data                               long   ->   wide
-----------------------------------------------------------------------------
Number of obs.                     2720   ->     544
Number of variables                  16   ->      51
j variable (10 values)         marketid   ->   (dropped)
xij variables:
                                  price   ->   price1 price2 ... price11
                            euclideankm   ->   euclideankm1 euclideankm2 ... eu
> clideankm11
                     scaledsubscription   ->   scaledsubscription1 scaledsubscr
> iption2 ... scaledsubscription11
                               rainfall   ->   rainfall1 rainfall2 ... rainfall
> 11
-----------------------------------------------------------------------------

. foreach v in price euclideankm scaledsubscription rainfall {
  2. gen delta_`v' = `v'2 - `v'1
  3. }
(272 missing values generated)
(272 missing values generated)
(272 missing values generated)
(272 missing values generated)

. regress delta_price delta_euclieankm delta_scaledsubscription delta_rainfall
> i.commodity i.pricetype
variable delta_euclieankm not found
r(111);

[Clyde Schechter]

This is one you should be able to troubleshoot yourself. The message is self explanatory: there is no such variable as delta_euclieankm: the variable's name correctly typed would be delta_euclideankm.

[Mohammed Islam]

I am extremely sorry about the careless mistake in message 25. I tried the dummy variables i.commodity and i.pricetype and receieved the following errors ( I then substituted for c.dummy and this did not work either)

regress delta_price delta_euclideankm delta_scaledsubscription delta_rainfall
>  i.commodity i.pricetype
commodity:  string variables may not be used as factor variables
r(109);

. regress delta_price delta_euclideankm delta_scaledsubscription delta_rainfall
>  c.commodity c.pricetype
no observations
r(2000);

[Clyde Schechter]

The two errors are both manifestations of the same underlying problem. You can't use a string variable in a regression. The first error message is very explicit about the problem: it's telling you that commodity is a string, and that you can't use it as a factor variable. When you change to c.commodity, you pass the "factor variable" filter, but now you have a string variable in the regression. The error message is, in this case, not so clearly informative. But in a regression command, string variables are treated as missing values, so you have, in effect, put a missing value into every observation in the regression, so there are no observations to estimate.

Looking over the data from your -list- output above, what you need to do is create a numeric commodity variable by using the -encode- command. Read the help file for -encode- if you don't know how it's used. Once you do that, enter that new numeric variable into the model as a factor variable. You may run into the same problems with pricetype once you fix up commodity. The solution is the same.

[Mohammed Islam]

Thank you very much for the help!

My project has changed slightly after discussions with my supervisor, but I am hugely grateful for the help