reshape error "too many variables specified"

[Pilar Gonalons]

Hi,

I'm trying to reshape a large dataset to long format. The reshape command includes 120 variables and it returns an error "too many variables specified." How this be fixed?

Thanks!

Pilar

[Attaullah Shah]

It is difficult to crack the problem without seeing what are your variables, what did you type and what Stata complained. Post an example of your data using dataex program and share with us what you typed

[Joseph Coveney]

As Attaullah mentions, you really do need to show what you typed. In the example below, Stata performed the reshaping of 120 variables in a 30k-observation dataset without any trouble.

.ÿversionÿ14.1

.ÿ
.ÿclearÿ*

.ÿsetÿmoreÿoff

.ÿsetÿseedÿ`=date("2016-05-15",ÿ"YMD")'

.ÿquietlyÿsetÿobsÿ30000

.ÿgenerateÿintÿpidÿ=ÿ_n

.ÿforvaluesÿiÿ=ÿ1/120ÿ{
ÿÿ2.ÿÿÿÿÿÿÿÿÿlocalÿvarlistÿ`varlist'ÿrandu`i'
ÿÿ3.ÿÿÿÿÿÿÿÿÿforvaluesÿjÿ=ÿ1/3ÿ{
ÿÿ4.ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿgenerateÿdoubleÿrandu`i'`j'ÿ=ÿruniform()
ÿÿ5.ÿÿÿÿÿÿÿÿÿ}
ÿÿ6.ÿ}

.ÿ
.ÿquietlyÿreshapeÿlongÿ`varlist',ÿi(pid)ÿj(vin)

.ÿ
.ÿdisplayÿinÿsmclÿasÿtextÿ`:ÿwordÿcountÿ`varlist''
120

.ÿ
.ÿexit

endÿofÿdo-file


.

[Pilar Gonalons]

You're right! Here's some code. I use local macro to list the variables. I also tried to just type in all variables (to check if there was something wrong in the stubs local) but I get the same error message. See code for both below.

Using local macro to list stubs, all variables are in the form of: nameSIPPM_YYYY.

unab mylist: *SIPP*


foreach v of local mylist {
local stubs `"`stubs' `=substr("`v'", 1, length("`v'")-4)'"'
}

reshape long `stubs', i(pid) j(year)

Listing all variables:

#delimit ;
reshape long wfeSIPP1_ wfeSIPP2_ [...list of variables...]
, i(pid) j(year)
#delimit cr

Many thanks!

Pilar

[Pilar Gonalons]

And I should add, I reshape 120 stubs, in the wide data set this is amounts to roughly 4680 variables - which will become 120 once I get it to long format.

[William Lisowski]

Welcome to Statalist, Pilar!

I've modified Joseph Coveney's example to reshape 120 stubs with 39 - rather than 3 - variables apiece = 4680 variables into 120 variables and successfully tested it. Perhaps you could test this example on your system and see if it returns the same error you received with your actual data. I will say that it appears to me that the loop you use to generate your stub list will produce a list with all 4680 variables. I don't see a problem with your example listing the 120 stubs by hand, though.

. set more off

. set seed `=date("2016-05-15", "YMD")'

. set obs 100
number of observations (_N) was 0, now 100

. gen pid = _n

. forvalues i = 1/120 {
  2.     local varlist `varlist' randu`i'_
  3.     forvalues j = 1/39 {
  4.         generate double randu`i'_`j' = runiform()
  5.     }
  6. }

. describe, short

Contains data
  obs:           100                          
 vars:         4,681                          
 size:     3,744,400                          
Sorted by:
     Note: Dataset has changed since last saved.

. quietly reshape long `varlist', i(pid) j(vin)

. describe, short

Contains data
  obs:         3,900                          
 vars:           122                          
 size:     3,763,500                          
Sorted by: pid  vin
     Note: Dataset has changed since last saved.

.

[Ariel Karlinsky]

What is your maxvar setting? maybe that's what holding you back. take a look at "set maxvar" at query memory.

[Pilar Gonalons]

Thanks for your help! The version listing all stupbs by hand did finally reshape, but I did not really change anything. Not sure what was producing the error to begin with.