Error r(900) - no room to add variables, up to 2048 variables are allowed, but I'm only using 340 variables

[Iz Monnick]

Hello,

I'm trying to add 9 variables (w_racel) for each wave of data (waves a to i) to a large dataset that contains all the other data I'm using. But it keeps coming up with r(900): 'no room to add more variables. Up to 2,048 variables are allowed with this version of Stata. Versions are available that allow up to 120,000 variables.'

However, the data only contains 340 variables

 describe

Contains data from C:\Users\iz9\w_all.dta
  obs:        86,094                          Substantive data for responding adults (16+)
 vars:           340                          28 Dec 2019 11:42
                                              (_dta has notes)

The merge command I am attempting is, for example for wave a:

merge 1:1 pidp using "C:\Users\iz9\a_indresp.dta", keepusing(a_racel) nogenerate

Many thanks for any help,

Iz

[Clyde Schechter]

There is nothing obviously wrong with your code. I'm not sure how -merge- works internally. It is possible that the -keepusing()- option simply tells Stata that after reading in the file to be merged in its entirety, all but those variables should be dropped. I don't know. If that is the case, then a large data set will break your code even with the -keepusing()- option specified. Try this workaround:

foreach x in a b c d e f g h i {
    use pidp a_race1 using "C:/Users/iz9/`x'_indresp.dta", clear
    tempfile wave`x'
    save wave`x'
}
use "C:/Users/iz9/w_all.dta", clear 
foreach x in a b c d e f g h i { 
      merge 1:1 pidp using `wave`x'', nogenerate      
}

[Iz Monnick]

Hi Clyde,

Thanks for your response! Unfortunately Stata comes up with error (601) (file not found) for the first part of the code and error (198) (invalid file specification) for the second part... Any idea what's going on?

[Nick Cox]

Show us the exact code you used. For example, using backward slashes in filepaths would be fatal here.

[Iz Monnick]

. foreach x in a b c d e f g h i {
  2. 
.     use pidp a_race1 using "C:\Users\iz9\`x'_indresp.dta", clear
  3. 
.     tempfile wave`x'
  4. 
.     save wave`x'
  5. 
. }
file C:\Users\iz9\`x'_indresp.dta not found
r(601);

. use "C:\Users\iz9\w_all.dta", clear 
(Substantive data for responding adults (16+), incl. proxies)

. 
. foreach x in a b c d e f g h i { 
  2. 
.       merge 1:1 pidp using `wave`x'', nogenerate      
  3. 
. }
invalid file specification
r(198);

.

[Iz Monnick]

I tried with forward slashes but got the same error codes

[Nick Cox]

The backslash is wrong before a local macro reference: https://www.stata-journal.com/sjpdf....iclenum=pr0042 That's why Clyde suggested forward slashes.

Otherwise can you confirm that all those files exist? Show us the result of

 dir C:\Users\iz9\*_indresp.dta

[Iz Monnick]

Thanks Nick! I must have made a mistake before (apologies), it seems to have worked except I've only got the w_racel variable for waves a and b (i.e. I only have a_racel and b_racel) :

. foreach x in a b c d e f g h i {
  2. 
.     use pidp `x'_racel using "C:/Users/iz9/`x'_indresp.dta", clear
  3. 
.     tempfile uswave`x'
  4. 
.     save uswave`x'
  5. 
. }
(Substantive data for responding adults (16+), incl. proxies)
file uswavea.dta saved
(Substantive data for responding adults (16+), incl. proxies)
file uswaveb.dta saved
(Substantive data for responding adults (16+), incl. proxies)
file uswavec.dta saved
(Substantive data for responding adults (16+), incl. proxies)
file uswaved.dta saved
(Substantive data for responding adults (16+), incl. proxies)
file uswavee.dta saved
(Substantive data for responding adults (16+), incl. proxies)
file uswavef.dta saved
(Substantive data for responding adults (16+), incl. proxies)
file uswaveg.dta saved
(Substantive data for responding adults (16+), incl. proxies)
file uswaveh.dta saved
(Substantive data for responding adults (16+), incl. proxies)
file uswavei.dta saved

. use "C:/Users/iz9/w_all.dta", clear 
(Substantive data for responding adults (16+), incl. proxies)

. 
. foreach x in a b c d e f g h i { 
  2. 
.       merge 1:1 pidp using `uswave`x'', nogenerate      
  3. 
. }

And yes all the files exist:

 87.5M  12/18/19 11:27  a_indresp.dta     
 107.6M  12/18/19 11:27  b_indresp.dta     
 168.3M  12/18/19 11:27  c_indresp.dta     
 117.3M  12/18/19 11:27  d_indresp.dta     
 132.4M  12/18/19 11:27  e_indresp.dta     
 111.6M  12/18/19 11:27  f_indresp.dta     
 137.8M  12/18/19 11:27  g_indresp.dta     
 102.9M  12/18/19 11:27  h_indresp.dta     
 127.1M  12/18/19 11:27  i_indresp.dta

[Nick Cox]

Good. but I can't help further beyond suggesting that you check again the variable names in your individual files. That might be

foreach x in a b c d e f g h i {
    di "{title:`x'}" 
    describe *racel* using "C:/Users/iz9/`x'_indresp.dta"
    di 
}

[Iz Monnick]

Managed to get this to work without the original error code (900) coming up:

foreach x in a b c d e f g h i {
    use pidp `x'_racel using "C:/Users/iz9/`x'_indresp.dta", clear
    tempfile awave`x'
    save awave`x'
}
foreach x in a b c d e f g h i { 
      merge 1:1 pidp using awave`x', nogenerate      
}
merge 1:1 pidp using "C:\Users\iz9\w_all.dta", nogenerate
save "C:\Users\iz9\w_all1.dta"

Thanks all for your help!