Problem with -import dbase-

[Paula de Souza Leao Spinola]

I am using -import dbase- to convert a DBF file to DTA. However, it is replacing some of the non-missing cells to missing values.
When I manually convert the same file with Stat/Transfer I get no missing values.

. import dbase using dn99, clear case(lower)
(21 vars, 3,256,433 obs)

. save SINASC_import_dbase, replace
file SINASC_import_dbase.dta saved

. codebook dtnasc

--------------------------------------------------------------------------------------------------------------------------------------------
dtnasc                                                                                                                                DTNASC
--------------------------------------------------------------------------------------------------------------------------------------------

                  type:  string (str8)

         unique values:  366                      missing "":  994,858/3,256,433

              examples:  ""
                         "05031999"
                         "13111999"
                         "22081999"

.
. use SINASC_stat_transfer, clear

. codebook dtnasc

--------------------------------------------------------------------------------------------------------------------------------------------
dtnasc                                                                                                                           (unlabeled)
--------------------------------------------------------------------------------------------------------------------------------------------

                  type:  string (str8)

         unique values:  366                      missing "":  0/3,256,433

              examples:  "06111999"
                         "12121999"
                         "19021999"
                         "25031999"

.
end of do-file

Does anyone know why would the command -import dbase- transform some non-missing values in missing values and how I can prevent this from happening?

Many thanks
Paula

[William Lisowski]

You don't tell us what values are being replaced with missing values.

Presumably your datasets contain some sort of unique identifier, I will assume it is a variable named id. I would

import dbase using dn99, clear case(lower)
generate flag = missing(dtnasc)
keep id flag
merge 1:1 id using SINASC_stat_transfer
tab dtnasc if flag, sort

to get some idea of the values that are being replaced.

Alternative idea: Perhaps the dbase data contains values that contain one or more blanks, and perhaps Stata/Transfer copies these directly as a Stata string value of a single blank character, and perhaps import dbf "trims" input strings, which replaces these stribngs with a string of length 0, which is a missing value.

. clear

. input str8 x

             x
  1. " "
  2. ""
  3. end

. generate l = length(x)

. generate flag = missing(x)

. list, clean

       x   l   flag  
  1.       1      0  
  2.       0      1  

.

[Paula de Souza Leao Spinola]

Many thanks William Lisowski
I ran below your first code suggestion:

. use SINASC_stat_transfer, clear

. generate id = _n

. codebook dtnasc

--------------------------------------------------------------------------------------------------------------------------------------------
dtnasc                                                                                                                           (unlabeled)
--------------------------------------------------------------------------------------------------------------------------------------------

                  type:  string (str8)

         unique values:  366                      missing "":  0/3,256,433

              examples:  "06111999"
                         "12121999"
                         "19021999"
                         "25031999"

. save SINASC_stat_transfer, replace 
file SINASC_stat_transfer.dta saved

. 
. clear

. import dbase using dn99, clear case(lower)
(21 vars, 3,256,433 obs)

. generate flag = missing(dtnasc)

. generate id = _n

. codebook dtnasc

--------------------------------------------------------------------------------------------------------------------------------------------
dtnasc                                                                                                                                DTNASC
--------------------------------------------------------------------------------------------------------------------------------------------

                  type:  string (str8)

         unique values:  366                      missing "":  994,858/3,256,433

              examples:  ""
                         "05031999"
                         "13111999"
                         "22081999"

. keep id flag

. merge 1:1 id using SINASC_stat_transfer

    Result                           # of obs.
    -----------------------------------------
    not matched                             0
    matched                         3,256,433  (_merge==3)
    -----------------------------------------

. tab dtnasc if flag, sort

     dtnasc |      Freq.     Percent        Cum.
------------+-----------------------------------
   01011999 |     25,161        2.53        2.53
   30031999 |      3,875        0.39        2.92
   20041999 |      3,751        0.38        3.30
   29031999 |      3,730        0.37        3.67
   05041999 |      3,724        0.37        4.04
   27041999 |      3,693        0.37        4.42
   10031999 |      3,688        0.37        4.79
   16041999 |      3,677        0.37        5.16
   04031999 |      3,662        0.37        5.52
   02031999 |      3,656        0.37        5.89
   12041999 |      3,650        0.37        6.26
   09041999 |      3,637        0.37        6.62
   31031999 |      3,631        0.36        6.99
   10021999 |      3,626        0.36        7.35
   03031999 |      3,617        0.36        7.72
   08041999 |      3,615        0.36        8.08
   19031999 |      3,614        0.36        8.44
   14041999 |      3,613        0.36        8.81
   11031999 |      3,610        0.36        9.17
   22031999 |      3,603        0.36        9.53
   23041999 |      3,598        0.36        9.89
   25051999 |      3,598        0.36       10.26
   15031999 |      3,592        0.36       10.62
   06041999 |      3,590        0.36       10.98
   07041999 |      3,585        0.36       11.34
   11021999 |      3,584        0.36       11.70
   19041999 |      3,583        0.36       12.06
   23031999 |      3,583        0.36       12.42
   30041999 |      3,583        0.36       12.78
   26031999 |      3,576        0.36       13.14
   26041999 |      3,571        0.36       13.50
   01031999 |      3,569        0.36       13.86
   12031999 |      3,569        0.36       14.21
   25031999 |      3,565        0.36       14.57
   05031999 |      3,561        0.36       14.93
   08031999 |      3,559        0.36       15.29
   14051999 |      3,555        0.36       15.65
   06051999 |      3,546        0.36       16.00
   09031999 |      3,542        0.36       16.36
   03051999 |      3,539        0.36       16.71
   18031999 |      3,536        0.36       17.07
   13041999 |      3,526        0.35       17.42
   15041999 |      3,519        0.35       17.78
   10051999 |      3,517        0.35       18.13
   19051999 |      3,499        0.35       18.48
   17031999 |      3,493        0.35       18.83
   18021999 |      3,492        0.35       19.18
   05051999 |      3,489        0.35       19.54
   09021999 |      3,484        0.35       19.89
   18051999 |      3,478        0.35       20.24
   24031999 |      3,478        0.35       20.59
   08021999 |      3,477        0.35       20.93
   16031999 |      3,475        0.35       21.28
   17051999 |      3,475        0.35       21.63
   12051999 |      3,472        0.35       21.98
   23021999 |      3,472        0.35       22.33
   21051999 |      3,469        0.35       22.68
   22021999 |      3,455        0.35       23.03
   26021999 |      3,454        0.35       23.37
   04051999 |      3,449        0.35       23.72
   11051999 |      3,445        0.35       24.07
   12011999 |      3,442        0.35       24.41
   12021999 |      3,435        0.35       24.76
   19021999 |      3,431        0.34       25.10
   25021999 |      3,430        0.34       25.45
   26051999 |      3,429        0.34       25.79
   24051999 |      3,428        0.34       26.14
   20051999 |      3,425        0.34       26.48
   28051999 |      3,424        0.34       26.83
   04011999 |      3,423        0.34       27.17
   31051999 |      3,422        0.34       27.51
   20011999 |      3,412        0.34       27.86
   18011999 |      3,408        0.34       28.20
   01061999 |      3,403        0.34       28.54
   02061999 |      3,398        0.34       28.88
   05021999 |      3,393        0.34       29.22
   24021999 |      3,393        0.34       29.57
   28041999 |      3,388        0.34       29.91
   05011999 |      3,384        0.34       30.25
   07051999 |      3,384        0.34       30.59
   04021999 |      3,381        0.34       30.93
   25011999 |      3,368        0.34       31.26
   06011999 |      3,362        0.34       31.60
   29041999 |      3,358        0.34       31.94
   13051999 |      3,344        0.34       32.28
   08061999 |      3,341        0.34       32.61
   02021999 |      3,337        0.34       32.95
   27051999 |      3,336        0.34       33.28
   22041999 |      3,321        0.33       33.62
   07061999 |      3,310        0.33       33.95
   11011999 |      3,300        0.33       34.28
   15061999 |      3,275        0.33       34.61
   03021999 |      3,272        0.33       34.94
   15011999 |      3,270        0.33       35.27
   14061999 |      3,258        0.33       35.60
   28011999 |      3,258        0.33       35.92
   16061999 |      3,256        0.33       36.25
   26011999 |      3,256        0.33       36.58
   18061999 |      3,247        0.33       36.90
   19011999 |      3,247        0.33       37.23
   22011999 |      3,245        0.33       37.56
   21011999 |      3,238        0.33       37.88
   01041999 |      3,237        0.33       38.21
   21061999 |      3,233        0.32       38.53
   22061999 |      3,233        0.32       38.86
   08011999 |      3,226        0.32       39.18
   10061999 |      3,224        0.32       39.51
   01021999 |      3,216        0.32       39.83
   27011999 |      3,206        0.32       40.15
   13011999 |      3,193        0.32       40.47
   14011999 |      3,184        0.32       40.79
   07011999 |      3,146        0.32       41.11
   25061999 |      3,146        0.32       41.42
   28061999 |      3,143        0.32       41.74
   04061999 |      3,133        0.31       42.05
   29011999 |      3,128        0.31       42.37
   05071999 |      3,113        0.31       42.68
   23061999 |      3,112        0.31       43.00
   17061999 |      3,110        0.31       43.31
   21041999 |      3,110        0.31       43.62
   24041999 |      3,106        0.31       43.93
   11061999 |      3,090        0.31       44.24
   09061999 |      3,088        0.31       44.55
   20021999 |      3,086        0.31       44.86
   17021999 |      3,085        0.31       45.17
   07071999 |      3,084        0.31       45.48
   12071999 |      3,080        0.31       45.79
   15051999 |      3,077        0.31       46.10
   01051999 |      3,070        0.31       46.41
   10041999 |      3,069        0.31       46.72
   17041999 |      3,066        0.31       47.03
   06031999 |      3,057        0.31       47.34
   01071999 |      3,035        0.31       47.64
   09071999 |      3,024        0.30       47.94
   06071999 |      3,017        0.30       48.25
   27031999 |      3,013        0.30       48.55
   20071999 |      3,009        0.30       48.85
   15071999 |      3,007        0.30       49.16
   24061999 |      2,999        0.30       49.46
   08051999 |      2,995        0.30       49.76
   29061999 |      2,990        0.30       50.06
   30061999 |      2,989        0.30       50.36
   20031999 |      2,978        0.30       50.66
   09081999 |      2,969        0.30       50.96
   19071999 |      2,962        0.30       51.25
   15021999 |      2,957        0.30       51.55
   13021999 |      2,955        0.30       51.85
   02071999 |      2,954        0.30       52.15
   30011999 |      2,951        0.30       52.44
   13031999 |      2,943        0.30       52.74
   08071999 |      2,935        0.30       53.03
   06021999 |      2,927        0.29       53.33
   13071999 |      2,923        0.29       53.62
   10081999 |      2,910        0.29       53.91
   27021999 |      2,904        0.29       54.21
   27071999 |      2,903        0.29       54.50
   03041999 |      2,899        0.29       54.79
   12081999 |      2,895        0.29       55.08
   03061999 |      2,893        0.29       55.37
   22051999 |      2,891        0.29       55.66
   21071999 |      2,886        0.29       55.95
   09011999 |      2,884        0.29       56.24
   05081999 |      2,868        0.29       56.53
   14071999 |      2,863        0.29       56.82
   16071999 |      2,857        0.29       57.10
   26071999 |      2,846        0.29       57.39
   23011999 |      2,837        0.29       57.68
   29051999 |      2,836        0.29       57.96
   02041999 |      2,834        0.28       58.25
   30071999 |      2,825        0.28       58.53
   28071999 |      2,818        0.28       58.81
   03081999 |      2,814        0.28       59.10
   21031999 |      2,808        0.28       59.38
   25041999 |      2,806        0.28       59.66
   28031999 |      2,803        0.28       59.94
   16081999 |      2,791        0.28       60.22
   23081999 |      2,779        0.28       60.50
   11081999 |      2,778        0.28       60.78
   12061999 |      2,776        0.28       61.06
   11041999 |      2,770        0.28       61.34
   02081999 |      2,768        0.28       61.62
   16021999 |      2,766        0.28       61.89
   23071999 |      2,763        0.28       62.17
   20081999 |      2,757        0.28       62.45
   29071999 |      2,748        0.28       62.73
   05061999 |      2,746        0.28       63.00
   16011999 |      2,746        0.28       63.28
   04081999 |      2,740        0.28       63.55
   09091999 |      2,725        0.27       63.83
   22071999 |      2,724        0.27       64.10
   04041999 |      2,720        0.27       64.37
   02011999 |      2,719        0.27       64.65
   02051999 |      2,713        0.27       64.92
   07021999 |      2,713        0.27       65.19
   20091999 |      2,712        0.27       65.47
   28021999 |      2,705        0.27       65.74
   18041999 |      2,702        0.27       66.01
   19061999 |      2,702        0.27       66.28
   01091999 |      2,693        0.27       66.55
   06081999 |      2,688        0.27       66.82
   07031999 |      2,683        0.27       67.09
   23051999 |      2,682        0.27       67.36
   30051999 |      2,669        0.27       67.63
   24011999 |      2,659        0.27       67.90
   14031999 |      2,658        0.27       68.16
   27081999 |      2,657        0.27       68.43
   14021999 |      2,644        0.27       68.70
   26061999 |      2,638        0.27       68.96
   27091999 |      2,638        0.27       69.23
   19081999 |      2,636        0.26       69.49
   24081999 |      2,628        0.26       69.76
   10011999 |      2,627        0.26       70.02
   18081999 |      2,623        0.26       70.28
   17011999 |      2,620        0.26       70.55
   03011999 |      2,619        0.26       70.81
   21021999 |      2,618        0.26       71.07
   21091999 |      2,617        0.26       71.34
   09051999 |      2,613        0.26       71.60
   02091999 |      2,607        0.26       71.86
   13091999 |      2,607        0.26       72.12
   26081999 |      2,604        0.26       72.38
   30081999 |      2,604        0.26       72.65
   25081999 |      2,602        0.26       72.91
   10091999 |      2,601        0.26       73.17
   16091999 |      2,600        0.26       73.43
   14091999 |      2,594        0.26       73.69
   17091999 |      2,567        0.26       73.95
   31011999 |      2,563        0.26       74.21
   03071999 |      2,558        0.26       74.46
   08091999 |      2,555        0.26       74.72
   13081999 |      2,552        0.26       74.98
   10071999 |      2,550        0.26       75.23
   17081999 |      2,550        0.26       75.49
   13061999 |      2,547        0.26       75.75
   16051999 |      2,544        0.26       76.00
   22091999 |      2,534        0.25       76.26
   06091999 |      2,523        0.25       76.51
   06061999 |      2,519        0.25       76.76
   17071999 |      2,502        0.25       77.01
   28091999 |      2,502        0.25       77.27
   31081999 |      2,501        0.25       77.52
   15091999 |      2,495        0.25       77.77
   03091999 |      2,487        0.25       78.02
   23091999 |      2,474        0.25       78.27
   20061999 |      2,460        0.25       78.51
   14081999 |      2,454        0.25       78.76
   24091999 |      2,447        0.25       79.01
   29091999 |      2,408        0.24       79.25
   04071999 |      2,400        0.24       79.49
   27061999 |      2,399        0.24       79.73
   30091999 |      2,371        0.24       79.97
   31071999 |      2,345        0.24       80.21
   24071999 |      2,341        0.24       80.44
   11071999 |      2,307        0.23       80.67
   28081999 |      2,298        0.23       80.90
   07081999 |      2,279        0.23       81.13
   08081999 |      2,243        0.23       81.36
   18071999 |      2,237        0.22       81.58
   21081999 |      2,236        0.22       81.81
   25071999 |      2,225        0.22       82.03
   18091999 |      2,213        0.22       82.25
   11091999 |      2,198        0.22       82.47
   19091999 |      2,161        0.22       82.69
   04091999 |      2,153        0.22       82.91
   01081999 |      2,141        0.22       83.12
   07091999 |      2,139        0.22       83.34
   15081999 |      2,136        0.21       83.55
   25091999 |      2,128        0.21       83.77
   22081999 |      2,121        0.21       83.98
   12091999 |      2,106        0.21       84.19
   04101999 |      2,102        0.21       84.40
   08101999 |      2,051        0.21       84.61
   14101999 |      2,043        0.21       84.81
   01101999 |      2,037        0.20       85.02
   07101999 |      2,035        0.20       85.22
   20101999 |      2,009        0.20       85.43
   26091999 |      2,002        0.20       85.63
   06101999 |      2,001        0.20       85.83
   05101999 |      1,995        0.20       86.03
   15101999 |      1,984        0.20       86.23
   29081999 |      1,980        0.20       86.43
   11101999 |      1,957        0.20       86.62
   19101999 |      1,939        0.19       86.82
   21101999 |      1,938        0.19       87.01
   05091999 |      1,928        0.19       87.21
   08111999 |      1,920        0.19       87.40
   18101999 |      1,901        0.19       87.59
   05111999 |      1,896        0.19       87.78
   22101999 |      1,885        0.19       87.97
   04111999 |      1,883        0.19       88.16
   27101999 |      1,883        0.19       88.35
   28101999 |      1,873        0.19       88.54
   25101999 |      1,868        0.19       88.73
   10111999 |      1,861        0.19       88.91
   09111999 |      1,851        0.19       89.10
   13101999 |      1,844        0.19       89.28
   26101999 |      1,823        0.18       89.47
   17111999 |      1,820        0.18       89.65
   19111999 |      1,816        0.18       89.83
   16111999 |      1,800        0.18       90.01
   03111999 |      1,788        0.18       90.19
   11111999 |      1,776        0.18       90.37
   22111999 |      1,765        0.18       90.55
   18111999 |      1,755        0.18       90.73
   01111999 |      1,750        0.18       90.90
   09101999 |      1,747        0.18       91.08
   29101999 |      1,741        0.17       91.25
   12111999 |      1,730        0.17       91.43
   02101999 |      1,726        0.17       91.60
   26111999 |      1,717        0.17       91.77
   06121999 |      1,691        0.17       91.94
   20121999 |      1,690        0.17       92.11
   23111999 |      1,682        0.17       92.28
   12101999 |      1,673        0.17       92.45
   29111999 |      1,663        0.17       92.62
   01121999 |      1,656        0.17       92.78
   15121999 |      1,656        0.17       92.95
   02121999 |      1,650        0.17       93.12
   23101999 |      1,640        0.16       93.28
   16101999 |      1,625        0.16       93.44
   10121999 |      1,621        0.16       93.61
   17121999 |      1,621        0.16       93.77
   16121999 |      1,620        0.16       93.93
   03121999 |      1,616        0.16       94.10
   25111999 |      1,606        0.16       94.26
   24111999 |      1,602        0.16       94.42
   30111999 |      1,594        0.16       94.58
   21121999 |      1,592        0.16       94.74
   06111999 |      1,591        0.16       94.90
   22121999 |      1,582        0.16       95.06
   13111999 |      1,579        0.16       95.22
   30101999 |      1,579        0.16       95.37
   15111999 |      1,552        0.16       95.53
   03101999 |      1,551        0.16       95.69
   07121999 |      1,546        0.16       95.84
   09121999 |      1,536        0.15       96.00
   24101999 |      1,525        0.15       96.15
   10101999 |      1,519        0.15       96.30
   13121999 |      1,510        0.15       96.45
   28121999 |      1,510        0.15       96.61
   14121999 |      1,508        0.15       96.76
   07111999 |      1,499        0.15       96.91
   27121999 |      1,480        0.15       97.06
   02111999 |      1,477        0.15       97.21
   08121999 |      1,470        0.15       97.35
   20111999 |      1,470        0.15       97.50
   17101999 |      1,468        0.15       97.65
   31101999 |      1,465        0.15       97.80
   23121999 |      1,457        0.15       97.94
   27111999 |      1,439        0.14       98.09
   29121999 |      1,395        0.14       98.23
   14111999 |      1,359        0.14       98.36
   21111999 |      1,348        0.14       98.50
   04121999 |      1,341        0.13       98.63
   12121999 |      1,328        0.13       98.77
   18121999 |      1,326        0.13       98.90
   28111999 |      1,308        0.13       99.03
   11121999 |      1,279        0.13       99.16
   24121999 |      1,277        0.13       99.29
   05121999 |      1,266        0.13       99.42
   30121999 |      1,219        0.12       99.54
   26121999 |      1,217        0.12       99.66
   19121999 |      1,207        0.12       99.78
   25121999 |      1,084        0.11       99.89
   31121999 |      1,082        0.11      100.00
------------+-----------------------------------
      Total |    994,858      100.00

. 
end of do-file

[William Lisowski]

I wrote

Presumably your datasets contain some sort of unique identifier, I will assume it is a variable named id.

You instead created an identifier called id using the observation number. I do not trust that the datasets are in the same order, so your tabulation is not necessarily correct.

If among the 21 variables there are one or more variables which constitute unique identification, replace "id" with those variables in your merge command.

If not, make some attempt to confirm that the datasets are in the same order by confirming that dtnasc is the same in the two datasets when it is not missing.

import dbase using dn99, clear case(lower)
rename dtnasc dtnasc_imp
keep id dtnasc_imp
merge 1:1 id using SINASC_stat_transfer
assert dtnasc_imp==dtnasc if ! missing(dtnasc_imp)
tab dtnasc if missing(dtnasc_imp), sort

[Paula de Souza Leao Spinola]

Hi William Lisowski. Sorry I do not have a unique identifier in the dataset, this is why I created one. Because I used the same DBF file to convert to DTA (with Stat/Transfer as well as with command -import dbase- within Stata), I assumed the order is the same. I also ran the code you suggested to check if dtnasc is the same among the two datasets when it is not missing, and this seems to be the case (there is no output to the command -assert-). I cannot see any particular pattern to non-missing values that Stata has converted to missing ones. You can find the output to your code below.

. use SINASC_stat_transfer, clear

. generate id = _n

. codebook dtnasc

--------------------------------------------------------------------------------------------------------------------------------------------
dtnasc                                                                                                                           (unlabeled)
--------------------------------------------------------------------------------------------------------------------------------------------

                  type:  string (str8)

         unique values:  366                      missing "":  0/3,256,433

              examples:  "06111999"
                         "12121999"
                         "19021999"
                         "25031999"

. save SINASC_stat_transfer, replace 
file SINASC_stat_transfer.dta saved

. 
. clear

. import dbase using dn99, clear case(lower)
(21 vars, 3,256,433 obs)

. generate id = _n

. rename dtnasc dtnasc_imp

. keep id dtnasc_imp

. merge 1:1 id using SINASC_stat_transfer

    Result                           # of obs.
    -----------------------------------------
    not matched                             0
    matched                         3,256,433  (_merge==3)
    -----------------------------------------

. assert dtnasc_imp==dtnasc if ! missing(dtnasc_imp)

. tab dtnasc if missing(dtnasc_imp), sort

     dtnasc |      Freq.     Percent        Cum.
------------+-----------------------------------
   01011999 |     25,161        2.53        2.53
   30031999 |      3,875        0.39        2.92
   20041999 |      3,751        0.38        3.30
   29031999 |      3,730        0.37        3.67
   05041999 |      3,724        0.37        4.04
   27041999 |      3,693        0.37        4.42
   10031999 |      3,688        0.37        4.79
   16041999 |      3,677        0.37        5.16
   04031999 |      3,662        0.37        5.52
   02031999 |      3,656        0.37        5.89
   12041999 |      3,650        0.37        6.26
   09041999 |      3,637        0.37        6.62
   31031999 |      3,631        0.36        6.99
   10021999 |      3,626        0.36        7.35
   03031999 |      3,617        0.36        7.72
   08041999 |      3,615        0.36        8.08
   19031999 |      3,614        0.36        8.44
   14041999 |      3,613        0.36        8.81
   11031999 |      3,610        0.36        9.17
   22031999 |      3,603        0.36        9.53
   23041999 |      3,598        0.36        9.89
   25051999 |      3,598        0.36       10.26
   15031999 |      3,592        0.36       10.62
   06041999 |      3,590        0.36       10.98
   07041999 |      3,585        0.36       11.34
   11021999 |      3,584        0.36       11.70
   19041999 |      3,583        0.36       12.06
   23031999 |      3,583        0.36       12.42
   30041999 |      3,583        0.36       12.78
   26031999 |      3,576        0.36       13.14
   26041999 |      3,571        0.36       13.50
   01031999 |      3,569        0.36       13.86
   12031999 |      3,569        0.36       14.21
   25031999 |      3,565        0.36       14.57
   05031999 |      3,561        0.36       14.93
   08031999 |      3,559        0.36       15.29
   14051999 |      3,555        0.36       15.65
   06051999 |      3,546        0.36       16.00
   09031999 |      3,542        0.36       16.36
   03051999 |      3,539        0.36       16.71
   18031999 |      3,536        0.36       17.07
   13041999 |      3,526        0.35       17.42
   15041999 |      3,519        0.35       17.78
   10051999 |      3,517        0.35       18.13
   19051999 |      3,499        0.35       18.48
   17031999 |      3,493        0.35       18.83
   18021999 |      3,492        0.35       19.18
   05051999 |      3,489        0.35       19.54
   09021999 |      3,484        0.35       19.89
   18051999 |      3,478        0.35       20.24
   24031999 |      3,478        0.35       20.59
   08021999 |      3,477        0.35       20.93
   16031999 |      3,475        0.35       21.28
   17051999 |      3,475        0.35       21.63
   12051999 |      3,472        0.35       21.98
   23021999 |      3,472        0.35       22.33
   21051999 |      3,469        0.35       22.68
   22021999 |      3,455        0.35       23.03
   26021999 |      3,454        0.35       23.37
   04051999 |      3,449        0.35       23.72
   11051999 |      3,445        0.35       24.07
   12011999 |      3,442        0.35       24.41
   12021999 |      3,435        0.35       24.76
   19021999 |      3,431        0.34       25.10
   25021999 |      3,430        0.34       25.45
   26051999 |      3,429        0.34       25.79
   24051999 |      3,428        0.34       26.14
   20051999 |      3,425        0.34       26.48
   28051999 |      3,424        0.34       26.83
   04011999 |      3,423        0.34       27.17
   31051999 |      3,422        0.34       27.51
   20011999 |      3,412        0.34       27.86
   18011999 |      3,408        0.34       28.20
   01061999 |      3,403        0.34       28.54
   02061999 |      3,398        0.34       28.88
   05021999 |      3,393        0.34       29.22
   24021999 |      3,393        0.34       29.57
   28041999 |      3,388        0.34       29.91
   05011999 |      3,384        0.34       30.25
   07051999 |      3,384        0.34       30.59
   04021999 |      3,381        0.34       30.93
   25011999 |      3,368        0.34       31.26
   06011999 |      3,362        0.34       31.60
   29041999 |      3,358        0.34       31.94
   13051999 |      3,344        0.34       32.28
   08061999 |      3,341        0.34       32.61
   02021999 |      3,337        0.34       32.95
   27051999 |      3,336        0.34       33.28
   22041999 |      3,321        0.33       33.62
   07061999 |      3,310        0.33       33.95
   11011999 |      3,300        0.33       34.28
   15061999 |      3,275        0.33       34.61
   03021999 |      3,272        0.33       34.94
   15011999 |      3,270        0.33       35.27
   14061999 |      3,258        0.33       35.60
   28011999 |      3,258        0.33       35.92
   16061999 |      3,256        0.33       36.25
   26011999 |      3,256        0.33       36.58
   18061999 |      3,247        0.33       36.90
   19011999 |      3,247        0.33       37.23
   22011999 |      3,245        0.33       37.56
   21011999 |      3,238        0.33       37.88
   01041999 |      3,237        0.33       38.21
   21061999 |      3,233        0.32       38.53
   22061999 |      3,233        0.32       38.86
   08011999 |      3,226        0.32       39.18
   10061999 |      3,224        0.32       39.51
   01021999 |      3,216        0.32       39.83
   27011999 |      3,206        0.32       40.15
   13011999 |      3,193        0.32       40.47
   14011999 |      3,184        0.32       40.79
   07011999 |      3,146        0.32       41.11
   25061999 |      3,146        0.32       41.42
   28061999 |      3,143        0.32       41.74
   04061999 |      3,133        0.31       42.05
   29011999 |      3,128        0.31       42.37
   05071999 |      3,113        0.31       42.68
   23061999 |      3,112        0.31       43.00
   17061999 |      3,110        0.31       43.31
   21041999 |      3,110        0.31       43.62
   24041999 |      3,106        0.31       43.93
   11061999 |      3,090        0.31       44.24
   09061999 |      3,088        0.31       44.55
   20021999 |      3,086        0.31       44.86
   17021999 |      3,085        0.31       45.17
   07071999 |      3,084        0.31       45.48
   12071999 |      3,080        0.31       45.79
   15051999 |      3,077        0.31       46.10
   01051999 |      3,070        0.31       46.41
   10041999 |      3,069        0.31       46.72
   17041999 |      3,066        0.31       47.03
   06031999 |      3,057        0.31       47.34
   01071999 |      3,035        0.31       47.64
   09071999 |      3,024        0.30       47.94
   06071999 |      3,017        0.30       48.25
   27031999 |      3,013        0.30       48.55
   20071999 |      3,009        0.30       48.85
   15071999 |      3,007        0.30       49.16
   24061999 |      2,999        0.30       49.46
   08051999 |      2,995        0.30       49.76
   29061999 |      2,990        0.30       50.06
   30061999 |      2,989        0.30       50.36
   20031999 |      2,978        0.30       50.66
   09081999 |      2,969        0.30       50.96
   19071999 |      2,962        0.30       51.25
   15021999 |      2,957        0.30       51.55
   13021999 |      2,955        0.30       51.85
   02071999 |      2,954        0.30       52.15
   30011999 |      2,951        0.30       52.44
   13031999 |      2,943        0.30       52.74
   08071999 |      2,935        0.30       53.03
   06021999 |      2,927        0.29       53.33
   13071999 |      2,923        0.29       53.62
   10081999 |      2,910        0.29       53.91
   27021999 |      2,904        0.29       54.21
   27071999 |      2,903        0.29       54.50
   03041999 |      2,899        0.29       54.79
   12081999 |      2,895        0.29       55.08
   03061999 |      2,893        0.29       55.37
   22051999 |      2,891        0.29       55.66
   21071999 |      2,886        0.29       55.95
   09011999 |      2,884        0.29       56.24
   05081999 |      2,868        0.29       56.53
   14071999 |      2,863        0.29       56.82
   16071999 |      2,857        0.29       57.10
   26071999 |      2,846        0.29       57.39
   23011999 |      2,837        0.29       57.68
   29051999 |      2,836        0.29       57.96
   02041999 |      2,834        0.28       58.25
   30071999 |      2,825        0.28       58.53
   28071999 |      2,818        0.28       58.81
   03081999 |      2,814        0.28       59.10
   21031999 |      2,808        0.28       59.38
   25041999 |      2,806        0.28       59.66
   28031999 |      2,803        0.28       59.94
   16081999 |      2,791        0.28       60.22
   23081999 |      2,779        0.28       60.50
   11081999 |      2,778        0.28       60.78
   12061999 |      2,776        0.28       61.06
   11041999 |      2,770        0.28       61.34
   02081999 |      2,768        0.28       61.62
   16021999 |      2,766        0.28       61.89
   23071999 |      2,763        0.28       62.17
   20081999 |      2,757        0.28       62.45
   29071999 |      2,748        0.28       62.73
   05061999 |      2,746        0.28       63.00
   16011999 |      2,746        0.28       63.28
   04081999 |      2,740        0.28       63.55
   09091999 |      2,725        0.27       63.83
   22071999 |      2,724        0.27       64.10
   04041999 |      2,720        0.27       64.37
   02011999 |      2,719        0.27       64.65
   02051999 |      2,713        0.27       64.92
   07021999 |      2,713        0.27       65.19
   20091999 |      2,712        0.27       65.47
   28021999 |      2,705        0.27       65.74
   18041999 |      2,702        0.27       66.01
   19061999 |      2,702        0.27       66.28
   01091999 |      2,693        0.27       66.55
   06081999 |      2,688        0.27       66.82
   07031999 |      2,683        0.27       67.09
   23051999 |      2,682        0.27       67.36
   30051999 |      2,669        0.27       67.63
   24011999 |      2,659        0.27       67.90
   14031999 |      2,658        0.27       68.16
   27081999 |      2,657        0.27       68.43
   14021999 |      2,644        0.27       68.70
   26061999 |      2,638        0.27       68.96
   27091999 |      2,638        0.27       69.23
   19081999 |      2,636        0.26       69.49
   24081999 |      2,628        0.26       69.76
   10011999 |      2,627        0.26       70.02
   18081999 |      2,623        0.26       70.28
   17011999 |      2,620        0.26       70.55
   03011999 |      2,619        0.26       70.81
   21021999 |      2,618        0.26       71.07
   21091999 |      2,617        0.26       71.34
   09051999 |      2,613        0.26       71.60
   02091999 |      2,607        0.26       71.86
   13091999 |      2,607        0.26       72.12
   26081999 |      2,604        0.26       72.38
   30081999 |      2,604        0.26       72.65
   25081999 |      2,602        0.26       72.91
   10091999 |      2,601        0.26       73.17
   16091999 |      2,600        0.26       73.43
   14091999 |      2,594        0.26       73.69
   17091999 |      2,567        0.26       73.95
   31011999 |      2,563        0.26       74.21
   03071999 |      2,558        0.26       74.46
   08091999 |      2,555        0.26       74.72
   13081999 |      2,552        0.26       74.98
   10071999 |      2,550        0.26       75.23
   17081999 |      2,550        0.26       75.49
   13061999 |      2,547        0.26       75.75
   16051999 |      2,544        0.26       76.00
   22091999 |      2,534        0.25       76.26
   06091999 |      2,523        0.25       76.51
   06061999 |      2,519        0.25       76.76
   17071999 |      2,502        0.25       77.01
   28091999 |      2,502        0.25       77.27
   31081999 |      2,501        0.25       77.52
   15091999 |      2,495        0.25       77.77
   03091999 |      2,487        0.25       78.02
   23091999 |      2,474        0.25       78.27
   20061999 |      2,460        0.25       78.51
   14081999 |      2,454        0.25       78.76
   24091999 |      2,447        0.25       79.01
   29091999 |      2,408        0.24       79.25
   04071999 |      2,400        0.24       79.49
   27061999 |      2,399        0.24       79.73
   30091999 |      2,371        0.24       79.97
   31071999 |      2,345        0.24       80.21
   24071999 |      2,341        0.24       80.44
   11071999 |      2,307        0.23       80.67
   28081999 |      2,298        0.23       80.90
   07081999 |      2,279        0.23       81.13
   08081999 |      2,243        0.23       81.36
   18071999 |      2,237        0.22       81.58
   21081999 |      2,236        0.22       81.81
   25071999 |      2,225        0.22       82.03
   18091999 |      2,213        0.22       82.25
   11091999 |      2,198        0.22       82.47
   19091999 |      2,161        0.22       82.69
   04091999 |      2,153        0.22       82.91
   01081999 |      2,141        0.22       83.12
   07091999 |      2,139        0.22       83.34
   15081999 |      2,136        0.21       83.55
   25091999 |      2,128        0.21       83.77
   22081999 |      2,121        0.21       83.98
   12091999 |      2,106        0.21       84.19
   04101999 |      2,102        0.21       84.40
   08101999 |      2,051        0.21       84.61
   14101999 |      2,043        0.21       84.81
   01101999 |      2,037        0.20       85.02
   07101999 |      2,035        0.20       85.22
   20101999 |      2,009        0.20       85.43
   26091999 |      2,002        0.20       85.63
   06101999 |      2,001        0.20       85.83
   05101999 |      1,995        0.20       86.03
   15101999 |      1,984        0.20       86.23
   29081999 |      1,980        0.20       86.43
   11101999 |      1,957        0.20       86.62
   19101999 |      1,939        0.19       86.82
   21101999 |      1,938        0.19       87.01
   05091999 |      1,928        0.19       87.21
   08111999 |      1,920        0.19       87.40
   18101999 |      1,901        0.19       87.59
   05111999 |      1,896        0.19       87.78
   22101999 |      1,885        0.19       87.97
   04111999 |      1,883        0.19       88.16
   27101999 |      1,883        0.19       88.35
   28101999 |      1,873        0.19       88.54
   25101999 |      1,868        0.19       88.73
   10111999 |      1,861        0.19       88.91
   09111999 |      1,851        0.19       89.10
   13101999 |      1,844        0.19       89.28
   26101999 |      1,823        0.18       89.47
   17111999 |      1,820        0.18       89.65
   19111999 |      1,816        0.18       89.83
   16111999 |      1,800        0.18       90.01
   03111999 |      1,788        0.18       90.19
   11111999 |      1,776        0.18       90.37
   22111999 |      1,765        0.18       90.55
   18111999 |      1,755        0.18       90.73
   01111999 |      1,750        0.18       90.90
   09101999 |      1,747        0.18       91.08
   29101999 |      1,741        0.17       91.25
   12111999 |      1,730        0.17       91.43
   02101999 |      1,726        0.17       91.60
   26111999 |      1,717        0.17       91.77
   06121999 |      1,691        0.17       91.94
   20121999 |      1,690        0.17       92.11
   23111999 |      1,682        0.17       92.28
   12101999 |      1,673        0.17       92.45
   29111999 |      1,663        0.17       92.62
   01121999 |      1,656        0.17       92.78
   15121999 |      1,656        0.17       92.95
   02121999 |      1,650        0.17       93.12
   23101999 |      1,640        0.16       93.28
   16101999 |      1,625        0.16       93.44
   10121999 |      1,621        0.16       93.61
   17121999 |      1,621        0.16       93.77
   16121999 |      1,620        0.16       93.93
   03121999 |      1,616        0.16       94.10
   25111999 |      1,606        0.16       94.26
   24111999 |      1,602        0.16       94.42
   30111999 |      1,594        0.16       94.58
   21121999 |      1,592        0.16       94.74
   06111999 |      1,591        0.16       94.90
   22121999 |      1,582        0.16       95.06
   13111999 |      1,579        0.16       95.22
   30101999 |      1,579        0.16       95.37
   15111999 |      1,552        0.16       95.53
   03101999 |      1,551        0.16       95.69
   07121999 |      1,546        0.16       95.84
   09121999 |      1,536        0.15       96.00
   24101999 |      1,525        0.15       96.15
   10101999 |      1,519        0.15       96.30
   13121999 |      1,510        0.15       96.45
   28121999 |      1,510        0.15       96.61
   14121999 |      1,508        0.15       96.76
   07111999 |      1,499        0.15       96.91
   27121999 |      1,480        0.15       97.06
   02111999 |      1,477        0.15       97.21
   08121999 |      1,470        0.15       97.35
   20111999 |      1,470        0.15       97.50
   17101999 |      1,468        0.15       97.65
   31101999 |      1,465        0.15       97.80
   23121999 |      1,457        0.15       97.94
   27111999 |      1,439        0.14       98.09
   29121999 |      1,395        0.14       98.23
   14111999 |      1,359        0.14       98.36
   21111999 |      1,348        0.14       98.50
   04121999 |      1,341        0.13       98.63
   12121999 |      1,328        0.13       98.77
   18121999 |      1,326        0.13       98.90
   28111999 |      1,308        0.13       99.03
   11121999 |      1,279        0.13       99.16
   24121999 |      1,277        0.13       99.29
   05121999 |      1,266        0.13       99.42
   30121999 |      1,219        0.12       99.54
   26121999 |      1,217        0.12       99.66
   19121999 |      1,207        0.12       99.78
   25121999 |      1,084        0.11       99.89
   31121999 |      1,082        0.11      100.00
------------+-----------------------------------
      Total |    994,858      100.00

. 
end of do-file

.

[William Lisowski]

Good! I think we've gone as far as we can to rule out the most obvious explanations.

I think this is a problem that should be reported to Stata Technical Services. The ado file for the import dbase command calls an (undocumented?) Mata function so there's nothing much to be learned by peeking behind the scenes. Even if there's some system limit that you've passed with the number of observations or some such thing, quietly wiping out data is not an effective error message.

Please let us know what you find out.

[Paula de Souza Leao Spinola]

Sure. I will do that and will let you know. For the time being I used the command below to convert the datasets from DBF to DTA, which keeps non-missing values as in the original file.

global StatTransfer_path `"C:\Users\Paula\Dropbox\Pesquisa\Softwares\StatTransfer7\st.exe"'
foreach a in 97 98 99 {
foreach i in AC AL AM AP BA CE DF ES GO MA MG MS MT PA PB PE PI PR RJ RN RO RR RS SC SE SP TO {
inputst "C:\Users\Paula\Dropbox\Pesquisa\DADOS\DATASUS\SINASC\dados_brutos\website\DN_dbf\DN`i'19`a'.dbf"
save "C:\Users\Paula\Dropbox\Pesquisa\DADOS\DATASUS\SINASC\dados_brutos\website\DN_dta\DN`i'`a'.dta", replace
}
}