Bug: Python Error in Stata Function Interface (sfi), Unicode-related

[Ilya Bolotov]

Bug in SFI?
Running this Python code in Stata 16.1

sysuse world-d, clear
python: import sfi; print(sfi.Data.get('name'))

produces the following error:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf4 in position 1: invalid continuation byte

The above exception was the direct cause of the following exception:

SystemError: <class 'UnicodeDecodeError'> returned a result with an error set

The above exception was the direct cause of the following exception:

SystemError: <class 'UnicodeDecodeError'> returned a result with an error set

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Applications/Stata/ado/base/py/sfi.py", line 659, in get
    return _stp._st_getdata(vars, obss, svar, 1, bvalabel)
SystemError: <built-in function _st_getdata> returned a result with an error set
r(7102);

My Python version is

    Python system information
      version              3.8.6
      architecture         64-bit

[William Lisowski]

That's odd, in my installation that sysuse dataset does not exist. Presumably it was installed as sample data for some community contributed command?

. about

Stata/SE 16.1 for Mac (Intel 64-bit)
Revision 15 Dec 2020

...

. sysuse dir
  auto.dta                 tsline1.dta              zhtageus.dta
  auto2.dta                tsline2.dta              zlenageius.dta
  autornd.dta              uslifeexp.dta            zlhagewho.dta
  bplong.dta               uslifeexp2.dta           zlhtageukwhopreterm.dta
  bpwide.dta               voter.dta                zlhtageukwhoterm.dta
  cancer.dta               xtline1.dta              zllageuk.dta
  census.dta               zacagewho.dta            zshtageuk.dta
  citytemp.dta             zbfageuk.dta             zssagewho.dta
  citytemp4.dta            zbmiageuk.dta            ztsagewho.dta
  educ99gdp.dta            zbmiageukwhopreterm.dta  zwsageuk.dta
  github.dta               zbmiageukwhoterm.dta     zwtagecomus.dta
  gnp96.dta                zbmiageus.dta            zwtageuk.dta
  lifeexp.dta              zbmiagewho.dta           zwtageukwhopreterm.dta
  network1.dta             zbmicat.dta              zwtageukwhoterm.dta
  network1a.dta            zbmicateg.dta            zwtagewho.dta
  nlsw88.dta               zhcageius.dta            zwthtus.dta
  nlswide1.dta             zhcageuk.dta             zwthtwho.dta
  pop2000.dta              zhcageukwhopreterm.dta   zwtlenius.dta
  sandstone.dta            zhcageukwhoterm.dta      zwtlenwho.dta
  sp500.dta                zhcagewho.dta            zwtukeg.dta
  surface.dta              zhtageuk.dta             zwtukwhoeg.dta

.

[William Lisowski]

Aha, the data is installed as part of the wbopendata package. Having installed it, your code succeeded on my system, pointing perhaps to configuration issues rather than a bug in the interface. Or perhaps a version-related issue: I seem to have 2.7.16.

. sysuse world-d, clear

. describe

Contains data from /Users/lisowskiw/Library/Application Support/Stata/ado/plus/w/world-d.dta
  obs:           250                          
 vars:            14                          25 Jan 2012 05:32
-------------------------------------------------------------------------------------------------
              storage   display    value
variable name   type    format     label      variable label
-------------------------------------------------------------------------------------------------
objectid        int     %10.0g                
area            float   %9.0g                
perimeter       float   %9.0g                
wb_             int     %9.0g                
wb_id           int     %9.0g                
countrycode     str3    %9s                  
name            str39   %39s                  
symbol          str46   %46s                  
label           str16   %16s                  
labl_frmt       str42   %42s                  
shape_leng      float   %9.0g                
shape_area      float   %9.0g                
wb_country      str2    %9s                  
_ID             int     %12.0g                
-------------------------------------------------------------------------------------------------
Sorted by: countrycode

. python: import sfi; print(sfi.Data.get('name'))
['Aruba (Neth.)', 'Afghanistan', 'Angola', 'Albania', 'Andorra', 'Netherlands Antilles (Neth.)',
> 'United Arab Emirates', 'Argentina', 'Armenia', 'American Samoa (US)', 'Antigua and Barbuda', '
> Australia', 'Austria', 'Azerbaijan', 'Burundi', 'Belgium', 'Benin', 'Burkina Faso', 'Bangladesh
> ', 'Bulgaria', 'Bahrain', 'Bahamas, The', 'Bosnia and Herzegovina', 'Belarus', 'Belize', 'Bermu
> da (UK)', 'Bolivia', 'Brazil', 'Barbados', 'Brunei', 'Bhutan', 'Botswana', 'Central African Rep
> ublic', 'Canada', 'Switzerland', 'Channel Islands (UK)', 'Chile', 'China', "C\xf4te d'Ivoire",
> 'Cameroon', 'Dem. Rep. of Congo', 'Congo', 'Colombia', 'Comoros', 'Cape Verde', 'Costa Rica', '
> Cuba', 'Cayman Islands (UK)', 'Cyprus', 'Czech Republic', 'Germany', 'Djibouti', 'Dominica', 'D
> enmark', 'Dominican Republic', 'Algeria', 'Ecuador', 'Arab Rep. of Egypt', 'Eritrea', 'Spain',
> 'Estonia', 'Ethiopia', 'Finland', 'Fiji', 'France', 'Faeroe Islands (Den.)', 'Federated States
> of Micronesia', 'Gabon', 'United Kingdom', 'Georgia', 'Ghana', 'Gibraltar (UK)', 'Guinea', 'Gua
> deloupe (Fr.)', 'The Gambia', 'Guinea-Bissau', 'Equatorial Guinea', 'Greece', 'Grenada', 'Green
> land (Den.)', 'Guatemala', 'French Guiana (Fr.)', 'Guam (US)', 'Guyana', 'Honduras', 'Croatia',
>  'Haiti', 'Hungary', 'Indonesia', 'Isle of Man (UK)', 'India', 'Ireland', 'Islamic Republic of
> Iran', 'Iraq', 'Iceland', 'Israel', 'Italy', 'Jamaica', 'Jordan', 'Japan', 'Kazakhstan', 'Kenya
> ', 'Kyrgyz Republic', 'Cambodia', 'Kiribati', 'St. Kitts and Nevis', 'Rep. of Korea', 'Kuwait',
>  'Lao P.D.R.', 'Lebanon', 'Liberia', 'Libya', 'St. Lucia', 'Liechtenstein', 'Sri Lanka', 'Lesot
> ho', 'Lithuania', 'Luxembourg', 'Latvia', 'Macao, China', 'Morocco', 'Monaco', 'Moldova', 'Mada
> gascar', 'Maldives', 'Mexico', 'Marshall Islands', 'FYR Macedonia', 'Mali', 'Malta', 'Myanmar',
>  'Mongolia', 'N. Mariana Islands (US)', 'Mozambique', 'Mauritania', 'Martinique (Fr.)', 'Maurit
> ius', 'Malawi', 'Malaysia', 'Mayotte (Fr.)', 'Namibia', 'New Caledonia (Fr.)', 'Niger', 'Nigeri
> a', 'Nicaragua', 'Netherlands, The', 'Norway', 'Nepal', 'Nauru', 'New Zealand', 'Oman', 'Pakist
> an', 'Panama', 'Peru', 'Philippines', 'Palau', 'Papua New Guinea', 'Poland', 'Puerto Rico (US)'
> , "Dem. People's Rep. of Korea", 'Portugal', 'Paraguay', 'West Bank and Gaza', 'French Polynesi
> a (Fr.)', 'Qatar', 'R\xe9union (Fr.)', 'Romania', 'Russian Federation', 'Rwanda', 'Saudi Arabia
> ', 'Sudan', 'Senegal', 'Singapore', 'Solomon Islands', 'Sierra Leone', 'El Salvador', 'San Mari
> no', 'Former Spanish Sahara', 'Somalia', 'S\xe3o Tom\xe9 and Pr\xedncipe', 'Suriname', 'Slovak
> Republic', 'Slovenia', 'Sweden', 'Swaziland', 'Seychelles', 'Syrian Arab Rep.', 'Chad', 'Togo',
>  'Thailand', 'Tajikistan', 'Turkmenistan', 'Timor-Leste', 'Tonga', 'Trinidad and Tobago', 'Tuni
> sia', 'Turkey', 'Tuvalu', 'Taiwan, China', 'Tanzania', 'Uganda', 'Ukraine', 'Uruguay', 'United
> States', 'Uzbekistan', 'St. Vincent and the Grenadines', 'R.B. de Venezuela', 'U.S. Virgin Isla
> nds (U.S.)', 'Vietnam', 'Vanuatu', 'Samoa', 'Anguilla', 'Falkland Islands (Islas Malvinas)', 'F
> rench Southern & Antarctic Lands', 'Glorioso Islands', 'Heard Island & McDonald Islands', 'Howl
> and Island', 'Jan Mayen', 'Jarvis Island', 'Johnston Atoll', 'Juan De Nova Island', 'Midway Isl
> ands', 'Antarctica', 'Montserrat', 'Niue', 'Norfolk Island', 'Pitcairn Islands', 'South Georgia
>  and the South Sandwich Is', 'St. Helena', 'St. Pierre and Miquelon', 'Svalbaard', 'Baker Islan
> d', 'Tokelau', 'Turks and Caicos Islands', 'Wake Island', 'Wallis and Futuna', '', '', '', 'Bou
> vet Island', 'British Indian Ocean Territory', 'British Virgin Islands (UK)', 'Christmas Island
> ', 'Cocos (Keeling) Islands', 'Cook Islands', 'Rep. of Yemen', 'Serbia and Montenegro', 'South
> Africa', 'Zambia', 'Zimbabwe']

.

[Bjarte Aagnes]

Try:

unicode translate world-d.dta
sysuse world-d, clear
python: import sfi; print(sfi.Data.get('name'))

[Ilya Bolotov]

Great, thanks! Problem solved.