repeated time values within panel

[Gabriel Bladh]

Dear Statalisters!

I have problems with my time variable. I tried to use this commando:

xtset importer1 Year
But I get this error message:

repeated time values within panel

My data looks like this:




I encoded Importer and Year with this commando:

encode Importer, gen(importer1)
encode Year1, gen(Year)




In my data, i have 27 Importers and around 165 Exporters. I want to examine how imports from the Exporters to the Importers change during crisis and depending on if the Importer have euro or not.
The problem seems to be that the same year is used several time for the same Importer. Is it even possible for me to use panel data with my data set? If it is, how should I proceed?

Best reg(ression)ards,

Gabriel Bladh
Stockholm
Sweden

[Clyde Schechter]

Well, your screenshots of the data are unreadable. That is one of the reasons why it is made clear in the FAQ that you should never use screenshots to show data. So before you re-post, please read the Forum FAQ, with emphasis on #12 to learn the effective and helpful ways to show data, code and results.

That said, your question as to whether this data can be treated as panel data cannot be answered in the abstract. It depends on specifics that might only be known actually seeing representative data. But here are some things to consider.

1. Perhaps the repeated use of the same year for a firm represents an error in the data. You can identify the observations with -duplicates tag importer1 year1, gen(flag)- and then -browse if flag-. If you see that these are simply entirely duplicate observations (on all variables) then you can run -duplicates drop- and rid yourself of the problem. If, however, these surplus observations differ on other variables, then you may not be able to rid yourself of the surplus unless you can figure out which one is correct.

2. Perhaps the repeated firm1 year1 combinations are not an error. Perhaps the reason that they are there is that the different observations refer to, let's say, different subdivisions of the same firm. In that case, you need to have a variable in your data set which identifies the subdivision. And then you need the panel variable to be not the firm, but the combination of firm and subdivision. So -egen panel_var = group(firm1 subdivision)- and -xtset panel_var year1-.

3. Perhaps the repeated firm1 year1 combinations are not an error. Perhaps the reason they are there is that the observations refer to more specific time periods such as quarters or months, etc. In that case you need to generate a more precise date variable (see -help datetime-, this gets complicated) and then you can -xtset firm1 precise_date_variable-.

4. Perhaps the repeated observations are not an error but neither of the above applies. Well, then you cannot treat this as panel variable in the strict sense. But maybe you don't need to. -xtset- does not require a time variable. You can just -xtset firm1- and for most purposes everything will be fine. What you lose when you have no time variable in your -xtset- is the ability to use time-series operators like lag, lead, seasonal difference, etc., and the ability to fit models with autoregressive correlation structure. But if you don't need any of those things, then there is no reason to bother with a time variable.

[Gabriel Bladh]

Thanks for the reply and sorry for the inconvenience. I will definitely try your suggestions, much appriciated.

In the dataex below, it´s just Austria as Importer and several other countries as Exporters for 2004. In the original data set, these observations continues from year 2004 to 2016. There, it´s also 26 more countries as Importers and the same amount of countries as Exporters.

PS. Is this post to messy and should i create a new post with this dataex instead?

* Example generated by -dataex-. 
clear
input str15 Importer str52 Exporter double Import int Year1 float logGDPeu long importer1 float Year
"Austria" "Liberia"                             76.164 2004 26.43006 1 1
"Austria" "Bermuda"                               2.16 2004 26.43006 1 1
"Austria" "Maldives"                            73.118 2004 26.43006 1 1
"Austria" "Singapore"                       174023.104 2004 26.43006 1 1
"Austria" "Bolivia"                           2667.613 2004 26.43006 1 1
"Austria" "Canada"                          360244.305 2004 26.43006 1 1
"Austria" "Israel"                          109582.968 2004 26.43006 1 1
"Austria" "Micronesia, Fed. Sts."                2.151 2004 26.43006 1 1
"Austria" "Bahrain"                           2845.958 2004 26.43006 1 1
"Austria" "Botswana"                            99.747 2004 26.43006 1 1
"Austria" "Guinea-Bissau"                        2.041 2004 26.43006 1 1
"Austria" "Macedonia, FYR"                   11286.426 2004 26.43006 1 1
"Austria" "Bahamas"                            145.849 2004 26.43006 1 1
"Austria" "Kuwait"                             204.341 2004 26.43006 1 1
"Austria" "Uruguay"                           1948.936 2004 26.43006 1 1
"Austria" "Ghana"                              9039.73 2004 26.43006 1 1
"Austria" "Nauru"                                1.166 2004 26.43006 1 1
"Austria" "Ethiopia"                          1672.655 2004 26.43006 1 1
"Austria" "Uzbekistan"                       61804.053 2004 26.43006 1 1
"Austria" "Niger"                                7.473 2004 26.43006 1 1
"Austria" "United Arab Emirates"              9646.941 2004 26.43006 1 1
"Austria" "Norway"                          359795.556 2004 26.43006 1 1
"Austria" "Comoros"                            390.154 2004 26.43006 1 1
"Austria" "Colombia"                         17455.683 2004 26.43006 1 1
"Austria" "Tunisia"                          62569.843 2004 26.43006 1 1
"Austria" "Gambia"                              43.908 2004 26.43006 1 1
"Austria" "India"                            300400.04 2004 26.43006 1 1
"Austria" "Senegal"                            360.704 2004 26.43006 1 1
"Austria" "Congo, Dem. Rep."                   392.619 2004 26.43006 1 1
"Austria" "Cameroon"                          7124.162 2004 26.43006 1 1
"Austria" "Argentina"                       103877.742 2004 26.43006 1 1
"Austria" "Brunei"                              63.215 2004 26.43006 1 1
"Austria" "St. Vincent and the Grenadines"      25.702 2004 26.43006 1 1
"Austria" "Croatia"                         726079.085 2004 26.43006 1 1
"Austria" "Honduras"                          6354.829 2004 26.43006 1 1
"Austria" "Benin"                                7.579 2004 26.43006 1 1
"Austria" "Ecuador"                          46917.015 2004 26.43006 1 1
"Austria" "Oman"                                637.23 2004 26.43006 1 1
"Austria" "Cayman Islands"                       1.599 2004 26.43006 1 1
"Austria" "East Timor"                           2.735 2004 26.43006 1 1
"Austria" "Palau"                                 .017 2004 26.43006 1 1
"Austria" "Jamaica"                           1681.759 2004 26.43006 1 1
"Austria" "Dominican Republic"                5822.215 2004 26.43006 1 1
"Austria" "Congo"                             1753.577 2004 26.43006 1 1
"Austria" "Costa Rica"                       43950.012 2004 26.43006 1 1
"Austria" "Philippines"                     189110.487 2004 26.43006 1 1
"Austria" "United States of America"       3678493.061 2004 26.43006 1 1
"Austria" "Kyrgyzstan"                         271.504 2004 26.43006 1 1
"Austria" "Haiti"                              110.585 2004 26.43006 1 1
"Austria" "Northern Mariana Islands"            261.38 2004 26.43006 1 1
"Austria" "Paraguay"                           376.487 2004 26.43006 1 1
"Austria" "Antigua and Barbuda"                 10.321 2004 26.43006 1 1
"Austria" "Seychelles"                         928.542 2004 26.43006 1 1
"Austria" "Korea, Rep."                     844871.825 2004 26.43006 1 1
"Austria" "Guinea"                            1206.792 2004 26.43006 1 1
"Austria" "Madagascar"                        3670.252 2004 26.43006 1 1
"Austria" "Mexico"                          126164.583 2004 26.43006 1 1
"Austria" "Andorra"                            894.345 2004 26.43006 1 1
"Austria" "Guyana"                              41.972 2004 26.43006 1 1
"Austria" "Azerbaijan"                       43783.961 2004 26.43006 1 1
"Austria" "Mauritania"                          59.044 2004 26.43006 1 1
"Austria" "Japan"                          2457270.973 2004 26.43006 1 1
"Austria" "Hong Kong"                        200660.02 2004 26.43006 1 1
"Austria" "Algeria"                           82947.54 2004 26.43006 1 1
"Austria" "Fiji"                                 60.89 2004 26.43006 1 1
"Austria" "Panama"                           12915.031 2004 26.43006 1 1
"Austria" "Somalia"                             25.891 2004 26.43006 1 1
"Austria" "Chad"                               145.624 2004 26.43006 1 1
"Austria" "Peru"                              8517.259 2004 26.43006 1 1
"Austria" "Jordan"                             568.076 2004 26.43006 1 1
"Austria" "Lao PDR"                           3193.498 2004 26.43006 1 1
"Austria" "Nigeria"                          207724.45 2004 26.43006 1 1
"Austria" "Tonga"                               14.025 2004 26.43006 1 1
"Austria" "Cambodia"                         27775.647 2004 26.43006 1 1
"Austria" "Thailand"                        360900.645 2004 26.43006 1 1
"Austria" "Indonesia"                       187491.765 2004 26.43006 1 1
"Austria" "Turkmenistan"                     13574.507 2004 26.43006 1 1
"Austria" "Albania"                           5559.538 2004 26.43006 1 1
"Austria" "Equatorial Guinea"                    17.48 2004 26.43006 1 1
"Austria" "Turkey"                          916384.393 2004 26.43006 1 1
"Austria" "Lesotho"                                 .2 2004 26.43006 1 1
"Austria" "Australia"                        70980.683 2004 26.43006 1 1
"Austria" "Mongolia"                           589.612 2004 26.43006 1 1
"Austria" "St. Kitts and Nevis"                184.813 2004 26.43006 1 1
"Austria" "China"                           2747319.02 2004 26.43006 1 1
"Austria" "Ukraine"                         418016.993 2004 26.43006 1 1
"Austria" "Mozambique"                        18999.17 2004 26.43006 1 1
"Austria" "Belize"                             225.084 2004 26.43006 1 1
"Austria" "Cuba"                              2258.336 2004 26.43006 1 1
"Austria" "Iceland"                           4490.293 2004 26.43006 1 1
"Austria" "El Salvador"                       3458.066 2004 26.43006 1 1
"Austria" "Cape Verde"                           8.441 2004 26.43006 1 1
"Austria" "Samoa"                               57.311 2004 26.43006 1 1
"Austria" "Brazil"                          360734.963 2004 26.43006 1 1
"Austria" "Dominica"                           349.772 2004 26.43006 1 1
"Austria" "Malawi"                            3573.884 2004 26.43006 1 1
"Austria" "Belarus"                           22677.83 2004 26.43006 1 1
"Austria" "New Zealand"                       44707.87 2004 26.43006 1 1
"Austria" "Tajikistan"                        6976.158 2004 26.43006 1 1
"Austria" "Uganda"                            1494.635 2004 26.43006 1 1
end
label values importer1 importer1
label def importer1 1 "Austria", modify

[Clyde Schechter]

So this is rather different from what you originally described. There are no firms in the data you show. There are pairs of Importer and Exporter countries, as well as years and then certain economic statistics. At least within the example data you show, the combination of Importer, Exporter, and Year1 uniquely identify the observations. You can check if this is true in your fall data set by running

isid Importer Exporter Year1

If these variables do jointly identify observations uniquely, you will see no output from the -isid- command. If they do not, then you will get an error message.

Assuming they do uniquely identify observations, then you can manage this as panel data by setting the combination of Importer and Exporter as your panel variable:

egen panel_var = group(Importer Exporter)
xtset panel_var Year1

This will enable you to use the full range of panel-data analyses Stata has to offer. Of course, in all of those analyses, if you are using fixed effects, it will be the importer-exporter pair that gives the fixed effect, and it will not be possible to separately estimate the effects of importers or separately estimate the effects of exporters because those will be colinear with the pair fixed effects.

By the way, the -dataex- output you showed was perfectly fine. Not sure what you thought was "too messy" about it.

[Gabriel Bladh]

Thanks for the reply!

I also thought that the variables would uniquely identify, but unfortunately they do not. I had this problem when I merged the data, where stata said that they did not uniquely identify, but I manage to solve the problem then. Do you have any suggestions about what could be the problem now? I have checked for duplicates and there is none in our data.

Best regards

[Nick Cox]

Tell us what your check for duplicates was, exactly.

[Gabriel Bladh]

My check for duplicates was:

Duplicates report

And when I removed the duplicates, I used this code:

sort Importer Exporter Import Year1
quietly by Importer Exporter Import Year1: gen dup = cond(_N==1,0,_n)
tabulate dup
drop if dup>1

[Nick Cox]

That was my private guess on what you would say. But

duplicates report

checks for duplicates across all variables. The issue is

duplicates report Importer Exporter Year1

and -- rather than just removing such duplicates -- the question should be why they occur at all and what that tells you.

[Gabriel Bladh]

You are totally right!

When I used duplicates report, stata said that I had no surplus at all. But when I used the code duplicates report Importer Exporter Year1, stata says this:


----------------------------------------------
copies observations surplus

1 48722 0
2 324 162
----------------------------------------------


I am not quite sure why they occur and what that tells me, but the problem could maybe have occured during the data collection?

[Nick Cox]

https://www.stata.com/support/faqs/d...d-time-values/

[Eric de Souza]

What exactly do you wish to do with your data?
Do you wish to analyse the trade between Importer and Exporter?

Then take the variable defined in the line of your dataex example beginning with input:

input str15 Importer str52 Exporter double Import int Year1 float logGDPeu long importer1 float Year

Just drop importer1 and Year
Rename Year1 to year
Then if you wish to analyse trade between importers and exporters, do the following:

encode Importer, gen(importer)
encode Exporter, gen(exporter)
egen group = group(importer exporter)
tsset group year

egen group combines (Importer and Exporter into one variable: an Importer, Exporter pair.

[Clyde Schechter]

Re #9. Given that only a small fraction of the observations are duplicates, it is likely that these represent errors in the data themselves or data management. It is much less likely that these duplicates are supposed to be there and that we are overlooking some additional variable that is needed to obtain unique identification of observation.

So the first step is to look at the offending observations:

duplicates tag Import Exporter Year1, gen(flag)
browse if flag

When you look at them you might immediately recognize why they are there or what is wrong with them and how to fix it. If not, you will have to trace them back through the data management process to see where things went wrong. Or perhaps you will find that they are already in the original source of the data you worked from, in which case you will have to contact whoever created that source and discuss it with them.

[Eric de Souza]

When I asked in #11 above what you wished to do with your data, I missed the sentence "In my data, i have 27 Importers and around 165 Exporters. I want to examine how imports from the Exporters to the Importers change during crisis and depending on if the Importer have euro or not." in your first post
My suggestion in #11 holds.

[Gabriel Bladh]

Thanks for the replys, very kind of you!

#10. Thank you, that link helped me solve the problem!

#11. What I wish to do is that I have imports from ~165 exporting countries (Rest of the World) and 27 importing countries (EU minus Croatia) and I want to examine how imports is affected during the financial crisis 2009 depending of if the importing country had euro or not. I also have more variables than those in the dataex, mostly dummies.


#12. I have managed to trace the problem now and fix it! Atleast, there is not any surplus left. The problem was with one exporting country, where it was duplicates with every importer.
I then used the code from your post #4 and got this message:

xtset panel_var Year1
panel variable: panel_var (unbalanced)
time variable: Year1, 2004 to 2016, but with gaps
delta: 1

But if I understand stata right (which is very uncertain) there should not be any problems with the fact that panel_var is unbalanced and that the time variable are with gaps?



PS. I apoligize if my english, explanations or lack of knowledge in stata is inadequate.

[Clyde Schechter]

But if I understand stata right (which is very uncertain) there should not be any problems with the fact that panel_var is unbalanced and that the time variable are with gaps?

Correct, this is not a problem. Balanced data are important for a very small number of analyses, but none that you have mentioned in your plans. Nearly all of Stata's -xt- commands work correctly with unbalanced data. Those that do not will give you an error message if you try to use them.

I apoligize if my english, explanations or lack of knowledge in stata is inadequate.

No apologies called for. We all have difficulties working in languages other than our mother tongue, and your mother tongue is an accident of birth. Your English is perfectly clear for getting the meaning across--which is what matters here. As for lack of knowledge of Stata, we were all beginners once. Nobody is born knowing Stata, not even Bill Gould.