_ID error with Spatial Weight Matrix

[Ben Spoer]

Good afternoon Statalist,
TL:DR - estat moran, errorlag(matrixname) produces an error, "_IDs in weighting matrix Widist do not match _IDs in estimation sample
There are places in Widist not in estimation sample and places in estimation sample not in Widist.", and I am not sure how to solve it.

Further detail:
I am trying to test for spatial auto-correlation in my data using Moran's I. I have followed the steps outlined in Intro 7 in SP manual, including the following:

• Download a shape file from the web
• Unzip said shape file and import it into STATA using spshape2dta
• Create a shared ID variable to use to merge into my data
• Open my data set and merge the spatial data into my dataset, used "keep if _merge ==3" to retain only matched records
• Created a spatial weight matrix called Widist using "spmatrix create"
• Ran regress on a few variables of interest
• Attempted to run "estat moran, errorlag(Widist)"

However, when I attempt run the estat command, STATA produces the following error:
"_IDs in weighting matrix Widist do not match _IDs in estimation sample
There are places in Widist not in estimation sample and places in estimation sample not in Widist."
I cannot image how this is possible considering the _ID var is generated by STATA, and I merged that variable in from the sp dataset I created. I also did not drop any data until after the merge, and that was only as per the manual's instructions.
Can anyone provide some insight?
Greatfully,
Ben Spoer

[Nick Garcia]

Struggled for days with the same error. Realized that when I performed the estat moran command after creating the contiguity matrix, I didn't notice the change in the sample size used in the regression. Did a quick sum command on my independent variables and found that one of my IV's was missing 26 cases. I dropped those cases, reformed the spmatrix, reran the regression, and the estat command worked.
***Also: Don't forget to spmatrix clear between sessions.

[Michael Coppedge]

I am trying to run -spregress, gs2sls- with multiple 720*720 custom weighting matrices. Generic version of my code:

-spregress DV X1 - Xn, gs2sls dvarlag(W1) errorlag(W1) dvarlag(W2) errorlag(W2) force-

The dataset has 720 observations, although with some missing values on X variables. In an earlier model with square weighting matrices of size 10860, I was always able to get spregress results by specifying the force option; then Stata would automatically create subsets of my custom weighting matrices to use in the calculation. But now, with the smaller matrices, I get the following errors:

(720 observations)
(24 observations excluded due to missing values)
(696 observations (places) used)
(weighting matrices define 720 places)
(you specified -force-)
_IDs in weighting matrix do not match _IDs in data

And Stata does not appear to have created the subsets for this calculation. I have tried running spmatrix clear before running the rest of the script (including -spmatrix fromdata-); same errors. It seems to me that the -force- option is not working the way it used to.
Another difference is that I'm now running it on Stata SE 15.0 my laptop, whereas I ran the larger data on a remote machine that runs Stata-MP. Could that be the reason?

[Michael Coppedge]

Update: I retried my do-file on Stata-MP on the remote machine, with the larger weighting matrix, and it worked. Output:
capture noisily {
. spregress DV X1 - Xn, gs2sls dvarlag(W1) errorlag(W1) force
(10860 observations)
(2487 observations excluded due to missing values)
(8373 observations (places) used)
(weighting matrix defines 10860 places)
(you specified -force-)
(weighting matrix matched 8373 places in data)
(weighting matrix W1_s001 created)

This replicates the successful runs I had in the past. So it seems that -force- works in Stata-mp but not Stata SE. Can anyone confirm that?

[Michael Coppedge]

Problem solved. -force- works; the problem was actually that when I made subsets of my analysis dataset and my weighting matrices, I needed to recreate the ID variable I was using to -spset- each file/matrix. So,
use DS, clear
spset, clear
drop id
generate id = _n /*I'm using custom weighting matrices*/
spset id

use W1.dta
spset, clear
drop id
generate id = _n
spset id

etc.

Now my output is (simplified):
(720 observations)
(24 observations excluded due to missing values)
(696 observations (places) used)
(weighting matrices define 720 places)
(you specified -force-)
(weighting matrices matched 696 places in data)
(weighting matrix W1_s001 created)
(weighting matrix W2_s001 created)

...
then the estimates from the model. Success!

[Maria Shabir]

After assigning the weights i am facing this issue.I have a panel of 30 economies with 780 observations "I am facing this issue""(780 observations (places) used)
(weighting matrix defines 30 places)
estimation sample defines places not in the weighting matrix
You must specify if or in to restrict the estimation sample to the places in the weighting matrix or use a different weighting
matrix."
How i can expand the Matix from 30 to 780.
I am using 30 countries. 780 is the total number of observations.

[Maria Shabir]

How I can resolve it? I tried my best but I just got an error. Firstly I downloaded the shapefile. Then I import it in Stata by using the spshape2dta"drive link". Later I merged this file with the panel data file by using the merger (m:m ) because 1:1 was not working and was showing this error"_IDs are not unique". After this, I used "keep if _merge ==3" to retain only matched records. When I assigned the weight matrix I faced this error again "_IDs are not unique".

[Andrew Musau]

You have panel data, so you need to use an -if- statement with spmatrix create. See an example here: https://www.statalist.org/forums/for...ate-contiguity