Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Contradictions Found in Assertion

    Hi Statalist,

    I hope you are all well during this time.

    I am in the process of cleaning up a dataset to get it ready to merge to the master dataset. Essentially, what I have to do is that I have to fill in country names and country numerical codes that are missing for the years 2000-2018 (including for the years 2000 and 2018, if applicable). Basically, they are missing somewhat at random. In the original dataset, the country names were only available if there was a case associated for the country and the year. I just have to fill in the missing gaps before I merge. I have used the following codes that have worked flawlessly until now:

    Code:
    gen long obs_no = _n
    gen block = ceil(_n/19)
    by block (obs_no), sort: assert year==1999+_n
    by block (Country), sort: assert Country==Country[_N] | missing(Country)
    by block (Country): replace Country=Country[_N]
    sort obs_no
    and
    Code:
    gen long obs_new = _n
    gen block1 = ceil(_n/19)
    by block1 (obs_new), sort: assert year == 1999 + _n
    by block1 (COW), sort: assert COW == COW[_N] | missing(COW)
    by block1 (COW): replace COW = COW[_N]
    sort obs_new
    (The reference point for the codes above can be found here: https://www.statalist.org/forums/for...within-a-range)

    Now, I am receiving the following error messages:
    Code:
     gen long obs_new = _n
    
    . gen block1 = ceil(_n/19)
    
    . by block1 (obs_new), sort: assert year == 1999 + _n
    
    . by block1 (COW), sort: assert COW == COW[_N] | missing(COW)
    7 contradictions in 14,022 observations
    assertion is false
    r(9);
    
    end of do-file
    
    r(9);
    and
    Code:
    gen long obs_no = _n
    
    . gen block = ceil(_n/19)
    
    . by block (obs_no), sort: assert year==1999+_n
    
    . by block (Country), sort: assert Country==Country[_N] | missing(Country)
    6 contradictions in 14,022 observations
    assertion is false
    r(9);
    
    end of do-file
    
    r(9);
    I went through the dataset manually last night to see if I can find some errors. From eyesight, it looks okay. (But obviously, it is not). I tried to go through the assert help file, but I still cannot find out what is causing these contradictions. I also tried looking online for help, and found
    Code:
    do check, nostop, run check
    This is a small window of the data, but these are the variables that are important for the research.
    Code:
    * Example generated by -dataex-. To install: ssc install dataex
    clear
    input int(year conflict_id) str10 start_date byte ep_end str10 ep_end_date str24 location int gwno_location byte armed_conflict_sending_nonstate
    2000 4564 ""           . ""           ""           . 0
    2001 4564 ""           . ""           ""           . 0
    2002 4564 ""           . ""           ""           . 0
    2003 4564 ""           . ""           ""           . 0
    2004 4564 ""           . ""           ""           . 0
    2005 4564 ""           . ""           ""           . 0
    2006 4564 ""           . ""           ""           . 0
    2007 4564 "2007-02-09" 1 "2007-12-07" "Iraq"     645 1
    2008 4564 ""           . ""           ""           . 0
    2009 4564 ""           . ""           ""           . 0
    2010 4564 ""           . ""           ""           . 0
    2011 4564 ""           . ""           ""           . 0
    2012 4564 ""           . ""           ""           . 0
    2013 4564 ""           . ""           ""           . 0
    2014 4564 ""           . ""           ""           . 0
    2015 4564 ""           . ""           ""           . 0
    2016 4564 ""           . ""           ""           . 0
    2017 4564 ""           . ""           ""           . 0
    2018 4564 ""           . ""           ""           . 0
    2000 4565 ""           . ""           ""           . 0
    2001 4565 ""           . ""           ""           . 0
    2002 4565 ""           . ""           ""           . 0
    2003 4565 ""           . ""           ""           . 0
    2004 4565 ""           . ""           ""           . 0
    2005 4565 ""           . ""           ""           . 0
    2006 4565 ""           . ""           ""           . 0
    2007 4565 ""           . ""           ""           . 0
    2008 4565 "2008-01-09" 1 "2008-12-26" "Somalia"  520 1
    2009 4565 ""           . ""           ""           . 0
    2010 4565 ""           . ""           ""           . 0
    2011 4565 ""           . ""           ""           . 0
    2012 4565 ""           . ""           ""           . 0
    2013 4565 ""           . ""           ""           . 0
    2014 4565 ""           . ""           ""           . 0
    2015 4565 ""           . ""           ""           . 0
    2016 4565 ""           . ""           ""           . 0
    2017 4565 ""           . ""           ""           . 0
    2018 4565 ""           . ""           ""           . 0
    2000 4566 ""           . ""           ""           . 0
    2001 4566 ""           . ""           ""           . 0
    2002 4566 "2002-03-27" 1 "2002-03-27" "Ghana"    452 1
    2003 4566 ""           . ""           ""           . 0
    2004 4566 ""           . ""           ""           . 0
    2005 4566 ""           . ""           ""           . 0
    2006 4566 ""           . ""           ""           . 0
    2007 4566 ""           . ""           ""           . 0
    2008 4566 ""           . ""           ""           . 0
    2009 4566 ""           . ""           ""           . 0
    2010 4566 ""           . ""           ""           . 0
    2011 4566 ""           . ""           ""           . 0
    2012 4566 ""           . ""           ""           . 0
    2013 4566 ""           . ""           ""           . 0
    2014 4566 ""           . ""           ""           . 0
    2015 4566 ""           . ""           ""           . 0
    2016 4566 ""           . ""           ""           . 0
    2017 4566 ""           . ""           ""           . 0
    2018 4566 ""           . ""           ""           . 0
    2000 4567 ""           . ""           ""           . 0
    2001 4567 ""           . ""           ""           . 0
    2002 4567 ""           . ""           ""           . 0
    2003 4567 ""           . ""           ""           . 0
    2004 4567 ""           . ""           ""           . 0
    2005 4567 ""           . ""           ""           . 0
    2006 4567 ""           . ""           ""           . 0
    2007 4567 ""           . ""           ""           . 0
    2008 4567 "2008-01-14" 1 "2008-04-22" "Nigeria"  475 1
    2009 4567 ""           . ""           ""           . 0
    2010 4567 ""           . ""           ""           . 0
    2011 4567 "2008-01-14" 1 "2011-10-17" "Nigeria"  475 1
    2012 4567 ""           . ""           ""           . 0
    2013 4567 ""           . ""           ""           . 0
    2014 4567 ""           . ""           ""           . 0
    2015 4567 ""           . ""           ""           . 0
    2016 4567 ""           . ""           ""           . 0
    2017 4567 ""           . ""           ""           . 0
    2018 4567 ""           . ""           ""           . 0
    2000 4568 ""           . ""           ""           . 0
    2001 4568 ""           . ""           ""           . 0
    2002 4568 ""           . ""           ""           . 0
    2003 4568 "2003-03-12" 1 "2003-07-11" "Nigeria"  475 1
    2004 4568 ""           . ""           ""           . 0
    2005 4568 ""           . ""           ""           . 0
    2006 4568 ""           . ""           ""           . 0
    2007 4568 ""           . ""           ""           . 0
    2008 4568 ""           . ""           ""           . 0
    2009 4568 ""           . ""           ""           . 0
    2010 4568 ""           . ""           ""           . 0
    2011 4568 ""           . ""           ""           . 0
    2012 4568 ""           . ""           ""           . 0
    2013 4568 ""           . ""           ""           . 0
    2014 4568 ""           . ""           ""           . 0
    2015 4568 ""           . ""           ""           . 0
    2016 4568 ""           . ""           ""           . 0
    2017 4568 ""           . ""           ""           . 0
    2018 4568 ""           . ""           ""           . 0
    2000 4569 ""           . ""           ""           . 0
    2001 4569 ""           . ""           ""           . 0
    2002 4569 "2002-06-30" 0 ""           "Ethiopia" 530 1
    2003 4569 "2002-06-30" 1 "2003-01-31" "Ethiopia" 530 1
    2004 4569 ""           . ""           ""           . 0
    end
    ------------------ copy up to and including the previous line ------------------

    Listed 100 out of 14022 observations
    Use the count() option to list more


    Please note that gwno_location is renamed to COW and gwno_location is named to Country.

    Thank you in advanced for any help.
    MW

  • #2
    Essentially, what I have to do is that I have to fill in country names and country numerical codes that are missing for the years 2000-2018 (including for the years 2000 and 2018, if applicable).
    Can't you use fillin for this?
    ---------------------------------
    Maarten L. Buis
    University of Konstanz
    Department of history and sociology
    box 40
    78457 Konstanz
    Germany
    http://www.maartenbuis.nl
    ---------------------------------

    Comment


    • #3
      Originally posted by Maarten Buis View Post

      Can't you use fillin for this?
      If I execute
      Code:
      fillin location gwno_location
      it does not seem to work, and I get the data below:

      Code:
      * Example generated by -dataex-. To install: ssc install dataex
      clear
      input int(year conflict_id) str10 start_date byte ep_end str10 ep_end_date str24 location str3 gwno_location byte armed_conflict_sending_nonstate
      2000 1 "1997-05-12" 0 ""           "Ecuador"     "130" 1
      2001 1 "1997-05-12" 0 ""           "Ecuador"     "130" 1
      2002 1 "1997-05-12" 0 ""           "Ecuador"     "130" 1
      2003 1 "1997-05-12" 0 ""           "Ecuador"     "130" 1
      2004 1 "1997-05-12" 0 ""           "Ecuador"     "130" 1
      2005 1 "1997-05-12" 1 "2005-10-28" "Ecuador"     "130" 1
      2006 1 ""           . ""           ""            ""    0
      2007 1 ""           . ""           ""            ""    0
      2008 1 ""           . ""           ""            ""    0
      2009 1 ""           . ""           ""            ""    0
      2010 1 ""           . ""           ""            ""    0
      2011 1 ""           . ""           ""            ""    0
      2012 1 ""           . ""           ""            ""    0
      2013 1 ""           . ""           ""            ""    0
      2014 1 ""           . ""           ""            ""    0
      2015 1 ""           . ""           ""            ""    0
      2016 1 ""           . ""           ""            ""    0
      2017 1 ""           . ""           ""            ""    0
      2018 1 ""           . ""           ""            ""    0
      2000 2 ""           . ""           ""            ""    0
      2001 2 ""           . ""           ""            ""    0
      2002 2 ""           . ""           ""            ""    0
      2003 2 ""           . ""           ""            ""    0
      2004 2 ""           . ""           ""            ""    0
      2005 2 ""           . ""           ""            ""    0
      2006 2 "2006-05-31" 0 ""           ""            ""    0
      2007 2 "2006-05-31" 1 "2007-08-15" ""            ""    0
      2008 2 ""           . ""           ""            ""    0
      2009 2 "2006-05-31" 1 "2009-08-02" ""            ""    0
      2010 2 ""           . ""           ""            ""    0
      2011 2 "2006-05-31" 0 ""           "South Sudan" "626" 1
      2012 2 "2006-05-31" 0 ""           "South Sudan" "626" 1
      2013 2 "2006-05-31" 1 "2013-08-08" "South Sudan" "626" 1
      2014 2 ""           . ""           ""            ""    0
      2015 2 ""           . ""           ""            ""    0
      2016 2 "2006-05-31" 0 ""           "South Sudan" "626" 1
      2017 2 "2006-05-31" 1 "2017-04-10" "South Sudan" "626" 1
      2018 2 ""           . ""           ""            ""    0
      2000 3 ""           . ""           ""            ""    0
      2001 3 ""           . ""           ""            ""    0
      2002 3 ""           . ""           ""            ""    0
      2003 3 ""           . ""           ""            ""    0
      2004 3 ""           . ""           ""            ""    0
      2005 3 ""           . ""           ""            ""    0
      2006 3 "2006-05-31" 0 ""           ""            ""    1
      2007 3 "2006-05-31" 1 "2007-08-15" ""            ""    1
      2008 3 ""           . ""           ""            ""    0
      2009 3 "2006-05-31" 1 "2009-08-02" ""            ""    1
      2010 3 ""           . ""           ""            ""    0
      2011 3 "2006-05-31" 0 ""           "Ethiopia"    "530" 1
      2012 3 "2006-05-31" 0 ""           "Ethiopia"    "530" 1
      2013 3 "2006-05-31" 1 "2013-08-08" "Ethiopia"    "530" 1
      2014 3 ""           . ""           "Ethiopia"    "530" 1
      2015 3 ""           . ""           "Ethiopia"    "530" 1
      2016 3 "2006-05-31" 0 ""           "Ethiopia"    "530" 1
      2017 3 "2006-05-31" 1 "2017-04-10" "Ethiopia"    "530" 1
      2018 3 ""           . ""           "Ethiopia"    "530" 1
      2000 4 "1991-12-31" 1 "2000-02-23" ""            ""    0
      2001 4 ""           . ""           ""            ""    0
      2002 4 ""           . ""           ""            ""    0
      2003 4 ""           . ""           ""            ""    0
      2004 4 ""           . ""           ""            ""    0
      2005 4 "1991-12-31" 1 "2005-11-30" "Ethiopia"    "530" 1
      2006 4 ""           . ""           ""            ""    0
      2007 4 ""           . ""           ""            ""    0
      2008 4 ""           . ""           ""            ""    0
      2009 4 "1991-12-31" 1 "2009-11-30" "Ethiopia"    "530" 1
      2010 4 ""           . ""           ""            ""    0
      2011 4 "1991-12-31" 1 "2011-08-07" ""            ""    0
      2012 4 ""           . ""           ""            ""    0
      2013 4 ""           . ""           ""            ""    0
      2014 4 ""           . ""           ""            ""    0
      2015 4 ""           . ""           ""            ""    0
      2016 4 ""           . ""           ""            ""    0
      2017 4 ""           . ""           ""            ""    0
      2018 4 ""           . ""           ""            ""    0
      2000 5 "1991-07-07" 1 "2000-07-16" ""            ""    0
      2001 5 ""           . ""           ""            ""    0
      2002 5 ""           . ""           ""            ""    0
      2003 5 "1991-07-07" 1 "2003-12-26" ""            ""    0
      2004 5 ""           . ""           ""            ""    0
      2005 5 "1991-07-07" 1 "2005-06-30" ""            ""    0
      2006 5 ""           . ""           ""            ""    0
      2007 5 ""           . ""           ""            ""    0
      2008 5 ""           . ""           ""            ""    0
      2009 5 ""           . ""           ""            ""    0
      2010 5 ""           . ""           ""            ""    0
      2011 5 ""           . ""           ""            ""    0
      2012 5 ""           . ""           ""            ""    0
      2013 5 ""           . ""           ""            ""    0
      2014 5 ""           . ""           ""            ""    0
      2015 5 ""           . ""           ""            ""    0
      2016 5 ""           . ""           ""            ""    0
      2017 5 "1991-07-07" 0 ""           "Djibouti"    "522" 1
      2018 5 "1991-07-07" 0 ""           "Djibouti"    "522" 1
      2000 6 ""           . ""           ""            ""    0
      2001 6 ""           . ""           ""            ""    0
      2002 6 ""           . ""           ""            ""    0
      2003 6 ""           . ""           ""            ""    0
      2004 6 "2004-01-25" 0 ""           "Guatemala"   "90"  1
      end

      Comment


      • #4
        Code:
        fillin year location
        seemed to retrieve similar results to the previous post.

        Comment


        • #5
          Code:
          by block (Country), sort: gen OK = Country==Country[_N] | missing(Country)
          list if !OK
          would save on the manual scrutiny, which failed. Note that multiple spaces would not qualify as missing strings, nor would uchar(160) which looks like a space, but isn't.

          Code:
          . di "|" uchar(160) "|"
          | |
          Code:
          replace Country = trim(Country)
          removes leading and trailing spaces, important if some substantive missings are really one or more spaces, or if some country entries contain leading or trailing spaces.

          Last edited by Nick Cox; 27 Mar 2020, 02:52.

          Comment


          • #6
            Originally posted by Nick Cox View Post
            Code:
            by block (Country), sort: gen OK = Country==Country[_N] | missing(Country)
            list if !OK
            would save on the manual scrutiny, which failed. Note that multiple spaces would not qualify as missing strings, nor would uchar(160) which looks like a space, but isn't.

            Code:
            . di "|" uchar(160) "|"
            | |
            Code:
            replace Country = trim(Country)
            removes leading and trailing spaces, important if some substantive missings are really one or more spaces, or if some country entries contain leading or trailing spaces.
            Nick, thank you. This is extremely helpful, and saved a lot of time. This is exactly what I was looking for. It allowed me to see where the errors where.

            Comment


            • #7
              Good, but what were the errors?

              Comment


              • #8
                Originally posted by Nick Cox View Post
                Good, but what were the errors?
                There were a few cases where there were more than one three-digit country code (gwno_location) and country (location) listed in the row.

                Comment

                Working...
                X