Why does a logical expression like x > 0 return 1 when x is missing?

[Nick Baradar]

Hello all,

I ran into an unexpected behavior when generating a binary indicator from a numeric variable that sometimes contains missing values.

When I run:

generate y = x > 0

I notice that if x is missing (.), then y becomes 1.
I had assumed that missing would propagate to missing or evaluate to false.


So my question is why Stata evaluates the expression . > 0 as true (i.e., 1)?

Sincerely,
Nick

[Felix Bittmann]

You may want to read

help missing

To quote:

Numeric missing values are represented by large positive values.

And a very large number is larger than 0, so the expression evaluates to 1.

[Nick Baradar]

Thank you, Felix! But what's the logic behind that decision?

Thank you,
Nick

[Leonardo Guizzetti]

There’s no deeper logic really, just a design decision. Stata treats the system missing (.) and extended missing values (.a through .z) as very large numbers for the purposes of logical comparisons (help missing values). Other software (like SAS) uses the opposite ranking, where missing values are the smallest numbers for such comparisons. Yet other languages (like R) have a 3-value logic system where missing values are separate from numeric literals.

[Leonardo Guizzetti]

Let me also add for you that the idiomatic way to exclude missing values from your variable definition is

generate y = x > 0 if !missing(x)   // note most experienced Stata programmers will use mi() instead of missing() since as it is less typing

[Nick Cox]

Here is another way to spell out that missings should map to missings.

generate wanted = cond(missing(x), ., x > 0)

which spells out the conditional

if missing(x) return .
else return x > 0

which in turn can be expanded mentally

if missing(x) return .
else if x > 0 return 1
else return 0

which in turn can be expressed syntactically

generate wanted = cond(missing(x), ., cond(x > 0, 1, 0))

The balance between being concise and being clear is largely personal. I am aware of the syntax mi(x) but usually spell out missing(x). Other way round I usually write gen not generate.

The main criterion is what will be clear to any future reader of your code, where often the most likely future reader is yourself 1 day, 1 week, 1 month, 1 year, 1 decade down the line.

Backing up: the rules attractive to any particular users on handlkng missing values are the rules that do what you they want with missing values, except that any particular user probably wants some combination of ignoring missing values some of the time, not ignoring them some of the time, treating them as if zero some of the time, and treating them as a special for missing some of the time.

Stata's design was influenced by a simple question: what happens to (numeric) missing values when you sort on a variable containing them? As Leonardo Guizzetti pointed out, SAS jumped one way on this, and Stata jumped the other way.

Naturally Nick Baradar is not the first Stata user to be puzzled by this, perhaps being bitten too (after 5+ years of using Stata?).

There have been many previous discussions, here on Statalist, and more durably, such as

SJ-19-1 dm0099 . . . . . . How best to generate indicator or dummy variables
. . . . . . . . . . . . . . . . . . . . N. J. Cox and C. B. Schechter
Q1/19 SJ 19(1):246--259 (no commands)
discusses how to best generate indicator or dummy variables

SJ-10-2 dm0049 . . . . . . . . . . . . . Stata tip 86: The missing() function
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B. Rising
Q2/10 SJ 10(2):303--304 (no commands)
tip on using the missing() function to deal with missing
values

SJ-10-1 dm0047 . . . . . . . . . . . . . . . . Stata tip 84: Summing missing
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . N. J. Cox
Q1/10 SJ 10(1):157--159 (no commands)
tip concerning treatment of missing data when summing

FAQ . . . . . . . . . . . . . . . . Logical expressions and missing values
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . W. Gould
2/03 Why is x > 1000 true when x contains missing value?
http://www.stata.com/support/faqs/data-management/
logical-expressions-and-missing-values/


-- where the last iterm may well be all that you need to read (at least at first).