Two sample Kolmogorov-Smirnov test results

[Mirko Hirschmann]

Hello everybody,

I would like to ask a question concerning the interpretation of a two sample Kolmogorov-Smirnov test result which we conducted as a formal goodness-of-fit test.
We conducted the test for the variables age and education which are ordinally scaled. First, is it the right test to assess formal goodness-of-fit?
If yes, how is it possible that we find in the first line (0) a difference of 0.0000 for both tests (see below)?

It would be great if you could help me. Thanks in advance!



ksmirnov Age, by (FinUn)

Two-sample Kolmogorov-Smirnov test for equality of distribution functions

Smaller group D P-value
-----------------------------------
0: 0.0000 1.000
1: -0.1030 0.064
Combined K-S: 0.1030 0.128

Note: Ties exist in combined dataset;
there are 4 unique values out of 656 observations.



. ksmirnov Education, by (FinUn)

Two-sample Kolmogorov-Smirnov test for equality of distribution functions

Smaller group D P-value
-----------------------------------
0: 0.0000 1.000
1: -0.0317 0.770
Combined K-S: 0.0317 1.000


Note: Ties exist in combined dataset;
there are 5 unique values out of 656 observations.

[Maarten Buis]

The Kolmogorov-Smirnov test tests the hypothesis that the distribution of age and education is the same in the two groups defined by the variable FinUn. I don't see how the equality of the distribution of age and education equates to a good fit of a model. Can you clarify, why you think that that is the case? Can you tell us what the model is? Can you tell us what FinUn is?

[Mirko Hirschmann]

Thanks for the quick answer, Maarten. The variable FinUn seperates our population in participants and non-participants. We wanted to show, that both groups equal each other which we already did by a z-test. However, we now want to further check the validity of the sample selection in the population by a formal goodness-of-fit test and thought the K-S test might be the right test.

[Maarten Buis]

Goodness of fit tests only exist relative to a model; you cannot look at how good something fits without first defining what it is that needs to fit. You did not specify the model, so the concept of a goodness of fit test just does not apply to your problem. I don't know if this is just some mistake in the use of terminology or if there is a larger underlying misunderstanding going on here.

My guess of your situation is that you have an experiment or a quasi experiment and you want to check whether the treated and control groups are comparable with respect to age and education. I assume that with a z-test you actually mean a t-test, i.e. you compared the means. The Kolmogorov-Smirnov test is not without merrit, but it is not without problems either. So if you are looking for additional evidence beyond the t-test, I would look at just graphs.