Skillings Mack test?

[Sneha Mantri]

Hello all,

I have a dataset of 25 participants who were sampled at 3 time-points (pre-intervention, post-intervention, 3-mo follow-up). I would like to see if there is a change in various prespecified outcome measures over that time. Because of the small sample size and some loss to follow-up, I chose the Skillings Mack test as a non-parametric version of repeated measures ANOVA. However some of the Stata output is inconsistent, so I wanted to know if I'm writing the code incorrectly.

When I run

skilmack updrs_2, id( record_id) repeated( visit)

I get this:

Weighted Sum of Centered Ranks

 visit |     N  WSumCRank        SE   WSum/SE  
-------+-------------------------------------
 1     |    17       9.00      4.12      2.18  
 2     |    17      -9.00      4.12     -2.18  
---------------------------------------------
 Total               0 

Note N= 8 not included as only had one observation

Skillings Mack    =   4.765
P-value (No ties) =   0.0290

  Ties exist. Above SEs and P-value approximate, if not too many ties;
  42 rows of [record_id, updrs_2_score]; 40 different combinations;  n(record_id) = 25

  Consider using the p-value below, (which is found from a simulated
        conditional null distribution of SM   - see options -
  simulating ...........)

Empirical P-value (Ties)    ~   0.0110

This suggests that there is a significant pre-post difference in the UPDRS_2 variable.


However when I run this:

by visit, sort : summarize updrs_2, detail

I get

-> visit = 1

                        updrs_2_score
-------------------------------------------------------------
      Percentiles      Smallest
 1%            2              2
 5%            3              3
10%            4              4       Obs                  25
25%            8              6       Sum of Wgt.          25

50%            8                      Mean               9.56
                        Largest       Std. Dev.      4.213866
75%           12             13
90%           15             15       Variance       17.75667
95%           17             17       Skewness       .6863663
99%           21             21       Kurtosis       3.820296

------------------------------------------------------------------------------------------------------
-> visit = 2

                        updrs_2_score
-------------------------------------------------------------
      Percentiles      Smallest
 1%            2              2
 5%            2              3
10%            3              4       Obs                  17
25%            6              5       Sum of Wgt.          17

50%            8                      Mean           8.176471
                        Largest       Std. Dev.      4.333522
75%            9             11
90%           13             13       Variance       18.77941
95%           20             13       Skewness       1.106565
99%           20             20       Kurtosis       4.390751

which shows that the median is the same (although the interquartile range is shifted down a bit).

I'm having trouble reconciling the SM p-value with the actual medians here. Note that this particular example has only two timepoints, although most of the other outcome variables do have a third. On the other outcome variables, the p-value only registers as significant when there is an obvious difference between medians (e.g. 31 to 24).

So my question is:
1. Is the Skillings-Mack an appropriate test to use in this situation?
1a. If so, do I need to adjust my command?
1b. If not, is there a different statistical test I should use?

Thank you very much!

[Dave Airey]

Do you want to look at the differences between time points within subjects? You might also look at the data with a spaghetti plot type graph.

[Sneha Mantri]

Do you want to look at the differences between time points within subjects? You might also look at the data with a spaghetti plot type graph.

Yes, I made a spaghetti plot, but it looks very messy with some people improving and others worsening. Perhaps that's why the medians are the same but SM says there's a difference?