Real change - minimum detectable change? Test-retest

[Natalia Gonzalez]

Good evening,

I was wondering if somebody could help me with this burning question.
In my work, we are using a device for measuring changes over time (tracking changes), we are currently studying the reliability of this test.

In addition to this, I was hoping to calculate/show what a real change in this test is. I mean, when a patient gets a follow-up test, I would like to calculate how much the results have to change in order for it to be a real change instead of random variability.
I have been calculating the ICC, Bland-Altman plots...(For reproducibility purposes) but besides that, I was hoping to calculate this other parameter.
I have read that standard error of measurement (SEM) followed by smallest real different (SRD) make sense in this case but I am not sure how to obtain them and how to interpret. Some psychometric tests have used this but there have been difference confidence interval levels (90%, 95%)...
I was wondering if somebody could shed some light regarding this question.

I really appreciate it. Thank you so much!

Best regards,
Natalia

[Natalia Gonzalez]

I have been doing further research on SEM and SRD/MDC and I think that for my test it is not a good option as the standard deviation is not fixed (it changes depending on each set of values) as well as the reliability (ICC for example, ICC tend to be higher in size is bigger than if smaller...)
Another option that I think it could work for calculating/estimating the change is by focusing on the upper and lower limits of agreement.
The whole interval wideness, we could theorize that a change bigger than the interval wideness could be a real change? Worse case scenario, test1 is -lower limit of agreement and test2 is +upperlimit of an agreement, that difference could be due to chance, but if we see a difference (from baseline) bigger than the confidence interval, then we would be confident that it is a real change.

Does it make sense?

Thank you so much!

[Phil Bromiley]

You didn't get a quick answer. You'll increase your chances of a useful answer by following the FAQ on asking questions - provide Stata code in code delimiters, readable Stata output, and sample data using dataex. Also, it is best not to include files or pictures.

The methods you're talking about seem to reflect a specific disciplinary area. If you try to program something and need help, this listserve is very helpful. We are much less able to help you with things that relate to a specific disciplinary database. Sometimes you interest one of the few on the list with experience in that area but that is somewhat random.