data collected at the "wrong" time

[Rich Goldstein]

I am involved in a longitudinal study where each person is measured, say,
quarterly for 4 quarters and their measurement occasion is defined as, say,
90 days from their prior measurement occasion, plus or minus 10 days (call this their "window"). However, some of the subjects
missed their window but came in later for a measure and this was done (i.e.,,
their data was collected); some were very close (e.g., they missed the window by 1 day or so) and some are not
nearly so close (e.g., they missed the window by more than 30 days).

I have been unable to find any literature on how to deal with subjects who
miss their measurement window but still come in and have their data collected,
but I believe and hope there is some relevant literature.

So, I am looking for 2 things: (1) advice on how to deal with the measurement
occasions that are outside of the pre-specified window and (2) citations, if
any, that discuss this issue in any way.

I do note that I was not involved in setting up the size of the windows but I
was told that the window size was set so that the clinicians would be
"comfortable" that the value was appropriate for that time point - and that
is all I know about the issue (and I don't fully understand the answer!)

[Leonardo Guizzetti]

I don’t have references to offer at hand. This scenario is a fairly common problem seen in clinical trials, especially exacerbated during the pandemic restrictions. What I have mostly seen done, which uses all available data, is to define analysis windows for visits. Using your example, you might state that for visits inside the nominal visit window (e.g., the target visit date -/+ 10 days), those serve as analysis values as was planned. For visits outside the nominal window, the visit closest to the as yet unallocated visits becomes that analysis visit value. An alternative approach is to use all visits as they are but use the actual time instead of nominal time in your analysis. This could greatly increase the complexity of your model.

[Rich Goldstein]

Leonardo Guizzetti - thank you; I should have mentioned that I have thought of each of these and, it I go with the first, I will probably analyze more than once with different sets of ways of treating those outside the window (e.g., as you suggest, or treat as missing, or widen the window depending on what the clinical experts are "willing" to accept

[Clyde Schechter]

My own practice in this situation, which I encounter often, is similar to Leonardo Guizzetti 's, except that I typically define multiple analysis windows. The first window is the pre-specified window from the study protocol. The next window is wide enough to include those who were very close, and another window is wide enough to strain credulity about being timely (in terms of the time stability of what the measure is) but not wide enough to induce howls of laughter. If the distribution of the data permits, I also include one or more windows in between. I then repeat the analysis separately using each window to determine whether the findings are robust to the window choice. If the protocol's windows were well chosen I usually find that the first expanded window produces results similar to the protocol window, and the wider windows are increasingly divergent from that.

There is another approach that I occasionally use. Sometimes the measurements that were obtained within the protocol window were obtained at the study's own facilities but the "late" windows were done elsewhere, raising the specter of method variation. In this situation, I have treated the out-of-protocol-window measurements as missing values and then done multiple imputation, with the imputation model including the out-of-window measurement, and, sometimes, the lateness of the out-of-window measurement.

[Rich Goldstein]

Clyde Schechter - thanks for your thoughts; I will probably do multiple analyses as implied by your first paragraph and see this as a form of "sensitivity" analysis; but, assuming I can get the actual dates of measurement, I am leaning to using the actual time of the measures but, because N is relatively small (about 75), this may run into problems

again, if anyone has a cite, that would be greatly appreciated

[Rich Goldstein]

Just to complete things - I sent this question also to a couple of friends, one of whom suggested I post it to datamethods.org; which I did; my post and answers from Marc Schwartz and Frank Harrell (who runs this site I believe) can be found at https://discourse.datamethods.org/t/...window/6861/8; another friend agree with what Frank sent me privately which is basically the same as what he posted in the linked discussion

[Clyde Schechter]

That link doesn't work. Can you repost? I'd like to see what was said there.

[Leonardo Guizzetti]

That link doesn't work. Can you repost? I'd like to see what was said there.

There was a stray semicolon included in the link.

[Clyde Schechter]

Thank you very much!

[Rich Goldstein]

for those following this, yesterday and then this morning, Marc and Frank added new comments and, particularly valuable, a cite of a new paper that appears directly relevant

sorry for the typo in the link above; as Leonardo Guizzetti points out, just ignore the last character (a semi-colon) in the link

[Joseph Coveney]

I gather the protocol for your study doesn't define a so-called primary endpoint or the statistical methods to be used, and there are no formal data-collection forms (case report forms, case record forms), either paper or electronic.

The latter would be where the staff declare what the "measurement occasion" is, as well as the date of visit, which might lie outside the prescribed window for the visit. Or the visit might be declared "unscheduled" (often pursuant to an adverse event and, as Frank Harrell alludes, would be among those where "measurement times are informative").

In the clinical studies that I am familiar with (regulated), these—as well as a stand-alone statistical analysis plan—are present, and the procedure to follow for out-of-window visits (unless the endpoints are something-free survival) is essentially identical to what Marc Schwartz states in the first comment in the thread there, that is:

Violations of those protocol defined follow up windows, and missed visits entirely, would be formally tracked as protocol violations.

In most cases, your ITT analysis would include those patients using their data as if collected at the discrete time interval.

. . . The protocol and/or the SAP for the study would pre-define your analysis cohorts such as ITT, Per Protocol, Safety, As Treated, and so forth, and the inclusion/exclusion criteria for each.

[Rich Goldstein]

First,thank you for your comments - I did not join the project until after data collection had started and the previous statistician had left for another position

but, no, neither the protocol nor the SAP, nor anything else I have seen deals with this question

yes, an ITT would include all people but not necessarily all "visits" but I do want to include them

I note that this went through at least one IRB but it is not what would be called a "regulated" study as the treatment has to do with learning how to prepare food in "better" ways; there is no drug and no device involved

[Joseph Coveney]

Yes, sorry, I did actually figure that it was a project that you joined in-progress, and when I wrote "your study" it wasn't intended to imply anything other than it's one that you happen to be involved in.

Because the protocol and SAP are silent on the matter, it seems that it would give you the latitude to handle out-of-window intervals in a way that makes most sense from an interpretive standpoint and that facilitates analysis. If you choose to go with the as-declared route, I doubt that I can offer any pointers to literature to cite that you haven't already searched. I don't recall the as-declared handling of follow-up interval of scheduled visits ever being explicitly critically treated (that is, a pros-and-cons discussion of as-declared versus as-recorded) in any industry best-practices whitepaper or agency guidance document. With the manner in which a (non-survival) primary endpoint is typically defined and statistically analyzed in these studies, the matter is moot.

[Rich Goldstein]

thank you