Detecting careless responses in STATA

[Tania Clarke]

Hello,


I wonder if anyone could share any tips or tricks for detecting careless responses (unengaged response patterns) in the context of self-report questionnaire. I'm looking for observations with response patterns that are even-odd or repeated (e.g. 1 1 1 1, or 12121). Any papers or STATA articles on this would be much appreciated!


So far, I've used the basic code below for inspecting and screening myself, but I know there will be more sophisticated code out there. Can anyone help me? In particular I'm interested in learning whether there is STATA code for the even-odd consistency index and the psychometric antonyms index.

--------
egen patternwbs = concat(wbs1 wbs2 wbs3 wbs4...*)
bigtab patternwbs uniqueID
------

Where 'patternwbs' is a new variable created displaying the patterns of responses on all the 'wellbeing at school' items (there are 21 in total), 'uniqueID' is to help me identify the problematic observation

Thanks so much in advance for any steers!

Tania

[Nick Cox]

bigtab is community-contributed and from SSC, as you are asked to explain (https://www.statalist.org/forums/help#stata). While visiting that, do swing by https://www.statalist.org/forums/help#spelling

bigtab has some helpful handles but they don't seem to include something that can be very helpful: sorting by frequency, such as is allowed by tabulate -- and -- if that command is awkward for some reason -- by groups from the Stata Journal,

Whatever command you use, I am not sure that it is at all easy to distinguish genuine and careless patterns. As someone who fills in (or out) questionnaires sometimes I am a little disconcerted by the idea that a constant response is careless, as it can match what I want to say, especially if I am minded to record what is overall a strongly negative or positive view. In student evaluations it can be a candid response expressing an overall view of some teaching. ("Quality of feedback" is particularly likely to be interpreted as "Did you like the grade you got?") Similarly employees are often asked to reply to a series of nuanced questions which are difficult to differentiate or to reply to given ignorance of whatever management or human relations jargon lies behind them.

Even-odd alternations I think you'd need to look for by first mapping to even and odd through say mod(answer, 2);

It is obvious, but perhaps worth underlining, that antipathy or indifference to questionnaires is most often expressed by ignoring them altogether.

[Tania Clarke]

Thanks for your reply Nick, much appreciated. And my apologies for not being more attentive to the statalist conventions and correct spelling of 'Stata'. I almost new this as a wrote it, as I often look at the alternate spelling by others with a raised eyebrow :')

Thanks for drawing my attention to number of important points to consider when it comes to different kinds of careless responses. In my case, I am looking at self-report measures from children that capture their mental wellbeing in school.

Certainly, it is possible that a constant response is genuine, but the tool I'm using by design includes negatively and positively worded items, such that it wouldn't necessarily make sense to answer 'Strongly agree' to every item (e.g. I feel happy; I feel miserable). This does make it somewhat possible to detect cases where a constant answer pattern to all items could be a sign of disengagement as opposed to genuine response.

You are right to suggest that simply missing questions would also suggest disengagement, and indeed there are cases like that, but when the respondents are children I think there might be other ways they express antipathy.

I'll definitely check out the mod command - thanks!

In any case, as with all 'outlier' exploration I will be consulting lots of different statistical evidence as removing observations is something I want to avoid at all costs.

Kind regards,
Tania