Can I combine results from different modes of administration?

In some cases, data may be collected using different approaches within the same study. Three situations are not uncommon:

data may be collected using a combination of self-complete versions and patient interview versions (e.g. in a study with trauma patients in which some patients can physically complete the questionnaire, but others cannot – though they are capable of responding in an interview);data may be collected in some cases using a paper version and in other cases using a digital version of the questionnaire, within the same study;data is collected from a combination of proxy-reporters and patients (e.g. in a study with stroke patients, where some patients can respond for themselves and others cannot respond at all, on either the self-complete or the interviewer version).

It is likely to be of interest to aggregate data collected in the same study using different modes of administration and/or from different types of respondent (patient versus proxy). However, this is currently probably only appropriate, at least without further in-study analysis of response patterns, in the case of b), where evidence suggests substantial levels of equivalence in response between electronic and paper versions of PRO instruments (see Muehlhausen et al, 2015). In the case of a) and c), there is insufficient evidence to show that EQ-5D responses collected using self-complete versus interview modes of administration or from proxies versus patients can be aggregated. One study with EQ-5D found that face-to-face and interview administration provided similar results (Chatterji et al, 2017), however, little work has been done in this area and studies with other PRO instruments in the literature have shown that results can differ between the two modes of administration.

Regarding the equivalence of proxy and patient responses, research to date suggests that responses should probably not be directly aggregated, at least without further within-study analysis of responses patterns. Hounsome et al (2011) found that proxy versions had good reliability and validity for use in dementia patients but highlighted that it was important to select appropriate proxies and that different proxies (e.g., family carers, institutional carers, and health-care professionals) provide different ratings for patients’ health. They also noted that there was a lack of association between patient and proxy ratings. In a review of the use of EQ-5D in stroke patients, Oczkowski et al (2010) found that most studies reported that proxy respondents overestimated impairments compared with patient self-reports. Gabbe et al (2012) found that, in trauma patients 12 months after injury, agreement between patient and proxy respondents was substantial for the mobility and self-care items and moderate for the remaining 3 dimensions.Based on these considerations, in the case of a) and c), we recommend that when different modes of administration are used, a variable indicating mode of administration should be recorded in the data set. Data should always be analysed separately before aggregating, and researchers should report consent rates, data completeness and other potential sources of bias for the different modes of administration.

Muehlhausen W et al. Equivalence of electronic and paper administration of patient-reported outcome measures: a systematic review and meta-analysis of studies conducted between 2007 and 2013. Health Qual Life Outcomes 2015;13:167.