Abstract

Background: The application of conjoint analysis (including discretechoice experiments and other multiattribute stated-preference methods) in health has increased rapidly over the past decade. A wider acceptance of these methods is limited by an absence of consensusbased methodological standards. Objective: The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Good Research Practices for Conjoint Analysis Task Force was established to identify good research practices for conjoint-analysis applications in health. Methods: The task force met regularly to identify the important steps in a conjoint analysis, to discuss good research practices for conjoint analysis, and to develop and refine the key criteria for identifying good research practices. ISPOR members contributed to this process through an extensive consultation process. A final consensus meeting was held to revise the article using these comments, and those of a number of international reviewers. Results: Task force findings are presented as a 10-item checklist covering: 1) research question; 2) attributes and levels; 3) construction of tasks; 4) experimental design; 5) preference elicitation; 6) instrument design; 7) datacollection plan; 8) statistical analyses; 9) results and conclusions; and 10) study presentation. A primary question relating to each of the 10 items is posed, and three sub-questions examine finer issues within items. Conclusions: Although the checklist should not be interpreted as endorsing any specific methodological approach to conjoint analysis, it can facilitate future training activities and discussions of good research practices for the application of conjoint-analysis methods in health care studies. Keywords: conjoint analysis, discrete-choice experimental, economic evaluation, good research practice. Copyright © 2011, International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Published by Elsevier Inc. Background to the task force report The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Preference-based Methods Special Interest Group's Conjoint Analysis Working Group developed a proposal for a task force on conjoint-analysis good research practices. With the increase in use of conjoint analysis, a structure to guide the development, analysis, and publication of conjoint analyses in health care studies would be useful for researchers, reviewers, and students. The task force proposal was submitted to the ISPOR Health Science Policy Council (HSPC) in November 2008. The HSPC recommended the proposal to the ISPOR Board of Directors where it was subsequently approved in January 2009. The ISPOR Conjoint Analysis Good Research Practices Task Force met regularly via teleconference and in person at ISPOR meetings to identify the important steps in a conjoint analysis, to develop and refine the key criteria for good research practices, write the outline, and draft the subsequent report. ISPOR members and invited international experts contributed to the consensus development of the task force report via comments made during a Forum presentation at the 2009 ISPOR 14th Annual International Meeting (Orlando, FL, USA), through the comments received from the draft report's circulation to the Conjoint Analysis Reviewer Group and an international group of reviewers selected by the task force chair. The task force met in person in September 2009 to discuss and come to consensus on the more controversial issues that arose. The draft report was revised as appropriate to address comments from these review opportunities. The final step in the consensus process was circulation of the report to the ISPOR membership in September 2010 with an invitation to review and comment. A total of 42 reviewers submitted written or verbal comments from all occasions to review the draft report. Introduction Understanding how patients and other stakeholders value various aspects of an intervention in health care is vital to both the design and evaluation of programs. Incorporating these values in decision making may ultimately result in clinical, licensing, reimbursement, and policy decisions that better reflect the preferences of stakeholders, especially patients. Aligning health care policy with patient preferences could improve the effectiveness of health care interventions by improving adoption of, satisfaction with, and adherence to clinical treatments or public health programs [1][2] Economists differentiate between two approaches to the measurement of preferences: revealed and stated In health, the term "preferences" includes methods beyond the stated and revealed preference paradigms. For example, methods such as the time-trade-off or standard gamble, which are used to calculate quality-adjusted life years (QALYs), are referred to as preference-based. Such methods are based on cardinal utility and are beyond the scope of a scientific report on stated preferences. Stated-preference studies are preferable to QALY or attitudinalbased valuation methods because they are grounded in consumer theory and the psychology of choice. Stated-preference methods fall into two broad categories: • Methods using ranking, rating, or choice designs (either individually or in combination) to quantify preferences for various attributes of an intervention (often referred to as conjoint analysis, discrete-choice experiments, or stated-choice methods), or • Methods using direct elicitation of monetary values of an intervention (including contingent valuation or willingness-to-pay and willingness-to-accept methods) A simple distinction between these two categories is that the latter aims to estimate demand for a single product, whereas the former aims to explore trade-offs between a product's attributes and its effect on choice. In practice, the distinctions between the two categories have blurred, with researchers estimating demand using multiple-question and discrete-choice formats, and researchers using preference estimates to calculate willingness-to-pay for attributes. This scientific report focuses on the first of these approaches. Following standard convention in health care, we refer to them as conjoint analysis. However, we acknowledge that many others would prefer the term "discrete-choice experiment" over "conjoint analysis." This said, most of the material in this report applies equally to discrete-choice experiments and other types of conjoint analysis. Conjoint analysis in health care studies There has been a rapid increase in the application of conjoint analysis in health care studies [6 -8]. Conjoint analysis is a decomposition method, in that the implicit values for an attribute of an intervention are derived from some overall score for a profile consisting (conjointly) of two or more attributes [9 -13]. Conjoint-analysis methods are particularly useful for quantifying preferences for nonmarket goods and services or where market choices are severely constrained by regulatory and institutional factors, such as in health care The task force thus endeavored to provide broad guidance on good research practices by suggesting a structure to guide the development, analysis, and publication of conjoint analyses in health care studies, without necessarily endorsing any one approach. For its report, the task force also decided to use the checklist format to guide research How the checklist can be used The checklist should be used to understand the steps involved in producing good conjoint-analysis research in health care. The final format of the checklist follows the format establish by Drummond and colleagues Description of the checklist The findings of the task force are presented as a 10-item checklist and summarized in Implicit in the structure of the checklist is that some tasks should be considered jointly or collectively. These joint tasks 404 V A L U E I N H E A L T H 1 4 ( 2 0 1 1 ) 4 0 3 -4 1 3 are arranged horizontally in In the remaining sections of this report, we describe issues to be considered in evaluating each of these 10 items and elaborate on additional points in each section of the checklist. These items are summarized in Research question Following generally accepted research practices in health care, a conjoint-analysis study must clearly state a well-defined research question that delineates what the study will attempt to measure [5]. For example, a conjoint analysis might be undertaken to quantify patients' relative preferences for cost, risk of complications, and health care service location for a given medical intervention. Specifying a testable hypothesis, defining a study perspective, and providing a rationale for the study are important good research practices for applications of conjoint analysis in health care. Testable hypothesis In addition to defining the research question, researchers should state any hypotheses to be tested in the study or acknowledge that the study is exploratory and/or descriptive. A testable hypothesis may be implicit in the research question itself. For example, if the research question is to determine whether changes in surgical wait time influence patient treatment choice, the testable null hypothesis is that the parameter estimate for the wait-time attribute is not statistically significantly different from zero. In other words, the hypothesis test is designed to infer whether a change in the level of the attribute (e.g., a change in surgical wait time from 1 to 2 months) is statistically significant. If the null hypothesis is rejected for a given attribute, then the parameter estimate on that attribute is statistically significant, indicating that it has played a role in the patients' responses. Study perspective Researchers should define the study perspective, including any relevant decision-making or policy context. The research question "What are patients willing to pay for treatment to reduce the rate of relapse in multiple sclerosis?" includes both the items to be measured-the trade-off between cost and reduction in relapse rate-and the perspective and decision context of the analysis, i.e., the patient's perspective in making treatment decisions. Here researchers may want to provide even more specifics in defining the study perspective by focusing on a particular type of patient or a particular timing or environment. Although it is good research practice to offer the most accurate study perspective possible, the more specific the perspective, the more difficult it may be to find respondents. Rational for using conjoint analysis A conjoint-analysis study should explain why conjoint methods are appropriate to answer the research question. Conjoint analysis is well suited to evaluate decision makers' willingness to trade off attributes of multi-attribute services or products. The multiple sclerosis research question posed in the previous paragraph involves explicit trade-offs between measurable attributes, so it can be answered using conjoint analysis. The research question also could be addressed using alternative methods such as contingent valuation. Researchers should identify not only whether conjoint analysis can be used to answer the research question but also why conjoint analysis is preferable to alternative methods. Attributes and levels The objective of conjoint analysis is to elicit preferences or values over the range of attributes and levels that define profiles in the conjoint-analysis tasks. Although all attributes that potentially characterize the alternatives should be considered, some may be excluded to ensure that the profiles are plausible to subjects. For the chosen attributes, the attribute levels should encompass the range that may be salient to subjects, even if those levels are hypothetical or not feasible given current technology. Again, the choice of attribute levels may need to be restricted. Authors should explain both inclusions and omissions of attributes and levels. Good research practices should include attribute identification, attribute selection, and level selection. Attribute identification 405 V A L U E I N H E A L T H 1 4 ( 2 0 1 1 ) 4 0 3 -4 1 3 attributes (and even possible attribute levels) that characterize the profiles to be evaluated. Attribute selection The subset of all possible attributes that should be included in the conjoint-analysis tasks can be determined on the basis of three criteria: relevance to the research question, relevance to the decision context, and whether attributes are related to one another. Attributes central to the research question or to the decision context must either be included or held constant across all profiles. It is important to control for any potential attributes that are omitted from the conjoint-analysis tasks but that correlate with attributes that are included in these tasks. In the United States health care market, insurance coverage and out-of-pocket medical expenses for procedures are routine for many patients. Cost may be perceived as correlated with improvements in medical outcomes or with access to advanced interventions. If cost is not included, it should be controlled for by informing subjects that it is constant across profiles. Discussion with experts and further pilot testing with subjects can be used to narrow the list of attributes. If the number of possible attributes exceeds what one may find possible to pilot in a conjoint analysis, it may prove beneficial to use other types of rating and/or ranking exercises (often referred to as compositional approaches) to assess the importance of attributes and to facilitate the construction of the final list of attributes to be included. Level selection Once the attributes have been decided upon, researchers must identify the levels that will be included in the profiles in the conjoint-analysis tasks. Levels can be categorical (e.g., a public or private hospital), continuous (a copayment of $10, $20, or $30), or a probability (a chance of rehospitalization of 2%, 5%, or 10%). Although an emerging literature discusses the subjective recoding of the levels Researchers also are cautioned against choosing too many attribute levels. Although some attributes may require more or fewer levels (especially those that are categorical), it is good research practice to limit levels to three or four per attribute. Finally, researchers should avoid the use of extreme values that may cause a grounding effect. Unless it is required for the research question, researchers 406 V A L U E I N H E A L T H 1 4 ( 2 0 1 1 ) 4 0 3 -4 1 3 need not span the full breadth of possible levels. For example, if one were constructing an attribute to define the distance to the nearest service in a national study, it might be plausible to have very small and very large distances, but these could be considered outliers. Instead one might form levels across the interquartile range or at plus and minus one standard deviation from the mean. Whatever the logic used to determine the levels of an attribute, researchers should make their decision making transparent, and assumptions need to be tested during the pilot testing. Construction of tasks Conjoint-analysis tasks are the mechanism by which possible profiles are presented to respondents for the purpose of preference elicitation. Conjoint-analysis tasks can be assembled in a number of ways by varying the numbers of attributes, profiles (options or choices), and other alternatives. Thus, researchers should consider the use of full or partial profiles, an assessment of the appropriate number of profiles per task, and the inclusion of opt-out or status-quo options. Full or partial profiles Within the tasks that respondents will evaluate, profiles (alternatives or choices) can be presented with all the attributes that are being considered in the study (a full profile) or with only a subset of the attributes (a partial profile). Although it is generally considered good practice in health care research to work with full profiles, researchers should determine, through qualitative research or pilot testing, whether subjects can reasonably evaluate the full profiles. If researchers believe that the complexity of the conjoint-analysis task will encourage respondents to develop or employ simplifying heuristics, such as focusing on only a few attributes while ignoring others, then partial profiles may be preferred. If the use of partial profiles is undesirable, then tasks can show full profiles, but researchers should constrain some attribute levels to be the same (i.e., overlap) between the profiles