The primary output of a decision making process is a decision and a key outcome measure is therefore decision quality. However, being a formative construct, ‘decision quality’ is both preference- and context-sensitive and legitimate alternative measures accordingly exist. A decision maker wishing to measure decision quality in the evaluation of a decision or decision making process needs to be aware of the attributes of the measures on offer. This paper establishes some of the key conceptual differences by examining two measures: Decision Quality Instruments and MyDecisionQuality. Four of their main conceptual differences relate to: the timing of the measurement (at the point of decision or at follow-up when the ‘downstream’ outcome is known); (whether or not an objective assessment of the information state of the individual is included (as opposed to self-reported state); whether the instrument itself is preference-sensitive; and whether the measure is to be used in the context of individualised clinical practice at the point of care or only in research to produce group level feedback. Establishing agreed measures of decision quality is necessary and useful, so long as it is accepted that it is a preference- and context-sensitive construct, in the way that is widely acknowledged in relation to, for example, Health-Related Quality of Life, with its many measures.