Tag Archives: qol

NIHR suggestion

NIHR in the UK are encouraging people to make suggestions for research.

So I did.

I would like studies to start collecting attitudinal data alongside the usual clinical and quality of life data. I found disturbing evidence in my Bristol data that the effect of negative perceptions of social empowerment and poor health exceeds the sum of their parts (there is a stonking interaction in a negative direction on quality of life). If this is better understood it would help in targeting the right treatment to the right person, who is at risk of catastrphic decline (a phenomenon a clinician told me about that is seen among some patients after total joint replacement), and in improving end-of-life care.

We shall see if it leads to anything.

lovely wishes

Sent “official” email to collaborators a couple of days ago with my new job details and was touched at so many lovely comments and wishes for the future.

I’ve got to get on to practicalities now – I already had to buy a new suitcase in Singapore on my journey back here to transport back the heavy boots and big coats I bought there…I’ve worn neither on a regular basis since I left the UK 5 years ago. I’m also liaising with various groups of collaborators regarding funding applications. I’m particularly interested in getting European initiatives going – though these are NOT limited just to Europe-based researchers, so don’t want to exclude existing collaborators based elsewhere. The Horizon 2020 initiative has been mentioned to me. So any existing collaborators or ones who would like to participate, please get in contact. I’m interested in getting going projects in any of:
(1) Risky decision-making
(2) End-of-life decision-making
(3) Ethical issues in medicine, particularly cross-country comparisons
(4) Quality of life, particularly anything using the Capabilities Approach of Amartya Sen
(5) Response time models
(6) Agent based models and systems dynamics with DCEs embedded.

So get your thinking caps on, folks!

Happiness isn’t quality of life if you’re old

The subject of happiness, particularly among older people, has come up (again) in the media. I reckon they trot out the latest survey results whenever there’s a slow news day. I think it’s no coincidence the newest stories have appeared in the slow month of August.

Anyway I shall keep this short as I’ll rant otherwise. Once again, neither happiness nor life satisfaction is the same as quality of life and we can argue til the cows come home as to which of the three (if any) is truly well-being.

First of all, if I can find the time to write up a follow-up to the paper I published on the mid 2000s survey of Bristolians I will show this:

Five year age bands showing mean levels (after rescaling) of self-rated happiness versus scored quality of life in Bristol

Five year age bands showing mean levels (after rescaling) of self-rated happiness versus scored quality of life in Bristol

The two track reasonably closely until retirement age. Then whilst happiness continues to rise, quality of life certainly does not. The wealth of other evidence on health, money, friends, etc from the survey suggests our QoL, the ICECAP-O instrument, is the better measure of overall well-being.

We are not the only ones to find this. A large US study pretty much concluded they didn’t know WTF older people were doing when they answered life satisfaction/happiness questions but they sure don’t answer them the same way that younger adults do. Older people use a different part of the numerical scale (typically a higher portion, all other things being equal). That’s rating scale bias and there is a huge and growing literature on it.

Stop asking these dumb questions. There are good alternatives.



Citation of Flynn Articles

As mentioned already, some of my papers are incorrectly cited, so here is a (handy?) list of the principal discrete choice related ones and the reasons for citing each 🙂

Flynn principal papers with primary relevance to choice models and reasons for citing

Coast J, Flynn TN, Sutton E, Al-Janabi H, Vosper J, Lavender S, Louviere JJ, Peters TJ. Investigating Choice Experiments for Preferences of Older People (ICEPOP): evaluative spaces in health economics. Journal of Health Services Research and Policy 2008;13(suppl 3):31-37

  • Paper which introduces the wider context of the BWS work (quality of life and the ICEPOP programme)


Flynn TN, Louviere JJ, Peters TJ, Coast J. Best-Worst Scaling: What it can do for health care research and how to do it. Journal of Health Economics 2007;26(1):171-89.

  • Original Case 2 (Profile case) user guide
  • Uses old term – attribute case – since dropped
  • But NOT the first case 2 published study in health – see Szeinbach et al 1999: Using conjoint analysis to evaluate health state preferences
  • Contrasts maxdiff and marginal models (though unfortunately not called these in the paper)


Lancsar E, Louviere JJ, Flynn TN. Several methods to estimate relative attribute impact in stated preference experiments. Social Science and Medicine, 2007;64:1738-1753.

  • First paper to discuss attribute IMPACT versus IMPORTANCE and how to get the former from BWS (amongst other techniques applied to DCE data)


Flynn TN, Louviere JJ, Peters TJ, Coast J. Estimating preferences for a dermatology consultation using Best-Worst Scaling: Comparison of various methods of analysis. BMC Medical Research Methodology 2008; 8:76.

  • Corrected the coding for the marginal sequential model from JHE article in order that model summary statistics be correct
  • Showed no difference in results from different models
  • Illustration of heterogeneity using effects coding and thereby showing how splitting out attribute impact out can be useful


Marley AAJ, Flynn TN, Louviere JJ. Probabilistic Models of Set-Dependent and Attribute-Level Best-Worst Choice. Journal of Mathematical Psychology 2008; 52:281-296.

  • Paper with mathematical proof of Case 2 estimator properties
  • The proof that you can’t, in fact, get attribute importance from BWS (contrary to McIntosh & Louviere original claim) though you do go a long way toward it, getting attribute IMPACT, (discussed in more detail in JOCM 2013)
  • Review of the literature on discrete choice tasks showing that despite 40 years of research estimation of attribute IMPORTANCE (in a discrete choice tasks framework) remains elusive
  • Presented hypothetical example of how manipulating context across two DCEs would finally allow estimation of attribute importance – this principle is used by Flynn et al (JOCM 2013)


Flynn TN, Marley AAJ, Louviere JJ, Peters TJ, Coast J. Rescaling quality of life tariffs from discrete choice experiments for use as QALYs: a cautionary tale. Population Health Metrics 2008; 6:6

  • Key paper showing why putting the death state into any RUT-based model is wrong.
  • Also first paper to discuss why the choice of health states used in any RUT-based model must take into account how scale might vary as a result – e.g. two states far apart on the latent health scale will have a large variance scale factor, much larger than two states close together on the latent health scale


Louviere JJ, Flynn TN. Using Best-Worst Scaling Choice Experiments To Measure Public Perceptions and Preferences for Healthcare Reform in Australia. The Patient: Patient-Centered Outcomes Research 2010;3(4):275-283

  • First Case 1 paper in health


Flynn TN, Peters TJ, Coast J. Quantifying response shift or adaptation effects in quality of life by synthesising best-worst scaling and discrete choice data. Journal of Choice Modelling 2013;6:34-43

  • Hopefully seminal paper as the first empirical application of the proposed model put forward by Marley et al (2008) and discussed by Flynn (2010) in which context is varied in order to get attribute importance.
  • This is used in an attempt to quantify response shift (adaptation) in a quality of life context.


Flynn TN, Louviere JJ, Peters TJ, Coast J. Using discrete choice experiments to investigate heterogeneity in preferences for quality of life. Variance scale heterogeneity matters. Social Science and Medicine 2010; 70:1957-1965

  • First paper in health to demonstrate the scale adjusted latent class (SALC) model to attempt to properly adjust for scale (the G-MNL being the other model)
  • Hypothesises how individual level valuations of health/QoL states is now possible


Louviere JJ, Flynn TN, Carson R. Discrete choice experiments are not conjoint analysis. The Journal of Choice Modelling 2010;3(3):57-72

  • Paper discussed theoretical underpinnings of DCEs
  • Why these are not part of the conjoint measurement paradigm in academia,
  • Why we need to be aware that we are concentrating on a tiny part of the whole decision making process that humans use


Flynn TN. Valuing citizen and patient preferences in health: recent developments in three types of best-worst scaling. Expert Review of Pharmacoeconomics & Outcomes Research 2010; 10(3):259-267.

  • First paper to set out the definitive naming of all three types (“Cases”) of BWS
  • The object case – Case 1, where the choice options are non-attribute based simple options,
  • The profile case – Case 2, where the choice options are attribute levels within a profile-based framework
  • The multi-profile case – Case 3, where the choice options are multi-attribute profiles
  • These names were chosen by the inventor of BWS (Jordan Louviere) and agreed by the three authors writing the definitive textbook (CUP 2014 forthcoming).
  • First paper in health to use the best-minus-worst scores (the “scores”), from Marley & Louviere (2005) in summarising outcomes


Flynn TN. Using conjoint analysis and choice experiments to estimate quality adjusted life year values: issues to consider. Pharmacoeconomics 2010;28(9):711-722

  • Paper notes likely problems getting interactions from Case 2
  • First paper to propose various options in valuing health states in discrete choice framework, ranging from using TTO to rescale estimates from a DCE without a duration attribute (cheap but not ideal) to including duration as an attribute (expensive, but theoretically more appealing)
  • First paper to discuss how Case 2 might change the context of the problem under investigation from a traditional DCE – and how the researcher should consider this when tempted to make inferences about the validity of one or both types of task (within- versus between-profile tasks)
  • Further discussion of how variance scale is important to consider in discrete choice based valuation studies in health
  • Further discussion of why including the “dead state” is wrong
  • Warns health economists about the possibility that respondents might interact with the design (“demand artefacts”) and the need for common designs in order that there is no confounding of design with utilities


Hawkins GE, Marley AAJ, Heathcote A, Flynn TN, Louviere JJ, Brown SD. Integrating cognitive process and descriptive models of attitudes and preferences. Cognitive Science (in press, accepted 23 May 2013

  • Collection of response times alongside DCE data
  • How, for the first time, inferences from a stated preference RUT-based model have been proved from a proper process model (the LBA).
  • Uses BWS as a vehicle for collecting data and providing additional proof of process model


Louviere JJ, Lings I, Islam T, Gudergan S, Flynn TN. An Introduction to the Application of (case 1) Best-Worst Scaling in Marketing Research. International Journal of Research in Marketing 2013;30(3):292-303.

  • First user guide paper for Case 1 but in marketing.
  • Many of the techniques already in use in early health papers including a recent paper in SSM by Swiss researchers.


Potoglou D, Burge P, Flynn TN, Netten A, Malley J, Forder J, Wall B, Brazier J. Best-worst Scaling vs. Discrete Choice Experiments: An Empirical Comparison using Social Care Data. Social Science and Medicine 2011;72(10):1717-1727

  • First comparison of DCE and Case 2 data, though not in a fully scale-adjusted model.


Coast J, Al-Janabi H, Sutton E, Horrocks S, Vosper J, Swancutt D, Flynn TN. Using qualitative methods for attribute development for discrete choice experiments: issues and recommendations. Health Economics 2012;21(6):730-741

  • Paper dealing with issues raised by Louviere, Hensher & Swait (2001 book) in terms of providing guidance on development of attributes
  • Compares various qualitative methods in the light of experiences from the ICEPOP programme

ICECAP-A UK scoring now available

For anyone wanting to use the ICECAP-A Capability instrument to measure and value well-being in the UK, we have the early view paper in Health Economics with scoring!


Terry N. Flynn, Elisabeth Huynh, Tim J. Peters, Hareth Al-Janabi, Sam Clemens, Alison Moody, Joanna Coast,

It is the first to produce both the population tariff and proper adjustment for scale (variance) heterogeneity in the same paper (rather than doing the latter as a secondary analysis). It also shows that Brits are not all alike, there are different “types” who value different attributes of life.

referenced in a blog

I was googling myself – as you do, and in this case to check that the youtube video with the two-naked-guys-plus-dildo thumbnail had been taken down/edited…it hadn’t – and found that my public lecture was blogged about back in 2010!

Thanks Lyrian! Hopefully I’ll get enough data from the recent survey to update the results.

PS neither of the guys in the video thumbnail is me and the video has nothing to do with me….honest! :-p

Your quality of life

Would you like to know you own personal quality of life score using the ICECAP-O instrument?

Would you like to know how you compare with other people your age? Gender? With others in your State or Territory (Australia)?

Try this totally anonymous survey – it’s in beta mode at the moment as we’re building up the numbers of responses to get better comparisons. However, it shows what we can do with ICECAP-O.