Tag Archives: individual models


eu_support_graph copy


Well, I’ve finally got round to programming a model that:

  • Asks you just five best-worst scaling questions – you choose your “most agreed with principle” and “least agreed with principle” – people take 2-3 mins to answer this tops.
  • Runs a best-worst scaling (BWS) exercise on just YOUR five answers.
  • Spits out three things:
    • A pie chart showing how likely each of the six main options (continued EU membership/Norway option/ Switzerland option/ Canadian option/ Turkish option/ World Trade Organisation option) would best satisfy YOUR principles
    • A pie chart showing the predicted chances of you personally supporting each of the five principles
    • A pie chart showing the predicted chances of you personally rejecting each of the five principles





Thus, the first chart tells you, based on which of these five principles we could “get” under each of the six models, what are the chances of getting “as much as we want” from each model of a new British-European relationship – the six models (one REMAIN, five BREXIT) .

This, like all CORRECT best-worst scaling, is an individual model, giving you PERSONALISED results, not “you averaged with others”.

We can, of course, average across people, slice and dice the results across sex/gender/political affiliation etc, to find out what model is most popular in certain groups. But the point is, my model doesn’t NEED to do that. All because just five BWS questions tell me everything I need to know about what you value.

Gold dust for all the campaigns – and the government, as it struggles to negotiate what type of new relationship would command majority support in the country.

I have deliberately answered the survey as a “hypothetical REMAINer” to show what they should have done – namely made the single European market something people understood and fought for, above other factors.

There are lots of scenarios – including what probably actually happened in that people were in reality “sure” they disliked free movement of people and/or EU budget contributions but unsure about their SEM/FTA/CU support – which lead to a BREXIT outcome as the most likely to achieve their preferences….your relative preferences for these determines which BREXIT model (hard/soft) is most likely to suit you.

Campaign managers/constituency parties/national party executives as well as Jo(e) Public would be very interested in this.


Does age bring wisdom?

Today’s blog entry will discuss a philosophical issue that has implications for choice of ICECAP instrument and these views are purely my own, not those of any other individual involved in development of any of the ICECAP instruments. The issue concerns the difference in attributes (dimensions/domains) between ICECAP-O (technically designed only for older people) and ICECAP-A (designed for adults of any age).

Now, in many ways the issue is moot: there is considerable overlap in the underlying conceptual attributes and three of the five are pretty much the same. Whilst published work showed ICECAP-O working well amongst older people in a large survey conducted in Bristol, UK, unpublished follow-on work showed it working equally well among the younger adults. Furthermore a national online valuation exercise plus survey conducted in Australia (published in the BWS book) showed it working well there among adults of all ages too.

However, the other two attributes (doing things that make you feel valued and concerns about the future in ICECAP-O vs achievement & progress and feeling settled & secure in ICECAP-A) are arguably different. It might reflect the priorities of different generations: younger generations may feel a great need to achieve and progress – the idea of “moving forward” may be driving this (particularly since we have many more people working in that group). ICECAP-O on the other hand stresses the act of doing things that make you feel valued in life, which (to me) does not necessarily imply “moving forward” (though my personal career changes may have coloured my views!).

Likewise in ICECAP-A feeling settled and secure may reflect current younger generations’ feelings of instability in a world of zero-hour contracts etc. ICECAP-O asks instead about “concerns about the future”. Whilst it might be seen merely as the ICECAP-A question “flipped”, it is phrased with respect to the amount of concern overall, unlike ICECAP-A which is phrased with respect to how many areas of life – there’s a subtle difference there. To illustrate, I will simply pose a question. If you otherwise have a very good quality of life, can you still have a lot of concern about the future? I’d argue yes. Now let’s think about ICECAP-A. If you otherwise have a very good quality of life, can you feel settled and secure in only a few areas of life? Playing devil’s advocate, it could be argued that “this respondent has already said they’re doing well on the other four attributes of quality of life – they have a lot of capability to achieve the levels they want – so how can they feel unsettled in key attributes too?”

Ultimately these are empirical issues, requiring researchers to look at correlation matrices of actual answers. In the Australian survey 5002 people were randomised to either ICECAP-O or ICECAP-A. The Spearman rank correlation coefficients of the respondents’ five tickbox answers were uniformly higher for any given pair of attributes in ICECAP-A compared to their ICECAP-O equivalent. However, a big caveat here is that the ICECAP-O arm was a properly done valuation exercise in which quota sampling – on the basis of own ICECAP-O tickbox answers – was done. There were no previous ICECAP-A data on which to choose quotas. Thus this is not a like-for-like comparison and ICECAP-O therefore had an artificial advantage. Using the Bristol adult ICECAP-O data (to correct, somewhat, for this) caused four of the ten pairwise correlations to be smaller for ICECAP-A but two of these were for attributes common to both instruments. Comparisons among groups from the same country and using the same sampling is therefore required before firm conclusions can be made.

Finally, it is worth considering the philosophy here and I’ll raise a final point. OK it seems that adding younger adults to the valuation sample has changed at least one and arguably two attributes. It raises the normative question of whether we should use these attributes in valuing their quality of life when they haven’t, by definition, lived a long life: perhaps age brings wisdom and it is the older people who “know what’s best for you”. Most people experience regret at some point and our “values” (defined both conceptually – the attributes themselves – and numerically – the tariff) can change with experience.

Of course using a single ICECAP instrument – ICECAP-O if one were persuaded of the above philosophical argument – would make things nicer and easier when it comes to “a single common denominator for valuation” but if, like me, you are keen for greater investigation of (and possibly use of) individual valuation, could we justify using ICECAP-O scoring for a 30 year old which may downweight “doing things that make you feel valued” because that person actually is more interested in achieving things and progressing (forward?) in life?  On the other hand, knowing what are the key conceptual attributes of ICECAP-A, maybe stressing, in the intro for ICECAP-O,  that “doing things that make you feel valued” can easily encompass “achieving and progressing in life” is a practical solution?

Another empirical issue!

And so we come full circle to whether practical solutions, or stricter ones fitting some theory, are the way forward. As usual in health economics, normative issues galore.

states worse than dead

No, this isn’t another moan by yours truly about how the valuation people deal (in)correctly with states worse than the death on in health economics valuation exercises (phew).

This tweet interested me.  There are all sorts of things you could do with a discrete choice experiment (DCE) to measure the trade-offs such patients make. When at UTS, we did a DCE that did two things, one novel and one not so novel. The first was an attitudinal one that found there are three segments among Australian retired people (our sample was around 1100 total) when you got them to tell you what statements about life they related to most and least – Best-Worst Scaling. We did something never done before – feed back to them their own results after that survey that they could print off, bring to their doctor to discuss, use as the starting point for and end-of-life care plan etc: results of this form a chapter in the book referenced. Of course the doctors at the sharp end in ICUs had warned us that thanks to TV programmes the general public has much higher expectations about the success/acceptablility of these dramatic interventions than is true in practice, but you could do the same survey with patients. In fact the bare bones of the survey are still live at the link and you can see how you compare with older Aussies.

The second DCE was (by DCE standards) very very simple, but was done to get a handle on the trade-offs people woul make regarding the kinds of interventions in the survey in this Twitter post and unfortunately won’t give you personalised results.

These types of DCEs should become routine. They can be done on touchscreen tablet PCs etc when the patient is waiting to see the doctor, they can give personalised results – not aggregated ones like in the bad old days. People like them, and like to know how they compare with others – the older generation love those surveys comparing them to others just as much as the younger “Facebook generations”. C’mon people, this survey is great and very very informative but we can move forward even further and do it today.

euroqol group funding

Sander Arons, Karin Oudshoorn and I have a EuroQoL funded project: how a *correctly* designed Case 2 best-worst scaling (BWS) study + a DCE can give us a tariff for EQ-5D-5L.

The study will be interesting as we work from first principles: what is the tariff for an INDIVIDUAL person. Then do all the things I’ve been harking on about for 5+ years to get a proper population tariff.

I don’t claim this will turn the world upside down. But I do think it will give some food for thought for whether the Secretariat wants to consider DCEs for the 5L tariff. I fully recognise that the EuroQoL group has all sorts of constraints to work under (linking with the -3L version, consistency of methods etc) which I totally understand. I’m pragmatic these days. But I think, if our testing results hold true in the main study, that we will raise some eyebrows in the results and give the EuroQoL Group the chance to get back in front in terms of methodology.

Interesting times ahead!