Tag Archives: eq-5d-5l

spring cleaning

New year = digital spring cleaning time! Ugh. No matter how future-proof you aim to be with how you structure files, aim to work seamlessly across PCs etc it never takes long for reality to change and you realise you need to do the rigmarole again.

When admin is done I’m back to the project looking at comparing Case 2 BWS estimates with DCE ones. I shall look with “fresh eyes” since I haven’t worked on it since before xmas. (Plus we need to get this rounded off so we can submit and get paid, hehe.)

Then it’s the (long-delayed) big marketing push for TF Choices LTD. I’ve had a good number of proposals and funded projects come my way so far but can’t rest on my laurels…time to make sure a load of marketers and others know what I can do for them, in addition to the academic community I was part of!

I can’t think of anything methodological I want to shout about today (phew, they think)….I’ll continue to post anything big or of key relevance but as there are only so many hours in the day and company stuff must come front and centre in 2017 it’s likely that my comments and posts will be related to things I’m doing at the time (like Case 2 vs DCEs) rather than detailed posts triggered by twitter or citation alerts I get.

 

efficient design problems redux

Two papers in the past week have given me need to remind people that efficient designs in DCEs may not be the bee’s knees. I posted a while back when my paper showing this was accepted and gave more detail, which ultimately became part of my SSRN paper, after publication.

I’ll quote from my latter post:

Street and Burgess had begun to provide CenSoC with designs whose efficiency was 100% (or close to it), rather than 30-70%. We loved them and used them practically all the time. In parallel with this, John Rose at the Institute of Transport and Logistics Studies at Sydney University had begun utilising highly efficient designs – though of a different sort. However, what efficient designs have in common – and really what contributes heavily to their efficiency – is a lack of level overlap. This means that if the respondent is presented with two pairs of options, each with five attributes, few, and in many cases none, of those attributes will have the same level in both options. Thus, the respondent has to keep in mind the differences in ALL FIVE ATTRIBUTES at once when making a choice. Now, this might be cognitively difficult. Indeed John Rose, to his immense credit, made abundantly clear in the early years in a published paper that his designs, although STATISTICALLY EFFICIENT, might not be “COGNITIVELY EFFICIENT”, in that people might find them difficult (pushing up their error variance) or, even worse, use a simplifying heuristic (such as “choose the cheapest option”) in order to get through the DCE. (Shame on us CenSoCers for not reading that paper more closely.) Clearly in the latter case you are getting biased estimates – not only are your parameter estimates biased (in an unknown direction) but the functional form of the utility function for such respondents is wrong. Now John merely hypothesised this problem – he had no empirical data to test his hypothesis, and recommended that people go collect data. For many years they didn’t.

My study was the first within-subject study to be published (though I know of at least one other within-subject study that was doing the conference rounds at about the same time and may well have been published since). It certainly has influenced my thinking and the paper in the current AHE Blog has found – although I believe using betwen-subject study only – that yes, efficient designs for “complete EQ-5D-5L described lives” seemed to cause problematic beta estimates. They advocate two-step designs – something a group I’m working with are already doing….hopefully we will have some interesting stuff to present next year at a EuroQoL Group meeting.

I shall simply end with a warning I put in the SSRN paper concerning efficient designs (which certainly have their place, don’t get me wrong, but you can’t use them unthinkingly):

Of course if it turns out that greater precision has been gained at the expense of bias, then efficient designs replace what is merely an annoyance with a crippling flaw.

 

 

protocol updates for all instruments?

On Monday I put forward an argument that it will soon be time to update protocols and conduct new valuation exercises for older instruments like ICECAP-O (though I’d include the valuation exercise I was part of for ASCOT too in this recommendation, since it drew heavily on the ICECAP-O methods and the finding that the BWS tariff more-or-less matches the DCE one could conceal important differences our sample was not set up to detect). Yesterday I gave a purely personal view on the relative merits of ICECAP-O and ICECAP-A, arguing that continued use of a population average tariff might be an argument in favour of ICECAP-O, whilst more individual-level valuation might dictate whatever instrument is most appropriate for your age group.

Today’s blog entry will discuss a problem people may not be aware of, but which concerns the use of the original British English ICECAP-A in contexts where, in fact, it may give misleading results (though that remains to be checked, once it is translated from British English to other forms of English – bear with me!)

For instance, we know already that ICECAP-A – the instrument for use among adults of any age and which uses British English – should only be used with caution even in other predominantly English-speaking countries. Here’s why. After I was given the finalised version of ICECAP-A, my team in Sydney ran some piloting of the choice experiment (BWS). On at least one attribute the “third” (one level down from top) level capability score was actually estimated to be larger than the “fourth” (top) level score. Now, there are design reasons why this could have happened (which I won’t discuss here – anyone with sufficient knowledge of DCE design should be able to work out why this can happen). However, I was able to discount this as the main reason. It got me very worried. I asked around the office – most of my colleagues spoke American or Australian English. I was also able to ask a few NZ and Canadian English speakers.

I discovered that millennials up to my generation (gen X) in particular, in Australia, Canada and New Zealand, have largely imported US English definitions of the qualifier “quite”: they regard “quite a lot” of something to be of greater magnitude than “a lot” of something, unlike Brits who think the other way and which is an assumption in the wording of ICECAP-A (which used different types of qualifiers than ICECAP-O – see yesterday’s discussion). It turns out this is a well-known problem.

During final estimation I had to put in restrictions on the scoring in at least one attribute so the “top” level did not have a lower capability score than the “third” level in order for ICECAP-A to work: clearly even some Brits in the (UK) valuation exercise had abandoned traditional British English (watching US films and TV?), certainly enough to skew the scoring. So, sobering though it is, we also need to do some more work on ICECAP-A, in addition to ICECAP-O and ASCOT.

US/Canadian/NZ/Australian valuation exercises will have to “translate” the British English ICECAP-A version into their local English before valuation. I don’t think we need be defensive about this – the EuroQoL Group have changed their protocols/been open to more than one (the original, the Paris etc) over the years and are currently funding a lot of work to make a bigger leap forward. (Full disclosure:  I am part of a group funded by them to investigate whether BWS can be used to produce an EQ-5D-5L tariff.) A health economist’s job is never done!