On Monday I put forward an argument that it will soon be time to update protocols and conduct new valuation exercises for older instruments like ICECAP-O (though I’d include the valuation exercise I was part of for ASCOT too in this recommendation, since it drew heavily on the ICECAP-O methods and the finding that the BWS tariff more-or-less matches the DCE one could conceal important differences our sample was not set up to detect). Yesterday I gave a purely personal view on the relative merits of ICECAP-O and ICECAP-A, arguing that continued use of a population average tariff might be an argument in favour of ICECAP-O, whilst more individual-level valuation might dictate whatever instrument is most appropriate for your age group.
Today’s blog entry will discuss a problem people may not be aware of, but which concerns the use of the original British English ICECAP-A in contexts where, in fact, it may give misleading results (though that remains to be checked, once it is translated from British English to other forms of English – bear with me!)
For instance, we know already that ICECAP-A – the instrument for use among adults of any age and which uses British English – should only be used with caution even in other predominantly English-speaking countries. Here’s why. After I was given the finalised version of ICECAP-A, my team in Sydney ran some piloting of the choice experiment (BWS). On at least one attribute the “third” (one level down from top) level capability score was actually estimated to be larger than the “fourth” (top) level score. Now, there are design reasons why this could have happened (which I won’t discuss here – anyone with sufficient knowledge of DCE design should be able to work out why this can happen). However, I was able to discount this as the main reason. It got me very worried. I asked around the office – most of my colleagues spoke American or Australian English. I was also able to ask a few NZ and Canadian English speakers.
I discovered that millennials up to my generation (gen X) in particular, in Australia, Canada and New Zealand, have largely imported US English definitions of the qualifier “quite”: they regard “quite a lot” of something to be of greater magnitude than “a lot” of something, unlike Brits who think the other way and which is an assumption in the wording of ICECAP-A (which used different types of qualifiers than ICECAP-O – see yesterday’s discussion). It turns out this is a well-known problem.
During final estimation I had to put in restrictions on the scoring in at least one attribute so the “top” level did not have a lower capability score than the “third” level in order for ICECAP-A to work: clearly even some Brits in the (UK) valuation exercise had abandoned traditional British English (watching US films and TV?), certainly enough to skew the scoring. So, sobering though it is, we also need to do some more work on ICECAP-A, in addition to ICECAP-O and ASCOT.
US/Canadian/NZ/Australian valuation exercises will have to “translate” the British English ICECAP-A version into their local English before valuation. I don’t think we need be defensive about this – the EuroQoL Group have changed their protocols/been open to more than one (the original, the Paris etc) over the years and are currently funding a lot of work to make a bigger leap forward. (Full disclosure: I am part of a group funded by them to investigate whether BWS can be used to produce an EQ-5D-5L tariff.) A health economist’s job is never done!