EDIT 27 July 2016: Just to clarify, this post is not due to any current issues/proposals/projects. It is a piece I have had in mind for a long while now but couldn’t realistically write whilst I was still under funding/contractual obligations. Recent encounters with other funders have (so far, fingers crossed, touch wood) been more positive!
Funding by UK health and EU funding councils has, in my experience, been less than satisfactory. Cuts that seem totally arbitrary have been made. In fact I cannot recall a single discrete choice experiment (BWS) study since the original ICECAP-O study that has been fully funded.
Universities I worked at pretty much absorbed the costs. Now, this was partly because all the valuation exercises I have been involved in have raised new issues. But more recent examples have been particularly annoying for my collaborators – they have ended up putting in a lot of extra time. It made me feel guilty, thinking “perhaps I should have costed more time”. But the truth is, whatever I asked for in the past, I got cut BIGTIME.
This comes back to issues that I have had close to my heart for a while now, the issues of (1) what exactly “discrete choice modelling” (DCM) is as a discipline and (2) where it sits within health services research (at least for its application in health). I personally consider choice modelling to be an entire discipline and when in health to be a subdiscpline of HSR akin to “qualitative research” or “biostatistics”. Now, funded trials would not dream of having a jobbing RA to do a fully-fledged qualitative project within a programme grant, nor have an RA do all the biostatistics. So why does DCM get treated as the poor relation? There should ALWAYS be an expert (with 15+ years of experience) in charge of the DCM, with a junior to learn. Because discrete choice experiments (DCEs) are NOT just a branch of health economics or biostatistics. Choice modelling is a discipline in its own right, with a totally different set of statistical skills, economic assumptions and especially psychological knowledge required to do it successfully. It most definitely IS NOT just another preference elicitation method.
Qualitative research quite rightly fought hard to gain acceptance as an equal partner in HSR studies. DCM should too. It’s simply not good enough to have juniors doing DCEs in major clinical trials. Again, would you get an RA without specific training to do all the biostats? No? Then why is it considered acceptable in DCM?
Those of us who spent over 15 years learning our trade find it, frankly, a little insulting, that someone from an MSc or PhD should be considered able to run a DCE in a trial. DCE analysis is not purely a science. It’s very difficult to teach. It’s part art. Do it for 10+ years – ACROSS MULTIPLE DISCIPLINES – then you are just about able to do it properly.
May we have some acknowledgement that what we do is important please?
This isn’t just for the benefit of us “insiders”: the industry loses since the standards of reporting and conduct have not, in my opinion, improved much at all in health. Sooner or later (and sooner if one study I have in mind in a major journal is compared with real preference data) the results of a poor DCE will be comprehensively discredited. Then we all lose. Actually I don’t. Because I do studies predominantly for the private sector who are very sensitive to incorrect results. They commission me because I get them good results. But it’d be a shame if DCM is lost to the public sector. All because nobody wanted to pay for senior people to do things correctly.
I’m told I can be too negative. Fair enough. Yes – standards are a LOT better than 10 years ago. But please remember, whilst the health field has moved on, those fields way ahead of you (marketing/environmental econ/transport econ etc) have also moved on. A lot. You’ve caught up a little. But not by enough. Experience has become doubly important – reading the literature won’t tell you how to do a DCE perfectly. Because of that dirty little secret…..there are an infinite number of solutions to a DCE. If you’ve not analysed 20+ DCEs are you really confident you know what solution to quote to policymakers? Particularly in health where the “tricks” available to solve the problem in other disciplines are not available? I’ll end with a quote:
Harry Callahan: “Uh uh. I know what you’re thinking. “Did he fire six shots or only five?” Well to tell you the truth in all this excitement I kinda lost track myself. But being this is a .44 Magnum, the most powerful handgun in the world and would blow your head clean off, you’ve gotta ask yourself one question: “Do I feel lucky?” Well, do ya, punk?
You are playing with a loaded gun if you don’t know enough about what solutions make sense and what will blow your reputation up. Do you feel lucky?