Tag Archives: rating scales

happiness redux

There is a piece on happiness at NakedCapitalism.com up today. It is a guest post from VoxEU and unfortunately, though trying to make valid points, falls into the usual holes: the key one is that the data all appear to be Likert-based self-reported happiness scales, which in two major countries (at the very least) have been shown to be deeply misleading (US and Australia). In short, even within these two countries, there are cohort and/or longitudinal effects: the number you state your happiness/life satisfaction to be is heavily dependent upon age (particularly if you are older), independent of (after adjusting for) a huge number of other factors (health, wealth, social empowerment, independence, etc). Moreover this is not “just” the infamous “mid-life dip”: the differences between such measures, and the more comprehensive well-being/quality-of-life ones, are particularly stark in extreme old age and have big implications for retirement age, what resources are needed by the very old etc.

To make comparisons across countries with different cultural backgrounds seems even more hazardous – Likert scales generally were pretty much discredited on such grounds by 2001:

Baumgartner H, Steenkamp J-BEM. Response styles in marketing research: a cross-national investigation. Journal of Marketing Research. 2001;38(2):143-56.

Steenkamp J-BEM, Baumgartner H. Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research. 1998;25(1):78-90.

Five year age bands showing mean levels (after rescaling) of self-rated happiness versus scored quality of life in Bristol

Five year age bands showing mean levels (after rescaling) of self-rated happiness versus scored quality of life in Bristol, UK

 

 

 

 

 

 

 

 

The above shows that the ICECAP-O measure (based on discrete choice based outcomes of McFadden, coupled with Capabilities Approach of Sen, both winners of the Economics “Nobel”) tracks happiness (after both rescaled to be on 0-1 scale) reasonably well til middle age. In old age people report suspiciously high life satisfaction/happiness scores even when they have a whole host of problems in their lives. We captured these in the ICECAP-O (collected from the same people who gave us life satisfaction scores), as well as their individual answers to a huge number of questions about these other factors in life. This has been found in the USA too:

US life satisfaction

US life satisfaction

 

 

 

 

 

 

 

In short, we don’t have a bloody clue what older people are doing when they answer these scales but sure aren’t doing the same thing as younger people.

I discussed further the contribution of trust toward a broad measure of well-being in a talk I gave years ago when in Sydney: in Australia it is basically the case that a lack of trust of those in the local community has a pretty huge (11%) detrimental effect to your quality of life in Sydney but a much smaller, though still significant (5%) effect elsewhere in Australia.

I wish these Likert-based happiness surveys would cease. They really don’t help the field, when much better alternatives are already in routine use.

disability sector survey disappointment

I’ve just tried out the survey promoted by Scope Victoria.

It’s a shame that despite a fair bit of interest expressed in the type of work I do generally (choice modelling) at a disability conference and my work specifically (in valuing social and health care and quality of life in both the UK and Australia), that the powers that be are sticking with old discredited methods.

In particular – there are a whole battery of category rating (visual analogue) scales where you have to give a number from 0 (or 1) to 10 expressing your level of agreement with, or current level experienced of, various aspects of life. I just have two questions:

(1) How are you going to deal with the numerous criticisms of these that Steenkamp and Baumgartner put forward over 10 years ago? Namely culture-specific effects and numerous other biases in how people use these scales (avoid end-points / only use end-points / etc etc)…

(2) How on earth do you hope to summarise how a respondent’s life is overall? You can’t just sum the scores, you have no idea how important they are to the individual since no properly validated task to measure the trade-offs (s)he would be prepared to make between them all was done?

This is simply going to provide a huge textbook of results that will allow service providers and politicians to cherry-pick what they want and concentrate on whatever scores seem to endorse their own views/provision. If you’re going to administer a long survey – and this is VERY long – then you should do a discrete choice experiment to find out what is important to a respondent and what their personal overall disability/quality of life score is.

Then we can track and compare and have a meaningful discussion about disabilitycare and who is being helped or not helped.