Tag Archives: aer

open letter to sen and mcfadden

Dear Professors McFadden and Sen,

I am writing to alert you to a paper in AER last year that makes serious failures in acknowledging:

(1)   your main paradigm-shifting work in discrete choice modelling for which you won the Nobel prize (McFadden) and

(2)   Work that used the work of McFadden to operationalise the Capabilities Approach for which you won the Nobel prize (Sen).

I wish to make it clear that one author of the paper in question has already publicly acknowledged (2) above and given that McFadden’s work is (arguably) “more mainstream” in economics, the failure to acknowledge (1) makes the paper’s errors even more serious – at least, in the eye of economists.

The paper in question is Beyond Happiness and Satisfaction: Toward Well-Being Indices Based on Stated Preference. Daniel J. Benjamin, Miles S. Kimball, Ori Heffetz, and Nichole Szembrot. Am Econ Rev. 2014 Sep; 104(9): 2698–2735.

I mention above that I have already had a recognition by Miles Kimball (who, by reputation, is probably the senior author) of a lack of referencing of the work to operationalise Professor Sen’s Capabilities Approach.

(see attached screenshot of Miles’s recognition of his poor referencing on Twitter)

ScreenHunter_01 Sep. 13 12.30





Yet, even if I were to accept “the “Sennian literature being often outside the mainstream economics literature” as a mitigating factor (which I do not), proposing a discrete choice model to elicit public values for elements of well-being is most definitely not “outside mainstream economics”. Of course your work on discrete choice models, Professor McFadden, is entirely “mainstream”.

A friend/colleague of mine working in the “more mainstream” branches of economics considers my shock and annoyance entirely justified – she cannot believe the authors were unaware of the work of the team in which I worked from 2001-2015 to use stated preference discrete choice models to value elements of the Capabilities Approach.

In terms of my reputation, and that of the wider group, I worked with Professor Jordan Louviere for many years and a textbook which we, together with Professor Tony Marley (acknowledged in Professor McFadden’s Nobel lecture) co-author is being published by CUP this month. (Please see my google scholar profile for details of my publications).

I do recognise and respect Professor Sen’s issues with putting numerical values on elements of Capabilities, but the “ICEPOP” team in which I worked has, I would posit, gone the furthest in providing a framework in which the importance of these can be valued, leaving the societal decision rule to be decided separately, by you Professor Sen, or others – please see numerous publications by Professor Joanna Coast and me.

Anyway, our group has been the worldwide leader in marrying your respective approaches in order to value well-being and I consider it insulting, to say the least, that a paper in AER should appear, claiming that crown, when our papers have been available in reputable (and in some cases hardly non-mainstream economics) peer-reviewed journals for a decade.

I make no request of you for further explicit action – I left academia several months ago, partly due to this kind of behaviour. I merely would ask that if further work by these authors or others, “passes by you or your colleagues”, you make them aware that valuing Capabilities using discrete choice models is well-progressed already and that they should be checking the publications of me, and the wider ICEPOP team.

Kind regards



Terry N Flynn MA(Cantab), MSc, PhD
Director, TF Choices LTD

Tel: +44 (0)115 888 0809 (UK); +61 (0)2 8006 0907 (AUS)
Twitter: @tfchoices @tflynnhealth


Copyright Terry N Flynn September 2015

top economists fail basic literature review shock horror

So, via a blog I follow, I learn that four economists have invented how to operationalise valuing well-being using stated preference discrete choice tasks.

One problem with that. They haven’t. We did it – at least, using the “Sennian stated preference approach” they follow, we did – others did so using health-related quality of life even earlier. Over 10 years ago. They either fail an undergraduate literature review or they are simply liars.

So, Daniel Benjamin, Miles Kimball, Ori Heffetz, and Nichole Szenbrot,  which is it?

The article was published in the American Economic Review, that bastion of “wonderful economics that proved so able to predict real life events”.

I shall simply reproduce the comments I made to Mike Norman, who posted the link to it. I left academia – partly because of clowns like these who run the profession. Thus I am under no obligation to go through every single piece of rubbish or already proven stuff they present.

Thurstone (1927) provided the statistical basis of what they propose – Random Utility Theory. If we are looking for an economist who made a real contribution to the field we need look no further than McFadden, the (co) winner of the “Economics Nobel” in 2000 for showing how this could predict demand (the BART in California – BEFORE it was even built!)

However, all the theory was also developed in mathematical psychology in the 1960s by Duncan Luce and Tony Marley (who is still active in the field, and with whom I have the honour of co-authoring).

Using this to elicit the public’s preferences for aspects of well-being (and specifically operationalising Sen’s approach) is NOT new. There are various groups who have been doing it for close to two decades. A lot of the early work limited itself to health-related quality of life, but work to value general well-being, including fully operationalising Sen’s Capabilities Approach, was well underway by 2006 by the group in which I was working. I see no references to that work by these authors. Again, I find it staggering that mainstream economists can get away with such poor reviews of the literature – but again, the literature in which we published all this stuff is literature they don’t consider “worthy”.

I could go through their article point by point demolishing the out of date rubbish they propose (preferences with descriptors such as “somewhat prefer” which have been shown to be rubbish in the marketing and applied economics literature and which are of no use in PREDICTING AN ACTUAL CHOOSE/NOT CHOOSE DECISION) but I now work in industry and have better things to do. My clients would not give their stuff a single glance, let alone a second one.

Some references showing how to do this using methods that have a 90 year pedigree are given below. It is a shame these “proper” economists do not see fit to reference them. And people wonder why I left academia? My first reference is a text-book that includes a chapter detailing work done several years ago demonstrating what they want to do, and with much better methods.

I find it astonishing that mainstream economists are STILL trying to pass off other people’s original work as their own, when it becomes clear their stuff is rubbish.



Best-Worst Scaling: Theory, Methods and Applications (2015). Louviere JJ, Flynn TN, Marley AAJ. Cambridge University Press.

A law of comparative judgment (1927). Thurstone, L. L. Psychological Review 34, 273-286

Conditional logit analysis of qualitative choice behaviour (1974). Frontiers in Econometrics (Zarembka P – ed); 105-142.

Developing attributes for a generic quality of life measure for older people: Preferences or capabilities? (2006) I Grewal, J Lewis, T Flynn, J Brown, J Bond, J Coast. Social science & medicine 62 (8), 1891-1901

Best-worst scaling: What it can do for health care research and how to do it. (2007) TN Flynn, JJ Louviere, TJ Peters, J Coast. Journal of Health Economics 26 (1), 171-189

Probabilistic models of set-dependent and attribute-level best-worst choice (2008). AAJ Marley, TN Flynn, JJ Louviere. Journal of Mathematical Psychology 52 (5), 281-296

Quantifying response shift or adaptation effects in quality of life by synthesising best-worst scaling and discrete choice data (2013). TN Flynn, TJ Peters, J Coast. Journal of choice modelling 6, 34-43

Using discrete choice experiments to understand preferences for quality of life. Variance-scale heterogeneity matters (2010). TN Flynn, JJ Louviere, TJ Peters, J Coast. Social Science & Medicine 70 (12), 1957-1965

Valuing the ICECAP capability index for older people (2008). J Coast, TN Flynn, L Natarajan, K Sproston, J Lewis, JJ Louviere. Social Science & Medicine 67 (5), 874-882

Developing attributes for a generic quality of life measure for older people: Preferences or capabilities? (2006) I Grewal, J Lewis, T Flynn, J Brown, J Bond, J Coast. Social science & medicine 62 (8), 1891-1901