Measuring fuzzy objects

vivianimbriotis | Sept. 25, 2019, 1:40 p.m.

An epidemiologist and a philosophy student started their morning with coffee and an argument.

“So I was reading this blog post about Goodhart’s law, and I was thinking about how it might apply to health,” said Eugene, a philosophy undergraduate currently burning himself on a sugarless long black. “I mean that’s your job, right? Or at least part of it. Measuring health?”

“Goodhart’s law...you mean that using a proxy measurement of something often causes it to be a poorer measurement?” said Asha, drinking the much more respectable latte.

“Yeah,” Eugene said, idly fiddling with his napkin. “Like the whole cobra thing.”

“Well I mean there’s definitely examples. Like some doctors think that incentivising shorter and shorter wait times might actually worsen access to healthcare, because everyone just gets seen and brushed off rapidly.”

“I don't know about that, but what about measurements of health itself going wrong? I mean you can measure health, right?”

“Well one must in order to know how well an intervention works, for example. Or to compare healthcare systems. It’s tricky, though. You have to not just measure life expectancy, because then you’d miss a lot of chronic disease that makes people miserable. And mental health, for example.”

“What about children in Flint, Michigan, whose life expectancy is about the same as their peers anywhere else, and how might not be actively suffering, but whose intellectual development has been delayed or reduced by lead poisoning? Are they less healthy than their peers?”

“Well...intuitively, yes. Neurological damage does seem like worse health.”

“Even if it doesn’t actually correlate with a shorter lifespan? Or with suffering?”

“Yeah. Like I’d rather not have a stroke that would reduce my functioning, or my intelligence, even if I wouldn’t be sadder or less long-lived afterwards.”

“Okay, so what is health about then? Living longer? Being happier? Being able to perform functional tasks?”

“It’s about all of those things! There’s this nice World Health Organisation definition...” Asha pulled out her phone. “‘Health is a state of complete physical, mental and social well-being and not merely the absence of disease or infirmity.’”

“Huh...that definition is sort of garbage, though. I mean if we want to measure health, surely we need to be able to say how much it is about each of those components. Like, compare someone who has lost a limb to someone who has depression. Who is less healthy?”

“Well, for that you use the Disability Adjusted Life-Year, or DALY. You decide how much each year of living without a limb is ‘worth’ in terms of a year of life at full health - say each year of limbless life is worth 0.83 DALYs. Then you add up the total expected number of DALYs each will get and whoever has more is healthier! At least on the population level, where the individual variance will balance out.”

“But you’re making all these value judgements there! What if the person without the limb is happier than a person with chronic kidney disease? Does that matter?”

“We’re talking about health, not happiness.”

“I guess, but the more I think about it, the more it kind of feels like disability and life expectancy are two completely different things.”

“Well often we do split things up into mortality, impact on survival, and morbidity, impact on disability. But DALYs are really essential for intercomparing an intervention with regards to its effect on health overall.”

“Shouldn’t epidemiology at least be actively discussing the politics and ethics of deciding on this system? Of even of deciding on the specific numbers for each disability?”

“Are you serious? Then we’d be glued down in debates all the time and never get anything done! Surely that’s more unethical than just picking something, some system we can use to measure health.”

“I still don’t know what you mean by health--”

“I GAVE you the WHO definition--”

“That’s not a definition, that’s a single point in health-space! It just tells me what optimal health looks like, it doesn’t help me compare the health of two people with two different sets of problems!”

“Right, which is why we need DALYs--”

“But how do you come up with how much each disability is worth--”

“Well you ask the experts.”

“Well how do THEY decide? Look, if this is the health profession being some kind of utilitarian then just acknowledge that and move on.”

“But we’re not! These measurements need to be a compromise between the values of everyone in society!”

“Why?”

“Because we live in a democracy, Eugene.”

“Oh, so did everyone vote on the definition of a DALY in a referendum? Or have I missed all the major parties’ policy stances on the DALY algorithm?”

“Look okay, maybe we can’t measure health directly, but we can definitely measure things that correlate with it! Like all those things you said before - life expectancy and happiness and whatever.”

“But do those things correlate with an Actual Thing called Health, or do they just correlate with each other often enough that we’re all pretending there’s some objective thing called health even if we can’t measure it--”

“And this,” said Asha, picking up her paper cup and standing, “is why no one likes philosophy majors.”

We all kind of know what people mean when they say “Health”. We know what a healthy person looks like and what an unhealthy person looks like. But our intuitive understanding seems to combine lots of things - probable remaining years of life, pain and discomfort that will be experienced during that remaining life, ability to perform functional tasks, even ancillary things like intelligence as Eugene mentioned. And while those things are often correlated, they’re not always correlated - like in limb amputation, where they may be little difference in life expectancy, or in the early stages of chronic kidney disease, where there may be no effect on day-to-day life. So when we try and measure “Health”, which our intuition says is a unitary concept, maybe we’re actually measuring many different things that are related only by statistical correlation - and there is no actual thing-in-the-world to which the label “Health” refers.

Compare beauty. We’d find it weird if a study purported to measure beauty in some objective sense, because we all understand that beauty is a constructed and ill-defined thing. It exists as a heuristic, as something on our abstracted map of reality, but it’s not a thing-in-the-world like blood pressure, or life expectancy, or mass or charge or colour. And if health is the same, then deciding what gets to go into the big bucket of health, and how we’ll measure it, is political.