top of page
Search

Value Complacency

Writer's picture: RickyRicky

This week, I sat in on a graduate-level computer science class.


The instructor asked under what conditions humans and AI would be synergistic—that is, when the combination of both would outperform either humans or AI working alone.


One student in the back raised his hand and said,

“When the AI model is 90% accurate.”

Sorry, what?


I share this not to put anyone down or be mean—that definitely wasn’t a career highlight where he’d been paying the most attention—but this numerical fixation with whether the AI earns an A is a helpful illustration. Kenneth Liberman calls this fascination with round numbers in particular a digital fetish.


I actually think this moment is symptomatic of a larger problem I’ve started to notice.

  1. We often assume that there is a best way to do things.

  2. We agree on a proxy—a shared metric or procedure—to help us measure our current performance against that ideal.

  3. When we do better on the proxy, we celebrate our measurably improved performance.

  4. We forget that the proxy is just a measure. (That’s value capture.)

  5. But we also assume that we agree on our values. (I’m gonna call this value complacency.)


In value complacency, because we’ve buried our real ethical values under a shared proxy, we’ve failed to investigate the fact that we probably don’t agree on what underlying ethical concepts like fairness mean.


So now when we do well on the proxy…that’s good, right?


Well, is it? How do you know?


This problem is happening all over AI spaces.


Say we’re developing an AI model to read CT scans and diagnose cancer. Here’s an important question: How does its performance compare across different subpopulations? Is it just as good for women as for men? For black patients as for white patients? How about interaction effects—is it just as good for black women?


We often assume that there is a best way to do things. So sure, a ‘perfect’ diagnostic model would just diagnose everyone correctly. But it would have other, subtler properties too. For example, its technical performance would not be compromised by ‘irrelevant’ factors like gender or race. That would indicate biased performance, but a ‘perfect’ model would be perfectly fair! (It would be equally ‘perfect’ for everyone.)


Okay, great. So how are we gonna measure the fairness of our model in real life? There are tons of fairness metrics floating around, but let’s say that you and I are researchers on a project and we agree on three that we like.


Now we try to drive improvements on those three metrics, so we can show that our model does particularly well by those lights. When we finally do, we congratulate ourselves for developing fairer techniques.


But we’ve forgotten that improving at proxies is different from improving at the real thing. And we never really discussed the real thing. What is fairness, anyway?


Since we never discussed that, we never deeply considered what it means for a factor to be ‘irrelevant.’ Which subpopulations were we gonna compare, anyway?


But there’s an even deeper problem. Since we each started from our own assumption of what a perfectly fair model would be like, we don’t actually know how far we agree or disagree. Sure, we ended up codifying some notion of fairness by agreeing to use these three metrics in our paper.


But this shared procedure can actually conceal our theoretical disagreements!


We may never realize we mean different things by unfairness until some punk philosopher asks a weird question during Q&A. Until then, our practical convergence hid any theoretical divergence.


I’m realizing I need to do more work on philosophy of measurement, a branch of philosophy of science full of interesting case studies like the history of the meter. From Wikipedia:

In 1799, the metre was redefined in terms of a prototype metre bar, the bar used was changed in 1889, and in 1960 the metre was redefined in terms of a certain number of wavelengths of a certain emission line of krypton-86. The current definition was adopted in 1983 and modified slightly in 2002 to clarify that the metre is a measure of proper length. From 1983 until 2019, the metre was formally defined as the length of the path travelled by light in vacuum in 1/299792458⁠ of a second. After the 2019 revision of the SI, this definition was rephrased to include the definition of a second in terms of the caesium frequency ΔνCs.

I’m interested in so much here. These are extraordinary technical leaps in how we try to standardize measures of length. Why think our understanding of length is improving? Why think we’re redefining the meter, or even modifying the current definition, as opposed to doing something else?


And remember, this is just the history of measuring length, not fairness! It seems like length should be a much easier to address as a technical problem.


For now, I think my job is still to show up at talks and be annoying. But in the long term, I have to figure out how to help folks step back from the proxies long enough to talk about the underlying constructs.


You know, the things you’ve supposedly agreed how to measure.

Related Posts

See All

Comments


Sign up for more philosophy in your life!

Thanks for subscribing!

bottom of page