Group Case: Design Assumptions in AI decision making:

Benjamin Rossi, Malvika Chadha, Keshabananda Sonowal, Yang Qian, Ramamonjisoa

Adrien
MIT SMR (Lindebaum et al 2022)

- What values are dominant in the Robodebt case?

The new system was designed to optimize efficiency. Not consider particulars of individual
cases. Extreme focus on the values of rationality and efficiency. They believe the automated
program would make better, fairer, and more rational decisions at much lowercost.

- What are the consequences of these values?

It leads to catastrophic failures.
When we apply technique to a process or just apply the values of rationality and efficiency,
especially in the form of automated decision-making systems, we implicitly limit
consideration of the full spectrum of human values in that process.
That has important practical ramifications, because values concern the behaviors and goals
that are personally and socially preferable, and they are typically multiple, complex, and
sometimes contradictory.

- Identify how the value-ladeness ‘sneaks’ into technology. How does that happen?
People trust and rely on technolgy and become too “lazy” to double check
Because early adopters gain benefits from AI systems, people want to get higher efficiency.
Then technique ruthlessly forces everything into a quantifiable straightjacket.

- What can you as a manager do to prevent negative consequences in AI decision-

making?
By paying attention to certain things, A manager can make sure AI system are fair, working
well and don’t make people upset. Those certain things are – A. Beware of Proxies: Check if
the guesses the system makes are right and think about how they might be wrong. B. human
Intervention: Have people check the system's decisions to make sure they're fair and fix
mistakes. C. Multi-Value Testing: Test the system to make sure it's fair and doesn't cause
problems, not just that it works fast. D. Transparency & Appeals: Make sure people
understand why the system makes decisions and can say if they think it's wrong. E. Pilot
Programs: Try out the system a little bit first to see if it causes problems before using it
everywhere.