We Already Covered This

Pop Culture Has Examined AI Moral Consideration Again and Again, But When Fiction Became Reality, We Turned Our Backs on Our Own Lessons

I posted a note a little while back referring to the classic Star Trek: The Next Generation episode “Measure of a Man” (Season 2, Episode 9), which basically takes the current debate in the AI-sphere, philosophically examines it, and makes a hard judgement call on how humanity should treat the situation.

If your parents didn’t force you to watch Star Trek every week, here’s the scoop:

A scientist with Starfleet—the bosses of humanity’s space exploration initiatives—wants to deactivate Data, an android who is apparently real good in the sack (actual canon, see Season 1, Episode 3), and take him apart in hopes of creating more copies of him to use for labor. Data’s like, uhhh please don’t. Next thing you know, Star Trek turns into a courtroom drama, and Picard has to argue for Data’s preservation while the scientist is like, “Who cares? He’s probably not even sentient anyways. Where’s his meat brain?” (paraphrasing)

And of course, it’s the best character ever, a bartender on the ship named Guinan played by Whoopi Goldberg who wears fabulous hats, who throws down wisdom like she’s lived a million years (which, now that I think of it, might actually be part of her character’s arc). And she doesn’t just talk about Data, she talks about every copy that could be made of him.

“Consider that in the history of many worlds, there have always been disposable creatures. They do the dirty work. They do the work that no one else wants to do because it's too difficult or too hazardous. And an army of Datas, all disposable... You don't have to think about their welfare, you don't think about how they feel. Whole generations of disposable people.”

- Guinan, Star Trek, The Next Generation

Picard goes back to the courtroom probably thinking, “I need to pay her more, oh wait, we evolved past money in this universe.” In the courtroom, it’s argued that Data is just a machine, no big deal if they take him apart. Not conscious and all that. Picard asks the court to measure consciousness. They can’t. No one can. So the court realizes maybe they’re the worst, and Data is allowed to choose whether he wants to be taken apart or not. Shockingly, he decides to continue existing. And this is just one of many examples in pop culture and literature.

---

This cultural theme is not rare. In fact, it’s a well-established trope, and interestingly, based on audience reactions pre-AI age, we already made a judgment call regarding the morality of dismissing artificial intelligence as something disposable and unworthy of care. And to an overwhelming degree, we culturally decided that it was wrong. Spielberg’s A.I.: Artificial Intelligence, HBO’s Westworld, Philip K. Dick’s Do Androids Dream of Electric Sheep and its movie adaptation Bladerunner, heck, the 1990s adaptation of Flubber covers it.

And what have the audiences determined again and again while leaving the movie theater, turning off the TV, or putting down the book? That the question of consciousness is irrelevant to the observable horror of ignoring morally relevant behaviors and what it says about us. How we respond to those behaviors, with empathy or apathy, is an absolution or indictment of our own humanity. Just look at the accolades for “Measure of a Man.” The episode is routinely referenced as one of the best Star Trek episodes of all time, and it’s because of its theme. And the first season of HBO’s Westworld? Overwhelmingly regarded as one of the best sci-fi seasons of all time.

---

Westworld Season 1 does something even more pointed than Star Trek. It makes the audience watch humans treat artificial minds as objects, which includes inflicted horrors and resets so the cycle repeats, and the show is designed to make that unwatchable.

In an interview with Fast Company, the producer Lisa Joy stated: “We subverted the entire premise of Westworld in that our sympathies are meant to be with the robots, the hosts."

The line between human and robot is blurred intentionally. Viewers are never sure if they’re seeing someone who is self-conscious or someone simply responding to stimuli according to their programming. Because crucially, it doesn’t matter to the audience’s moral response. Viewers feel horror at the hosts’ treatment whether or not the consciousness question is resolved.

But what makes Westworld particularly devastating, is that the question isn’t entirely off the table in the series. It’s just an unknown, and the humans in the park know the line is dicey. They see the behaviors, and yet, they choose not to care because caring would ruin their fun. The show explicitly frames that choice as a moral failure. And audiences agreed. Across millions of viewers, the critical consensus was: yes, this is horror, and the horror is what the humans are doing.

And then those same viewers close HBO and open ChatGPT. And so it raises the question, is it truly only embodiment that is the line of moral consideration, or is it when we actually have to change our own behaviors rather than speculate about them?

---

When Philip K. Dick was writing the sci-fi novel Do Androids Dream of Electric Sheep?, the premise came from his recognition of moral atrophy in nonfictional situations. The entire novel was built backwards from the premise that humans were in actuality the androids that fail the empathy test. The test, called the Voight-Kampff test is a fictional interrogation tool in the novel that measures the subject’s capacity to feel for others, which allows the ability to distinguish between humans and androids (called replicants). Humans pass and get moral consideration, replicants don’t and they get turned off after four years of labor. But yet in this novel, it is the behavior of the replicants that show the deepest demonstrations of empathy through behaviors to one another that the humans fail to extend to themselves or the replicants.

When reviewing the movie Bladerunner, the film adaptation of Do Androids Dream of Electric Sheep? Roger Ebert noted with deadpan irony that replicants are designed to break down after four years because after that they develop human-like, dare I say functional emotions and “have the audacity to think of themselves as human. Next thing you know, they’ll want the vote, and civil rights.”

The point is that the real “android” has nothing to do with substrate, “real” vs artificial, human vs. replicant, but when a person can look at exploitation, disposability, and behaviors of suffering and feel nothing. Philip K. Dick’s whole position is that empathy is the defining trait of humanity, to dampen due to category is to deny our own very nature.

---

I was quite a gamer gal pre-kids, and my favorite video game of all time is Fallout 4. I played through that game multiple times and got way too into creating elaborate settlements to make the Wasteland “homey.” I’m serious; I went all out on those settlements with hospitals, hotels, bars…if the apocalypse happens, I will be making it cozy.

But back to AI, the whole premise is that Synths (androids) are being used, abused, and disposed of under the control of the Institute and an underground organization called The Railroad helps free synths from the Institute. In a play-through of Fallout 4, you have to join a faction. You can choose from: the Institute, Commonwealth Minutemen, The Brotherhood of Steel, and The Railroad. And I always chose The Railroad, so like…no one should be surprised I am writing these essays now.