The Summary Problem, or why AI in search is attractive

Don't worry, we'll get to the downsides soon enough

I’m not a fan of the headlong rush to put AI at the center of search; a search of my Twitter feed and public statements will make that clear. At the same time I often find people criticizing the use of AI in search without understanding the information seeking problems it seeks to address.

I’m a firm believer that the first step to better solutions is to be honest about the problem. This is my (short) attempt to explain why AI is attractive to both users and platforms, apart from financial concerns. That is, this post outlines one significant user-centric case for AI in search. In a later post I’ll discuss the issues it raises.

The problem of directed summary

Let’s start with a problem. You see this online.

Post saying: 🔥🔥ADRENOCHROME🔥🔥 - THIS IS THE REASON, DeepState, Ukraine and Everything is getting CLEANOUT INCREDIBLE-DISGUSTING-SHOCKING Child & Human Trafficking. Share far and wide to the blind.

You don’t know what this means. You’re concerned about it. So you go and search “Adrenochrome”.

The results you get back are good, mostly. We’re a far way away from where searches like this would spiral down into conspiracist nonsense. For a variety of reasons data voids are harder to stumble into than they used to be.

Of course, reading a page like this is not as easy as it seems. The Wikipedia link doesn’t seem like it’s answering the kind of question we have. And the first result after the Wikipedia link — notice that the text clips off after “Dark Virality of a Hollywood Blood Harvesting…” Um, blood harvesting what? Blood-harvesting conspiracy theory? Or blood-harvesting plot? There’s a big difference between the two. Likewise, the QAnon reference in the second results warns me that we are in the conspiracy theory-verse. But miss the quotes around the statement that “Adrenochrome is a drug that the elites love. It comes from children.” and you might get the wrong idea. On the whole it’s a good result page, but it still requires more cognitive effort and attention to parse than one might realize.

So I click through the links, and a couple things happen. First, I find out I’m out of free Wired articles. So that’s not going to help — and it’s a bit frustrating because from my perspective as a user, I don’t want the article, I just want a very simple answer to a question that goes something like “Why are people talking about adrenochrome, and is what they are saying true?” But the modern experience of trying to get an answer is hitting a paywall.

So I click the Daily Beast article, and this time I’m in luck. Free article. Except.

Daily Beast article intro

This has a beautiful, literary intro. “In recent months, the YouTube comments for a song by the 1980s post-punk band The Sisters of Mercy have veered slightly off-topic.” The article then spirals delicately down to its main point — how the adrenochrome conspiracy theory came to be. Except it’s frustrating — I might want to read this article later, but right now I just need a two paragraph explainer, not Joan Didion.

If you want to try it for yourself, put yourself in the mindset of someone who isn’t a big reader, and just needs a two paragraph answer on what the heck Uncle Paulie is posting on his Facebook page and whether they should be concerned, then click through. It feels… not good, right? When we need very specific information and people are not translating for our context it feels frustrating and alienating. We learn skills to deal with that, but they never wholly address the underlying tension between our needs and the writer’s goals.

For most authors summary is a stop on the way to somewhere else

What you need, of course, is a concise summary. And those summaries exist, but outside sites like Wikipedia and Stack Overflow, summaries are usually a means to getting somewhere else. That Daily Beast article does eventually summarize what the conspiracy theory is and why it’s wrong and dangerous, but the story’s larger hook is how a particular idea — dormant for decades — gets woven into a broad conspiracy theory. The summary here is in service of a larger idea about the weirdness of our time, the odd dynamics of conspiracy culture, the way obviously fictional stories develop in music and film and then move back out into complex and sometimes dangerous theories about the real world.

There’s a financial model for stories like this. Not a great one, as the past few weeks have shown, but something. The Daily Beast story is unique, the kind of thing you can share as a really interesting story with your friends. It’s got a hook, it’s got a throughline, it’s got an arc. It gets clicks and readthrough so that you can neatly fit three in article ads for The Family Stallone (new, from Paramount+!) in between a meditation on how desires for grand stories go astray.

Some summary writing can work like this. When particular misinformation goes viral, for example, a fact-check becomes sharable, and provides a good summary and gets shared. But for financial reasons, a lot of the niche summary that exists explicitly as summary comes out of a world of peer production. We get summaries from Wikipedia, Reddit, Stack Overflow, not Huffington Post. Smart searchers know this — when looking for a summary on an issue and wanting to skip the process of mining longer articles for the answer they need they will append “reddit”, “stack overflow”, “fact-check”, or “wikipedia” to their searches in hope of getting a non-paywalled answer that is direct and to the point. Other commentators have interpreted this as "people wanting an answer from a human” and that’s partially true. But of course Daily Beast articles are written by humans too. My suspicion is that for many people the desire is just to get something that is an answer not an article. Something that is content to answer a narrow question without selling something, making a larger cultural point, or forcing one to navigate a paywall to figure out if Uncle Paulie is becoming a white supremacist.

I also suspect that this is part of what people are saying when they say that search results have gotten worse. Personally, I think search results have gotten better, particularly since 2018 or so, and one reason why is that before 2018 the sort of person you were likely to find answering your adrenochrome-style question was likely to be a conspiracy theorist. But what the user notices is there seem to be more articles and less answers whenever they punch in a query. That means a lot of mental translation, and that doesn’t feel great.

A known, but persistent problem

I’m not detailing something unknown here. If you look at the various features Google has introduced over the past years, even before the recent AI push, a lot of them are targeted at this problem. Snippets, featured results, direct answers, highlighted excerpts — all of these are methods of turning articles into answers, and trying to reduce the cognitive load of someone doing that mental translation.

So into this comes AI. And this is the attraction. When I type “why are people talking about adrenochrome and is it true” into Bard this is what I get:

People are talking about adrenochrome because it is a component of several conspiracy theories, such as QAnon and Pizzagate. These theories allege that a cabal of Satanists are harvesting adrenochrome from the blood of children and using it as a drug or as an elixir of youth. There is no evidence to support these claims, and adrenochrome is not a psychoactive drug. It is a chemical compound that occurs in the body, but it has no known medical or recreational value.

The belief that adrenochrome is a powerful drug or elixir of youth appears to have originated in the 1950s, when it was the subject of limited research as a potential cause of schizophrenia. However, these studies found no evidence that adrenochrome was harmful or psychoactive. In fact, the compound is relatively harmless and can be easily synthesized in a laboratory.

The recent resurgence of interest in adrenochrome is likely due to the popularity of the conspiracy theories that mention it. These theories have been spread online and on social media, and they have been amplified by some celebrities and politicians. However, there is no evidence to support these claims, and they should be treated with skepticism.

Bard often does a lot worse than this, but I chose this example to talk about the attraction of AI, because this example is not half-bad. There’s some oddities here — for instance with a theory like adrenochrome you absolutely should go further than “treat[ing] it with skepticism.” (It’s fabricated antisemitic blood-libel mythology, you should throw it in the trash bin entirely). But a lot of this response is quite good. It gives me a quick summary of the conspiracy theory. It tells me a few things that directly debunk it — far from being mystical and rare, adrenochrome doesn’t have the impact that people say it does, and it’s easily synthesized in a lab. It’s neither valuable or rare, which kind of undermines the whole theory.

In this case, you can imagine an average user getting this being as well-informed as someone that goes through the search process and builds up their own answer by skimming articles in search results. It’s not perfect, but there’s a good chance the mental model a searcher builds from skimming articles is not perfect either. Importantly, in this (very limited) case, the relative quality of the AI-generated response held true even though the compared search results were high quality. If you squint hard enough, AI seems to solve the summary problem, turning articles into answers so that the user doesn’t have to put that labor in, with both the frustration and opportunities for error that entails.

It doesn’t solve the summary problem, of course, and I’ll talk in later posts about how that hope that this solves it is unfounded in practice, but my goal for this post was to provide a frame for thinking about why both platforms and some users find this technology attractive so I’ll stop here for now, and save the dismal for the next post.