Answers from nowhere solving nothing

Why do AI examples increasingly feel like those Cybertruck "offroading" videos?

From the New York Times today (bolding mine):

Apple struck a deal with OpenAI, the maker of ChatGPT, to support some of its A.I. capabilities. Requests that its system can’t field will be directed to ChatGPT. For example, a user could say that they have salmon, lemon and tomatoes and want help planning dinner with those ingredients. Users would have to choose to direct those requests to ChatGPT, ensuring that they know that the chatbot — not Apple — is responsible if the answers are unsatisfying. Sam Altman, the chief executive of OpenAI, attended the Apple event.

Great! Thank God you don’t have to rely on search for such a complex request. Here’s the Google result for that question, which… gives you a bunch of recipes actually:

Search result for “what can i make with salmon, lemon, and tomatoes”, showing multiple recipes

Scrolling further down that page…

More results including some Reddit forums…

Now I suppose some will quibble, because of course maybe your experience with such a search might involve going to these pages and finding out you don’t have one of the other ingredients, and maybe there is some way AI would solve that hypothetical problem. Thank God for AI, right?

But you’d be wrong there, because that problem doesn’t exist. From the first result after the recipe carousel:

Ingredients list for the first recipe: you need tomatoes, olive oil, pepper, salt, lemons, and salmon.

I’m increasingly confused by the use cases of AI. To me they resemble those fan videos of people driving Cybertrucks, where people excitedly film themselves going “off-roading” on something that is actually a dirt road you could navigate with a minivan, or showing how you can fit four bags of horse feed in the bed of a Cybertruck, when you’re looking at it and thinking “I’m pretty sure I could fit those in my VW Golf with the seat down.”

What’s driving this odd and embarassing behavior where we all have to pretend technology is currently at some 1998 level? I have to imagine it’s not anything to do with current, pre-AI technology, which seems to effortlessly solve all the problems that are being presented as intractable. It’s something else, maybe about legal issues, or market share, or the regulatory environment. There are absolutely some good use cases for LLM tech. But it’s telling that in most cases the pitch is answers from nowhere, solving nothing.