I appreciate this post so much! I have been frustrated with critics I respect and admire for similar reasons.
I also think that if educators never experience moments of wonder and excitement about the capabilities of current AI, we will not be able to fully understand where students are coming from or where society is going.
I do think even folks who end up refusing to use AI or invite students to use it on ethical grounds should probably engage for a period just to experience this.
Exactly. Btw, I notice you have done previous (and perhaps current) work in argument analysis (and maybe argumentation theory?) I think you might like this thing I've been working on a few months and plan to start recording on YouTube soon -- teaching people Toulmin analysis using AI. It's super super cool and I can't believe it works as well as is does. It's what pulled me away from being a skeptic.
This is roughly where I am with regard to AI. I'm a programmer, and while in some senses it is a glorified autocomplete, it's one that's often spookily good at guessing what I want to do next. It gets things wrong, but it gets them wrong in ways which have a kind of logic to them.
Like many, I am concerned about the power usage and companies profiting off other people's work, the deluge of AI generated slop, the labour issues, and so forth. But I think there's often a tendency to conflate bad consequences with not being an impressive technology.
It's worth pointing out that the reverse image search is also AI. It's machine learning classifying and recognising features of images in order to index and match them. It's not an LLM, it's not generative AI, but it's using some of the same core technology. So saying something is 'just' using reverse image search is not really an argument.
Yeah, it's telling that there are so few people who think "AI is horribly unethical" and "It's surprisingly good at things". The critique generally runs it's horrible, and does nothing, as if it were crypto or NFTs. And it's not.
I am worried about the ethical issues around its production. But I'm just as worried, if not more, that people of good intent steer clear of it and the worst people embrace it then that is ceding important ground and important tools.
It’s interesting that the ai actually reads the wrong usdot number in its reasoning (it reads 254406 instead of 354406). So I decided to dig into it a bit more.
It mentions that range being common in NH, but NH doesn’t actually require usdot numbers, it’s one of 15 states not requiring it (I assume it’s just that the bus operator is from Ohio.)
As far as I can find, usdot numbers have no type nor specific documented structure either and there isn’t any verifiable way to get ranges matched to geographic areas (many are assigned for inter-state transports of hazardous materials with no related types), just direct lookups.
So what you have here is something that looks like a good response, but with broken reasoning.
Another thing it does that’s like a psychic con/cold reading is take a general thing like “is operated by "First Student," which is one of the largest school bus operators in North America.”
And then also uses it as validation in “It's operated by First Student, a major contractor in New Hampshire”.
But because it’s the biggest one, it operates in like 30+ states and even Canada, and runs half of all the school busses in the US. So the statement “a major contractor in <state>” is like “has lost someone close to them” — there’s almost no significance there except to the audience.
Rather than actually being valid reasoning it sounds like after-the fact conjuring of a flawed explanation. Like is the rock wall behind really the kind of stuff farmers would have built (right next to an asphalt street), or is it just an after the fact rationalization once a location was “picked” as an answer and then rationales are invented?
How else would one explain near-okay answers obtained by such flawed processes?
As a side note, we should probably be careful about attributing human properties to processes that do not operate that way. There’s a chance that merely wanting to believe it to be true lowers our defences and makes us ignore flaws in the mechanism and eventually lead to inadequate use of technology due to misplaced trust.
Yep, the more detailed it gets the more I would distrust it. Which is why my advice in the post previous to this is to evaluate the reasons that make sense and discard those that don't. There are reasons in this that you can know are valid, ones that you can check and ones that are too out there.
Sorry if I wasn't more clear that it gets some of this stuff wrong, but gets enough verifiable stuff that you could make a case. It's the same way with the bus route being regular if you look at it, yeah, it maybe rules out the first day, where people aren't slouched looking at their phones, but not the first week.
On the other hand, the point about the small town New England feel and the sidewalks is pretty solid -- if you want a a sidewalk neighborhood in suburbia in NH then you do end up in a college town. The stone wall is rebuilt and not original, sure, but it's the sort of thing you see in new England a lot. Your job is to look at the case it makes and see what makes sense.
I was kind of doing this as a follow-up to the previous reasoning post, but I realize I did not pull enough of that in. So thanks for that.
Thank you. Can you provide a pointer to, or just, a definition of "reasoning pattern?" And can you speculate on the internal mechanisms by which recent GAIs are acquiring and applying them?
It actually gets the year right in the month reasoning. Did you give it that or did it get there with the month nudge?
Good eye -- yeah it asked me if 2013 was right after the first one and I said no, 2011 before I continued but I cut that out for brevity
Excellent argument, Mike.
I appreciate this post so much! I have been frustrated with critics I respect and admire for similar reasons.
I also think that if educators never experience moments of wonder and excitement about the capabilities of current AI, we will not be able to fully understand where students are coming from or where society is going.
I do think even folks who end up refusing to use AI or invite students to use it on ethical grounds should probably engage for a period just to experience this.
Exactly. Btw, I notice you have done previous (and perhaps current) work in argument analysis (and maybe argumentation theory?) I think you might like this thing I've been working on a few months and plan to start recording on YouTube soon -- teaching people Toulmin analysis using AI. It's super super cool and I can't believe it works as well as is does. It's what pulled me away from being a skeptic.
This is roughly where I am with regard to AI. I'm a programmer, and while in some senses it is a glorified autocomplete, it's one that's often spookily good at guessing what I want to do next. It gets things wrong, but it gets them wrong in ways which have a kind of logic to them.
Like many, I am concerned about the power usage and companies profiting off other people's work, the deluge of AI generated slop, the labour issues, and so forth. But I think there's often a tendency to conflate bad consequences with not being an impressive technology.
It's worth pointing out that the reverse image search is also AI. It's machine learning classifying and recognising features of images in order to index and match them. It's not an LLM, it's not generative AI, but it's using some of the same core technology. So saying something is 'just' using reverse image search is not really an argument.
Yeah, it's telling that there are so few people who think "AI is horribly unethical" and "It's surprisingly good at things". The critique generally runs it's horrible, and does nothing, as if it were crypto or NFTs. And it's not.
I am worried about the ethical issues around its production. But I'm just as worried, if not more, that people of good intent steer clear of it and the worst people embrace it then that is ceding important ground and important tools.
Good point about image search.
In case you missed it. Reconsider use of the word “ethics” when using tools built on stolen words and artwork.
https://substack.com/@glovelace/note/c-83933460?r=410aa5
These arguments and examples make me think all about what is reasoning and what is intelligence.
It’s interesting that the ai actually reads the wrong usdot number in its reasoning (it reads 254406 instead of 354406). So I decided to dig into it a bit more.
It mentions that range being common in NH, but NH doesn’t actually require usdot numbers, it’s one of 15 states not requiring it (I assume it’s just that the bus operator is from Ohio.)
As far as I can find, usdot numbers have no type nor specific documented structure either and there isn’t any verifiable way to get ranges matched to geographic areas (many are assigned for inter-state transports of hazardous materials with no related types), just direct lookups.
So what you have here is something that looks like a good response, but with broken reasoning.
Another thing it does that’s like a psychic con/cold reading is take a general thing like “is operated by "First Student," which is one of the largest school bus operators in North America.”
And then also uses it as validation in “It's operated by First Student, a major contractor in New Hampshire”.
But because it’s the biggest one, it operates in like 30+ states and even Canada, and runs half of all the school busses in the US. So the statement “a major contractor in <state>” is like “has lost someone close to them” — there’s almost no significance there except to the audience.
Rather than actually being valid reasoning it sounds like after-the fact conjuring of a flawed explanation. Like is the rock wall behind really the kind of stuff farmers would have built (right next to an asphalt street), or is it just an after the fact rationalization once a location was “picked” as an answer and then rationales are invented?
How else would one explain near-okay answers obtained by such flawed processes?
As a side note, we should probably be careful about attributing human properties to processes that do not operate that way. There’s a chance that merely wanting to believe it to be true lowers our defences and makes us ignore flaws in the mechanism and eventually lead to inadequate use of technology due to misplaced trust.
Yep, the more detailed it gets the more I would distrust it. Which is why my advice in the post previous to this is to evaluate the reasons that make sense and discard those that don't. There are reasons in this that you can know are valid, ones that you can check and ones that are too out there.
Sorry if I wasn't more clear that it gets some of this stuff wrong, but gets enough verifiable stuff that you could make a case. It's the same way with the bus route being regular if you look at it, yeah, it maybe rules out the first day, where people aren't slouched looking at their phones, but not the first week.
On the other hand, the point about the small town New England feel and the sidewalks is pretty solid -- if you want a a sidewalk neighborhood in suburbia in NH then you do end up in a college town. The stone wall is rebuilt and not original, sure, but it's the sort of thing you see in new England a lot. Your job is to look at the case it makes and see what makes sense.
I was kind of doing this as a follow-up to the previous reasoning post, but I realize I did not pull enough of that in. So thanks for that.
Thank you. Can you provide a pointer to, or just, a definition of "reasoning pattern?" And can you speculate on the internal mechanisms by which recent GAIs are acquiring and applying them?