I get about 2,000 views on an average post here, and on the better ones I crack 10,000, maybe once a month or so.
Recently I started a YouTube project where I narrate solving verification and contextualization challenges using AI Mode, and the results have been a bit sobering.
Or they would be sobering figures if I hadn’t already gone through this with SIFT back in the day. Early on with SIFT I had incredibly good luck with my printed explanations and uptake on the video stuff was poor. But it was the small number of people watching the videos where I tried out different techniques who were actually bringing SIFT into classrooms, seeing what worked and what didn’t. They became a core group of people who changed the face of online information literacy instruction, not only in the U.S., but across the world.
So I’m going to keep doing these.
Here’s a little FAQ though for people wondering what these videos are.
Why am I making these?
Here’s the thing about online information literacy instruction. It’s fucking hard.
It really is. I’ve taught other things. I taught undergrads normal distribution, and the mechanics of base rate fallacy. I’ve taught courses on research methods to people who could not care less what an active control randomized trial was and why the active part matters. All these things are hard of course.
But online information literacy is hard in a different way. It’s all about the demos and the walkthroughs. You need to be able to get up in front of a group of students and with 40% of your mind on the students, 40% of your mind on the example you’re working, and 40% of your mind on the lesson and skills you’re trying to transmit and if you add that up its impossible. You’re up in front of a class popping in a Google search and trying to read and skim and synthesize a set of search results that is somehow completely different than when you tried the same demo last night, and bring that back to the lesson.
What you learn when you do this is that there is a sort of “flow” to approaching a given skill in the classroom. You’ve got to hit a middle ground between looseness and tightness. And what you learn when you try to do that is that the ability to be loose is a function of knowing lots of little things that let you see the opportunity in the examples.
Take the 3I/Atlas example in the set. I didn’t know where that was going to go, but when I ran it, I saw this quote
And thought, hey, “nonsense on stilts”. maybe we could track that down. That led to an intermediate source (USA Today), and reading quickly I didn’t notice at first that the article pointed me to the original source being Live Science:
But when I did notice that after a bit of fumbling, I tried this approach, just asking if AI Mode could get me that link:
It did…
And we verified that quote:
In this case I found something surprising to me — it’s actually an even stronger condemnation of the theory in context. Which is a kind of neat reesult.
I could write a 200 page book on how to use LLMs to get context on things, and put every last little trick I’ve picked up in it for people to read. But I think text often makes things look very planned and methodical, and it doesn’t always stick. What text misses is a sort of vibe to the approach. “Can you get me to the original Live Science article” isn’t some magic sequence of words. It’s what I describe as an evidence-focused follow-up, which is less a recipe and more a way of thinking about interacting with the system.
You can get into that groove, and if you see enough examples of that approach, you can react in the moment and teach this stuff. And it’s my belief that it’s far easier to get in that groove by watching videos then running through the examples than reading text. Maybe I’m wrong. But I feel knowing what working through an example looks like is incredibly useful, and not always something text captures well.
Am I seeing these examples for the first time?
One question that comes up a lot when I do these is whether I am seeing these examples for the first time or if I have run through them before. The answer is — it depends.
Sometimes I have run them before, and sometimes I haven’t. A lot of times I have run them once before and I find I am getting different results back so it’s kind of a new example anyway. In any case, I use the common teaching method of trying to capture the thinking process of a person working through it and displaying that thought process to the audience whether it is my first time or not.
Often, because of the cognitive demands of demoing and explaining at the same time, you can do that better if you’ve been through it once or twice before. Every teacher knows this. I actually feel there’s a bit of a sweet spot where you’ve run an example a couple times and it’s still fresh enough that you remember what the struggle is but known to you enough that people aren’t waiting around while you read large swaths of text.
Incidentally, one thing I find sometimes in the comments is people complaining that I know something about the result and that’s why I was able to take a certain path. That might be true, though I would counter my single greatest ability — from the early days of designing eLearning in the 1990s to what I do now — has been that I have a gift of being able to put myself in the seat of a novice when it comes to developing these materials.
But my bigger point is this — that’s not what these videos are about. The question of whether a student adopting these methods makes them better at navigating an issue is not going to be solved by me turning myself into anthropological experiment on camera. It’s ultimately going to be solved by funded research that compares students adopting these methods to an active control group. If you want to know if these work, please push people to fund pilots. They aren’t cheap, but the bang for the buck is incredibly good.
All this being said, one of the interesting elements of working with AI is the answer coming back is always new and different, so in a way I’m always reacting in real time. What I say in the section above, for example, is quite true — in the 3I/Atlas demo I decided in the moment to try to track down the quote. I’d gone a different way with that example the couple of times I’d done it before. It’s cool that the non-deterministic output of AI keeps demos fresh! But in general, please don’t insult the videos and the work I put into them by treating them as some Big Brother game show. That’s not what they are. I absolutely discover some techniques in the moment, which is cool, but that’s not the point.
Why AI Mode?
Honestly I don’t know if I’ll stay with AI Mode. We’ll see. I’m interested in it partially because it seems like no one is focusing on it. What I’ve seen online from others (particularly on LinkedIn) is 800,000 people hawking their services to teach people to design complex prompts for different LLMs. I think that’s useful, even if the space is overcrowded and has a lot of hucksterism in it.
But if the past is any guide, the majority of people are not going to start their information seeking journey by designing a lengthy prompt or even firing up a custom project in Claude that does deep dives into things. They are going to do what decades of habit formation has trained them to do which is put something simple into a textbox or image upload area and hit enter.
I’m no stranger to writing complex prompts that get excellent results. But I also know that one of the most common uses of Google is people typing “amazon” into the search bar to go to Amazon. There is a very simple bookmarking function in every browser that has been there since the 1990s and yet still somehow people can’t be bothered to take ten seconds to make bookmarks to the top five sites they go to.
Maybe I’m wrong, but I think that behavior will persist, not just in AI Mode but with other platforms — as much as using a variety of tools would be helpful, people tend to gravitate to common, low-friction starting points. We saw this with SIFT too — my early passes at online information literacy pushed people to a host of specialized tools and syntax. It didn’t work. The minute someone thinks “What tool should I use to do this?” is the minute most people decide not to do it.
The methods that stick are ones that have low friction and supply tools to the person at the point they have their informational need. So my question becomes how can we take that low-friction process level it up a bit. That’s the idea of Get it in, track it down, follow up.
I feel like there are people who are doing online information literacy as “prompt engineering” and others that are doing it as “look out for these pitfalls!” (hallucinations, bad summaries, stochastic parrot analogies). But there is a void in this area of how students can work with the back-and-forth of this in a productive way that is a positive vision,1 accessible to all. I aim to fill that.
Anyway, check out the playlist, and I’ll try to keep adding walkthroughs to it.
By positive vision I don’t mean shilling. I mean that there is a negative vision (don’t do this!) and a positive vision (try this to get what you need!) and those two things have to be in balance.