Reading the Room with SIFT Toolbox

It's not perfect, but it will ease you into a scholarly conversation

In our book Verified, my co-author Sam Wineburg and I talk about the talent of "Reading the Room" -- which is a way to try to understand the structure of the professional or scholarly discourse you are entering before you start to engage with it using search.

I've been playing around with SIFT Toolbox (the extended prompt I have developed) as a tool for Reading the Room. It works surprisingly well! I think it could provide a useful tool for students to bootstrap themselves into a scholarly discussion. Here's an experiment I ran tonight:

https://checkplease.neocities.org/sift-toolbox/azar-gat-claims/

I imagine this as a starting point for getting into a topic -- you don't have to start with a tweet of course, you can put in any sourced scholarly (or unscholarly) claim!

Readable text is all at the link above, but here are some screenshots (This is not the whole page, just some of it).

Everything gets linked and sourced, the summaries are good. And yes, maybe you don't like Pinker -- but the point is to lay out the discourse space for a student, not to come to a conclusion.

Claude did this "Analysis of perspectives" spontaneously (all the other stuff on the page was meticulously detailed by me in the mega-prompt). It was a nice surprise!

Is it perfect? Does an expert look at this and go "precisely!"?

I don't know. I've gotten expert eyes on these for other topics and generally experts think they are pretty good. But the idea is not precision anyway, any more than the idea of a Google result is that it be a perfectly written intro to a subject.

The idea is to give a person enough of a starting point that they can begin to make sense of a space that is overwhelming to swallow all at once. People don't get this. It doesn't have to be as good as an expert! It has to be better than alternative reading approaches.

The educational question is different from the “publishing” question. I don’t know if people get that. The question is whether a student that approaches a subject using this tool gains a better understanding than the student approaching the subject another way. In that framework some errors can be quite bad (forming poor initial assessments that lead to misunderstandings) and some errors don’t matter much at all.

LLMs are a cultural technology that people use to explore cultural products and disciplinary ways of thinking. Products like this can be thought of as discourse maps, and you can think of them in the same way you think of a subway map. Not “is this a perfect representation” but does it get the reader where they need to go. Not “what percent accurate is this?” but is “is this fit for purpose?”

Does it get you to your destination or get you lost? I’m biased, but I think the map the Toolbox creates is going to get student at least partway to their destination.

Prompt to run it is here. Works in Claude 3.7 Sonnet.