(Just need the example prompts? Click here. For prompt code, click here.)
One thing I have struggled with in SIFT Toolbox for Claude is its tendency to guess when it doesn’t quite know. The problem is that you can use language to turn down its tendency to guess, but then you often miss important information. Turn the tendency to guess up, and suddenly you have the opposite problem, with the Toolbox making large leaps. I could never get it right. In fact, sometimes when I told it not to guess it would end up even more credulous.
A couple days ago I stumbled on a solution. Instead of trying to set the right credulity level I added a section called “Potential Leads”. And that’s about it, really, just added a section to put potential leads into.
And that… seemed to work? It’s not perfect, but the system having a place to put things it is not sure about without having to decide on them has solved a lot of the problems I was seeing. You can use the Potential Leads section to follow up, well, “potential leads”. It’s a place where the tool can park an assessment like “I think the photo here might be from this event but I’m not sure,” etc.
Of course, this tends to build the solution around the sorts of things I use it for, where we are looking at poor contextualized information online and trying to contextualize it. But after hammering on the new revision for a couple days (I know how to party on my vacation) I’m confident its an improvement and am releasing it. As usual the prompt language is below, followed by a couple test prompts. You can use text prompts too — they are actually far easier for the system to process. I put the screenshot prompts because they tend to be more complex.
Also, Anna Mills did a recent test of the code in Gemini 2.5 with some good results, so you might try it there. I had been disappointed with the way the code performed in the previous version of Gemini, but the new version looks very promising! Let me know how that works if you try it.
Instructions Text
The most recent instructions can be found on the SIFT Toolbox site.
Sample prompts
Jabbari Claim (For research purposes only)
Dead Loop Claim (For research purposes only)
Fauci Net Worth Claim (For research purposes only)
Rocket-Looking Thing (This one goes badly wrong — common problem with LLM “vision”)
Alexa privacy (For research purposes only)
Pope Lambo (For research purposes only)
Dye Phase-Out (for research purposes only)
Trump on Crimea/Ukraine (for research purposes only)
CT Income (Research purposes only)
I love the idea of creating a section for the chatbot to put more tentative ideas! I guess it's more or less accurate about its degree of certainty...
I guess there's no straightforward way of making a custom bot out of this? I think the prompt is too long for ChatGPT GPTs, and the performance is not as good. Maybe Poe.com? Oh, I just saw that Marc Watkins made a custom Gem.
This is useful! I made a custom Gem, but I don't seem to have a share button for sharing the Gem itself. But here's a link to a result from my Gem: https://g.co/gemini/share/f552e8a6c5dc