Critical Reasoning with AI: Worst Decade

This graph was circulating, being shared uncritically as evidence that things were worse than they had ever been.

I asked Claude to do a Toulmin analysis.

Do a Toulmin analysis of this graph as evidence that we are living in the worst era along with rebuttals.

I've been playing around with a lot of ways to ask Claude to do analysis but this is one of the most reliable I've found. In this case, it did not disappoint:

Let me break down this graph using the Toulmin method of argument analysis.

CLAIM: The present era (2020s) represents the worst time period compared to previous decades across multiple societal measures.

DATA (EVIDENCE):

- The graph shows eight metrics comparing the 1930s to 2020s

- All metrics show dramatic spikes in negative assessments for the 2020s

- Categories cover broad spectrum: political division, news reliability, community cohesion, morality, crime, family stability, work-life balance, and economy

WARRANT (REASONING):

If multiple independent measures of societal wellbeing show historic peaks in negative assessments, then we are living in an exceptionally troubled time.

BACKING:

- The metrics appear to be based on survey data asking people to compare different decades

- The measures cover both objective (crime, economy) and subjective (morality, happiness) factors

- The consistency across different measures suggests a robust pattern

REBUTTALS AND QUALIFIERS:

1. Present Bias:

- People tend to view current problems as more severe than historical ones

- The 1930s included the Great Depression, rise of fascism, and massive societal upheaval, yet respondents rate it as "better"

- Psychological phenomenon of "declinism" - tendency to view the present more negatively than the past

2. Measurement Issues:

- The graph doesn't show actual data about crime rates, economic indicators, etc.

- Instead shows subjective perceptions which may not match reality

- Modern media and social networks amplify negative news and perspectives

3. Missing Context:

- No data shown for intervening decades (1940s-2010s)

- Limited to comparison with just the 1930s

- Many objective measures (life expectancy, poverty rates, technological advancement) show improvement

4. Survey Methodology Questions:

- How were respondents selected?

- How were questions framed?

- Possible sampling biases or leading questions

5. Historical Amnesia:

- Many respondents likely have no direct experience of the 1930s

- Romanticization of the past while forgetting its problems

- No consideration of progress made in civil rights, medicine, technology

CONCLUSION:

While the data presents a compelling narrative of decline across multiple measures, the evidence is primarily based on subjective perceptions rather than objective measures. The stark contrast with the 1930s - an era of genuine global crisis - suggests these results may reflect present bias and declinism more than actual societal deterioration. A more balanced analysis would need to include objective metrics and broader historical context.

(Added note: Claude whiffs a bit on the “missing context” bullet and its methodological critique is a bit overblown, but otherwise this is pretty good)

I also did a follow-up:

Does this graph show that people are more unhappy now than in the past?

This is admittedly a question that reflects a nuanced understanding of the graph, but my assumption is at least some students reading the Toulmin analysis might be able to formulate it, perhaps in the context of a facilitated discussion.

Here is the result of that:

No, this graph doesn't actually show that people are more unhappy now than in the past. Here's why:

1. Measurement Issue

- The graph shows how many people TODAY think the 2020s represent "the least happy families" compared to the 1930s

- It's not measuring actual happiness levels from the 1930s vs now

- It's measuring current people's perceptions and opinions about different eras

2. Critical Data Problem

- We don't have happiness surveys from the 1930s to compare against

- The people answering this survey likely weren't alive in the 1930s

- They're making assumptions about happiness levels in an era they didn't experience

3. Historical Reality Check

- The 1930s included:

- The Great Depression

- Widespread poverty and unemployment

- Limited social safety nets

- Fewer civil rights and opportunities

- Lower life expectancy

- Higher infant mortality

- Less access to healthcare

4. Psychological Factors

- This graph may better illustrate psychological phenomena like:

- "Declinism" (tendency to think things are getting worse)

- Rosy retrospection (viewing the past more positively than it was)

- Present bias (current problems feel more intense than past ones)

- Availability bias (current unhappiness is more visible and memorable)

What the graph actually demonstrates is how people today perceive relative happiness levels across time periods - not actual historical happiness levels. It's more a measure of current attitudes about the present than a reliable comparison of happiness across eras.

(Added note: Again, Claude seems to think this is just a 1930s vs now comparison because of the weird split response in the survey, but otherwise this is good).

Having taught quantitative literacy for three years to undergraduates, I can tell you without AI assistance you would be lucky to have even one student in a class of 50 who spots this, and you would likely never have any student that could process this at this level. Could they get to this understanding while using AI as a reasoning aid, especially if they are given some generally usable ways to prompt it? Could some novices operate at a level above novice level?

These are open questions, but ones I think we need to explore, as the potential benefits could be substantial.