Know the AI thinking traps (Part 1)

In an incredibly short time period, AI has become a routine part of daily life for many. No one can say just what the impact will be, but we do know it can create certain thinking or cognitive traps that have the potential to affect how we think.

This is especially true for young people who are still developing their cognitive capacities.

The following is not an exhaustive list, but a selection of relevant and known problems that can occur when we rely on AI to do too much of our thinking - or start to trust it more than ourselves.

Some of these concepts pre-date AI, but are taking on new life in this era. Others are wholly new phenomena that are emerging alongside the technology. All of them are worth understanding, watching out for and sharing with students of any age.

1. Cognitive offloading

What it is: Handing our thinking over to a tool. Humans have always done this – we use calculators, search engines, GPS. AI now allows us to offload far more complex forms of thinking.

Potential outcome: Cognitive muscles atrophy, affecting our ability to adequately develop judgment, discernment, complex reasoning, the ability to comprehend longer-form and/or complex text.

Ideas for what to do: Intentionally carve out time for human-only thinking and commit to the struggle of figuring out, writing and/or looking up something on your own. Even if it's just for 10 minutes, even if it's something you are only doing for the sole purpose of counteracting this effect.

2. Automation bias

What it is: The tendency to trust that what a machine says is right. This behaviour becomes easier to fall into given how authoritative and confident AI-generated answers can sound. We may start to accept those answers over our own judgment, knowledge or even over explicit evidence that says something different.

Potential outcome: Healthy intellectual skepticism is turned off and we become more intellectually passive and pliable.

Ideas for what to do: Vigilantly remind yourself that the machine can be wrong and double check key facts before you accept them. For students, you can proactively have an AI generate a summary with errors and have them practice fact-checking.

3. Intellectual helplessness

What it is: In the context of AI, this phenomenon takes hold when we lose the resilience required to struggle through and solve complex challenges because we’ve become used to instant and friction-free answers.

Potential outcome: We give up when tasks require a modicum of brain effort, self-infantilizing our intellectual development.

Ideas for what to do: You can lean into hands-on activities where you can see the fruits of your labour - cook a meal using a difficult recipe, learn to embroider, build a structure with blocks, grow a plant from seed. Look for tasks where accomplishment is a reward.

4. Epistemic dependence

What it is: A person has epistemic agency when they believe in their own capacity to acquire knowledge and verify facts. When we begin to view AI as an omniscient all-knowing oracle, we can lose that agency.

Potential outcome: From politics to science to history, our sense of what is true is shaped by an AI, which not incidentally is corporate-owned.

Ideas for what to do: For students, introduce them to different sources of knowledge, from credible websites, to organizations, to local newspapers. As well, if you're using AI and have access to different AIs, show them how AIs often disagree with each other - it's the ultimate proof that an AI is not infallible. 

5. Illusion of competence

What it is: When we mistake our access to summarized answers for actually possessing the knowledge that is contained or explained in the answer. In other words, we read an AI-generated blurb and think we are an expert.

Potential outcome: Satisfied with reading an AI summary, we no longer feel it necessary to dig any further – and then we make decisions or seek to persuade others based on something we don’t truly understand. This is very prevalent in social media.

Ideas for what to do: First and foremost, read books! But for a fun activity, you can create a role-playing activity where one student reads an AI-generated summary of a topic and pretends to be an expert - and then students ask questions that expose the illusion of competence. Or simply provide an AI-generated summary of a topic and have students list questions that the summary does not asnwer.

6. Map-territory confusion

What it is: When we look at an AI-generated summary and forget that the real world is infinitely bigger, messier and more complicated than the text on the screen, which was generated through a mathematical process. We mistake the AI’s explanation for actual experience and knowledge of what it is describing.

Potential outcome: We lose our desire for details, nuance, context and perspective seeking. We lose confidence in our own eyes, memories and experiences. We forget to look beyond the slop.

Ideas for what to do: Provide experiences of the real world.

• • Read an AI-generated description of a leaf or a tree or grass, etc. Then go outside and experience and observe these yourselves. Note the differences, and/or write your own more fulsome descriptions in which you draw from that experience. 
  • You can also read an AI review of, for example, a local cafe, then go there and experience it yourself - again making sure to note the differences. This isn't to show that AI is wrong, it's to underscore the importance of real world experiences. 
  • Also, discover oral history, read short stories, interact with people.

7. Algorithmic homogenization

What it is: The danger that everyone’s ideas, writing and creative work start sounding exactly the same. Because AI is trained to blend together millions of existing web pages, it tends to spit out predictable and middle-of-the-road answers.

Potential outcome: Original thinking is completely flattened out. We lose individual voices, and unique perspectives. Daring, original and wonderfully weird ideas dissipate because everyone is using the exact same machine to do their thinking for them. Everything just becomes sort of average and “normal.”

Ideas for what to do: Read fiction and poetry. Provide ample (and rubric-free) opportunities for creative writing, both fiction and non-fiction. Revel in language. 

8. Automation complacency

What it is: When we stop paying close attention to a machine’s work because we assume it is always working correctly. While automation bias is about believing the machine is right, complacency is worse. It’s about no longer bothering to check at all.

Potential outcome: We lose the habit – and eventually the skill – of doing our best to ensure the information we pass on is correct. We become purveyors of misinformation, unintentionally yet casually spreading major mistakes, incorrect AI facts (hallucinations) and perspectives grounded in falsehoods.

Ideas for what to do: Research instances of AI error or misbehaviour. There are ample. 

9. Loss of cognitive sovereignty

What it is: When we begin relying on AI to determine what we ourselves actually think, value or believe. Instead of using AI as an extra tool, we start outsourcing our inner decision-making and morals to an algorithm.

Potential outcome: We lose confidence in our own minds. We become unable to form independent opinions, make judgments or decide what matters to us without first consulting an AI system to tell us how to feel. We infantilize ourselves, which leaves us open to greater manipulation by others.

Ideas for what to do: Read short stories or novels that explore this theme, from Ray Bradbury to Ursula K. Le Guin, to George Orwell. Authors writing in the 20th-century and who witnessed the rise of totalitarianism and technology, speculated on the future through captivating and eerily prescient science fiction. (Look for a post on this in the future.)

10. Algorithmic herd behaviour

What it is: The digital version of peer pressure, where groups of people get funnelled into thinking, buying and believing the exact same things because they are all being fed content by the same recommendation engines. Instead of humans organically sharing ideas, an algorithm rounds us up into massive, predictable cultural herds. 

Potential outcome: Societies and culture lose their vibrancy because the algorithms generate - and reward - conformity and compliance. Plus, it opens the door to fascism.

Ideas for what to do: Celebrate local shops and small businesses. (No this is not a shameless plug 😁). But be conscious of this phenomenon and its consequences, and act against it when you can by stepping outside the herd. Discover different people to follow on social media, embrace local news, try new products, write your own vacation itinerary, value your opinions but be open to evolving them, embrace your differences. 

A last word for educators & parents

Although I've provided a list of ideas for what to do to counteract the above AI risks, an overall approach is to outsource it to young people themselves. You can provide the thinking trap and then have them come up with ideas for how to counteract it. That not only puts the thinking on them, it builds up their agency, critical thinking and imagination at the same time.

>> As well, activities that build up critical and creative thinking and conversation are fundamental to counteracting the potential negative effects of AI over-reliance. Check out the Think, Talk, Imagine bundle and/or the Think Like a Storyteller bundle - both are made to enliven and energize young minds.