I hear this a lot - and to me it sounds like pushback against how Large Language Models (LLMs) keep making advancements. But here is a different perspective: Maybe it’s less about LLMs being “smart” in a human sense and more about the fact that we vastly overestimate how clever our own brains actually are.

To understand this, we can just go look at human psychology:

Imagine a study where people are asked to participate in a “guess the price” experiment. They enter a room, provide the last two digits of their social security number, and are then shown common household goods. They are asked to estimate the cost of these items.

The results are remarkably consistent. People whose last two digits are higher (50-99) consistently make higher cost estimates than those with lower digits (00-49).

This is essentially a "hack" on the human brain, a predictable cognitive shortcut (anchoring effect ). In a similar way, LLMs can be "tricked" by priming them with specific information in their context window. We think we are interacting with a complex reasoning engine, but often we are just interacting with a system that is highly sensitive to the "anchors" provided in the prompt.

The reality is that there are many such well-known phenomena in psychology and neuropsychology where our brains act much more “simple” and predictable than we would like to believe. We like to think of our consciousness as this impenetrable fortress of complex reasoning, but much of it operates on heuristics and biases that can be bypassed with the right “trick”.

[Sidenote: You should definitely use the anchoring effect in salary negotiations btw. It’s an actual practical application of a brain “glitch”.]

Another example: The primacy effect ensures that the first information we encounter carries the most weight. Similarly, the recency effect means that the most recent information tends to dominate our judgment. Our brains are essentially optimizing for speed and ease of processing, often at the expense of accuracy. We are hardwired to be influenced by the "edges" of our experience.

I have a psychologist in my immediate family - and the similarity between well-known cognitive biases in humans and LLM shortcomings is fascinating to watch.