Part 2 of 5 in a blog series looking at AI’s journey since the release of ChatGPT in November 2022.

After November 2022, many people who normally acted with care and caution seemed to lose the plot.

It was clear that Large Language Models (LLMs) were often misleading or just plain wrong. The ChatGPT user interface, like others later on, said so right up front. And there was no shortage of social media posts illustrating how an LLM had confidently invented a biography, fact, or citation – what came to be called LLM hallucinations.

And then everyone carried on using them anyway – not just for fun, but in the workplace. Why?

When a new tool saves you time, it offers more than convenience. It offers relief, from the cognitive burden you carry all day, every day. It offers to make you faster, more articulate, more competent, more productive. All this for free, at a time when the modern world is relentlessly making you ever more overloaded, interrupted, and anxious.

So we all started using LLMs to summarise documents and draft new ones. Then to help us make decisions, at every level from forming company strategy to deciding on new hires. Human judgement dies not through ignorance, but through exhaustion.

Plausible is not the same as true

An LLM in its raw form doesn’t generate truth, but patterned plausibility. It creates something that sound right, without any internal mechanism to say whether it’s true or not. With equal confidence, an LLM will  compose a fluent essay about Shakespeare and cite non-existent legal cases or statistics. But fluent doesn’t mean reliable. Years on from ChatGPT’s public debut, hallucinations are as prevalent as ever. A major study in October 2025 showed that almost half of all AI answers contain errors and a third have missing, misleading or incorrect attributions.

Startlingly, attorneys have repeatedly been accused of citing fabricated legal authorities in official brief. In a Connecticut case in early 2026, a lawyer was accused of including over a dozen non-existent cases in a motion, then amending it only to include further inaccuracies. In similar UK and US federal filings lawyers apologised for including AI-generated citations that didn’t exist at all. These aren’t interns but qualified legal practitioners expected to verify facts before they appear in front of judges. Courts globally are now moving on from warnings to imposing significant sanctions for AI hallucinations, primarily involving fabricated case law, non-existent statutes, and fake quotes in legal filings.

Other professional associations around the world are now debating standards and policies for AI use in research and drafting. Regulatory attention is increasing. News reports now routinely highlight AI errors in healthcare, public health information, and legal domains. Inaccurate AI health summaries, for example, could dangerously mislead millions.

But the underlying problem remains. LLMs still hallucinate, providing confident but incorrect responses without transparent links to verifiable evidence. Yet studies continue to find high error rates across AI-produced material in critical domains such as news, law, research, and policy. Why do we continue to trust LLMs?

The psychology of misplaced trust

The psychological dynamic is deep. Humans are almost always cognitively overloaded – certainly in the modern workplace. Further, we respect fluency and instinctively trust prose that sound authoritative. When an AI gives a concise, well-structured answer, many of us feel, emotionally, that we don’t need to verify every fact – especially if the answer conforms to our prior expectations and prejudices, which LLM responses are designed to do. As every successful politician knows, this is the basis of rhetoric.

And as many politicians learn to their dismay, the rubber hits the road when smooth and plausible statements are used as the basis for professional decisions. That’s when the system breaks and careers suddenly come crashing to an end.

The silent surrender of critical thinking

The real shift here isn’t about truth. People, like LLMs, always have and made mistakes. The fundamental change is in accountability. People across the world are starting to believe that an LLM has greater authority than they do personally, ceasing to take responsibility for their decisions, and losing sight of the fact that they are still accountable for the consequences. The world has already seen how, whatever happens to ChatGPT, Sam Altman won’t lose his job. But you might.

For the foreseeable future, the real threat of AI isn’t runaway superintelligence taking over the world. It’s our willingness to outsource human judgment to tools that have only the veneer of authority – and none of the accountability.

To find out more about why we invented Dedoctive, keep reading this blog series.

Author

Keith Harrison-Broninski is an author, researcher, and keynote speaker specialising in cross-boundary collaboration, community antifragility, and technology for good. Keith’s awards include from Gartner for social enterprise and from the NHS for technology to replicate healthcare innovations.

Keith’s first book “Human Interactions” (2005) was described by Information Age as “the overarching framework for 21st century business technology”. Keith’s other books include two for Springer and three for the Workflow Management coalition. Keith’s most recent book “Supercommunities” (2021) was described by the Chief Executive of the RSA in his foreword as “Ranging from ancient history to economics to psychology to public policy ‘Supercommunities’ is both authoritative and highly readable. It puts our current challenges in context, shows why change is necessary and provides a trove of practical ideas for change makers.”

Vint Cerf, co-inventor of the Internet, in a second foreword, wrote “Supercommunities offers a path away from social and economic meltdown … We will need to replace short-term thinking with long term planning and execution if we are to regain upward motion towards common benefit for everyone on Spaceship Earth. To begin, read this book!”

Keith currently focuses on creating the Internet of Communities via his company’s revolutionary technology “Dedoctive” – a fundamental AI innovation, academically-validated for the MoD as fully trustworthy, that empowers reliable access to complex information for all.

Keith is also a jazz pianist, traditional folk musician, and classical composer. He has released 9 albums and performs regularly with a wide range of other musicians.

More information

Email Keith Harrison-Broninski: khb@dedoctive.ai