When AI can outthink the expert, who decides what’s true?

“We’re right at the end of human beings’ capacity to say we’re better than it is.”

Shelly Palmer knows the line is provocative. He thinks boards need to hear it.

He argues that the most advanced models can draw on more information, process it faster and identify connections no director, executive or subject-matter expert could hope to replicate alone.  

“What happens when the model gives you an answer and you don’t know if it is new information you’ve never seen before or whether it is hallucinating?”

Palmer has already encountered that problem in a field he knows intimately.

Before becoming an adviser to Fortune 500 companies, chief executive of The Palmer Group and a director of NASDAQ-listed 1-800-Flowers.com, he began his career as a composer and producer. His parents were music educators. He has about 5,000 pieces of music in his catalogue and decades of experience with the theory and technology behind it.

During a late-night argument with fellow musicians about functional harmony – how a chord operates within the context of a piece – the group put the question to several AI models. Most returned familiar interpretations. One produced an answer none of them had encountered.

It stopped the argument.

Palmer and his friends could not tell whether the model had found something genuinely new or simply made it up. They contacted other professional musicians and academics. A text thread grew to around 40 people. Two weeks later, they still had no definitive answer.

“We are the guys you call,” he says. “And the guys you call don’t know.”

Boards can rely on experience, judgement and knowing when to seek specialist advice. What happens when the specialist cannot decide whether an AI-generated insight is brilliant or false?

“We’re not saying it’s better,” Palmer says. “It’s got access to information and it’s got a mixture of experts and reasoning that’s beyond your capacity as a human to have read all the things it’s read and synthesised all the things it’s synthesised.”

Given reliable, grounded data, a model may find patterns people cannot, Palmer says.  

“Then you have to figure out whether you believe it. That’s the harder problem. And it’s not the problem people are trying to solve for right now.

“I don’t fear artificial intelligence. I fear artificial control.”

People already outsource wayfinding to mapping software, memory to devices and search to platforms. They trust the suggested route is the fastest one without knowing why it was selected. With AI, users hand over something more revealing than keywords – context, intention and meaning.

A person asking for a recipe is no longer simply searching for “chicken enchiladas”. They may tell the system they are hosting 10 people that evening, what they like to eat and what kind of event they are planning. The model receives a semantic picture of the person, not just a search term.

“How much influence do these tools have over us that we are submitting to by giving them our data?” Palmer asks.  

That is one of the questions he will bring to the Institute of Directors Leadership Conference in September. His keynote, AI and the boardroom: Governing through rapid technological change, will give directors ways to think about the technology rather than a tidy set of predictions or instructions.

“You’re a subject-matter expert. You’ve got all your priors. You’re bringing knowledge into this conversation. I’m going to give you some very clear frameworks for how to think about AI.”

Defence contractors, retailers, banks and not-for-profits do not face the same exposure, hold the same data or need the same capability. Palmer wants each board to work out its own position rather than wait for a universal answer.

For decades, Palmer argues, the basic unit of business has been the deliverable – the report, presentation, analysis, campaign, plan or recommendation. Meetings are held to commission them, review them and prepare the next one.

He expects much of that work to become cheap and immediate.  

“What do you do now that every single human who works for you presses a button and gets a fit-for-purpose deliverable in 90 seconds?” Palmer asks.

“The atomic unit of business has to shift to the decision.”

That shift would leave employees hired to execute someone else’s vision spending less time producing and more time judging. Palmer wants boards to ask whether those people are ready.

“Are they good decision-makers? Are you able to have a Socratic debate? Are you able to have a dialectic discussion? Is there someone who can play devil’s advocate to a deliverable?”

Palmer says most organisations have trained people to make things, not to interrogate the assumptions behind them. AI may expose that gap quickly.  

His shorthand is blunt: “Technical debt is meaningless in 2026 and beyond. The high hill to climb is cultural debt.

“You throw money and engineers at a problem, they will solve the technical problem,” he says.  

Inside organisations, Palmer encounters a deeply conflicted response. People may be frightened of AI while feeling pressure to present themselves as confident users of it. For experienced professionals, the threat is not only to their role. It is to their sense of who they are.

Palmer calls it “identity threat” – the moment a capable person recognises that the machine has produced a better answer.

“Agents work better for people with agency,” he says.

“If you know what you're looking for, you’re going to get amplified. If you don’t know what you’re looking for, you’re going to get snookered.”

He wants directors to study the technology themselves and get direct experience of using it.

Palmer intends to end his conference address with a challenge: “Roll up your sleeves, get your hands on the keyboard and lead from the front, as opposed to talking like you know something.”  

The IoD’s National Leadership Conference brings together more than 500 directors and business leaders for two days of governance discussion and networking on 3–4 September at the Tākina Convention and Events Centre in Wellington.  Local and international speakers will discuss governance and leadership issues affecting Aotearoa New Zealand and the wider world.  Register here.