26/10/2026
Cracking Copilot: Why Most AI Testing Gets It Wrong
As organisations race to adopt AI, many are focusing on the technology itself while paying less attention to how it is evaluated.
One insight continues to emerge from our work with leaders:
The quality of an AI answer is often influenced as much by how we frame, test and challenge the problem as by the technology itself.
This was a recurring theme during Tirian's recent Critical Thinking in the Age of AI workshop with KPMG. The discussion quickly moved beyond tools and prompts to questions of judgement, assumptions, evidence, bias and decision-making.
As AI becomes increasingly embedded in organisational decision-making, the ability to critically evaluate AI outputs may prove more important than the ability to simply use AI.
Our latest article explores why testing AI is becoming a critical thinking capability, and why leaders need to look beyond simple model comparisons when assessing AI solutions.
🔗 https://andrewgaiagrant.substack.com/p/cracking-copilot-why-testing-ais?r=o972c
If every organisation has access to the same AI tools, will competitive advantage come from the technology... or from the quality of thinking used to challenge it?