The Leadership Problem Tech Organisations Refuse to Name
In most technology organisations — from cloud infrastructure teams to GDPR compliance departments — the person who ends up in charge is rarely the one who most deeply understands the domain. They are, almost without exception, the one who spoke first, interrupted most, and radiated certainty in every meeting. The confidence vs competence leadership gap is not a matter of opinion or bad luck. It is one of the most consistently documented findings in organisational psychology, and its consequences reach far beyond awkward management hierarchies into decisions that directly affect data governance, security architecture, and regulatory compliance.
You almost certainly know this person. They hold strong opinions on zero-trust network architecture despite having never configured one. They sit on AI ethics committees with no background in machine learning. They volunteer confidently in procurement discussions about GDPR-compliant cloud vendors despite having never read Article 28 of the Regulation. And they get promoted — faster, more reliably, and with more institutional enthusiasm than the quiet engineer three desks over who has actually read the documentation, tested the tools, and could answer the hard questions if anyone thought to ask them.
When that promoted person eventually reaches a level of responsibility that their actual abilities cannot support, the organisation is genuinely surprised. The research suggests it should not be.
What the Dunning-Kruger Research Actually Found — and What It Means for Technical Hiring
The foundational paper on this subject was published in 1999 by Dr Justin Kruger and Dr David Dunning at Cornell University, in the Journal of Personality and Social Psychology, under the title "Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments." Across four studies, testing participants on grammar, logical reasoning, and humour, Kruger and Dunning found that the worst-performing participants were the most confident in their own abilities.
The specific numbers are striking. Participants who scored in the bottom quartile — at roughly the twelfth percentile in objective performance — estimated themselves to be performing at the sixty-second percentile. They believed themselves more capable than approximately two thirds of their peers. In reality, they were outperformed by roughly seven eighths of the group.
The mechanism behind this, Kruger and Dunning argued, is structurally difficult to resolve. The metacognitive skill required to accurately evaluate your own performance at a task is the same skill required to perform the task well in the first place. A developer who lacks competence in security architecture also, on the same measurement, lacks the ability to recognise the vulnerabilities they are introducing. Their confidence is not despite their incompetence. It is a direct symptom of it. They cannot see what they cannot see — and that inability registers, subjectively, as certainty rather than as ignorance.
For IT decision makers and privacy professionals, this has a specific operational meaning. When you are evaluating a vendor's security claims, assessing a colleague's GDPR implementation, or hiring a data protection officer, the person who presents with the most confidence may be the one least equipped to identify what they are getting wrong. As Kruger and Dunning's research suggests, the most dangerous thing about low competence is not that it produces hesitation — it is that it produces the opposite.
Why Even Experienced Teams Cannot Tell Overconfidence from Genuine Expertise
The second half of the problem — the part that actually drives bad promotion decisions — was documented in a 2012 paper by Cameron Anderson, Sebastien Brion, Don Moore, and Jessica Kennedy, working across the Haas School of Business at Berkeley, IESE Business School at the University of Navarra, and what was then the Wharton School. Published in the Journal of Personality and Social Psychology under the title "A Status-Enhancement Account of Overconfidence," the paper ran six separate studies testing whether the confidence a person displays influences how competent observers believe them to be.
The finding was unambiguous. Participants who were more overconfident — who believed themselves to be more skilled than they actually were — were consistently rated as more competent by their peers. They received higher social status. They were listened to more carefully. Their contributions were weighted more heavily in group decisions.
The mechanism was not that overconfident people made better arguments. It was behavioural. Overconfident people speak first. They speak more frequently. They use more assertive body language. They express fewer hedges and qualifications. They interrupt. And observers, watching this pattern, read it as evidence of genuine underlying competence rather than as self-belief that has come untethered from the underlying ability. Anderson and colleagues called this the "behavioural signature" of overconfidence — a set of observable cues that the human brain has been conditioned to interpret as expertise.
"The most dangerous aspect of overconfidence in technical organisations is not the bad decision it produces in isolation — it is the institutional trust it generates over time, which makes those bad decisions progressively harder to challenge."
— Tomas Chamorro-Premuzic, Professor of Business Psychology, University College London and Columbia UniversityWhen the Anderson findings are combined with the Kruger and Dunning findings, the result is a self-reinforcing loop that points directly toward bad organisational outcomes. The least competent people are the most confident. The most confident people are read as the most competent. And the organisation, which cannot reliably distinguish between them, promotes them into roles that require exactly the judgment they lack.
For teams working in technical domains — cloud infrastructure, cybersecurity, data privacy compliance — this loop has consequences that go beyond office politics. A CTO who rose through overconfidence rather than technical depth may resist implementing zero-trust architecture because they do not fully understand why it matters. A data protection officer who presents confidently but has not deeply engaged with GDPR may approve vendor contracts that introduce serious compliance risk. The damage is not always visible immediately. But it accumulates.
How Confidence Bias Shapes Tech Leadership — and Distorts Security and Privacy Decisions
The most direct analysis of what this pattern does to organisational leadership was published in the Harvard Business Review in August 2013 by Tomas Chamorro-Premuzic, professor of business psychology at University College London and Columbia University, under the title "Why Do So Many Incompetent Men Become Leaders?" Chamorro-Premuzic argued, drawing on his own research and the surrounding literature, that the everyday inability of hiring managers and promotion committees to distinguish between confidence and competence has produced a systematic drift toward a leadership population that is, on average, less capable than the people they lead.
The traits that help someone rise into a leadership role — overconfidence, self-promotion, willingness to interrupt, comfort with dominance — are almost the exact opposite of the traits that predict performing well in the role once obtained. Which produces, at the level of the organisation, the pattern that every competent engineer recognises: the people who could most improve the organisation's technical decisions are not the ones making them.
Research by McKinsey & Company on leadership effectiveness has consistently found that organisations with stronger technical depth in leadership make better technology investment decisions and sustain lower rates of costly project failure — yet standard hiring and promotion frameworks rarely assess domain competence directly. Instead, they rely heavily on interview performance and peer impressions, both of which the Anderson research shows are highly susceptible to overconfidence bias.
| Trait | Helps you get promoted? | Predicts performing well in the role? |
|---|---|---|
| High confidence, few qualifications | ✅ Yes — strongly | ❌ No — negative correlation |
| Deep domain expertise | ⚠️ Weakly — often overlooked | ✅ Yes — strong positive predictor |
| Assertive body language and interrupting | ✅ Yes — peers rate as competent | ❌ No — unrelated to actual output quality |
| Careful hedging and accurate uncertainty | ❌ No — perceived as weak | ✅ Yes — marker of genuine calibration |
| Strong performance review scores | ✅ Yes | ⚠️ Partially — depends on how reviews are structured |
For organisations operating under GDPR, the NIS2 Directive, or emerging EU AI Act requirements, the stakes attached to leadership competence are not abstract. Regulatory frameworks in Europe now impose direct personal liability on senior management for data protection failures, cybersecurity incidents, and non-compliant AI deployments. A leadership team selected primarily for behavioural confidence rather than technical depth is one that is specifically ill-equipped to navigate these obligations — and one that is less likely, by the Kruger and Dunning analysis, to recognise the gap.
What Technically Rigorous Organisations Are Actually Doing Differently
The research does not end at diagnosis. A growing body of organisational behaviour literature, including work published in the Academy of Management Journal on structured decision-making and competence-based hiring, points to several practical interventions that reduce the influence of overconfidence bias on promotion and hiring outcomes.
The most effective approaches share a common principle: they replace subjective impression with objective measurement wherever possible. Structured interviews with domain-specific technical questions, scored against predetermined criteria, significantly reduce the impact of confident-sounding non-answers. Work sample tests — where candidates actually perform a representative task rather than discuss how they would approach it — are among the highest-validity predictors of job performance identified in the hiring literature, according to research compiled by Schmidt and Hunter in their widely-cited meta-analysis in Psychological Bulletin.
For privacy and security roles specifically, organisations such as the International Association of Privacy Professionals (IAPP) have long advocated for certification-based competence standards that create an independent baseline against which candidates can be assessed, separate from how they present in interviews. Several European tech organisations have begun pairing CIPP/E certification requirements with structured technical assessments before considering candidates for data protection leadership roles — a direct structural response to the confidence-competence gap the research describes.
