The Intelligence of Failure

What breakdown reveals about behaviour, institutions and change

Failure, it seems, can be a good thing: digital culture has talked about the value of failing fast, as failure is necessary for innovation. Resilience in the face of failure is praised, with universities and workplaces encouraging people to think about setbacks as positive lessons. So, failure, it seems, is not something to be embarrassed about or hidden but something useful.

In many ways this seems preferable to older, more punishing cultures where failure was treated simply as shameful, a case of incompetence or moral weakness. But the newer language surely comes with its own problems. And first amongst these is the way that, while failure is acceptable, this is only so if it can be rapidly turned into future success, with the failure simply used as evidence of agility, self-improvement and innovation. In other words, failure cannot remain failure for very long. And while the failed start-up founder who later succeeds can redescribe earlier failure as formative, this rear-view account surely inflates the merits of failure and downplays its pain. Which means failure is turned into a staging post in a much wider success story, so we are no longer talking about failure in its own right.

But can failure always be considered a lesson? Sometimes, maybe, there is not much to learn from our failures, and there are no redeeming outcomes. And if we always insist that failure is useful for us, then perhaps this is simply a way of refusing to look at it properly.

To be human is to fail – all of us in some way fail at different points in different ways. There is a rich literature written by those who have dwelt on failure, from Charlie Chaplin and Leo Tolstoy to Arthur Miller and George Orwell. All of these have written about how failure is an integral yet under-theorised part of life, offering valuable lessons in humility about the limitations of our power and place in the world.

The question, then, is what failure reveals before it is converted into anything else and all too quickly forgotten about.

Failure is not the same as disappointment

To better understand failure, perhaps we can consider what it is not. Disappointment might, at first glance, seem similar to failure: it begins with an expectation of how life should be, which has not been met, and of how something better might have happened. Failure goes a step further than this: beyond unmet expectations, it shows those expectations to be misguided or wrong, based on a misinformed view.

Hence, a disappointed person may say: the future I expected and reasonably hoped for did not arrive. By contrast, failure says something more challenging: the model that made our imagined future seem plausible in the first place now no longer works. This is why failure is perhaps more than the emotion of disappointment, telling us something important, whether about a system, a category, an institution, a script, or a form of life that no longer works.

This is philosopher Costică Brădățan’s key point, when he writes about the importance of resisting the temptation to make failure cosy. He sets out the way that failure is a clash with the limits of our agency - which does not always produce a lesson or reveal a better path. Sometimes failure simply shows us our precariousness in the face of a world that is not as manageable as we had assumed.

This is where failure becomes valuable, because it contains information that we otherwise might not see - success tends to confirm the existing model; failure tests it. And while success allows us to keep telling the same story about ourselves, failure tells the (surely more important) lesson of showing where the story no longer holds.

Experiencing real failure, in this sense, is therefore a hidden asset: hidden because shame, stigma and defensiveness make it difficult to examine and an asset because it shows us what success often conceals - the assumptions people are living within, the constraints they are navigating, and the systems that shape what action is possible. One of these concerns the limitations of many organisations.

How institutions can create failure

The way organisations can manifest failure is the topic of psychologist Norman Dixon’s book, ‘On the Psychology of Military Incompetence’. Having served in the army for ten years during World War II before becoming an academic psychologist, he was actually less interested in arrogant commanders or lost battles; instead, he asked how organisational cultures in the military could make certain kinds of failure more likely. This is despite many of the people inside them being individually intelligent, diligent and seemingly competent.

While Dixon’s setting is the military world, we can see how this applies more broadly. Research suggests that, within organisations, confidence is often treated as a cue to credibility, even though it is not the same as accuracy. Studies of leadership selection similarly suggest that overconfident individuals may be seen as more leader-like, not because they necessarily have better judgement, but because confidence itself is read as evidence of authority. At the same time, research on employee voice and organisational silence shows how doubt, dissent and caution can be suppressed when people believe that speaking up is risky, unwelcome or futile.

Doubt, in these settings, is not read as a form of intelligence or care but a sign of weakness, uncertainty or lack of command. We can therefore see how easily organisations can end up selecting for the very qualities that make failure all the more likely.

One of Dixon’s examples of how these pitfalls can play out, in the military at least, is the fall of Singapore in 1942. The episode is often remembered as a significant military humiliation: British forces, under the command of General Arthur Percival, were defeated by the Japanese advance despite Singapore’s status as a key imperial stronghold. Dixon’s concern, however, is not simply with the final collapse, but with the organisational conditions that made such a collapse possible. The failure lay partly in a strategic imagination that had become too narrow – based in no small part on over-confidence and the downplaying of doubt.

British planning had long treated Singapore as a fortress whose principal danger would come from the sea, while giving insufficient weight to the possibility of a landward advance through Malaya. Japanese capabilities were underestimated; the jungle was treated as more of a barrier than it proved to be; and evidence that challenged these assumptions struggled to dislodge the confident, settled view. The problem was therefore not only that one commander made a poor decision at the end of the process. It was that the wider system had narrowed what could be seen, said and regarded as credible, as there was little room for doubt to be expressed.

Dixon’s challenge is that failure should be shifted away from the people at the end of the line who enact the failure, in this case the commanders, and towards the wider conditions that make bad decisions possible. Military failure is not only the absence of competence, but evidence of how an institution thinks. The disastrous decision is only the visible endpoint of a more deeply ingrained dysfunction.

Time-shifted failure

If Dixon helps us see that failure is often produced long before it becomes visible, there is a related form of failure that appears in another temporal location. In the Singapore case, failure was made in advance through planning assumptions, institutional confidence and a failure to take disconfirming evidence seriously. The collapse came later, but the conditions of collapse had already been assembled.

Something similar can happen in everyday behaviour, although in a different way, as some failures do not arrive after action, whether as a consequence of a poor decision, weak motivation or inadequate planning. They can arrive before action properly begins, when the future attached to an action no longer feels credible enough to organise effort in the present.

To unpack a little, the ‘fail fast’ story we mentioned earlier assumes that people are already able to try: they act, fail, learn and try again. But this misses a subtler and perhaps more harmful form of failure. Anticipated failure can travel backwards from the imagined future, making present action feel risky, pointless or simply not for ‘people like me’.

In these cases, failure lies in the collapse of possibilities: the sense that a course of action could realistically lead somewhere. This matters because behaviour often depends on a plausible long-term future: we study or train because qualifications and new skills appear to lead somewhere, we save because later life feels real enough to prepare for. These actions require more than information, intention or discipline; they require a future that feels sufficiently credible to act towards.

This credibility is not created by the individual alone. Imagination is socially shaped, formed through family histories, peer worlds, institutions, media and the visible lives around them. Some futures are easy to imagine because they are repeatedly modelled and socially endorsed while others feel distant, risky or marked as belonging to different kinds of people. A young person may be told that education, training or work will open doors, but if the doors they have encountered have mostly stayed closed, the instruction to ‘believe in the future’ can begin to sound less encouraging.

This changes how we should understand failure. A young person who misses out on stable work at 19 or 22 does not simply lose income in the present. They may also lose identity, confidence, contacts and the everyday institutional knowledge through which adult life is assembled. The failure therefore does not stay in the moment. It travels forward, shaping what later opportunities feel like and whether effort seems likely to be repaid.

This is where the familiar language of aspiration can be misleading. Many people are not failing to imagine the future because they lack ambition. They are trying to imagine a future from within a life in which desired futures feel fragile or simply unavailable. The question is not simply whether people can imagine a better life, but whether the social world gives them enough material from which to imagine it credibly.

So, whilst we might treat future-oriented action as a problem of planning, time preference, or self-control, there is a danger that this overlooks the deeper problem of plausible futures. People may understand the advice and accept the goal but still struggle to act if the future attached to that action feels remote, insecure or not really theirs.

Failure, then, is not always a discrete setback from which a person can simply learn and recover. Sometimes it changes the conditions under which future action becomes possible. For those with fewer reserves, weaker networks or less institutional recognition, failure is not fast. Instead it lingers, accumulates and changes what the future can be made to mean.

There is also a more personal version of this time-shifting. People often protect themselves from failure by putting distance between who they are now and who they were then. We say things like ‘that was a different me’ or ‘I’ve moved on from that’, not only because time has passed, but because the failure feels easier to live with if it belongs to a former self. Temporal self-appraisal theory makes the point that people may feel subjectively further away from past experiences that reflect badly on them, and closer to past experiences that support a more positive view of who they are now. Failure, then, is not a simple event sitting neatly in the past: it is worked on psychologically, pushed away, reinterpreted or disowned, even while its practical consequences may continue to travel forward.

Failure and the fantasy of mastery

This brings us back to Brădățan’s point that failure is so often made acceptable only when it can be redeemed: when it becomes a lesson, a pivot, a useful scar, a moment of growth, or an early chapter in a later success story. But this is also a way of ‘domesticating’ failure and, in doing so, minimising it by allowing us to keep the basic fantasy intact. We can continue to think that the world is manageable, that agency is secure, that better knowledge will produce better control, and that every setback can eventually be folded back into progress.

And surely this shows up in the way that so many stories of failure are told in reverse. We are used to hearing about the musician or novelist who experiences vast numbers of rejections, the artist who fails to sell their paintings, the politician who did not achieve the position they wanted or the entrepreneur whose early ventures collapsed, but only because we already know where the story ends. There is later success, and then we are redeemed from our earlier failure; its meaning fundamentally changes. What might have been experienced at the time as shame and genuine defeat is converted into evidence of perseverance in the face of adversity. It’s in this way that the failed moment is no longer a failure but a test of character before eventual vindication.

These stories also fail to account for luck. They make success look like the person who eventually succeeds appears to have been moving all along towards a destination that was somehow waiting for them. But success is often far more uncertain than these stories allow. It can depend on timing, inherited advantage, social networks, institutional sponsorship, being noticed by the right person, entering a field at the right moment, or simply not being destroyed by an unlucky event that might have ended the story altogether. As Liu and de Rond argue in their review of luck in management scholarship, performance is often shaped by chance, context and attribution, even though success is routinely redescribed as skill, effort or judgement after the event.

This matters because luck means that while failure is made a personal failing, success is made deserved. The failed person is asked what they did wrong, what they failed to see, why they lacked resilience, discipline or adaptability. The successful person, meanwhile, is more easily granted depth, foresight and character. Their luck is swiftly absorbed into the story as if it had always been part of their innate talent.

And there are plenty of times where failure does not lead to eventual success – it is simply a failure. But there is value here as it creates in us a humility that forces us to better understand the world: we can see our dependence on others and the communities we live in, the extent to which we have limitations, how we can misplace our confidence, and see more acutely the fragile nature of the world we thought we had agency in.

It is against this backdrop that Brădățan suggests failure is less an unfortunate exception to an otherwise successful existence and, in fact, lies close to the centre of human life, stripping away the fictions we may have about the world, showing us where our confidence exceeded our understanding.

In a related point, Brădățan draws on philosopher Simone Weil’s observation that those who do the crushing often feel nothing, while the person crushed is the one who feels what is happening. While this is pretty bleak, it does point out that failure is able to locate knowledge where some institutions assume we least expect to find it: with those who experience failure most directly. The person who is harmed by a system may understand its reality way more clearly than the person who benefits from what appears, from a distance, to be its smooth operation. In this sense, failure is not only humbling because it exposes our limits, but it is humbling in another direction as it redistributes authority over what is real.

This matters because failure is not only something to be explained by those looking on from above. If it is often understood most sharply by those who have lived through the breakdown itself and failure redistributes authority over what is real, then how do we allow that knowledge to be heard, examined and acted upon?

Failure, evaluation and evidence

This is where evaluation of our organisational activity becomes important, not as a bureaucratic afterthought, but as one of the ways it is decided what counts as evidence. Evaluation can be treated as a judgement passed at the end of a process: did the change programme achieve its outcomes, did the intervention move the metric, was the investment justified? And while these are necessary questions, they run the risk of making something that failed into a bland albeit regretful outcome, rather than a place where searching questions are asked.

Perhaps a more useful evaluation activity would ask not only whether the hoped-for outcome occurred, but what the failure to arrive at that outcome shows us. Was the analysis used to design the activity good enough? Was the behaviour we were trying to influence the right one? Were people being asked to act in ways that did not in fact make sense to them? Did the people or organisations involved have legitimacy? In this sense, evaluation should not simply measure failure but help interpret it in the wider context.

For example, we might launch a campaign for healthy eating that fails to register much, if any, change in the target audience. So we can look at how well the materials for communication were designed, whether there was sufficient investment to ensure they reached the target audience, whether the message was compelling enough and so on. But those sorts of questions do not look at whether any campaign was going to succeed given the context of people not having access to healthy foods, whether due to location or affordability. Perhaps the weight of advertising promoting less healthy options was simply too great for any campaign to get cut-through.

Also, any sort of behavioural evaluation cannot only ask those who designed or funded the intervention what went wrong. Instead, we need to speak to those who felt the failure most directly, and while they may not describe it in technocratic terms, they may understand with great precision where the offer failed to meet the real world. Asking questions directly of someone who is attempting to make food choices for themselves and others is often the best way to understand why a healthy eating campaign may have failed.

This would make evaluation less of an audit function and more of a discipline of enquiry. Of course, we would still care about outcomes and value for money, but we would also ask in more detail what failure has now made visible: whose assumptions were protected, whose knowledge was ignored, what contexts were flattened, and what kind of person the programme imagined. In that sense, evaluation becomes one of the practical ways policy can learn from failure before it is tidied away.

Conclusions

The point is not to romanticise failure as it can be painful, wasteful, humiliating and damaging. But also, we cannot deny that people and organisations can learn from it. But learning is not the only value failure has – as, perhaps more importantly, it can disclose what success conceals.

This is why Beckett’s widely used term to ‘fail better’ perhaps needs to be rescued from the start-up culture that has adopted it. In contemporary business language, it is often made to sound like a slogan for innovation: try, fail, iterate, improve. But Beckett’s notion of failure is not simply a method for getting closer to success and master the world more effectively. Instead, it may simply mean proceeding through with fewer illusions.

So, this is a case for failure not to be rescued too quickly. Before it is turned into a case study or a story of resilience, it can be allowed to remain troublesome for a while. Its lessons lie in the awkward evidence it offers about what our models could not see, what we could not hear, and what our preferred stories could no longer hold.



Implications for policymakers

• Treat failure as evidence about the system, not simply as poor delivery or public non-compliance.

• Ask what failure reveals about the assumptions behind the policy: e.g. trust, agency, motivation, access and everyday fit.

• Use evaluation to interpret failure, not just to record whether targets were met.

• Listen to those who lived through the breakdown; they may understand the failure more clearly than those who designed the intervention.

Implications for marketers

• Treat campaign failure as diagnosis, not just a prompt for optimisation.

• Ask whether the brand misunderstood its role in people’s lives, rather than only blaming creative, targeting or media.

• Use non-response as evidence about consumer reality: credibility, permission, effort, timing and cultural fit.

• Be cautious with success stories; success may reflect luck, timing, context or competitor weakness as much as brand brilliance.