Why we gloss over great ideas – and invest in bad ones

Few decisions change the world – and the truly revolutionary choices are rarely made by doctoral students. But, in 1995, a young computer scientist at Stanford University was trying to pick the subject of his dissertation. He knew that his subject choice could influence his whole career, and so he made a list of about 10 ideas that had caught his eye.

One of those options concerned the relatively new-fangled World Wide Web, which had only been invented six years prior. The student wondered whether it might be possible to chart the hyperlinks between sites, in much the same way that academics chart their citations.

As you may have guessed, that student was Larry Page, and his project – initially called BackRub – would eventually form the basis of the PageRank algorithm behind Google’s search engine. Google’s parent company Alphabet is now worth more than a trillion dollars.

In hindsight, the genius of the idea – which permanently changed the way we navigate the internet, and earned Page around $100bn (£70bn) in personal wealth – may seem obvious. Yet, it’s easy to imagine an alternative world in which Page instead settled on another project that might have seemed like the perfect material to gain the PhD.

In general, research shows people rarely assess the value of their own ideas accurately. This means that we often fail to see the potential in our best ideas, leading us to jettison them and waste time on less promising ventures. Fortunately, these studies also suggest some clever ways to avoid making this mistake.

The benefits of being second place

One of the most insightful studies comes from Professor Justin Berg at the Stanford Graduate School of Business. In a series of experiments, he asked participants to invent new products or services – such as novel forms of fitness equipment or exciting travel experiences. The participants first brainstormed a list of initial ideas, which they personally ranked in order of preference, after which they were given more time to develop their concepts. A team of independent judges then rated the quality of the final designs.

Analysing the data across the group, Berg found that, on average, the participant’s first choice often failed to impress. Instead, it tended to be the participant’s second favourite idea that generated more excitement from others.

Larry Page recognised the importance of his idea – but research shows that many of us undervalue our ideas (Credit: Alamy)
Larry Page recognised the importance of his idea – but research shows that many of us undervalue our ideas (Credit: Alamy)

Investigating further, Berg found that the participants’ rankings were being skewed by the level of abstraction of the raw ideas. In general, the participants preferred the concepts that were the most concrete and which initially appeared to be the easiest to apply. “An idea that is already concrete is usually closer to its ultimate potential, so you don’t need as much foresight,” says Berg. “The value of the idea is already relatively apparent.” This meant that they often overlooked slightly more abstract ideas that were not immediately practical.

With development, however, the potential uses of the abstract ideas soon became apparent – leading them to be more popular with the independent judges.

As a simple example, consider one participant’s proposal for a “gravity or wind machine” that provided resistance against your movements – a bit like swimming but without the “wet mess of a pool”. It’s initially hard to imagine how that might be put into action, but with development time, the participant had explored the potential use of a magnetic platform, combined with magnetic clothes. “This would make a common thing like high-stepping a weight-training exercise,” wrote the participant. By reversing the magnets, the exerciser might even be able to levitate, easing activities like handstands. With those details, the plan ultimately proved to be highly popular with the independent judges.

How to identify brilliance

As Berg’s results show, we may be consistently missing out on our best ideas, thanks to a simple inability to recognise the nuggets of gold in front of us. How, then, can we improve our judgements?

Given that our hunches are often wrong, one possibility may be to encourage the right kind of reflective thinking. Berg, for instance, found he could improve participants’ forecasts by asking them to justify why they had chosen their rankings – a simple reflective exercise that helped them to appreciate the more abstract ideas. “The key is that you have to seriously engage in this thinking and be open to changing your mind based on it,” he says.

With extra thought, an abstract idea that initially appeared less promising will begin to gain substance and might soon start to seem more exciting
More generally, he thinks that we might delay making a final choice until we have developed a couple of competing options. With extra thought, an abstract idea that initially appeared less promising will begin to gain substance and might soon start to seem more exciting. If I were looking to start writing a new book, for example, I might brainstorm a number of concepts before writing out the chapter summaries for my two favourites. Only then would I decide which to pursue. “This reduces the risk of rejecting an idea too early,” says Berg.

Rogelio Puente Díaz, a professor of marketing at Universidad Anahuac México Norte, agrees that we should be wary of relying on our initial judgements. His own work has shown that we often rely on “metacognitive feelings” – the hunch that something works, or doesn’t – to judge an idea’s quality, and these can be subject to bias. In his opinion, however, one of the biggest problems may be unwarranted optimism about an idea’s potential success. “Our tendency is just to look at the positive side,” he says. For this reason, he recommends that we make a conscious effort to think about the many reasons an idea might fail.

If you were to apply this alongside Berg’s suggestion, you might develop those two favourite ideas and, after you have a better sense of what they would entail, you would compile a list of pros and cons for each. “Put as much effort in thinking about what could go wrong, as well as what could go right.” This way, you can pre-empt the issues that would ultimately lead to failure – so that you can be sure that the brilliant idea that you choose is achievable.

Perhaps the surest way of ensuring success is to gauge the opinions of others – and preferably the people who might be interested in your product or service. Laura Kornish, a professor of marketing at the University of Colorado Boulder, says that most organisations currently engage in consumer research at a relatively late stage of a product’s development, whereas her research shows that it can be usefully employed at an earlier stage, soon after the initial brainstorm, to determine which ideas will be most successful. “If you can get 20 or 30 people to rate your ideas, you can have a real signal of whether it is good or not,” she says.

We may never know what led Larry Page to focus on BackRub and leave his other ideas – but these techniques can help us all to identify the projects with the greatest potential. By putting more effort into the initial development, considering the best- and worst-case scenarios, and gauging the opinions of others, we can sift the gold from the debris of our creativity – and never risk wasting a trillion-dollar idea.