When evaluating whether to invest in a new idea, senior executives often rely on experts. But these advisers often favor ideas that are easy to execute over tough-to-pull-off but potentially groundbreaking plans.
The risk for companies: Brilliant innovations might wind up on the cutting room floor, suggests new research based on an international competition to design a NASA robot.
When experts have a been-there-done-that mindset, potential breakthroughs may hit the discard pile before companies can evaluate them in full.
Harvard Business School Assistant Professor Jacqueline Lane and colleagues studied how experts judged contest entries and found that the greater an evaluator’s expertise, the more likely they were to nix less feasible proposals in favor of safer bets.
“Who we select to do our evaluations and to pick these ideas has important implications on the products and solutions that are selected—and the ones that get implemented,” Lane says.
An increasingly on-demand world has raised the stakes for innovation at many companies, whose leaders face pressure to constantly update, upgrade, and introduce products. But when experts have a been-there-done-that mindset, potential breakthroughs may hit the discard pile before companies can evaluate them in full—causing companies to miss out on big competitive leaps, write the researchers in their working paper.
Lane, who is affiliated with the Laboratory for Innovation Science at Harvard (LISH), teamed with Karim R. Lakhani, the Dorothy and Michael Hintze Professor of Business Administration at HBS, and LISH research scientist Michael Menietti; as well as George Washington University’s Zoe Szajnfarber and Jason Crusan.
“Given that these types of decisions can affect a firm’s strategic direction, it raises the question of whether managers should aim to deliberately increase the diversity of their evaluator pools,” the researchers write.
Experts prefer feasible robot designs
To test how open experts are to unique ideas, the researchers worked with NASA and Freelancer.com to recruit people to evaluate contest entries for the US space agency’s Astrobee system in 2018. Entries proposed solutions to technical issues with a robotic arm for Astrobee, a free-flying robot that handles inventory, documents astronauts’ experiments, and moves cargo. Astrobee is key to NASA’s mission to return to the moon, among other operations.
NASA ran 17 challenges with prizes that ranged from $250 to $5,000 and included a promise that features from winning entries would be incorporated in future Astrobee designs. The contests drew more than 250 proposals.
The researchers asked 374 evaluation recruits with expertise both inside and outside of the robotics field to assess 10 of 101 design proposals. That yielded some 3,869 pairs of evaluators and entries to analyze.
There are limits to domain expertise. When they see a novel design that they have never encountered before, suddenly, their expertise becomes less relevant.
Lane and her colleagues sorted responses evaluating robot arm designs into three categories of evaluator: unscreened, screened through a human resources-like questionnaire examining their credentials, and those that took a robotics skills test. The authors then sorted critique criteria using the evaluator ratings for novelty, feasibility, and quality of design. They also analyzed individual written comments justifying evaluators’ recommendations.
The results showed that experts made a “tradeoff” between the possibility a proposal will work and the risk of taking a big leap forward through untested ideas, Lane says. Compared to the unscreened evaluators, the experts weighed feasibility roughly 30 percent more in their ratings of a design’s quality.
On average, experts screened through the human resources test and the skills test were more critical of unusual robotic arm entries than the non-expert group, the researchers found. This shows what the authors called a “feasibility preference.”
Novel ideas appear riskier to experts
Why do experts react this way? The behavior of chess masters offers some clues, the authors say. Chess masters show better memory skills than beginning players because they’ve memorized positions on the board in “chunks” of four- to five-piece configurations. But the chess master’s memory advantage disappears when pieces are randomly placed on the board, rather than in established patterns.
When it comes to robotics designs, experts are more familiar with existing solutions in the domain. That advantage helps them identify designs that would be easiest to implement.
“But there are limits to domain expertise. When they see a novel design that they have never encountered before, suddenly, their expertise becomes less relevant,” the researchers write.
Imagine what you want to create and where you want to be 50 years from now—little incremental improvements probably won't get you there.
Novel ideas could seem even riskier to experts because they challenge their competence and knowledge in the domain, the researchers say.
One potential reason decision-makers hold back: They’re concerned about safety. Another could be time pressures, Lane notes. Still, she’s concerned such a feasibility preference could kill solutions before they are given a chance.
“The experts are not the final decision-makers. It's really the innovation manager, or the person who's in charge of the project, who's going to say, ‘OK, these are the reviews that I got, and these are the ones that I'm going to go forward with.’ But they're going to be informed by the experts that they select to evaluate,” Lane says.
Managers need to consider long-term gains
So, how can managers who act as gatekeepers spot breakthrough ideas that are brushed aside by experts? First, realize feasibility and novelty are two distinct parts of an idea—one doesn’t necessarily mean the other isn’t worthy of consideration, Lane says. Then consider the project’s goal and the risk needed to make big leaps.
“Doing something that works, that's quick, that is going to be reliable is one approach forward,” Lane says. “But imagine what you want to create and where you want to be 50 years from now—little incremental improvements probably won't get you there.”
You Might Also Like:
Feedback or ideas to share? Email the Working Knowledge team at hbswk@hbs.edu.
Image: iStockphoto/cokada
