Why AI explanations can make reviewers reject promising ideas

An AI explanation can make rejecting an idea too easy

An AI system can make a weak rejection easier to defend. For companies using AI to assess proposed projects, a written explanation may therefore weaken the human review it is meant to help. In an experiment on initial proposal reviews, which researchers call “early-stage innovation screening,” evaluators became more willing to follow incorrect recommendations when the model supplied a rationale. This increased the risk of discarding ideas that experts would advance.

Discarding a promising idea is one of two costly screening errors. A false negative rejects an idea that could have succeeded; the researchers illustrate this risk with Xerox terminating early Ethernet and PostScript projects. A false positive advances a project that ultimately fails, with Google Glass and Amazon’s Fire Phone as their examples. These cases show the business stakes, while the experiment assessed screening decisions by their agreement with experts.

Screening often happens with limited time and information, which makes AI assistance attractive. Large language models, or LLMs, can recommend whether a proposal should advance and explain that recommendation in writing. But an explanation also influences whether a reviewer investigates further, so its value depends on how it changes the reviewer’s judgment. The experiment tested whether a rationale improved that judgment.

The experiment compared explanations with independent judgment

To test the effect of explanations, researchers associated with Harvard Business School, MIT, and the University of Washington recruited 228 experienced evaluators to assess nearly 50 submissions from an MIT challenge. A common pool of proposals let the researchers compare decisions under three conditions. Evaluators either worked without AI, received an LLM recommendation with a written rationale, or received a black-box recommendation: an AI pass-fail decision without an explanation.

The researchers assessed those decisions against judgments from four human experts. An evaluator’s choice counted as ‘correct’ when it matched the experts’ positive or negative assessment. Under this standard, a false positive meant advancing a submission the experts would reject, while a false negative meant rejecting one they would advance. Correctness therefore meant agreement with expert screening judgments; eventual commercial success was a different question.

Alongside decision quality, the researchers examined how evaluators used the model’s advice. Evaluators could follow a recommendation or override it by making a different choice. A productive override involved independently checking a persuasive model output before deciding, so disagreement alone did not demonstrate better judgment. Comparing this behavior with expert agreement helped distinguish willingness to accept AI advice from improvements in screening.

Explained rejections reduced one error while increasing another

Against the expert judgments, recommendations without explanations produced the stronger overall result. Black-box assistance improved evaluators’ decisions, while assistance with written rationales did not. “Our findings reveal that LLM explanations do not necessarily improve decision-making,” the researchers wrote. Evaluators made worse decisions with the model’s rationale than with its recommendation alone.

Despite that difference in quality, AI recommendations attracted substantial agreement. The researchers reported that evaluators accepted LLM recommendations 67% of the time. They also reported agreement with both black-box and narrative LLM decisions roughly 75% of the time, alongside agreement with human decisions 54% of the time. The model’s influence extended to cases where its recommendation conflicted with the expert panel.

In those cases, evaluators followed AI advice in both directions: they rejected proposals the experts favored and approved proposals the experts considered sub-par. Written rationales increased the problem because evaluators were more inclined to follow an incorrect recommendation when it included an explanation. Making the recommendation easier to understand therefore also made an erroneous recommendation easier to accept, moving reviewers further from the expert judgment.

This effect mattered especially when the model recommended rejection. Evaluators disproportionately agreed with explained rejections, shifting their decisions toward eliminating proposals. That shift reduced false positives because fewer submissions opposed by the expert panel received support. For a company trying to avoid spending resources on weak proposals, this was a meaningful benefit.

But the researchers reported that the same treatment “substantially” increased false negatives. More submissions that experts would have advanced were eliminated, so filtering out weaker proposals also removed promising options. The overall quality comparison captures the trade-off: explanations reduced one error category without improving agreement with experts across the screening task.

For companies evaluating such systems, this trade-off makes acceptance an incomplete measure of success. A high acceptance rate shows that reviewers follow the model, while decision quality shows whether following it helps them reach better judgments. Assessing both, alongside independent checking, can reveal whether an interface improves screening or simply makes its recommendations persuasive. The researchers’ next question was why explanations made rejection particularly easy to accept.

A persuasive rationale may discourage verification

To explain that pattern, the researchers propose that narratives can “suppress” productive overrides. In their account, a convincing reason to reject a proposal reduces the reviewer’s incentive to check whether the reason holds. Their proposed mechanism is that accepting the rationale discourages the independent verification needed to catch a mistaken recommendation.

One possible influence is negativity bias, the tendency to give negative information greater weight than positive information. In screening, this could make a clear account of potential failure more influential than an idea’s uncertain promise. The researchers argue that this helps explain the disproportionate agreement with rationales supporting rejection. A fluent account of risk can give the reviewer an immediate reason to stop considering an option.

Rejection may also combine a sense of responsibility with the convenience of leaving existing commitments unchanged. “Rejection is an active, eliminative decision that feels more consequential and accountable than preserving optionality,” the researchers wrote. They also describe rejection as maintaining the status quo, avoiding risk and bias, and requiring no resource commitment. A reviewer can therefore experience rejection as consequential while choosing an outcome that requires no new investment.

A written rationale can make that choice easier to defend because it supplies what the researchers call “ready-made justifications.” Evaluators may adopt these reasons without checking them independently, treating fluency, coherence, and apparent credibility as grounds for acceptance. The explanation then gives reviewers a defensible position before they have established whether its reasoning is sound. In this account, the model takes over part of the thinking that human review was intended to provide.

That apparent clarity can also create an “illusion of explanatory depth,” in which someone feels they understand a decision more fully than they do. As the researchers put it, “individuals tend to overestimate their understanding of a decision despite limited insight into its reasoning,” which can weaken the perceived need for further scrutiny. Expert-like language may therefore leave a reviewer feeling informed while discouraging the work needed to assess the recommendation. For explanation design, the question becomes how the text changes the reviewer’s behavior.

Design explanations around the task and the cost of errors

Because explanations can change review behavior, the researchers identify “clear implications” for AI-assisted evaluation and urge caution in high-stakes settings. “Effective human-AI collaboration requires designs that preserve rather than supplant independent human judgment,” they write. A deployment should therefore be evaluated on whether reviewers retain the ability and motivation to check recommendations. Reasons are useful when they help people examine the decision.

To assess that effect before deployment, the researchers recommend testing whether explanations improve accuracy, help reviewers detect errors, and develop their understanding of model behavior. Better understanding could also increase human-AI agreement by helping reviewers recognize when the model is reliable. Both agreement and disagreement need to be assessed against the quality of the final choice, since a higher override rate offers little value unless those overrides reflect sound verification.

What counts as a sound choice also depends on the task’s error costs. In quality control, compliance screening, and fraud detection, the researchers suggest that explanations could reinforce useful caution. Innovation screening has a different requirement because eliminating a proposal can prevent the company from gathering the information needed to judge it more fully. An explanation format should therefore be tested against the consequences of both advancing and rejecting an option.

For innovation screening, the black-box result gives companies a reason to compare a recommendation-only interface with an explained version. Simpler recommendations may leave reviewers more room for discretion and independent checking. As an application of the findings, this comparison should examine whether reviewers catch erroneous approvals and rejections, and whether their final choices improve against the chosen quality standard. The rationale itself becomes a design choice whose effect must be evaluated.

To give reviewers more to examine, the researchers propose testing contrasting narratives: reasons to reject an idea alongside reasons to accept it. They also suggest testing disclosures of uncertainty based on a fixed threshold rather than purely binary decisions. These approaches would change the information available during review, but they remain proposals for testing. Their effects need to be assessed through the decisions reviewers make.

Another proposed approach is to structure the system so that it invites human disagreement while allowing agreement when a recommendation withstands scrutiny. The aim is to preserve independent assessment even when the model sounds fluent and confident. The researchers express the design principle directly: “organizations should treat AI explanations not as universally beneficial transparency tools, but as behavioral interventions whose effects depend on how evaluators process information under uncertainty.”

Later decision stages may change the value of an explanation

Because the experiment concerned early screening, the researchers propose studying explanations at later decision stages. Reviewers at those stages may face fewer options, have more information, and have stronger incentives to verify outputs and deliberate. Those conditions could change the value of a rationale by giving reviewers more evidence against which to examine it. Testing at those stages would show how explanations affect judgment when people have more opportunity and reason to check the model’s reasoning.

Main highlights

• Measure decision quality against a clear benchmark: The experiment compared 228 evaluators’ screening decisions with judgments from four experts. Innovation teams evaluating AI tools need a defined quality standard; agreement with experts measures screening quality, while commercial success requires separate evidence.
• Track both types of screening error: Written AI rationales reduced approvals of weak proposals but increased rejections of ideas experts favored. Innovation managers need to weigh the cost of missed opportunities alongside the resources saved by rejecting proposals.
• Preserve independent verification: Persuasive explanations may discourage reviewers from checking incorrect recommendations, especially rejections. Review owners can build explicit verification steps into the screening process and assess whether those checks catch errors.
• Test explanation formats before deployment: Recommendation-only assistance improved decisions in this experiment, while explained recommendations did not. AI product owners can compare both formats on accuracy, error detection, and productive overrides before choosing an interface.
• Validate explanations at each decision stage: The findings concern early screening, where information and time are limited. Later-stage evaluation warrants separate testing because reviewers may have more evidence and stronger incentives to check the model’s reasoning.