Harder technical interview questions may not improve hiring

Harder interviews can leave the hiring problem unchanged

A team hires an engineer who writes clean whiteboard code, gives confident system-design answers, and handles behavioral questions smoothly. Then work begins. Delivery slips, unclear tasks go untouched, and the new “senior” engineer needs more support than the junior they were expected to mentor. The team responds with harder coding challenges, another round, or more interviewers, making the next hiring process more demanding while leaving the original evaluation problem intact.

The unnamed interview-process reviewer behind Technical Interview Questions From the Other Side of the Table describes observations made “After years” of reviewing hiring processes. The reviewer argues that teams often choose questions before agreeing on what they should measure. Interviewers then apply different standards, compare impressions, and mistake a polished performance for the capability the job requires. An unsuccessful hire prompts more testing, even though the way interviewers interpret answers stays the same.

To change that outcome, an engineering manager, VP, or CTO needs to define the evidence before asking more of candidates. The reviewer recommends a calibrated interview loop: a sequence of interviews with questions, role-level expectations, and scoring standards aligned across interviewers. The team also needs to compare its judgments with performance after hiring to see whether they were useful. Even a flawless answer provides limited evidence when nobody has agreed on what it demonstrates.

Structured assessment gives interviewers a shared basis for judgment

Research supports replacing impression-led interviewing with structured assessment, which evaluates candidates against predefined questions and scoring standards. Decades of personnel-selection research support structured interviews as useful predictors of job performance. Google’s hiring research also emphasizes structured behavioral questions and consistent evaluation. The findings cover different aspects of assessment, but each gives employers a reason to make their evaluation process explicit.

The research also distinguishes between kinds of performance an interview may help assess. Task performance means executing assigned work, while contextual performance means contributing to the conditions that help people work effectively. The 2025 meta-analysis connects interview evaluations with both. Its finding about structured scoring is especially relevant when employers want to assess contributions beyond technical execution.

Useful assessment becomes more pressing when recruiting is difficult. ManpowerGroup, which has a commercial interest in employers’ need to find skilled workers, reported in its 2025 U.S. Talent Shortage Survey that nearly three in four U.S. employers struggled to find the skilled talent they needed. This broad U.S. labor-market measure describes the wider hiring environment rather than engineering hiring specifically. A process that uses up a scarce, expensive search without distinguishing relevant capability from interview preparation adds to that pressure.

For the reviewer, the practical response starts with an explicit account of what success in the role requires. Questions should give candidates a fair chance to demonstrate those requirements, and scoring should explain how their answers meet the standard. Interviewers can then examine confidence, familiarity, and personal rapport before those impressions influence the decision.

Define the skills and scoring before choosing the questions

To make those requirements concrete, the reviewer recommends identifying four to six core skills and defining the proficiency needed in each. For a senior backend engineer, these might include relevant programming languages, depth in application programming interfaces (APIs), microservices, which divide an application into smaller services, or continuous integration and continuous delivery (CI/CD), which automate integrating and preparing code changes for release. Problem decomposition and system design may also be essential. The list should reflect the work the person will own, because a broad catalogue of desirable technologies obscures the capabilities that matter most.

The required level needs equal attention because one question can reveal different levels of proficiency. When comparing “senior” with “mid-level,” interviewers need to agree on how the expected reasoning, independence, and technical depth change. A competent response can still fall short of the role’s requirements. Without an agreed standard, one interviewer may accept that answer while another sees it as a concern.

Once the skills and levels are clear, the reviewer recommends using each question to assess one, or at most two, key skills. Before interviewing, describe a strong response and give interviewers a shared rubric: written standards for scoring the relevant skill. Record separate competency scores so the panel can see where evidence is strong, weak, or incomplete. This replaces judgments such as “we liked them” with assessments the team can compare across candidates and over time.

Separate scores also require interviewers to recognize the evidence each question category provides. Coding can reveal how a candidate breaks down a problem, while a retrospective behavioral question can establish what that person contributed to past work. The following measurement map makes those purposes clear before the conversation begins.

A shared rubric still needs relevant questions, as the reviewer’s memorization example shows: “What’s the default heap size of the JVM?” The Java Virtual Machine (JVM) runs Java programs, and its heap is memory used to store objects. A working engineer may look up that setting when needed, so testing immediate recall can disadvantage someone whose normal work includes checking facts. Agreeing on how to score recall does not establish that it measures a capability the role requires.

The reviewer applies the same test to gotchas, brainteasers, and familiar questions from popular interview lists. Candidates routinely practice publicly available questions, so prior exposure can turn an apparently demanding exercise into a preparation test. Remove a question when seeing it beforehand makes it dramatically easier, the reviewer recommends. Deliberately making candidates uncomfortable also substitutes pressure for a clear assessment of technical ability, problem solving, or judgment, and can damage the employer’s reputation among applicants.

Google made a related change in 2013, moving away from brainteaser-style questions after finding them unhelpful as predictors of job performance. Consistent scoring cannot make an irrelevant measurement relevant, which is why question selection belongs inside the assessment process. The reviewer’s five-check audit helps a team examine both question selection and scoring before the next hiring round.

A failed check identifies a specific change the team can make before meeting candidates. An extra interviewer cannot supply a missing skill definition, and a harder exercise cannot establish an absent scoring standard. Once those foundations are in place, the next task is to define useful evidence within each kind of answer.

Score reasoning and judgment against the role’s requirements

Coding: examine decisions before and during implementation

For coding, the reviewer recommends observing decomposition, data-structure choice, complexity reasoning, edge-case handling, and how candidates handle missing requirements. Decisions before and during implementation make these qualities visible. But an unfamiliar puzzle can obscure them when success depends heavily on discovering one particular solution.

A prompt can instead make those decisions explicit: “Given a stream of events arriving out of order, design a function that returns the events of the last five minutes in order.” A strong response clarifies event volume and what the ordering guarantees mean, because those answers affect the data structure and implementation. Immediately choosing a sorted list leaves those decisions unexplained. The five-minute window is a hypothetical requirement in the reviewer’s example that the interviewer can use to examine the candidate’s reasoning.

Existing code offers another way to examine that reasoning: “Here is a function that works but is slow on large inputs. Walk me through how you’d find out why, then make it faster.” The reviewer recommends listening for measurement or profiling, which means observing where the program spends time or resources, before optimization. The candidate should then explain the improvement and connect time and space complexity to the actual inputs and operations. Big O notation describes how resource requirements grow with input size; it becomes useful when that connection is explicit.

Missing requirements reveal another decision: whether the candidate recognizes behavior that needs agreement before implementation. The reviewer illustrates this with “Write a function that merges two customer records, where fields may conflict.” A useful first step is to identify possible conflicts and propose explicit rules for resolving them. Silently inventing those rules may produce working code while hiding decisions that affect the result.

To preserve that distinction, interviewers should score clarification, assumptions, and implementation separately enough to show what happened. They should also record whether candidates notice edge cases independently or only after a prompt, explain alternatives, recognize defects, and correct their approach. The reviewer argues that slightly suboptimal code with clear reasoning and self-correction can provide more usable evidence than a flawless solution without explanation, particularly when rehearsal is possible. Correctness remains part of the assessment alongside the reasoning that produced it.

System design: examine the consequences of a decision

The same attention to reasoning applies to system design, where the reviewer recommends testing scalability, performance, and choices under constraints. Open prompts give interviewers room to assess these skills if they follow the explanation beneath the technology names. The scale and growth figures in the reviewer’s examples are hypothetical requirements.

The personal-system prompt makes generic preparation less useful because the candidate must explain a system they know. The interviewer can examine why its current design works, which constraint would become limiting, and how the proposed change affects other behavior. So a prepared architecture description must lead to an explanation of consequences to answer the follow-ups adequately.

Caching and message queues give interviewers concrete ways to explore those consequences. A cache stores data for faster access, so a strong explanation covers where it sits, when its entries are invalidated, and what happens when it contains stale data. A message queue holds work or messages for later processing; the candidate should explain how it changes delivery behavior. The reviewer also recommends examining whether better write performance requires accepting eventual consistency, where different parts of the system may temporarily hold different versions of data.

When a choice sounds plausible but remains shallow, the reviewer suggests asking, “What does that decision break?” This brings disadvantages into the assessment and gives the candidate a reason to explain the trade-off. Still, the interviewer must avoid steering the candidate toward a preferred architecture. A different design deserves credit when its reasoning fits the stated constraints, because the skill being measured is judgment within those constraints.

Behavioral and hypothetical questions: establish responsibility and priorities

Technical decisions also depend on how an engineer handles people and uncertain work. The reviewer identifies conflict, ambiguity, ownership, and communication as frequent sources of difficulty in senior hiring, so behavioral assessment needs deliberate design alongside coding and system design. Past-experience questions examine what a candidate has done. Hypothetical scenarios examine how they might approach unfamiliar work.

Past-experience questions connect those qualities to specific actions and consequences. The following prompts focus on a wrong decision, a disagreement, and a struggling project. In each case, the interviewer needs to establish the candidate’s contribution.

To organize that evidence, the reviewer recommends STAR: situation, task, action, result. The framework helps the evaluator identify the setting, responsibility, conduct, and outcome. A polished account of the situation can still conceal vague action, especially when the candidate repeatedly describes the team’s work without establishing their own responsibility. The evaluator needs a clear personal account, represented by “I.”

When responsibility remains unclear, the follow-up “What did you specifically do?” gives the candidate room to separate team activity from personal decisions and actions. If the account remains vague, the claimed ownership deserves closer examination. But the reviewer treats STAR as an assessment framework rather than a mandatory speaking format. A response that supplies the relevant evidence in a different order should receive appropriate credit.

Hypotheticals extend that assessment to work the candidate may not yet have had a chance to own. Senior candidates should have substantial past experience to discuss, while career switchers and people moving into greater responsibility may face unfamiliar requirements. Scenarios let the interviewer examine their priorities while keeping their actual experience clear. They add evidence about judgment alongside the retrospective account.

In both scenarios, the reviewer recommends examining a plausible first action and the reason for choosing it. The service question requires sequencing work during the first week of a three-week release window. The pull-request question requires addressing a concrete behavior, including the interpersonal work involved. Shared standards should help interviewers recognize useful judgment across different reasonable responses.

Uncertainty: examine the investigation plan

Those scenarios involve incomplete information, as does an unfamiliar technical topic. The reviewer offers this illustrative answer: “I don’t know how that works internally, but here’s how I’d find out, here’s what I’d test first, and here’s what I’d expect to see,” giving the interviewer an investigation plan to examine. The proposed test and expected result create follow-up questions about how the candidate would resolve uncertainty. A plan can reveal more about reasoning than a correct guess.

Interviewers need to recognize that difference before a favorable impression makes confident guessing persuasive. A correct answer may reveal little about the reasoning behind it, while an incomplete answer can expose a sound method for investigation. The rubric should preserve both observations. The reviewer’s comparisons make these expectations concrete across question types.

Applying these standards requires enough flexibility for candidates to demonstrate the same skill in different ways. An imperfect implementation can expose useful reasoning, and a behavioral answer can establish responsibility without following a prescribed presentation order. The interviewer’s task is to record evidence against the role’s requirements. This keeps discretion tied to observable decisions and conduct.

Separate AI assistance from AI capability

AI adds two assessment questions: whether a candidate could receive live assistance, and whether they can use AI competently in engineering work. The reviewer notes that generic remote coding exercises can receive real-time model-generated answers. This changes what an unaided-looking solution tells the interviewer and creates a reason to assess how well the candidate inspects generated work.

For live assistance, the reviewer recommends questions anchored in “a system you built” or “a decision you regretted,” followed by exchanges such as “what does that decision break?” These follow-ups return to the personal-history and consequence questions already described, requiring engagement with the details under discussion. During the dialogue, an interviewer can introduce a constraint and examine whether the explanation remains coherent. Such exchanges are harder to outsource than a standalone puzzle, although assistance remains possible.

Because the aim is to elicit evidence, the reviewer favors live dialogue over surveillance. Personal history gives the interviewer material to examine through follow-up questions about decisions and outcomes. A harder puzzle can leave the outsourcing problem unchanged when a model can still produce a plausible solution.

Workplace AI fluency requires a separate exercise: give the candidate an AI-generated solution or code sample and ask them to review it. The reviewer recommends looking for checks on claims, detection of subtle bugs, and an explanation of when relying on a model would be inappropriate. Reviewing a junior engineer’s code is already a relevant senior responsibility, and generated code also demands explicit verification. This exercise makes review judgment visible without treating unaided answer production as a complete measure of readiness for AI-assisted engineering.

Calibrate every interviewer, and check scores against performance

The evidence from these exercises needs to remain comparable when an external screening partner conducts part of the process. A nearshore partner, an external team in a nearby country, may run the first technical screen or interview jointly with the employer. The reviewer argues that assessments still need to match the employer’s role requirements, because generic praise can hide a mismatch until onboarding. Another organization’s participation does not by itself align its conclusions with the internal team’s scores.

Before relying on that screening, the reviewer recommends checking three parts of the arrangement. Together, they establish what the partner will assess, how it will score the evidence, and whether its interviewers understand the employer’s expectations.

After those checks, ask the partner to explain a scored example. The discussion shows how an answer becomes a skill assessment and lets both parties identify differences in interpretation. Feedback such as “strong candidate, good communication skills,” leaves the employer unable to see the observed behavior, the standard applied, or the reason for the score. A useful handoff preserves the connection between the question, the candidate’s evidence, and the assessment.

That connection also makes review possible after the hiring decision. The reviewer recommends periodically comparing interview assessments with performance after people join the team, including those screened by a partner. If the strongest interview scores do not correspond to stronger performance, examine the criteria, the scoring, and how interviewers apply the rubric. A mismatch can have several causes, so the review should examine those possibilities before assigning an explanation.

Written standards let the team examine its original judgment instead of reconstructing it from memory. For the team whose polished interview hire later struggled with delivery and unclear tasks, the useful question is which interview evidence led to those expectations and how it relates to the work observed after hiring. The employer can use that comparison to revise its questions, proficiency standards, or scoring practice. Calibration continues as the team learns how its hiring judgments relate to performance.

Recap

For engineering leaders, the first response to a disappointing hire should be to examine the evidence behind the decision. A harder coding exercise or another interview round adds demands on candidates and interviewers. Neither resolves uncertainty about what the role requires or how an answer demonstrates readiness for it.

Before the next hiring round, agree on the essential skills, the required proficiency, and the evidence that earns a strong score. Apply those standards across internal interviewers and external screening partners. Assess technical correctness alongside reasoning, ownership, and judgment, so a polished answer does not stand in for capabilities the job requires.

Then compare those assessments with performance after hiring. Where expectations and results diverge, review the questions, scoring standards, and interviewer decisions before adding more tests. This gives leaders a basis for improving the process rather than simply making it longer.

The executive responsibility is to make hiring judgments clear enough to examine and consistent enough to compare. A demanding interview is useful only when it produces evidence that helps the business choose someone who can do the work.