Feature comparisons are easy to game. Vendors write their own, reviewers copy them, and the resulting tables tell you everything about marketing positioning and almost nothing about what actually happens when a real candidate goes through the process.
So we did something different. We built a single candidate profile — a mid-level backend engineer with four years of experience, solid Python and distributed systems exposure, some genuine architectural decisions on their resume and some that were clearly team efforts with limited personal contribution. Then we walked that candidate through five of the most widely used AI interview platforms and asked one question of each: what did it actually capture, and what did it miss?
The results were instructive. Not because any platform was obviously broken — they all work, in their own way — but because each one has a precise and very specific blind spot that no feature list will ever tell you about.
The Candidate Profile
To make this useful, the candidate needed to have real depth in some areas and real weakness in others — so we could see whether the platform found each. Here's what we were working with:
- Genuine ownership: Led the rewrite of a payment processing service — architectural decisions, vendor choices, on-call responsibilities were all theirs
- Inflated ownership: Lists a "real-time data pipeline" on their resume that they contributed to but didn't design — the architecture was a colleague's
- Technical depth: Strong on database design, transactions, and API design; can speak precisely about trade-offs
- Technical shallowness: System design at scale is surface-level — they can name patterns but struggles to reason about why they'd choose one over another
- Confidence signal: Engaged, articulate, well-prepared — but the preparation shows in certain areas (lots of polished answers to common questions)
The job: senior backend engineer at a 60-person fintech. The hiring bar: someone who can own a service end-to-end and make good architectural decisions without hand-holding.
Platform 1: HireVue (Async Video Screening)
HireVue's format — pre-set questions, recorded responses, AI-scored output — is the most widely deployed in enterprise hiring. We knew going in what it was optimised for: volume reduction at the top of the funnel for structured, comparable roles.
What it captured: Communication clarity. Response structure. Whether the candidate stayed on topic and hit expected themes. Our candidate's polished preparation showed through immediately — their answers were well-framed, coherent, and hit the keywords a hiring manager would expect.
What it missed: Everything underneath the surface. The inflated ownership claim on the pipeline project was stated confidently and recorded without challenge. The shallow system design thinking never surfaced because no follow-up question asked "why" — only "what." The score HireVue produced reflected communication quality and answer completeness, which is a real signal. It just isn't the signal that predicts senior engineering performance.
Strong filter for cultural fit and communication. Zero signal on technical judgment. If your senior engineering bar is high, HireVue tells you who can talk — not who can build.
Platform 2: Codility (Technical Assessment)
Codility is built around code. Its core product is a timed coding challenge — implement a solution, pass the tests, get scored. We gave our candidate a medium-difficulty task involving API design and data handling.
What it captured: Implementation ability under time pressure. Our candidate did well here — their Python is solid and they understood the problem quickly. Codility correctly identified them as technically competent at the task level.
What it missed: Everything that isn't a coding task. There was no mechanism to surface the inflated resume claim — Codility doesn't ask about your resume at all. There was no evaluation of system design reasoning. And the time pressure format actively disadvantaged a candidate who thinks well but codes methodically — which is often true of good senior engineers. Codility has recently introduced Cody, an AI tool that observes how candidates use AI assistants during the assessment. That's genuinely interesting — but it's still a code-correctness frame, not a thinking-quality frame.
The most reliable signal for "can they actually code." Tells you almost nothing about whether they should be making architectural decisions or leading a service.
Platform 3: Willo (Async Video — Candidate-Experience Focus)
Willo positions itself around the candidate experience — no-login, browser-based, low friction. We went through it specifically watching for what the format change (no-login vs. app-based) did to candidate behaviour.
What it captured: Very similar to HireVue in signal terms — recorded responses to structured questions, presented for recruiter review. The browser-based format genuinely reduced friction; our candidate reported the process felt more natural than HireVue's interface. Willo's AI surface provides a written summary of each response alongside the video, which reduces the time recruiters spend watching full recordings.
What it missed: The same things HireVue missed, for the same reasons. Async video without adaptive follow-up is structurally unable to probe. The candidate's confident surface presentation sailed through. The pipeline ownership claim was accepted at face value. The system design shallowness was invisible because no question was asked that would surface it.
Better candidate experience than HireVue, comparable signal. The format improvement is real but doesn't fix the deeper problem: a recorded monologue is still a monologue.
Platform 4: HackerEarth OnScreen (Autonomous AI Interviewer)
HackerEarth's OnScreen product is the most ambitious in the field in terms of what it's trying to do — an autonomous AI interviewer that conducts a full conversation, applies a rubric, and produces a structured assessment. This is the category closest to what a human technical interview is trying to accomplish.
What it captured: More than any other platform on this list. The rubric-applied scoring meant the assessment had structure beyond keyword matching. The conversational format meant follow-up questions were possible — and in some cases, the platform did probe deeper on a claim before moving on.
What it missed: The follow-up logic was rule-based rather than genuinely responsive. When our candidate made the inflated pipeline claim, the system asked a follow-up — but a pre-configured one about the technology stack, not a challenge to the ownership framing itself. It captured that the candidate knew the stack. It couldn't distinguish between someone who designed it and someone who used it. The confidence signal was picked up better here than on any other platform — structured communication and commit speed were reflected in the output. But depth probing on system design reasoning remained limited.
The most complete assessment of the five, with real structured scoring. The gap between its follow-up logic and genuine adaptive probing is where the remaining signal lives.
Platform 5: myInterview (Async Video + AI Summaries)
myInterview's differentiator is its AI-generated written summaries alongside video responses — designed to let reviewers skim rather than watch. We were specifically looking at whether the summarisation introduced or removed signal.
What it captured: The summaries were accurate — they faithfully represented what the candidate said. For recruiters doing a first-pass triage across dozens of candidates, the summarisation genuinely saves time. The video itself preserved the energy and delivery signals that async video captures.
What it missed: Summarisation of a surface-level answer produces a surface-level summary. The AI didn't flag what wasn't said — the pipeline claim appeared in the summary as stated, with no signal that it warranted scrutiny. The most important gap isn't what the summaries said incorrectly; it's that they created an impression of assessment rigour that the underlying data didn't support. A recruiter reading a clean AI summary of a confident but shallow system design answer might advance a candidate a human reviewer watching the video would have hesitated on.
Genuinely useful for time-pressed recruiters at the triage stage. But automated summaries can create false confidence — the signal quality of the summary is bounded by the signal quality of the question, and neither was challenged.
The Pattern Across All Five
Every platform captured something real. None of them captured the thing that actually matters for a senior technical hire.
| Platform | Strongest Signal | Critical Blind Spot |
|---|---|---|
| HireVue | Communication quality | Technical judgment; ownership authenticity |
| Codility | Implementation ability | Architectural reasoning; resume accuracy |
| Willo | Candidate experience, response coherence | Same as HireVue — no probing possible |
| HackerEarth OnScreen | Structured rubric scoring; some follow-up | Adaptive depth; ownership probing |
| myInterview | Triage speed via AI summaries | Summary confidence exceeds signal quality |
The shared blind spot across all five is some version of the same thing: none of them can do what a skilled human interviewer does when they hear something interesting and pull on it. Our candidate's inflated ownership claim passed through every single platform unchallenged. Their shallow system design reasoning was invisible to every platform that didn't specifically structure a question around it — and even the one that did couldn't probe with genuine adaptive follow-up.
Every platform we tested was good at what it was designed for. The problem is that none of them were designed for the thing that separates a good senior hire from a bad one.
What This Means for Hiring Teams
The practical takeaway isn't "don't use these tools." It's "know exactly what each one is telling you, and what it isn't."
Using HireVue or Willo to reduce 400 applications to 40 is a legitimate use of the technology — they do that well. Treating the output as a proxy for technical judgment or ownership authenticity is where teams get into trouble. The 31% of candidates who abandon applications because AI video screening feels impersonal are responding to something real: the format communicates, correctly, that no one is actually listening. As we covered in our analysis of candidate walkouts, that perception has measurable downstream effects on offer acceptance.
Codility and similar technical assessment platforms are honest about what they measure — code correctness — and for roles where that's the primary bar, they're the right tool. The error is using them as a proxy for senior engineering judgment, which they don't measure and weren't designed to.
The deeper question — whether any current AI tool can surface the signal that matters for senior technical hiring — is what tools like Ray by Diyam AI are explicitly trying to answer. We covered their approach in detail in our comparison piece. The short version: adaptive probing in a live AI conversation gets meaningfully closer to the signal these five platforms leave on the table.
But the more important lesson from this exercise is the one that applies to any tool you're currently using: map what it actually measures, not what the marketing says it measures. They're rarely the same thing — and the gap between them is where your hiring mistakes live.
Sharingan AI evaluates recruitment technology through independent testing and analysis. No vendor sponsorships. No affiliate links. Just what the tools actually do.