All customers
NVIDIASemiconductors

NVIDIA found the systems engineers its Boolean strings kept missing

Kernel and compiler work hides behind a hundred different job titles. Widening the search string surfaced more noise rather than more people, until the unit of matching changed from a keyword to a description of the work.

2.7xMore qualified profiles surfaced
41%Of hires missed by the previous string
9 daysOff the median time to first interview

The problem

Low-level systems work does not have a settled job title. The same person, doing the same job, is a Compiler Engineer at one company, a Kernel Developer at the next, and a Member of Technical Staff at the third. NVIDIA's sourcing strings had grown to accommodate this, and the growth was not helping.

Adding another OR clause solved the specific miss somebody had noticed and did nothing for the class of misses it belonged to, because the team was enumerating a vocabulary with no fixed size. Meanwhile every widening pulled in more people who had listed CUDA on a skills line after a weekend tutorial.

The measurement that settled the argument

Rather than debate it, the team ran a check. They took twelve strings written by experienced sourcers, ran them against a pool where a panel had manually labelled who was genuinely qualified, and measured what came back.

42%Of the qualified pool the median string found
2.7xMore qualified profiles after the change
41%Of eventual hires the old string missed

The sourcers, shown the misses afterwards, agreed with the label in almost every case. The strings were not badly written. They were being asked to do something a string cannot do.

What replaced the string

Two things, neither of which is "natural language search" as a marketing phrase.

The first is a representation that survives paraphrase, so that "ran production Kubernetes" and "operated containerised workloads at scale" land in the same place. This is what embeddings are genuinely good at, and it is why the Platform Engineer nobody thought to type now appears.

The second is a judgement step over what retrieval returns. Broad retrieval on its own trades a precise-and-narrow tool for a fuzzy-and-wide one, which is not obviously an improvement. Something has to read each profile against the actual requirements and say why it does or does not fit.

Where Boolean stayed

NVIDIA kept operators for the things that are genuinely binary. Work authorisation, a hard location boundary, an export control constraint. Handing those to a language model is worse in every respect: slower, more expensive, and less predictable than an index lookup that has been correct since 1974.

The right shape turned out to be a filter for the things that are actually filters, and a description for the things that are actually descriptions.

Keep reading

More customer stories

The future of recruiting is here.

Source
Screen
Sequence
Interview
Recruiters using Arbi on a laptop