A face recognition system is an AI-based technology that detects a face in an image or video, extracts its biometric features as a numerical vector, and compares that vector against a reference to confirm or establish a person’s identity. The technology is used in access control, time and attendance tracking, video surveillance, banking, retail, and smartphones — the range of applications is far wider than most people assume. Below: how the whole process works step by step, what FAR and FRR mean, why accuracy can’t be expressed as a single number, and what real limitations the technology has.
What a Face Recognition System Is
At its core, a face recognition system solves one task: compare a face in an image against an already-known reference and produce a decision — match or no match. Everything else — detection, alignment, computing the feature vector — consists of steps that make that comparison technically possible and reasonably reliable.
It’s worth separating two closely related but distinct concepts right away: face recognition (comparing a face against a reference) and face detection (simply finding a face in the frame, without determining identity). These two are confused more often than anything else, and it’s the first thing worth clarifying before getting into the technology itself.
How Face Recognition Works: From Photo to Result
Technically, the face recognition process is a sequence of steps, not one “magic” action performed by a neural network:
- Face detection — the system finds the region of the frame where a face is located.
- Alignment — the face is normalized to a standard position based on key landmark points.
- Feature extraction (embedding) — the neural network converts the face into a numerical vector.
- Comparison — the vector is matched against one or more references.
- Threshold — the degree of similarity is checked against a set boundary.
- Result — the system produces a decision: match or no match.
Face Detection
Face detection answers a simple question: is there a face in the frame, and exactly where is it? The algorithm looks for an image region that resembles a face and outlines its boundaries — a purely geometric task, with no judgment about the person’s identity involved. A system that only detects faces can’t say who’s in front of the camera: it can only find faces as a class of objects, the same way, for example, a smartphone camera’s autofocus does.
Alignment and Creating the Biometric Vector
A detected face is rarely positioned perfectly — it can be turned, tilted, or shot at an angle. Alignment normalizes its position using characteristic points (landmarks) — the eyes, the tip of the nose, the corners of the mouth — so that the subsequent comparison happens in a single coordinate system, regardless of the incidental camera angle.
After alignment, the neural network converts the face into a biometric vector (embedding) — a set of several hundred numbers describing the unique characteristics of that specific face. It’s an abstract numerical representation designed so that similar faces produce similar sets of numbers, while different faces produce substantially different ones. The vector is not a photograph and does not store the original image in its original form. It cannot, however, be treated as irreversible: inversion attacks can reconstruct an approximate image or identifying features, which is why the vector requires protection as sensitive data.
Comparison and the Decision Threshold
Next, the two vectors — from the captured face and from the reference — are compared using one of two approaches: a similarity measure or a distance measure. With the first, a higher value generally means greater similarity; with the second, a smaller distance does. Which metric a given implementation uses varies, and the result needs to be interpreted with that choice in mind.
This is where the threshold — the decision boundary — comes in. The system doesn’t “know” the correct answer in advance: it compares the computed degree of similarity against a predefined boundary and, based on that, decides whether to treat the faces as matching or not. Changing the threshold isn’t a minor technical detail — it’s a direct trade-off: a stricter threshold lowers the chance of letting an impostor through but raises the chance of wrongly rejecting a legitimate person, and vice versa. That trade-off is the subject of the next section.
1:1 Verification and 1:N Identification
Face recognition is applied in two fundamentally different modes, and their metrics and accuracy expectations shouldn’t be confused.
1:1 verification answers the question: “Is this the person they claim to be?” The system compares a face against one specific reference — for example, a passport photo or a previously stored picture of an account holder. Typical examples: KYC checks during remote identification, logging into an app by face, and access control when a person’s identity has already been declared (say, by a badge).
1:N identification answers a different question: “Is there a suitable candidate in the gallery, and who might it be?” Here, the system compares a face against many references — a database sometimes called a gallery — and returns a candidate or a ranked list of candidates whose similarity exceeded the threshold, or reports that no match was found. This is not a guaranteed identity determination but a result that is then verified according to the specific system’s own scenario. Typical examples: searching for a person against a watchlist, video surveillance in public places, and searches against a pre-registered database of employees or visitors.
The difference has real practical consequences: the larger the reference database in 1:N identification, the higher the chance of a random false match — with the exact same technology and the exact same threshold. That’s precisely why evaluating a system’s accuracy “in general,” without specifying the mode and the database size, is meaningless.
How Accurate Are Face Recognition Systems
It’s worth dropping the habit of looking for a single “X% accuracy” figure right away — that doesn’t work for face recognition, much as the accuracy of a single medical test doesn’t mean much without stated conditions. Any accuracy metric only makes sense together with three things: which type of error it is, at what threshold, and on what dataset it was measured.
FAR and FRR
FAR (False Acceptance Rate) — the probability that the system mistakenly accepts the wrong person as the intended one: it registers a match where there genuinely isn’t one.
FRR (False Rejection Rate) — the probability that the system mistakenly rejects a genuine, legitimate user: it fails to recognize someone it should have recognized.
These two types of errors are linked through the threshold, and the relationship is inverse: lowering the threshold makes the system more “lenient” — it rejects legitimate users less often (FRR drops) but lets impostors through more often (FAR rises); raising the threshold works the other way. Choosing a threshold is always a deliberate trade-off for a specific scenario: for bank onboarding, it’s usually more important not to let an impostor through (low FAR), even at the cost of having to manually re-check some legitimate users.
NIST terminology uses related but more strictly defined comparison-level metrics: FMR (False Match Rate) and FNMR (False Non-Match Rate). They coincide with the applied FAR/FRR only in the simple “one attempt — one comparison” scenario; at the transaction level, where there may be multiple samples or attempts, FAR/FRR and FMR/FNMR values can diverge. TAR (True Acceptance Rate) is the share of correct matches; in that same simple binary scenario, it equals 1 − FNMR.
| Applied term | NIST term | What it means |
|---|---|---|
| FAR | FMR (a related comparison-level metric) | Share of impostors wrongly accepted |
| FRR | FNMR (a related comparison-level metric) | Share of legitimate users wrongly rejected |
| — | TAR | Share of legitimate users correctly accepted (in the simple scenario, 1 − FNMR) |
A detailed look at how these metrics are used in practice when comparing algorithms is in the article Face Recognition Accuracy in KYC: FAR/FRR Metrics and NIST FRVT.
What NIST Testing Shows
Independent evaluation of face recognition algorithms is run by NIST (the National Institute of Standards and Technology, USA) — the program was historically called FRVT (Face Recognition Vendor Test), and its current name is FRTE (Face Recognition Technology Evaluation). Testing is split into tracks: 1:1 Verification and 1:N Identification — just as in real-world use, these are different tasks with different metrics. One of the key metrics is FNMR at a fixed FMR level, for example FNMR@FMR=1e-4 (the share of false rejections at one false match per ten thousand comparisons).
NeuroVision’s Enface face recognition algorithm undergoes testing in NIST FRTE 1:1 Verification. Participation in independent testing is itself a meaningful fact, but the result of any specific run is tied to a specific algorithm version, a specific dataset, and a specific FMR level — NIST testing does not produce a single, universal “accuracy” figure that could be quoted as one number without that context.
NIST separately studies accuracy differences across demographic groups (demographic effects). Modern algorithms have improved noticeably in this respect compared with earlier generations, but the spread of results depends heavily on the specific algorithm and testing conditions — a blanket statement like “face recognition is biased” is inaccurate without specifying exactly which algorithm and dataset is being discussed.
What Accuracy Depends On
Recognition accuracy in a real-world scenario depends not only on the algorithm but also on capture conditions and data quality:
- lighting — low contrast or strong overexposure degrades frame quality
- resolution and the size of the face within the frame
- the angle and rotation of the head relative to the camera
- motion blur from movement
- the age of the reference photo — a person’s face changes over time
- glasses, headwear, medical masks, and other face occlusions
- facial expression
- image compression, which loses detail
- the quality of the reference image itself being compared against
- the size of the database (gallery) in 1:N identification — the more references it holds, the higher the risk of a random match
- the chosen threshold
A good algorithm doesn’t guarantee the same result under any conditions — the quality of the lighting and camera at a specific installation point often affects final accuracy just as much as the choice of algorithm itself.
Where Face Recognition Is Used
Face recognition is used far more broadly than banking identification — here are the main scenarios. The scenarios below are described from a purely technological standpoint. In Russia, their legal regime is set by 572-FZ: organizations are not permitted to maintain their own database of employee or customer faces for identification and authentication (Article 15) — the permitted paths are covered in the section on choosing a system.
Access Control Systems
In offices, industrial facilities, and restricted areas, face recognition replaces or supplements passes and access cards. Depending on the scale, this can be 1:1 verification (the employee is already identified by a card, and the face only confirms identity) or 1:N identification against a small database of authorized employees.
Time and Attendance Tracking
Recording entry and exit by face automates the time and attendance log and reduces the risk of clocking in “for a colleague” — a typical weakness of card- or PIN-based systems — especially when recognition is combined with liveness/PAD and organizational controls; it does not guarantee the complete elimination of matching errors or workaround schemes.
Banking and Fintech
One application scenario — not the only one, and not central to the technology itself — is remote customer identification during onboarding (KYC), in-app login authentication, and anti-fraud checks. This is where some of the strictest accuracy requirements apply, since an error translates directly into financial risk. For more on how face recognition fits into the KYC process, see Biometric Face-Based KYC: How Recognition Improves Onboarding Accuracy and Security.
Retail
In retail, face recognition is used to find previously flagged offenders against a watchlist (loss prevention) and for loyalty programs that identify customers by face. In Russia, running either scenario on a store’s own biometric database is prohibited under Article 15 of 572-FZ, and a customer’s consent does not lift that prohibition (more detail in the section on choosing a system). This should be distinguished from anonymous visitor video analytics, which counts people and builds aggregated traffic statistics without establishing any specific individual’s identity — these are different technologies with very different levels of privacy intrusion.
Video Surveillance and Security
A classic 1:N identification scenario is searching camera video streams for matches against a wanted-persons database or watchlist, and perimeter control at guarded facilities. Screening and checks at transport hubs and airports fall into the same category, where confirmed scenarios match a passenger against a document or database at boarding.
Smartphones
Unlocking a phone by face (Face ID and similar systems) is the most widespread and easily understood example of 1:1 verification: the device compares the face in front of the camera against the single stored reference of the owner.
An accurate face-matching algorithm does not, on its own, protect against spoofing — companies building a customer verification loop separately need a liveness and anti-spoofing layer
Face Recognition, Liveness, and Deepfakes
Recognition and liveness solve different problems, and that’s worth understanding in advance, before an incident occurs. Face recognition answers the question “does the presented face match the reference?” Liveness answers a different question: “is there a live person in front of the camera right now, or a spoof?”
The recognition algorithm itself does not check whether there’s a live person in front of the camera: it compares whatever image it received against the reference, regardless of that image’s source. This means recognition without a separate liveness layer is vulnerable to a photo, a video played on a screen, a silicone mask, or a synthetic image — a deepfake — being presented to it. For how the live-presence verification technology itself works, see Liveness Detection from NeuroVision.
A separate threat involves substituting the incoming video signal: an attacker can technically replace the camera feed with a pre-prepared or generated one, and the recognition algorithm receives a synthetic image instead of a real person, without ever “noticing” the difference at its own level of the task. This is a distinct class of attack from presenting a photo or mask to the camera: liveness/PAD specifically counters presentation attacks, while defending against video-stream substitution requires different measures — controlling the capture source, session integrity, and specialized injection detection; standard PAD may not catch this kind of substitution. That’s why recognition is typically paired with several layers at once, rather than a single universal anti-spoofing mechanism. A detailed breakdown of this scenario is in the article Deepfakes in KYC: Detecting Face Spoofing and Protecting Video Verification, and specific ways of bypassing liveness checks are covered in How Liveness Detection Is Bypassed: Presentation Attacks, Replay, and Virtual Cameras.
Limitations and Common Myths
- “The camera always knows who’s in front of it.” No — without a reference database, the system can at best detect a face; it cannot name an identity.
- “99% accuracy means one error in a hundred.” Without stating which metric this is (FAR or FRR) and at what threshold, that figure says nothing about the system’s real-world behavior.
- “Any photo works for recognition.” Quality, angle, lighting, and the age of the picture directly affect the comparison result.
- “Face recognition automatically checks that a live person is in front of the camera.” No — that’s a separate task, liveness.
- “Face recognition protects against deepfakes.” On its own — no: without an anti-spoofing layer, it compares whatever it received without checking the source of the image.
- “All recognition algorithms are equally accurate.” Different algorithms vary substantially in accuracy and in robustness under difficult conditions — independent tests like NIST’s exist precisely so this can be compared.
What Businesses Should Consider When Choosing a System
When choosing a face recognition system for a business task, there are several practical questions worth settling before deployment — not after the first incident.
First, determine the mode: 1:1 verification or 1:N identification — this determines the expected metrics and the permissible database size. Second, deliberately choose the FAR/FRR balance for the specific scenario: the security requirements for a payment and the comfort requirements for an employee at a turnstile are different. Third, rely not on a marketing accuracy percentage but on independent tests like NIST — with an understanding of exactly which metric and conditions they refer to. Fourth, separately budget for liveness and anti-spoofing — without this layer, even an accurate face-matching algorithm does not protect against a photo, video, or deepfake being presented to it. And finally, account for the legal context: under 152-FZ (Article 11), a facial image becomes biometric personal data specifically when it is used to establish identity — not every photograph automatically acquires that status. 572-FZ adds a special regime: biometric identification of a person is permitted only through the EBS (Article 9), and authentication — through the EBS or through the systems of accredited organizations operating on EBS vectors (Articles 16–17; the industry calls these commercial biometric systems, or CBS, though the law itself does not use that term). Organizations are not permitted to maintain their own database of customer or employee faces for these purposes (Article 15), including for access control systems and time and attendance tracking, and a person’s consent does not lift that prohibition. For entry onto premises, the law sets out the path directly: critical infrastructure facilities and restricted-access sites — through the EBS; other organizations — through regional EBS segments or accredited systems operating on EBS vectors (Article 13). The law does not apply to state functions (defense, security, law enforcement, migration records, and others) or to cases where a match against a photograph is verified by an official rather than an algorithm (Article 1(2)); automatic recognition at a turnstile does not fall under this exception.
In practice, face recognition usually doesn’t work in isolation but as part of a broader identification loop — for example, NeuroVision KYC combines face matching with a liveness check within a single process.
We’ll help you choose between verification and identification mode and tune the decision threshold for your task — from access control to remote customer identification
A face recognition system isn’t a single “smart eye” — it’s a sequence of technical steps: face detection, alignment, conversion into a biometric vector, comparison against a reference, and a decision based on a set threshold. Detection and recognition, as well as 1:1 verification and 1:N identification, are different tasks with different metrics, and shouldn’t be conflated. The accuracy of such a system can’t be captured in a single number: any result only makes sense together with the type of error (FAR or FRR), the chosen threshold, and the testing conditions — and that applies to independent tests like NIST’s as well. Real-world accuracy at a specific site further depends on lighting, camera quality, angle, and the quality of the reference photo just as much as on the algorithm itself. For a business, this means choosing a system is about choosing the right balance for a specific scenario, not chasing the highest percentage in marketing materials. And it’s worth keeping in mind separately that face recognition and liveness checking solve different problems: an accurate face-matching algorithm, on its own, does not protect against a photo, video, or deepfake — that requires a separate layer of protection.
FAQ
