Metrics and Constraints for Tuning Anti-Fraud Without Increasing False Blocks
Setting up an anti-fraud loop starts not with rules and thresholds but with defining what counts as success and failure. Without clear metrics and constraints, the team optimizes the system blindly: it reduces fraud at the cost of mass rejections of legitimate users or, conversely, lets fraud through for the sake of a smooth customer journey. Before changing scoring, adding signals, or moving thresholds, it is necessary to fix measurable boundaries of what is acceptable and build a control loop that will immediately show if changes make the situation worse.
What Counts as a False Block in Onboarding, Login, and Transactions
A false block (false positive) is a situation in which an anti-fraud system rejects or stops an action by a legitimate user, mistakenly classifying it as fraudulent. At different points of the customer journey, a block manifests differently and causes different damage.
At the onboarding and KYC stage, a false block is a refusal to register or verify a real customer. The cause may be an erroneous document recognition, an incorrect comparison of a selfie with the passport photo, a liveness check failure, or a rule firing that does not account for a legitimate scenario. According to industry research, 38 to 68% of users do not return to the registration process after a failed verification attempt. Every falsely rejected customer means lost acquisition costs (CAC), forgone LTV, and reputational risk.
At account login and access recovery, a false block looks like a forced request for additional authentication, a temporary account lock, or denial of access when a device, network, or geolocation legitimately changes. The damage here is not only churn: if login to a financial service is blocked, the customer may miss a payment deadline, lose access to funds, or contact customer support, increasing operating costs.
When confirming transactions and profile changes, a false block is a rejected payment, a blocked transfer, or a stopped change of payment details for a bona fide user. According to estimates from analytical sources, 2 to 10% of declined e-commerce transactions are legitimate. About 40% of users who encounter a false payment decline never return to that merchant, which makes the cost of one false positive comparable to, or even higher than, the cost of a missed fraudulent transaction.
It follows that a false block must be defined and measured separately for each stage — onboarding, login, transactions — with its own acceptable thresholds. A single “rejection rate” metric for the whole loop hides the real picture and does not allow rules to be tuned precisely.
Measuring false blocks separately at onboarding, login, and transactions is the basis of precise tuning, but implementing such granularity without a suitable platform is difficult. We will deploy the NeuroVision KYC subsystem with configurable verification scenarios for each stage: the composition of steps, the set of required documents, scoring thresholds, and routing rules are set separately for registration, authentication, and transaction confirmation. The platform aggregates the results of 40+ anti-fraud algorithms and returns structured scoring with decision reasons to your backend — you will be able to track FPR and conversion for each decision point, rather than a “rejection rate” for the whole loop. Connection via REST API and SDKs for Web, iOS, and Android takes from 24 hours for the API to 3–7 days for full implementation, depending on the environment and information security requirements. Leave a request — together we will determine which scenarios and thresholds your business needs.
How to Set the Target Balance Between Fraud Losses and Rejection Losses
An anti-fraud system works under a trade-off: tightening rules reduces missed fraud (false negatives) but increases the number of false blocks (false positives). Loosening the rules does the opposite. The task is to find the point where the business’s total losses are minimal.
For this, a loss function is used that converts both types of errors into comparable units — as a rule, into money. Losses from missed fraud are made up of the amount of damage from the fraudulent transaction, investigation costs, regulatory fines, and reputational costs. Losses from a false block include the forgone revenue from the declined transaction, the cost of acquiring the customer who left, and the costs of manual review and support requests. Once both types of losses are expressed on one scale, it becomes possible to calculate the optimal decision threshold — the one at which the total cost of errors is minimal.
The loss function depends on the business model. In a low-margin service, one missed fraudulent transaction requires hundreds of successful transactions to compensate, so the acceptable share of false negatives is minimal, even at the cost of a moderate increase in false positives. In a high-margin business, each falsely rejected customer costs more than an individual fraud case, and the priority shifts toward reducing false blocks.
The target balance is set not as a fixed number but as a set of constraints:
- The acceptable level of fraud losses — the maximum share of fraudulent transactions or the amount of damage the business is prepared to accept over a period. The indicator is tied to regulatory requirements, payment system conditions, or the internal risk management policy.
- The acceptable level of false blocks — the maximum share of legitimate actions the system can reject without a critical impact on conversion and revenue. For onboarding, the threshold is stricter than for transactions, since losing a customer at the entrance wipes out all future revenue.
- The acceptable cost of manual review — the budget for reviewing ambiguous cases, which limits the share of decisions sent to manual review.
These constraints are revised when the business model, the threat landscape, regulatory requirements, or traffic volumes change. The balance is determined by the specific business and its current context. The requirement is that it be fixed, agreed between teams (anti-fraud, product, finance, compliance), and revised regularly, not after an incident.
Which Control Metrics Capture Deterioration After Changes to Rules and Thresholds
Every change in the anti-fraud loop — a new rule, a shifted threshold, an updated model — potentially affects the balance between fraud detection and the level of false blocks. To detect degradation in time, you need a set of control metrics that are tracked in near real time and compared with baseline values from before the changes.
False Positive Rate (FPR) — the share of legitimate actions mistakenly rejected by the system. It is calculated as the ratio of false blocks to the total number of legitimate events. It is the main indicator that tightening the rules has caught bona fide users. In AML transaction monitoring, the industry-average FPR reaches 95% of triggered alerts, while leading organizations manage to reduce it to 30–50%. In anti-fraud at onboarding and in transactions, target values are usually much lower and are determined by the specific scenario.
False Negative Rate (FNR) — the share of fraudulent actions missed by the system. A rise in FNR after changes means the system has become less sensitive to real threats.
Precision — the share of confirmed fraud cases among all blocked ones. Low precision means the system generates a lot of noise: it blocks often but rarely hits the target. A drop in precision after a rule update is a signal that the new conditions are too broad.
Conversion at control stages — the percentage of users who successfully complete onboarding, login, or a transaction. The metric links anti-fraud indicators to the business result. A decline in conversion with a stable fraud level indicates an increase in false blocks, even if FPR has formally changed only slightly.
Volume and share of manual review — the number and percentage of events sent for manual review. A sharp increase after changes indicates that the system has stopped making confident decisions and is shifting the load to operators.
Average time to decision — the time from the user initiating an action to the system’s final decision (approve / reject / review). An increase worsens the user experience and may increase the share of rejections at stages where speed matters.
Distribution of scores — a histogram of risk scores before and after changes. A shift of the distribution toward higher or lower values helps detect a systemic bias that is not always visible in aggregate metrics.
These metrics work as an early warning system provided that baseline values and acceptable deviation thresholds are defined for each of them in advance. A deviation beyond the set corridor after changes is a trigger for analysis and, if necessary, for rolling back to the previous configuration. Without such a control loop, even competent rule tuning turns into an experiment without feedback.
The Customer Journey Map and the Places Where a Decision Layer Is Needed
The anti-fraud loop is embedded in specific points of the user’s interaction with the service. If checks are placed where the real risk does not arise, the result is predictable: fraudsters go unnoticed, or legitimate customers face excessive friction and leave. The task is to determine at which nodes of the customer journey a decision layer is needed and what level of checking is adequate for each node.
Decision points are grouped around three phases: a new customer entering the system (onboarding and KYC), repeat access to the account (authentication and recovery), and actions inside the account (transactions and profile changes). Each phase has its own threats, its own acceptable levels of friction, and its own cost of error.
Decisions at Onboarding and KYC
Onboarding is the first and often the only opportunity to cut off a fraudster before they gain access to the service’s resources. This is where the baseline level of trust in the customer is established, and this is where the cost of missed fraud is highest: a fake account that has passed KYC becomes a springboard for all subsequent abuse — from money laundering to social engineering.
The decision layer at this stage covers several successive checkpoints.
Document intake and validation. The system determines the document type, extracts the data, and checks its internal consistency: whether the MRZ matches the visual zone, whether the check digits are correct, and whether the expected security features are present. Already at this step, the first decision is made — whether the quality of the input data is sufficient to continue or a repeat capture is needed. If the document is recognized but signs of digital processing or template mismatches are detected, the risk scoring records a signal, and the user’s route may change: a request for an additional document, redirection to manual review, or a soft refusal with an explanation of the reason.
Biometric verification and liveness check. Comparing a selfie with the photo from the document (face matching), combined with a liveness check, answers two questions: Is this person the owner of the presented document? Is there a live person in front of the camera, rather than a photo, a video recording, or a generated image? At the anti-fraud loop level, the face match score and the PAD (Presentation Attack Detection) result feed into the session’s overall risk profile and affect further routing.
Checks against external sources and compliance screening. With a risk-based approach, the scope of external checks depends on the accumulated risk score. A customer with clean documents, a high face match score, and a typical device can pass basic AML screening (sanctions lists, PEP, terrorist lists). A customer with an elevated score gets an extended package: checking full name–phone number links, credit scoring, court debts, and, in some jurisdictions, verification through state registers. The principle: do not request the maximum set of data from every customer, but increase the depth of checking in proportion to the risk detected. This reduces the cost of verification and lowers friction for most legitimate users.
Final decision and routing. Based on all the steps, the rules engine produces one of the decisions: automatic approval, referral to manual review by an operator, or refusal. In mature systems, instead of a binary “yes/no,” an intermediate zone is used — conditional approval with restrictions: reduced limits, deferred access to certain functions, a request for additional confirmation within a set period. This approach makes it possible not to lose customers with a borderline risk profile while limiting potential damage.
Decisions at onboarding affect all subsequent work with the customer. If the system retains the risk profile formed at registration, it becomes the starting point for scoring at login and in transactions. Data quality at the KYC stage is not only a compliance task but also the foundation for the accuracy of anti-fraud decisions at all subsequent stages.
Decisions at Login and Access Recovery
After onboarding, the user returns to the system dozens and hundreds of times. Each authentication is a point at which an attacker may try to take over someone else’s account (account takeover, ATO). According to industry reports, ATO is among the three most common types of online fraud in the financial sector. At the same time, friction at each login directly affects retention: a service that regularly demands excessive confirmations loses users.
The decision layer at authentication works differently than at onboarding. There is no task of verifying identity from scratch — it is necessary to make sure that the user logging in matches the previously verified profile. This means quickly comparing the context of the current session with the norm for the specific account.
| Category | Description |
|---|---|
| Session context assessment | At each login, the system collects a set of parameters: the device (fingerprint, model, OS), the network (IP address, provider, geolocation), the login time, and behavioral signals (typing speed, navigation patterns). This data is compared with the account’s history. A login from a familiar device, from a familiar geolocation, at a familiar time — low risk, minimal friction. A login from a new device from another country at an unusual time — elevated risk, activation of step-up authentication. |
| Adaptive authentication | The principle of risk-based authentication is that the level of checking is proportional to the risk level of the current session. At low risk, a standard factor is sufficient (password, on-device biometrics). At medium risk, the system requests an additional factor: a one-time code via SMS or push notification, confirmation via email, or server-side biometric verification. At high risk, a temporary session lock with escalation to an operator check is possible. The decision must be made automatically and in real time: a login delay noticeable to the user should not exceed a few seconds. |
| Access recovery | Password reset or access recovery is one of the points most exploited by fraudsters. An attacker who has gained control of the victim’s email or SIM card can go through the standard recovery procedure and hijack the account. Therefore, access recovery requires a separate set of rules in the decision engine: repeat biometric verification with liveness detection, document verification (especially for accounts with high limits), and temporary restrictions on transactions after the reset (cooling-off period). For example, after a password change, the account may be restricted from transfers for 24–48 hours — enough for the real owner to notice the unauthorized action and react. |
The task of the decision layer at login is not to get in the user’s way in the 95–98% of cases when the login is legitimate, and to fire reliably in the remaining 2–5% when the session parameters deviate from the norm. Achieving this balance requires constant calibration: of trigger thresholds, signal weights, the length of cooling-off periods, and the conditions for activating step-up checks.
Decisions When Confirming Transactions and Profile Changes
Even if the user has passed KYC and successfully authenticated, individual actions inside the account carry their own risk. A transfer of a large amount, a change of the linked phone number, adding a new payee, changing limits — each of these actions may indicate fraud, especially if it is performed for the first time or deviates from usual behavior.
Transaction control. Every transaction involving the movement of funds or valuable assets passes through the risk assessment layer. The rules engine analyzes the transaction parameters (amount, recipient, channel, time) in the context of the account history and the current session. A typical transaction that fits the behavior profile goes through without additional friction. A transaction that deviates from the profile — in amount, recipient geography, frequency, or time of day — triggers a step-up check: a request for an additional confirmation factor, a temporary delay for analysis, or escalation to an operator.
Rule granularity is important here. Blocking all transactions above a fixed threshold is a crude tool that generates a mass of false positives. Dynamic thresholds tied to the profile of a specific user work more precisely: for one customer, a transfer of ₽50,000 is a routine transaction, for another, an anomaly. This approach requires the system to accumulate and update a behavioral profile, and the rules engine to be able to access it with each decision.
Changes to the profile and critical parameters. Changing the phone number, email address, password, payee details, or personal data are actions fraudsters take to consolidate control over a hijacked account. Each such change should go through a separate decision branch. Practices that have proven effective: mandatory verification via the previous communication channel before switching to a new one (confirmation to the old number before linking a new one), a cooling-off period after critical changes (restricting transactions for 24–72 hours), and notifying the owner through all channels (push, SMS, email) with the option of an emergency block.
The combination of changes as a separate signal. The anti-fraud loop should track not only individual actions but also chains. If within one session the user changes the password, links a new phone number, and immediately initiates a large transfer, each action on its own may not exceed the threshold, but their sequence forms a distinct account takeover pattern. A rules engine capable of analyzing sequences and correlations of actions within a time window detects such scenarios much more precisely than systems that assess each event in isolation.
The general principle for all three phases: the decision layer is not static. It should receive feedback — from investigation results and from false block metrics — and be adjusted based on real data.
Which Risk Signals to Use in Anti-Fraud
The quality of anti-fraud decisions is determined not by the number of checks but by which signals the system takes into account and how it combines them. A single isolated indicator — a suspicious IP or a name mismatch — rarely gives grounds for a block. False positives arise when a decision relies on a narrow set of parameters without regard to context. Missed fraud is a consequence of blind spots: the system sees the document but does not analyze the device; it records the geolocation but ignores the user’s behavior in the session.
A risk-based approach requires working with four groups of signals: biometrics and documents, the device and network environment, behavioral patterns, and account history and links between entities. Each group covers its own vulnerability zone, and accuracy is achieved at their intersection — when an anomaly in one layer is confirmed or refuted by data from another.
Signals From Document, Face, and Liveness Checks
Document checking, biometric comparison, and liveness detection form the first line of anti-fraud defense at onboarding and repeat authentication. These signals answer three questions: is the presented document genuine, does it belong to the person going through the check, and is that person physically present at the time of the session.
Document verification generates several types of risk signals. The first is structural validation: whether the layout, fonts, field placement, and security features (holograms, microtext, UV features) match the reference template for the specific document type and issuing country. AI-OCR extracts data from the machine-readable zone (MRZ) and the visual zone (VIZ) and compares them — discrepancies between the MRZ and VIZ are a typical marker of forgery. The second is image metadata: resolution, EXIF tags, traces of editing (pixel cloning, inserted fragments, mismatched compression levels in different areas of the frame). The third is document liveness: checking that the image was captured from a physical document, not from a screen, a printout, or a digital mock-up. According to industry estimates, up to 90% of fraud in digital onboarding is associated with document forgery or presentation attacks.
Biometric comparison (face matching) — matching the user’s selfie with the photo from the presented document. The risk signal is the match score: a value below the set threshold indicates possible identity substitution. At the same time, a low score may be the result of poor lighting or an outdated photo in the document rather than fraud. Therefore, the face match score is passed to the overall risk scoring as a weighted factor, not as the sole decision criterion.
The liveness check (liveness/PAD) counters presentation attacks: photos on a screen, replayed video, 3D masks, and deepfakes. According to industry reports for 2025, the number of attacks using fake selfies increased by 58% over the year, and injection attacks (video stream substitution at the API or camera driver level) grew by 40%. Liveness detection works in two modes. Passive mode analyzes skin texture, micro-movements, depth, and light reflection without user participation — this reduces friction and does not increase drop-off. Active mode requires the user to perform an action — turning the head, blinking, saying a phrase — and more reliably withstands advanced attacks. Adaptive systems combine both modes depending on the session’s risk level: for low-risk scenarios, a passive check is sufficient; at elevated risk, an active one is added.
Each of the three layers returns a set of attributes to the anti-fraud loop: binary flags (pass/fail), numeric scores, and text reasons for the decision. Together, they form a trust profile of the presented identity. The more layers confirm each other, the higher the confidence in legitimacy — and the fewer grounds for additional friction.
The three layers — document, face, and liveness check — form a trust profile only when their results feed into the session’s overall risk scoring rather than being processed in isolation. The NeuroVision platform combines AI-OCR for 16,000+ document types from 200+ countries with Enface biometric comparison (verification accuracy — 99.74%, NIST FRVT result — first place among Russian algorithms) and liveness/PAD with 99.9% accuracy, including countering deepfakes and injection attacks. Each module returns a numeric score and a set of reason flags, which are aggregated into a single risk profile and determine the route: automatic approval, step-up, or escalation to an operator. Document recognition takes less than 1 second, and face comparison less than 0.1 seconds, which makes it possible to make a decision without a delay noticeable to the user. The solution is available in the cloud, on-premises, or in a hybrid option — we will select an architecture for your information security and regulatory requirements.
Device, Network, and Session Geography Signals
The context of the environment from which the request comes often reveals anomalies earlier than is possible at the level of documents or the face. Fraudsters can present a genuine (stolen) document and a high-quality deepfake, but reproducing the normal digital environment of a real user is much harder for them.
The device fingerprint consists of dozens of attributes: device type and model, OS and browser version, screen resolution, installed fonts and plugins, canvas/WebGL rendering parameters, and audio context configuration. Together, these parameters create a unique identifier that is resistant to IP changes, cookie clearing, and switching between networks. Risk signals from this layer: detection of an emulator or virtual machine, a rooted/jailbroken device, use of an anti-detect browser, a mismatch between the declared model and the actual rendering parameters. Each of them is not an unconditional block but a factor that increases the risk weight in the overall scoring.
Network signals are the characteristics of the connection: the IP address, ASN (belonging to a provider or a data center), connection type (mobile network, Wi-Fi, wired), and the presence of a proxy, VPN, or Tor. Modern detection methods go beyond checking IPs against blacklists: they analyze mismatches between the device time zone and the IP geolocation, identify residential proxies (which disguise traffic as that of an ordinary home user), and detect signs of geolocation spoofing via GPS spoofing or the injection of fake Wi-Fi networks. A VPN connection does not in itself mean fraud — corporate users work through VPNs en masse. But a VPN combined with a new device and an anomalous session time is a weighty combination for raising the risk score.
Geolocation and “impossible travel” is a classic but still effective pattern. If between two sessions 30 minutes apart the user has “moved” from Moscow to London, this is a strong signal that the credentials have been compromised. Detection accuracy increases when GPS coordinates (for mobile apps), IP geolocation, and telecom operator data are compared. A reasonable approach is not to block instantly but to initiate a step-up check: request additional confirmation if the distance between login points is physically impossible to cover in the time that has passed.
For the practical application of these signals, device history matters: binding the fingerprint to the account and checking at each login whether the device is familiar. The first login from a new device is a reason for heightened attention, but not for a block. A repeat login from an already known device from a familiar location is grounds for reducing friction. With this arrangement, 96–99% of legitimate sessions go through without additional requests, and checks are concentrated on genuinely suspicious cases.
Behavioral Signals of Input and Pace of Actions
Behavioral biometrics analyzes not “what” the user does but “how” they do it. A fraudster may possess stolen credentials and a genuine document, but reproducing an individual’s manner of interacting with a device is a task of a different order of difficulty. Behavioral signals are especially valuable as an additional layer that works continuously throughout the session.
Keystroke dynamics captures typing speed, the rhythm of alternating key presses, key hold time (dwell time) and the intervals between presses (flight time), characteristic errors, and the frequency of hotkey use. For a new user, the system has no personal profile, but it can detect signs of a bot or automated input: a perfectly even rhythm, abnormally high speed, no pauses or errors. For a returning user, a deviation of the current pattern from the historical profile indicates a possible account takeover — even if the password was entered correctly.
Cursor movement and touchscreen interaction. Mouse trajectory, movement speed, the curvature of movements, and the click pattern form a unique behavioral signature. On mobile devices, touch pressure, swipe speed, preferred screen area, and gyroscope and accelerometer data (tilt angle, orientation, the way the device is held) are tracked. Bots and remote access tools (RATs) are given away by unnaturally straight cursor lines, the absence of micro-movements, and mechanically precise timing. A fraudster working through a remote desktop tool on the victim’s computer may have all the account data, but their digital handwriting will differ from the owner’s.
Pace and sequence of actions in the session is the macro level of behavioral analysis. This includes the time taken to fill in a form, the order of moving between fields, the number of corrections, and the method of data entry (typing manually or pasting from the clipboard). Pasting a full card number or passport series and number from the clipboard is atypical behavior: an ordinary user types their own data manually. A sharp acceleration or slowdown at the stage of entering payment details may indicate the use of a script or that the user is acting on external instructions (a characteristic sign of social engineering).
Behavioral signals are among the few that work continuously throughout the entire session and can detect user substitution, session hijacking, or a handover of control to malware in real time. At the same time, the user does not perform any additional actions — the data is collected in the background through an SDK or a JavaScript agent in the browser. This makes it possible to strengthen security and at the same time add no friction to the customer journey.
Account History, Repetition, and Links Between Entities
The signals from the previous three groups describe the current session. The fourth group works with accumulated data: the history of a specific account, the frequency of events, and links between users, devices, and data points. This layer makes it possible to detect organized fraud, synthetic identities, and schemes that look harmless within a single session but form a pattern at scale.
| Category | Description |
|---|---|
| Account age and history | A basic but informative signal. A recently created account performing a high-risk transaction (a large payment, a change of the linked phone number, a withdrawal of funds) carries significantly greater risk than the same action from an account with a months-long history of legitimate activity. An account that has long been inactive and suddenly “comes back to life” with atypical actions is a sign of a possible takeover (ATO). Useful for scoring: registration date, volume of past transactions, history of successful verifications, presence of confirmed contact details and payment instruments. |
| Velocity checks (frequency and speed control) | They track how many times a certain event has occurred in a given time window. Examples: the number of registration attempts from one device per hour, the number of transactions from one card per day, the number of different accounts using one phone number per week. Velocity rules detect testing of stolen cards (mass micropayments), automated account creation by bots, and promotion abuse (multi-accounting). To reduce false positives, velocity thresholds are calibrated to the real activity profile: the normal frequency of transactions differs radically between a marketplace and a SaaS platform. |
| Links between entities (entity linking / link analysis) | One of the most powerful tools for detecting organized fraud. Each user, device, email, phone, IP address, and payment instrument is a node in a graph. Edges between nodes arise when two accounts use one phone, one device fingerprint, a matching address, or overlapping payment data. Graph analysis detects clusters of linked accounts — fraud rings, mule networks, chains of synthetic identities. Each individual account in such a network may look legitimate: a correct document, a successful liveness check, a familiar device. But dozens of accounts linked through two or three common nodes are a clear marker of organized fraud. |
Link analysis is implemented in two ways. The first is a check at onboarding: a new account is compared against the base of existing ones on all key attributes (email, phone, device fingerprint, address, payment data). A match with accounts from a blacklist or with previously blocked accounts is a direct signal for rejection or manual review. The second is retrospective analysis: periodically recalculating the link graph from accumulated data to identify patterns that were not noticeable at the time of registration. For example, a fraud ring in which each account was created from a unique device, but a month later all the accounts begin performing the same kind of transactions through one IP range.
The combination of all four groups of signals — documents and biometrics, the device environment, behavior, history and links — forms a multidimensional risk profile. No group is self-sufficient: the greatest accuracy and the lowest level of false blocks are achieved through cross-correlation of signals from different layers. A multi-signal architecture allows the anti-fraud loop to make differentiated decisions — let through, request an additional check, or block — with a clear trace of which signals led to a specific decision.
Cross-correlation of document, biometric, environmental, and behavioral signals gives the lowest level of false blocks, but requires a platform capable of bringing heterogeneous data together into a single score with transparent tracing. NeuroVision combines document authenticity checks (AI-OCR, MRZ validation, editing detection), biometric comparison, and liveness/PAD with external checks: confirming full name–phone and full name–email links, checking the TIN and SNILS (individual insurance account number), and screening against sanctions lists, the Federal Bailiff Service (FSSP), and bankruptcy registers. The result of each check is structured JSON with a numeric score, risk flags, and decision reasons, which provides full tracing for incident analysis and audit. Up to 90% of cases go through without manual review, and disputed ones are automatically routed to the operator interface with the full set of collected signals and the grounds for triggering. We will audit your current set of signals and propose a loop configuration that will close blind spots without excessive friction.
Risk Scoring for Onboarding and Transactions
Risk signals collected at each stage of the customer journey do not make decisions on their own. They need to be combined into a single numeric assessment — a risk score — that determines what happens next: instant admission, an additional check, or a block. The quality of this assessment directly affects two indicators that usually conflict: the share of missed fraud and the share of false blocks.
The Risk Scale and Levels for Routing Decisions
A risk score is a number on a predefined scale that reflects the overall probability that a user or transaction is associated with fraud. The scale can be anything — from 0 to 100, from 0 to 1,000 — but what matters fundamentally is not the boundaries but how the scale is divided into levels (risk buckets), each of which corresponds to a specific processing route.
The minimum viable configuration includes three levels. Low risk — automatic approval without additional steps. Medium risk — a step-up check: a request for an additional document, repeat biometric verification, confirmation via a second communication channel. High risk — a block or handover to an analyst for manual review. As data accumulates and volumes grow, three levels may prove insufficient: intermediate zones appear for which a separate set of actions is optimal. Four to five levels is a common configuration in the mature anti-fraud loops of fintech services and banks.
The boundaries between levels are thresholds that set the operational balance of the system. Shifting the “medium → high” threshold down by a few points increases the number of blocks and manual reviews but reduces missed fraud. Shifting it up has the opposite effect. That is why thresholds are determined based on a loss function that takes into account the cost of each type of error — both financial and reputational.
Levels should be tied not to a number but to an action. If the “medium risk” level does not have a clearly defined route (which step-up, which checks, which timeout), it turns into a gray zone where decisions are made manually and unpredictably. This is a direct source of rising false blocks: an analyst without instructions tends to play it safe.
How to Combine Model Scoring and Rules Without Conflicts or Duplication
In a real anti-fraud loop, risk assessment almost never relies on a single source. In parallel, there is an ML model that produces a probabilistic score based on hundreds of features, and a rules engine that encodes hard constraints and heuristics based on expert knowledge. The task is to build their interaction so that they complement each other rather than conflict.
A conflict arises when the model and a rule evaluate the same signal but produce opposite results. Example: the ML model rated a session as low-risk (a score of 12 out of 100), but the rule “IP country does not match document country → medium risk” raises the level. If both decisions are simply added up, the final score is inflated — and a legitimate user working through a VPN gets blocked. If the rule has unconditional priority, the ML model is effectively zeroed out for an entire category of cases.
A robust architecture involves separating roles. The ML model is responsible for baseline scoring: it processes a large set of weak and medium signals, detects non-linear patterns, and produces a probabilistic assessment. The rules engine performs two functions. The first is applying hard constraints (hard rules), which act unconditionally and do not depend on the model’s score: a data match with a sanctions list or a confirmed fact of document compromise. Such rules work as an override — they intercept the decision regardless of the score. The second is corrective rules (soft rules), which modify the model’s score: they add or subtract points for signals that the model does not take into account or takes into account insufficiently. For example, if the model was trained on data from one region and the business enters a new market, a corrective rule can temporarily lower the score for that market’s patterns until the model is retrained.
To avoid duplication, each signal should be counted exactly once. If a feature is already among the ML model’s features, a rule should not penalize it again — except when the rule reacts to an absolute threshold value (for example, an account age of less than 1 hour), while the model processes this feature as one of many in a gradient.
A practical way to keep this under control is shadow mode. A new rule runs in parallel with the model but does not affect the decision. Shadow data is used to assess how many cases the rule would reclassify and what percentage of them are real fraud. If the rule creates more false positives than the losses it prevents, it is either adjusted or not enabled.
Calibrating the Score and Checking Quality by Segment
A model’s score is useful for decision-making only when it is calibrated: a value of 70 out of 100 should mean approximately the same probability of fraud for different user segments. Without calibration, the model may systematically overestimate risk for one segment and underestimate it for another, even with a high overall AUC-ROC.
Calibration is checked by comparing the predicted probability with the actual share of fraud in each score group. Scores are split into bins (for example, 0–10, 10–20, …, 90–100), and for each bin the real share of fraudulent events is calculated. If the model is calibrated, the points on the “predicted probability vs. actual share” chart lie close to the diagonal. Significant deviations signal the need for recalibration — using Platt scaling or isotonic regression, without retraining the model itself.
Checking quality by segment is a step that is often skipped. Aggregate metrics (precision, recall, FPR across all traffic) can mask a problem: the model works excellently on the main segment and critically poorly on a minority one. Typical cuts for segmentation: geography (country, region), device type and channel (mobile app, web version, API), account age (new vs. existing users), and transaction type (onboarding, login, payment, change of details).
For each segment, target metrics are fixed: acceptable FPR, recall, and manual review volume. If FPR in a segment goes beyond the acceptable threshold, this is a reason not for a general recalibration but for a targeted adjustment: shifting the threshold for that segment, adding an exception rule, or refining the model’s features.
Targeted adjustment of thresholds by geography, device type, or account age requires a tool in which rules are configured separately and the result is visible in real time for each segment. The NeuroVision platform back office provides configuration of scenarios with separate scoring thresholds, step-up conditions, and routing rules for each type of check, and dashboards show conversion, incidents, and the share of manual reviews by transaction. The platform processes up to 10,000 requests per minute, which makes it possible to collect statistically significant data for calibration even on high-load services. A full verification cycle — document, face, liveness, and AML screening — costs approximately ₽35–50 per check, depending on the connected modules and volume, and deployment is possible both in the cloud and in the customer’s secure environment. We will analyze your current traffic segmentation and agree on thresholds that take into account the real audience profile and the acceptable FPR for each segment.
How often to check depends on the dynamics of the data. For fintech services with high transaction volumes, weekly monitoring of key metrics by segment is a reasonable minimum. When external conditions change sharply (a new product launch, entry into a new market, a mass attack), the check is carried out immediately.
Decision Reasons and Signal Tracing for Analyzing False Blocks
Even a well-tuned system will generate false blocks. The question is not how to eliminate them completely (that is impossible without zeroing out protection), but how to detect, analyze, and eliminate their cause quickly. For this, each decision should be accompanied by a trace — a transparent chain of signals and rules that led to a specific score and action.
The minimum set of data in a trace: the final score and level; a list of signals indicating the weight of each; the triggered rules (hard and soft) with identifiers and conditions; the assigned action (approve, step-up, block, manual review); a timestamp and the identifier of the model version and rule set active at the time of the decision.
This information solves several tasks. When reviewing a complaint, an analyst determines within minutes why the user was blocked: whether the model, a rule, or their combination fired, and which signal was decisive. By aggregating decision reasons for all false blocks over a period, it is possible to identify a rule or signal that generates a disproportionate number of errors. Tracing is also critical for regulatory requirements: a number of jurisdictions require that an organization be able to explain the reason for refusal to a customer, and KYC/AML standards presuppose logging the grounds for every decision.
On the ML model side, explainability is provided through reason codes — codes ranked by their degree of influence on the final score. For models based on gradient boosting (GBDT), reason codes are formed based on the feature importance of a specific prediction. For more complex architectures, local interpretation methods are used — SHAP (SHapley Additive exPlanations) or LIME. The choice of method depends on the acceptable latency: SHAP on the full set of features may be too slow for real-time scoring, and then approximate calculations or pre-computed tables of the most frequent combinations are used.
In practice, a trace shows not all the hundreds of features but the top 5–7 signals that determined the decision. This is sufficient for prompt analysis and does not overload the analyst. If a false block was caused by a rule, the trace should contain a direct reference to the rule identifier — so that it can be found, its logic checked, its trigger statistics assessed, and it can be adjusted.
The absence of tracing turns the anti-fraud system into a black box that both customers and the company’s own team lose trust in.
The “Signal → Action” Matrix and Risk-Based Step-Up Verification
Risk scoring becomes a practical tool when each score range corresponds to a specific system action. Without a formalized “signal → action” link, operators make decisions by intuition, rules duplicate each other, and threshold values are set arbitrarily. An action matrix eliminates uncertainty — it sets deterministic logic: at which risk level which check is launched, which scenario is opened to the user, and when manual processing is brought in.
The principle of the matrix is proportionality of response. Low risk does not require additional friction: the user passes the check in seconds. Medium risk triggers targeted reinforcement — step-up verification, in which the system requests exactly the factor that closes the identified gap in trust. High risk moves the case to quarantine or manual review, but even here a block is a last resort. This approach preserves conversion for the bulk of users and concentrates the team’s resources on genuinely suspicious sessions.
Actions for Low, Medium, and High Risk
Division into three levels is the minimum viable routing scheme. There may be more levels, but the logic is the same: the higher the risk score, the deeper the check and the more confirmation points the user goes through.
| Risk level | Description |
|---|---|
| Low risk | Automatic approval without additional steps. The session passes the basic set of checks (document, face, liveness), all signals are within acceptable ranges, and the account history is clean. The user gains access or completes the transaction in one pass. The system’s task is to minimize latency and not create unnecessary screens. This scenario serves 70–85% of legitimate traffic, and its conversion determines the commercial effectiveness of the entire loop. |
| Medium risk | Launch of step-up verification. The system has detected at least one signal that reduces confidence: a geolocation mismatch with the usual pattern, a new device, a borderline liveness score, a suspicious pace of filling in a form. Instead of a block, additional confirmation is requested: repeat biometrics, an OTP to the linked phone, upload of a second document, address verification. The choice of a specific step-up is determined by which signal turned out to be borderline. |
| High risk | Suspension of the action and escalation. Several signals simultaneously point to fraud, or there are matches with known fraud patterns: a document with signs of forgery, a face from a blacklist, a device linked to previously rejected applications. The system puts the transaction or application on hold and passes the case to manual review with the full set of collected signals and trigger reasons. A full block is allowed only if the combination of signals leaves no room for a legitimate scenario — and this decision must be justified in the log. |
The boundaries between levels are not constants. They depend on the business context, the audience profile, and the acceptable ratio of fraud losses to losses from false rejections. Thresholds are set based on the loss function and revised with every significant change in rules or traffic.
Step-Up Verification and Risk-Based Adaptive Authentication
Step-up verification is a measured increase in the level of checking in response to a specific risk signal. Unlike a static approach, where all users go through the same chain of steps, adaptive authentication builds the check dynamically: the volume and type of requested factors are determined in real time based on the session profile.
The logic of adaptive authentication: request only what will close the identified gap. If the score is lowered because of a new device while all other parameters match, confirmation of channel ownership (OTP) is sufficient. If the liveness result is borderline, a repeat biometric check with a reinforced scenario is needed. If the document raises doubts, an alternative data source or a second document is requested. Each step-up factor should target the zone of uncertainty that the scoring recorded, rather than add checks “just in case.”
The criterion for a correct step-up: if an additional check does not reduce the uncertainty recorded by the signal, it is unnecessary. Excessive step-up reduces conversion without increasing security.
Reinforcing Checks of Documents and Data Sources
When the score is lowered because of document signals — borderline image quality, a data mismatch between fields, suspicious artifacts in the MRZ or security features — step-up is aimed at obtaining additional evidence of authenticity.
Typical reinforcement measures: requesting a second document of a different type (for example, if the main one was a passport — a driver’s license or an ID card); recapturing the document with quality requirements (no glare, in focus, the full spread); cross-checking the extracted data against external registers — state databases, credit bureaus, telecom operators (confirming the full name–phone link, checking the TIN or SNILS). If a discrepancy is found between the address in the document and the one stated at registration, confirmation of the address via a utility bill or a bank statement may be required.
Two types of document problems require different approaches. A technical problem (poor image quality, cropped corners, glare) is solved by recapture — the user is offered a reshoot with instructions. A substantive problem (data mismatch, signs of editing) requires an alternative source or escalation, since reshooting the same document will not eliminate the discrepancy.
Reinforcing Biometrics and the Liveness Check
Biometric step-up is activated when the result of comparing the face with the photo in the document is in the borderline zone, or when liveness/PAD has returned an inconclusive result. The reasons may be legitimate: poor lighting, a low-resolution camera, glasses, a change in appearance since the document was issued.
The first level of reinforcement is a repeat attempt with an active liveness scenario: the user is asked to perform certain actions in front of the camera (turning the head, blinking, saying a sequence of digits). This makes it harder to use static photos, pre-recorded videos, and masks. If the initial check was passive (a single frame), moving to an active model increases resistance to presentation attacks.
The second level is video verification: the system records a short video and analyzes it frame by frame, assessing the three-dimensionality of the face, micro-movements, and skin texture. This scenario significantly raises the entry barrier for attackers using deepfakes or video stream injection.
If uncertainty remains after reinforced biometric checks, the case is passed to an operator for visual comparison — but this is already the zone of manual processing, not automatic step-up.
Two levels of biometric step-up — active liveness and frame-by-frame video analysis — raise the entry barrier for attackers using deepfakes and video stream injection, but their effectiveness depends on the accuracy of the algorithms and the speed of response. The liveness/PAD module of the NeuroVision platform implements both modes: the passive check analyzes skin texture, micro-movements, and frame depth without user participation, while the active one requests actions in front of the camera to counter advanced presentation attacks. Liveness detection accuracy is 99.9%, the check takes less than 1 second, and the choice of mode — passive, active, or a combination — is determined by the session’s risk level through configurable rules in the engine. Integration is carried out via SDKs for iOS, Android, and Web with ready-made UI components for capturing the selfie and the document, which reduces development on your side. We will assess your current liveness check scenario and propose a configuration adapted to the threat profile and the acceptable level of friction for your audience.
Checks of Channel and Device Ownership
This class of step-up is aimed at confirming that the action is being performed by the account owner, not a third party with access to the credentials. An ownership check is relevant when the device changes, when logging in from a new region, when recovering access, or when confirming a critical transaction (a password change, a withdrawal of funds, a change of contact details).
Standard mechanisms: a one-time code (OTP) to the linked phone number or email; a push notification to the mobile app requesting confirmation; verification via a linked trusted device (device binding). A stricter option is requesting a code sent simultaneously to two independent channels (SMS and email), which reduces the risk of a single channel being compromised.
If the device fingerprint does not match the account history but the other signals are normal, an OTP to the linked channel is the minimally sufficient measure. If the device is new and the IP region has also changed, it is reasonable to combine an OTP with repeat biometrics.
Orchestrating KYC Checks by Risk in the Rules Engine
The rules engine is the central component that links scoring results to specific actions. It determines which checks are performed, in what order, and under what conditions step-up is launched. Without orchestration, checks work in isolation: each module (OCR, biometrics, liveness, AML screening) returns a result, but no one puts the whole picture together and makes a single decision.
Orchestration by risk means that the set of KYC checks is not fixed but determined dynamically. For a low-risk user, the engine launches the minimum mandatory set: a document check, face matching, basic liveness. For medium risk, it adds AML screening against extended lists, cross-checking data against registers, and reinforced liveness. For high risk, it activates the full set of checks, including a source of funds check and escalation to manual processing.
The implementation is built on a set of “if — then” rules organized into chains with priorities. Each rule contains a condition (a combination of signal values and the score), an action (launching a specific check, setting a status, sending a notification), and a priority that determines the order of processing in case of conflict. The engine must support rule versioning, so that any change can be rolled back, and decision logging — a full trace of each case: which rules fired, what data was at the input, what decision was made and why.
Two typical orchestration errors. The first is cascading duplication, when several rules launch the same check at different stages, increasing processing time and cost without a gain in accuracy. The second is a priority conflict, when one rule sends a case for approval and another for a block, and the result depends on the order of execution. The solution is an explicit hierarchy: security rules (block, quarantine) always take priority over pass rules; at equal priority, the stricter action is applied; all conflicts are recorded in the log for subsequent analysis.
Orchestrating KYC checks by risk level eliminates cascading duplication and priority conflicts — provided that the engine supports configuration versioning and logging of every case. In the back office of the NeuroVision platform, scenarios are configured modularly: the order of steps, the set of documents, scoring thresholds, step-up conditions, and rules for routing to manual review are set through the operator interface without changing code. Action logs record which rules fired, what data was at the input, and what decision was made — this covers both operational and regulatory requirements for explainability, including the standards of 115-FZ and the GDPR. The platform is deployed in the cloud or in your secure environment (container delivery via Docker/VM), and the API availability SLA is 99.99%. We will agree on a set of scenarios and rule logic for your business model and connect a test environment for up to 1 month so that you can evaluate the engine’s performance on real traffic.
How to Replace a Block With a Scenario Restriction Without Losing Control
A direct block is the simplest but also the most expensive anti-fraud tool. Every false block is a lost customer, a load on support, and reputational damage. Replacing a block with a scenario restriction makes it possible to keep control over a risky session without closing off the user’s access entirely.
A scenario restriction works on the principle of “admit with conditions.” Instead of a refusal, the system narrows the functional perimeter: it allows registration, but without the ability to make transactions until additional verification is passed; it allows login, but blocks changing contact details and withdrawing funds; it allows a transaction, but with a reduced limit until identity is confirmed. The user gets access to the service and sees clear conditions for lifting the restrictions, rather than a wall with a refusal.
To implement restrictions, the rules engine operates not with binary decisions (approve/decline) but with a set of permissions tied to the level of trust. Each scenario — registration, login, payment, profile change — has a minimum and a full set of permissions. At medium risk, the minimum set is assigned with the conditions for escalation specified. The conditions should be clear to the user: “upload a second document to increase your limit,” “confirm your phone to unlock transfers.”
A scenario restriction must not turn into a hidden pass. If the user has received restricted access but has not passed additional verification, the system should automatically close the session or lower the access level after a set time. A restriction is a temporary measure, not a permanent state.
A Retry and Exception Policy That Doesn’t Open Loopholes
An anti-fraud loop must allow repeat attempts to pass a check; otherwise, a single technical failure or an unsuccessful photo becomes a final verdict for a legitimate user. But unlimited repeat attempts are an open door to brute-forcing: an attacker can iteratively try documents, adjust a deepfake, or exploit borderline score values.
A retry policy sets three parameters: the number of allowed attempts, the time window, and the conditions for blocking. Typical values for onboarding: 2–3 repeat document check attempts within 24 hours; 2 repeat liveness check attempts within one session; after the attempts are exhausted — a cooling-off period of 1 to 48 hours. For authentication at login, the window is shorter: 3–5 OTP attempts in 10–15 minutes, then a temporary lock of the channel.
Each failed attempt should raise the session’s internal risk score. If a user has uploaded a document with signs of editing three times, this is not a technical problem, and the system should not provide a fourth attempt automatically. The rule: a repeat attempt is allowed in case of a technical failure (image quality, timeout, connection failure); for substantive reasons (data mismatch, signs of forgery, a blacklist), a repeat is possible only through manual escalation.
Exceptions to the standard policy are necessary for atypical but legitimate cases: the user has changed their passport and the new document differs from the data in the system; the customer is roaming and cannot receive an SMS; the face has changed after a medical procedure. For such cases, a separate route is created — an exception with manual approval. Each exception requires a justification from the operator, a link to the session identifier, and an entry in the audit log. Mass exceptions following one template are an indicator that a rule or threshold is configured incorrectly and needs to be revised, not bypassed manually.
An additional safeguard is automatic monitoring of exception statistics. If the share of exceptions for a specific rule exceeds a set threshold (for example, 5% of the total volume), the system generates an alert for the anti-fraud team. This prevents a situation in which exceptions de facto become the norm and undermine the integrity of the loop.
Tuning Risk Thresholds Without Increasing False Positives
The scoring model and the set of rules do not protect against fraud on their own — what protects is the point at which the system makes a decision: let through, reinforce the check, or block. This point is set by thresholds. A block threshold that is too low cuts off fraudsters, but legitimate customers along with them. One that is too high reduces friction but lets attacks through. Tuning thresholds without degrading the customer experience requires a formalized approach: selection based on a loss function, controlled changes to rules in the engine, staged verification on real traffic, and automatic rollback when metrics deviate.
Choosing Thresholds by Loss Function and a Constraint on False Blocks
A threshold is a numeric value on the risk score scale at which the system switches the action: from automatic pass to a request for additional verification or to a block. Choosing it “by eye” or copying it from someone else’s case is a mistake, because the optimal point depends on the ratio of the costs of the two types of errors in a specific business.
The loss function formalizes this ratio. The cost of missed fraud is direct losses: the transaction amount, investigation and refund costs, regulatory fines, reputational damage. The cost of a false block is lost revenue from the declined transaction, the cost of handling a support request, and the risk of churn.
Knowing these values, you can build a curve of total losses for different threshold values. The optimal threshold is at the minimum of this curve: for each point in the score range, the sum of losses from false positives and from missed fraud is calculated, and the value with the smallest total is chosen.
The cost of errors in anti-fraud is not symmetrical. Missed fraud, as a rule, costs several times more than a false block. Therefore, optimization often sets a constraint: the acceptable level of false blocks must not exceed a certain percentage of the total flow. This constraint works as a safeguard — the threshold cannot shift into a zone where false positives go beyond the acceptable limit.
Thresholds can and should differ for different decision points. At the onboarding stage, the cost of a false block is higher: the customer is not yet attached to the service, and a refusal most likely means losing them forever. When confirming a large transaction on an existing account, a stricter check is acceptable, because the customer has already invested effort in the relationship with the service and is more likely to complete an additional step. The cost of errors also varies by segment: average order value, product type, region, entry channel. If resources allow, thresholds are calibrated separately for key segments rather than setting a single value for all traffic.
Evaluation metrics: precision, recall, false positive rate, and the overall monetary loss function. Research in the field of financial fraud shows that optimizing the threshold by the F2 or MCC metrics gives more stable results compared with the standard threshold of 0.5, since it takes into account the asymmetry of classes and of the costs of errors. A threshold chosen without an explicit definition of the loss function almost always turns out to be suboptimal.
Tuning Rules in the Rules Engine Using Priorities, Exceptions, and Time Windows
The rules engine is the layer where the model’s score turns into a specific action. This is also where conflicts, duplication, and false positives arise that were not present in the model. Correct tuning involves working with three mechanisms: priorities, exceptions, and time windows.
Controlling the interaction between rules and the model is a separate aspect of tuning. A common architecture: the model generates a baseline score, rules adjust it up or down, and the final threshold is applied to the adjusted value. An alternative option is that rules work as filters before or after the model: at the input, they cut off events that obviously do not require scoring (trusted devices, allowlists); at the output, they intercept the model’s decisions for cases with special business logic. For each rule, metrics need to be recorded: the number of triggers, the share of false and true ones, and the impact on final conversion. Rules without a measurable contribution are candidates for removal.
Checking Changes Before Rolling Them Out to All Traffic
Any change to thresholds or rules is a hypothesis, not an improvement by default. The hypothesis needs to be tested before it affects all users. For this, three successive stages are used: a retrospective test, shadow mode, and a canary release.
A retrospective test (backtesting) is running the new thresholds and rules on historical data with known outcomes. A sample for a representative period is taken (several weeks, including peak and normal days), the new configuration is applied to it, and the result is compared with the actual decisions. How did the false positive rate change? Were confirmed cases of fraud missed? How did the manual review load change? A retrospective test requires no infrastructure changes and takes from a few hours to one or two days. Its main limitation is that it does not take into account the behavioral reactions of users and fraudsters to the new rules.
Shadow mode closes this limitation. The new configuration is deployed in parallel with the current one: each request is processed by both sets of rules, but only the current version makes the decision for the user. The shadow version writes its decisions to a log. This makes it possible to compare: in which cases the new configuration would have made a different decision, and how many of the discrepancies are improvements and how many are degradation. Shadow mode requires additional computing resources but carries no risks for users. The recommended duration is from a few days to a week.
A canary release is the transition from observation to action. The new configuration is applied to a limited share of real traffic: usually from 1% to 10%. The remaining users are served by the previous version. At this stage, the same metrics are measured, but now on live data with real consequences. The share of traffic is increased gradually: 1% → 5% → 10% → 25% → 50% → 100%. Moving to the next step happens only after confirming that the control metrics remain within the acceptable corridor. For a service with tens of thousands of events per day, one or two days per step is enough to collect a statistically significant sample.
At all three stages, the comparison must use the same metrics defined when the thresholds were chosen (false positive rate, recall, loss function). If the team evaluated precision in the retrospective test and switched to F1 in the canary, the results are not comparable, and the decision to roll out turns out to be untested.
Stop Conditions and Automatic Rollback When False Blocks Increase
Even after successfully passing all stages of checking, a new configuration may degrade in production. The causes: drift in the traffic distribution, the emergence of a new fraud pattern, seasonal changes in user behavior, a technical failure in one of the signal sources. To keep degradation from turning into a mass problem, the loop must include stop conditions with automatic rollback.
A stop condition is a threshold value of a control metric at which the system automatically returns to the previous configuration. It is formulated explicitly and in advance, before the changes are launched. Examples: the false positive rate over a rolling 1-hour window has exceeded X% — roll back; the share of blocked sessions has grown by more than Y% relative to the previous week’s baseline — roll back; the number of support requests about blocks has exceeded Z per day — roll back. The thresholds for stop conditions are determined based on historical data: the normal range of the metric is taken, and an acceptable deviation is added to its upper boundary.
Automatic rollback means that the system switches the rules engine to the previous configuration without operator involvement. This requires configuration versioning and infrastructure for instant switching. Each change to thresholds and rules is saved as a separate version with a timestamp. On rollback, the system activates the last stable version. Switching time should be measured in seconds — a delay means continued false blocks.
In addition to automatic rollback, an intermediate reaction is useful — an alert without rollback. Not every metric deviation requires an immediate return: a short-term spike in the false positive rate may be caused by a legitimate anomalous event (mass registration after an advertising campaign, unusual activity on a holiday). A two-tier system — a warning at the first threshold, a rollback at the second — reduces the risk of false rollbacks.
After each stop condition is triggered, an analysis is carried out: what caused the rise in false blocks, in which segment, on which rule, which signals deviated from expectations. The results are fed back into the tuning cycle: the loss function, exceptions, time windows, or the composition of signals are adjusted. Without a feedback cycle, stop conditions block any changes. The goal is not to avoid rollbacks, but to extract from each rollback information that makes the next iteration more accurate.
Operating the Loop and Maintaining Quality Without Increasing False Blocks
Launching an anti-fraud loop is not the final point but the beginning of a continuous cycle. Fraudsters adapt their tactics, the composition of the customer base changes, and channels and product scenarios are updated. If regular operation is not established after launch, the quality of decisions will begin to degrade: the model will miss new fraud schemes, and a threshold that yesterday gave an acceptable level of false blocks will start cutting off legitimate users tomorrow.
Labeling Outcomes and the Source of Truth on Fraud
Assessing the quality of an anti-fraud loop begins with answering the question: which cases turned out to be fraudulent and which did not. Without labeled outcomes, it is impossible to calculate either precision or recall, or to detect a rise in false positives. The first task at the operating stage is to fix the labeling process and define the source of truth (ground truth).
The source of truth depends on the scenario. For payment fraud — chargebacks and confirmed customer claims. For onboarding fraud — investigation results, data on identity duplication, confirmed cases of using forged or someone else’s documents. For account takeover — the owner’s requests and the conclusions of the investigations team.
No single source covers all cases. Chargebacks arrive with a delay of several days to several months; not every victim files a claim; some fraud remains undetected because the amount of damage is below the threshold for filing a claim. This creates the problem of incomplete labeling: the model is evaluated on data in which part of the real fraud is not labeled. If this effect is not taken into account, the metrics will be overstated.
To minimize distortions, labeling is built in layers. The first layer is automatic marking based on the occurrence of an external event (a chargeback, a block by decision of the security service, a confirmed incident). The second is manual labeling of a sample of blocked and passed cases by an analyst. The third is periodic retrospective reconciliation: after a fixed window (30, 60, or 90 days), a case is considered finally labeled based on the combination of signals.
If automatic and manual labeling diverge, fixed priority rules are needed: which source takes priority and under what conditions. Without such regulations, analysts will interpret disputed cases differently, and training the model on contradictory labeling leads to unstable scoring.
A separate question is labeling false blocks. The source of truth here is the result of manual review or successful completion of step-up verification: if the user has confirmed their identity and completed the transaction, the initial block is recognized as a false positive. This data needs to be stored with a trace: which signals led to the block, what the score was, and which rule fired. These labeled cases are used to calibrate thresholds and refine the logic of the rules.
Regular Review of Signals, Thresholds, and the Action Matrix
Signals that a year ago reliably separated fraud from legitimate actions may lose their discriminating power: fraudsters study the logic of the checks and adapt their behavior. At the same time, new data sources with a more accurate signal appear. A review of the set of signals, threshold values, and the “signal → action” matrix should be regular and documented.
The pace of review is determined by the speed of change. For payment services with a high volume of transactions, a typical cycle is once every two to four weeks. For KYC processes with a relatively stable flow of applications, a monthly or quarterly review may be sufficient. An extraordinary review is launched immediately in the event of an anomaly: a sharp rise in the share of rejections, a jump in false blocks, the emergence of a new attack vector, or a significant shift in the composition of incoming traffic.
Reviewing signals involves three steps. The first is analyzing the information value, or a similar metric of the discriminating power, of each feature on a fresh labeled sample. If a signal has stopped contributing to distinguishing fraud from non-fraud, it is modified or excluded. The second is checking new candidates: whether additional data sources have appeared for inclusion in the scoring. The third is analyzing the correlation between signals: strongly correlated features duplicate each other and can distort the model’s weights.
Reviewing thresholds relies on labeled data for the most recent period. A threshold is shifted only on a statistically significant basis: if on a fresh sample the current threshold leads to fraud losses or false blocks above the target constraint. Each change is formalized as an experiment, recording the “before” and “after” metrics, the predicted effect, and the rollback conditions.
The action matrix is reviewed taking into account accumulated statistics on step-up verification. If the data shows that a certain category of customers consistently passes the reinforced check, the level of friction for that segment can be reduced. If a specific type of step-up does not stop fraudsters, it is worth replacing it with a more effective scenario.
The result of each review cycle is recorded in a change log: the date, what exactly was changed, the justification, and the expected and actual effect. The log makes it possible to trace which decisions led to an improvement and which to a regression, and reduces diagnostic time at the next incident.
Monitoring Score Drift and Degradation by Segment
Even with a properly functioning model, the distribution of scores in production shifts over time relative to the training sample. Score drift can be caused by a change in the profile of the customer base, seasonal effects, an update of acquisition channels, or a change in fraudster tactics. If drift is not tracked, thresholds lose their relevance, and the false positive rate and false negative rate grow unnoticed.
The standard monitoring tool is the Population Stability Index (PSI). The index compares the distribution of scores (or input features) for the current period with the baseline. Generally accepted reference points: a PSI below 0.1 means stability, from 0.1 to 0.2 a shift that requires attention, and above 0.2 a significant change at which the model or thresholds need to be revised. These values should be adapted to the specific context: they depend on the number of bins, the sample size, and the specifics of the data.
PSI is not the only metric. To monitor drift in input features, the Kolmogorov–Smirnov test or the Wasserstein distance is used; to monitor calibration, a comparison of predicted and actual fraud shares by score decile.
Drift needs to be tracked not only on the overall flow but also by segment. A model may show stable aggregate metrics but degrade on individual segments — for example, on applications from a new acquisition channel, for a certain type of document, or in a specific region. Segmenting monitoring by channel, transaction type, geography, and device type makes it possible to detect local degradation before it shows up in the overall figures.
Implementation: an automated pipeline once a day (or more often, depending on volume) calculates the PSI of the distribution of scores and key features for each segment. When the threshold is exceeded, an alert is generated for the analyst on duty. The analyst carries out a diagnosis: whether the shift is caused by a real change in input data, a technical failure in the feature stream, or model degradation. Based on the results, a decision is made: the shift is insignificant (a seasonal effect within the norm), the thresholds are adjusted, recalibration is launched, or retraining is initiated.
Time passes between detecting drift and deploying an updated model. For this period, prepared measures are needed: a temporary adjustment of thresholds, enabling an additional rule in the engine, expanding the manual review pool. This makes it possible to maintain the quality of the loop without waiting for a full retraining cycle.
Quality Control of Manual Review and Feedback Into Rules and Models
Manual review is an essential element of the anti-fraud loop. It processes cases from the gray zone of scoring, serves as a source of labeled data for training the model, and acts as the last line of defense before an erroneous refusal to a customer. But it is only as effective as it is well organized.
Consistency of decisions between operators. If two analysts reach different conclusions on the same case, this is a signal of insufficiently clear instructions or differences in qualification. Consistency is measured on a cross-sample: some cases are sent to two operators independently, after which the share of matches is calculated. For typical verification of documents and biometrics, the benchmark is no lower than 90% matches.
The share of operator errors is identified during retrospective reconciliation. When the labeled data shows that an analyst missed confirmed fraud or blocked a legitimate customer without grounds, this is recorded as an error. Systematic tracking for each operator makes it possible to identify the need for additional training and to maintain a uniform standard.
Processing time. A prolonged manual review creates a delay for the customer and reduces conversion. Monitoring the median and the 95th percentile of processing time by case type helps identify bottlenecks: excessive complexity of instructions, a lack of resources, or unclear routing.
Feedback from manual review into rules and models is a critical part of the process. The results of operator decisions, especially in cases where the operator disagreed with the system’s automatic decision, form a valuable data set. If operators regularly overturn blocks following a certain pattern, this is grounds for adjusting a rule or threshold. If they consistently confirm fraud in cases that the model rates as medium risk, the model underestimates this type of threat.
For feedback to work, a formalized process is needed: periodic export of operator decisions, their comparison with automatic scores and rules, analysis of discrepancies, and prioritization of adjustments. The results are sent to the team responsible for updating the rules and the model and are included in the next review cycle.
A closed loop — from an automatic decision through manual review back to updating the logic — ensures continuous refinement of the anti-fraud system. The system not only responds to new threats but also reduces the share of false blocks thanks to operator experience translated into formal rules and model features.
Setting up an anti-fraud loop without increasing false blocks is the result not of a single successful rule or an accurate model, but of a built chain: measurable constraints on losses, multi-signal scoring, a proportionate response through the action matrix and step-up verification, controlled rollout of changes, and a closed feedback cycle from outcome labeling to threshold adjustment. Each link relies on the previous one: without a loss function, thresholds are chosen blindly; without decision tracing, it is impossible to analyze a false block; without regular review of signals and drift monitoring, the system degrades along with changes in fraud tactics and the audience profile.
Such a loop does not eliminate the trade-off between fraud detection and customer experience — it makes this trade-off manageable, transparent, and adaptive to changes in the business and the threat landscape. Teams that build this architecture in at the start and maintain it as a continuous process gain the ability to strengthen protection iteratively, based on data — and to preserve conversion at every stage of the customer journey.