Private Recognition Model Evaluation: A Practical Guide

# Private Recognition Model Evaluation: A Practical Guide

A private model evaluation is the most reliable way to choose a face recognition model for production. Public benchmarks are a useful starting point, but the model that wins on your data, at your operating point, on your deployment, is the one that should win the procurement decision. This guide walks through how to run such an evaluation in practice, using the InsightFace private model evaluation process as the worked example.

Step 1: Scope the evaluation

Before requesting access, write down:

• Use case — KYC verification, employee access control, member identification, watchlist, or another workload.
• Deployment — Cloud API, on-premise, or SDK.
• Volume — expected request rate or, for SDK, expected device count.
• Operating point — the false match rate and false non-match rate you actually need.
• Validation data — what you have, where it lives, and whether it can leave your environment.

This is the input to the scoping conversation with the InsightFace team.

Step 2: Submit the evaluation request

Use the enterprise inquiry form with the scope above. The team will follow up to confirm the right evaluation path: private API access, or on-premise evaluation instructions for teams that cannot move data.

Step 3: Put the lightweight evaluation agreement in place

Where appropriate, both sides put a short evaluation agreement in place to cover scope, confidentiality, and acceptable use. This is intentionally lightweight — its purpose is to make the evaluation possible, not to commit either side to a commercial contract.

Step 4: Receive evaluation access

Qualified teams receive credentials for a private model endpoint or, for on-premise scenarios, instructions to evaluate inside their own environment. The proprietary models exposed during evaluation are the same ones available under a commercial license — not a downgraded preview.

Step 5: Build and run the benchmark

Some practical recommendations:

• Use the same preprocessing pipeline you intend to ship.
• Run on the hardware you intend to deploy on, especially for latency numbers.
• Report FMR and FNMR at your real operating point, not just AUC.
• Slice results by the demographics, devices, and capture conditions that matter for your business.
• Compare against your current baseline (vendor or open-source) on the same set, the same way.

Step 6: Tune the threshold

Threshold selection is part of the evaluation, not a separate step. Pick the threshold that hits your FMR target on your data, then report FNMR at that threshold. The InsightFace team can help interpret results and confirm a sensible default.

Step 7: Make the procurement decision

With the benchmark in hand, the procurement decision usually has three dimensions:

1. Does the candidate clear your accuracy and fairness bar at your operating point?

2. Does the deployment model fit your operational reality?

3. Do the licensing terms cover the scope, geography, and volume you actually need?

A clear "yes" on all three means it is time to move to a commercial agreement.

Common pitfalls to avoid

• Benchmarking on the wrong distribution. Public datasets do not look like your users.
• Locking the threshold from a different model. Each model has its own operating curve.
• Ignoring fairness slices. A model that wins on average can lose on the cohort you care about.
• Confusing accuracy with deployment fit. A model that is great on a benchmark cluster may not fit your edge device.
• Skipping the latency measurement. Teams discover deployment-blocking latency far too late.

Next steps

To start a private model evaluation against your own data, submit an enterprise inquiry with the scope from Step 1. For background on the proprietary models and the supporting workflows, see the Private Recognition Model Evaluation page.