Face recognition has the potential to significantly improve how organizations search, analyze and act on video data, but it is also one of the most sensitive and misunderstood types of video intelligence technology in use today. In this article, we explain how face recognition works, where it is most effective, where its limitations lie and how responsible principles are applied in real-world video analytics systems, including within BriefCam.
Face recognition is a video analytics capability powered by deep learning that analyzes faces in video to support identification, verification and review across a range of security and operational scenarios.
At a technical level, face recognition works by detecting faces in video and encoding their visual characteristics into a mathematical representation known as a feature vector, sometimes described as an anonymous numerical signature. These signatures capture complex patterns in facial features and are used to compare faces mathematically rather than visually.
Earlier face recognition systems relied on explicit facial measurements, such as the distance between the eyes, the width of the nose or the shape of the mouth. While innovative for their time, these approaches were highly sensitive to changes in lighting, camera angle and image quality, which limited accuracy and scalability.
Modern, AI-driven face recognition systems work differently. Using deep neural networks trained on large and diverse datasets, facial images are converted into anonymous numerical signatures rather than being compared as raw images. Matching signatures instead of images improves precision while reducing reliance on image data.
Importantly, face recognition does not produce a definitive identity decision. Instead, it generates a probability-based match score that indicates how closely two facial signatures align. Any potential match must always be reviewed and confirmed by a human operator before action is taken. This human-in-the-loop approach is a critical safeguard that ensures the technology supports decision-making rather than replacing it.
In practice, the evolution from manual facial measurements to AI-driven, signature-based comparison has made today’s face recognition technology more accurate, more scalable and better suited for responsible deployment when paired with clear governance and oversight.
As described above, face recognition systems can be used in different ways, depending on whether the goal is to verify a known identity or to find potential matches within video. In traditional verification scenarios, an individual’s live or captured image is compared against a pre-enrolled identity stored in a database.
BriefCam Face Recognition is designed primarily around face matching, rather than identity verification. Instead of confirming who someone is, the system correlates visual features extracted from faces in video to identify potential similarities. The result of this correlation is a probability-based match score, which approximates the likelihood of a match and must always be reviewed by a human operator.
In practice, BriefCam applies the same signature-based approach described earlier. Facial features detected in video are encoded into a feature vector, sometimes described as an anonymous numerical signature, which is used solely to compare visual similarity between faces rather than to store or identify individuals.
Because matching is performed using these signatures, BriefCam Face Recognition does not store biometric data in the traditional sense and does not connect to third-party image databases for monitoring or identifying people. All comparisons remain within customer-controlled environments and are subject to human review.
Before continuing, it is important to understand the broader biometric technology landscape and where face recognition fits within it.
Hard biometrics are technologies designed to uniquely identify or confirm an individual’s identity. Face recognition falls into this category as it can link facial patterns to known identities when configured to do so.
This example from BriefCam shows how soft biometric attributes, like clothing color, can be used to identify and track patterns in video instead of hard biometric identification like face recognition.
Soft biometrics describe visual or behavioral attributes such as clothing color, accessories, height or movement patterns. While these characteristics are not unique identifiers on their own, they can still be valuable for filtering, searching and analyzing.
This distinction matters because not every situation requires hard biometric identification. In many scenarios, soft biometric approaches or non-biometric video analytics can deliver the insight needed while reducing privacy impact. Human operators remain responsible for evaluating each situation and choosing the least intrusive technology that meets the objective.
Face recognition is one of many identification approaches, ranging from hard biometrics to softer, context-based and non-biometric methods that can be applied depending on the use case.
If a situation can benefit from face recognition, it can be applied in two primary ways, each with different operational and governance considerations.
Post-event analysis refers to applying face recognition to recorded video after an incident has occurred. Investigators can review footage, search for a person’s specific appearance and correlate events without real-time intervention. This approach allows for careful review and validation.
Real-time face recognition processes live video streams and can trigger alerts when a potential match is detected. These deployments are typically used in controlled or high-risk environments, such as access points or large public venues, where timely response is critical. Because real-time applications carry higher risk of false positives, they require stricter controls and clearly defined response procedures.
Face recognition supports several distinct use cases, each with different privacy and operational implications.
- Face verification (1:1)
In verification scenarios, a person claims an identity and the system confirms it. These use cases typically involve approved access lists, high-resolution cameras and controlled environments, and are commonly used for frictionless access control.
- Face recognition with watchlists (1:N)
In identification scenarios, detected faces are compared against a defined watchlist. These deployments may operate in real-time or during post-event review and are often used to locate missing persons, identify known offenders and support investigative workflows. Because environments are less controlled, careful configuration and human review are essential.
- Face re-identification (N:N)
Re-identification focuses on anonymized analysis rather than identity. Faces are converted into anonymous signatures to measure movement patterns, dwell time or flow through spaces. Data retention is minimized, and identity is not the goal. This approach enables organizations to convert video into anonymous, actionable data for operational insight.
Face re-identification is used here to analyze unique visitor patterns—such as dwell time, frequency and movement—using anonymous facial signatures rather than identifying individuals.
The effectiveness of face recognition depends heavily on the quality of the video input. At a foundational level, the quality of the feature-vector extraction—and therefore the reliability of a potential match—is directly tied to the clarity, detail and consistency of the facial image captured by the camera.
To support accurate matching and reduce the potential for false positives, BriefCam Face Recognition relies on high-quality video input. For successful feature extraction, the system requires a minimum facial resolution of 40×40 pixels across the face, or approximately 20 pixels between the eyes. When this threshold is not met, the system may not be able to extract sufficient facial detail for reliable comparison.
Several environmental and technical factors influence whether these conditions are met:
- Camera placement: Cameras should be positioned to capture faces clearly and consistently. In general, a vertical angle of approximately 45 degrees or more improves visibility and reduces distortion, increasing the likelihood that facial features can be extracted accurately.
- Occlusion: Objects that partially block the face—such as hats, masks, hair or other people—can interfere with feature extraction. Camera positioning and scene design should minimize occlusion wherever possible.
- Lighting and focus: Sufficient lighting, good contrast and proper focus are essential for producing crisp images. Poor lighting or soft focus can reduce facial detail and negatively affect matching reliability.
- Resolution and frame rate: Higher resolution and a consistent frame rate generally improve video quality, provided the facial pixel threshold is met. Sudden drops in frame rate or excessive compression can degrade the input and impact results.
These factors also influence fairness and consistency. Stable lighting, appropriate camera placement and well-calibrated systems help mitigate performance variation across different environments and populations. When conditions are not suitable for reliable face recognition, organizations should consider alternative, soft analytics, such as appearance similarity or attribute-based search, rather than forcing the technology beyond its effective limits.
For detailed technical requirements related to resolution, compression and engine-specific thresholds, organizations should consult the relevant BriefCam technical documentation or their Milestone representative to ensure configurations align with their deployment environment and use case.
Up to this point, we’ve looked at face recognition as a technology—how it works, where it performs best and the limitations that shape responsible use. The next question is practical: how are those principles applied in a real video analytics system?
This is where platform design and governance matter.
Within the BriefCam platform, responsible use of face recognition is supported through system-level controls that allow organizations to align technology use with legal, ethical and operational requirements.
BriefCam Face Recognition does not connect to third-party identity databases or external image repositories, and it does not automatically identify individuals. All images and video analyzed by the system remain the property of the customer, and BriefCam does not store personal data outside of customer-controlled environments.
Face Recognition can be excluded entirely at the license level or deactivated by administrators at any time. When disabled, all other video analytics capabilities remain available, ensuring organizations can continue to derive value from video without processing biometric data.
Administrators can enable or disable face recognition based on policy or regulatory requirements, while maintaining access to other video analytics features.
These design choices reflect Milestone’s broader perspective on responsible face recognition. Today’s AI-driven face recognition technology is significantly more precise and safer than earlier approaches, but its use still requires balance. Privacy concerns are valid, and regulation continues to evolve. With transparent governance, human oversight and thoughtful deployment, that balance is achievable.
Face recognition should be applied with clear purpose, appropriate controls and a willingness to use alternatives when they better fit the scenario. When used responsibly, it can support access control, anonymous analytics, investigations and even real-time prevention—without compromising trust.
Interested in exploring which video analytics capabilities are right for your organization? Contact us to see how Milestone can support your video intelligence needs.