Due to the wide range of tasks that digital face recognition technologies solve, interest in these systems is very high. Based on this technology, various scenarios can be supported: authenticating users for access, counting people in a space for crowd management, and more. You can read more about the areas of application of face recognition here.
For this review, we have compared the capabilities of Face ID — a face recognition service by Evergreen — and the three most popular face recognition solutions: Microsoft Azure Face API, Google Cloud Vision API, and Amazon Rekognition Image (please, click to jump to a specific section).
During the analysis, we set ourselves the goal of testing the following functional blocks:
1. Face detection.
2. Face comparison (verification).
3. Determination of facial attributes (gender, age, ethnicity, emotions).
4. Search for similar faces — whether the system can identify the face of a certain person in a group of photos (to find a face that looks most like the target person).
We focused on the following criteria (they refer to testing each functional block listed above):
1. Accuracy (confidence score).
2. Service processing speed — time between receiving and responding to the request.
3. Recognition cost for each of the functions.
4. Additional features and recognition capabilities (in terms of facial recognition).
For objective evaluation, we used the same photos and analyzed them in turns in each of the services. Let's briefly compare the features of all three face recognition services and Face ID for ease of reference:
Azure Face API is part of Azure Cognitive Services by Microsoft for advanced facial recognition. Azure Face API can be used to detect, identify, and analyze faces in images and videos and uses state-of-the-art cloud-based face algorithms.
Azure Face service detects human faces and returns the rectangle coordinates of their locations in an image. Face detection can optionally extract a series of related attributes such as head pose, gender, age, emotion, facial hair, glasses, etc. with their detection confidence for a given face. On the below example, the detected face is of a 36 y.o. male who is smiling and the perceived emotion is happiness, according to the system’s evaluation.
Face ID can detect and locate human faces within an image and returns precise face bounding boxes. It can also locate facial landmarks (chin, eyes, eyebrows, mouth corners, nose contour) and return attributes like gender, age, smile intensity, head pose, blurriness, eye status (open or closed eyes, eyewear). The output is JSON.
For Face ID, the average detection time on a larger group of photos is 1.4 s.
In Azure Face, this feature is called Face Verification. Practically, the system evaluates whether two faces belong to the same person and locates faces with bounding boxes. As you can see below, Azure Face verified that our sample photos belong to different people. The response time took approximately 394 ms.
FaceID also compares two photos and decides whether they belong to the same person. You can upload your image or use a URL for face comparison. Additionally, the system evaluates face similarity (59.2% in this case — different people). Processing took 454 ms.
Facial Expressions and Attributes
Azure Face API can detect facial expressions such as neutral, happiness, anger, contempt, disgust, sadness, fear, and surprise. This feature is called Perceived Emotion Recognition.
Azure can extract facial attributes at the detection stage. In this mode, it returns the face rectangle position and corresponding confidence scores for each emotion we mentioned above.
At the moment, emotion recognition is not included in the Face ID service package and is currently under development. We can implement an individual solution that will recognize the following facial emotions: happiness, neutral, surprise, sadness, disgust, anger, and fear. Follow up with our release updates, or request an individual solution.
We didn’t have an opportunity to test the demo, but Azure Face service can compare a target face against a set of candidates' faces to find a set of similar-looking ones. The program has two working modes: the matchPerson mode returns similar faces after it filters for the same person, and the matchFace mode returns a list of similar candidate faces that might or might not belong to the same person. It ignores the same-person filter.
Up to date, we didn’t include Similarity Search in Face ID. But we can develop and implement this feature for your project. Given a target face and a collection of faces, the search API will return a collection of similar faces, along with confidence scores to evaluate the similarity. It would look something like this:
1. Comparison with a group of candidates:
2. Search results and response in JSON format:
In this example, we can see that the system recognizes the same person on photo 1 (same photo we used for search inquiry), and 2 with the highest level of probability. For other candidates, the probability is low.
Cloud Vision API is a comprehensive recognition product by Google that uses pre-trained models. It can detect objects and faces in photos, recognize text, automatically assign metadata, and much more. For this review, we focused on features that are related to face recognition and comparison.
Face detection in Google Cloud Vision API can detect a face or multiple faces in an image. Additionally, it detects general image properties (underexposed, blurred). Specific individual facial recognition is not supported.
Along with multiple objects recognized in a single image, it can also detect a person. Cloud Vision returns the following elements: a textual description, a confidence score, and normalized vertices [0, 1] for the bounding polygon around the object (in JSON).
Face ID returns precise face bounding boxes and locates facial landmarks (chin, eyes, eyebrows, mouth corners, nose contour). Also, the algorithm can return attributes such as gender, age, smile intensity, head pose, blurriness, eye status (open or closed eyes, eyewear) in JSON report.
Cloud Vision API doesn’t support face comparison and verification. It only confirms that a particular photo has a human face or multiple faces in it.
Using FaceID, you can compare two images and verify if detected faces belong to the same individual. If the similarity score is <80%, the system will indicate that there are two different people in the photos.
Two photos of the same person: 93.1% similarity (faces match), response time 1.163 s.
Photos of two different people:
Although these two actors are very look-alike, the system evaluated their similarity to be 57.6% (faces don’t match), with a response time of 0.459 s.
Facial Expressions and Attributes
Cloud Vision API locates faces with bounding polygons and identifies specific facial landmarks (eyes, ears, nose, mouth, etc.) along with their corresponding confidence values. It returns likelihood ratings for emotion (joy, sorrow, anger, surprise) and general image properties (underexposed, blurred, headwear present).
At the moment, emotion recognition is not included in the Face ID service package, as we already mentioned above. But we can implement an individual solution for your project at any time.
Our service works well with gesture recognition. It can be set up as an additional verification rule in your authentication systems. We can configure the recognition of up to 16 different gestures. You can see an example of a similar service below:
Amazon Rekognition uses advanced technology for face detection in images and video. It is Amazon's answer to Google's Cloud Vision API, being a complex product for the segmentation and classification of visual content. For this article, we will be focusing on its components for face recognition and analysis.
We have tested face detection as part of the Amazon Rekognition Object and Scene Detection Demo. It can automatically label objects, concepts, and scenes within images and provide a confidence score. To detect faces in an image, you need to configure your face-detection algorithm following the developer guide and run the DetectFaces operation.
As we can see, the system is 99.9% confident the face on the sample photo represents a human, and with 96.3% it is a performer. You can access the full response in JSON format.
Face ID recognizes the face in the sample photo as belonging to a 37 y.o. male locating it with a bounding box and also returns a JSON response containing facial landmarks and their coordinates (chin, eyes, eyebrows, mouth corners, nose contour), and other attributes.
Amazon Rekognition Image can compare a face in a source image with each face in the target image. For a source image that contains multiple faces, the service detects the largest face and uses it to compare with each face that is detected in the target image. In response, you get an array of face matches, source face information, source, and target image orientation. For each matching face, the system will return a similarity score (how similar the face is to the source face) and face metadata (bounding box and facial landmarks).
Face ID can compare two photos and decide whether they belong to the same person. The system will decide that the detected faces are of the same individual if the similarity score is >80%. In our example, there is a 93.1% probability that we recognized the same person in both photos.
Also, Face ID can automatically detect faces in documents, for example, in a passport photo. This feature can be useful for various authentication services (anti-fraud). The average processing time, in this case, is approximately 700 ms.
Facial Expressions and Attributes
For Amazon Recognition, we tested the Face Analysis Demo. The algorithm can return the following information for each detected face:
- coordinates of the bounding box that surrounds the face;
- the level of confidence that the bounding box contains a face;
- facial landmarks (an array of facial landmarks such as the left eye, the right eye, eyebrows, mouth, etc.);
- facial attributes (such as whether the face has a beard, is wearing sunglasses), gender (male or female);
- quality (the brightness and the sharpness of the face);
- pose (the rotation of a face inside the image);
- emotions (disgusted, happy, surprised, angry, confused, calm, sad).
Face ID returns the above attributes (except for emotion detection) as well (in JSON). As we already mentioned earlier, facial landmarks, gender, age, and other characteristics are being evaluated during face detection.
Amazon Rekognition can store information about detected faces in server-side containers knows as collections. You can use the facial information to search for known faces in images and videos, and use indexing to persist information about the detected facial features into a collection. After you create a face collection and store the facial information for all faces, you can search the collection for face matches. These collections can be used in a variety of scenarios.
The current version of Face ID can detect, compare, and verify facial information in photos and digital documents. Our team can develop and implement Similarity Search for your project.
If you want to learn more about the technology, discuss additional recognition services, try our demo or order a boxed solution — please contact us. We are also ready to adapt the face recognition service to your specific requirements and processes.