Face Capture
Introduction
Description
The Face Capture SDK is the client side library to be used with the Face Liveness Server product for determining the liveness of a subject. It is used to capture face images and package them according to the Face Liveness Server specifications. The Face Liveness Server will make a determination for whether or not the subject is live. The library does not directly communicate with any servers; it provides a package for the application to deliver to the Face Liveness Server. It is up to the application developer to implement any server communication and security.
Facial images are captured based on the selected capture profile. Capture profiles specify the criteria the subject must meet in order to successfully capture images and create a server package. Profiles are provided with this installer to be used as a base for capturing faces that are optimized for liveness processing on the back-end. Profiles can be customized to meet any additional requirements for capture.
Design Features
- Light-weight on-device size
- Easy to integrate API
- No direct UI activity to allow for a fully customizable user experience
- Handles operation of device hardware
- Optimized data collection for back-end analysis
- No direct server interaction. Applications have full control over when and how they interact with the back-end services.
Platforms
- Android 8.0 and higher
System Requirements
Android
- Android 8.0 or higher
- Android NDK 20 or higher
- API 24 or higher supported on device
- Device supports Camera2 integration
- Users must grant access to CAMERA permissions
Installation
Installed Components
The SDK installs various files in the subdirectories it creates. The subdirectories and their contents are listed below, along with a short description of the contents of each directory or file.
Top Level Directory bin, java, lib, manuals, samples, src-demo.
- changes.txt - Change log for each release of Face Capture.
- LICENSE - Aware license agreement
- bin - Directory containing pre-built Face Capture Demo APKs.
- lib - Directory containing the Android AAR files to use in integration.
- manuals - Directory containing Face Capture documentation.
- samples - Directory containing profiles.
- src-demo - Directory containing Aware demo source code.
Application Design
Overview
The Face Capture SDK’s purpose in an application is to initiate and coordinate a controlled capture of a subject’s Face biometric data. The library is re-entrant and can be used from multiple threads. However, only one capture session can occur at a time. The library provides functionality for:
- Intializing and operating the device’s cameras
- Selecting which camera to use in facial capture
- Providing preview image data to the application
- Processing images and returning feedback codes used to guide the user experience
- Specifying a maximum duration to attempt face capture
- Updating the application with the current capture session status
- Creating a JSON package compatible with the Aware Face Liveness Server back-end
Integration Tasks
The following is a quick checklist of objects to create and parameters to configure in order to capture a face.
- Create a Face Capture library
- Create a Workflow
- Set the Capture Profile for the Workflow
- Specify a Capture Timeout for the Workflow (optional, but recommended)
- Specify a Username for the Workflow (optional)
- Get the list of cameras for either the front or back side of the device
- Select a camera from the list
- Set up a polling thread for getting Capture States
- Start a Capture Session with the Workflow and Camera
- Display images and feedback from the capture session to guide the user to a successful capture
- Handle the three cases where the Capture Session will end
- Upon receiving the Completed status, the Capture Session has successfully captured. The application should retrieve the server package from the Workflow used.
- Upon receiving the Timedout status, the Capture Session was unsuccessful. The application has been notified that the user did not capture in the time allowed.
- Upon receiving the Aborted status, the Capture session was unsuccessful. The application should handle this case as a camera/hardware failure.
- If successful, send the server package to the back-end.
Server Communication
The Face Capture SDK does not communicate directly with any server. All server communication is to be handled by the application integrating against it. Instead, the Face Capture SDK provides a request package to be delivered to the back-end. Some of the properties of the request package will differ based on which Workflow was created and what properties were used in the capture session.
The Face Capture SDK does not parse or interpret results from the Face Liveness server. Interpretation of the server results are left to the integrator and how they have delegated responsibilities between their application and back-end.
Data Security
The FaceCapture function getEncryptedServerPackage() returns an encrypted JSON package. There is a matching decryption function in the Face Liveness server. The encryption used is asymmetric RSA based on a AES 256 cipher block chain.
Workflows
Overview
Workflows are a pre-defined set of rules and parameters used to perform automatic collection of subject data for determining liveness. Workflows are created by the Face Capture library using their name. Capture Sessions are started using a Workflow and Camera to manage its settings. Some settings are available for modifcation through the Workflow such as:
- A username to identify the subject in the transmitted JSON for reporting purposes.
- A timeout duration which stops the Capture Session once the specified amount of time has passed. Setting a zero second timeout disbales the Capture Session from timing out.
- A capture profile specifying the criteria used to determine when to automatically capture the subject’s face.
Please see the (Table 1) table below for a general overview of the available Workflows.
Feature | Charlie | Foxtrot |
---|---|---|
Device Type | Mobile Device | Mobile Device |
Device Type - The type of device to use this workflow with. Different workflows perform captures differently based on device type. For example, using a mobile device workflow when capturing with a kiosk will result in inaccurate or poor performance.
Note : The Charlie workflow is only available through the Android JAVA and iOS Swift interfaces.
Workflow Package Types
After a successful capture session, the back-end request package will be available through the Workflow. There are three different types of back-end package that can be obtained:
- High Usability - A request for the back-end focussed on usability.
- High Security - A request for the back-end focussed on security.
- Balanced - A request for the back-end with a mix of security and usability.
It is recommended to start with a Balanced approach and adjust to either High Usability or High Security based on testing.
JAVA API
FaceCaptureJNI
Library object that manages all other Face Capture API classes.
Destroy
Description: Destroy the FaceCaptureJNI object. This will free the memory used by the native library. Calling destroy will invalidate this object and all objects created by it. Attempting to use any functions with this or any other invalidated object will result in the NULL_FACE_CAPTURE_OBJ error being thrown.
Function Definition:
void destroy()
Returns: None
Throws: None
Workflow Create
Description: Create a Workflow object using a workflow name. The Workflow object will become invalid if the FaceCaptureJNI used to create it is destroyed.
Function Definition:
IFaceCapture.IWorkflow workflowCreate(String workflowName)
Parameter Name | Parameter Type | Description |
---|---|---|
workflowName | String | Name of the workflow to create |
Returns: A new Workflow object.Throws:
- NULL_FACE_CAPTURE_OBJ - The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
- UNKNOWN_WORKFLOW - The workflowName specified did not match any of the valid workflow names. Please see the Constants section for the list of valid names.
Get Camera List
Description: Get a list of available Camera objects. Use CameraPosition.FRONT to get all “selfie”/front facing cameras. Use CameraPosition.BACK to get all cameras on the back of the device.
Function Definition:
IFaceCapture.ICamera[] getCameraList(CameraPosition position)
Parameter Name | Parameter Type | Description |
---|---|---|
position | CameraPosition | Position of the cameras on the device |
Returns: An array of Cameras for the camera position specified.Throws:
- NULL_FACE_CAPTURE_OBJ - The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
- INVALID_CAMERA_POSITION - The position value specified was not a valid CameraPosition.
Start Capture Session
Description: Start a capture session using the specified Workflow and Camera objects. While capturing, you cannot modify either object until the capture session ends. A capture session can end via a stopCaptureSession() call, if the capture session timeout expires, if there is a hardware issue, or if it captures successfully.
Function Definition:
void startCaptureSession(IFaceCapture.IWorkflow workflow, IFaceCapture.ICamera camera)
Parameter Name | Parameter Type | Description |
---|---|---|
workflow | IWorkflow | Workflow containing settings for the capture session |
camera | ICamera | Camera to be used in the capture session |
Returns: NoneThrows:
- NULL_FACE_CAPTURE_OBJ - The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
- INVALID_WORKFLOW - The Workflow object has been invalidated by destroy() and cannot be used.
- INVALID_CAMERA - The Camera object has been invalidated by destroy() and cannot be used.
- NO_CAPTURE_PROFILE - The CAPTURE_PROFILE WorkflowProperty was not specified. A valid CAPTURE_PROFILE must be set.
- INVALID_CAPTURE_PROFILE - The CAPTURE_PROFILE WorkflowProperty specified was not profile valid XML.
Stop Capture Session
Description: Stop the current capture session. If there is no capture session running, this is a no-op. The next CaptureState will contain the STOPPED CaptureSessionStatus indicating that the session has successfully stopped. The Frame will not be retrievable and the Feedback may be safely ignored from this CaptureState.
Function Definition:
void stopCaptureSession()
Returns: NoneThrows:
- NULL_FACE_CAPTURE_OBJ - The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
Get Capture Session State
Description: Blocking function that retrieves the capture session state object.
Function Definition:
ICaptureState getCaptureSessionState()
Returns: A new CaptureState object.Throws:
- NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by destroy() and cannot be used.
Get Server Package
Description: Get the JSON server package as a string value.
Function Definition:
String getServerPackage(IWorkflow workflow, PackageType packageType)
Parameter Name | Parameter Type | Description |
---|---|---|
workflow | IWorkflow | Workflow containing settings for the capture session |
packageType | PackageType | The requested package type. |
Returns: The string representation of the JSON server package.Throws:
- NULL_FACE_CAPTURE_OBJ - The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
- INVALID_WORKFLOW - The Workflow object has been invalidated by destroy() and cannot be used.
- CAPTURE_SESSION_UNAVAILABLE - There has not been a successfully completed capture session. No data available.
Get Encrypted Server Package
Description: Get the encrypted JSON server package as a string value.
Function Definition:
String getEncryptedServerPackage(EncryptionType encryptionType, String encryptionKey, IWorkflow workflowHandle, PackageType packageType)
Parameter Name | Parameter Type | Description |
---|---|---|
encryptionType | EncryptionType | Type of encryption |
encryptionKey | String | String containing encryption key |
workflowHandle | Workflow | Workflow object containing settings for the capture session |
packageType | PackageType | Type of JSON request package |
Returns: The string representation of the encrypted JSON server package.Throws:
- NULL_FACE_CAPTURE_OBJ - The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
- INVALID_WORKFLOW - The Workflow object has been invalidated by destroy() and cannot be used.
- INVALID_ENCRYPTION_TYPE - The EncryptionType enumeration is not a valid type and cannot be used.
- INVALID_ENCRYPTION_PUBLIC_KEY - The String containing the encryption key is not a valid key and cannot be used.
- CAPTURE_SESSION_UNAVAILABLE - There has not been a successfully completed capture session. No data available.
Get Captured Image
Description: Retrieves the stored capture image from the given Workflow.
Function Definition:
byte[] getCapturedImage( IFaceCapture.IWorkflow workflow )
Parameter Name | Parameter Type | Description |
---|---|---|
workflow | IWorkflow | Specified workflow to retrieve a capture image from. |
Returns: The capture image that was retrieved from the Workflow. The image is returned as a JPG.Throws:
- NULL_FACE_CAPTURE_OBJ - The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
- INVALID_WORKFLOW - The Workflow object has been invalidated by destroy() and cannot be used.
- CAPTURE_SESSION_UNAVAILABLE - There has not been a successfully completed capture session. No data available.
Capture Session Get Capture Region
Description: Function that retrieves the region of the image where the face will be captured in the current Capture Session.
Function Definition:
Rectangle captureSessionGetCaptureRegion()
Returns: A rectangle defining the region where the face will be captured. The region is a rectangle is defined by a point (x,y) at the upper left corner along with a width and height.Throws:
- NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by destroy() and cannot be used.
- CAPTURE_SESSION_UNAVAILABLE - There are no capture sessions currently running
Capture Session Enable Autocapture
Description: Enable or disable the autocapture for the capture session. If there is no capture session running, this is a no-op.
Function Definition:
void captureSessionEnableAutocapture(boolean enableAutocapture)
Parameter Name | Parameter Type | Description |
---|---|---|
enableAutocapture | Boolean | Boolean value on whether to enable autocapture |
Returns: NoneThrows:
- NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by destroy() and cannot be used.
- CAPTURE_SESSION_UNAVAILABLE - There are no capture sessions currently running
Get Version
Description: Get the Face Capture SDK version as an integer value.
Function Definition:
int getVersion()
Returns: The integer representation of the Face Capture SDK version in AABBCCDD format.
- AA is the major version
- BB is the minor version
- CC is the bug fix version
- DD is the build version
Throws: None
Get Version String
Description: Get the Face Capture SDK version as a string value.
Function Definition:
String getVersionString()
Returns: The string representation of the Face Capture SDK version.
Throws: None
CaptureState
The Capture State object provides a snap-shot of the capture session’s state when created. The data provided by a single Capture State will never change. A new Capture State must be retrieved in order to get a new status update from the capture session.
Destroy Capture State
Description: Destroys the CaptureState object. This will free the memory used by the native library. Calling destroy will invalidate this object. Attempting to use any functions with this invalidated CaptureState will result in the INVALID_CAPTURE_STATE error being thrown.
Function Definition:
void destroy()
Returns: NoneThrows:
- NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by destroy() and cannot be used.
- INVALID_CAPTURE_STATE - The CaptureState object has been invalidated by destroy() and cannot be used.
Get Status
Description: Function that retrieves the capture session’s status from when the Capture State was created.
Function Definition:
CaptureSessionStatus getStatus()
Returns: The capture sessions status from when the Capture State was created. See the CaptureSessionStatus enumeration descriptions for details.Throws:
- NULL_FACE_CAPTURE_OBJ - The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
- INVALID_CAPTURE_STATE - The CaptureState object has been invalidated by destroy() and cannot be used.
Get Frame
Description: Function that retrieves the preview frame from when the Capture State was created.
Function Definition:
byte[] getFrame()
Returns: A byte array containing a preview frame from the capture session. The frame is in JPG format. The frame is either 480x640 in portrait mode or 640x480 in landscape mode.Throws:
- NULL_FACE_CAPTURE_OBJ - The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
- INVALID_CAPTURE_STATE - The CaptureState object has been invalidated by destroy() and cannot be used.
- NO_FRAME_AVAILABLE - The capture session ended before the next frame was ready. There is no frame available for this Capture State.
Get Feedback
Description: Function that retrieves the subject feedback from when the Capture State was created.
Function Definition:
AutoCaptureFeedback getFeedback()
Returns: The most recent AutoCaptureFeedback update when the Capture State was created. See the AutoCaptureFeedback enumeration descriptions for more details.Throws:
- NULL_FACE_CAPTURE_OBJ - The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
- INVALID_CAPTURE_STATE - The CaptureState object has been invalidated by destroy() and cannot be used.
Workflow
Workflow object used to store capture session configurations.
Destroy
Description: Destroy the Workflow object. This will free the memory used by the native library. Calling destroy will invalidate this object. Attempting to use any functions with this invalidated Workflow will result in the NULL_FACE_CAPTURE_OBJ or INVALID_WORKFLOW error being thrown.
Function Definition:
void destroy()
Returns: None
Throws: None
Set Property String
Description: Set a string property value for the specified workflow property. Attempting to set a non-string property will result in a WORKFLOW_PROPERTY_TYPE_MISMATCH error being thrown. See the WorkflowProperty enumeration descriptions for more details.
Function Definition:
void setPropertyString(WorkflowProperty property, String value)
Parameter Name | Parameter Type | Description |
---|---|---|
property | WorkflowProperty | Workflow property having its value set |
value | String | New value for the specified workflow property |
Returns: NoneThrows:
- NULL_FACE_CAPTURE_OBJ - The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
- INVALID_WORKFLOW - The Workflow object has been invalidated by destroy() and cannot be used.
- INVALID_WORKFLOW_PROPERTY - The property value was not a valid WorkflowProperty.
- WORKFLOW_PROPERTY_TYPE_MISMATCH - The property value specified does not accept a String value.
Set Property Double
Description: Set a double property value for the specified workflow property. Attempting to set a non-double property will result in a WORKFLOW_PROPERTY_TYPE_MISMATCH error being thrown. See the WorkflowProperty enumeration descriptions for more details.
Function Definition:
void setPropertyDouble(WorkflowProperty property, double value)
Parameter Name | Parameter Type | Description |
---|---|---|
property | WorkflowProperty | Workflow property having its value set |
value | double | New value for the specified workflow property |
Returns: NoneThrows:
- NULL_FACE_CAPTURE_OBJ - The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
- INVALID_WORKFLOW - The Workflow object has been invalidated by destroy() and cannot be used.
- INVALID_WORKFLOW_PROPERTY - The property value was not a valid WorkflowProperty.
- WORKFLOW_PROPERTY_TYPE_MISMATCH - The property value specified does not accept a double value.
Camera
Camera object used to store capture session configurations.
Destroy
Description: Destroy the Camera object. This will free the memory used by the native library. Calling destroy will invalidate this object. Attempting to use any functions with an invalidated Camera will result in the NULL_FACE_CAPTURE_OBJ or INVALID_CAMERA error being thrown.
Function Definition:
void destroy()
Returns: None
Throws: None
Get Name
Description: Get the name of the Camera.
Function Definition:
String getName()
Returns: String containing the name of the Camera.Throws:
- NULL_FACE_CAPTURE_OBJ - The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
- INVALID_CAMERA - The Camera object has been invalidated by destroy() and cannot be used.
Set Orientation
Description: Set the orientation of the camera. This will affect how the capture session expects the camera to be oriented. See the CameraOrientation enumeration descriptions for more details.
Function Definition:
void setOrientation(CameraOrientation orientation)
Parameter Name | Parameter Type | Description |
---|---|---|
orientation | CameraOrientation | New camera orientation for this camera to be used in capture sessions |
Returns: NoneThrows:
- NULL_FACE_CAPTURE_OBJ - The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
- INVALID_CAMERA - The Camera object has been invalidated by destroy() and cannot be used.
- INVALID_CAMERA_ORIENTATION - The orientation specified was not a valid CameraOrientation.
- CAMERA_IN_USE - The Camera is currently in use in a capture session and cannot be modified.
Enumerations
Enumerations
AutoCaptureFeedback
Feedback provided during a capture session indicating any facial quality issues with the subject.
Code | Value | Description |
---|---|---|
FACE_COMPLIANT | 0 | The subject’s face was compliant. |
NO_FACE_DETECTED | 1 | No faces were detected in the image. The subject’s face must be wholly in frame. |
MULTIPLE_FACES_DETECTED | 2 | Multiple faces were detected in the image. The subject must be the only face in frame. |
INVALID_POSE | 3 | The subject’s pose is too far off center. The subject should directly face the camera. |
FACE_TOO_FAR | 4 | The subject’s face is too far away. The subject should move closer to the camera. |
FACE_TOO_CLOSE | 5 | The subject’s face is too close. The subject should move away from the camera. |
FACE_ON_LEFT | 6 | The subject’s face is too far to the left. The subject should move to the center of the frame. |
FACE_ON_RIGHT | 7 | The subject’s face is too far to the right. The subject should move to the center of the frame. |
FACE_TOO_HIGH | 8 | The subject’s face is too high. The subject should move down towards the center of the frame. |
FACE_TOO_LOW | 9 | The subject’s face is too low. The subject should move up towards the center of the frame. |
INSUFFICIENT_LIGHTING | 10 | There is insufficient lighting. The subject should move to an area with more uniform lighting. |
LEFT_EYE_CLOSED | 11 | The subject’s left eye is closed. The subject should have both eyes open and visible to the camera. |
RIGHT_EYE_CLOSED | 12 | The subject’s right eye is closed. The subject should have both eyes open and visible to the camera. |
DARK_GLASSES_DETECTED | 13 | The subject is wearing dark or tinted glasss. The subject should remove the dark or tinted glasses. |
CameraOrientation
The device’s camera orientation.
Code | Value | Description |
---|---|---|
PORTRAIT | 0 | Portrait orientation - Camera is held vertically. |
LANDSCAPE | 1 | Landscape orientation - Camera is held horizontally. |
CameraPosition
The position of the camera on the device.
Code | Value | Description |
---|---|---|
FRONT | 0 | Camera is on the front of the device. Used for selfie captures. |
BACK | 1 | Camera is on the back of the device. Used for capturing other subjects. |
PackageType
The type of JSON request package to send to the Face Liveness back-end.
Code | Value | Description |
---|---|---|
HIGH_USABILITY | 0 | Request focused on high usability when analyzing on the back-end. |
BALANCED | 1 | Request focused on a balance of usability and security when analyzing on the back-end. |
HIGH_SECURITY | 2 | Request focused on high security when analyzing on the back-end |
WorkflowProperty
Configurable properties for Workflows.
Code | Value | Description |
---|---|---|
USERNAME | 0 | A string value used to set the Username/ID fields when communicating with the back-end. |
CAPTURE_TIMEOUT | 1 | A double value indicating the maximum duration in seconds for attempting face capture before ending the capture session. |
CAPTURE_PROFILE | 2 | A string value used to set the capture criteria for Face Capture. |
EncryptionType
The type of encryption for server packages.
Code | Value | Description |
---|---|---|
RSA_AES_256_CBC | 1 | RSA AES-256 CBC encryption. |
Constants
Workflow Names
Name | Value | Description |
---|---|---|
CHARLIE | Charlie | Name used in creating a Charlie workflow. Only available in JAVA and Swift. |
FOXTROT | Foxtrot | Name used in creating a Foxtrot workflow. |
Error Codes
Please see the API chapter for specific details on what each error means from any given function.
Error Codes
Error Code | Description |
---|---|
NULL_FACE_CAPTURE_OBJ | The Face Capture object was NULL. |
TRIAL_EXPIRATION_PASSED | The trial period for Face Capture has expired. Please contact Aware. |
OUT_OF_MEMORY | Failed to allocate sufficient memory for this function. |
INITIALIZATION_FAILED | Could not initialize Face Capture or a required component. |
INVALID_WORKFLOW | The Workflow object was invalid. |
UNKNOWN_WORKFLOW | No Workflow exists with the given name. |
WORKFLOW_IN_USE | Workflows cannot be modified while a capture session is in progress. |
INVALID_PACKAGE_TYPE | An invalid package type was specified. |
INVALID_WORKFLOW_PROPERTY | An invalid workflow property was specified. |
WORKFLOW_PROPERTY_TYPE_MISMATCH | The specified value is the wrong type for the specified property. |
INVALID_PROPERTY_VALUE | The specified value is not valid for the specified property. |
INVALID_CAMERA | The Camera object was invalid. |
UNKNOWN_CAMERA | No camera exists with the given name. |
CAMERA_IN_USE | Cameras cannot be modified while a capture session is in progress. |
CAMERA_INITIALIZATION | Camera could not be initialized. |
NO_INITIALIZABLE_CAMERAS | No cameras were detected. |
INVALID_CAMERA_POSITION | The camera position specified was an invalid value. |
INVALID_CAMERA_ORIENTATION | The camera orientation specified was an invalid value. |
INVALID_CAPTURE_STATE | The Capture State object was invalid. |
NO_FRAME_AVAILABLE | No frame data is available from the capture state object. |
CAPTURE_IN_PROGRESS | A capture session is already in progress. |
CAPTURE_SESSION_UNAVAILABLE | The previous capture session for the specified Workflow was aborted. |
NO_CAPTURE_PROFILE | No capture profile was specified by the Workflow. |
INVALID_CAPTURE_PROFILE | The capture profile set in the Workflow was invalid. |
WORKFLOW_INCOMPATIBLE_WITH_CAMERA | The selected Camera and Workflow are incompatible. Please see the Workflow description for usable camera criteria. |
INVALID_ENCRYPTION_TYPE | An invalid encryption type was specified. |
INVALID_ENCRYPTION_PUBLIC_KEY | The encryption public key specified was invalid or malformed. |
Software Acknowledgments
Aware Face Capture libraries incorporate third-party software. See the LICENSE file installed with the FaceCapture software for the full license agreement.