Face Capture
Introduction
Description
The Face Capture SDK is the client side library to be used with the Face Liveness Server product for determining the liveness of a subject. It is used to capture face images and package them according to the Face Liveness Server specifications. The Face Liveness Server will make a determination for whether or not the subject is live. The library does not directly communicate with any servers; it provides a package for the application to deliver to the Face Liveness Server. It is up to the application developer to implement any server communication and security.
Facial images are captured based on the selected capture profile. Capture profiles specify the criteria the subject must meet in order to successfully capture images and create a server package. Profiles are provided with this installer to be used as a base for capturing faces that are optimized for liveness processing on the back-end. Profiles can be customized to meet any additional requirements for capture.
Design Features
- Light-weight on-device size
- Easy to integrate API
- No direct UI activity to allow for a fully customizable user experience
- Handles operation of device hardware
- Optimized data collection for back-end analysis
- No direct server interaction. Applications have full control over when and how they interact with the back-end services.
Platforms
- iOS 12.0 and higher
System Requirements
iOS
- iOS 10 or higher
- Users must grant access to CAMERA permissions
Application Design
Overview
The Face Capture SDK’s purpose in an application is to initiate and coordinate a controlled capture of a subject’s Face biometric data. The library is re-entrant and can be used from multiple threads. However, only one capture session can occur at a time. The library provides functionality for:
- Intializing and operating the device’s cameras
- Selecting which camera to use in facial capture
- Providing preview image data to the application
- Processing images and returning feedback codes used to guide the user experience
- Specifying a maximum duration to attempt face capture
- Updating the application with the current capture session status
- Creating a JSON package compatible with the Aware Face Liveness Server back-end
Integration Tasks
The following is a quick checklist of objects to create and parameters to configure in order to capture a face.
- Create a Face Capture library
- Create a Workflow
- Set the Capture Profile for the Workflow
- Specify a Capture Timeout for the Workflow (optional, but recommended)
- Specify a Username for the Workflow (optional)
- Get the list of cameras for either the front or back side of the device
- Select a camera from the list
- Set up a polling thread for getting Capture States
- Start a Capture Session with the Workflow and Camera
- Display images and feedback from the capture session to guide the user to a successful capture
- Handle the three cases where the Capture Session will end
- Upon receiving the Completed status, the Capture Session has successfully captured. The application should retrieve the server package from the Workflow used.
- Upon receiving the Timedout status, the Capture Session was unsuccessful. The application has been notified that the user did not capture in the time allowed.
- Upon receiving the Aborted status, the Capture session was unsuccessful. The application should handle this case as a camera/hardware failure.
- If successful, send the server package to the back-end.
Server Communication
The Face Capture SDK does not communicate directly with any server. All server communication is to be handled by the application integrating against it. Instead, the Face Capture SDK provides a request package to be delivered to the back-end. Some of the properties of the request package will differ based on which Workflow was created and what properties were used in the capture session.
The Face Capture SDK does not parse or interpret results from the Face Liveness server. Interpretation of the server results are left to the integrator and how they have delegated responsibilities between their application and back-end.
Data Security
The FaceCapture function getEncryptedServerPackage() returns an encrypted JSON package. There is a matching decryption function in the Face Liveness server. The encryption used is asymmetric RSA based on a AES 256 cipher block chain.
Workflows
Overview
Workflows are a pre-defined set of rules and parameters used to perform automatic collection of subject data for determining liveness. Workflows are created by the Face Capture library using their name. Capture Sessions are started using a Workflow and Camera to manage its settings. Some settings are available for modifcation through the Workflow such as:
- A username to identify the subject in the transmitted JSON for reporting purposes.
- A timeout duration which stops the Capture Session once the specified amount of time has passed. Setting a zero second timeout disbales the Capture Session from timing out.
- A capture profile specifying the criteria used to determine when to automatically capture the subject’s face.
Please see the (Face Capture Workflow Summary) table below for a general overview of the available Workflows.
Feature | Charlie | Foxtrot |
---|---|---|
Device Type | Mobile Device | Mobile Device |
Device Type - The type of device to use this workflow with. Different workflows perform captures differently based on device type. For example, using a mobile device workflow when capturing with a kiosk will result in inaccurate or poor performance.
Note : The Charlie workflow is only available through the iOS Swift interface.
Workflow Package Types
After a successful capture session, the back-end request package will be available through the Workflow. There are three different types of back-end package that can be obtained:
- High Usability - A request for the back-end focussed on usability.
- High Security - A request for the back-end focussed on security.
- Balanced - A request for the back-end with a mix of security and usability.
It is recommended to start with a Balanced approach and adjust to either High Usability or High Security based on testing.
Swift API
Classes
FaceCapture
Library object that manages all other Face Capture API classes.
Dispose
Description: Dispose the FaceCapture object. This will free the memory used by the native library. Calling dispose will invalidate this object and all objects created by it. Attempting to use any functions with this or any other invalidated object will result in the NULL_FACE_CAPTURE_OBJ error being thrown.
Function Definition:
func dispose() -> Void
Returns: None
Throws: None
Workflow Create
Description: Create a Workflow object using a workflow name. The Workflow object will become invalid if the FaceCapture used to create it is disposed.
Function Definition:
func workflowCreate(workflowName: String) throws -> Workflow
Parameter Name | Parameter Type | Description |
---|---|---|
workflowName | String | Name of the workflow to create |
Returns: A new Workflow object.Throws:
- NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
- UNKNOWN_WORKFLOW - The workflowName specified did not match any of the valid workflow names. Please see the Constants section for the list of valid names.
Get Camera List
Description: Get a list of available Camera objects. Use CameraPosition.FRONT to get all “selfie”/front facing cameras. Use CameraPosition.BACK to get all cameras on the back of the device.
Function Definition:
func getCameraList(position: CameraPosition) throws -> [Camera]
Parameter Name | Parameter Type | Description |
---|---|---|
position | CameraPosition | Position of the cameras on the device |
Returns: An array of Cameras for the camera position specified.Throws:
- NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
- INVALID_CAMERA_POSITION - The position value specified was not a valid CameraPosition.
Start Capture Session
Description: Start a capture session using the specified Workflow and Camera objects. While capturing, you cannot modify either object until the capture session ends. A capture session can end via a stopCaptureSession() call, if the capture session timeout expires, if there is a hardware issue, or if it captures successfully.
Function Definition:
func startCaptureSession(workflowHandle: Workflow, cameraHandle: Camera) throws -> Void
Parameter Name | Parameter Type | Description |
---|---|---|
workflowHandle | Workflow | Workflow containing settings for the capture session |
cameraHandle | Camera | Camera to be used in the capture session |
Returns: NoneThrows:
- NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
- INVALID_WORKFLOW - The Workflow object has been invalidated by dispose() and cannot be used.
- INVALID_CAMERA - The Camera object has been invalidated by dispose() and cannot be used.
- NO_CAPTURE_PROFILE - The CAPTURE_PROFILE WorkflowProperty was not specified. A valid CAPTURE_PROFILE must be set.
- INVALID_CAPTURE_PROFILE - The CAPTURE_PROFILE WorkflowProperty specified was not profile valid XML.
Stop Capture Session
Description: Stop the current capture session. If there is no capture session running, this is a no-op. The next CaptureState will contain the STOPPED CaptureSessionStatus indicating that the session has successfully stopped. The Frame will not be retrievable and the Feedback may be safely ignored from this CaptureState.
Function Definition:
func stopCaptureSession() throws -> Void
Returns: NoneThrows:
- NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
Get Capture Session State
Description: Blocking function that retrieves the capture session state object.
Function Definition:
func getCaptureSessionState() throws -> CaptureState
Returns: A new CaptureState object.Throws:
- NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
- INVALID_CAPTURE_STATE - The capture state is invalid.
Get Server Package
Description: Get the JSON server package as a string value.
Function Definition:
func getServerPackage(workflowHandle: Workflow, packageType: PackageType) throws -> String
Parameter Name | Parameter Type | Description |
---|---|---|
workflowHandle | Workflow | Workflow containing settings for the capture session |
packageType | PackageType | The requested package type. |
Returns: The string representation of the JSON server package.Throws:
- NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
- CAPTURE_SESSION_UNAVAILABLE - There are no capture sessions currently running
Get Encrypted Server Package
Description: Get the encrypted JSON server package as a string value.
Function Definition:
func getEncryptedServerPackage(encryptionType: EncryptionType, encryptionKey: String, workflowHandle: Workflow, packageType: PackageType) throws -> String
Parameter Name | Parameter Type | Description |
---|---|---|
encryptionType | EncryptionType | Type of encryption |
encryptionKey | String | String containing encryption key |
workflowHandle | Workflow | Workflow object containing settings for the capture session |
packageType | PackageType | Type of JSON request package |
Returns: The string representation of the encrypted JSON server package.Throws:
- NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by destroy() and cannot be used.
- INVALID_WORKFLOW - The Workflow object has been invalidated by destroy() and cannot be used.
- INVALID_ENCRYPTION_TYPE - The EncryptionType enumeration is not a valid type and cannot be used.
- INVALID_ENCRYPTION_PUBLIC_KEY - The String containing the encryption key is not a valid key and cannot be used.
- CAPTURE_SESSION_UNAVAILABLE - There has not been a successfully completed capture session. No data available.
Get Captured Image
Description: Retrieves the stored capture image from the given Workflow.
Function Definition:
func getCapturedImage( workflow: Workflow ) throws -> [UInt8]
Parameter Name | Parameter Type | Description |
---|---|---|
workflow | Workflow | Specified workflow to retrieve a capture image from. |
Returns: The capture image that was retrieved from the Workflow. The image is returned as a JPG.Throws:
- NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by destroy() and cannot be used.
- INVALID_WORKFLOW - The Workflow object has been invalidated by destroy() and cannot be used.
- CAPTURE_SESSION_UNAVAILABLE - There has not been a successfully completed capture session. No data available.
Get Capture Session Region of Interest
Description: Get the area where the user’s face is expected for a capture to occur for the current capture session.
Function Definition:
func captureSessionGetCaptureRegion() throws -> CGRect
Returns: A rectangle defining the region where the face will be captured. The region is a rectangle is defined by a point (x,y) at the upper left corner along with a width and height.Throws:
- NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
- CAPTURE_SESSION_UNAVAILABLE - There are no capture sessions currently running
Capture Session Enable Autocapture
Description: Enable or disable the autocapture for the capture session. If there is no capture session running, this is a no-op.
Function Definition:
func captureSessionEnableAutocapture(enableAutocapture: Bool) throws -> Void
Parameter Name | Parameter Type | Description |
---|---|---|
enableAutocapture | Boolean | Boolean value on whether to enable autocapture |
Returns: NoneThrows:
- NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
Get Version
Description: Get the Face Capture SDK version as an integer value.
Function Definition:
static func getVersion() throws -> Int32
Returns: The integer representation of the Face Capture SDK version in AABBCCDD format.
- AA is the major version
- BB is the minor version
- CC is the bug fix version
- DD is the build version
Throws: None
Get Version String
Description: Get the Face Capture SDK version as a string value.
Function Definition:
static func getVersionString() throws -> String
Returns: The string representation of the Face Capture SDK version.
Throws: None
CaptureState
Library object that abstracts the Face Capture Capture State.
Dispose
Description: Dispose the CaptureState object. This will free the memory used by the native library. Calling dispose will invalidate this object. Attempting to use any functions with this invalidated CaptureState will result in the NULL_FACE_CAPTURE_OBJ or INVALID_CAPTURE_STATE error being thrown.
Function Definition:
func dispose() -> Void
Returns: NoneThrows:
- NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
- INVALID_CAPTURE_STATE - The CaptureState object has been invalidated by dispose() and cannot be used.
Get Capture Session Status
Description: Retrieves the most recent status update from the capture session.
Function Definition:
func getCaptureSessionStatus() throws -> CaptureSessionStatus
Returns: The most recent status update from the capture session. See the CaptureSessionStatus enumeration descriptions for details.Throws:
- NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
Get Capture Session Frame
Description: Blocking function that retrieves the most recent preview frame from the capture session. If there is no capture session in progress, this function will throw the CAPTURE_SESSION_UNAVAILABLE error.
Function Definition:
func getCaptureSessionFrame() throws -> [UInt8]
Returns: A byte array containing a preview frame from the capture session. The frame is in JPG format. The frame is either 480x640 in portrait mode or 640x480 in landscape mode.Throws:
- NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
- CAPTURE_SESSION_UNAVAILABLE - There are no capture sessions currently running.
Get Capture Session Feedback
Description: Retrieves the most recent preview feedback from the capture session. If there is no capture session in progress, this function will throw the CAPTURE_SESSION_UNAVAILABLE error.
Function Definition:
func getCaptureSessionFeedback() throws -> AutoCaptureFeedback
Returns: The most recent AutoCaptureFeedback update from the capture session. See the AutoCaptureFeedback enumeration descriptions for more details.Throws:
- NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
- CAPTURE_SESSION_UNAVAILABLE - There are no capture sessions currently running.
Workflow
Workflow object used to store capture session configurations.
Dispose
Description: Dispose the Workflow object. This will free the memory used by the native library. Calling dispose will invalidate this object. Attempting to use any functions with this invalidated Workflow will result in the NULL_FACE_CAPTURE_OBJ or INVALID_WORKFLOW error being thrown.
Function Definition:
func dispose() -> Void
Returns: None
Throws: None
Set Property Double
Description: Set a double property value for the specified workflow property. Attempting to set a non-double property will result in a WORKFLOW_PROPERTY_TYPE_MISMATCH error being thrown. See the WorkflowProperty enumeration descriptions for more details.
Function Definition:
func setPropertyDouble(property: WorkflowProperty, value: Double) throws -> Void
Parameter Name | Parameter Type | Description |
---|---|---|
property | WorkflowProperty | Workflow property having its value set |
value | double | New value for the specified workflow property |
Returns: NoneThrows:
- NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
- INVALID_WORKFLOW - The Workflow object has been invalidated by dispose() and cannot be used.
- INVALID_WORKFLOW_PROPERTY - The property value was not a valid WorkflowProperty.
- WORKFLOW_PROPERTY_TYPE_MISMATCH - The property value specified does not accept a double value.
Set Property String
Description: Set a string property vlaue for the specified workflow property. Attempting to set a non-string property will result in a WORKFLOW_PROPERTY_TYPE_MISMATCH error being thrown. See the WorkflowProperty enumeration descriptions for more details.
Function Definition:
func setPropertyString(property: WorkflowProperty, value: String) throws -> Void
Parameter Name | Parameter Type | Description |
---|---|---|
property | WorkflowProperty | Workflow property having its value set |
value | String | New value for the specified workflow property |
Returns: NoneThrows:
- NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
- INVALID_WORKFLOW - The Workflow object has been invalidated by dispose() and cannot be used.
- INVALID_WORKFLOW_PROPERTY - The property value was not a valid WorkflowProperty.
- WORKFLOW_PROPERTY_TYPE_MISMATCH - The property value specified does not accept a String value.
Camera
Camera object used to store capture session configurations.
Dispose
Description: Dispose the Camera object. This will free the memory used by the native library. Calling dispose will invalidate this object. Attempting to use any functions with an invalidated Camera will result in the NULL_FACE_CAPTURE_OBJ or INVALID_CAMERA error being thrown.
Function Definition:
func dispose() -> Void
Returns: None
Throws: None
Get Name
Description: Get the name of the Camera.
Function Definition:
func getName() throws -> String
Returns: String containing the name of the Camera.Throws:
- NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
- INVALID_CAMERA - The Camera object has been invalidated by dispose() and cannot be used.
Set Orientation
Description: Set the orientation of the camera. This will affect how the capture session expects the camera to be oriented. See the CameraOrientation enumeration descriptions for more details.
Function Definition:
func setOrientation(orientation: CameraOrientation) throws -> Void
Parameter Name | Parameter Type | Description |
---|---|---|
orientation | CameraOrientation | New camera orientation for this camera to be used in capture sessions |
Returns: NoneThrows:
- NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
- INVALID_CAMERA - The Camera object has been invalidated by dispose() and cannot be used.
- INVALID_CAMERA_ORIENTATION - The orientation specified was not a valid CameraOrientation.
- CAMERA_IN_USE - The Camera is currently in use in a capture session and cannot be modified.
Enumerations
Enumerations
AutoCaptureFeedback
Feedback provided during a capture session indicating any facial quality issues with the subject.
Code | Value | Description |
---|---|---|
FACE_COMPLIANT | 0 | The subject’s face was compliant. |
NO_FACE_DETECTED | 1 | No faces were detected in the image. The subject’s face must be wholly in frame. |
MULTIPLE_FACES_DETECTED | 2 | Multiple faces were detected in the image. The subject must be the only face in frame. |
INVALID_POSE | 3 | The subject’s pose is too far off center. The subject should directly face the camera. |
FACE_TOO_FAR | 4 | The subject’s face is too far away. The subject should move closer to the camera. |
FACE_TOO_CLOSE | 5 | The subject’s face is too close. The subject should move away from the camera. |
FACE_ON_LEFT | 6 | The subject’s face is too far to the left. The subject should move to the center of the frame. |
FACE_ON_RIGHT | 7 | The subject’s face is too far to the right. The subject should move to the center of the frame. |
FACE_TOO_HIGH | 8 | The subject’s face is too high. The subject should move down towards the center of the frame. |
FACE_TOO_LOW | 9 | The subject’s face is too low. The subject should move up towards the center of the frame. |
INSUFFICIENT_LIGHTING | 10 | There is insufficient lighting. The subject should move to an area with more uniform lighting. |
LEFT_EYE_CLOSED | 11 | The subject’s left eye is closed. The subject should have both eyes open and visible to the camera. |
RIGHT_EYE_CLOSED | 12 | The subject’s right eye is closed. The subject should have both eyes open and visible to the camera. |
DARK_GLASSES_DETECTED | 13 | The subject is wearing dark or tinted glasss. The subject should remove the dark or tinted glasses. |
CameraOrientation
The device’s camera orientation.
Code | Value | Description |
---|---|---|
PORTRAIT | 0 | Portrait orientation - Camera is held vertically. |
LANDSCAPE | 1 | Landscape orientation - Camera is held horizontally. |
CameraPosition
The position of the camera on the device.
Code | Value | Description |
---|---|---|
FRONT | 0 | Camera is on the front of the device. Used for selfie captures. |
BACK | 1 | Camera is on the back of the device. Used for capturing other subjects. |
PackageType
The type of JSON request package to send to the Face Liveness back-end.
Code | Value | Description |
---|---|---|
HIGH_USABILITY | 0 | Request focused on high usability when analyzing on the back-end. |
BALANCED | 1 | Request focused on a balance of usability and security when analyzing on the back-end. |
HIGH_SECURITY | 2 | Request focused on high security when analyzing on the back-end |
WorkflowProperty
Configurable properties for Workflows.
Code | Value | Description |
---|---|---|
USERNAME | 0 | A string value used to set the Username/ID fields when communicating with the back-end. |
CAPTURE_TIMEOUT | 1 | A double value indicating the maximum duration in seconds for attempting face capture before ending the capture session. |
CAPTURE_PROFILE | 2 | A string value used to set the capture criteria for Face Capture. |
EncryptionType
The type of encryption for server packages.
Code | Value | Description |
---|---|---|
RSA_AES_256_CBC | 1 | RSA AES-256 CBC encryption. |
Constants
Workflow Names
Name | Value | Description |
---|---|---|
CHARLIE | Charlie | Name used in creating a Charlie workflow. Only available in JAVA and Swift. |
FOXTROT | Foxtrot | Name used in creating a Foxtrot workflow. |
Error Codes
Please see the API chapter for specific details on what each error means from any given function.
Error Codes
Error Code | Description |
---|---|
NULL_FACE_CAPTURE_OBJ | The Face Capture object was NULL. |
TRIAL_EXPIRATION_PASSED | The trial period for Face Capture has expired. Please contact Aware. |
OUT_OF_MEMORY | Failed to allocate sufficient memory for this function. |
INITIALIZATION_FAILED | Could not initialize Face Capture or a required component. |
INVALID_WORKFLOW | The Workflow object was invalid. |
UNKNOWN_WORKFLOW | No Workflow exists with the given name. |
WORKFLOW_IN_USE | Workflows cannot be modified while a capture session is in progress. |
INVALID_PACKAGE_TYPE | An invalid package type was specified. |
INVALID_WORKFLOW_PROPERTY | An invalid workflow property was specified. |
WORKFLOW_PROPERTY_TYPE_MISMATCH | The specified value is the wrong type for the specified property. |
INVALID_PROPERTY_VALUE | The specified value is not valid for the specified property. |
INVALID_CAMERA | The Camera object was invalid. |
UNKNOWN_CAMERA | No camera exists with the given name. |
CAMERA_IN_USE | Cameras cannot be modified while a capture session is in progress. |
CAMERA_INITIALIZATION | Camera could not be initialized. |
NO_INITIALIZABLE_CAMERAS | No cameras were detected. |
INVALID_CAMERA_POSITION | The camera position specified was an invalid value. |
INVALID_CAMERA_ORIENTATION | The camera orientation specified was an invalid value. |
INVALID_CAPTURE_STATE | The Capture State object was invalid. |
NO_FRAME_AVAILABLE | No frame data is available from the capture state object. |
CAPTURE_IN_PROGRESS | A capture session is already in progress. |
CAPTURE_SESSION_UNAVAILABLE | The previous capture session for the specified Workflow was aborted. |
NO_CAPTURE_PROFILE | No capture profile was specified by the Workflow. |
INVALID_CAPTURE_PROFILE | The capture profile set in the Workflow was invalid. |
WORKFLOW_INCOMPATIBLE_WITH_CAMERA | The selected Camera and Workflow are incompatible. Please see the Workflow description for usable camera criteria. |
INVALID_ENCRYPTION_TYPE | An invalid encryption type was specified. |
INVALID_ENCRYPTION_PUBLIC_KEY | The encryption public key specified was invalid or malformed. |
Software Acknowledgments
Aware Face Capture libraries incorporate third-party software. See the LICENSE file installed with the FaceCapture software for the full license agreement.