Face Capture

Estimated reading: 19 minutes 768 views

Introduction

Description

The Face Capture SDK is the client side library to be used with the Face Liveness Server product for determining the liveness of a subject. It is used to capture face images and package them according to the Face Liveness Server specifications. The Face Liveness Server will make a determination for whether or not the subject is live. The library does not directly communicate with any servers; it provides a package for the application to deliver to the Face Liveness Server. It is up to the application developer to implement any server communication and security.

Facial images are captured based on the selected capture profile. Capture profiles specify the criteria the subject must meet in order to successfully capture images and create a server package. Profiles are provided with this installer to be used as a base for capturing faces that are optimized for liveness processing on the back-end. Profiles can be customized to meet any additional requirements for capture.

Design Features

Light-weight on-device size
Easy to integrate API
No direct UI activity to allow for a fully customizable user experience
Handles operation of device hardware
Optimized data collection for back-end analysis
No direct server interaction. Applications have full control over when and how they interact with the back-end services.

Platforms

iOS 12.0 and higher

System Requirements

iOS

iOS 10 or higher
Users must grant access to CAMERA permissions

Application Design

Overview

The Face Capture SDK’s purpose in an application is to initiate and coordinate a controlled capture of a subject’s Face biometric data. The library is re-entrant and can be used from multiple threads. However, only one capture session can occur at a time. The library provides functionality for:

Intializing and operating the device’s cameras
Selecting which camera to use in facial capture
Providing preview image data to the application
Processing images and returning feedback codes used to guide the user experience
Specifying a maximum duration to attempt face capture
Updating the application with the current capture session status
Creating a JSON package compatible with the Aware Face Liveness Server back-end

Integration Tasks

The following is a quick checklist of objects to create and parameters to configure in order to capture a face.

Create a Face Capture library
Create a Workflow
Set the Capture Profile for the Workflow
Specify a Capture Timeout for the Workflow (optional, but recommended)
Specify a Username for the Workflow (optional)
Get the list of cameras for either the front or back side of the device
Select a camera from the list
Set up a polling thread for getting Capture States
Start a Capture Session with the Workflow and Camera
Display images and feedback from the capture session to guide the user to a successful capture
Handle the three cases where the Capture Session will end

Upon receiving the Completed status, the Capture Session has successfully captured. The application should retrieve the server package from the Workflow used.
Upon receiving the Timedout status, the Capture Session was unsuccessful. The application has been notified that the user did not capture in the time allowed.
Upon receiving the Aborted status, the Capture session was unsuccessful. The application should handle this case as a camera/hardware failure.

If successful, send the server package to the back-end.

Server Communication

The Face Capture SDK does not communicate directly with any server. All server communication is to be handled by the application integrating against it. Instead, the Face Capture SDK provides a request package to be delivered to the back-end. Some of the properties of the request package will differ based on which Workflow was created and what properties were used in the capture session.

The Face Capture SDK does not parse or interpret results from the Face Liveness server. Interpretation of the server results are left to the integrator and how they have delegated responsibilities between their application and back-end.

Data Security

The FaceCapture function getEncryptedServerPackage() returns an encrypted JSON package. There is a matching decryption function in the Face Liveness server. The encryption used is asymmetric RSA based on a AES 256 cipher block chain.

Workflows

Overview

Workflows are a pre-defined set of rules and parameters used to perform automatic collection of subject data for determining liveness. Workflows are created by the Face Capture library using their name. Capture Sessions are started using a Workflow and Camera to manage its settings. Some settings are available for modifcation through the Workflow such as:

A username to identify the subject in the transmitted JSON for reporting purposes.
A timeout duration which stops the Capture Session once the specified amount of time has passed. Setting a zero second timeout disbales the Capture Session from timing out.
A capture profile specifying the criteria used to determine when to automatically capture the subject’s face.

Please see the (Face Capture Workflow Summary) table below for a general overview of the available Workflows.

Feature	Charlie	Foxtrot
Device Type	Mobile Device	Mobile Device

Device Type - The type of device to use this workflow with. Different workflows perform captures differently based on device type. For example, using a mobile device workflow when capturing with a kiosk will result in inaccurate or poor performance.

Note : The Charlie workflow is only available through the iOS Swift interface.

Workflow Package Types

After a successful capture session, the back-end request package will be available through the Workflow. There are three different types of back-end package that can be obtained:

High Usability - A request for the back-end focussed on usability.
High Security - A request for the back-end focussed on security.
Balanced - A request for the back-end with a mix of security and usability.

It is recommended to start with a Balanced approach and adjust to either High Usability or High Security based on testing.

Swift API

Classes

FaceCapture

Library object that manages all other Face Capture API classes.

Dispose

Description: Dispose the FaceCapture object. This will free the memory used by the native library. Calling dispose will invalidate this object and all objects created by it. Attempting to use any functions with this or any other invalidated object will result in the NULL_FACE_CAPTURE_OBJ error being thrown.

Function Definition:

func dispose() -> Void

Returns: None

Throws: None

Workflow Create

Description: Create a Workflow object using a workflow name. The Workflow object will become invalid if the FaceCapture used to create it is disposed.

Function Definition:

func workflowCreate(workflowName: String) throws -> Workflow

Parameter Name	Parameter Type	Description
workflowName	String	Name of the workflow to create

Returns: A new Workflow object.Throws:

NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
UNKNOWN_WORKFLOW - The workflowName specified did not match any of the valid workflow names. Please see the Constants section for the list of valid names.

Get Camera List

Description: Get a list of available Camera objects. Use CameraPosition.FRONT to get all “selfie”/front facing cameras. Use CameraPosition.BACK to get all cameras on the back of the device.

Function Definition:

func getCameraList(position: CameraPosition) throws -> [Camera]

Parameter Name	Parameter Type	Description
position	CameraPosition	Position of the cameras on the device

Returns: An array of Cameras for the camera position specified.Throws:

NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
INVALID_CAMERA_POSITION - The position value specified was not a valid CameraPosition.

Start Capture Session

Description: Start a capture session using the specified Workflow and Camera objects. While capturing, you cannot modify either object until the capture session ends. A capture session can end via a stopCaptureSession() call, if the capture session timeout expires, if there is a hardware issue, or if it captures successfully.

Function Definition:

func startCaptureSession(workflowHandle: Workflow, cameraHandle: Camera) throws -> Void

Parameter Name	Parameter Type	Description
workflowHandle	Workflow	Workflow containing settings for the capture session
cameraHandle	Camera	Camera to be used in the capture session

Returns: NoneThrows:

NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
INVALID_WORKFLOW - The Workflow object has been invalidated by dispose() and cannot be used.
INVALID_CAMERA - The Camera object has been invalidated by dispose() and cannot be used.
NO_CAPTURE_PROFILE - The CAPTURE_PROFILE WorkflowProperty was not specified. A valid CAPTURE_PROFILE must be set.
INVALID_CAPTURE_PROFILE - The CAPTURE_PROFILE WorkflowProperty specified was not profile valid XML.

Stop Capture Session

Description: Stop the current capture session. If there is no capture session running, this is a no-op. The next CaptureState will contain the STOPPED CaptureSessionStatus indicating that the session has successfully stopped. The Frame will not be retrievable and the Feedback may be safely ignored from this CaptureState.

Function Definition:

func stopCaptureSession() throws -> Void

Returns: NoneThrows:

NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.

Get Capture Session State

Description: Blocking function that retrieves the capture session state object.

Function Definition:

func getCaptureSessionState() throws -> CaptureState

Returns: A new CaptureState object.Throws:

NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
INVALID_CAPTURE_STATE - The capture state is invalid.

Get Server Package

Description: Get the JSON server package as a string value.

Function Definition:

func getServerPackage(workflowHandle: Workflow, packageType: PackageType) throws -> String

Parameter Name	Parameter Type	Description
workflowHandle	Workflow	Workflow containing settings for the capture session
packageType	PackageType	The requested package type.

Returns: The string representation of the JSON server package.Throws:

NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
CAPTURE_SESSION_UNAVAILABLE - There are no capture sessions currently running

Get Encrypted Server Package

Description: Get the encrypted JSON server package as a string value.

Function Definition:

func getEncryptedServerPackage(encryptionType: EncryptionType, encryptionKey: String, workflowHandle: Workflow, packageType: PackageType) throws -> String

Parameter Name	Parameter Type	Description
encryptionType	EncryptionType	Type of encryption
encryptionKey	String	String containing encryption key
workflowHandle	Workflow	Workflow object containing settings for the capture session
packageType	PackageType	Type of JSON request package

Returns: The string representation of the encrypted JSON server package.Throws:

NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by destroy() and cannot be used.
INVALID_WORKFLOW - The Workflow object has been invalidated by destroy() and cannot be used.
INVALID_ENCRYPTION_TYPE - The EncryptionType enumeration is not a valid type and cannot be used.
INVALID_ENCRYPTION_PUBLIC_KEY - The String containing the encryption key is not a valid key and cannot be used.
CAPTURE_SESSION_UNAVAILABLE - There has not been a successfully completed capture session. No data available.

Get Captured Image

Description: Retrieves the stored capture image from the given Workflow.

Function Definition:

func getCapturedImage( workflow: Workflow ) throws -> [UInt8]

Parameter Name	Parameter Type	Description
workflow	Workflow	Specified workflow to retrieve a capture image from.

Returns: The capture image that was retrieved from the Workflow. The image is returned as a JPG.Throws:

NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by destroy() and cannot be used.
INVALID_WORKFLOW - The Workflow object has been invalidated by destroy() and cannot be used.
CAPTURE_SESSION_UNAVAILABLE - There has not been a successfully completed capture session. No data available.

Get Capture Session Region of Interest

Description: Get the area where the user’s face is expected for a capture to occur for the current capture session.

Function Definition:

func captureSessionGetCaptureRegion() throws -> CGRect

Returns: A rectangle defining the region where the face will be captured. The region is a rectangle is defined by a point (x,y) at the upper left corner along with a width and height.Throws:

NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
CAPTURE_SESSION_UNAVAILABLE - There are no capture sessions currently running

Capture Session Enable Autocapture

Description: Enable or disable the autocapture for the capture session. If there is no capture session running, this is a no-op.

Function Definition:

func captureSessionEnableAutocapture(enableAutocapture: Bool) throws -> Void

Parameter Name	Parameter Type	Description
enableAutocapture	Boolean	Boolean value on whether to enable autocapture

Returns: NoneThrows:

NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.

Get Version

Description: Get the Face Capture SDK version as an integer value.

Function Definition:

static func getVersion() throws -> Int32

Returns: The integer representation of the Face Capture SDK version in AABBCCDD format.

AA is the major version
BB is the minor version
CC is the bug fix version
DD is the build version

Throws: None

Get Version String

Description: Get the Face Capture SDK version as a string value.

Function Definition:

static func getVersionString() throws -> String

Returns: The string representation of the Face Capture SDK version.

Throws: None

CaptureState

Library object that abstracts the Face Capture Capture State.

Dispose

Description: Dispose the CaptureState object. This will free the memory used by the native library. Calling dispose will invalidate this object. Attempting to use any functions with this invalidated CaptureState will result in the NULL_FACE_CAPTURE_OBJ or INVALID_CAPTURE_STATE error being thrown.

Function Definition:

func dispose() -> Void

Returns: NoneThrows:

NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
INVALID_CAPTURE_STATE - The CaptureState object has been invalidated by dispose() and cannot be used.

Get Capture Session Status

Description: Retrieves the most recent status update from the capture session.

Function Definition:

func getCaptureSessionStatus() throws -> CaptureSessionStatus

Returns: The most recent status update from the capture session. See the CaptureSessionStatus enumeration descriptions for details.Throws:

NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.

Get Capture Session Frame

Description: Blocking function that retrieves the most recent preview frame from the capture session. If there is no capture session in progress, this function will throw the CAPTURE_SESSION_UNAVAILABLE error.

Function Definition:

func getCaptureSessionFrame() throws -> [UInt8]

Returns: A byte array containing a preview frame from the capture session. The frame is in JPG format. The frame is either 480x640 in portrait mode or 640x480 in landscape mode.Throws:

NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
CAPTURE_SESSION_UNAVAILABLE - There are no capture sessions currently running.

Get Capture Session Feedback

Description: Retrieves the most recent preview feedback from the capture session. If there is no capture session in progress, this function will throw the CAPTURE_SESSION_UNAVAILABLE error.

Function Definition:

func getCaptureSessionFeedback() throws -> AutoCaptureFeedback

Returns: The most recent AutoCaptureFeedback update from the capture session. See the AutoCaptureFeedback enumeration descriptions for more details.Throws:

NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
CAPTURE_SESSION_UNAVAILABLE - There are no capture sessions currently running.

Workflow

Workflow object used to store capture session configurations.

Dispose

Description: Dispose the Workflow object. This will free the memory used by the native library. Calling dispose will invalidate this object. Attempting to use any functions with this invalidated Workflow will result in the NULL_FACE_CAPTURE_OBJ or INVALID_WORKFLOW error being thrown.

Function Definition:

func dispose() -> Void

Returns: None

Throws: None

Set Property Double

Description: Set a double property value for the specified workflow property. Attempting to set a non-double property will result in a WORKFLOW_PROPERTY_TYPE_MISMATCH error being thrown. See the WorkflowProperty enumeration descriptions for more details.

Function Definition:

func setPropertyDouble(property: WorkflowProperty, value: Double) throws -> Void

Parameter Name	Parameter Type	Description
property	WorkflowProperty	Workflow property having its value set
value	double	New value for the specified workflow property

Returns: NoneThrows:

NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
INVALID_WORKFLOW - The Workflow object has been invalidated by dispose() and cannot be used.
INVALID_WORKFLOW_PROPERTY - The property value was not a valid WorkflowProperty.
WORKFLOW_PROPERTY_TYPE_MISMATCH - The property value specified does not accept a double value.

Set Property String

Description: Set a string property vlaue for the specified workflow property. Attempting to set a non-string property will result in a WORKFLOW_PROPERTY_TYPE_MISMATCH error being thrown. See the WorkflowProperty enumeration descriptions for more details.

Function Definition:

func setPropertyString(property: WorkflowProperty, value: String) throws -> Void

Parameter Name	Parameter Type	Description
property	WorkflowProperty	Workflow property having its value set
value	String	New value for the specified workflow property

Returns: NoneThrows:

NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
INVALID_WORKFLOW - The Workflow object has been invalidated by dispose() and cannot be used.
INVALID_WORKFLOW_PROPERTY - The property value was not a valid WorkflowProperty.
WORKFLOW_PROPERTY_TYPE_MISMATCH - The property value specified does not accept a String value.

Camera

Camera object used to store capture session configurations.

Dispose

Description: Dispose the Camera object. This will free the memory used by the native library. Calling dispose will invalidate this object. Attempting to use any functions with an invalidated Camera will result in the NULL_FACE_CAPTURE_OBJ or INVALID_CAMERA error being thrown.

Function Definition:

func dispose() -> Void

Returns: None

Throws: None

Get Name

Description: Get the name of the Camera.

Function Definition:

func getName() throws -> String

Returns: String containing the name of the Camera.Throws:

NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
INVALID_CAMERA - The Camera object has been invalidated by dispose() and cannot be used.

Set Orientation

Description: Set the orientation of the camera. This will affect how the capture session expects the camera to be oriented. See the CameraOrientation enumeration descriptions for more details.

Function Definition:

func setOrientation(orientation: CameraOrientation) throws -> Void

Parameter Name	Parameter Type	Description
orientation	CameraOrientation	New camera orientation for this camera to be used in capture sessions

Returns: NoneThrows:

NULL_FACE_CAPTURE_OBJ - The FaceCapture object has been invalidated by dispose() and cannot be used.
INVALID_CAMERA - The Camera object has been invalidated by dispose() and cannot be used.
INVALID_CAMERA_ORIENTATION - The orientation specified was not a valid CameraOrientation.
CAMERA_IN_USE - The Camera is currently in use in a capture session and cannot be modified.

Enumerations

AutoCaptureFeedback

Feedback provided during a capture session indicating any facial quality issues with the subject.

Code	Value	Description
FACE_COMPLIANT	0	The subject’s face was compliant.
NO_FACE_DETECTED	1	No faces were detected in the image. The subject’s face must be wholly in frame.
MULTIPLE_FACES_DETECTED	2	Multiple faces were detected in the image. The subject must be the only face in frame.
INVALID_POSE	3	The subject’s pose is too far off center. The subject should directly face the camera.
FACE_TOO_FAR	4	The subject’s face is too far away. The subject should move closer to the camera.
FACE_TOO_CLOSE	5	The subject’s face is too close. The subject should move away from the camera.
FACE_ON_LEFT	6	The subject’s face is too far to the left. The subject should move to the center of the frame.
FACE_ON_RIGHT	7	The subject’s face is too far to the right. The subject should move to the center of the frame.
FACE_TOO_HIGH	8	The subject’s face is too high. The subject should move down towards the center of the frame.
FACE_TOO_LOW	9	The subject’s face is too low. The subject should move up towards the center of the frame.
INSUFFICIENT_LIGHTING	10	There is insufficient lighting. The subject should move to an area with more uniform lighting.
LEFT_EYE_CLOSED	11	The subject’s left eye is closed. The subject should have both eyes open and visible to the camera.
RIGHT_EYE_CLOSED	12	The subject’s right eye is closed. The subject should have both eyes open and visible to the camera.
DARK_GLASSES_DETECTED	13	The subject is wearing dark or tinted glasss. The subject should remove the dark or tinted glasses.

CameraOrientation

The device’s camera orientation.

Code	Value	Description
PORTRAIT	0	Portrait orientation - Camera is held vertically.
LANDSCAPE	1	Landscape orientation - Camera is held horizontally.

CameraPosition

The position of the camera on the device.

Code	Value	Description
FRONT	0	Camera is on the front of the device. Used for selfie captures.
BACK	1	Camera is on the back of the device. Used for capturing other subjects.

PackageType

The type of JSON request package to send to the Face Liveness back-end.

Code	Value	Description
HIGH_USABILITY	0	Request focused on high usability when analyzing on the back-end.
BALANCED	1	Request focused on a balance of usability and security when analyzing on the back-end.
HIGH_SECURITY	2	Request focused on high security when analyzing on the back-end

WorkflowProperty

Configurable properties for Workflows.

Code	Value	Description
USERNAME	0	A string value used to set the Username/ID fields when communicating with the back-end.
CAPTURE_TIMEOUT	1	A double value indicating the maximum duration in seconds for attempting face capture before ending the capture session.
CAPTURE_PROFILE	2	A string value used to set the capture criteria for Face Capture.

EncryptionType

The type of encryption for server packages.

Code	Value	Description
RSA_AES_256_CBC	1	RSA AES-256 CBC encryption.

Constants

Workflow Names

Name	Value	Description
CHARLIE	Charlie	Name used in creating a Charlie workflow. Only available in JAVA and Swift.
FOXTROT	Foxtrot	Name used in creating a Foxtrot workflow.

Error Codes

Please see the API chapter for specific details on what each error means from any given function.

Error Codes

Error Code	Description
NULL_FACE_CAPTURE_OBJ	The Face Capture object was NULL.
TRIAL_EXPIRATION_PASSED	The trial period for Face Capture has expired. Please contact Aware.
OUT_OF_MEMORY	Failed to allocate sufficient memory for this function.
INITIALIZATION_FAILED	Could not initialize Face Capture or a required component.
INVALID_WORKFLOW	The Workflow object was invalid.
UNKNOWN_WORKFLOW	No Workflow exists with the given name.
WORKFLOW_IN_USE	Workflows cannot be modified while a capture session is in progress.
INVALID_PACKAGE_TYPE	An invalid package type was specified.
INVALID_WORKFLOW_PROPERTY	An invalid workflow property was specified.
WORKFLOW_PROPERTY_TYPE_MISMATCH	The specified value is the wrong type for the specified property.
INVALID_PROPERTY_VALUE	The specified value is not valid for the specified property.
INVALID_CAMERA	The Camera object was invalid.
UNKNOWN_CAMERA	No camera exists with the given name.
CAMERA_IN_USE	Cameras cannot be modified while a capture session is in progress.
CAMERA_INITIALIZATION	Camera could not be initialized.
NO_INITIALIZABLE_CAMERAS	No cameras were detected.
INVALID_CAMERA_POSITION	The camera position specified was an invalid value.
INVALID_CAMERA_ORIENTATION	The camera orientation specified was an invalid value.
INVALID_CAPTURE_STATE	The Capture State object was invalid.
NO_FRAME_AVAILABLE	No frame data is available from the capture state object.
CAPTURE_IN_PROGRESS	A capture session is already in progress.
CAPTURE_SESSION_UNAVAILABLE	The previous capture session for the specified Workflow was aborted.
NO_CAPTURE_PROFILE	No capture profile was specified by the Workflow.
INVALID_CAPTURE_PROFILE	The capture profile set in the Workflow was invalid.
WORKFLOW_INCOMPATIBLE_WITH_CAMERA	The selected Camera and Workflow are incompatible. Please see the Workflow description for usable camera criteria.
INVALID_ENCRYPTION_TYPE	An invalid encryption type was specified.
INVALID_ENCRYPTION_PUBLIC_KEY	The encryption public key specified was invalid or malformed.

Software Acknowledgments

Aware Face Capture libraries incorporate third-party software. See the LICENSE file installed with the FaceCapture software for the full license agreement.