Face Capture

Estimated reading: 20 minutes 100 views

Introduction

Description

The Face Capture SDK is the client side library to be used with the Face Liveness Server product for determining the liveness of a subject. It is used to capture face images and package them according to the Face Liveness Server specifications. The Face Liveness Server will make a determination for whether or not the subject is live. The library does not directly communicate with any servers; it provides a package for the application to deliver to the Face Liveness Server. It is up to the application developer to implement any server communication and security.

Facial images are captured based on the selected capture profile. Capture profiles specify the criteria the subject must meet in order to successfully capture images and create a server package. Profiles are provided with this installer to be used as a base for capturing faces that are optimized for liveness processing on the back-end. Profiles can be customized to meet any additional requirements for capture.

Design Features

  • Light-weight on-device size
  • Easy to integrate API
  • No direct UI activity to allow for a fully customizable user experience
  • Handles operation of device hardware
  • Optimized data collection for back-end analysis
  • No direct server interaction. Applications have full control over when and how they interact with the back-end services.

Platforms

  • Android 8.0 and higher

System Requirements

Android

  • Android 8.0 or higher
  • Android NDK 20 or higher
  • API 24 or higher supported on device
  • Device supports Camera2 integration
  • Users must grant access to CAMERA permissions

Installation

Installed Components

The SDK installs various files in the subdirectories it creates. The subdirectories and their contents are listed below, along with a short description of the contents of each directory or file.

Top Level Directory bin, java, lib, manuals, samples, src-demo.

  • changes.txt – Change log for each release of Face Capture.
  • LICENSE – Aware license agreement
  • bin – Directory containing pre-built Face Capture Demo APKs.
  • lib – Directory containing the Android AAR files to use in integration.
  • manuals – Directory containing Face Capture documentation.
  • samples – Directory containing profiles.
  • src-demo – Directory containing Aware demo source code.

Application Design

Overview

The Face Capture SDK’s purpose in an application is to initiate and coordinate a controlled capture of a subject’s Face biometric data. The library is re-entrant and can be used from multiple threads. However, only one capture session can occur at a time. The library provides functionality for:

  • Intializing and operating the device’s cameras
  • Selecting which camera to use in facial capture
  • Providing preview image data to the application
  • Processing images and returning feedback codes used to guide the user experience
  • Specifying a maximum duration to attempt face capture
  • Updating the application with the current capture session status
  • Creating a JSON package compatible with the Aware Face Liveness Server back-end

Integration Tasks

The following is a quick checklist of objects to create and parameters to configure in order to capture a face.

  • Create a Face Capture library
  • Create a Workflow
  • Set the Capture Profile for the Workflow
  • Specify a Capture Timeout for the Workflow (optional, but recommended)
  • Specify a Username for the Workflow (optional)
  • Get the list of cameras for either the front or back side of the device
  • Select a camera from the list
  • Set up a polling thread for getting Capture States
  • Start a Capture Session with the Workflow and Camera
  • Display images and feedback from the capture session to guide the user to a successful capture
  • Handle the three cases where the Capture Session will end
  • Upon receiving the Completed status, the Capture Session has successfully captured. The application should retrieve the server package from the Workflow used.
  • Upon receiving the Timedout status, the Capture Session was unsuccessful. The application has been notified that the user did not capture in the time allowed.
  • Upon receiving the Aborted status, the Capture session was unsuccessful. The application should handle this case as a camera/hardware failure.
  • If successful, send the server package to the back-end.

Server Communication

The Face Capture SDK does not communicate directly with any server. All server communication is to be handled by the application integrating against it. Instead, the Face Capture SDK provides a request package to be delivered to the back-end. Some of the properties of the request package will differ based on which Workflow was created and what properties were used in the capture session.

The Face Capture SDK does not parse or interpret results from the Face Liveness server. Interpretation of the server results are left to the integrator and how they have delegated responsibilities between their application and back-end.

Data Security

The FaceCapture function getEncryptedServerPackage() returns an encrypted JSON package. There is a matching decryption function in the Face Liveness server. The encryption used is asymmetric RSA based on a AES 256 cipher block chain.

Workflows

Overview

Workflows are a pre-defined set of rules and parameters used to perform automatic collection of subject data for determining liveness. Workflows are created by the Face Capture library using their name. Capture Sessions are started using a Workflow and Camera to manage its settings. Some settings are available for modifcation through the Workflow such as:

  • A username to identify the subject in the transmitted JSON for reporting purposes.
  • A timeout duration which stops the Capture Session once the specified amount of time has passed. Setting a zero second timeout disbales the Capture Session from timing out.
  • A capture profile specifying the criteria used to determine when to automatically capture the subject’s face.

Please see the (Table 1) table below for a general overview of the available Workflows.

FeatureCharlieFoxtrot
Device TypeMobile DeviceMobile Device

Device Type – The type of device to use this workflow with. Different workflows perform captures differently based on device type. For example, using a mobile device workflow when capturing with a kiosk will result in inaccurate or poor performance.

Note : The Charlie workflow is only available through the Android JAVA and iOS Swift interfaces.

Workflow Package Types

After a successful capture session, the back-end request package will be available through the Workflow. There are three different types of back-end package that can be obtained:

  • High Usability – A request for the back-end focussed on usability.
  • High Security – A request for the back-end focussed on security.
  • Balanced – A request for the back-end with a mix of security and usability.

It is recommended to start with a Balanced approach and adjust to either High Usability or High Security based on testing.

JAVA API

FaceCaptureJNI

Library object that manages all other Face Capture API classes.

Destroy

Description: Destroy the FaceCaptureJNI object. This will free the memory used by the native library. Calling destroy will invalidate this object and all objects created by it. Attempting to use any functions with this or any other invalidated object will result in the NULL_FACE_CAPTURE_OBJ error being thrown.

Function Definition:

void destroy()

Returns: None

Throws: None

Workflow Create

Description: Create a Workflow object using a workflow name. The Workflow object will become invalid if the FaceCaptureJNI used to create it is destroyed.

Function Definition:

IFaceCapture.IWorkflow workflowCreate(String workflowName)
Parameter NameParameter TypeDescription
workflowNameStringName of the workflow to create

Returns: A new Workflow object.Throws:

  • NULL_FACE_CAPTURE_OBJ – The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
  • UNKNOWN_WORKFLOW – The workflowName specified did not match any of the valid workflow names. Please see the Constants section for the list of valid names.

Get Camera List

Description: Get a list of available Camera objects. Use CameraPosition.FRONT to get all “selfie”/front facing cameras. Use CameraPosition.BACK to get all cameras on the back of the device.

Function Definition:

IFaceCapture.ICamera[] getCameraList(CameraPosition position)
Parameter NameParameter TypeDescription
positionCameraPositionPosition of the cameras on the device

Returns: An array of Cameras for the camera position specified.Throws:

  • NULL_FACE_CAPTURE_OBJ – The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
  • INVALID_CAMERA_POSITION – The position value specified was not a valid CameraPosition.

Start Capture Session

Description: Start a capture session using the specified Workflow and Camera objects. While capturing, you cannot modify either object until the capture session ends. A capture session can end via a stopCaptureSession() call, if the capture session timeout expires, if there is a hardware issue, or if it captures successfully.

Function Definition:

void startCaptureSession(IFaceCapture.IWorkflow workflow, IFaceCapture.ICamera camera)
Parameter NameParameter TypeDescription
workflowIWorkflowWorkflow containing settings for the capture session
cameraICameraCamera to be used in the capture session

Returns: NoneThrows:

  • NULL_FACE_CAPTURE_OBJ – The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
  • INVALID_WORKFLOW – The Workflow object has been invalidated by destroy() and cannot be used.
  • INVALID_CAMERA – The Camera object has been invalidated by destroy() and cannot be used.
  • NO_CAPTURE_PROFILE – The CAPTURE_PROFILE WorkflowProperty was not specified. A valid CAPTURE_PROFILE must be set.
  • INVALID_CAPTURE_PROFILE – The CAPTURE_PROFILE WorkflowProperty specified was not profile valid XML.

Stop Capture Session

Description: Stop the current capture session. If there is no capture session running, this is a no-op. The next CaptureState will contain the STOPPED CaptureSessionStatus indicating that the session has successfully stopped. The Frame will not be retrievable and the Feedback may be safely ignored from this CaptureState.

Function Definition:

void stopCaptureSession()

Returns: NoneThrows:

  • NULL_FACE_CAPTURE_OBJ – The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.

Get Capture Session State

Description: Blocking function that retrieves the capture session state object.

Function Definition:

ICaptureState getCaptureSessionState()

Returns: A new CaptureState object.Throws:

  • NULL_FACE_CAPTURE_OBJ – The FaceCapture object has been invalidated by destroy() and cannot be used.

Get Server Package

Description: Get the JSON server package as a string value.

Function Definition:

String getServerPackage(IWorkflow workflow, PackageType packageType)
Parameter NameParameter TypeDescription
workflowIWorkflowWorkflow containing settings for the capture session
packageTypePackageTypeThe requested package type.

Returns: The string representation of the JSON server package.Throws:

  • NULL_FACE_CAPTURE_OBJ – The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
  • INVALID_WORKFLOW – The Workflow object has been invalidated by destroy() and cannot be used.
  • CAPTURE_SESSION_UNAVAILABLE – There has not been a successfully completed capture session. No data available.

Get Encrypted Server Package

Description: Get the encrypted JSON server package as a string value.

Function Definition:

String getEncryptedServerPackage(EncryptionType encryptionType, String encryptionKey, IWorkflow workflowHandle, PackageType packageType)
Parameter NameParameter TypeDescription
encryptionTypeEncryptionTypeType of encryption
encryptionKeyStringString containing encryption key
workflowHandleWorkflowWorkflow object containing settings for the capture session
packageTypePackageTypeType of JSON request package

Returns: The string representation of the encrypted JSON server package.Throws:

  • NULL_FACE_CAPTURE_OBJ – The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
  • INVALID_WORKFLOW – The Workflow object has been invalidated by destroy() and cannot be used.
  • INVALID_ENCRYPTION_TYPE – The EncryptionType enumeration is not a valid type and cannot be used.
  • INVALID_ENCRYPTION_PUBLIC_KEY – The String containing the encryption key is not a valid key and cannot be used.
  • CAPTURE_SESSION_UNAVAILABLE – There has not been a successfully completed capture session. No data available.

Get Captured Image

Description: Retrieves the stored capture image from the given Workflow.

Function Definition:

byte[] getCapturedImage( IFaceCapture.IWorkflow workflow )
Parameter NameParameter TypeDescription
workflowIWorkflowSpecified workflow to retrieve a capture image from.

Returns: The capture image that was retrieved from the Workflow. The image is returned as a JPG.Throws:

  • NULL_FACE_CAPTURE_OBJ – The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
  • INVALID_WORKFLOW – The Workflow object has been invalidated by destroy() and cannot be used.
  • CAPTURE_SESSION_UNAVAILABLE – There has not been a successfully completed capture session. No data available.

Capture Session Get Capture Region

Description: Function that retrieves the region of the image where the face will be captured in the current Capture Session.

Function Definition:

Rectangle captureSessionGetCaptureRegion()

Returns: A rectangle defining the region where the face will be captured. The region is a rectangle is defined by a point (x,y) at the upper left corner along with a width and height.Throws:

  • NULL_FACE_CAPTURE_OBJ – The FaceCapture object has been invalidated by destroy() and cannot be used.
  • CAPTURE_SESSION_UNAVAILABLE – There are no capture sessions currently running

Capture Session Enable Autocapture

Description: Enable or disable the autocapture for the capture session. If there is no capture session running, this is a no-op.

Function Definition:

void captureSessionEnableAutocapture(boolean enableAutocapture)
Parameter NameParameter TypeDescription
enableAutocaptureBooleanBoolean value on whether to enable autocapture

Returns: NoneThrows:

  • NULL_FACE_CAPTURE_OBJ – The FaceCapture object has been invalidated by destroy() and cannot be used.
  • CAPTURE_SESSION_UNAVAILABLE – There are no capture sessions currently running

Get Version

Description: Get the Face Capture SDK version as an integer value.

Function Definition:

int getVersion()

Returns: The integer representation of the Face Capture SDK version in AABBCCDD format.

  • AA is the major version
  • BB is the minor version
  • CC is the bug fix version
  • DD is the build version

Throws: None

Get Version String

Description: Get the Face Capture SDK version as a string value.

Function Definition:

String getVersionString()

Returns: The string representation of the Face Capture SDK version.

Throws: None

CaptureState

The Capture State object provides a snap-shot of the capture session’s state when created. The data provided by a single Capture State will never change. A new Capture State must be retrieved in order to get a new status update from the capture session.

Destroy Capture State

Description: Destroys the CaptureState object. This will free the memory used by the native library. Calling destroy will invalidate this object. Attempting to use any functions with this invalidated CaptureState will result in the INVALID_CAPTURE_STATE error being thrown.

Function Definition:

void destroy()

Returns: NoneThrows:

  • NULL_FACE_CAPTURE_OBJ – The FaceCapture object has been invalidated by destroy() and cannot be used.
  • INVALID_CAPTURE_STATE – The CaptureState object has been invalidated by destroy() and cannot be used.

Get Status

Description: Function that retrieves the capture session’s status from when the Capture State was created.

Function Definition:

CaptureSessionStatus getStatus()

Returns: The capture sessions status from when the Capture State was created. See the CaptureSessionStatus enumeration descriptions for details.Throws:

  • NULL_FACE_CAPTURE_OBJ – The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
  • INVALID_CAPTURE_STATE – The CaptureState object has been invalidated by destroy() and cannot be used.

Get Frame

Description: Function that retrieves the preview frame from when the Capture State was created.

Function Definition:

byte[] getFrame()

Returns: A byte array containing a preview frame from the capture session. The frame is in JPG format. The frame is either 480×640 in portrait mode or 640×480 in landscape mode.Throws:

  • NULL_FACE_CAPTURE_OBJ – The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
  • INVALID_CAPTURE_STATE – The CaptureState object has been invalidated by destroy() and cannot be used.
  • NO_FRAME_AVAILABLE – The capture session ended before the next frame was ready. There is no frame available for this Capture State.

Get Feedback

Description: Function that retrieves the subject feedback from when the Capture State was created.

Function Definition:

AutoCaptureFeedback getFeedback()

Returns: The most recent AutoCaptureFeedback update when the Capture State was created. See the AutoCaptureFeedback enumeration descriptions for more details.Throws:

  • NULL_FACE_CAPTURE_OBJ – The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
  • INVALID_CAPTURE_STATE – The CaptureState object has been invalidated by destroy() and cannot be used.

Workflow

Workflow object used to store capture session configurations.

Destroy

Description: Destroy the Workflow object. This will free the memory used by the native library. Calling destroy will invalidate this object. Attempting to use any functions with this invalidated Workflow will result in the NULL_FACE_CAPTURE_OBJ or INVALID_WORKFLOW error being thrown.

Function Definition:

void destroy()

Returns: None

Throws: None

Set Property String

Description: Set a string property value for the specified workflow property. Attempting to set a non-string property will result in a WORKFLOW_PROPERTY_TYPE_MISMATCH error being thrown. See the WorkflowProperty enumeration descriptions for more details.

Function Definition:

void setPropertyString(WorkflowProperty property, String value)
Parameter NameParameter TypeDescription
propertyWorkflowPropertyWorkflow property having its value set
valueStringNew value for the specified workflow property

Returns: NoneThrows:

  • NULL_FACE_CAPTURE_OBJ – The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
  • INVALID_WORKFLOW – The Workflow object has been invalidated by destroy() and cannot be used.
  • INVALID_WORKFLOW_PROPERTY – The property value was not a valid WorkflowProperty.
  • WORKFLOW_PROPERTY_TYPE_MISMATCH – The property value specified does not accept a String value.

Set Property Double

Description: Set a double property value for the specified workflow property. Attempting to set a non-double property will result in a WORKFLOW_PROPERTY_TYPE_MISMATCH error being thrown. See the WorkflowProperty enumeration descriptions for more details.

Function Definition:

void setPropertyDouble(WorkflowProperty property, double value)
Parameter NameParameter TypeDescription
propertyWorkflowPropertyWorkflow property having its value set
valuedoubleNew value for the specified workflow property

Returns: NoneThrows:

  • NULL_FACE_CAPTURE_OBJ – The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
  • INVALID_WORKFLOW – The Workflow object has been invalidated by destroy() and cannot be used.
  • INVALID_WORKFLOW_PROPERTY – The property value was not a valid WorkflowProperty.
  • WORKFLOW_PROPERTY_TYPE_MISMATCH – The property value specified does not accept a double value.

Camera

Camera object used to store capture session configurations.

Destroy

Description: Destroy the Camera object. This will free the memory used by the native library. Calling destroy will invalidate this object. Attempting to use any functions with an invalidated Camera will result in the NULL_FACE_CAPTURE_OBJ or INVALID_CAMERA error being thrown.

Function Definition:

void destroy()

Returns: None

Throws: None

Get Name

Description: Get the name of the Camera.

Function Definition:

String getName()

Returns: String containing the name of the Camera.Throws:

  • NULL_FACE_CAPTURE_OBJ – The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
  • INVALID_CAMERA – The Camera object has been invalidated by destroy() and cannot be used.

Set Orientation

Description: Set the orientation of the camera. This will affect how the capture session expects the camera to be oriented. See the CameraOrientation enumeration descriptions for more details.

Function Definition:

void setOrientation(CameraOrientation orientation)
Parameter NameParameter TypeDescription
orientationCameraOrientationNew camera orientation for this camera to be used in capture sessions

Returns: NoneThrows:

  • NULL_FACE_CAPTURE_OBJ – The FaceCaptureJNI object has been invalidated by destroy() and cannot be used.
  • INVALID_CAMERA – The Camera object has been invalidated by destroy() and cannot be used.
  • INVALID_CAMERA_ORIENTATION – The orientation specified was not a valid CameraOrientation.
  • CAMERA_IN_USE – The Camera is currently in use in a capture session and cannot be modified.

Enumerations

Enumerations

AutoCaptureFeedback

Feedback provided during a capture session indicating any facial quality issues with the subject.

CodeValueDescription
FACE_COMPLIANT0The subject’s face was compliant.
NO_FACE_DETECTED1No faces were detected in the image. The subject’s face must be wholly in frame.
MULTIPLE_FACES_DETECTED2Multiple faces were detected in the image. The subject must be the only face in frame.
INVALID_POSE3The subject’s pose is too far off center. The subject should directly face the camera.
FACE_TOO_FAR4The subject’s face is too far away. The subject should move closer to the camera.
FACE_TOO_CLOSE5The subject’s face is too close. The subject should move away from the camera.
FACE_ON_LEFT6The subject’s face is too far to the left. The subject should move to the center of the frame.
FACE_ON_RIGHT7The subject’s face is too far to the right. The subject should move to the center of the frame.
FACE_TOO_HIGH8The subject’s face is too high. The subject should move down towards the center of the frame.
FACE_TOO_LOW9The subject’s face is too low. The subject should move up towards the center of the frame.
INSUFFICIENT_LIGHTING10There is insufficient lighting. The subject should move to an area with more uniform lighting.
LEFT_EYE_CLOSED11The subject’s left eye is closed. The subject should have both eyes open and visible to the camera.
RIGHT_EYE_CLOSED12The subject’s right eye is closed. The subject should have both eyes open and visible to the camera.
DARK_GLASSES_DETECTED13The subject is wearing dark or tinted glasss. The subject should remove the dark or tinted glasses.

CameraOrientation

The device’s camera orientation.

CodeValueDescription
PORTRAIT0Portrait orientation – Camera is held vertically.
LANDSCAPE1Landscape orientation – Camera is held horizontally.

CameraPosition

The position of the camera on the device.

CodeValueDescription
FRONT0Camera is on the front of the device. Used for selfie captures.
BACK1Camera is on the back of the device. Used for capturing other subjects.

PackageType

The type of JSON request package to send to the Face Liveness back-end.

CodeValueDescription
HIGH_USABILITY0Request focused on high usability when analyzing on the back-end.
BALANCED1Request focused on a balance of usability and security when analyzing on the back-end.
HIGH_SECURITY2Request focused on high security when analyzing on the back-end

WorkflowProperty

Configurable properties for Workflows.

CodeValueDescription
USERNAME0A string value used to set the Username/ID fields when communicating with the back-end.
CAPTURE_TIMEOUT1A double value indicating the maximum duration in seconds for attempting face capture before ending the capture session.
CAPTURE_PROFILE2A string value used to set the capture criteria for Face Capture.

EncryptionType

The type of encryption for server packages.

CodeValueDescription
RSA_AES_256_CBC1RSA AES-256 CBC encryption.

Constants

Workflow Names

NameValueDescription
CHARLIECharlieName used in creating a Charlie workflow. Only available in JAVA and Swift.
FOXTROTFoxtrotName used in creating a Foxtrot workflow.

Error Codes

Please see the API chapter for specific details on what each error means from any given function.

Error Codes

Error CodeDescription
NULL_FACE_CAPTURE_OBJThe Face Capture object was NULL.
TRIAL_EXPIRATION_PASSEDThe trial period for Face Capture has expired. Please contact Aware.
OUT_OF_MEMORYFailed to allocate sufficient memory for this function.
INITIALIZATION_FAILEDCould not initialize Face Capture or a required component.
INVALID_WORKFLOWThe Workflow object was invalid.
UNKNOWN_WORKFLOWNo Workflow exists with the given name.
WORKFLOW_IN_USEWorkflows cannot be modified while a capture session is in progress.
INVALID_PACKAGE_TYPEAn invalid package type was specified.
INVALID_WORKFLOW_PROPERTYAn invalid workflow property was specified.
WORKFLOW_PROPERTY_TYPE_MISMATCHThe specified value is the wrong type for the specified property.
INVALID_PROPERTY_VALUEThe specified value is not valid for the specified property.
INVALID_CAMERAThe Camera object was invalid.
UNKNOWN_CAMERANo camera exists with the given name.
CAMERA_IN_USECameras cannot be modified while a capture session is in progress.
CAMERA_INITIALIZATIONCamera could not be initialized.
NO_INITIALIZABLE_CAMERASNo cameras were detected.
INVALID_CAMERA_POSITIONThe camera position specified was an invalid value.
INVALID_CAMERA_ORIENTATIONThe camera orientation specified was an invalid value.
INVALID_CAPTURE_STATEThe Capture State object was invalid.
NO_FRAME_AVAILABLENo frame data is available from the capture state object.
CAPTURE_IN_PROGRESSA capture session is already in progress.
CAPTURE_SESSION_UNAVAILABLEThe previous capture session for the specified Workflow was aborted.
NO_CAPTURE_PROFILENo capture profile was specified by the Workflow.
INVALID_CAPTURE_PROFILEThe capture profile set in the Workflow was invalid.
WORKFLOW_INCOMPATIBLE_WITH_CAMERAThe selected Camera and Workflow are incompatible. Please see the Workflow description for usable camera criteria.
INVALID_ENCRYPTION_TYPEAn invalid encryption type was specified.
INVALID_ENCRYPTION_PUBLIC_KEYThe encryption public key specified was invalid or malformed.

Software Acknowledgments

Aware Face Capture libraries incorporate third-party software. See the LICENSE file installed with the FaceCapture software for the full license agreement.

CONTENTS