Android SDK
Using our Android SDK makes it easy to integrate Shen.AI into a new or existing native Android app.
The best way to get started is to download and build the example app - available here on GitHub.
Installing the SDK package
Download the Shen.AI aar packages (shenai_sdk
and shenai_sdk_android
) and add them to your project as a dependencies in build.gradle
.
dependencies {
implementation files('./path/to/your/shenai_sdk.aar')
implementation files('./path/to/your/shenai_sdk_android.aar')
}
Needed permissions
To use Shen.AI your app needs to have certain permissions. Obviously, permission to use camera but also Internet access to check the license. It is possible to use Shen.AI without Internet connection, but first usage on the device always needs to check the license, so the permission is obligatory. Adding following lines to your AndroidManifest.xml
will suffice.
<uses-permission android:name="android.permission.CAMERA" />
<uses-feature android:name="android.hardware.camera" android:required="true" />
<uses-permission android:name="android.permission.INTERNET"/>
Preparing necessary app components
To use Shen.AI your Android app needs to fulfill two conditions. First you need to have an Android Activity
, which you can later attach to Shen.AI handler, as in the example app:
import ai.mxlabs.shenai_sdk_android.ShenAIAndroidSDK
...
private val shenaiSDKHandler = ShenAIAndroidSDK()
...
shenaiSDKHandler.attachToActivity(this)
The attached activity is used for camera initialization and to access app files.
The second condition is to provide an Android Surface
into which Shen.AI can render the camera stream along with the measurement information so that you can display it in your app. In example providing the surface to Shen.AI is implemented inside the surfaceCreated
callback:
override fun surfaceCreated(holder: SurfaceHolder) {
renderingSurface = holder!!.surface
shenaiSDKHandler.provideSurfaceForRendering(renderingSurface)
...
}
Initializing the SDK
Now you should be ready to initialize Shen.AI. You have to provide API_KEY
and USER_ID
to check the Shen.AI license.
val settings = shenaiSDKHandler.defaultInitializationSettings
var result = shenaiSDKHandler.initialize(API_KEY, USER_ID, settings);
result
variable can take values according to enum
enum InitializationResult {
OK, // Initialization successful!
INVALID_API_KEY, // Invalid api key, check your license
CONNECTION_ERROR, // Couldn't check the license, check your connection
INVALID // Initialization not successful because of internal error
}
You can check if Shen.AI in initialized by calling isInitialized()
. You can also deinitialize it, to start a new session with deinitialize()
.
Starting the measurement
After initialization Shen.AI is ready to perform the measurement. For that purpose, the user needs to locate their well-lightened face in the middle of the screen. You can provide the feedback for the user about their face position and lighting conditions by using getFaceState()
and getLightingConditions()
calls. They return the following enums accordingly:
enum FaceState {
OK,
NOT_CENTERED,
TOO_CLOSE,
TOO_FAR,
UNSTABLE, // too much head movement
INVALID,
}
enum LightingConditions {
OK,
DARK,
TOO_DARK,
TOO_BRIGHT_BACKGROUND,
TOO_BRIGHT_FACE,
TOO_BRIGHT,
INVALID
}
You can check if the conditions are satisfied by calling isReadyForMeasurement()
. If the face is located properly you should be able to start the measurement. beginMeasurement()
call returns following enum informing if the measurement has successfully started or the reason of failure:
enum BeginMeasurementStatus {
STARTED, // Measurement started
RESUMED, // Measurement resumed after being paused TODO: should be in the interface?
NOT_STARTED_CAMERA, // Measurement not started because of camera problem
NOT_STARTED_FACE, // Measurement not started because of wrong face position
NOT_STARTED_LIGHTING, // Measurement not started because of wrong lighting conditions
INVALID // Measurement not started because of internal error
}
You can also check that the measurement has started by calling isMeasuring()
. If you want to abort the measurement call abortMeasurement()
.
During the measurement
Shen.AI is performing the measurement in the background and rendering the blood flow on the given surface, but you can still provide user feedback about their face position and lighting conditions.
To see what is happening underneath you should be calling getEngineState()
. It returns the following enum informing you about the state of the Shen.AI engine and the measurement.
enum EngineState {
NOT_READY, // Not ready (generic)
NOT_READY_FACE, // Face not detected
NOT_READY_PARTIAL_FACE, // Face is not fully in the frame
READY, // Face detected in acceptable position, ready to start
PERFORMING_INITIAL_FIT, // Facial model fitting in progress
TRACKING_SIGNAL_SHORT, // Tracking active, signal too short for any conclusions
TRACKING_BAD_SNR, // Tracking active, last frames have poor signal quality
TRACKING_GOOD_SNR, // Tracking active, last frames have good signal quality
TRACKING_LOST, // Tracking lost, trying to relocate
FINALIZING, // Measurement has satisfied end conditions and is performing final computations
SUCCESS, // Measurement has satisfied end conditions and finished
FAILURE, // Measurement has failed and finished
ENGINE_NOT_INITIALIZED // Engine not initialized
}
In particular, you should wait for FINALIZING
, SUCCESS
or FAILURE
values, because they indicate that the measurement has finished.
For first 5-10 seconds you can call getHeartRateProgressPercentage()
returning float
between 0 and 100 informing about progress of heart rate calibration. When the value is 100 you can check current heart rate in real-time using getLatestHeartRate()
call. Other metrics, such as HRV, are available after the full measurement is completed.
To check the progress of the whole measurement you can call getMeasurementProgressPercentage()
returning float
from 0 to 100.
For measured values to be accurate the quality of the extracted blood flow signal is important. You can check how this quality varies calling getCurrentSignalQualityMetric()
returning float
value representing how good was last 5 seconds of the signal. If the quality is too low, you will see no progress in the measurement completeness.
Measurement completed
When the Shen.AI engine enters the SUCCESS
state it means that the measurement has finished. You can now obtain the results calling getLatestMeasurementSummary()
, which returns the following structure:
class MeasurementSummary {
public float hr_bpm; // Heart rate in "beats per minute"
public float hrv_sdnn_ms; // HRV using SDNN metric in ms
public float br_bpm; // Breathing rate in "breaths per minute"
}
Testing
For integration testing purposes we provide also crash()
function in the interface. After calling it you should expect the Shen.AI to crash.