Android SDK

Using our Android SDK makes it easy to integrate Shen.AI into a new or existing native Android app.

The best way to get started is to download and build the example app - available here on GitHub .

Installing the SDK package

Download the Shen.AI aar package (shenai_sdk.aar) and add it to your project as a dependency in build.gradle.

dependencies {
    implementation files('./path/to/your/shenai_sdk.aar')
}

Needed permissions

To use Shen.AI your app needs to have certain permissions. Obviously, permission to use camera but also Internet access to check the license. It is possible to use Shen.AI without Internet connection, but first usage on the device always needs to check the license, so the permission is obligatory. Adding following lines to your AndroidManifest.xml will suffice.

<uses-permission android:name="android.permission.CAMERA" />
<uses-feature android:name="android.hardware.camera" android:required="true" />
<uses-permission android:name="android.permission.INTERNET"/>
 

Preparing necessary app components

To use Shen.AI your Android app, you need to have an Android Activity, which you can use to initialize the SDK, as in the example app:

import ai.mxlabs.shenai_sdk.ShenAIAndroidSDK;
 
...
 
private val shenaiSDKHandler = ShenAIAndroidSDK()
 
...
 
shenaiSDKHandler.initialize(this, "YOUR_API_KEY", ...);
 

The attached activity is used for camera initialization and to access app files.

The activity is also necessary to create the View for displaying the SDK’s UI

import ai.mxlabs.shenai_sdk.ShenAIView;
 
...
 
ShenAIView shenaiView = new ShenAIView(this);
setContentView(shenaiView);
 

It is expected that the Activity passed to the ShenAIView is an instance of LifecycleOwner - in such a case, the view will properly react to app lifecycle events, releasing resources when the activity is paused.

If for some reason your Activity is not a LifecycleOwner, you can use the ShenAIView.activityPaused() and ShenAIView.activityResumed() methods to manually notify the view about lifecycle events.

Custom frame input

If your app already owns the video pipeline, you can initialize the SDK with CameraMode.CUSTOM_FRAMES. In this mode the SDK will not open a camera and will wait for frames from your app.

val settings = shenaiSDKHandler.getDefaultInitializationSettings()
settings.cameraMode = ShenAIAndroidSDK.CameraMode.CUSTOM_FRAMES
 
val initResult = shenaiSDKHandler.initialize(this, API_KEY, USER_ID, settings)

Frames are submitted through the native Android API as I420 data:

val metadata = ShenAIAndroidSDK.FrameMetadata().apply {
  timestampUs = System.nanoTime() / 1_000
  cameraFacingUser = true
}
 
val accepted = shenaiSDKHandler.handleNextFrame(i420FrameBytes, width, height, metadata)

timestampUs is optional. When you provide it, it should be a monotonic capture timestamp in microseconds. If you leave it unset or negative, the SDK synthesizes monotonic submission-time timestamps. For best accuracy, provide source timestamps when available and prefer uniform sampling of at least 30 FPS. In CustomFrames mode, submit frames already rotated and resized as you want them processed; SDK-side rotation, frameWidth, and frameHeight are ignored.

Further steps

Please see permissions, initialization, configuration and video measurement for further steps.