Initialization

Before starting the video measurement, you’ll need to initialize the SDK.

Under the hood that step verifies the license and connects to a native platform camera device. The app must have camera permissions before this step.

You’ll need to provide an API key - see Authorization for more information.

lib/main.dart

import 'package:shenai_sdk/shenai_sdk.dart';
 
var API_KEY = ""
var USER_ID = ""
 
final _initResult = await ShenaiSdk.initialize(API_KEY, USER_ID);

App.tsx

import { initialize, InitializationResult } from "react-native-shenai-sdk";
 
const API_KEY = "";
 
const initResult = await initialize(API_KEY);

AppDelegate.swift

import ShenaiSDK
 
let API_KEY = "";
 
let init_result = ShenaiSDK.initialize(API_KEY, userID: nil, settings: nil)

MainActivity.kt

import ai.mxlabs.shenai_sdk.ShenAIAndroidSDK
 
val API_KEY = ""
val USER_ID = ""
 
val initResult = shenaiSDKHandler.initialize(this, API_KEY, USER_ID)

index.ts

import CreateShenaiSDK from "shenai-sdk";
 
const shenaiSDK = await CreateShenaiSDK();
 
const API_KEY = ""; // Use your API key here
const USER_ID = "";
 
shenaiSDK.initialize(
  API_KEY,
  USER_ID,
  {
    /* settings */
  },
  (initResult) => {
    /* handle result */
  }
);

Error handling

You should check if the initialization was successful before proceeding further.

Possible errors at this stage include:

INVALID_API_KEY - the provided API key is malformed or inactive
CONNECTION_ERROR - there is a problem with network connection to the licensing server
INTERNAL_ERROR - other errors such as missing camera permissions

enum InitializationResult {
  success,
  failInvalidApiKey,
  failConnectionError,
  failInternalError
}
 
if (_initResult != InitializationResult.success) {
  // handle error
}

enum InitializationResult {
  OK,
  INVALID_API_KEY,
  CONNECTION_ERROR,
  INTERNAL_ERROR,
}
 
if (initResult != InitializationResult.OK) {
  // handle error
}

enum InitializationResult { 
  case success
  case failureInvalidApiKey
  case failureConnectionError
  case failureInternalError 
};
 
if (init_result != .success) {
// handle error
}

enum InitializationResult {
  OK,
  INVALID_API_KEY,
  CONNECTION_ERROR,
  INVALID
}
 
if (initResult != InitializationResult.OK) {
  // handle error
}

import { InitializationResult } from "shenai-sdk";
 
// Initialize the SDK and handle result in callback
shenaiSDK.initialize(API_KEY, USER_ID, {}, (initResult) => {
  switch (initResult) {
    case InitializationResult.OK:
      // initialization succeeded
      break;
    case InitializationResult.INVALID_API_KEY:
      // handle invalid API key
      break;
    case InitializationResult.CONNECTION_ERROR:
      // handle connection error
      break;
    case InitializationResult.INTERNAL_ERROR:
      // handle internal error
      break;
  }
});

If the SDK has been successfully initialized, repeated calls to initialize will immediately return success - to start a new session first deinitialize the SDK.

Custom initialization options

The SDK can be configured at the initialization stage to determine it’s behavior and visual appearance. The options can also be changed at runtime.

Calibration Integration

To use calibration modes (CALIBRATION or CALIBRATED_MEASUREMENT), be sure to set initializationMode. See the Calibration page for more details.

Initialization Settings

If you’d like to change these initialization settings from the Shen.AI Portal instead of app code, see Remote Configuration.

enum Event {
  startButtonClicked,
  stopButtonClicked,
  measurementFinished,
  userFlowFinished,
}
 
typedef EventCallback = void Function(Event event);
 
class InitializationSettings {
InitializationMode initializationMode;
PrecisionMode precisionMode;
OperatingMode operatingMode;
MeasurementPreset measurementPreset;
CameraMode cameraMode;
OnboardingMode onboardingMode;
bool? showUserInterface;
bool? showFacePositioningOverlay;
bool? showVisualWarnings;
bool? enableCameraSwap;
bool? showFaceMask;
bool? showBloodFlow;
bool? hideShenaiLogo;
bool? enableStartAfterSuccess;
bool? enableSummaryScreen;
bool? enableHealthRisks;
bool? showOutOfRangeResultIndicators;
bool? showSignalQualityIndicator;
bool? showSignalTile;
bool? showTrialMetricLabels;
EventCallback? eventCallback;
}

export type EventName =
  | "START_BUTTON_CLICKED"
  | "STOP_BUTTON_CLICKED"
  | "MEASUREMENT_FINISHED"
  | "USER_FLOW_FINISHED";
 
interface InitializationSettings {
  initializationMode?: InitializationMode;
  precisionMode?: PrecisionMode;
  operatingMode?: OperatingMode;
  measurementPreset?: MeasurementPreset;
  cameraMode?: CameraMode;
  onboardingMode?: OnboardingMode;
  showUserInterface?: boolean;
  showFacePositioningOverlay?: boolean;
  showVisualWarnings?: boolean;
  enableCameraSwap?: boolean;
  showFaceMask?: boolean;
  showBloodFlow?: boolean;
  hideShenaiLogo?: boolean;
  enableStartAfterSuccess?: boolean;
  enableSummaryScreen?: boolean;
  enableHealthRisks?: boolean;
  showOutOfRangeResultIndicators?: boolean;
  showTrialMetricLabels?: boolean;
  showSignalQualityIndicator?: boolean;
  showSignalTile?: boolean;
}

public enum Event {
  START_BUTTON_CLICKED,
  STOP_BUTTON_CLICKED,
  MEASUREMENT_FINISHED,
  USER_FLOW_FINISHED
}
 
public interface EventCallback {
  void onEvent(Event event);
}
 
public class InitializationSettings {
  public InitializationMode initializationMode;
  public PrecisionMode precisionMode;
  public OperatingMode operatingMode;
  public MeasurementPreset measurementPreset;
  public CameraMode cameraMode;
  public OnboardingMode onboardingMode;
  public boolean showUserInterface;
  public boolean showFacePositioningOverlay;
  public boolean showVisualWarnings;
  public boolean enableCameraSwap;
  public boolean showFaceMask;
  public boolean showBloodFlow;
  public boolean hideShenaiLogo;
  public boolean enableStartAfterSuccess;
  public boolean enableSummaryScreen;
  public boolean enableHealthRisks;
  public boolean showOutOfRangeResultIndicators;
  public boolean showTrialMetricLabels;
  public boolean showSignalQualityIndicator;
  public boolean showSignalTile;
  public EventCallback eventCallback;
}

class InitializationSettings {
  var initializationMode InitializationMode;
  var precisionMode: PrecisionMode
  var operatingMode: OperatingMode
  var measurementPreset: MeasurementPreset
  var cameraMode: CameraMode
  var onboardingMode: OnboardingMode
  var showUserInterface: Bool
  var showFacePositioningOverlay: Bool
  var showVisualWarnings: Bool
  var enableCameraSwap: Bool
  var showFaceMask: Bool
  var showBloodFlow: Bool
  var hideShenaiLogo: Bool
  var enableStartAfterSuccess: Bool
  var enableSummaryScreen: Bool
  var enableHealthRisks: Bool
  var showOutOfRangeResultIndicators: Bool
  var showTrialMetricLabels: Bool
  var showSignalQualityIndicator: Bool
  var showSignalTile: Bool
}

export type EventName =
  | "START_BUTTON_CLICKED"
  | "STOP_BUTTON_CLICKED"
  | "MEASUREMENT_FINISHED"
  | "USER_FLOW_FINISHED";
 
interface InitializationSettings {
  initializationMode?: InitializationMode;
  precisionMode?: PrecisionMode;
  operatingMode?: OperatingMode;
  measurementPreset?: MeasurementPreset;
  cameraMode?: CameraMode;
  onboardingMode?: OnboardingMode;
  showUserInterface?: boolean;
  showFacePositioningOverlay?: boolean;
  showVisualWarnings?: boolean;
  enableCameraSwap?: boolean;
  showFaceMask?: boolean;
  showBloodFlow?: boolean;
  hideShenaiLogo?: boolean;
  enableStartAfterSuccess?: boolean;
  enableSummaryScreen?: boolean;
  enableHealthRisks?: boolean;
  showOutOfRangeResultIndicators?: boolean;
  showTrialMetricLabels?: boolean;
  showSignalQualityIndicator?: boolean;
  showSignalTile?: boolean;
  eventCallback?: (event: EventName) => void;
  onCameraError?: () => void;
}

See Configuration for detailed information on each option.

Providing the user interface

Unless you’re implementing a custom solution that doesn’t require a user interface, you’ll need to embed the SDK’s UI in your app. The SDK uses OpenGL/WebGL for rendering, embedded in a platform-specific native view component.

You can use the ShenaiView widget to embed the SDK’s UI in your Flutter app.

lib/main.dart

import 'package:shenai_sdk/shenai_view.dart';
...
 
@override
Widget build(BuildContext context) {
  return Center(
      child: ShenaiView(),
    );
}

You can use the ShenaiSdkView component to embed the SDK’s UI in your React Native app.

App.tsx

import {ShenaiSdkView} from 'react-native-shenai-sdk';
 
const App = () => {
  ...
  return (
    <View>
      <ShenaiSdkView />
    </View>
  );
};

You can use the ShenaiView class to embed the SDK’s UI in your iOS app.

Swift example:

AppDelegate.swift

import UIKit
import ShenaiSDK
 
@main
class AppDelegate: UIResponder, UIApplicationDelegate {
 
    func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {
        window = UIWindow(frame: UIScreen.main.bounds)
        let mainVC = ShenaiView()
        window?.rootViewController = mainVC
        window?.makeKeyAndVisible()
 
        return true
    }
}

Objective-C example:

AppDelegate.m

#import "AppDelegate.h"
#import <ShenaiSDK/ShenaiView.h>
 
@interface AppDelegate ()
 
@end
 
@implementation AppDelegate
 
- (BOOL)application:(UIApplication _)application didFinishLaunchingWithOptions:(NSDictionary _)launchOptions {
  self.window = [[UIWindow alloc] initWithFrame:[[UIScreen mainScreen] bounds]];
  ShenaiView *vc = [[ShenaiView alloc] init];
  self.window.rootViewController = vc;
  [self.window makeKeyAndVisible];
 
  return YES;
}
 
@end

You can use the ShenAIView class to embed the SDK’s UI in your Android app. To construct the view, you’ll need to provide the current Activity as a parameter.

src/main/java/MainActivity.java

import ai.mxlabs.shenai_sdk.ShenAIView;
 
 
public class MainActivity extends ComponentActivity {
  ...
  ShenAIView shenaiView = new ShenAIView(this);
  setContentView(shenaiView);
  ...
}

To display the SDK output, you’ll need to provide a canvas element.

If you create a canvas with id="mxcanvas", the SDK will automatically connect to it on initialization and use it for rendering.

<canvas id="mxcanvas" />

Alternatively, you can connect the canvas manually using the attachToCanvas method. This can be done either before or after the initialization, or even during runtime. However, keep in mind that switching canvases at runtime might cause UI to be not responding for some time.

<canvas id="not-mxcanvas" />
...
shenaiSDK.attachToCanvas("#not-mxcanvas");

In such a case, it’s a good idea to also specify the same custom canvas for the preload display (which gives the instructions before the SDK is loaded and initialized). This should be done by editing shenai-sdk/index.mjs file:

createPreloadDisplay("not-mxcanvas");

It is also possible to attach to multiple canvases. For this, you can use the optional second parameter exclusive of the attachToCanvas method. If exclusive is false it allows to attach to another canvas without detaching from the current one.

<canvas id="canvas1" />
<canvas id="canvas2" />
...
shenaiSDK.attachToCanvas("#canvas1", true);
shenaiSDK.attachToCanvas("#canvas2", false);

You should use CSS to provide size boundaries for the canvas. The SDK will automatically expand the canvas to fit the available space. Don’t set directly the width and height attributes of the canvas as those will be set by the SDK itself to properly size the rendering buffer.

You should include the following <meta> viewport element in HTML of your website to ensure that canvas has the proper resolution on mobile devices:

<meta name="viewport" content="width=device-width,initial-scale=1.0">

We encourage to disable zooming as well by using the following <meta> viewport element instead:

<meta name="viewport" content="width=device-width,initial-scale=1.0,maximum-scale=1.0,user-scalable=0"/>

Deinitialization

After finishing using the SDK you should deinitialize it - that will free up any allocated resources and disconnect from the camera.

ShenaiSdk.deinitialize();

import { deinitialize } from "react-native-shenai-sdk";
 
deinitialize();

ShenaiSDK.deinitialize()

shenaiSDKHandler.deinitialize()

shenaiSDK.deinitialize();

Deinitializing the Web SDK will keep the runtime alive, allowing you to intialize the SDK again afterwards.

To release all memory held by the SDK in the current browser context, you should call the destroyRuntime method - note that you won’t be able to initialize the SDK again after that. You don’t need to call deinitialize before destroyRuntime.

shenaiSDK.destroyRuntime();

Note that in order for all resources to be released, you need to release all references you may have kept to the SDK runtime after creating it.

Installation and permissions Configuration