Introduction

Shen.AI SDK Documentation

Shen.AI SDK (Software Development Kit) by Shen.AI is a B2B platform that provides precise and easy-to-use camera-based diagnostics of vital signs and wellness.

Evaluating the SDK

You can try out the Shen.AI SDK on your own device or in a web browser using our publicly available demos. See https://demo.shen.ai/demo.

What metrics are currently available?

Our SDK currently exposes the following metrics, based on real-time on-device video analysis:

  • Heart Rate (real-time and aggregated)
  • Interbeat Intervals (real-time and aggregated)
  • Heart Rate Variability (HRV) - SDNN, lnRMSSD
  • Breathing Rate
  • Cardiac Stress
  • Parasympathetic Activity
  • Cardiac Workload
  • Blood Pressure (systolic and diastolic) - beta version
  • Age and BMI estimate

See Results for more details on the video-based metrics.

The SDK also computes Cardiovascular health risks based on provided risk factors.

What platforms are supported?

The current version of the SDK supports mobile and web platforms:

  • Android (Java/Kotlin)
  • iOS (Swift/Objective-C)
  • Web (Desktop and Mobile browsers: JavaScript/TypeScript)

It also supports cross-platform frameworks (for iOS/Android):

  • Flutter
  • React Native

See system requirements for more details.

We are open to expanding our platform support based on customer demand — feel free to contact us if you need support for a different environment.

How does it work?

The SDK connects to the camera of a mobile or desktop device and uses real-time Computer Vision to isolate a human face in the video stream. A 3D model of the face is constructed, tracked, and used to extract a high-quality, dense signal of blood pulsations using remote photoplethysmography (rPPG). Based on that dense signal, the SDK accurately determines the timings and shapes of all observed heartbeats and provides precise heart-related metrics. All computations happen locally on the end device or within the web browser.

What does it consist of?

The SDK consists of:

  • Compiled machine code (native shared library on Android/iOS, WebAssembly on Web) that contains high-performance real-time Computer Vision algorithms and neural networks.
  • An embedded UI guiding the user through the measurement process and presenting the results.
  • Platform or framework-specific components for easy integration and camera access.
  • Example app code for each supported platform.
Getting started
;