Introduction

Shen.AI SDK Documentation

Shen.AI SDK (Software Development Kit) by Shen.AI is a B2B platform that provides precise and easy-to-use camera-based diagnostics of vital signs and wellness.

Evaluating the SDK

You can try out the Shen.AI SDK on your own device or in a web browser using our publicly available demos. See https://demo.shen.ai/demo.

What metrics are currently available?

Our SDK currently exposes the following metrics, based on real-time on-device video analysis:

  • Heart Rate (real-time and aggregated)
  • Interbeat Intervals (real-time and aggregated)
  • Heart Rate Variability (HRV) - SDNN, lnRMSSD
  • Breathing Rate
  • Cardiac Stress
  • Parasympathetic Activity
  • Cardiac Workload
  • Blood Pressure (systolic and diastolic) - beta version
  • Age and BMI estimate

See Results for more details on the video-based metrics.

The SDK also computes Cardiovascular health risks based on provided risk factors.

What platforms are supported?

The current version of the SDK supports mobile and web platforms:

  • Android (Java/Kotlin)
  • iOS (Swift/Objective-C)
  • Web (Desktop and Mobile browsers: JavaScript/TypeScript)

It also supports cross-platform frameworks (for iOS/Android):

  • Flutter
  • React Native

See system requirements for more details.

We are open to expanding our platform support based on customer demand — feel free to contact us if you need support for a different environment.

What languages are supported?

The current version of the SDK supports the following languages:

  • Arabic | العربية (ar)
  • Czech | čeština (cs)
  • German | Deutsch (de)
  • English (en)
  • Spanish | español (es)
  • Filipino | filipino (fil)
  • French | français (fr)
  • Hebrew | עברית (he)
  • Hindi | हिन्दी (hi)
  • Italian | italiano (it)
  • Japanese | 日本語 (ja)
  • Korean | 한국어 (ko)
  • Lithuanian (Lithuania) | lietuvių (Lietuva) (lt)
  • Polish | polski (pl)
  • Portuguese | português (pt)
  • Romanian | română (ro)
  • Slovak | slovenčina (sk)
  • Thai | ไทย (th)
  • Simplified Chinese | 简体中文 (zh-Hans)
  • Traditional Chinese | 繁體中文 (zh-Hant)

If you need support for a language that’s not listed here, feel free to contact us. We are open to expanding our language support based on customer demand.

How does it work?

The SDK connects to the camera of a mobile or desktop device and uses real-time Computer Vision to isolate a human face in the video stream. A 3D model of the face is constructed, tracked, and used to extract a high-quality, dense signal of blood pulsations using remote photoplethysmography (rPPG). Based on that dense signal, the SDK accurately determines the timings and shapes of all observed heartbeats and provides precise heart-related metrics. All computations happen locally on the end device or within the web browser.

What does it consist of?

The SDK consists of:

  • Compiled machine code (native shared library on Android/iOS, WebAssembly on Web) that contains high-performance real-time Computer Vision algorithms and neural networks.
  • An embedded UI guiding the user through the measurement process and presenting the results.
  • Platform or framework-specific components for easy integration and camera access.
  • Example app code for each supported platform.
Getting started
;