Skip to Content
Introduction

Shen.AI (CE) Documentation

Introduction:

Shen.AI (CE) is a medical-grade B2B platform that provides camera-based vital sign measurements. The software is designed to support healthcare and digital health applications with a strong focus on patient safety, reliability, and regulatory compliance. Shen.AI (CE) is intended for the European market and is classified as a Software as a Medical Device (SaMD), Risk Class IIa, in accordance with Rule 11 of Regulation (EU) 2017/745 on Medical Devices (MDR).

How does it work?

Shen.AI (CE) is a Software as a Medical Device (SaMD) intended for non-invasive estimation of physiological parameters from video images acquired using a standard RGB camera. The software analyses optical remote photoplethysmography (rPPG) and remote ballistocardiography (rBCG) signals reconstructed from facial video recordings.

Deep learning models are used to detect and track the face, identify facial landmarks, reconstruct physiological signals, and estimate vital signs. The reconstructed rPPG signal reflects temporal variations in light intensity caused by pulsatile blood flow within the facial microvasculature, while the rBCG signal captures subtle head movements resulting from cardiac mechanical activity.

Heart Rate (HR) and Heart Rate Variability (HRV) are estimated from pulse detection and beat-to-beat interval analysis.

Breathing Rate (BR) is estimated from respiratory modulation of the reconstructed rPPG and rBCG signals.

Blood Pressure (BP) is estimated cufflessly using neural network models that analyse multiple pulse waveform features, including waveform morphology, rise time, amplitude, temporal variability, and regional pulse arrival time differences.

All processing is performed locally on the user’s device or within the web browser.

What metrics are currently available?

Shen.AI (CE) currently exposes the following metrics, based on real-time on-device video analysis:

  • Heart Rate (real-time and aggregated)
  • Heart Rate Variability (SDNN)
  • Breathing Rate
  • Systolic Blood Pressure
  • Diastolic Blood Pressure

See Results for more details on the video-based metrics.

What platforms are supported?

The current version of the SDK supports mobile and web platforms:

  • Android (Java/Kotlin)
  • iOS (Swift/Objective-C)
  • Web (Desktop and Mobile browsers: JavaScript/TypeScript)

It also supports cross-platform frameworks for iOS and Android:

  • Flutter
  • React Native
  • Capacitor
  • MAUI

See system requirements for more details.

We are open to expanding our platform support based on customer demand — feel free to contact us  if you need support for a different environment.

What languages are supported?

The current version of the SDK supports the following languages:

  • English (en)
  • German | Deutsch (de)
  • Spanish | Español (es)
  • French | Français (fr)
  • Polish | Polski (pl)

How does it work?

The SDK connects to the camera of a mobile or desktop device and uses real-time Computer Vision to isolate a human face in the video stream. A 3D model of the face is constructed, tracked, and used to extract a high-quality, dense signal of blood pulsations using remote photoplethysmography (rPPG). Based on that dense signal, the SDK accurately determines the timings and shapes of all observed heartbeats and provides precise heart-related metrics. All computations happen locally on the end device or within the web browser.

What does it consist of?

The SDK consists of:

  • Compiled machine code (native shared library on Android/iOS, WebAssembly on Web) that contains high-performance real-time Computer Vision algorithms and neural networks.
  • An embedded UI guiding the user through the measurement process and presenting the results.
  • Platform or framework-specific components for easy integration and camera access.
  • A Shen.AI (CE) example app  for each supported platform.
Authorization