🦻 OpenEarable - Firmware v1.4.0

OpenEarable is a new, open-source, Arduino-based platform for ear-based sensing applications. It provides a versatile prototyping platform with support for various sensors and actuators, making it suitable for earable research and development.

This firmware is compatible with hardware version: - 1.3.0 (9-axis IMU, pressure sensor, speaker, in-ear ultrasound microphone) - 1.4.0 (adds dual ultrasound microphones support: one in-ear and one out-of-ear)

⚠️ $${\rm\color{red}Please~Note:}$$ We recommend deploying the firmware using the open-earable-PlatformIO wrapper.

 

Get OpenEarable device now! 

Table of Contents

• Introduction
• Setup
• Arduino IDE
• Arduino Libraries
• SD Card Setup
• SPI Setup
• sdfat Library Setup
• BMP280 Library Setup
• Usage
• Install OpenEarable
• Default Firmware
• Dashboard
• edge-ml
• BLE Specification
• Firmware Internals
• Bootloader
• Pin Configuration
• Earable
• Sensor Configuration
• Button
• LED
• Cite

Introduction

OpenEarable is designed to enable ear-based sensing applications by offering a flexible and open-source hardware platform. It incorporates a range of sensors, including a 9-axis inertial measurement unit, an ear canal pressure and temperature sensor, an inward-facing ultrasound microphone, a speaker, a push button, and a controllable RGB LED. With these features, OpenEarable provides researchers and developers with the ability to explore various application scenarios. For more information visit the OpenEarable website.

OpenEarable is controlled and streams sensor data via BLE (Bluetooth Low Energy). Audio is played from and recorded to the internal SD card (required card SanDisk Extreme Class 3, must be formatted as exFAT). OpenEarable is compatible with the provided dashboard and edge-ml.

Setup

⚠️ $${\rm\color{red}Please~Note:}$$ We recommend deploying the firmware using the open-earable-PlatformIO wrapper.

Arduino IDE

Download and install the Arduino IDE. OpenEarable is based on the "Arduino Nano 33 BLE Sense" board. Therefore, you first have to install the required dependencies ("Arduino Mbed OS Nano Boards" via boards manager) in your Arduino IDE following this Setup Guide.

Arduino Libraries

The following Arduino Libraries have to be installed in your Arduino IDE by navigating to Sketch -> Include Library -> Manage Libraries: - EdgeML-Arduino (version 1.3.3), which includes the following dependencies that are also required and automatically installed: - ArduinoBLE - Adafruit_BMP280 - DFRobot_BMX160 - SdFat - Adafruit Fork

SD Card Setup

In order to be compatible with the OpenEarable library the SD card needs to be formatted with the exFAT format. Make sure to have a sufficiently fast SD card. (Required SD Card: SandDisk class 10 and class A30)

SPI, Wire, and Variant Setup

The default Arduino implementation of the SPI library does not meet the required speed. To address this, optimized SPI files are provided. Follow the steps below to integrate these files into Arduino. All referenced files can be found in the "resources" folder in the "spi_files" subfolder.

To fully integrate the optimized SPI files, changes to the Arduino Nano 33 BLE board files have to be made. Follow the steps below to accomplish this:

1. Navigate to the Arduino15 folder as described in this tutorial.

2. Navigate to the following directory: packages/arduino/hardware/mbed_nano/4.0.4/libraries. Please note: If you have a different version than mbed_nano 4.0.4 installed or you don't have it installed at all, you can change or add it by navigating to Tools -> Board -> Boards Manager. Search for Arduino Mbed OS Nano Boards and install the desired version.

3. Locate the existing SPI library folder within this directory. It needs to be swapped with the provided folder named "SPI" in "resources/spi_files" found in this repository.

4. Locate the existing Wire library folder within the libraries folder. It needs to be swapped with the provided folder named "Wire" in "resources/wire_files" found in this repository (make sure to delete the Wire.cpp file).

5. Navigate to the following directory in the Arduino15directory: packages/arduino/hardware/mbed_nano/4.0.4/cores/arduino.

6. Place the files RingBuffer.h and RingBuffer.cpp from "resources/wire_files" into this folder.

7. Similarly, navigate to the following directory: packages/arduino/hardware/mbed_nano/4.0.4/cores/arduino/mbed/targets/TARGET_NORDIC/TARGET_NRF5x/TARGET_SDK_15_0/modules/nrfx/drivers.

8. In the subdirectory include, locate the file named nrfx_spim.h and replace it with the provided nrfx_spim.h file from the resources/spi_files folder of this repository.

9. In the subdirectory src, place the nrfx_spim.c file provided under resources/spi_files of this repository (note: you will not find an existing file named nrfx_spim.c).

10. Navigate back to the Arduino15 folder. Navigate to packages/arduino/hardware/mbed_nano/4.0.4/variants/ARDUINO_NANO33BLE. Replace pins_arduino.h and variant.cppwith the files provided under resources/variant of this repository.

sdFat Library Setup

One of the library dependencies is the SdFat library from Bill Greiman. This library is used to send data to the SD card. To achieve the desired write speeds of up to 1.5Mbps the library has to be modified slighlty.

1. Go to the Arduino/libraries folder, locate the SdFat_-_Adafruit_Fork library folder and open it. (Note: typically, the Arduino libraries folder is located at ~/Documents/Arduino/libraries)

2. Inside the src folder, replace the SdFatConfig.h with the provided SdFatConfig.h file found in the resources/sdfat_config folder of this repository.

BMP280 Library Setup

The BMP280 library has to be slightly modified. 1. Go to the Arduino/libraries folder (commonly found in your Documents folder) and locate the Adafruit_BMP280_Library folder. 2. Replace the files Adafruit_BMP280.cpp and Adafruit_BMP280.h with the files found in the resources/Adafruit_BMP280_Library folder of this repository.

Reboot

To make sure that all your changes are applied correctly, restart your computer.

Usage

Install OpenEarable

Now that all dependencies are configured, the last step is to install this repository as a library as follows:

1. In your Arduino IDE click on Sketch -> Include Library -> Add .ZIP Library....
2. Navigate to the location where you saved the downloaded zip file.
3. Select the zip file and click Open.

Once done, the library should be available in the Sketch -> Include Library menu.

Default Firmware

The easiest way to use OpenEarable is with the provided App sketch. It can be found within Arduino under File -> Examples-> OpenEarable -> App.

The absolute minimum needed to run the code successfully is the following:

#include "Arduino.h"
#include "OpenEarable.h"

void setup()
{
    open_earable.begin();
}

void loop()
{
    open_earable.update();
}

With this minimum sketch, all internal functionality is activated and OpenEarable becomes controllable via our Dashboard, via EdgeML, and via the BLE API.

Flashing

To flash the firmware, make sure you select Arduino Nano 33 BLE as target and the port that your OpenEarable is connected to. Then simply press the Upload arrow.

Dashboard

OpenEarable can be directly connected to our Dashboard.

edge-ml

OpenEarable is seamlessly integrated with edge-ml.org, an open-source and browser-based toolchain for machine learning on microcontrollers.

BLE Specification

The following table contains the BLE specifications with the available Services and Characteristics as well as UUIDs. The first 3 Services are from the EdgeML-Arduino library.

Note: to configure sensors via BLE see Sensor Configuration.

Specification Table:

Service Name	Service UUID	Characteristic Name	Characteristic UUID	Permissions
Sensor Service	34c2e3bb-34aa-11eb-adc1-0242ac120002	Sensor Configuration	34c2e3bd-34aa-11eb-adc1-0242ac120002	Write
		Sensor Data	34c2e3bc-34aa-11eb-adc1-0242ac120002	Read/Notify
Device Info Service	45622510-6468-465a-b141-0b9b0f96b468	Device Identifier	45622511-6468-465a-b141-0b9b0f96b468	Read
		Device Generation	45622512-6468-465a-b141-0b9b0f96b468	Read
Parse Info Service	caa25cb7-7e1b-44f2-adc9-e8c06c9ced43	Scheme	caa25cb8-7e1b-44f2-adc9-e8c06c9ced43	Read
Audio Service	5669146e-476d-11ee-be56-0242ac120002	Source	566916a8-476d-11ee-be56-0242ac120002	Read/Notify/Write
		State	566916a9-476d-11ee-be56-0242ac120002	Read/Notify/Write
Battery Service	180F	Battery Level	2A19	Read/Notify
Button Service	29c10bdc-4773-11ee-be56-0242ac120002	Button State	29c10f38-4773-11ee-be56-0242ac120002	Read/Notify
LED Service	81040a2e-4819-11ee-be56-0242ac120002	LED Set State	81040e7a-4819-11ee-be56-0242ac120002	Write

Sensor Configuration Characteristic

Permissions: Write

This characteristic is used to send a sensor configuration to the Earable.

Configuration Package:

Byte 0	Byte 1-4	Byte 5-8
SensorID	Sample Rate	Latency
uint8	float	uint32

SensorID: ID of the sensor.

Sample Rate: Desired sample rate.

Latency: Legacy field. Repurposed to control microphone gain as shown later.

Each sensor or audio IO can be enabled individually or together at the same time with predefined configurations. It is recommended to use the predefined configurations.

To use the redefined configurations, Please refer to the implementation details below.

Sensor Data Characteristic

Permissions: Read/Notify

This Characteristic is responsible for sending data packages from the Earable to the connected device.

Data Package:

Byte 0	Byte 1	Byte 1-4	Byte 5-X
SensorID	Size	Time Stamp	Data Array
uint8	uint8	uint32	---

SensorID: ID of the sensor.

Size: Length of the data array.

Time Stamp: Timestamp in milliseconds.

Data Array: Array of bytes, which need to be parsed according the sensors parsing scheme.

Device Identifier Characteristic

Permissions: Read

This characteristic is used to get the Device Identifier string.

Device Generation Characteristic

Permissions: Read

This characteristic is used to get the Device Generation string.

Scheme Characteristic

Permissions: Read

With this characteristic the parsing scheme information can be requested from the device. The parsing scheme is needed to convert a received data package to usable values. More information about parsing the scheme can be found in the EdgeML-Arduino library.

Audio Play Service

The Audio Player service consists of two characteristics. One to set the audio source and one two controll the playing state of the player. To properly set a new source.

Source Characteristic

Permissions: Read/Write

This characteristic is used to set the source of the audio player. Writing to this characteristic will result in stopping the player and playing from the new source.

The first byte of the characteristic indicates the source type being: - 0: Idle - 1: Mono WAV File from SD Card - 2: Single Tone between 300-22000 Hz - 3: Jingle (chosen from a list of predefined sounds)

The meaning of the remaining remaining bytes depends on the source type and is given for each type below.

Play Package:

Idle

Byte 0	Byte 1	Byte 2-X
0	-	-
uint8	uint8	char array

WAV File

| Byte 0 | Byte 1 | Byte 2-X