ESP8266Audio - supports ESP8266 & ESP32 & Raspberry Pi Pico RP2040 and Pico 2 RP2350

Arduino library for parsing and decoding MOD, WAV, MP3, FLAC, MIDI, AAC, and RTTL files and playing them on an I2S DAC or even using a software-simulated delta-sigma DAC with dynamic 32x-128x oversampling.

For real-time, autonomous speech synthesis, check out ESP8266SAM, a library which uses this one and a port of an ancient formant-based synthesis program to allow your ESP8266 to talk with low memory and no network required.

ESP32, Raspberry Pi Pico (RP2040 and RP2350) Users

Consider using BackgroundAudio instead of this library as it can provide a simpler usage model and better results and performance on the Pico by using an interrupt-based, frame-aligned output model.

Disclaimer

All this code is released under the GPL, and all of it is to be used at your own risk. If you find any bugs, please let me know via the GitHub issue tracker or drop me an email.

• The MOD and MP3 routines were taken from StellarPlayer and libMAD respectively.
• The software I2S delta-sigma 32x oversampling DAC was my own creation, and sounds quite good if I do say so myself.
• The AAC decode code is from the Helix project and licensed under RealNetwork's RSPL license. For commercial use you're still going to need the usual AAC licensing from Via Licensing. On the ESP32, AAC-SBR is supported (many webradio stations use this to reduce bandwidth even further). The ESP8266, however, does not support it due to a lack of onboard RAM.
• MIDI decoding comes from a highly ported MIDITONES combined with a massively memory-optimized TinySoundFont, see the respective source files for more information.
• Opus is from Xiph.org with the Xiph license and patent described in src/{opusfile,libggg,libopus}/COPYING.

Neat Things People Have Done With ESP8266Audio

If you have a neat use for this library, I'd love to hear about it!

• My personal use of the ESP8266Audio library is only to drive a 3D-printed, network-time-setting alarm clock for my kids which can play an MP3 instead of a bell to wake them up, called Psychoclock.
• Harald Sattler has built a neat German word clock with MP3 alarm. Detailed discussion on the process and models are included.
• Erich Heinemann has developed a Stomper (instrument for playing samples in real-time during a live stage performance) that you can find more info about here.
• Dagnall53 has integrated this into a really neat MQTT based model train controller to add sounds to his set. More info is available here, including STL files for 3D printed components!
• JohannesMTC has built a similar project especially for model trains: https://github.com/JohannesMTC/ESP32_MAS
• A neat MQTT-driven ESP8266 light-and-sound device (alarm? toy? who can say!) was built by @CosmicMac, available at https://github.com/CosmicMac/ESParkle
• A very interesting "linear clock" with a stepper motor, NTP time keeping, and configurable recorded chimes with schematics, 3D printer plans, and source code, is now available https://janderogee.com/projects/linear_clock/linear_clock.htm
• Source and instructions for a gorgeous wooden MP3-playing clock, FM radio and a walkie-talkie using the ESP8266 and AVR microcontrollers is available https://github.com/zduka/mp3-player

Prerequisites

First, make sure you are running the 2.6.3/later or GIT head version of the Arduino libraries for ESP8266, or the latest ESP32 SDK from Espressif.

You can use GIT to pull right from GitHub: see this README for detailed instructions.

Installation

Install the library in your ~/Arduino/libraries

mkdir -p ~/Arduino/libraries
cd ~/Arduino/libraries
git clone https://github.com/earlephilhower/ESP8266Audio

When in the IDE please select the following options on the ESP8266:

Tools->lwIP Variant->v1.4 Open Source, or V2 Higher Bandwidth
Tools->CPU Frequency->160MHz

Usage

Create an AudioInputXXX source pointing to your input file, an AudioOutputXXX sink as either an I2S, I2S-sw-DAC, or as a "SerialWAV" which simply writes a WAV file to the Serial port which can be dumped to a file on your development system, and an AudioGeneratorXXX to actually take that input and decode it and send to the output.

After creation, you need to call the AudioGeneratorXXX::loop() routine from inside your own main loop() one or more times. This will automatically read as much of the file as needed and fill up the I2S buffers and immediately return. Since this is not interrupt driven, if you have large delay()s in your code, you may end up with hiccups in playback. Either break large delays into very small ones with calls to AudioGenerator::loop(), or reduce the sampling rate to require fewer samples per second.

Example

See the examples directory for some simple examples, but the following snippet can play an MP3 file over the simulated I2S DAC:

#include <Arduino.h>
#include "AudioFileSourceSPIFFS.h"
#include "AudioGeneratorMP3.h"
#include "AudioOutputI2SNoDAC.h"

AudioGeneratorMP3 *mp3;
AudioFileSourceSPIFFS *file;
AudioOutputI2SNoDAC *out;
void setup()
{
  Serial.begin(115200);
  delay(1000);
  SPIFFS.begin();
  file = new AudioFileSourceSPIFFS("/jamonit.mp3");
  out = new AudioOutputI2SNoDAC();
  mp3 = new AudioGeneratorMP3();
  mp3->begin(file, out);
}

void loop()
{
  if (mp3->isRunning()) {
    if (!mp3->loop()) mp3->stop(); 
  } else {
    Serial.printf("MP3 done\n");
    delay(1000);
  }
}

AudioFileSource classes

AudioFileSource: Base class which implements a very simple read-only "file" interface. Required because it seems everyone has invented their own filesystem on the Arduino with their own unique twist. Using this wrapper lets that be abstracted and makes the AudioGenerator simpler as it only calls these simple functions.

AudioFileSourceSPIFFS: Reads a file from the SPIFFS filesystem

AudioFileSourcePROGMEM: Reads a file from a PROGMEM array. Under UNIX you can use "xxd -i file.mp3 > file.h" to get the basic format, then add "const" and "PROGMEM" to the generated array and include it in your sketch. See the example .h files for a concrete example.

AudioFileSourceHTTPStream: Simple implementation of a streaming HTTP reader for ShoutCast-type MP3 streaming. Not yet resilient, and at 44.1khz 128bit stutters due to CPU limitations, but it works more or less.

AudioFileSourceBuffer - Double buffering, useful for HTTP streams

AudioFileSourceBuffer is an input source that simply adds an additional RAM buffer of the output of any other AudioFileSource. This is particularly useful for web streaming where you need to have 1-2 packets in memory to ensure hiccup-free playback.

Create your standard input file source, create the buffer with the original source as its input, and pass this buffer object to the generator.

...
AudioGeneratorMP3 *mp3;
AudioFileSourceHTTPStream *file;
AudioFileSourceBuffer *buff;
AudioOutputI2SNoDAC *out;
...
  // Create the HTTP stream normally
  file = new AudioFileSourceHTTPStream("http://your.url.here/mp3");
  // Create a buffer using that stream
  buff = new AudioFileSourceBuffer(file, 2048);
  out = new AudioOutputI2SNoDAC();
  mp3 = new AudioGeneratorMP3();
  // Pass in the *buffer*, not the *http stream* to enable buffering
  mp3->begin(buff, out);
...

AudioFileSourceID3 - ID3 stream parser filter with a user-specified callback

This class, which takes as input any other AudioFileSource and outputs an AudioFileSource suitable for any decoder, automatically parses out ID3 tags from MP3 files. You need to specify a callback function, which will be called as tags are decoded and allow you to update your UI state with this information. See the PlayMP3FromSPIFFS example for more information.

AudioGenerator classes

AudioGenerator: Base class for all file decoders. Takes a AudioFileSource and an AudioOutput object to get the data from and to write decoded samples to. Call its loop() function as often as you can to ensure the buffers are always kept full and your music won't skip.

AudioGeneratorWAV: Reads and plays Microsoft WAVE (.WAV) format files of 8 or 16 bits.

AudioGeneratorMOD: Reads and plays Amiga ModTracker files (.MOD). Use a 160MHz clock as this requires tons of SPIFFS reads (which are painfully slow) to get raw instrument sample data for every output sample. See https://modarchive.org for many free MOD files.

AudioGeneratorMP3: Reads and plays MP3 format files (.MP3) using a ported libMAD library. Use a 160MHz clock to ensure enough compute power to decode 128KBit 44.1KHz without hiccups. For complete porting history with the gory details, look at https://github.com/earlephilhower/libmad-8266

AudioGeneratorFLAC: Plays FLAC files via ported libflac-1.3.2. On the order of 30KB heap and minimal stack required as-is.

AudioGeneratorMIDI: Plays a MIDI file using a wavetable synthesizer and a SoundFont2 wavetable input. Theoretically up to 16 simultaneous notes available, but depending on the memory needed for the SF2 structures you may not be able to get that many before hitting OOM.

AudioGeneratorAAC: Requires about 30KB of heap and plays a mono or stereo AAC file using the Helix fixed-point AAC decoder.

AudioGeneratorRTTTL: Enjoy the pleasures of monophonic, 4-octave ringtones on your ESP8266. Very low memory and CPU requirements for simple tunes.

AudioOutput classes

AudioOutput: Base class for all output drivers. Takes a sample at a time and returns true/false if there is buffer space for it. If it returns false, it is the calling object's (AudioGenerator's) job to keep the data that didn't fit and try again later.

AudioOutputI2S: Interface for any I2S 16-bit DAC. Sends stereo or mono signals out at whatever frequency set. Tested with Adafruit's I2SDAC and a Beyond9032 DAC from eBay. Tested up to 44.1KHz. To use the internal DAC on ESP32, instantiate this class as AudioOutputI2S(0,AudioOutputI2S::INTERNAL_DAC), see example PlayMODFromPROGMEMToDAC and code in AudioOutputI2S.cpp for details. To use the hardware Pulse Density Modulation (PDM) on ESP32, instantiate this class as AudioOutputI2S(0,AudioOutputI2S::INTERNAL_PDM). For both later cases, default output pins are GPIO25 and GPIO26.

AudioOutputI2SNoDAC: Abuses the I2S interface to play music without a DAC. Turns it into a 32x (or higher) oversampling delta-sigma DAC. Use the schematic below to drive a speaker or headphone from the I2STx pin (i.e. Rx). Note that with this interface, depending on the transistor used, you may need to disconnect the Rx pin from the driver to perform serial uploads. Mono-only output, of course.

AudioOutputSPDIF (experimental): Another way to abuse the I2S peripheral to send out BMC encoded S/PDIF bitstream. To interface with S/PDIF receiver it needs optical or coaxial transceiver, for which some examples can be found at https://www.epanorama.net/documents/audio/spdif.html. It should work even with the simplest form with red LED and current limiting resistor, fed into TOSLINK cable. Minimum sample rate supported by is 32KHz. Due to BMC coding, actual symbol rate on the pin is 4x normal I2S data rate, which drains DMA buffers quickly. See more details inside AudioOutputSPDIF.cpp

AudioOutputSerialWAV: Writes a binary WAV format with headers to the Serial port. If you capture the serial output to a file you can play it back on your development system.

AudioOutputSPIFFSWAV: Writes a binary WAV format with headers to a SPIFFS filesystem. Ensure the FS is mounted and SPIFFS is started before calling. USe the SetFilename() call to pick the output file before starting.

AudioOutputNull: Just dumps samples to /dev/null. Used for speed testing as it doesn't artificially limit the AudioGenerator output speed since there are no buffers to fill/drain.

I2S DACs

I've used both the Adafruit I2S +3W amp DAC and a generic PCM5102 based DAC with success. The biggest problems I've seen from users involve pinouts from the ESP8266 for GPIO and hooking up all necessary pins on the DAC board. The essential pins are:

I2S pin	Common label*	ESP8266 pin
LRC	D4	GPIO2
BCLK	D8	GPIO15
DIN	RX	GPIO3

* The "common label" column applies to common NodeMCU and D1 Mini development boards. Unfortunately some manufacturers use different mappings so the labels listed here might not apply to your particular model.

Adafruit I2S DAC

This is quite simple and only needs the GND, VIN, LRC, BCLK< and DIN pins to be wired. Be sure to use +5V on the VIN to get the loudest sound. See the Adafruit example page for more info.

PCM5102 DAC

I've used several versions of PCM5102 DAC boards purchased from eBay. They've all had the same pinout, no matter the form factor. There are several input configuration pins beyond the I2S interface itself that need to be wired: * 3.3V from ESP8266 -> VCC, 33V, XMT * GND from ESP8266 -> GND, FLT, DMP, FMT, SCL * (Standard I2S interface) BCLK->BCK, I2SO->