Audio Hacking On The ESP8266

Wemos Audio

The esp8266 is quite powerful for audio applications with a CPU frequency of 160MHz and 4MB flash and easily outperforms the Volca Sample with a WiFi/Web GUI.

The goal for this blog series is to build synthesizers on the esp8266 platform so we will also go through adding MIDI, Sync24 and trigger inputs.

And use the WiFi for easy uploading and editing of samples.

But the PDM DAC also works for webradio and other audio streaming applications.

We will use the Wemos D1 Mini board with the Arduino IDE and all the source will be available.

WeMos D1 mini

The series starts off with creating the 44.1KHz, 16-bit PDM Audio DAC.

My work on these free synthesizers is based on donations from people.
If you find the code useful, please consider a $3 donation to keep future developments open source.

We will start by adding an audio output.

esp8266 i2s interface

The esp8266 handles audio through something called i2s.
i2s is high speed shifting out of 2 16-bit serial words, left and right channel, and a shift clock powered by DMA.

i2s

This interface normally requires an external i2s DAC that converts the serial stream to analog signals.

To make it more easy we are going to build a PDM (Pulse Density Modulation) DAC based on the i2s interface.

PDM is a high rate bitstream and at 44.1KHz sample rate it will be 32 times higher or about 1.4MHz.

PDM

Pulse Density Modulation being a 1-bit DAC gives us a dynamic range of 6dB.
That will generate ALOT of noise or 90dB to be exact.

PDM Spectrum

The good thing is that the noise is in a frequency range far above the audio spectrum and can easily be filtered off with a lowpass filter leaving us just the audio signal.

So delta-sigma coding our 16-bit sample words to PDM will give us one 16-bit DAC output with only an external passive filter.

This is the schematics for the audio output:

PDM LPF

But why is it connected to the RX pin?
Isn’t that the serial input pin?

It’s also the i2s data output pin.

Lets show some code

This is the setup() for our first test.

It turns off the WiFi radio to reduce power to about 15mA and setting up the pins and DMA for the i2s subsystem at a 44100Hz sample rate:

#include <Arduino.h> 
#include "ESP8266WiFi.h"
#include <i2s.h>
#include <i2s_reg.h>

void setup() {
  //WiFi.forceSleepBegin();             
  //delay(1);                               
  system_update_cpu_freq(160);

  i2s_begin();
  i2s_set_rate(44100);
  
}

To test the DAC we generate a slow sine wave:

uint8_t phase;
  void loop() {
  writeDAC(0x8000+sine[phase++]);
}

And the sinewave data:

int16_t sine[256] = {
  0x0000, 0x0324, 0x0647, 0x096a, 0x0c8b, 0x0fab, 0x12c8, 0x15e2,
  0x18f8, 0x1c0b, 0x1f19, 0x2223, 0x2528, 0x2826, 0x2b1f, 0x2e11,
  0x30fb, 0x33de, 0x36ba, 0x398c, 0x3c56, 0x3f17, 0x41ce, 0x447a,
  0x471c, 0x49b4, 0x4c3f, 0x4ebf, 0x5133, 0x539b, 0x55f5, 0x5842,
  0x5a82, 0x5cb4, 0x5ed7, 0x60ec, 0x62f2, 0x64e8, 0x66cf, 0x68a6,
  0x6a6d, 0x6c24, 0x6dca, 0x6f5f, 0x70e2, 0x7255, 0x73b5, 0x7504,
  0x7641, 0x776c, 0x7884, 0x798a, 0x7a7d, 0x7b5d, 0x7c29, 0x7ce3,
  0x7d8a, 0x7e1d, 0x7e9d, 0x7f09, 0x7f62, 0x7fa7, 0x7fd8, 0x7ff6,
  0x7fff, 0x7ff6, 0x7fd8, 0x7fa7, 0x7f62, 0x7f09, 0x7e9d, 0x7e1d,
  0x7d8a, 0x7ce3, 0x7c29, 0x7b5d, 0x7a7d, 0x798a, 0x7884, 0x776c,
  0x7641, 0x7504, 0x73b5, 0x7255, 0x70e2, 0x6f5f, 0x6dca, 0x6c24,
  0x6a6d, 0x68a6, 0x66cf, 0x64e8, 0x62f2, 0x60ec, 0x5ed7, 0x5cb4,
  0x5a82, 0x5842, 0x55f5, 0x539b, 0x5133, 0x4ebf, 0x4c3f, 0x49b4,
  0x471c, 0x447a, 0x41ce, 0x3f17, 0x3c56, 0x398c, 0x36ba, 0x33de,
  0x30fb, 0x2e11, 0x2b1f, 0x2826, 0x2528, 0x2223, 0x1f19, 0x1c0b,
  0x18f8, 0x15e2, 0x12c8, 0x0fab, 0x0c8b, 0x096a, 0x0647, 0x0324,
  0x0000, 0xfcdc, 0xf9b9, 0xf696, 0xf375, 0xf055, 0xed38, 0xea1e,
  0xe708, 0xe3f5, 0xe0e7, 0xdddd, 0xdad8, 0xd7da, 0xd4e1, 0xd1ef,
  0xcf05, 0xcc22, 0xc946, 0xc674, 0xc3aa, 0xc0e9, 0xbe32, 0xbb86,
  0xb8e4, 0xb64c, 0xb3c1, 0xb141, 0xaecd, 0xac65, 0xaa0b, 0xa7be,
  0xa57e, 0xa34c, 0xa129, 0x9f14, 0x9d0e, 0x9b18, 0x9931, 0x975a,
  0x9593, 0x93dc, 0x9236, 0x90a1, 0x8f1e, 0x8dab, 0x8c4b, 0x8afc,
  0x89bf, 0x8894, 0x877c, 0x8676, 0x8583, 0x84a3, 0x83d7, 0x831d,
  0x8276, 0x81e3, 0x8163, 0x80f7, 0x809e, 0x8059, 0x8028, 0x800a,
  0x8000, 0x800a, 0x8028, 0x8059, 0x809e, 0x80f7, 0x8163, 0x81e3,
  0x8276, 0x831d, 0x83d7, 0x84a3, 0x8583, 0x8676, 0x877c, 0x8894,
  0x89bf, 0x8afc, 0x8c4b, 0x8dab, 0x8f1e, 0x90a1, 0x9236, 0x93dc,
  0x9593, 0x975a, 0x9931, 0x9b18, 0x9d0e, 0x9f14, 0xa129, 0xa34c,
  0xa57e, 0xa7be, 0xaa0b, 0xac65, 0xaecd, 0xb141, 0xb3c1, 0xb64c,
  0xb8e4, 0xbb86, 0xbe32, 0xc0e9, 0xc3aa, 0xc674, 0xc946, 0xcc22,
  0xcf05, 0xd1ef, 0xd4e1, 0xd7da, 0xdad8, 0xdddd, 0xe0e7, 0xe3f5,
  0xe708, 0xea1e, 0xed38, 0xf055, 0xf375, 0xf696, 0xf9b9, 0xfcdc
};

And the resulting waveform output , a sine wave at 172Hz:

waveform

Two important things to keep in mind:

The esp8266 is a RTOS system and other things happen in the background.
So don’t use delay() or other blocking functions.
Use yield() if something takes a long time.

The DMA buffer is 512 samples long and will exhaust in 11.5mS
To have uninterrupted audio output you need to feed it samples before it exhaust.

Feel free to try and get this running and I’ll be back with a sample player.

(* Someone claimed that feeding the PDM algorithm a flatline 0x0001 DAC value will make it fail with a 22Hz hum. While that is true it is an extreme case that seldom happens. Normal flatline is 0x8000 which produces a 50/50 squarewave at 700KHz and the occasional DAC values that dip into values below 1000 wont last long so its not a problem in real world wave data.)

(* The PDM bitrate runs at 1.4MHz. To make it run at the more professional bitrate of 3MHz just bump the sample rate up from 44.1KHz to 96KHz.)

A simple 909 drum synth

rolandtr909

Using our sampling knowledge we are going to do a simple 909 drum sample player.

The sample player is a 11-voice fully polyphonic 44.1KHz 16-bit 1-shot wave player.

For this we need about 300Kbyte worth of 44.1KHz drum samples.

const uint16_t BD16[3796] PROGMEM = {
40, 85, 137, 144, -30, -347, -609, -785, // 0-7

const uint16_t CP16[4445] PROGMEM = {
-42, 74, -1236, -2741, -3134, -11950, -13578, -7572, // 0-7

The definitions above are only a small sample of the 16-bit wave data. The full arrays of the 11 sounds are included in the downloadable sketch.

We also need some declares for the sample engine.

uint32_t BD16CNT;
uint32_t CP16CNT;
uint32_t CR16CNT;
uint32_t HH16CNT;
uint32_t HT16CNT;
uint32_t LT16CNT;
uint32_t MT16CNT;
uint32_t CH16CNT;
uint32_t OH16CNT;
uint32_t RD16CNT;
uint32_t RS16CNT;
uint32_t SD16CNT;

#define BD16LEN 3796UL
#define CP16LEN 4445UL
#define CR16LEN 48686UL
#define HH16LEN 1734UL
#define HT16LEN 5802UL
#define LT16LEN 7061UL
#define MT16LEN 7304UL
#define OH16LEN 4772UL
#define RD16LEN 52850UL
#define RS16LEN 1316UL
#define SD16LEN 5577UL

This defines the sample counters and their lenght.

To keep the sample engine running a function needs to be defined that calculates the drum sounds.


uint16_t SYNTH909() {
 int32_t DRUMTOTAL=0;
 if (BD16CNT<BD16LEN) DRUMTOTAL+=(pgm_read_word_near(BD16 + BD16CNT++)^32768)-32768;
 if (CP16CNT<CP16LEN) DRUMTOTAL+=(pgm_read_word_near(CP16 + CP16CNT++)^32768)-32768;
 if (CR16CNT<CR16LEN) DRUMTOTAL+=(pgm_read_word_near(CR16 + CR16CNT++)^32768)-32768;
 if (HH16CNT<HH16LEN) DRUMTOTAL+=(pgm_read_word_near(HH16 + HH16CNT++)^32768)-32768;
 if (HT16CNT<HT16LEN) DRUMTOTAL+=(pgm_read_word_near(HT16 + HT16CNT++)^32768)-32768;
 if (LT16CNT<LT16LEN) DRUMTOTAL+=(pgm_read_word_near(LT16 + LT16CNT++)^32768)-32768;
 if (MT16CNT<MT16LEN) DRUMTOTAL+=(pgm_read_word_near(MT16 + MT16CNT++)^32768)-32768;
 if (OH16CNT<OH16LEN) DRUMTOTAL+=(pgm_read_word_near(OH16 + OH16CNT++)^32768)-32768;
 if (RD16CNT<RD16LEN) DRUMTOTAL+=(pgm_read_word_near(RD16 + RD16CNT++)^32768)-32768;
 if (RS16CNT<RS16LEN) DRUMTOTAL+=(pgm_read_word_near(RS16 + RS16CNT++)^32768)-32768;
 if (SD16CNT<SD16LEN) DRUMTOTAL+=(pgm_read_word_near(SD16 + SD16CNT++)^32768)-32768; if (DRUMTOTAL>32767) DRUMTOTAL=32767;
 if (DRUMTOTAL<-32767) DRUMTOTAL=-32767;
 DRUMTOTAL+=32768;
 return DRUMTOTAL;
}

In the main loop we add a call to the sample engine.

void loop() {
DAC=SYNTH909();

//Pulse Density Modulated 16-bit I2S DAC
for (uint8_t i=0;i<32;i++) { 
      i2sACC=i2sACC<<1; if(DAC >= err) {
        i2sACC|=1;
        err += 0xFFFF-DAC;
      }
        else
      {
        err -= DAC;
      }
     }
     bool flag=i2s_write_sample(i2sACC);

}

And finally the MIDI drum trigger function.

void MidiNoteOn(uint8_t channel, uint8_t note, uint8_t velocity) {

/* 909 MIDI Triggers
Bass Drum MIDI-35
Bass Drum MIDI-36
Rim Shot MIDI-37
Snare Drum MIDI-38
Hand Clap MIDI-39
Snare Drum MIDI-40
Low Tom MIDI-41
Closed Hat MIDI-42
Low Tom MIDI-43
Closed Hat MIDI-44
Mid Tom MIDI-45
Open Hat MIDI-46
Mid Tom MIDI-47
Hi Tom MIDI-48
Crash Cymbal MIDI-49
Hi Tom MIDI-50
Ride Cymbal MIDI-51
*/

if (channel==10) {
if(note==35) BD16CNT=0;
if(note==36) BD16CNT=0;
if(note==37) RS16CNT=0;
if(note==38) SD16CNT=0;
if(note==39) CP16CNT=0;
if(note==40) SD16CNT=0;
if(note==41) LT16CNT=0;
if(note==42) HH16CNT=0;
if(note==43) LT16CNT=0;
if(note==44) HH16CNT=0;
if(note==45) MT16CNT=0;
if(note==46) OH16CNT=0;
if(note==47) MT16CNT=0;
if(note==48) HT16CNT=0;
if(note==49) CR16CNT=0;
if(note==50) HT16CNT=0;
if(note==51) RD16CNT=0;
}
}

MIDI data for this can come from edge triggers on GPIO’s, serial MIDI or rtpMIDI.

You can easily add velocity data to scale the samples in the engine to make accented drums.

Download the sketch here:

ESP909.ino

Rearrange the code for ISR

Using the CPU to fill the DMA buffer is not nice if you have a DMA and not good if you want to run MIDI input events.

So we are going to rearrange the code into a ISR serviced at a 2mS intervall.

The definitions are the same except for the added Ticker library.

#include <Arduino.h>
#include <i2s.h>
#include <i2s_reg.h>
#include <Ticker.h>

uint32_t i2sACC;
uint8_t i2sCNT=32;
uint16_t DAC=0x8000;
uint16_t err;

And our test sine waveform.

int16_t sine[256] = {
0x0000, 0x0324, 0x0647, 0x096a, 0x0c8b, 0x0fab, 0x12c8, 0x15e2,
0x18f8, 0x1c0b, 0x1f19, 0x2223, 0x2528, 0x2826, 0x2b1f, 0x2e11,
0x30fb, 0x33de, 0x36ba, 0x398c, 0x3c56, 0x3f17, 0x41ce, 0x447a,
0x471c, 0x49b4, 0x4c3f, 0x4ebf, 0x5133, 0x539b, 0x55f5, 0x5842,
0x5a82, 0x5cb4, 0x5ed7, 0x60ec, 0x62f2, 0x64e8, 0x66cf, 0x68a6,
0x6a6d, 0x6c24, 0x6dca, 0x6f5f, 0x70e2, 0x7255, 0x73b5, 0x7504,
0x7641, 0x776c, 0x7884, 0x798a, 0x7a7d, 0x7b5d, 0x7c29, 0x7ce3,
0x7d8a, 0x7e1d, 0x7e9d, 0x7f09, 0x7f62, 0x7fa7, 0x7fd8, 0x7ff6,
0x7fff, 0x7ff6, 0x7fd8, 0x7fa7, 0x7f62, 0x7f09, 0x7e9d, 0x7e1d,
0x7d8a, 0x7ce3, 0x7c29, 0x7b5d, 0x7a7d, 0x798a, 0x7884, 0x776c,
0x7641, 0x7504, 0x73b5, 0x7255, 0x70e2, 0x6f5f, 0x6dca, 0x6c24,
0x6a6d, 0x68a6, 0x66cf, 0x64e8, 0x62f2, 0x60ec, 0x5ed7, 0x5cb4,
0x5a82, 0x5842, 0x55f5, 0x539b, 0x5133, 0x4ebf, 0x4c3f, 0x49b4,
0x471c, 0x447a, 0x41ce, 0x3f17, 0x3c56, 0x398c, 0x36ba, 0x33de,
0x30fb, 0x2e11, 0x2b1f, 0x2826, 0x2528, 0x2223, 0x1f19, 0x1c0b,
0x18f8, 0x15e2, 0x12c8, 0x0fab, 0x0c8b, 0x096a, 0x0647, 0x0324,
0x0000, 0xfcdc, 0xf9b9, 0xf696, 0xf375, 0xf055, 0xed38, 0xea1e,
0xe708, 0xe3f5, 0xe0e7, 0xdddd, 0xdad8, 0xd7da, 0xd4e1, 0xd1ef,
0xcf05, 0xcc22, 0xc946, 0xc674, 0xc3aa, 0xc0e9, 0xbe32, 0xbb86,
0xb8e4, 0xb64c, 0xb3c1, 0xb141, 0xaecd, 0xac65, 0xaa0b, 0xa7be,
0xa57e, 0xa34c, 0xa129, 0x9f14, 0x9d0e, 0x9b18, 0x9931, 0x975a,
0x9593, 0x93dc, 0x9236, 0x90a1, 0x8f1e, 0x8dab, 0x8c4b, 0x8afc,
0x89bf, 0x8894, 0x877c, 0x8676, 0x8583, 0x84a3, 0x83d7, 0x831d,
0x8276, 0x81e3, 0x8163, 0x80f7, 0x809e, 0x8059, 0x8028, 0x800a,
0x8000, 0x800a, 0x8028, 0x8059, 0x809e, 0x80f7, 0x8163, 0x81e3,
0x8276, 0x831d, 0x83d7, 0x84a3, 0x8583, 0x8676, 0x877c, 0x8894,
0x89bf, 0x8afc, 0x8c4b, 0x8dab, 0x8f1e, 0x90a1, 0x9236, 0x93dc,
0x9593, 0x975a, 0x9931, 0x9b18, 0x9d0e, 0x9f14, 0xa129, 0xa34c,
0xa57e, 0xa7be, 0xaa0b, 0xac65, 0xaecd, 0xb141, 0xb3c1, 0xb64c,
0xb8e4, 0xbb86, 0xbe32, 0xc0e9, 0xc3aa, 0xc674, 0xc946, 0xcc22,
0xcf05, 0xd1ef, 0xd4e1, 0xd7da, 0xdad8, 0xdddd, 0xe0e7, 0xe3f5,
0xe708, 0xea1e, 0xed38, 0xf055, 0xf375, 0xf696, 0xf9b9, 0xfcdc
};

uint8_t phase=0; //Sine phase counter

The setup function now has some Timer code added.

void setup() {
i2s_begin(); //Start the i2s DMA engine
i2s_set_rate(44100); //Set sample rate
pinMode(2, INPUT); //restore GPIOs taken by i2s
pinMode(15, INPUT);
timer1_attachInterrupt(onTimerISR); //Attach our sampling ISR
timer1_enable(TIM_DIV16, TIM_EDGE, TIM_SINGLE);
timer1_write(2000); //Service at 2mS intervall
}

The main loop is now as you see empty.

void loop() {

}

This is because the DMA engine has moved to a ISR.

void ICACHE_RAM_ATTR onTimerISR(){ //Code needs to be in IRAM because its a ISR

while (!(i2s_is_full())) { //Don’t block the ISR if the buffer is full

DAC=0x8000+sine[phase++];

//Pulse Density Modulated 16-bit I2S DAC
for (uint8_t i=0;i<32;i++) { 
      i2sACC=i2sACC<<1; if(DAC >= err) {
        i2sACC|=1;
        err += 0xFFFF-DAC;
      }
        else
      {
        err -= DAC;
      }
     }
     bool flag=i2s_write_sample(i2sACC);
}

timer1_write(2000);//Next in 2mS
}

This does the same as the first example but you are now free to put what ever you like in the main loop because the timer takes care of loading data to the DMA.

The DMA is automatically serviced at a 2mS intervall and you can process MIDI data in the main loop instead.

Reading serial MIDI data

MIDI

How do we actually read the MIDI data? Our serial port is used by the i2s stream so cant be used as a serial port.

We do this by moving the RX and TX pins to the alternate pins.

Serial.swap();

This moves the RX pin to GPIO13 and the TX pin to GPIO15.

You need to setup the serial port before you start the i2s engine because the serial setup will destroy the i2s GPIO setup.

void setup() {
  Serial.begin(31250); //Start the serial port with default MIDI baudrate
  Serial.swap(); //Move the TX and RX GPIOs to 15 and 13
  i2s_begin(); //Start the i2s DMA engine
  i2s_set_rate(44100); //Set sample rate
  pinMode(2, INPUT); //restore GPIOs taken by i2s
  pinMode(15, INPUT);
  timer1_attachInterrupt(onTimerISR); //Attach our sampling ISR
  timer1_enable(TIM_DIV16, TIM_EDGE, TIM_SINGLE);
  timer1_write(2000); //Service at 2mS intervall
}

Add the MIDI process definitions.

uint8_t MIDISTATE=0;
uint8_t MIDIRUNNINGSTATUS=0;
uint8_t MIDINOTE;
uint8_t MIDIVEL;

And the MIDI processor.

void processMIDI(uint8_t MIDIRX) {

/*
Handling “Running status”
1.Buffer is cleared (ie, set to 0) at power up.
2.Buffer stores the status when a Voice Category Status (ie, 0x80 to 0xEF) is received.
3.Buffer is cleared when a System Common Category Status (ie, 0xF0 to 0xF7) is received.
4.Nothing is done to the buffer when a RealTime Category message is received.
5.Any data bytes are ignored when the buffer is 0.
*/
if ((MIDIRX>0xBF)&&(MIDIRX<0xF8)) { MIDIRUNNINGSTATUS=0; MIDISTATE=0; return; } if (MIDIRX>0xF7) return;
if (MIDIRX & 0x80) {
  MIDIRUNNINGSTATUS=MIDIRX;
  MIDISTATE=1;
return;
}
if (MIDIRX < 0x80) {
  if (!MIDIRUNNINGSTATUS) return;
  if (MIDISTATE==1) {
  MIDINOTE=MIDIRX;
  MIDISTATE++;
  return;
}
if (MIDISTATE==2) {
  MIDIVEL=MIDIRX;
  MIDISTATE=1;
  //if (MIDIRUNNINGSTATUS==0x80) handleMIDInoteOFF(MIDIRUNNINGSTATUS,MIDINOTE,MIDIVEL);
  //if (MIDIRUNNINGSTATUS==0x90) handleMIDInoteON(MIDIRUNNINGSTATUS,MIDINOTE,MIDIVEL);
  //if (MIDIRUNNINGSTATUS==0xB0) handleMIDICC(MIDINOTE,MIDIVEL);
}
}
}

You need to add the handlers for noteOFF, noteON and MIDICC.

Now we can process incoming MIDI bytes in the main loop.

void loop() {
  if (Serial.available()) processMIDI(Serial.read());
}

Now you can apply our new DMA engine and serial MIDI processor on the simple drum player and play it from a keyboard or sequencer.

rtpMIDI on the ESP8266

What about rtpMIDI, or Apple-MIDI over WiFI?

Well it works pretty well on our drum machine.

To use it you need to download and install the Apple-MIDI library:
https://github.com/lathoub/Arduino-AppleMIDI-Library

Our drum machine defines.

#include <Arduino.h> 
#include "ESP8266WiFi.h"
#include <WiFiClient.h>
#include <WiFiUdp.h>
#include <i2s.h>
#include <i2s_reg.h>
#include <pgmspace.h>
#include "AppleMidi.h"
#include <Ticker.h>

extern “C” {
#include “user_interface.h”
}

char ssid[] = “YourSSID”; //  your network SSID (name)
char pass[] = “YourKEY”;    // your network password (use for WPA, or use as key for WEP)

APPLEMIDI_CREATE_INSTANCE(WiFiUDP, AppleMIDI); // see definition in AppleMidi_Defs.h

// Forward declaration
void OnAppleMidiConnected(uint32_t ssrc, char* name);
void OnAppleMidiDisconnected(uint32_t ssrc);
void OnAppleMidiNoteOn(byte channel, byte note, byte velocity);
void OnAppleMidiNoteOff(byte channel, byte note, byte velocity);

uint32_t i2sACC;
uint8_t i2sCNT=32;
uint16_t DAC=0x8000;
uint16_t err;

uint32_t BD16CNT;
uint32_t CP16CNT;
uint32_t CR16CNT;
uint32_t HH16CNT;
uint32_t HT16CNT;
uint32_t LT16CNT;
uint32_t MT16CNT;
uint32_t CH16CNT;
uint32_t OH16CNT;
uint32_t RD16CNT;
uint32_t RS16CNT;
uint32_t SD16CNT;

#define BD16LEN 3796UL
#define CP16LEN 4445UL
#define CR16LEN 48686UL
#define HH16LEN 1734UL
#define HT16LEN 5802UL
#define LT16LEN 7061UL
#define MT16LEN 7304UL
#define OH16LEN 4772UL
#define RD16LEN 52850UL
#define RS16LEN 1316UL
#define SD16LEN 5577UL

const uint16_t BD16[3796] PROGMEM = {
40,  85, 137, 144, -30, -347, -609, -785, // 0-7

The defines above are the same as the original drum sampler code but with the Apple MIDI added.

The 909 synth engine is still the same.

uint16_t SYNTH909() {
  int32_t DRUMTOTAL=0;
  if (BD16CNT<BD16LEN) DRUMTOTAL+=(pgm_read_word_near(BD16 + BD16CNT++)^32768)-32768;
  if (CP16CNT<CP16LEN) DRUMTOTAL+=(pgm_read_word_near(CP16 + CP16CNT++)^32768)-32768;
  if (CR16CNT<CR16LEN) DRUMTOTAL+=(pgm_read_word_near(CR16 + CR16CNT++)^32768)-32768;
  if (HH16CNT<HH16LEN) DRUMTOTAL+=(pgm_read_word_near(HH16 + HH16CNT++)^32768)-32768;
  if (HT16CNT<HT16LEN) DRUMTOTAL+=(pgm_read_word_near(HT16 + HT16CNT++)^32768)-32768;
  if (LT16CNT<LT16LEN) DRUMTOTAL+=(pgm_read_word_near(LT16 + LT16CNT++)^32768)-32768;
  if (MT16CNT<MT16LEN) DRUMTOTAL+=(pgm_read_word_near(MT16 + MT16CNT++)^32768)-32768;
  if (OH16CNT<OH16LEN) DRUMTOTAL+=(pgm_read_word_near(OH16 + OH16CNT++)^32768)-32768;
  if (RD16CNT<RD16LEN) DRUMTOTAL+=(pgm_read_word_near(RD16 + RD16CNT++)^32768)-32768;
  if (RS16CNT<RS16LEN) DRUMTOTAL+=(pgm_read_word_near(RS16 + RS16CNT++)^32768)-32768;
  if (SD16CNT<SD16LEN) DRUMTOTAL+=(pgm_read_word_near(SD16 + SD16CNT++)^32768)-32768; if (DRUMTOTAL>32767) DRUMTOTAL=32767;
  if  (DRUMTOTAL<-32767) DRUMTOTAL=-32767;
  DRUMTOTAL+=32768;
  return DRUMTOTAL;
}

Setup includes some new code to add the ESP8266 to your WiFi network.

void setup() {
//WiFi.forceSleepBegin();
//delay(1);
system_update_cpu_freq(160);

//Serial.begin(9600);

WiFi.begin(ssid, pass);

while (WiFi.status() != WL_CONNECTED) {
delay(500);
}

//Serial.print(F(“IP address is “));
//Serial.println(WiFi.localIP());

AppleMIDI.begin(“ESP909”); // ‘ESP909’ will show up as the session name

AppleMIDI.OnReceiveNoteOn(OnAppleMidiNoteOn);

i2s_begin();
i2s_set_rate(44100);
timer1_attachInterrupt(onTimerISR); //Attach our sampling ISR
timer1_enable(TIM_DIV16, TIM_EDGE, TIM_SINGLE);
timer1_write(2000); //Service at 2mS intervall

}

The main loop now has the Apple MIDI status code in it.

void loop() {
AppleMIDI.run();
}

And our main sampling ISR is the same.

void ICACHE_RAM_ATTR onTimerISR(){

while (!(i2s_is_full())) { //Don’t block the ISR

DAC=SYNTH909();

//Pulse Density Modulated 16-bit I2S DAC
for (uint8_t i=0;i<32;i++) { 
      i2sACC=i2sACC<<1; if(DAC >= err) {
        i2sACC|=1;
        err += 0xFFFF-DAC;
      }
        else
      {
        err -= DAC;
      }
     }
     bool flag=i2s_write_sample(i2sACC);

}

timer1_write(2000);//Next in 2mS
}

But now there is a new function added to handle MIDI events.

void OnAppleMidiNoteOn(byte channel, byte note, byte velocity) {

/* Triggers
Bass Drum MIDI-35
Bass Drum MIDI-36
Rim Shot MIDI-37
Snare Drum MIDI-38
Hand Clap MIDI-39
Snare Drum MIDI-40
Low Tom MIDI-41
Closed Hat MIDI-42
Low Tom MIDI-43
Closed Hat MIDI-44
Mid Tom MIDI-45
Open Hat MIDI-46
Mid Tom MIDI-47
Hi Tom MIDI-48
Crash Cymbal MIDI-49
Hi Tom MIDI-50
Ride Cymbal MIDI-51
*/

if (channel==10) {
if(note==35) BD16CNT=0;
if(note==36) BD16CNT=0;
if(note==37) RS16CNT=0;
if(note==38) SD16CNT=0;
if(note==39) CP16CNT=0;
if(note==40) SD16CNT=0;
if(note==41) LT16CNT=0;
if(note==42) HH16CNT=0;
if(note==43) LT16CNT=0;
if(note==44) HH16CNT=0;
if(note==45) MT16CNT=0;
if(note==46) OH16CNT=0;
if(note==47) MT16CNT=0;
if(note==48) HT16CNT=0;
if(note==49) CR16CNT=0;
if(note==50) HT16CNT=0;
if(note==51) RD16CNT=0;
}
}

To make this work you need to set up Apple rtpMIDI on your Mac, iPad or PC.

I cant show you how to do this because it is up to your platform how it is done.

And you need to find the IP address of your ESP8266 to pair it with your MIDI computer.

Enable the serial debug code and watch the serial console for your IP address.

But once that is done it works pretty well.

Download the sketch here:

rtpMIDI909.ino

This is the most basic sample player.
As long as the samples fits into your flash space you can play whatever you want and control it over MIDI or WiFi.

Compared to the samplers of the 90’s this is way better. 16-bit audio and 4Mbyte memory in a tiny space is way to good.

Next up is making a sample player synth on our sampling MIDI platform.

Have fun!

#include <Arduino.h> 
#include "ESP8266WiFi.h"
#include <WiFiClient.h>
#include <WiFiUdp.h>
#include <i2s.h>
#include <i2s_reg.h>
#include <pgmspace.h>
#include "AppleMidi.h"
#include <Ticker.h>
 
extern "C" {
#include "user_interface.h"
}
 
#define GPIO_IN ((volatile uint32_t*) 0x60000318)    // register contains gpio pin values of ESP8266 in read mode + some extra values (need to be truncated)
 
char ssid[] = "YourSSID"; //  your network SSID (name)
char pass[] = "YourKEY";    // your network password (use for WPA, or use as key for WEP)
 
APPLEMIDI_CREATE_INSTANCE(WiFiUDP, AppleMIDI); // see definition in AppleMidi_Defs.h
 
// Forward declaration
void OnAppleMidiConnected(uint32_t ssrc, char* name);
void OnAppleMidiDisconnected(uint32_t ssrc);
void OnAppleMidiNoteOn(byte channel, byte note, byte velocity);
void OnAppleMidiNoteOff(byte channel, byte note, byte velocity);
 
uint32_t i2sACC;
uint8_t i2sCNT=32;
uint16_t DAC=0x8000;
uint16_t err;
 
uint32_t BD16CNT;
uint32_t CP16CNT;
uint32_t CR16CNT;
uint32_t HH16CNT;
uint32_t HT16CNT;
uint32_t LT16CNT;
uint32_t MT16CNT;
uint32_t CH16CNT;
uint32_t OH16CNT;
uint32_t RD16CNT;
uint32_t RS16CNT;
uint32_t SD16CNT;
 
 
uint32_t samplecounter=100;
uint32_t TRIG0, TRIG1, TRIG2, TRIG3, TRIG4, TRIG5, TRIG6, TRIG7, TRIG8, TRIG9, TRIG10;
uint32_t OLDTRIG0,OLDTRIG1,OLDTRIG2,OLDTRIG3,OLDTRIG4,OLDTRIG5,OLDTRIG6,OLDTRIG7,OLDTRIG8,OLDTRIG9,OLDTRIG10;
 
 
#define BD16LEN 3796UL
#define CP16LEN 4445UL
#define CR16LEN 48686UL
#define HH16LEN 1734UL
#define HT16LEN 5802UL
#define LT16LEN 7061UL
#define MT16LEN 7304UL
#define OH16LEN 4772UL
#define RD16LEN 52850UL
#define RS16LEN 1316UL
#define SD16LEN 5577UL
 
const uint16_t BD16[3796] PROGMEM = {
40,  85, 137, 144, -30, -347, -609, -785, // 0-7
 
uint16_t SYNTH909() {
  int32_t DRUMTOTAL=0;
  if (BD16CNT<BD16LEN) DRUMTOTAL+=(pgm_read_word_near(BD16 + BD16CNT++)^32768)-32768;
  if (CP16CNT<CP16LEN) DRUMTOTAL+=(pgm_read_word_near(CP16 + CP16CNT++)^32768)-32768;
  if (CR16CNT<CR16LEN) DRUMTOTAL+=(pgm_read_word_near(CR16 + CR16CNT++)^32768)-32768;
  if (HH16CNT<HH16LEN) DRUMTOTAL+=(pgm_read_word_near(HH16 + HH16CNT++)^32768)-32768;
  if (HT16CNT<HT16LEN) DRUMTOTAL+=(pgm_read_word_near(HT16 + HT16CNT++)^32768)-32768;
  if (LT16CNT<LT16LEN) DRUMTOTAL+=(pgm_read_word_near(LT16 + LT16CNT++)^32768)-32768;
  if (MT16CNT<MT16LEN) DRUMTOTAL+=(pgm_read_word_near(MT16 + MT16CNT++)^32768)-32768;
  if (OH16CNT<OH16LEN) DRUMTOTAL+=(pgm_read_word_near(OH16 + OH16CNT++)^32768)-32768;
  if (RD16CNT<RD16LEN) DRUMTOTAL+=(pgm_read_word_near(RD16 + RD16CNT++)^32768)-32768;
  if (RS16CNT<RS16LEN) DRUMTOTAL+=(pgm_read_word_near(RS16 + RS16CNT++)^32768)-32768;
  if (SD16CNT<SD16LEN) DRUMTOTAL+=(pgm_read_word_near(SD16 + SD16CNT++)^32768)-32768; if (DRUMTOTAL>32767) DRUMTOTAL=32767;
  if  (DRUMTOTAL<-32767) DRUMTOTAL=-32767;
  DRUMTOTAL+=32768;
  return DRUMTOTAL;
}
 
 
void setup() {
  //WiFi.forceSleepBegin();             
  //delay(1);                               
  system_update_cpu_freq(160);
 
  //Serial.begin(9600);
 
  WiFi.begin(ssid, pass);
 
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
  }
 
  //Serial.print(F("IP address is ")); 
  //Serial.println(WiFi.localIP()); 
 
 
  AppleMIDI.begin("ESP909"); // 'ESP909' will show up as the session name
 
  AppleMIDI.OnReceiveNoteOn(OnAppleMidiNoteOn);
 
  i2s_begin();
  i2s_set_rate(44100);
  timer1_attachInterrupt(onTimerISR); //Attach our sampling ISR
  timer1_enable(TIM_DIV16, TIM_EDGE, TIM_SINGLE);
  timer1_write(2000); //Service at 2mS intervall
 
 
}
 
void loop() {
 AppleMIDI.run();
} 
 
void ICACHE_RAM_ATTR onTimerISR(){
 
  while (!(i2s_is_full())) { //Don't block the ISR
 
    DAC=SYNTH909();
 
    //----------------- Pulse Density Modulated 16-bit I2S DAC --------------------
     for (uint8_t i=0;i<32;i++) { 
      i2sACC=i2sACC<<1; if(DAC >= err) {
        i2sACC|=1;
        err += 0xFFFF-DAC;
      }
        else
      {
        err -= DAC;
      }
     }
     bool flag=i2s_write_sample(i2sACC);
    //-----------------------------------------------------------------------
 
  }
 
  timer1_write(2000);//Next in 2mS
}
 
void OnAppleMidiNoteOn(byte channel, byte note, byte velocity) {
 
/* Triggers
Bass Drum MIDI-35
Bass Drum MIDI-36
Rim Shot MIDI-37
Snare Drum MIDI-38
Hand Clap MIDI-39
Snare Drum MIDI-40
Low Tom MIDI-41
Closed Hat MIDI-42
Low Tom MIDI-43
Closed Hat MIDI-44
Mid Tom MIDI-45
Open Hat MIDI-46
Mid Tom MIDI-47
Hi Tom MIDI-48
Crash Cymbal MIDI-49
Hi Tom MIDI-50
Ride Cymbal MIDI-51
*/
 
  if (channel==10) {
    if(note==35) BD16CNT=0;
    if(note==36) BD16CNT=0;
    if(note==37) RS16CNT=0;
    if(note==38) SD16CNT=0;
    if(note==39) CP16CNT=0;
    if(note==40) SD16CNT=0;
    if(note==41) LT16CNT=0;
    if(note==42) HH16CNT=0;
    if(note==43) LT16CNT=0;
    if(note==44) HH16CNT=0;
    if(note==45) MT16CNT=0;
    if(note==46) OH16CNT=0;
    if(note==47) MT16CNT=0;
    if(note==48) HT16CNT=0;
    if(note==49) CR16CNT=0;
    if(note==50) HT16CNT=0;  
    if(note==51) RD16CNT=0;
  }
}

 

__________________________________________________
Like to contribute to open source development?
Please consider a small donation if downloading.

donate

 

37 Replies to “Audio Hacking On The ESP8266”

  1. Hi, the file “user_interface.h” is missing. This could be a problem for “beginners”.
    Because of the size of the ESP8266-baords, this could be a good mod for a “Volca-Beat” to get “808” or more “909”-sounds.

    Great work!

  2. Thank you Jan for for all the work you have done with ESP8266 and everything else. Your knowledge and the will to share is indispensable. I have cracked my head trying to make some things work, found this blog and solved everything in five minutes 🙂 Thank you a million times.

      1. Can you generate the sample data to use in the sketch with Audacity?
        I normally use a program called wav2sketch (for the teensy). It takes a .wav file as input and generates a .cpp file.
        What sample rate are your data supposed to be?

  3. Hi jan, is the function writeDAC is built in ?

    or it just this as a function:

    //—————– Pulse Density Modulated 16-bit I2S DAC ——————–
    for (uint8_t i=0;i<32;i++) {
    i2sACC=i2sACC<= err) {
    i2sACC|=1;
    err += 0xFFFF-DAC;
    }
    else
    {
    err -= DAC;
    }
    }
    bool flag=i2s_write_sample(i2sACC);
    //———————————————————————–

  4. Hi Jan,

    Please give me some support to get the circuit running.
    I set up the circuit and installed the software ESP909.ino.
    After switching on I hear a crash cymbal.
    I send Midi Notes via channel 10 but hear nothing.
    Midi is connected via an optocoupler to D7.
    Thanks in advance and best regards
    Andreas

      1. It works, thank you very much!
        Well done Jan!
        Which program do you use for converting .wav to program data and XX16LEN?
        Cheers
        Andreas

        1. I belive I did it with a custom Visual Basic program.

          Some have had luck with Audacity export function.
          The data is 16-bit signed.
          Same as in a 16-bit mono wavefile “DATA” chunk.

  5. Hi Jan, why is the drum output need to be xor-ed?

    is this because we need to get two complement number from the sampler?

    if (CP16CNT<CP16LEN) DRUMTOTAL+=(pgm_read_word_near(CP16 + CP16CNT++)^32768)-32768;

    thanks

  6. Hi, when I try to compile rtpMIDI909.INO with Arduino IDE (1.8.5 and older) failed:

    timer1_attachInterrupt(onTimerISR); exit status 1
    ‘onTimerISR’ was not declared in this scope.
    What’s going wrong?
    Tks a lot for your reply

  7. Hi Janost, maybe you could provide me some tips if I just want to read data from MIDI out (from a Yamaha WX5) and transmit it to an iPad or Macbook?
    Thanks
    Edilberto

    1. Hi Edilberto.
      You can use the same AppleMIDI library for that.
      I’m planning to make a post on building your own WiFi-MIDI network.

  8. Hello and thank you for sharing all these great information and source codes. Your code is working here in tehran, Iran on several arduinos. mercy…

    Sorry for off-topic questions, but here’s the only page I could reach you.

    1 – Is it possible to add volume control for each drum triggers (BD,SD,..) in dsp-D8 or O2? I see cv control for mute. could it be used for volume control?

    2 – Is there any chance your release dsp-G1 source code some day? I believe it’s one of your highlight works. So many great features packed into a little chip. we can learn a lot from every line of your code.

    My country is heavily sanctioned at the moment, wish I could order your chips. but that forced me to learn electronics and programing my own chip.

    Respect and Regards,
    Shawn

  9. Thank you for directions!

    and a compatibility question: can I use ATtiny24 family instead of ATtiny85 ones? the specs are nearly the same. the problem is 85s are rare to find over here.

      1. I would really like to Jan, but I’m living in iran where we have problem with online payments and generally international banking. My country’s banking system is not connected to worldwide networks. No credit carts, no paypal, nothing! and add sanctions to top of that 🙁

        The postal system also sucks since you gotta get gov permissions to receive electronic stuff from abraod..

        otherwise i would love to purchase a lot of your stuff specially the drum chips. I’m still researching how to buy them and possibly get them here in tehran. maybe i can receive them through private post services like DHL.

  10. Great project! Thanks a lot. Is it possible to create project which reads sample from I2S and records on SPIFF flash for looping? It is interesting of possibility to create simple looper on ESP8266 or delay. Thank you!

    1. Thank you.
      Yes, it’s possible but you need a i2s ADC.
      And then my DAC code wont work anymore.

      And if you add a i2s ADC you might add a i2s DAC aswell.

      1. Sounds cool. I tried to hook stream of i2s data one year ago, from ADC and DAC which I scavenged from broken fx-processor, but my knowledge of i2s protocol (left justified) is less then beginner level :))) Didnt work. There are a lot of stereo ADC and DAC (16 bit) can be recycled

  11. Hi Jan,
    Hope you’re enjoying summer, and sorry for bothering you with my questions.. 🙂

    I’m modifying D8 code so the 6 ADC ports control volume level of instruments (kick,snare,..). I could figure out the pitch system you coded, but where in code I can control the volume? Is it the value 127 in your code?

    if (samplecntSD) {
    phaccSD+=pitchSD;
    if (phaccSD & 128) {
    phaccSD &= 127; // <—– is it here?
    total+=(pgm_read_byte_near(SN + samplepntSD)-128);
    samplepntSD++;
    samplecntSD–; }

    And so, in ADC block how can I scale the incoming value to volume level? It's already set for pitch:

    uint16_t pitch=((ADCL+(ADCH<>3)+1;

    And finally a dumb question about ATMEGA328 pins: Pins 7 and 8 are VCC and GND on chip, yet we set them up as input triggers:

    //Drumtrigger inputs
    pinMode(7,INPUT_PULLUP);
    pinMode(8,INPUT_PULLUP);

    Is this ok and or I’m doing wrong?

    Thank you.

    1. The row, total+= is where you can scale the sample.
      Either by shifting it down or by dividing the value.

      Digital pin 7 is not physical pin 7.

  12. Hello,
    I just can’t seem to get the code to work with just a simple Din 5 Midi input. I keep getting errors in my code relating to

    error: ‘onTimerISR’ was not declared in this scope

    May I ask someone to post or PM me their code so I can get further? = )

  13. Hi Jan,

    I am getting a click sound after flashing ESP909.ino or rtpMIDI909.ino. My loop() looks like this:

    void loop() {
    //int n = random(35,51);
    for (byte n=35; n<=51; n++) {
    MidiNoteOn(10,n,100);
    delay(500);
    }
    }

    Do you have an idea whats wrong? Thanks, Torsten

    1. The clicking sound comes from the wifi radio pulling alot of current when transmitting.

      Either turn the radio off or use a better power supply to the ESP.

Leave a Reply

Your email address will not be published. Required fields are marked *