This project description is also available as PDF document.

Do you know a musical instrument that is played with the hands without the hands, or any other part of the body, touching the instrument? Such a thing has existed since the 1920s. Its name is "Theremin".

The original instrument consisted of two high frequency transmitters with antennas. By bringing the hands closer to these antennas and the resulting detuning of the transmission frequency, the volume and pitch of the instrument was ultimately influenced.

Today I will show you that it is also possible without high frequency and without antennas. So welcome to

MicroPython on the ESP32 and ESP8266

today

The ESP32 as a musician

While experimenting with a fantastic module, which actually comes from a completely different application corner than the music area, the idea of using this ToF module to generate tones gradually matured.

ToF is the acronym for Time of Flight and the module VL53L0X Time-of-Flight (ToF) Laser Distance Sensor does nothing else but measure the time of flight of a laser pulse emitted by it until it is reflected by an obstacle and back.

Image 1: ESP32 - Magic Music Synthesizer

Measurement results can be retrieved via the I2C bus. The numerical values range from two to four digits. This can be converted one-to-one into the frequency of sound signals. And for this the output of a PWM signal at any ESP-GPIO pin can be used.

But there were two problems to solve for the volume control. Right at the beginning it turned out that the above mentioned module has a fixed hardware address and thus only one module can be used on the same bus. There are I2C multiplexers that act as switches, and thus can switch between two modules with the same address. My original approach was to use the second module also via a PWM signal with variable Duty Cycle to drive an LED, which is connected via a voltage divider with a LDR should have controlled the volume.

But sometimes you can't see the forest for the trees. I couldn't find a solution without the LDR, so it had to serve as a volume control in any case. But then you can simply illuminate it with ambient light and shade it by hand to change its resistance value.

These considerations finally led to the circuit diagram for the ESP32-Theremin.

Image 2: Magic Music - circuit

Hardware

1	ESP32 Dev Kit C unsoldered or ESP32 NodeMCU Module WLAN WiFi Development Board or NodeMCU-ESP-32S Kit
1	VL53L0X Time-of-Flight (ToF) Laser Distance Sensor
1	PAM8403 digital mini audio amplifier 2x 3 Watt DC 5V
1	2 pieces DFplayer Mini 3 Watt 8 Ohm mini speakers
1	KY-018 photo LDR resistor or Photo Resistor Photo Resistor Diodes Set 150V 5mm LDR5528
1	Resistor 47kΩ
various	Jumper cable
1	Minibreadboard or Breadboard Kit - 3 x 65pcs Jumper Wire Cable M2M and 3 x Mini Breadboard 400 Pins

Image 3: VL53L0X - Laser Rangefinder

Image 4: Mini loudspeaker and amplifier

The software

For flashing and programming the ESP32:

Thonny or

µPyCraft

Used firmware for the ESP32:

MicropythonFirmware

Please be sure to flash the version given here, otherwise the PWM control will not work correctly. With newer versions the minimum PWM frequency is 700Hz upwards! Why, the vulture knows!

ESP32 with 4MB Version 1.15 Status 18.04.2021

The MicroPython programs for the project:

VL53L0X.py modified driver module for the ToF sensor adapted to ESP32(S)

magicmusic.py extensible software for ESP32-Theremin

original software on github for wipy made by pycom

Sample files:

magic1.py

magic2.py

magic3.py

magic4.py

magic5.py

MicroPython - Language - Modules and programs

For the installation of Thonny you find here a detailed manual (english version). In it there is also a description how the Micropython firmware (state 05.02.2022) on the ESP chip. burned is burned.

MicroPython is an interpreter language. The main difference to the Arduino IDE, where you always and only flash whole programs, is that you only need to flash the MicroPython firmware once at the beginning to the ESP32, so that the controller understands MicroPython instructions. You can use Thonny, µPyCraft or esptool.py to do this. For Thonny, I have described the process here described here.

Once the firmware is flashed, you can casually talk to your controller one-on-one, test individual commands, and immediately see the response without having to compile and transfer an entire program first. In fact, that's what bothers me about the Arduino IDE. You simply save an enormous amount of time if you can do simple tests of the syntax and the hardware up to trying out and refining functions and whole program parts via the command line in advance before you knit a program out of it. For this purpose I also like to create small test programs from time to time. As a kind of macro they summarize recurring commands. From such program fragments sometimes whole applications are developed.

Autostart

If you want the program to start autonomously when the controller is switched on, copy the program text into a newly created blank file. Save this file as boot.py in the workspace and upload it to the ESP chip. The program will start automatically at the next reset or power-on.

Test programs

Manually, programs are started from the current editor window in the Thonny IDE via the F5 key. This is quicker than clicking on the Start button, or via the menu Run. Only the modules used in the program must be in the flash of the ESP32.

In between times Arduino IDE again?

If you want to use the controller together with the Arduino IDE again later, simply flash the program in the usual way. However, the ESP32/ESP8266 will then have forgotten that it ever spoke MicroPython. Conversely, any Espressif chip that contains a compiled program from the Arduino IDE or the AT firmware or LUA or ... can easily be flashed with the MicroPython firmware. The process is always like here described.

The driver module for the VR53L01

The data sheet for the VR53L01 does not provide a register map and a description for the values to be set as usual. Instead an API is described, which serves as interface to the module. Long identifiers in a colorful variety pelt down on the reader. This very quickly brought my eagerness to write a module down to zero. Fortunately, after a bit of searching, I came across Github I found what I was looking for. There is a package with a readme file, an example for the application and with the very extensive module VL53L0X.py (648 lines). The first start of the example file main.py brought a disillusionment, although I had uploaded the module to the ESP32S and rewritten the pins for the ESP32.

The original

from machine import I2C
i2c = I2C(0)
i2c = I2C(0, I2C.MASTER)
i2c = I2C(0, pins=('P10','P9')) 

becomes

from machine import SoftI2C
i2c = I2C(0,scl=Pin(21),sda=Pin(22))

The MicroPython interpreter bleated a missing Chrono object, in the module VL53L0X.VL53L0X in line 639. After the module is written for another port, wipy devices made by Pycomit could well be that more such errors would appear. Other firmware, other identifiers for GPIO pins, other integration of hardware modules of the ESP32...

But since no other connection between ESP32 and VR53L01 is needed in the example except the I2C pins, I rated the probability of further trouble spots as low. So I took the method perform_single_ref_calibration() in the module VL53L0X.VL53L0X in the editor. The name Chrono indicated a time object.

And indeed, in the firmware of the WiPy the use of the hardware timers is solved differently than in the native ESP32. But in principle it is only about the initialization and checking of a timeout.

from machine import Timer
...
...
    def perform_single_ref_calibration(self, vhv_init_byte):
        chrono = Timer.chrono()
        self._register(SYSRANGE_START, 0x01|vhv_init_byte)
        chrono.start()
        while self._register((RESULT_INTERRUPT_STATUS & 0x07) = 0):
            time_elapsed = chrono.read_ms()
            if time_elapsed > _IO_TIMEOUT:
                return False
        self._register(SYSTEM_INTERRUPT_CLEAR, 0x01)
        self._register(SYSRANGE_START, 0x00)
        return True 

But for this I have my own software solution. A function, sorry, a method, we are moving in the definition of a class, so a method TimeOut() that takes a duration in milliseconds and returns the reference to a function inside it. Such a construct is called Closure. With this I have now simply replaced the time control, shorter but just as effective.

    def TimeOut(self,t):
        start=time.ticks_ms()
        def compare():
            return int(time.ticks_ms()-start) >= t
        return compare

    def perform_single_ref_calibration(self, vhv_init_byte):
        self._register(SYSRANGE_START, 0x01|vhv_init_byte)
        chrono = self.TimeOut(_IO_TIMEOUT)
        while self._register((RESULT_INTERRUPT_STATUS & 0x07) == 0):
            if chrono() :
                return False
        self._register(SYSTEM_INTERRUPT_CLEAR, 0x01)
        self._register(SYSRANGE_START, 0x00)
        return True

Furthermore, I was relieved when no further error message appeared when I restarted the demo program. On the contrary, the terminal displayed nice distance values in mm.

import sys,os
import time
from machine import Pin,PWM
from machine import I2C
import VL53L0X
​
i2c = I2C(0,scl=Pin(21),sda=Pin(22))
​
# Create a VL53L0X object
tof = VL53L0X.VL53L0X(i2c)
sound=PWM(Pin(18), freq=20, duty=512)
​
tof.set_Vcsel_pulse_period(tof.vcsel_period_type[0], 18)
​
tof.set_Vcsel_pulse_period(tof.vcsel_period_type[1], 14)
​
tof.start()
​
while True:
# Start ranging
    tof.read()
    print(tof.read())

The values were directly in the usable audio frequency range. So I only needed to pass the values to the PWM frequency setting instead of the output to the terminal.

import sys,os
import time
from machine import Pin,PWM
from machine import SoftI2C
import VL53L0X
​
i2c = SoftI2C(scl=Pin(21),sda=Pin(22))
​
# Create a VL53L0X object
tof = VL53L0X.VL53L0X(i2c)
sound=PWM(Pin(18), freq=20, duty=512)
​
tof.set_Vcsel_pulse_period(tof.vcsel_period_type[0], 18)
​
tof.set_Vcsel_pulse_period(tof.vcsel_period_type[1], 14)
​
tof.start()
print("started")
​
while True:
    sound.freq(tof.read()) 

Did you build everything? Does the module VR53L0X.py already live in the flash of the ESP32? Is the amplifier switched on? Then start magic1.py in an editor window. And if now a happy droning sounds from the loudspeakers and the sound frequency changes by raising and lowering the palm over the VR53L01, then you have won.

The volume can be adjusted steplessly in the good old analog way via the voltage divider of 47kΩ resistor and LDR, by shading the LDR - louder, or exposing it - softer. The value of the resistor should be chosen in a way that the volume is nearly zero at normal exposure. The resistor value to be selected thus depends on the prevailing ambient brightness.

Depending on what you do with the values read in from the VR53L01, you can achieve different sound results. In the following I simply present some program snippets. You can find them in the example files magicX.py, with X in range (1,6), to express it in MicroPython jargon.

tof.start()
f1=500
print("started")
​
while True:
    f2=(tof.read())
    df=(f2-f1)
    if df != 0:
        s= df//10 if df >= 50 else  df//abs(df)
    if s != 0:
        for f in range(f1,f2,s):
            sound.freq(f)
            time.sleep_ms(10)
    f1=f2

tof.start()
print("started")
f1=500
f2=(tof.read())
while True:
    f2=(tof.read())
    f=(f2+f1*os.urandom(1)[0]*2)//2
    sound.freq(f)
    f1=f2

tof.start()
f1=500
print("started")
f2=(tof.read())
while True:
    f2=(tof.read())*2
    df=(f2-f1)
    s= df//40 
    if s != 0:
        for f in range(f1,f2,s):
            sound.freq(f)
            time.sleep_ms(10)
    f1=f2

tof.start()
f1=500
print("started")
f2=(tof.read())
while True:
    f2=((tof.read())+7*f1)//8
    sound.freq(f2)
    time.sleep_ms(20)
    f1=f2

The examples show that the sound can be influenced by the following measures. Combinations of these are conceivable.

Frequency f2 and processing with f1

•  Feedback
•  Addition, subtraction
•  Multiplication with a random number

…

• Inserting a for loop for soft transitions
• Presetting the playing time per frequency
• Periodic volume change by finger fan or flickering light

One more tip at the end:

Open leads to the amplifier can pick up spurious radiation from the environment, e.g. 50Hz hum. This can also come from artificial lighting by fluorescent lamps but also from power supplies of LED lamps and be picked up by the LDR.

And now, have fun experimenting!