In this chapter, we set up a development environment that allows us to create the firmware, its unit tests, and the documentation. For this we install the Espressif IoT Development Framework (ESP-IDF) which, in addition to the obvious support for ESP chips, can also be compiled for Linux. This has the advantage that our unit tests can run directly on the host.

We recommend either an Arch (e.g. Garuda or Manjaro) or Ubuntu based distribution, so all of the following steps refer to those.

Prerequisites

In order to start developing the firmware, we need to meet quite a few prerequisites. Fortunately, most of them can be obtained directly from the package manager. But before we do that, let's bring our system up to date.

Arch
sudo pacman -Syu --noconfirm
Ubuntu 24.04
sudo apt update -y

sudo apt upgrade -y

Without going into detail about each individual dependency, the most important ones are CMake, a build system, GCC, a host compiler, Ninja, a build tool, Doxygen, a documentation generator, and Graphviz, a graph visualization software.

CMake ( >= 3.25 )
GCC ( >= 13.2.0 )
ESP-IDF ( >= 5.3 )
Ninja ( >= 1.10.2 )
Optional
- for building documentation
  - Doxygen ( >= 1.12.0 )
  - Graphviz ( >= 12.1.1 )

Arch
sudo pacman -S --noconfirm bison ccache cmake dfu-util doxygen flex gcc git gperf graphviz jdk-openjdk libbsd libusb make ninja python ruby
Ubuntu 24.04
sudo apt-get install -y bison build-essential ccache clang-format cmake dfu-util doxygen flex git gperf graphviz libbsd-dev libffi-dev libssl-dev libusb-1.0-0 ninja-build openjdk-21-jdk python3 python3-pip python3-venv ruby wget

ESP-IDF

The ESP-IDF framework is the only dependency that we cannot get directly from the packet manager. Instead, we follow Espressif's instructions and clone the ESP-IDF repository into a directory called esp in our home directory.

GitHub allows you to clone a repository either over SSH or HTTPS. If you have the option, we recommend using SSH as we believe it simplifies commit signing.

SSH
mkdir -p ~/esp

cd ~/esp

git clone -b --recursive git@github.com:espressif/esp-idf.git
HTTPS
mkdir -p ~/esp

cd ~/esp

git clone -b --recursive https://github.com/espressif/esp-idf.git

Attentive readers will immediately notice that we have explicitly checked out version [](https://github.com/espressif/esp-idf/releases/tag/) here. We advise against using a different version than the one currently used by the master branch.

In addition to the ESP-IDF source code, the framework also requires some tools such as the Xtensa compiler or various Python packages. To set these up, Espressif provides an installation script that must be executed.

Fish
cd ~/esp/esp-idf

./install.fish esp32s3
Bash
cd ~/esp/esp-idf

./install.sh esp32s3

This script is available in different shell flavors. Please be careful to run the correct one.

Note: If you're not sure which shell you're running you might execute this cute snippet here.
sh -c 'ps -p $$ -o ppid=' | xargs -I'{}' readlink -f '/proc/{}/exe'

It prints the path to the executable of the shell from which it was executed.

Flutter (optional)

It is optionally possible to build the Frontend from source while compiling the firmware. However, this requires the installation of Flutter, which is currently only available in chaotic-aur or snap. We recommend using chaotic-aur over snap, but cannot provide a guide on how to set it up.

Arch (chaotic-aur)
sudo pacman -S --noconfirm flutter-bin
Arch (snap)
sudo pacman -S --noconfirm snapd

sudo systemctl enable --now snapd.socket

sudo snap install flutter --classic
Ubuntu 24.04
sudo snap install flutter --classic

VSCode (optional)

We generally recommend VSCode for development, but this step remains entirely optional. Feel free to otherwise fire up your favorite editor or IDE.

Arch
sudo pamac install visual-studio-code-bin
Ubuntu 24.04
snap install code --classic

Clone

The firmware source code is also hosted on GitHub. As before, we can use either SSH or HTTP to clone the repository. Using git clone without any additional arguments will clone the latest version of the master branch to the current working directory. After that, we can change into the Firmware directory we've just created.

SSH
git clone git@github.com:OpenRemise/Firmware.git

cd Firmware
HTTPS
git clone https://github.com/OpenRemise/Firmware.git

cd Firmware

Build

Before we can start building the actual firmware, we need to add the ESP-IDF framework and its tools to the PATH environment variable. For this, there is again a script available that needs to be sourced.

Fish
. esp-idf.fish
Bash
. esp-idf.sh

Warning: Please make sure that you do not just execute the script! The script must be sourced, the leading dot and the space are important. Furthermore, the variables that this script creates are only valid for the current terminal session. Each time a new session is opened, the script must be re-run.

If everything has worked up to this point, Espressif will, after a series of debug outputs, reward you with the following message.

Done! You can now compile ESP-IDF projects.

We'll try this out right away by starting one of three CMake build configurations. These configurations differ in the compiler's optimization level (Debug corresponds to -Og, Release corresponds to -Os), and above all in whether the USB peripheral is initialized as a built-in JTAG interface.

Debug
cmake --preset "Debug"
Debug JTAG
cmake --preset "Debug JTAG"
Release
cmake --preset "Release"

Note: In principle, the sdkconfig files can be stacked in any way, e.g. to create a Release JTAG build. For obvious reasons though, a firmware compiled with the built-in JTAG interface can no longer be used for other USB connections. See Configuration for more details.

After the CMake configure stage has been successful, we just need to run the actual build stage.

cmake --build build --parallel

The terminal output of our build tool tells us if the binary has been built successfully.

Creating esp32s3 image...
Merged 2 ELF sections
Successfully created esp32s3 image.
Generated OpenRemise/Firmware/build/Firmware.bin

Flash

In order to flash the generated binary (or actually, binaries) onto a board, we need to put the board into the bootloader. This is done by

Connecting the board to the USB-C port
Setting the BOOT jumper
Switching on the power supply

Afterwards we can use the flash command of the idf.py frontend to upload the firmware.

idf.py flash

Warning: Be careful, this command actually flashes 5 binaries at once and not just the application. This includes things like the bootloader, the partition table and the NVS storage. Again, you can find further details in Configuration. If you already have a board with a running firmware and just want to flash the application, you can do this with the following command.
idf.py app-flash

Debug

The idf.py frontend we just saw is only one of two ways to communicate with our ESP chip over USB. The other way is the built-in JTAG interface. Espressif's own documentation has a diagram that illustrates the two possibilities well. The convenient part is that both connections are made via the same USB cable. This is made possible by the ESP32-S3 chip itself, which provides two USB channels, one for JTAG and the other for the USB terminal connection we have already used.

JTAG debugging

Before we can establish a debugging connection with GDB though, we need to grant the appropriate rights to communicate with the OpenOCD device. To do this, we download the udev rules from Espressif's OpenOCD repository to /etc/udev/rules.d.

sudo wget https://github.com/espressif/openocd-esp32/raw/master/contrib/60-openocd.rules -P /etc/udev/rules.d

With the udev rules installed, we can now start a debug session. We recommend using VSCode for this, but of course you can also do it manually via CLI.

VSCode To debug the firmware directly in VSCode, we install the GDB debugger extension Native Debug from the Visual Studio Marketplace. If you are not familiar with VSCode extensions, you can take a look at the official documentation.

Running the debugger is done by
- Connecting the board to the USB-C port
- Switching on the power supply
- Hitting F5
CLI To debug the firmware directly via the CLI we start 2x terminal sessions. In the first session we launch OpenOCD with the built-in JTAG interface configuration.
cd ~/.espressif/tools/openocd-esp32/v0.12.0-esp32-20240318/openocd-esp32

bin/openocd -f board/esp32s3-builtin.cfg

And in the second session, started in our project folder, we connect to GDB.
xtensa-esp32-elf-gdb -x gdbinit build/Firmware.elf

Test

A very useful feature of the ESP-IDF framework is the ability to run applications on the host. This allows us to write unit tests that do not need to be run directly on a board, but can be run on the host (and in the CI pipeline). The tests themselves are written with GoogleTest.

To build the tests executable we run the Tests build configuration.

cmake --preset "Tests"