Introduction: Getting Started With Python for ESP8266 & ESP32

Bacground

The ESP8266 and its younger big brother ESP32 are low-cost Wi-Fi microchips with full TCP/IP stack and micro-controller capability. The ESP8266 chip first came to attention of the maker community back in 2014. Since then, the low price (<5 USD), its Wi-Fi capability, a built in flash memory of 1 or 4 MB, and a variety of available development boards, has made the ESP chip one of the most popular micro-controllers for WiFi and IoT DIY projects.

MicroPython is a lean and efficient implementation of the increasingly popular Python programming language that includes a small subset of the Python standard library and is optimised to run on microcontrollers.

The combination of these two is a very interesting option for DIY projects, both for beginners and more advanced users.

The MiPy-ESP project

Back in 2015 my first projects with ESP8266 started out with the ESP-01 chip using Arudions for running chip AT commands over serial connection. After that, over the next years I applied the Arduino core for ESP8266 for programming the chips with the C++ language. This works fine, but for a Python enthusiast, my discovery of the MicroPython implementation of Python 3 was great news.

The MiPy-ESP project is a flexible framework applying MicroPython for full-stack Python IoT projects on the ESP-family micro-controllers.

The framework is developed by the LeGarage Technical Comittee Software Developer Team (LG-TC-SWDT-01) aiming at replacing already established C++ based code for our microcontroller applications.

The project provides basic features such as

  • Network connection procedures
  • Chip access point webserver (for wifi connection and serving of chip webpages for data I/O)
  • MQTT functionalities
  • Logging/debugging
  • Microcontroller event scheduling
  • Hardware I/O routines

With one main compact code script (main.py), all with global configuration (config.py).

This mictocontroller code runs with robust maintenance of chip connections to WiFi network and MQTT brokers. Existing MicroPython modules for various hardware can be easily integrated into the system.

The MiPy-ESP framework has become the backbone of all our hobby electronics IoT projects involving ESP-family micro-controllers. It has been tested on several ESP-family boards, like the NodeMCU, Wemos and Lolin boards.

The following tutorial is a guide for how to getting started with ESP-family microcontrollers and MicroPython using the MiPy-ESP framework.

Step 1: The Wemos D1 Mini ESP8266 Board

The MiPy-ESP framework works with most ESP8266 based microcontrollers.

The Wemos D1 mini development board is based on ESP-8266EX chip. On a footprint of 2.5 x 3.5 cm, it features 4MB flash memory, 11 digital input/output pins, all pins supports interrupt, PWM, I2C, SPI, serial and 1 analog input with 3.3V maximum input, can run on 5V power, has micro USB connection and is breadboard compatible. The low price and its small size has made it my favorite ESP board.

In addition, the D1 mini pro version of the board comes with option for connecting an external antenna, increasing the connection range significantly (+100 m range). Adding to that, the board also comes with a variety of out-of-the box extension boards boards with similar compact size.

Step 2: Getting Ready for MicroPython on the ESP Chip

In this first step, you will

  • Connect the ESP board via USB to your computer
  • Install the Esptool software for flashing the chip
  • Erase chip memory
  • Flash the chip with the MicroPython firmware
  • Install Rshell for enabling command line interaction with your chip
  • Install mpy-cross (for compilation of .py files to binary)

Connecting the board to your computer via USB
Boards with a built-in USB-serial port make the UART available to your PC and is the easiest option for getting started. For boards without USB connection, an FTDI module with USB to serial can be used to connect the GPIO pins for flashing connected to the outside world, but this is not covered in this tutorial.

For MicroPython using the MiPy-ESP code, the minimum requirement for chip flash size is 1MB. There is also a special build for boards with 512kB, but this has no support for a filesystem, which MiPy-ESP depends on.

When using a USB cable, the board gets powered by your computer while connected. This also allows for programming and debugging over the serial connection. When the project code is uploaded and your project is deployed, external power is applied over the power supply pins of the board.

Installing Esptool
Information about the Esptool software can be found in at Esptool GitHub repository. If you want to use the Windows/Linux/OSX(MAC), the link above also covers that. The Python package can be installed by

pip install esptool

For Linux users, packages for Esptool are maintained for Debian and Ubuntu, and can also be installed with

sudo apt install esptool


Erasing ESP flash memory
Using Esptool, you then erase the ESP flash memory with the command

esptool.py --port /dev/ttyUSB0 erase_flash

Downloading the MicroPyton firmware
The MicroPython firmware resides in a .bin file that can be downloaded from the MicroPython website.

Current project master branch of the repo has been tested and is operational with Micropython v.1.12. In order to ensure success with the MiPY-ESP framework, download file 'esp8266-20191220-v1.12.bin' from this link and write the firmware to the chip by the command:

esptool.py --port /dev/ttyUSB0 --baud 460800 write_flash --flash_size=detect 0 esp8266-20191220-v1.12.bin


Installing Rshell
The Rshell package enables command line interaction with your MicroPython environment installed on the chip. It can be found in this link. Rshell is is a simple shell which runs on the host and uses MicroPython's raw-REPL to send python snippets to the pyboard in order to getting filesystem information, and to copy files to and from MicroPython's filesystem. REPL stands for Read Evaluate Print Loop, and is the name given to the interactive MicroPython prompt that you can access on the ESP8266. Using the REPL is by far the easiest way to test out your code and run commands. Install Rshell by the command: 

sudo pip install rshell

Installing the mpy-cross compiler
MicroPython can be applied with ascii .py files uploaded to the chip filesystem. MicroPython also defines the concept of .mpy files which is a binary container file format that holds precompiled code, and which can be imported like a normal .py module. By compiling .py files to .mpy, more RAM memory will be available for your running code - and this is needed in order to have a functioning core module of the MiPy-ESP framework.

For MiPy-ESP code deployment, a the mpy-cross MicroPython cross compiler compiles the .py scripts to .mpy prior to chip upload. Install the mpy-cross package by the instructions in this link. Alternatively, the mpy-cross command can be installed by Python pip command or run from the mpy-cross folder path if you clone the MicroPython repository from GitHub here.

You now have MicroPython and all the needed tools installed for getting started with building your first MiPy-ESP project!

Step 3: Getting Started With MiPy-ESP

In this step you will

  • Download the MyPy-ESP framework

Downloading the MiPy-ESP framework
The MiPy-ESP project can be found at GitHub in this code repository. From GitHub you can download the repository file structure, or clone it to your computer by

git clone https://github.com/aslake/mipy_esp

With the code repository installed on your computer, you now have all the code modules you need for building an out-of-the-box ESP IoT project. More details on the toolbox in the next step.

Step 4: The MiPy-ESP Framework Architecture

In this step you will

  • learn about the MiPy-ESP code workflow

MiPy-ESP code architecture

All Python framework modules are found in the /src folder of the MiPY-ESP code repository. The src/core folder contains the core modules that go into every project. The src/drivers folder has a selection of modules for various hardware to be connected to your chip. The src/utilities folder contains optional utility modules to include in your project.

The files main.py and config.py are found in the src/ folder. These are the main files to edit for building your project:

config.py :

This file is the global configuration file for your project. It has various settings, all with descriptive comments in the file.

main.py :

This is the main script for the micro-controller code loop. It contains the application-specific code in the framework. Upon chip boot, main.py runs and imports all the project-dependent modules with given inputs from the config.py file. The above flowchart shows the layout of the main.py script.

The figure above describes the workflow of main.py:

  1. Upon boot, the code attempts to connect the chip to Wi-Fi network
    Previously applied networks and their passwords (encrypted on chip) are stored in flash memory.
    Network SSIDs and their passwords can be provisioned in the file wifi.json on the format {"SSID1":"Password", "SSID":"Password2"}. The given networks in this file are stored, passwords are encrypted, and the file is deleted upon boot.
  2. If no already know networks are found, the code sets up an access point (AP) webserver
    The chip AP server SSID and password are set in the config.py file. By logging on to the chip SSID, a webpage for logon of the chip to Wi-Fi is served at 192.168.4.1.
    Detected networks are shown in a menu, or SSID can be entered manually (hidden networks) together with Wi-Fi password. Upon successful connection of the chip to Wi-Fi, the AP server shuts down and the main.py code proceeds to its next steps.
  3. In the Setup section of main.py,
    • functions for jobs and callbacks (etc. MQTT callbacks) and regular events are defined.
    • Different timed jobs for running functions are set.
    • MQTT broker client is established
  4. The code then goes into the main micro-controller loop,
    • continuously checking network and MQTT broker connections,
    • MQTT subscriptions,
    • hardware I/O
    • and scheduled jobs.
    • Upon lost network or MQTT broker connection, the code attempts re-establishment.

Step 5: Preparing Your Project Code

In this step you will

  • learn about the MiPy-ESP repository file structure
  • prepare your project code for chip upload


Repository folder structure
The figure above describes the repository folder structure and lists the current modules of the framework. Your project is stages in the src/ folder. Core MiPy-ESP framework modules reside in src/core, optional utility modules in src/utilities and hardware modules in src/drivers.

Most available MicroPython hardware libraries can go into the drivers/ folder without any modifications. All present drivers are tested with the MiPy-ESP framework. Regarding modules in the utilities/ folder, more will be added as they come to life.

Staging of project code
Your project specific code should be placed in the src/ folder. Already there, is the main.py and config.py files you can edit. Also copy the wanted project utilities from src/utilities and src/drivers to src/.

In case you would like to provision known Wi-Fi networks and passwords to the chip, add the file wifi.json to src/.

Compiling and prepare for upload
A provided Makefile can be applied for preparing files for transfer to chip by compiling .py files in /src, compiling the core modules and transfering the compiled files to a new folder named build/ by the command

make build

The files in build are ready for upload to the chip filesystem. By default, the main.py and config.py are not compiled to binary, in order to easy access them for inspection of deployed chips. The command: 

make clean

Deletes the build/ folder and its contents.

Step 6: Compiling and Uploading the Code to the Microcontroller

In this section you will

  • upload the prepared files in build/ from last section
  • start and monitor the running code

Uploading build/ files with Rshell

Upload all files in /build directory to ESP chip using Rshell. With the microcontroller connected to USB, from the build/ folder start Rshell with the command

rshell -p /dev/ttyUSB0

Then inspect the chip files (if any) by

ls /pyboard

All files on chip can be deleted by

rm /pyboard/*.*

Copy all project files in build/ to the chip:

cp *.* /pyboard

Then start the interactive Python terminal by the command

repl

You can now invoke Python commands or import modules and monitor chip serial output from the MiPy-ESP logger module.

Restart the chip by pressing the reset button, or from command line by

import main

or 

import machine

and then

machine.reset()

Depending on your logging/debug settings in the project config file, the repl will now show debugging messages from the ESP chip over the serial connection.

This should hopefully get you started.

Happy hacking!