Namaste people! This is a project that I worked on for one of my courses ( Real-Time Digital Signal Processing) in my bachelor's program. The project aims at making a DSP system that "listens" audio data and outputs MIDI messages of corresponding notes over UART. Arduino Nano was used for this purpose. Long story short the micro-controller does an FFT on incoming audio data and does some analysis of the peaks and sends appropriate MIDI message. Don't be bothered about the MOSFETs though because they are for some other project (which will be put up later on instructables as well) and are not required for this project. So let's get started already!!

Step 1: Components Required

We'll need the following components to build this project although many of these are generic and can be substituted with their equivalents. Also refer the circuit diagram to work out and hunt for better implementations.

Component Quantity

1. Electret Microphone. 1

2. 30 Kilo Ohm resistor. 1

3. 150 Kilo Ohm resistor. 1

4. 100 ohm resistor. 1

5. 2.2 Kilo Ohm resistors. 3

6. 10 Kilo Ohm preset pot. 1

7. 10 Kilo Ohm trimmer pot. 1

8. 47 Kilo Ohm stereo pot. 1

9. 470 Ohms resistors. 2

10. 0.01uF capacitors. 2

11. 2.2uF capacitors. 3

12. 47uF capacitors. 2

13. 1000uF capacitor. 1

14. 470uF capacitor. 1

15. 7805 voltage regulator. 1

16. Female and Male header strip. 1 each

17. Barrel Jack connector. 1

18. 12 V 1 Amp DC Adapter. 1

19. SPST switch. (Optional) 1

20. Perfboard. 1

Step 2: Technical Specifications.

Sampling frequency: 3840 samples/sec

Number of samples per FFT: 256

Frequency Resolution: 15Hz

Refresh rate: Around 15 Hz

The lower and higher scales of the musical notes are not captured correctly. Lower notes suffer from low frequency resolution where as higher frequencies suffer from low sampling rates. The arduino is already out of memory so there is no way to get better resolution. And better resolution will come at a cost of reduced refresh rate so trade-off is inevitable. Layman version of Heisenberg's uncertainty principle.

The primary difficulty is the exponential spacing between notes (As seen in the figure. Every impulse on frequency axis is a musical note). Algorithms like LFT might help but that's a bit advanced and little complicated for a device like arduino Nano.

Step 3: Circuit Diagrams.

Note: Do not be bothered by the three MOSFETs and the screw terminals in the pictures. They are not required for this project. Notice that the microphone input board is removable or as they call it Modular. A small description of the various blocks is given below.

1) The two 470 ohm resistors combine the stereo audio signal to mono audio signal. Ensure that the ground of signal in goes to virtual ground (vg in the circuit diagram) and not to the ground of the circuit.

2) The next block is a 2nd order sallen-key low pass filter which is responsible for band limiting the input signal to avoid aliasing. Since we are working with only +12v supply we bias the op-amp by making a RC voltage divider. that fools the op-amp into thinking that the supply is 6 0 -6 volts supply (dual rail) where vg is the ground reference for the op amp.

3) Then the output is low pass filtered to block DC offset of 6 volts and coupled with DC of around 0.55 volts because the ADC will be configured to use the internal 1.1 v as Vref.

Note: The pre-amplifier for the electret microphone is not the best circuit on the internet. A circuit involving op-amp would've been a better choice. We desire the frequency response to be as flat as possible. The 47 kilo ohms stereo pot is used to define the cut-off frequency which should be typically half the sampling frequency. The 10 kilo ohm preset (The small pot with white head) is used to tune the gain and the Q value of the filter. The 10 kilo ohm trimmer pot (one with a metallic tuning knob that looks like a small flat head screw) is used to set the voltage to be as close as half Vref.

Note: When you're connecting the Nano to P.C. keep the SPST switch open else closed. Keep special care of this failing to do so can harm the circuit/computer/voltage regulator or any combination of the above.

Step 4: Necessary Applications and IDEs.

For coding the Arduino Nano I went with the primitive AVR studio 5.1 because it seems to work for me. You can find the installer here.
For programming the Arduino Nano I used Xloader. Its really easy to use light weight tool to burn .hex files to Arduinos. You can get it here.
For little bonus mini project and tuning the circuit I used processing. You can get it from here although make there are major changes in every revision so you might have to fiddle with deprecated functions to make the sketch work.
FL studio or any other MIDI processing software. You can get FL studio limited access version for free from here.
Loop MIDI creates a virtual MIDI port and is detected by FL studio as if it was a MIDI device. Get a copy of the same from here.
Hairless MIDI is used to read MIDI messages from COM port and send it to loop MIDI port. It also debugs MIDI messages real-time which makes debugging convenient. Get Hairless MIDI from here.

Step 5: Relevant Codes for Everything.

I would like to thank Electronic Lifes MFG (Website Here!!) for the fixed point FFT library that I utilized in this project. The library is optimized for mega AVR family. This is the link to library files and codes that he used. I am attaching my code below. It includes the processing sketch and the AVR C code as well. Please note that this is the configuration that worked for me and I do not take any responsibility what-so-ever if you damage any thing because of these codes. Also, I had lot of issues trying to make the code work. For example, DDRD (Data Direction Register) has DDDx (x = 0-7) as bit masks instead of the conventional DDRDx (x = 0-7). Watch out for these errors while compiling. Also changing the micro-controller affects these definitions so keep an eye on this as well while dealing with compilation errors. And if you're wondering why the project folder is called DDT_Arduino_328p.rar, well let's just say that it was very dark in the evening when I started and I was lazy enough to not turn on the lights. :P

Coming to the processing sketch, I used processing 3.3.6 to write this sketch. You'll need to set the COM port number in the sketch manually. You can check the comments in the code.

If anyone can help me port the codes to Arduino IDE and latest processing version, I'd be glad and will give credits to the developers\contributors as well.

Step 6: Setting It Up.

Open the code and compile the code with #define pcvisual uncommented and #define midi_out commented.
Open xloader and browse to the directory with code, browse to the .hex file and burn it to nano by selecting appropriate board and COM port.
Open the processing sketch and run it with appropriate COM port index. If everything goes well you should be able to see a spectrum of the signal on pin A0.
Get a screw driver and turn the trimmer pot until the spectrum is flat (DC component should be close to zero). Do not input any signal to the board then. (Do not attach the microphone module).
Now use any sweep generator tool like this to give input to the board from the micro-phone and observe the spectrum.
If you do not see a sweep of frequencies, decrease the cut-off frequency by changing the 47 kilo ohm resistance. Also increase the gain by using the 10 kilo ohm preset pot. Try to obtain a flat and prominent sweep output by changing these parameters. This is the fun part (the little bonus!), play your favorite songs and enjoy their real-time spectrum. ( Watch the video)
Now compile the embedded C code again this time with #define pcvisual commented and #define midi_out uncommented.
Reload the new compiled code onto arduino Nano.
Open LoopMidi and create a new port.
Open FL studio or other MIDI interface software and make sure the loop midi port is visible in the MIDI port settings.
Open hairless MIDI with arduino connected. Select output port to be the LoopMidi port. Go to settings and set the Baud rate to be 115200. Now select the COM port corresponding to Arduino Nano and open the port.
Play some "pure" tones near the microphone and you should hear the corresponding note hit in the MIDI software as well. If there is no response try lowering up_threshold defined in the C code. If the notes are getting triggered randomly then increase the up_threshold.
Get your piano and test how fast your system is!! Best thing is that in the goldy-lock zone of notes it can comfortably detect multiple simultaneous key presses easily.

Note: When the COM port is accessed by one application it cannot be read by another. For example if Hairless MIDI ws reading COM port, Xloader would not be able to flash the board.

Step 7: Results/Videos.

That's it for now guys! Hope you like it. If you have any suggestions or improvements in the project let me know in the comment section. Peace!

Introduction: Real-time Audio to MIDI Converter.