Arduino

The Arduino input and output pins and programming are what enable the dancensor to be able to accept incoming raw data from the arrays of buttons and sensors into useable data that can be accessed by the Linux OS and Pure Data. Arduino is a small micro controller board with onboard General Purpose Input Output (GPIO) pins and onboard programming that can be accessed through an Integrated Development Environment (IDE ) via USB. The library I am using on the Arduino is Standard Firmata, which is compiled to allow interaction between the Arduino micro controller and software elements.

 Linux / Udoo

The computer I am using as the brains of the arrays and on-board processing of Pure Data and transmission of wireless MIDI data is called Udoo. The Udoo computer comes equipped with a quad core 1 GHz ARM i.MX6 processor, 1 GB of DDR3 ram, a WiFi module and an on-board Arduino with a SAM3X8E ARM Cortex-M3 with 76 fully available GPIO that is used to accept incoming sensor data. The Udoo computer runs a version of Ubuntu Studio 12.04, the MIDI in Ubuntu 12.04 studio for Udoo was not working, I was able to update a module to the kernel and gain access to MIDI in the Linux OS. The Linux OS is responsible for many parts of the process I will explain in detail further but encompassing: Arduino input, the program Pure Data Extended, patching of midi between programs within the OS, wireless networking, sending and receiving wireless MIDI data, and having a Virtual Network Computing (VNC) server running that allows for remote administration. The Udoo computer is a fully functioning version of Linux with email clients, web browsers, document and office applications, media players, synthesizers and a myriad of other programming and multimedia applications that will not be used in this project but could be expanded upon in future designs.

Pure Data Extended

Pure Data Extended is an open source object oriented programming language originally written by Miller Puckette. Pure data can be used to send and receive data from sensors and MIDI directly with the ability to then program that incoming and outgoing data into usable MIDI data to be used with a MIDI capable program. Pure Data is capable of sending and receiving MIDI data wirelessly however for this project I have chosen to use QmidiNet and ipMIDI to handle the wireless data traffic. The patch I used to get data from the Arduino microcontroller into Pure Data utilizes Pduino, which is an Arduino object for Pure Data with a matching firmware for Arduino. This allows the programmer the ability to change the I/O and make changes to the way the Arduino sensors are interpreted and behave without having to load a new firmware into the Arduino. The Arduino firmware once loaded must be uploaded again and reset for any changes to take place, which can present a problem when working with performers in a live environment. Pure Data and the Pduino object solve this problem allowing for administration and changes to be made instantly while the dancensor is in use allowing for a more flexible and adaptable environment for expression. The first array sends MIDI note values in the key of C 4 major as it’s tonality. Covering one octave of notes from C 4 to C 5, with the DAW. The programming for each digital button input utilizes a debounce object to stop button presses that could be a result of the button sending multiple signals when it is close to it’s voltage threshold. The second array utilizes a series of analog and digital inputs comprising of a joystick sending X,Y, output that is spring loaded to reset to center with a digital mini arcade style button next to it used to activate the FX that the joystick is controlling. All of the analog inputs on this array utilize an autoscale object that makes all incoming data convert to a range of 0 -127. After that an average object that is set in my programming to take an average of the last 6 readings of the analog sensor to help keep a more stable number stream without this object the analog sensors can vary with small inconsistencies that will affect the output signal greatly. Next in the signal chain is a change object that blocks outgoing data unless it is different from the current number allowing for the MIDI data stream to not be slowed down by excess data being sent. There is an analog 10K Rotary potentiometer used to send MIDI control data to the DAW that is used to control which instrument is currently being controlled by the performer, this is accomplished by sending a series of MIDI control data changes set by high and low voltage value sections. When the potentiometer crosses a preset threshold the programming sends preset MIDI control values allowing the DAW (Host) to interpret those changes with customizable MIDI mapping. Lastly there is an analog linear 10K potentiometer used to control the volume of the channel the performer is currently on. The third array is a 2×2 button matrix. Two of the four buttons control the on and off of an analog sonar sensor that sends out control data relational to the physical distance between an object placed in front of it. The second two buttons are used to send MIDI control data that currently in my setup controls the starting and stopping of a loop based feature of the DAW I am using for performance. The programming for the analog sensor includes the earlier parts of code for analog inputs comprising of an autoscale and average object as well as a smoothing object used to keep numbers in a stream moving so that there are no skips. It takes the point at where the sensor was before receiving a change and where the new input sensor data is and creates a smooth line without jumping. There is also some math equations in this programming. Part of the math was found only by calibration of the distance I needed the sensor to ready and deciphering a useable value from it, the second part of the math is to remove the bottom amount of data because the sensor itself does not send a zero signal so I had to subtract the bottom ledge of input data to allow for a zero reading.

JACK

JACK Audio Connection Kit is a sound server daemon that allows for connections of audio and MIDI between applications. In the dancensor JACK is responsible for routing the MIDI data from Pure Data’s output and connecting it to the program QmidiNet to be sent out as wireless MIDI Data.

QmidiNet

QmidiNet is a MIDI network gateway via UDP/IP multicast. QmidiNet is an open source application written by Rui Nuno Capela that is designed to send MIDI information using JACK over a local area network; QmidiNet was designed for use with ipMIDI. I needed to add the Qt4 library to the Udoo computer Linux OS in order for QmidiNet to function. Qt4 is a library of code used to create GUI applications in multiple distributions.

Wireless Networking / VNC

The dancensor is connected to a Wireless Local Area Network (WLAN). The initial setup of the wireless network settings on the dancensor must be completed while the Udoo computer is plugged into either an HDMI equipped monitor or TV or connected to a Low Voltage Differential Signaling (LVDS) touch screen. The next step was to install TightVNC, which is a free open source VNC compatible remote desktop application that allows for the host computer to access full functionality of the Linux OS. You can start, stop, install, configure and modify all aspects of the Udoo computer the dancer/user is wearing. Allowing full system control in a live environment where performance based variables have the possibility of changing, in an effort to create maximum flexibility and expression for the performer.

ipMIDI

ipMIDI is used on the host DAW computer, the program allows the host DAW computer to communicate wireless MIDI with QmidiNet on the dancensor. The program acts as a MIDI interface allowing the incoming and outgoing MIDI data to be treated as though it were plugged in directly to the host DAW. ipMIDI utilizes the UDP multicast network, it provides up to 20 Ethernet MIDI in ports and 20 MIDI out ports.

 DAW / Ableton Live

The DAW I am using on the host computer is Ableton Live. The dancensor system is able to run on any DAW that can accept and output MIDI considering the host computer can also use the program ipMIDI. Other wireless options are available if ipMIDI is not a useable option on the host system you can run Pure Data on both the dancensor Linux OS and the DAW (Host) computer. I have chosen to run through QmidiNet and ipMIDI in order to make troubleshooting the MIDI easier. A system could be used without any knowledge of Pure Data giving users time to use the system and become familiar with Pure Data programming with hope of users gradually integrating their own pure data programming on the DAW (Host) computer. Some of the choices for my design to utilize Ableton Live as the DAW are Ableton’s stability and functionality in a live environment. The MIDI mapping in Ableton Live provides many options for how the DAW will handle incoming MIDI data. This allows for a large amount of control of the DAW without the need for interaction through conventional methods of keyboard and mouse interaction. The supplied looper with Ableton Live in this situation is used for a single performer using the dancensor to stack multiple loops of audio into a structured but improvisational song by changing instruments. For example can play and loop a drum track, switch instruments to a bass sample, loop a bass riff, and then switch instruments to a piano or horn and loop a melody; all without the need for interacting with the computer physically. All data is sent wirelessly through sensors connected to their body leaving the performer free to move and interact with the music they have created. With FX in the signal chain the music can be changed live to add more depth to a dance performance of music. If multiple dancensor-equipped performers are playing together they could then form an ensemble and create more dramatic music with each member taking sonic space and purpose. By keeping each performer inside the same set of Ableton Live you have control over global quantization to help the performers keep in time while still allowing freedom of self expression.

Array 1 is 8 buttons mapped to C4 through C5 in the key of C major, sending out MIDI note on, velocity and note off messages

Array 2 is comprised of a 4 digital buttons used to send MIDI control data and an analog sonar sensor for detecting distance then sending out relative MIDI data

Array 3 is comprised of an analog rotary 10K potentiometer sending MIDI control data, an analog X,Y, joystick sending MIDI control data, a linear 10k potentiometer sending MIDI control data and a digital button sending MIDI control data.