Magenta, Google’s research project that looks at how AI can help people be more creative, just released an open source experimental instrument called NSynth Super. NSynth Super utilizes Magenta’s NSynth, Google’s neural network that generates sounds, and directions for building your own NSynth Super are available on Github.
The NSynth Super is a piece of hardware that brings the NSynth technology to life. As Douglas Eck, research scientist on the Google Brain team, says in the video above, NSynth doesn’t generate notes, but rather, the actual sound of an instrument. The NSynth algorithm learns the core qualities of what makes up an individual sound and then is able to combine sounds to create something completely new.
The hardware Google has devised to play with the NSynth algorithm features a central X / Y pad where each quadrant can be assigned an instrument (sort of like using effects on a Kaoss Pad). These instruments can be mixed by gliding your finger across the pad. What’s particularly unique is that NSynth Super isn’t just layering sounds on top of each other. Rather, it’s synthesizing an entirely new sound based upon the acoustic qualities of the individual instruments. This gives some unexpected results. In the demo video above, blending a flute and a snare makes a sound that’s glassy and quasi-sharp, without any overtly “drum-like” qualities.
The NSynth Super has some parallel to the German-made Hartmann Neuron, a polyphonic synth produced from 2002 to 2005 that analyzes and creates computer models of sounds which can be controlled in various ways. Ultimately, while the idea was novel and the synth creates otherworldly noises, it was quite expensive and not entirely user friendly.
NSynth Super can be played via any MIDI source, like a DAW, sequencer, or keyboard, and features some onboard parameter shaping, like basic ADSR (attack, decay, sustain, release). Although the NSynth Super isn’t available for purchase, Google has offered directions to make one from scratch using Raspberry Pi. It even includes details on how to create the PCB (printed circuit board) and schematics for the housing. The directions won’t give you the pretty, rainbow screen seen in Google’s prototype, but you will get a cool neural instrument that generates wildly inventive sounds.