Tiny tiny beat tracking
The Pixel Operator a.k.a. PO Pixel
Together with Google and Teenage Engineering, I developed this cute little video sampler app. Exclusively available for Google’s line of Pixel phones.
Chantlings – an interactive choir
XLN XO: intelligent drum machine
XLN Audio just released their new drum machine XO, where drum sounds are laid out as a 2D map such that similar sounds are near eachother (also known as an embedding). We helped them build the space using a modified version of t-SNE, as well as some other neat things including auto-trimming, auto-gain, and more. It turned out really cool, download the demo and give it a spin!
This web app will synchronize awesome dance videos with whatever the mic on your device picks up, preferably some music. An instant VJ experience, originally created for the hackfest.lol hackathon. And we won! Twice! CLICK HERE to start the party!
Baby Talk For Dads
We did some signal processing to enable this web app that teaches dads to speak to their young kids in “baby talk”. This one was coded up in C++ and converted to js using emscripten.
The World’s Largest Sequencer
Clever Cat Collar
Catterbox from Tash and Dan on Vimeo.
Pacemaker DJ now does the work for you
Auxy – Easy music creation on iPad
CAPE – the Cut-And-Paste Engine
To be continued – maybe?!?!?
Interactive Audiovisual Experiment
This video was shot under the Liljeholmen Bridge in Stockholm, Sweden on September 8, 2012, where the piece was shown as part of John Essing‘s “The Dark Light” release party.
Brand new synth – GForce Re-Tron
We are proud to be involved in the new Mellotron simulation from GForce: Re-Tron. It is a Rack Extension for the excellent audio software Reason. We painstakingly ported the code from the VST plugin and implemented the pretty 3D interface (designed by Fredrik Hurtig), and added a bunch of new features, making it an even excellenter instrument!
Check out this beatiful demo by Rob Purcelli
Then go buy the thing in the Propellerhead Online Shop
Lessons learned from the 24-hour iPhone app
The app is called Vocal Shapes and is a visualizer for monophonic audio signals such as your voice or a musical instrument. Here is a quick demo video:
Here is a breakdown of how the hours were spent:
|Audio Signal Processing||2 hours|
|OpenGL graphics programming & design||4 hours|
|Splash Screen Artwork & Animation||3 hours|
|Fine tuning of processing & graphics||3 hours|
|App Store Submission||3 hours|
|Promo Song & video||3 hours|
|This blog post & product web page||1 hour|
So, 2 hours remain, which I will probably spend making another nice demo video. Lessons I have learned so far:
- Simple apps require dramatically less development time
- Simple apps are more fun to develop
- Even the simplest apps apparently get downloaded by some people
NOW GO TO THE APP STORE AND BUY THE THING FOR $0.99!
Note: this app is no longer available, I haven’t had the time to redo it for newer phones. If I were to resurrect this, it would probably be as a web page, using WebGL.
Square Pixels are So Last Century
1. Rotatable diamonds
By using diamond shapes that are either divided horizontally or vertically, we get a really nice effect. A simple algorithm just chooses the orientation of each diamond that minimizes image distortion. Click the image for full-size svg vector art.
2. Hexagon segments
Here, equilateral triangles are used to build up hexagons. Hexagons happen to be the optimal shapes to arrange points in 2D space for vector quantization. Click the image for full-size svg vector art.
Auto-tuning for Acne Digital
I’ve been busy for two months working with the digital production company Acne Digital (only slightly related to the clothes brand). It is a campaign site for Radiotjänst*, where you become the star of a music video. Of course, no modern pop music video is complete without heavy vocal processing including Auto-Tuning, and that is what I have provided in this case.
The vocal processor is written from scratch in c++ and includes auto-tuning, filter-distortion-tempo-delay, and a multiband normalizer/compressor. Initially, I wrote a command line VST host to run 3rd party tuners (Antares, Yamaha, etc), but they turned out to sound bad for regular speech input, because they are designed for singing voice, which is more periodic and predictable. My tuner is superior because it can find the globally optimal pitch track using dynamic programming.
Go here to make your own silly music video, and click here to watch mine on the original site.
*Footnote: For our foreign readers, Radiotjänst is the agency that collects funds for Swedish public service media. In this campaign, you’re told you’re a star for paying the fee (basically because it’s easy to avoid paying, by just saying that you don’t own a TV set).
Vocoding for Robyn
One of my latest projects is now online: an interactive music video for Robyn. It was produced on a shoe string budget with volunteers from DinahMoe, B-Reel, North Kingdom and Blip Boutique, among others. In other words, a bit of a dream team! Jakob Nylund dreamt up the VCR-ish design, Johan and Erik at DinahMoe did the sound interaction bit. My part was generating the vocoded voice that says the name of each beat creator (under “stream”). This was done using the open source speech synth flite, the mda vocoder, and a fair amount of custom code to nice up the sound, all running on a minimal virtual fedora machine over at rackspace. Enjoy!