Tiny tiny beat tracking

Don’t let the size fool you, this thing I put together might be the best realtime beat-tracker around. Responsive yet robust thanks to power of adaptive optimal filtering.

Chantlings – an interactive choir

Over the last year I’ve collab’d with iorama to create this lovely experience. We’re using voice analysis and synthesis to create virtual singers that will follow your lead and harmonize your singing. It’s great for practicing to sing together with others. Check it out here

Baby Talk For Dads


We did some signal processing to enable this web app that teaches dads to speak to their young kids in “baby talk”. This one was coded up in C++ and converted to js using emscripten.

Clever Cat Collar

We worked with We Do to build a smart cat collar that tries to translate cat language to english. The whole thing is a bit tongue-in-cheek as you can see in the video below, but the thing actually works, even though we only built one copy. You can even see me in the promo video making cat noises…

Catterbox from Tash and Dan on Vimeo.

Pacemaker DJ now does the work for you

We developed a state-of-the-art automatic DJ system for Pacemaker so you can spend your time on more important things. Go get it for your iPhone & iPad, it’s also FREE!!!

CAPE – the Cut-And-Paste Engine

In 2009 I spent a little time developing an audio mosaic engine, which basically built a database of sound snippets from a bunch of recordings, then tries to recreate another recording using only those snippets. As I’m now getting back into automatic remixing and such, I though it was time to revisit this thing. Here’s a demo video:

To be continued – maybe?!?!?

Interactive Audiovisual Experiment

Here is the latest iteration of my as yet untitled interactive art piece. A sensor reacts to body movements, which are translated to colors in software and projected onto the bodies in real time. Future versions will also react to incoming sound. Stay tuned, and contact me for more information on this piece.

This video was shot under the Liljeholmen Bridge in Stockholm, Sweden on September 8, 2012, where the piece was shown as part of John Essing‘s “The Dark Light” release party.

Merry Christmas!

This year I’ve made a “conscious” x-mas song for you all to enjoy. It is a dwelling on one of the more moronic topics in the 1984 hit Do They Know It’s Christmas by Band-Aid. Listen, take heed, and become enlightened!

Lessons learned from the 24-hour iPhone app

As an experiment, I have beset myself upon producing a finished iPhone-app with only 24 hours of work, including everything from idea musing to writing this blog post. To save my own sanity and health, these 24 hours were spread over a 2-week period (including the 5-day App Store review process).

The app is called Vocal Shapes and is a visualizer for monophonic audio signals such as your voice or a musical instrument. Here is a quick demo video:

Here is a breakdown of how the hours were spent:

Idea 1 hour
Audio Signal Processing 2 hours
OpenGL graphics programming & design 4 hours
Splash Screen Artwork & Animation 3 hours
Fine tuning of processing & graphics 3 hours
Testing 2 hours
App Store Submission 3 hours
Promo Song & video 3 hours
This blog post & product web page 1 hour
Sum 22 hours

 

So, 2 hours remain, which I will probably spend making another nice demo video. Lessons I have learned so far:

  • Simple apps require dramatically less development time
  • Simple apps are more fun to develop
  • Even the simplest apps apparently get downloaded by some people

NOW GO TO THE APP STORE AND BUY THE THING FOR $0.99!


Note: this app is no longer available, I haven’t had the time to redo it for newer phones. If I were to resurrect this, it would probably be as a web page, using WebGL.
 

Square Pixels are So Last Century

I like mosaic-ized versions of images, but the standard photoshop pixelize (or half-tone for that matter) is done to death. So I asked myself, what other geometric shapes can be used to build up images? I came up with two fairly simple ones that give the mosaic an arty and luxurious feel.

1. Rotatable diamonds
By using diamond shapes that are either divided horizontally or vertically, we get a really nice effect. A simple algorithm just chooses the orientation of each diamond that minimizes image distortion. Click the image for full-size svg vector art.
Debbie Harry image encoded with diamond mosaic 		2. Hexagon segments
Here, equilateral triangles are used to build up hexagons. Hexagons happen to be the optimal shapes to arrange points in 2D space for vector quantization. Click the image for full-size svg vector art.
Debbie Harry image encoded with hexagon segments

Auto-tuning for Acne Digital

I’ve been busy for two months working with the digital production company Acne Digital (only slightly related to the clothes brand). It is a campaign site for Radiotjänst*, where you become the star of a music video. Of course, no modern pop music video is complete without heavy vocal processing including Auto-Tuning, and that is what I have provided in this case.

The vocal processor is written from scratch in c++ and includes auto-tuning, filter-distortion-tempo-delay, and a multiband normalizer/compressor. Initially, I wrote a command line VST host to run 3rd party tuners (Antares, Yamaha, etc), but they turned out to sound bad for regular speech input, because they are designed for singing voice, which is more periodic and predictable. My tuner is superior because it can find the globally optimal pitch track using dynamic programming.

Go here to make your own silly music video, and click here to watch mine on the original site.

*Footnote: For our foreign readers, Radiotjänst is the agency that collects funds for Swedish public service media. In this campaign, you’re told you’re a star for paying the fee (basically because it’s easy to avoid paying, by just saying that you don’t own a TV set).

Vocoding for Robyn

"we dance to the beat" interactive beat machine

One of my latest projects is now online: an interactive music video for Robyn. It was produced on a shoe string budget with volunteers from DinahMoe, B-Reel, North Kingdom and Blip Boutique, among others. In other words, a bit of a dream team! Jakob Nylund dreamt up the VCR-ish design, Johan and Erik at DinahMoe did the sound interaction bit. My part was generating the vocoded voice that says the name of each beat creator (under “stream”). This was done using the open source speech synth flite, the mda vocoder, and a fair amount of custom code to nice up the sound, all running on a minimal virtual fedora machine over at rackspace. Enjoy!