Swap your face into pictures (2012)

Using a depth camera, it extracts your face and pastes it on top of all the faces in the pictures, in real-time. If the tracking is lost, the music stops. Nowadays using deep learning this can look perfect, but at the time they were just getting started…

Beer Ninja (2012)

Hand tracking was hard back in the day. During CES when testing the algorithm, we found out that it actually tracks beer cans as well. Fruit Ninja with a beer? Why not.

iPhone as controller for an RC Helicopter

Before drones there were helicopters!
I didn’t like the original remote controller for the RC helicopter I got as a gift. I wanted to fly the thing with my hand! So I did a quick hack of the original controller using an Arduino, a digital potentiometer, Processing and an iPhone. You hold the iPhone in your hand, the pitch and yaw correspond to your hand’s pitch and yaw while a simple slider on the iPhone screen controls the height. I wish the cat in the background could be controlled that easily…

Streetside Diary

We have many thoughts and see many things while driving but it’s difficult to remember it all later because we are busy driving (at least until autonomous cars arrive). So what if you could make verbal, deictic and environmentally-contextual notes and collect all these things in a sort of a “diary” for later?

So I built a prototype that:

• picks up driver’s pointing gesture around the steering wheel using an IR camera
• embeds the Google Street View of the correct side of the road as a visual reference
• records the address, nearby POIs
• generates relevant tags for you (e.g. location, time, date, audio, notes etc…)
• records and transcribes your audio message
• stores everything as an entry in your private “diary”

Music tracks in Chuck

Some simple tracks I programmed in Chuck. Yes, it turns out you can program music, you just have to write your own synthesizer and sequencer heh :-).

Realtime motion capture animation in C++ (2005)

Each jellyfish is controlled by a user holding a 6 DOF electromagnetic tracker. The animation is actually a C++ program that dynamically reads the tracking data and synchronizes it to the music. There is also a flock of fish following each jellyfish according to a simple flocking algorithm.

Haunted Windows desktop (2012)

I wanted to see how interactive can you make the Windows desktop with Windows API… turns out you can do quite a bit! Here the icons chase your cursor.