Category Archives: Media Lab

Meemoo: Hackable Web App Framework (the thesis)

That’s it. That is all of the words. The thesis is done. If I successfully defend it on May 2nd I’ll be a MA.

Check out the awesome art that my buddy Jyri Pieniniemi designed for the project and thesis cover:

Oh no wait, I was zoomed way in. It is actually:
Talk about attention to detail. The screws symbolize hackability, which is the main theme of the research project.

words on paper

If you want to read the paper, be my guest:
Meemoo: Hackable Web App Framework, Forrest Oliphant
It is kind of lame to have a thesis about the web locked in a PDF, so I’ll make an HTML version soon.

The project:

Swing Thing: playful interaction

Matti Niinimäki and I created this installation from Michihito Mizutani’s “poetic interaction” prompt at Winter 2010 Demo Day.

swing thing poster

Two people enter a darkened room with two swings. Classical music plays softly, and a cryptic symbol is projected between the curtains. They sit on the swings, and the music get louder. They begin to swing together, and the symbols start to move. When their swinging gets out of sync, the music’s pitch bends uncomfortably. If they shift their weight correctly while swinging, the symbol assembles itself and the mystery is solved!

Source (Arduino, Pure Data, and Quartz Composer). One technical discovery that was especially useful was figuring out how to route all system audio through Pure Data.

(Crossposted to the Media Lab Helsinki Students Blog)

Slithering: trading dimensions in video

Computational photography is a wide concept with many possible interpretations and directions. I couldn’t choose one project for the course, so I made two, and showed them with my classmates at Pixelache in Augusta Gallery. One project was an extension of some earlier computational photography experiments with Flash+webcam, called PaintCam. The other was a collaboration with Timo Wright called Slithering.

We shot two dance scenes, one with Anna Mustonen and the other with Lucía Merlo, Charlotte Lovera, Elise Giordano and myself. Thanks to Lume studio for helping us set up the lights.

We shot HD video with the Canon 5D Mark III. We thought that it would be important for the dancers to have a preview image to see how different movements would affect the final image, so I made a Processing sketch that would approximate the aspect ratio of the 5D’s output.

Now... how to turn this into a self-texturewrapped 3d model?

We used Kinect for the preview, with the idea that the depth data might interesting to use somehow in the final. The slitscan video (three dimensions shuffled) ended up interesting enough that I left the depth data for future experiments. What could four shuffled dimensions look like?

This is the description that Timo came up with for our piece when we were thinking that the final product of our piece would be still images:

Slithering is a alternative dance documentation where the dancer dances and reacts with the Slithering program. The program scans from a camera a one pixel wide segment and orders these segments to become one long picture in time. In this project the dancer has to find a completely new kind of movement, if she wants to control the visual end result. It also changes the documentation of dance in time and space to now happen only in time. What is dance minus space?

Slithering single still 2

Slithering single still 3

Slithering group still 3

Slithering group still 4

To make the first still images, I wrote a Photoshop script that would take one column of pixels from each frame of video. It took ages. I noticed that changing the column variable ended up with a very different image. What would they look like animated? I managed to write my first C++ application, with the help of Cinder, to shuffle the billions of pixels from one video to an output video. The source for Redimensionator is available freely, without warranty. Here are some experiments with the software, Redimensionating some videos found on YouTube:

Redimensionator wasn’t written until after the dances were filmed. We had the still-image preview while dancing, but had no idea what it would look like in Redimensionated video form. In the future, it would be fun to choreograph a dance piece or music video with the output in mind. Putting some planning into costumes, props, and choreography could make for very interesting output.

(Cross-posted to the new Media Lab Helsinki students blog.)

Introducing PaintCam

I joined the Computational Photography course in November of last year. We were working towards presenting at Pixelache, and it was nice to have the goal of showing some work outside of our little Media Lab family here. I couldn’t decide on one project, so I took on two: Slithering and PaintCam.

PaintCam is an extension of the MegaCam app that I started last year while living in beautiful Tomahawk, NZ. Some of these webcam toys were inspired by Lomo cameras, but this one goes beyond imitating analog photographic effects. My goal was to make a single-purpose application that allows you to composite short video loops in real-time. It works like a paint application, but instead of a color chooser you get the color, image, texture and motion from the webcam.

If you would like to try PaintCam, go to I don’t have a one-click method for saving the animations yet, but if this form doesn’t scare you off you can upload the 3×4 frame image to your favorite image host site and put the image url in. Send me any nice ones that you come up with.

It was fun to see people interacting with PaintCam and the other webcam toys in the gallery setting.

Sembiki-KaleidoCam-1300027890 Sembiki-KaleidoCam-1299939176 Sembiki-ScanCam-1299933667 Sembiki-ScanCam-1299929361

More images made by Pixelache visitors.

Transmediale Berlin 2011

I took part in Transmediale Festival in Berlin last week with a few other people from Media Lab Helsinki. Here are a few of the things that I enjoyed:

Multiscreen Films

I have not seen many multiscreen film installations, so I was glad to see Reynold Reynolds’ Secrets Trilogy in installation form. One particular shot that was interesting to me was stop-motion/pixilation of a woman playing piano, which became smashed as she played. Then she began to climb into (or be devoured by) the piano. Part of the installation was a simultaneous behind-the-scenes view, with the stated goal of shattering the illusion of film. I would have have preferred the illusion to remain intact, at least for that shot. It was painful to see a piano smashed with a sledge hammer over and over. Without that view the emotion would have been much more subtle, as it was quite a beautiful image.

I went to see the Finnish film Where Is Where? knowing nothing about the film or filmmaker. The story is set in a mixture of Algeria and Finland, told in Finnish, and visually arranged in a grid of four screens. Sometimes two adjacent screens became a panoramic image, which was nice. The whole thing was quite beautifully shot, and I was hoping to meet the filmmaker, Eija-Liisa Ahtila, but she wasn’t there.

I’m working on a multiscreen live video editor/sequencer, stemming from the Interactive Cinema workshop at the beginning of the year. Seeing these films with multiple screens was a good insight into the storytelling possibilities of this format used in a linear way.

I saw a few of the “modules” presented in Cinechamber, a 360° 10-HD-screen 8.1-channel audio space. These works tended to be more abstract. I think that this system has more potential for interactive content, since there could be the possibility of moving around in the space. People sat on the floor to watch these works, but choosing your vantage point and staying there meant that you couldn’t see all of the screens, and the ceiling was open. If the works somehow encouraged movement, I think that the space would be more interesting. A planetarium/Omnimax system would be better for this kind of immersive+passive viewing.
This system would be better for a dance party, and solve the “let’s all stare at the back of the DJ’s laptop” problem 😉

More Films

I was especially taken by Ho Tzu Nyen’s self-effacing charm in talking about his films. For Earth, he prefaced the screening by saying that it would be hard to stay awake, and even encouraged the audience to sleep, because it would add to the experience. It did! The film started by panning from vignette to vignette of one person each, dead or sleeping, in a richly textured environment. I watched intently for quite a while, but eventually the pace, dark lighting, and music soothed me to sleep. When I woke up the camera had pulled back to reveal the entire tableau, which seemed like a pile of fifty people… quite a dramatic jump.

He said that there have been seven different soundtracks, and I imagine that they would give quite a different feel to the film.

As an aspiring mad scientist, I appreciated Deconstructing Dad, a personal look into Raymond Scott’s life by his son. Scott was described as the Frank Zappa of the 1930s and 40s because of the “wierdness” that he inserted into the popular consciousness through music. He went on to be a pioneer of electronic music, making machines to play, sequence, and even compose music. One thing that he seemed to regret later in his life was the secrecy in which he operated. I plan to give away everything that I make with the hope that my ideas will be fruitful and multiply and divide and become new things that I never imagined.

Check out some of Scott’s “descriptive jazz” numbers, one set to the wonderful weirdness of Betty Boop, and one with some sweet tap:

The Raymond Scott Quintette – War Dance For Wooden Indians


Himalaya Variations was a good reminder that an old-school overhead projector is more high definition than digital video, when it comes to colors and textures (and frame rate). I had seen Daito Manabe’s Face Visualizer on YouTube, and it was cool to see it as a live performance. The Braun Tube Jazz Band was fun to see as well: a performance involving drumming on pulsing TV screens and letting the electromagnetic energy flow through the body to make music.
Himalaya Variations

Open Zone

The talks that I joined made me realize that I read too many articles online… there wasn’t too much new news for me. I thought that the idea of making a film (or, ahem, “an immediated autodocumentary”) at the festival was cool, and they articulated a lot of the thoughts that were floating around:

The Future of Art from KS12.

Paper Workshop

After the festival we joined some Universität der Künste students for a couple of days to made some “paper-based electroacoustical instruments.” I overheard that UDK is an evil lair of SuperCollider, which is a programming language for making sounds. Since I have been getting comfortable with PureData, I thought that I would keep on that track, since one esoteric generative noisemaking system is probably enough for me.

I wanted to make a monotone instrument that could be used as the root note in a chording harmonizer. I made an oboe out of cardboard and a drinking straw, which sounded like a (loud) duck call. A glass bottle sounded nicer. I was able to get it working on my iPod at the last moment, and it made people laugh, so I consider it a success. I’ll make the Rjdj scene public once I polish it a bit. I’ll record a song once I learn how to play it.

It was cool to see the wide variety of instruments and sounds that came from the open prompt of “paper.”
UDK paper workshop