Tag Archives: media

The long-tail of tool design and transformative programming

Simon St. Laurent connects some conceptual components in Transformative Programming: Flow-based, functional, and more. He explains the connections between web services, punch cards, Unix pipes, functional programming, and flow-based programming. I have been thinking about these connections for some time, and I’m glad somebody articulated them.

After years of watching and wondering, I’m starting to see a critical mass of developers working within approaches that value loose connections. …they share an approach of applying encapsulation to transformations, rather than data.

I think that people want tools to solve problems. It is amazing to see the lengths that computer novices will go to get the wrong tools to do what they want. (I talked about this in my JSConf.eu talk this year. It isn’t on YouTube yet, so I have no idea how coherent I was.)

If we make it easier to stitch together minimal tools, then we could make our own environment for different kinds of tasks. Wire in a spreadsheet when we need tabular data, timeline when we have linear media, dataflow graph when we want to make A/V synthesis and filtering… building software should be the practice of recognizing how to stitch these components together.

More people should have this skill. My main research interest is in making this skill (really, superpower) more accessible. I want to do this for myself, but also for my parents, kids, friends, and myself as a 9-year-old.

Monolithic software suites try to solve every problem in a broad domain with a giant toolbox, and then get abused to solve problems in other domains. Photoshop isn’t a web design tool, and it isn’t a dynamic web/mobile app design tool, but it is bent to solve those problems. Toolboxes are useful things to have and understand, but every digital media challenge is different.

meemoo-illo-by-jyri-pieniniemi I think that the upcoming custom elements web standard + NoFlo is going to be a powerful combination to make tools to get stuff done. I agree with St. Laurent that making the high-level picture dataflow is an effective way to make this work. My Meemoo.org and Mozilla App Maker are two potentially-compatible concepts for building toy programs like this. NoFlo is bringing this method to general purpose JavaScript, which now includes many spheres of possibility: server, browser, audio, video, 3D.

At JSConf.eu Jason Frame demoed a prototype window manager made for slicing up a screen and stitching together tools like this, using dataflow wires to connect widgets. I imagine a system like that which can also zoom in (open up?) to make more complex logic in a NoFlo graph.

This is the long-tail of tool design. 5 billion people will come online for the first time in the next 10 years. What problems will they be interested in solving? How many of these problems will be too obscure or not profitable enough to be suitable for a company to attempt to solve?

Right now, today, we can’t see the thing, at all, that’s going to be the most important 100 years from now.
Carver Mead


St. Laurent also writes about “Humans as Transformers.” Lauren McCarthy made a project where she farmed out all decisions during some dates to strangers on the internet in real-time. This got me to imagine a “Mechanical Turk” component for NoFlo: inputs data, text for directions, and price per item processed; outputs the human-transformed data. You could run these in parallel to compare answers, and use NoFlo’s normal flow controls to handle the asynchronous nature of the component. This would be a quick and dirty way to encapsulate any programming challenge too complex or difficult to express in code.

Transmediale Berlin 2011

I took part in Transmediale Festival in Berlin last week with a few other people from Media Lab Helsinki. Here are a few of the things that I enjoyed:

Multiscreen Films

I have not seen many multiscreen film installations, so I was glad to see Reynold Reynolds’ Secrets Trilogy in installation form. One particular shot that was interesting to me was stop-motion/pixilation of a woman playing piano, which became smashed as she played. Then she began to climb into (or be devoured by) the piano. Part of the installation was a simultaneous behind-the-scenes view, with the stated goal of shattering the illusion of film. I would have have preferred the illusion to remain intact, at least for that shot. It was painful to see a piano smashed with a sledge hammer over and over. Without that view the emotion would have been much more subtle, as it was quite a beautiful image.

I went to see the Finnish film Where Is Where? knowing nothing about the film or filmmaker. The story is set in a mixture of Algeria and Finland, told in Finnish, and visually arranged in a grid of four screens. Sometimes two adjacent screens became a panoramic image, which was nice. The whole thing was quite beautifully shot, and I was hoping to meet the filmmaker, Eija-Liisa Ahtila, but she wasn’t there.

I’m working on a multiscreen live video editor/sequencer, stemming from the Interactive Cinema workshop at the beginning of the year. Seeing these films with multiple screens was a good insight into the storytelling possibilities of this format used in a linear way.

I saw a few of the “modules” presented in Cinechamber, a 360° 10-HD-screen 8.1-channel audio space. These works tended to be more abstract. I think that this system has more potential for interactive content, since there could be the possibility of moving around in the space. People sat on the floor to watch these works, but choosing your vantage point and staying there meant that you couldn’t see all of the screens, and the ceiling was open. If the works somehow encouraged movement, I think that the space would be more interesting. A planetarium/Omnimax system would be better for this kind of immersive+passive viewing.
This system would be better for a dance party, and solve the “let’s all stare at the back of the DJ’s laptop” problem 😉

More Films

I was especially taken by Ho Tzu Nyen’s self-effacing charm in talking about his films. For Earth, he prefaced the screening by saying that it would be hard to stay awake, and even encouraged the audience to sleep, because it would add to the experience. It did! The film started by panning from vignette to vignette of one person each, dead or sleeping, in a richly textured environment. I watched intently for quite a while, but eventually the pace, dark lighting, and music soothed me to sleep. When I woke up the camera had pulled back to reveal the entire tableau, which seemed like a pile of fifty people… quite a dramatic jump.

He said that there have been seven different soundtracks, and I imagine that they would give quite a different feel to the film.

As an aspiring mad scientist, I appreciated Deconstructing Dad, a personal look into Raymond Scott’s life by his son. Scott was described as the Frank Zappa of the 1930s and 40s because of the “wierdness” that he inserted into the popular consciousness through music. He went on to be a pioneer of electronic music, making machines to play, sequence, and even compose music. One thing that he seemed to regret later in his life was the secrecy in which he operated. I plan to give away everything that I make with the hope that my ideas will be fruitful and multiply and divide and become new things that I never imagined.

Check out some of Scott’s “descriptive jazz” numbers, one set to the wonderful weirdness of Betty Boop, and one with some sweet tap:

The Raymond Scott Quintette – War Dance For Wooden Indians


Himalaya Variations was a good reminder that an old-school overhead projector is more high definition than digital video, when it comes to colors and textures (and frame rate). I had seen Daito Manabe’s Face Visualizer on YouTube, and it was cool to see it as a live performance. The Braun Tube Jazz Band was fun to see as well: a performance involving drumming on pulsing TV screens and letting the electromagnetic energy flow through the body to make music.
Himalaya Variations

Open Zone

The talks that I joined made me realize that I read too many articles online… there wasn’t too much new news for me. I thought that the idea of making a film (or, ahem, “an immediated autodocumentary”) at the festival was cool, and they articulated a lot of the thoughts that were floating around:

The Future of Art from KS12.

Paper Workshop

After the festival we joined some Universität der Künste students for a couple of days to made some “paper-based electroacoustical instruments.” I overheard that UDK is an evil lair of SuperCollider, which is a programming language for making sounds. Since I have been getting comfortable with PureData, I thought that I would keep on that track, since one esoteric generative noisemaking system is probably enough for me.

I wanted to make a monotone instrument that could be used as the root note in a chording harmonizer. I made an oboe out of cardboard and a drinking straw, which sounded like a (loud) duck call. A glass bottle sounded nicer. I was able to get it working on my iPod at the last moment, and it made people laugh, so I consider it a success. I’ll make the Rjdj scene public once I polish it a bit. I’ll record a song once I learn how to play it.

It was cool to see the wide variety of instruments and sounds that came from the open prompt of “paper.”
UDK paper workshop