Tag Archives: video

Design for Hackability talk at JSConf.eu

I finally summoned the courage to watch my JSConf.eu talk, and (besides the 30 ums per minute) it isn’t so bad.

I talk about:

  • software eating the world
  • metamedia / metanetwork
  • hacking the wrong tools to get things done
  • potential for ux design in coding:
  • text interfaces to gui to touchscreens invited more people to use software, how can we invite more people to hack software?
  • design not for hackability
    (in this section I talk about how consumer electronics are not hackable, but I was happy to find out how I was wrong about that in this TED talk: Vinay Venkatraman: Technology crafts for the digitally underserved.)
  • layers of abstraction:
    making it easy to dive from gui to dataflow to code and back
  • demos, and the long tail of tool design
  • artistic potential
  • future plans and NoFlo

lost in compression

I have been doing lots of video communication living abroad from my family and fellowship, and it is a challenge.

At first there was a honeymoon novelty phase, but now I feel like we are starting to get used to the rhythm of it, and annoyed by the shortcomings. It is somewhere in the uncanny valley of communication… feels a little like a face-to-face conversation, but then lag, A/V dropouts and freezes, and the lack of eye contact really drive home that this isn’t natural. Have you ever tried to sing together?

Lag seriously messes with the conversational flow, especially with multiple parties experiencing different lags from each other. With casual video chatting these don’t matter so much, but in a meeting where people are trying to make decisions it is a bit of a battle.

I hear myself rambling sometimes, rewording my point a few times, to fill in the lack of the normal subtle cues that all parties are on the same page. I catch myself looking at the camera “in the eye,” which is really backwards.

We’ll get used to it, and develop systems for explicitly communicating the subtle, subconscious things that are lost in compression. I’m noticing “sounds good to me” verbally filling in what might have been understood without words in person. Lag will decrease, A/V quality will be better, and maybe depth cameras will adjust the angle of our gaze to better approximate eye contact.

fam hangout

My daughters don’t know a world without video chatting. In 2009 I was 14300km away and my first was able to communicate with me with sign language. This wouldn’t have happened with just audio. My second didn’t meet my folks in person for 3 months, but when she did there was immediate recognition.

Some thought-provoking writing on the video-communication-centric future we might be heading towards:

Your children will know a very different way of relating to people who are not physically present. It will change the way they work, maintain friendships, relate to family members, fall in love, and experience the world. It will change their sense of self, and self-worth. It may be a boon, or it may be harmful. Most likely, it’ll be a bit of both, because after all, it’s still about people.

Alex Payne (via Henri Bergius)

Slithering: trading dimensions in video

Computational photography is a wide concept with many possible interpretations and directions. I couldn’t choose one project for the course, so I made two, and showed them with my classmates at Pixelache in Augusta Gallery. One project was an extension of some earlier computational photography experiments with Flash+webcam, called PaintCam. The other was a collaboration with Timo Wright called Slithering.

We shot two dance scenes, one with Anna Mustonen and the other with Lucía Merlo, Charlotte Lovera, Elise Giordano and myself. Thanks to Lume studio for helping us set up the lights.

We shot HD video with the Canon 5D Mark III. We thought that it would be important for the dancers to have a preview image to see how different movements would affect the final image, so I made a Processing sketch that would approximate the aspect ratio of the 5D’s output.

Now... how to turn this into a self-texturewrapped 3d model?

We used Kinect for the preview, with the idea that the depth data might interesting to use somehow in the final. The slitscan video (three dimensions shuffled) ended up interesting enough that I left the depth data for future experiments. What could four shuffled dimensions look like?

This is the description that Timo came up with for our piece when we were thinking that the final product of our piece would be still images:

Slithering is a alternative dance documentation where the dancer dances and reacts with the Slithering program. The program scans from a camera a one pixel wide segment and orders these segments to become one long picture in time. In this project the dancer has to find a completely new kind of movement, if she wants to control the visual end result. It also changes the documentation of dance in time and space to now happen only in time. What is dance minus space?

Slithering single still 2

Slithering single still 3

Slithering group still 3

Slithering group still 4

To make the first still images, I wrote a Photoshop script that would take one column of pixels from each frame of video. It took ages. I noticed that changing the column variable ended up with a very different image. What would they look like animated? I managed to write my first C++ application, with the help of Cinder, to shuffle the billions of pixels from one video to an output video. The source for Redimensionator is available freely, without warranty. Here are some experiments with the software, Redimensionating some videos found on YouTube: http://www.youtube.com/view_play_list?p=B2540182DE868E85

Redimensionator wasn’t written until after the dances were filmed. We had the still-image preview while dancing, but had no idea what it would look like in Redimensionated video form. In the future, it would be fun to choreograph a dance piece or music video with the output in mind. Putting some planning into costumes, props, and choreography could make for very interesting output.

(Cross-posted to the new Media Lab Helsinki students blog.)

Phase Five (Connection)

Stream from Google Video: Phase 5 (Connection) (Flash)
Download from archive.org: Phase_5.divx.avi (20 MB 640×480 DivX)

16mm black and white film. Concept by me. Starring Cari Moskow. Cinematography by Meg Barnhill, Cari Moskow, myself, and CJ Thompson. Produced by James Cahill.
I finished the day of filming for this short so high and giddy… measuring and moving lights all day–what fun… We got the film back from the lab and I edited this short together. I love how the full day of work and preparation boils down to 2 minutes. I put a few tiny snips of audio in there from Tom Waits. I hope he doesn’t mind.