Category Archives: NoFlo

the toybox and the toolbox

Talking about design for hackability at Assembly Helsinki Summer 2014:


Video: youtu.be/ZBCFBbxtBAw
Slides: slides.com/forresto/asm-2014

The first comment was one that we have heard many times. Programmers tend to be dismissive of visual programming, or anything that claims to be for making apps without coding. Visual programming has been around since 1969 in many forms, coders still work primarily in text editors, so visual programming must not work.

Our design has value for both play and serious coding, the toybox and the toolbox. Building modular code has been satisfying for play, VJing, tinkering, and experimentation. Working in a team on ambitious projects, FBP patterns become a powerful coding standard that make it possible to see other people’s work at a high level.

The primary design goal of my work with Flowhub and Meemoo isn’t to replace code with graphs. Rather, I want to be able to “wire in” the correct metaphor for the task at hand. So, editing a curve should happen by moving points on a canvas, keyframed animation in a timeline, tabular data in a spreadsheet, and logical manipulation in code.

We’re finding flow-based programming is a good fit to model UI event listeners, data transformation pipelines, A/V signal processing, electronics sensors and actuators, and Flux-like model/view synchronization for responsive UIs. This isn’t the entirety of programming, but we’re designing Flowhub to be flexible enough to switch and wire together metaphors as needed.

Coming soon: demo of wiring a keyframed timeline widget into a dataflow graph.

The long-tail of tool design and transformative programming

Simon St. Laurent connects some conceptual components in Transformative Programming: Flow-based, functional, and more. He explains the connections between web services, punch cards, Unix pipes, functional programming, and flow-based programming. I have been thinking about these connections for some time, and I’m glad somebody articulated them.

After years of watching and wondering, I’m starting to see a critical mass of developers working within approaches that value loose connections. …they share an approach of applying encapsulation to transformations, rather than data.

I think that people want tools to solve problems. It is amazing to see the lengths that computer novices will go to get the wrong tools to do what they want. (I talked about this in my JSConf.eu talk this year. It isn’t on YouTube yet, so I have no idea how coherent I was.)

If we make it easier to stitch together minimal tools, then we could make our own environment for different kinds of tasks. Wire in a spreadsheet when we need tabular data, timeline when we have linear media, dataflow graph when we want to make A/V synthesis and filtering… building software should be the practice of recognizing how to stitch these components together.

More people should have this skill. My main research interest is in making this skill (really, superpower) more accessible. I want to do this for myself, but also for my parents, kids, friends, and myself as a 9-year-old.

Monolithic software suites try to solve every problem in a broad domain with a giant toolbox, and then get abused to solve problems in other domains. Photoshop isn’t a web design tool, and it isn’t a dynamic web/mobile app design tool, but it is bent to solve those problems. Toolboxes are useful things to have and understand, but every digital media challenge is different.

meemoo-illo-by-jyri-pieniniemi I think that the upcoming custom elements web standard + NoFlo is going to be a powerful combination to make tools to get stuff done. I agree with St. Laurent that making the high-level picture dataflow is an effective way to make this work. My Meemoo.org and Mozilla App Maker are two potentially-compatible concepts for building toy programs like this. NoFlo is bringing this method to general purpose JavaScript, which now includes many spheres of possibility: server, browser, audio, video, 3D.

At JSConf.eu Jason Frame demoed a prototype window manager made for slicing up a screen and stitching together tools like this, using dataflow wires to connect widgets. I imagine a system like that which can also zoom in (open up?) to make more complex logic in a NoFlo graph.

This is the long-tail of tool design. 5 billion people will come online for the first time in the next 10 years. What problems will they be interested in solving? How many of these problems will be too obscure or not profitable enough to be suitable for a company to attempt to solve?

Right now, today, we can’t see the thing, at all, that’s going to be the most important 100 years from now.
Carver Mead

***

St. Laurent also writes about “Humans as Transformers.” Lauren McCarthy made a project where she farmed out all decisions during some dates to strangers on the internet in real-time. This got me to imagine a “Mechanical Turk” component for NoFlo: inputs data, text for directions, and price per item processed; outputs the human-transformed data. You could run these in parallel to compare answers, and use NoFlo’s normal flow controls to handle the asynchronous nature of the component. This would be a quick and dirty way to encapsulate any programming challenge too complex or difficult to express in code.

NoFlo clock demo

I have been working on the NoFlo graph editor, building on my experience making Meemoo.

noflo clock demo
press play on the iframe to start the clock

This graph runs the analog clock in the corner. We’re discussing a few ways that the graph might be simpler. For example, most of the components could be encapsulated into one subgraph, which would output just the numbers and rotation percentages needed to draw the clock.

Also, we’re talking about allowing equations like “=x/(60*1000)” in input ports that expect numbers. When the date is x, that equation will give you the correct rotation of the second hand (60 seconds * 1000 milliseconds). This will be a powerful feature, and reduce the number of nodes in the graph.

If you have ideas for the graph editor or future demos, please chime in.