Processing + Arduino. LightBrush
i've been working on this project off and on (a bit more off than on) since early 2009, and most recently i've given my self a deadline of wrapping it up by the end of 2011. from the looks of it, the conclusion however as it stands will be incomplete. i'm posting it here for posterity sake as well as in hope that maybe some of the successes and failures i've had may be of interest to others.

the idea
the crux of the project was to create light drawings in the air, but using light as a medium to recreate images, displayed on screen, in the air. a device with lights on it would somehow have x and y coordinates based on it's position in the air and correlate to x and y coordinates on the screen.

the beginning
i whipped an easy pixel grabber program, which used a black & white image and sent the location's pixel value (0 or 255) to a very crude prototype using one blue led. the single color led turns on or off based on the brightness of the pixel on screen. the biggest challenge was getting coordinates of the device in space. i ended up using a wii mote (via OSC) and an IR led to tell the computer the x and y coordinates of the device.

the initial tests with the prototype were promising but it was obvious i needed to tweak it (i.e. larger light source) in order to achieve the resolution i was hoping for. i knew i would need more LEDs and contemplated a stick or maybe an led matrix (multiples of either or a combination of the two). subtle pitch and yaw movements of the "brush" caused unwanted distortion so i used plexi glass as a surface to press on and the images got better, but lacked the "magic" i was hoping for.

everyone else
then i put the whole project on hold as my wife and i uprooted and relocated to europe, and in the meantime the inevitable happened and others developed the same idea (zeitgeist?). the first being by berg for dentsu london using an ipad to draw 3D type in the air.

and i should also mention timo arnall's other work with light painting.

although very close to my idea, no one had done it exactly as i pictured it in my head. there was still hope, yet i still was busy with other things and didn't have the time to invest in pursuing the project any further. by now it was near the end of 2010, when i stumbled across the light scythe, a very elegant execution which uses a large stick clad with RGB led's to create stunning images in the air. *drat* exactly what i wanted.

light scythe

onwards and upwards
i thought of just giving up and maybe i should have, but i decided to keep trudging on, because now i knew i wanted to create a more handheld version, which could be used untethered to the computer and maybe even together with other similar devices. which lead me to where i am now.

i began a couple months ago by focusing on using an 8x8 RGB matrix and a colorduino (shield), in wanting to make it completely mobile i started by putting together an iphone app and interface using openFrameworks and it was going well, i hit a couple of snags programming in objC. those problems aside, i had an app that would work theoretically, but in practice did not, because (so far as i know) communicating with external hardware devices from an iphone requires additional licenses and permissions from apple. so in my quest to find an alternative method of communicating to the arduino from an iphone, i found arms22's soft modem, i've included the parts needed to build one for yourself, but i quickly found out that it's baud rate of 1225bps would be way too low for sending 64*3 bytes (64 leds, and 3 bytes per color) at a speed that would be responsive enough for light drawing.

soft modem hardware
2 0.47µf electrolytic capacitor (blue can)
1 47pF tantalum capacitor (yellow)
2 100k resistor
3 10k resistor
1 4.7k resistor
1 3.5mm 4 pole phono input

that was a hard pill to swallow, and i was still hopeful that i could some how pull of a completely mobile computer-less version of my light brush. i had created an overly complicated series of sending a wifi signal from the phone to my laptop (via OSC) and then converting the OSC signal to serial and sending it to the arduino. luckily kevin cannon snapped me back to reality, and suggested that if i want to go mobile that interfacing with an android device would be much easier. if you've read this far, you know the dream of making what i envision is fading, but my stubborn determination kept me moving forward.

the result
abandoning all hopes of using the iphone, i switched to processing since i can prototype much quicker with it. and in the switch i abandoned the wiimote, as communicating with it via OSC was slow and cumbersome. once again kevin suggested just using the webcam and an IR filter. since apurely computer-less operation is on ice, i created a custom case which houses the arduino, colorduino (shield), 8x8 led matrix and 4 IR leds.

light brush hardware
4 IR Leds
8x8 rgb matrix
colorduino (shield)

after hours of coding, i finally had a desktop application that allowed me test my light brush. i can send 8 bits of RGB data at once (i.e. an image or parts of an image) and track the device in real time. i recruited my friend ben to help me test the hardware and software. we set up everything and set about drawing pictures in the air...

...but it didn't work at all, like i/we had thought/hoped it would. as we tried different settings and code optimizations, it became obvious why it wouldn't work. we learned you can only draw by moving the device in one direction (which is how the light scythe and others work) and also, the 8x8 led matrix at 60mm (2.3in) is most likely too small. below you can see the results of us trying to draw a simple "2" and that it didn't work. although my experiment with light drawing has ended in failure, i've upload large chunks of code that i've written for this, such as sending massive amounts of data over serial to arduino and a tracking class, maybe somebody out there will find them useful.

i failed to draw this "2" in the air, and learned that drawing images works better when the image size corresponds with the matrix size

as with previous projects, i'm really happy with what i've learned but i'm disappointed that i wasn't able to get it to work as i had hoped. maybe i'll let it sit for another two years and come back to it.

this is the class i created for tracking blobs (i.e. IR leds) and averaging their coordinates for smooth operation, (i wrapped it in a thread so that the overall frame rate of the main app isn't affected)

here's the class i wrote for sending an image via serial to arduino (also wrapped in a thread)

this is the corresponding code for the arudino (using a colorduino) which also needs my modified version of the colorduino library

Labels: , , , ,