Update
Ok, you can now grab the MotionTracker source code (AS2 & AS3). Version 2 eventually will include the other methods for detecting and tracking motion which I mentioned. For now I have just included code for the technique used in the demo.
Download: AS3 Webcam Motion TrackingFor those of you without access to a webcam (and as an example of a practical use for this class) here is a short video demonstrating the program I wrote for the installation piece.
End of update
Webcam required to view the demo (obviously…)
I’m currently working on putting up a show, part of which will be a live generative piece, constructed from the movement of visitors in the gallery space. So of course I needed a method for not only detecting but tracking motion. I’ve seen some of Grant Skinner and Mario Klingemann’s incredible experiments in this field, the most accurate of which used a coloured object in the camera’s field of vision (such as a ball or glove) to track movement, however I wanted the live response of the artwork to be a surprise to the viewer, and so asking them to hold a coloured object would have detracted from the overall impact.
If you ever want to be amazed by the human mind, try recreating the tasks that it performs, particularly those which we take for granted; you will quickly realise the incredible amount of computation which must be happening just to, for example, detect movement and track an object, or to discern more than one object and their relative motions, velocities and distances. This isn’t an essay on Neuroscience though so I’ll save that for another time…
Anyway; I researched the concept of motion tracking, and made some notes on my own ideas. A few possible approaches emerged.
One of the most attractive ideas was to divide the screen into a grid, and average the colours within each segment at regular intervals. The difference in brightness within each segment (from one calculation to the next) would determine whether or not motion had occurred in this area of the screen. Using this data, clusters of segments with a high rate of change could be used to assume an object in motion. This approach is quite accurate, however it would require a lot of getPixel and setPixel, as well as converting these results into Hue, Saturation, Brightness objects; all of which would be quite processor heavy. A decent computer could handle this fairly easily, however I had to also consider the processing load of the actual generative work running over the top.
The method I used in the end was much lighter, and providing the parameters could be adjusted to suit the environment, it would yield enough accuracy to provide a quite responsive tracking point for the artwork to follow.
Here’s how it works
- After the camera object is set up, it is passed to a MotionTracker class I wrote, which extends the Point object. The MotionTracker class contains two BitmapData objects; one representing the current data from the webcam, the other is used to store the previous frame.
- When the update method is called, the new frame is drawn on top of the previous using the difference filter.
- The result then has a ColorMatrixFilter applied to it, increasing the contrast of the image and dropping the darker areas (with less movement) further into the background.
- A blur filter is then applied in order to further reduce noise and to form blobs from areas where more movement is occurring.
- The threshold method is then called in the resulting BitmapData, effectively causing all near black pixels to be ignored and all light pixels (where movement has been detected) to be set to a certain colour.
So that takes care of the motion detection, but what about the tracking?
- First we call the getColorBoundsRect method on the BitmapData object, which gives us a Rectangle object representing the area occupied by pixels of the colour we used when applying the threshold filter.
- If the rectangle has an area larger than a specified constant, then sufficient movement is occurring and it can be tracked (choosing to ignore small areas of movement helps to eradicate noise caused by background and other interferences).
- We then find the centre point of this rectangle, and then move the x and y position of the MotionTracker instance to these coordinates, applying some gentle easing to give more continuity to the movement.
I will post some source code once the MotionTracker class has been refined, and once I have build a nicer API so that, amongst other things, you can create your own UI for tweaking parameters at runtime; a useful feature if you have requirements such as mine where getting the most accuracy from a particular environment is important.
Download: AS3 Webcam Motion Tracking
You can also download the AS3 Motion Tracking source code from the soulwire code repository.
hi…i see your flash and enjoy…but i wana know that how i can replace circle by a movie clip
Thanks for sharing this.
I’ve been working on a little webcam experiment for a couple of months and this really helped out when I started.
You can see a demo here:
http://hostville.com.au/labs/blog/?p=10
and the site here:
http://www.nakedpuppetry.com/
cheers
-donal
Great and lightweight! I have used it in my home page. And it won an FWA Site of the Day. Thanks for your incredible work!
I’m not a programmer, but I’ve always been jealous of people that can. I read and researched online to understand this motion tracking using flash, but I lack the processing power of that part of the brain. I need help getting a movieclip to follow the tracking point.
[// myMC is a MovieClip either on your stage when you compile // with an instance name myMC or a MovieClip attached via code // from your library... function track ():Void { // Tell the MotionTracker to update itself _motionTracker.track (); // Set the position of myMC to the current tracking position myMC._x = _motionTracker.x; myMC._y = _motionTracker.y; } ]what else do I need to get this to work using your source code.
I’m a Senior Concept Designer for Massiveblack. I will be happy to do a trade if you can help me out.
I’m trying to compile the AS3 code in FlashDevelop w/ Flex SDK 3.4. I’m lots of errors:
C:\Users\Matthew\Documents\FlashDevelop\MT_dl\src\Main.as(102): col: 4 Error: Access of undefined property blurStep.
blurStep.addEventListener( Event.CHANGE, setValues );
^
C:\Users\Matthew\Documents\FlashDevelop\MT_dl\src\Main.as(103): col: 4 Error: Access of undefined property brightnessStep.
brightnessStep.addEventListener( Event.CHANGE, setValues );
^
C:\Users\Matthew\Documents\FlashDevelop\MT_dl\src\Main.as(104): col: 4 Error: Access of undefined property contrastStep.
contrastStep.addEventListener( Event.CHANGE, setValues );
^
C:\Users\Matthew\Documents\FlashDevelop\MT_dl\src\Main.as(105): col: 4 Error: Access of undefined property areaStep.
areaStep.addEventListener( Event.CHANGE, setValues );
^
C:\Users\Matthew\Documents\FlashDevelop\MT_dl\src\Main.as(145): col: 10 Error: Access of undefined property blurStep.
case blurStep :
^
C:\Users\Matthew\Documents\FlashDevelop\MT_dl\src\Main.as(146): col: 28 Error: Access of undefined property blurStep.
_motionTracker.blur = blurStep.value;
^
C:\Users\Matthew\Documents\FlashDevelop\MT_dl\src\Main.as(149): col: 10 Error: Access of undefined property brightnessStep.
case brightnessStep :
^
C:\Users\Matthew\Documents\FlashDevelop\MT_dl\src\Main.as(150): col: 55 Error: Access of undefined property brightnessStep.
_motionTracker.brightness = _matrix.brightness = brightnessStep.value;
^
C:\Users\Matthew\Documents\FlashDevelop\MT_dl\src\Main.as(153): col: 10 Error: Access of undefined property contrastStep.
case contrastStep :
^
C:\Users\Matthew\Documents\FlashDevelop\MT_dl\src\Main.as(154): col: 51 Error: Access of undefined property contrastStep.
_motionTracker.contrast = _matrix.contrast = contrastStep.value;
^
C:\Users\Matthew\Documents\FlashDevelop\MT_dl\src\Main.as(157): col: 10 Error: Access of undefined property areaStep.
case areaStep :
^
C:\Users\Matthew\Documents\FlashDevelop\MT_dl\src\Main.as(158): col: 31 Error: Access of undefined property areaStep.
_motionTracker.minArea = areaStep.value;
Can you assist me with compilation? Thanks! -Matt
Soulwire, this is a extreemly cool creation!
I’m currently in a project involving interactive shopping/display windows. This class could be quite usefull at some point. I’ll start diggin into it right away :)
When can we expect to see an updated version?
Hi Matt,
Those are references to components on the stage – meaning you need the Flash IDE to compile with (open the FLA in the demo. For example blurStep is a slider component on the timeline.
If you don’t have flash, you will need to remove references to the components in the code and set the properties on the Motion Tracker another way.
Thanks~
Wow, this is great work! Congrats and thanks for sharing!
I was just wondering if it is possible to create Actionscript code that tracks a user-chosen object in a video, something similar to what http://innovid.com does. It really intrigues me how they can place an image behind a character in the video, just like you would hang a painting on a wall. And they animate that that image as the video plays, on the path defined by the object it’s covering. And they say they’re not modifying the original video. So what kind of magic are they doing?
This is quite amazing. I recently learned how to play with motion detection with webcam on flash. I am actually really beginner at the flash actionscript and this was quite a exercise. I still have a lot to learn but yeah I am really amazed by how you created this. Keep up the good work.
Hello,
I’m stumbled upon your work through a classmate of mine. We are currently learning how to create motion capture pieces through Dan Zen’s Flash Feathers – Ostrich. With this template of ActionScript coding, I was able to create a basic piece that was able to track motion.
Looking at your piece posted above is truly inspirational. Coming from a graphic design + art background I relate a lot more to beautiful and interesting art work. Reviewing your work has reinforced my motives to create beautiful, functional art work combined with experimentation and new technology.
Thank you for creating this, it is awe-inspiring. I would love to see this piece applied to a much larger scale.
Congratz on a great piece!
Nancy