AS3 Motion Tracking and Detection

Update

Ok, you can now grab the MotionTracker source code (AS2 & AS3). Version 2 eventually will include the other methods for detecting and tracking motion which I mentioned. For now I have just included code for the technique used in the demo.

Download: AS3 Webcam Motion Tracking

For those of you without access to a webcam (and as an example of a practical use for this class) here is a short video demonstrating the program I wrote for the installation piece.

End of update

Webcam required to view the demo (obviously…)

I’m currently working on putting up a show, part of which will be a live generative piece, constructed from the movement of visitors in the gallery space. So of course I needed a method for not only detecting but tracking motion. I’ve seen some of Grant Skinner and Mario Klingemann’s incredible experiments in this field, the most accurate of which used a coloured object in the camera’s field of vision (such as a ball or glove) to track movement, however I wanted the live response of the artwork to be a surprise to the viewer, and so asking them to hold a coloured object would have detracted from the overall impact.

If you ever want to be amazed by the human mind, try recreating the tasks that it performs, particularly those which we take for granted; you will quickly realise the incredible amount of computation which must be happening just to, for example, detect movement and track an object, or to discern more than one object and their relative motions, velocities and distances. This isn’t an essay on Neuroscience though so I’ll save that for another time…

Anyway; I researched the concept of motion tracking, and made some notes on my own ideas. A few possible approaches emerged.

One of the most attractive ideas was to divide the screen into a grid, and average the colours within each segment at regular intervals. The difference in brightness within each segment (from one calculation to the next) would determine whether or not motion had occurred in this area of the screen. Using this data, clusters of segments with a high rate of change could be used to assume an object in motion. This approach is quite accurate, however it would require a lot of getPixel and setPixel, as well as converting these results into Hue, Saturation, Brightness objects; all of which would be quite processor heavy. A decent computer could handle this fairly easily, however I had to also consider the processing load of the actual generative work running over the top.

The method I used in the end was much lighter, and providing the parameters could be adjusted to suit the environment, it would yield enough accuracy to provide a quite responsive tracking point for the artwork to follow.

Here’s how it works

  • After the camera object is set up, it is passed to a MotionTracker class I wrote, which extends the Point object. The MotionTracker class contains two BitmapData objects; one representing the current data from the webcam, the other is used to store the previous frame.
  • When the update method is called, the new frame is drawn on top of the previous using the difference filter.
  • The result then has a ColorMatrixFilter applied to it, increasing the contrast of the image and dropping the darker areas (with less movement) further into the background.
  • A blur filter is then applied in order to further reduce noise and to form blobs from areas where more movement is occurring.
  • The threshold method is then called in the resulting BitmapData, effectively causing all near black pixels to be ignored and all light pixels (where movement has been detected) to be set to a certain colour.

So that takes care of the motion detection, but what about the tracking?

  • First we call the getColorBoundsRect method on the BitmapData object, which gives us a Rectangle object representing the area occupied by pixels of the colour we used when applying the threshold filter.
  • If the rectangle has an area larger than a specified constant, then sufficient movement is occurring and it can be tracked (choosing to ignore small areas of movement helps to eradicate noise caused by background and other interferences).
  • We then find the centre point of this rectangle, and then move the x and y position of the MotionTracker instance to these coordinates, applying some gentle easing to give more continuity to the movement.

I will post some source code once the MotionTracker class has been refined, and once I have build a nicer API so that, amongst other things, you can create your own UI for tweaking parameters at runtime; a useful feature if you have requirements such as mine where getting the most accuracy from a particular environment is important.

Download: AS3 Webcam Motion Tracking

You can also download the AS3 Motion Tracking source code from the soulwire code repository.

Posted on 06 Jun 2008
277 Comments
51 Trackbacks

Meta

AS3 Webcam Motion Tracking was posted on June 6th 2008 in the category Code / Actionscript 3.0, Code, Flash, Lab, Open Source and tagged; , , , , , , .

You can Leave a comment.


Warning: file_get_contents(http://search.twitter.com/search.atom?q=from:soulwire&rpp=1) [function.file-get-contents]: failed to open stream: HTTP request failed! HTTP/1.0 401 Unauthorized in /home/soulwire/webapps/soulwire_blog/wp-content/themes/soulwire/functions.php on line 203

Discussion

251 Responses to AS3 Webcam Motion Tracking

1 19 20 21

Leave a Reply

Pingbacks / Trackbacks

  1. 5 years ago dbis interactive limited - dbis interactive communication blog » Motion Tracking in Flash

    [...] is another reference of Flash based motion tracking. This entry is filed under Uncategorized. You can follow any [...]

  2. 5 years ago AS3 Colour class for extracting the color palette from a BitmapData image or photo, ColourUtils.as

    [...] AS3 webcam Motion Tracking 12% similar Tracking an object’s movement [...]

  3. 5 years ago Week 7: Loading External Content, Video and Sound Playback, Microphone and Camera - Advanced Flash: Actionscript

    [...] The Camera class is used to playback video that is pulled in from a web camera attached to your computer. The video can be used to build streaming web cam applications as well as computer vision projects. [...]

  4. 5 years ago Mindfock - Basic (twitchy) Motion Tracking

    [...] Most of the concepts are explained in detail here and there. [...]

  5. 4 years ago Technical Overview « iDAT 203 - Negotiated Project

    [...] The other day I found someone who created a tracking system entirely in flash (follow the link) I thought this was pretty cool, it uses flash and papervision 3D to produce the graphics. [...]

  6. 4 years ago SOULWIRE! «

    [...] http://blog.soulwire.co.uk Posted by joshshard Filed in Code, Motion Tracking No Comments » [...]

  7. 4 years ago [Actionscript 3] Motion-Tracking - FlepStudio forum

    [...] Ciao, ho trovato il file as, in as3 e as2 per il motion Tracking a questo indirizzo: Flash AS3 Webcam Motion Tracking | Detecting and tracking webcam movement prova a darci un’occhiata….. volevo chiederti, ma il movimento

  8. 4 years ago REDBLIND » Blog Archive » An Intro to Color/Motion tracking

    [...] Tracking http://blog.soulwire.co.uk/2008/06/06/webcam-motion-detection-tracking/ This is motion tracking it it’s simplest form. This is the post I started on.  Ultimately, [...]

  9. 4 years ago .net Magazine关于Jam个人网站www.jpeg.cn访谈 at With Love And Joy

    [...] Soulwire Motion Detection [...]

  10. 4 years ago 01Media Blog β1.5 » .net Magazine关于Jam个人网站www.jpeg.cn访谈

    [...] 3) Soulwire Motion Detection 虽然功能简单不太精确,但由于消耗CPU较低且数据量小,成为了我追踪脸部运动的最终选择。 [...]

  11. 4 years ago SHOWGAME

    [...] 3) Soulwire Motion Detection 虽然功能简单不太精确,但由于消耗CPU较低且数据量小,成为了我追踪脸部运动的最终选择。 [...]

  12. 4 years ago Catherine Grevet » Blog Archive » Motion Tracking

    [...] http://blog.soulwire.co.uk/flash/actionscript-3/webcam-motion-detection-tracking/ We ended up using this for our final prototype and it worked like a charm. [...]

  13. 4 years ago flairjax » Apple ISlate insider info.

    [...] on the screen. This is similar to how flash is able to do this by recognizing the bitmapdata (seen here), as well as incorporating the touch sensing. So as the Apple invite shows you can place a cut of [...]

  14. 4 years ago Tutorial Flash – Webcam Motion Tracking | Fedeweb

    [...] la risposta: AS3 Webcam Motion Tracking, articolo di soulwire, dove troverai i sorgenti da scaricare e una spiegazione di come funziona il [...]

  15. 4 years ago Pitching the Microphone with Flash Player 10.1 Beta

    [...] Looks like he’s really getting somewhere from those example videos. I guess one approach might be to render the sound data samples to bitmaps and run the sort of blend, colour threshold comparisons that you achieve with Flash WebCam motion detection. [...]

  16. 3 years ago Soulwire « MovebyNature

    [...] http://blog.soulwire.co.uk/code/actionscript-3/webcam-motion-detection-tracking ▶ No Responses /* 0) { jQuery('#comments').show('', change_location()); jQuery('#showcomments a .closed').css('display', 'none'); jQuery('#showcomments a .open').css('display', 'inline'); return true; } else { jQuery('#comments').hide(''); jQuery('#showcomments a .closed').css('display', 'inline'); jQuery('#showcomments a .open').css('display', 'none'); return false; } } jQuery('#showcomments a').click(function(){ if(jQuery('#comments').css('display') == 'none') { self.location.href = '#comments'; check_location(); } else { check_location('hide'); } }); function change_location() { self.location.href = '#comments'; } }); /* ]]> */ Click here to cancel reply. [...]

  17. 3 years ago Pixel Expander Project » Plus d’info sur la détection de mouvement

    [...] Autre source Classé dans : Uncategorized — Gauth @ 17/05/2010 Comments (0) [...]

  18. 3 years ago AnthemGDA » Blog Archive » 15 Detailed Expert Tutorials for ActionScript 3

    [...] 3. AS3 Webcam Motion Tracking [...]

  19. 3 years ago AS3 Webcam Motion Tracking « That the way Kiwi can do

    [...] blog.soulwire.co.uk Categories: Actionscript 3 Comments (0) Trackbacks (0) Leave a comment [...]

  20. 3 years ago code review:: : « summer placement::2010/dublin

    [...] we import from global class files folder. MotionTracker and ColorMatrix are class files i found on this amazing website. if i would open last 3 class files, they would appear as longs sheets of code. i [...]

  21. 3 years ago Webcam-Controlled Games - Pahgawks Animations

    [...] Wanting to try something like this on your own? Well, basically what it does is it stores the webcam’s previous image, and then uses Actionscript 3 to apply it on top of the webcam’s current image with a “difference” filter. That way, the only part left that you see is the part of the image that changed. With that, you can determine the approximate location of where the person in front of the camera is (because nothing else in the picture moves.) From there you can move other things around the screen in relation to that area or, alternatively, you can adjust other variables (you could make sounds play based on the amount of movement, move the mouse cursor based on the location, or even use it to control external hardware.) Some websites you could look at for info: http://www.spikything.com/blog/index.php/2007/10/09/webcam-motion-detection-in-as3/ http://blog.soulwire.co.uk/code/actionscript-3/webcam-motion-detection-tracking [...]

  22. 3 years ago [Flash CS3] - Bildflackern - Flashforum

    [...] [...]

  23. 3 years ago NETLab Toolkit » Motion Tracking & HubFeeds

    [...] to the SoulWire blog for the code that this is based [...]

  24. 3 years ago 2.3 – Detecção de movimento em um ponto específico – outra abordagem | Tutoriais de desenvolvimento de jogos

    [...] classe disponível na internet e bastante utilizada para essa tarefa é a Motion Tracker. Inicialmente irei mostrar como utilizar essa classe e em seguida explicar como ela [...]

  25. 3 years ago My First (Crude) Multitouch Table « RKDspot

    [...] code depends heavily on this awesome library/example from soulwire, which is a website you should check every single day (and pour through the archives [...]

  26. 2 years ago motion tracking met webcam | Interactieve Installaties

    [...] motion tracking dmv een simpele webcam, kant en klare code hiervoor met een goede uitleg hoe je hier in kunt wijzigen [...]

  27. 2 years ago A supercool night with MOTION TRACKING | Kushagra's Blog

    [...] The motion tracking algorithm is itself very basic but gives amazing results with little tweaking. One restriction though, you should have only one object moving on the screen (hand, head etc) to properly get it tracked to play the game. More about the API can be read here. [...]

  28. 2 years ago Motion Detection & HYPE Framework | Brondbjerg Design & Development Blog

    [...] around for webcam motion detection lead me to Justin Windle‘s excellent post & code [...]

  29. 2 years ago Flash Daily

    [...] Motion Tracking: http://blog.soulwire.co.uk/code/actionscript-3/webcam-motion-detection-tracking Kategorie: Flash Platform. Autor: admin. Dodaj zakładkę do bezpośredniego [...]

  30. 2 years ago Flash Daily

    [...] Motion Tracking: http://blog.soulwire.co.uk/code/actionscript-3/webcam-motion-detection-tracking Kategorie: Bez kategorii. Autor: daily. Dodaj zakładkę do bezpośredniego [...]

  1. Le Vinh 2 years ago

    Thank you so much. This is useful for a lot of idea using Flash

    Reply to this comment

  2. amber 2 years ago

    hey, i’m really interested in making my own motion tracking camera.
    So can you help me with it??

    Reply to this comment

  3. Abhinov 2 years ago

    Nice article,it seems that it would be a great apps.but can you post the code so that i can understand….

    Reply to this comment

  4. Roman 2 years ago

    Super thing u do! Thanks

    Reply to this comment

  5. CK 2 years ago

    Hi,

    Is it possible to set up webcam to track motion to play certain music track in the background ?

    For example, having 4 different instrument like electric guitar, bass, drums and keyboard and by having certain movement or the speed of the moment to play certain instrument.

    Do pardon the noob question as i’m a print designer trying to score a freelance project involving motion tracking with sound trigger.

    Let me know if you’re know something, we def can speak more on payment if we require to collaborate on this project.

    Reply to this comment

  6. CK 2 years ago

    Is it possible to program it in a way that your movement will affect background ambient music ?

    Thanks !
    CK

    Reply to this comment

1 16 17 18