Sometimes it’s good to break things…

During a recent project I needed to find a way of simulating digital interference on an image / video stream. At first, it seemed the best approach might be to use the graphics API or Bitmap effects, but why imitate when you can have the real thing.

The principle is incredibly simple. Corrupt / alter a byte (or several bytes) in the ByteArray of a JPEG and then load this back into a DisplayObject using Loader.loadBytes(). Because of the way in which JPEGs work, the image will still display but the corrupt bytes will mangle the output to varying degrees, depending on how many bytes have been corrupted. The result is an interesting glitch pattern, a bit like those which were accidentally created when my webcam drivers lost the plot.

It is also possible to corrupt other image formats (png, gif etc) in the same way, by first passing the image ByteArray through a JPEGEncoder.

I’ve implemented this technique in the Glitchmap class, which you can download and play with at the bottom of this post. I hope that it will provide a few moments of fun, if not a practical use (I’m thinking image gallery transitions or music visualisations).

The small gottcha is that it isn’t desirable to mess with the JPEG headers, so when the clean ByteArray is passed to the Glitchmap class, it will determine the length of the header by reading through the bytes until it finds the SOS (start of scan) declaration (0xFFDA). The next two bytes represent the length of the SOS, so it skips forward by this amount – the resulting position in the ByteArray should be the end of the header, and so between this point and the EOI (end of image) the data corruption takes place. I’m not sure whether this is the best way of determining the JPEG header size; perhaps any CS graduates / students out there can help me out?

After I posted the webcam glitch images, Jon used them in his title sequences for the 48 hour film gala. If you find a similar creative use for this script or the glitch generator then please let us all know.

I think the next step is to experiment with merging the bytes of multiple glitched images in order to create transition effects. I may also build an AIR app and see what the best way of outputting video might be (using NativeProcess for screen capture perhaps?).

You can also download the Glitchmap Class and the Glitch Generator source code from the soulwire code repository.

Related posts:

1. Webcam Glitch Art
2. BitmapData Colour Palette
3. AS3 Webcam Motion Tracking

I’ve been revisiting one of my favourite topics lately – recursion – and have been getting together a bunch of sketches for a big personal project I’m working on (more on that another time). Anyway, I inadvertently stumbled across quite a nice way of making things grow in a convincing manner, and whilst this script doesn’t actually use recursion ‘per se’, I thought it was kind of cool so I ran with it.

It also presented a good opportunity to try out another Flash Player 10 feature that I’ve been eager to try, which is using Pixel Bender and ShaderJob. Although I’ve used Pixel Bender before, I’ve never used it in this way.

Basically, the idea is to exploit Pixel Bender’s performance and ask it to do some of the heavier number crunching for you – a concept that is currently being used by the Papervision3D team to give some juice to the next release. The other great thing about this approach is that these computations can run in another thread, leaving you more resources for other tasks (like rendering a fishing boat load of tentacles)

So, although this isn’t the best example of using Pixel Bender’s capabilities, it was a nice opportunity to exercise the idea. Each Tentacle starts a new ShaderJob and Pixel Bender whips up some tasty trigonometry for me, before passing it back inside a Vector object (typed Array). From there it’s just a question of making those numbers look nice… Kind of reminds me of the illustrative style used for the O’Reilly book covers…

Related posts:

1. Perlin Noise Flow Field
2. Double Sided 3D Plane in FP10
3. 2D Cellular Automata

We’ve all been there. You’re building an image gallery and you need to create consistently sized thumbnails from a set of images, of various sizes and orientations and with differing aspect ratios – or you need a fullscreen background which always fills the stage, regardless of a user’s screen resolution and browser window size – or maybe you just need to fit a DisplayObject into a rectangular area whilst maintaining the correct proportions of the original image.

Introducing DisplayUtils

Since this is a common task, one that I find myself coding over and over, I’ve whipped up a little class called DisplayUtils which will help with the scenarios mentioned above and more.

The main method in DisplayUtils is FitIntoRect. FitIntoRect can work in several ways. You can use it to resize a DisplayObject to fit inside a rectangle, or you can use it to generate a transformation matrix, which can be applied to an object or used with the BitmapData.draw method to create thumbnail previews, or both. There is also another method called createThumb, which uses fitIntoRect to quickly and easily create nicely cropped and scaled thumbnails from a source image with any aspect ratio.

Have a play with the demo above to see what you can do with these methods.

Using DisplayUtils and fitIntoRect

There are several parameters to control how fitIntoRect works. Here’s what you can pass to the method:

• displayObject : The DisplayObject to resize
• rectangle : A Rectangle object representing the space the DisplayObject should fit into.
• fillRect : Whether the DisplayObject should be transformed to fill the entire rectangle, or whether it should fit completely within it.
• align : The alignment of the DisplayObject within the rectangle, for example Top Left or Middle.
• applyTransform : Whether the calculated transformation matrix should be applied to the DisplayObject within the fitIntoRect method. If false, the DisplayObject will not be modified, but the Matrix returned by the fitIntoRect method can be used to transform objects or modify the draw command on a BitmapData object.

The download includes all necessary classes as well as several examples of how to use the fitIntoRect method. Here are some code snippets for getting started with…

Example Usage

Fit a DisplayObject into a Rectangle

var area:Rectangle = new Rectangle( 0, 0, 200, 100 );
DisplayUtils.fitIntoRect( myDisplayObject, area, true, Alignment.MIDDLE );

Create a fullscreen background image

stage.align = StageAlign.TOP_LEFT;
stage.scaleMode = StageScaleMode.NO_SCALE;

function onStageResized( event:Event = null ):void
{
	var area:Rectangle = new Rectangle( 0, 0, stage.stageWidth, stage.stageHeight );
	DisplayUtils.fitIntoRect( myDisplayObject, area, true, Alignment.MIDDLE );
}

stage.addEventListener( Event.RESIZE, onStageResized );
onStageResized();

Create a thumbnail

var thumb:Bitmap = DisplayUtils.createThumb( myBitmap.bitmapData, 100, 100, Alignment.MIDDLE, true );
addChild( thumb );

Download

You can download the DisplayUtils class, as well as the Alignment class, which simply contains enumeration for the various alignment options when using fitIntoRect and createThumb.

A example of some of the uses discussed above is also available in the zip.

Related posts:

1. AS3 Webcam Motion Tracking
2. Double Sided 3D Plane in FP10
3. Flipping you the Boid

I’ve uploaded the Boid source code. Sorry for the delay. I’ve also included some basic examples of how you can create nice behaviors using the Boid class. If you set Main.as as your document class then you’ll see that within that you can specify the demo to run when you compile.

The basic demos included are:

• Chase – Boids chase each other
• Flock – Boids flock together with some wander
• Graphics – How to use custom graphics with a Boid instance
• Seek – Boids seek the mouse position
• Wander – You guessed it, they wander

I’ve commented all of the methods and properties and there is also documentation (open index.html in the docs folder) to get you started. Before you mention it, yes, there is some redundant code ;) (such as the constrainToRect method which doesn’t really work) but I need to do some optimisations anyway so I just left it as is for now, rough and ready.

Download: AS3 Boid Class & Demos

It’s been a little while since I posted! I’m in the process of moving house and office, plus I’ve been repeatedly slammed at work. Apologies to those of you who’ve left comments and haven’t got much by way of reply. I’ll hopefully be visible again over the next few weeks.

I actually coded this experiment back in November, but only recently decided that enough was enough and hijacked an evening so I could tidy up the code and build a UI for the demo.

Since coming across Craig Reynold’s paper; “Steering Behaviors for Autonomous Characters” whilst I was researching complexity theory for a university project (it wasn’t as academic as it sounds!), I’ve found myself becoming fascinated by flocking and swarm beahviors and the notion that you can create complex systems from relatively simple components. Much like the Cellular Automata, you can program an agent to have only a limited set of abilities and awareness, define a set rules for it to follow, and then create a multitude of instances of the agent to see what patterns begin to emerge through their interactions.

Flocking is a great example of this, as Reynolds demonstrated. His agents, or “Boids”, could perform certain maneuvers, such as seeking a target, fleeing from a predator, avoiding obstacles or wandering randomly. Combine these behaviors though, and allow each Boid to consider it’s neighbors when deciding on its locomotion, and complex behaviors begin to emerge.

The most infamous of these combinations is that of separation, cohesion and alignment, as it can produce a swarm behavior reminiscent of flocking birds or shoals of fish.

Separation means that each Boid attempts to maintain a certain distance from its immediate neighbors, Cohesion ensures that each Boid attempts to stay close to the centre of the immediate flock and Alignment will calculate the average heading of the flock and steer towards this vector.

Each of these behaviors is fairly simple in isolation, but the results can give the impression that some significant AI is at work, when all that’s really going on is some simple vector math.

Robert Hodgin has done some amazing work with flocking behaviors, so too has Psyop, and of course Reynolds is the daddy of all this stuff. There are some good libraries out there too, such as OpenSteer, and Keith Peters talks about this too in Advanced Actionscript Animation.

I’ve been trying to sharpen up my mathematical skills lately though, and was eager to play around with the new (new when I created this experiment anyway!) Flash Player 10 API – the 3D aspects in particular. So at the core of the Boid class are the native Vector3D, Matrix3D and PerspectiveProjection classes. Ok, so we’ve all got our own Vector3D and Matrix3D class in our library, but I thought it’d be good to use Adobe’s, especially seeing you can then apply the Matrix3D directly to a DisplayObject’s transform property.

Another cool feature I came across was the Matrix3D.pointAt() method, which will rotate a Matrix around a given axis and in this case can be used to orientate the Boid to it’s heading vector… Nice!

I’ve had a lot of fun putting together the Boid demo anyway – tweaking various settings on the individual Boids as well as the group behaviors (my personal favorite is ‘Chase’ mode with ‘Number of Boids’ cranked right up!).

If you have a play with the settings though, you’ll quickly see what I mean about the plethora of different effects that can be accomplished, even by simply adjusting the locomotive abilities of the Boids.

Due to said lack of free time as of late, I haven’t documented the code fully, though I have started and so if anyone is interested in the source code then please say so in the comments and I’ll try and finish the job (after a mere 6 months!) and put it up for download.

Related posts:

1. Perlin Noise Flow Field
2. AS3 Shuffle Grid Class
3. Double Sided 3D Plane in FP10

Just a quick (and slightly belated) announcement.

The distinguished Bandcamp has recently been regaled with another round of audio-visual snacks. I’ve had a lot of fun working on them, and there will be even more in the not too distant future.

The whole Bandcamp site in fact is bursting with more and more tasty new features which you should check out. In terms of my little contribution on the music visualistions front, there are currently four new full fat flavors to feast on…

Ribbons » which uses my Papervision3D Ribbons Class and is designed for crunching riffs and hard beats.

Strata » which was inspired by stream-graphs and rock formations, and flows like a charred pizza in a log-flume along an imaginary Teflon oven dish.

Plughole » which comprises of disembodied appendages and iconography encapsulating the zeitgeist of generation stupid – mashes the aforementioned archetypes up into crumbs before offering them to the esophagus of oblivion.

Hypno » which was designed to bring about universal harmony and spiritual equilibrium but accidentally caused the credit crunch.

You can see them in action by visiting an artist’s page (such as Hot Bitch Arsenal or Pattern), hitting play and selecting a visualisation from the pop-out menu underneath the cover art.

Related posts:

1. Bandcamp Music Visualisations
2. AS3 Music Visualisation
3. Disabed Algebra

Drag & Drop the images to shuffle them around…

A friend of mine, Dale Sattler over at No Ponies recently posted a grid sorting class. He said it was based on my old dynamic stacking menu; but that was pure modesty as his script is based on a 2D grid and much more sophisticated.

Anyway – it inspired me to create a grid sorting / shuffling algorithm of my own. It behaves differently to Dales but I think it works pretty well.

The basic logic of the algorithm works by determining how far a dragged item travels along the grid (rows and columns) and then shuffling the surrounding items in the opposite direction by these values, therefore creating space for the dragged item to slot into when released.

It currently only works for grids which are fully populated, but adding more flexibility for incomplete grids will only be a matter of adding a few conditions – functionality which I’ll add when I get a moment.

You can extend the ShuffleGridItem class to add the functionality you need. In the demo I’ve made a simple image class which grabs a photo from Flickr.

Creating a ShuffleGrid is very simple, and can be done like so:

var grid:ShuffleGrid = new ShuffleGrid(4,7,40,40,1);

for (var i : int = 0; i < grid.numCells; i++)
{
	grid.addItem (new ShuffleGridItem());
}

addChild (grid);

The constructor takes the following arguments:

new ShuffleGrid(rows, cols, cellWidth, cellHeight, spacing);

Feel free to download the class and a simple demo and have a play around. I can imagine many uses for it, so hopefully it will come in handy.

Related posts:

1. AS3 Webcam Motion Tracking
2. 2D Cellular Automata
3. Double Sided 3D Plane in FP10

I’ve made a couple of small improvements to the PaperSprite class:

1. The visible face of the plane is updated automatically only when required. This saves on processing load and also means you can just set up the PaperSprite and not have to worry about anything else, just tween the crap out of all its properties and it won’t mind one bit. (Thanks to Jesse for the pointers re. using stage.invalidate for this process)
2. The back face is now automatically flipped horizontally, so text and graphics are no longer mirrored and will display correctly
3. I’ve changed the logic which sets the dynamic registration point for the pivot. This should now correctly line up the front and back faces of the 3D plane, regardless of where their individual registration points are.

Also, some people have asked how to use it, so here is a really quick example to get you up and running with PaperSprite if you are not familiar with using classes and are using the Flash IDE to compile your movie:

1. First, download the PaperSprite class and put the soulwire folder in the same directory as your FLA or source code.
2. Then paste this code into the main timeline of your FLA:

// Import the PaperSprite class

import soulwire.display.PaperSprite;

/*
Create a new PaperSprite:

If your front and back faces already exist, you can pass them straight
into the constructor, like so:

myPaperSprite = new PaperSprite( myFrontMc, myBackMc );
*/

var myPaperSprite:PaperSprite = new PaperSprite();

/*
Optionally specify a pivot point, in this example the centre is used
(this is also the default so there is no need to set this normally).

To pivot around the top left use:
myPaperSprite.pivot = new Point(0,0);

or for bottom right:
myPaperSprite.pivot = new Point(1,1);

and so on...
*/

myPaperSprite.pivot = new Point(0.5,0.5);

// Centre it on the stage

myPaperSprite.x = stage.stageWidth / 2;
myPaperSprite.y = stage.stageHeight / 2;

/*
Set the front and back faces:

These can be any type of DisplayObject, for example, to use a MovieClip
from your library use:

myPaperSprite.front = new MyLibraryClip();
*/

myPaperSprite.front = createSprite(0x66FF00);
myPaperSprite.back = createSprite(0xFF3399);

// Add the PaperSprite to the display list

addChild ( myPaperSprite );

// This method just creates coloured boxes for the demo

function createSprite (colour:uint):Sprite
{
	var mySprite:Sprite = new Sprite();
	mySprite.graphics.beginFill ( colour );
	mySprite.graphics.drawRect (0,0,100,100);
	mySprite.graphics.endFill ();
	return mySprite;
}

// Listen for the enterFrame event

addEventListener ( Event.ENTER_FRAME, spin );

/*
Move and rotate your PaperSprite in any way you want - it will
update it's visible face automatically!
*/

function spin ( event:Event ):void
{
	var mX:Number = ((mouseX / stage.stageWidth) - 0.5) * 360;
	var mY:Number = ((mouseY / stage.stageHeight) - 0.5) * 360;

	myPaperSprite.rotationY += (mX - myPaperSprite.rotationY) / 40;
	myPaperSprite.rotationX += (mY - myPaperSprite.rotationX) / 40;
}

Publish the movie and you’re all done!

Background

Flash Player 10’s new 3D capabilities are a much welcomed and exiting addition, and although their implementation at first seemed a little obtuse after using libraries such as Papervision3D, Adobe’s implementation is beginning to make a lot more sense to me. Rather than being handed a complete 3D library, developer’s have been given the necessary building blocks to use 3D, or 2.5D with less ‘hacks’ and arguably better performance where it matters.

Time will tell, but I personally think this will be a positive development for existing frameworks like Papervision and Sandy, because whilst Flash can now natively handle certain tasks like triangle drawing, 3D matrices etc – implementing cameras and shaders, loading and building meshes and so on is something which developers will still require. If this can plug neatly into Flash Player’s capabilities and possibly yield increased performance then all the better.

In more basic situations though, the ability to rotate a DisplayObject around its X, Y and Z axis, as well as finally having a Z property, is useful for adding basic 3D effects, even if your project doesn’t require anything more sophisticated than that.

But this is where some gaps begin to arise. Z sorting, for example, will be required, and Ralph Hauwert (a member of the Papervision3D team) has already written a useful script for this.

Two Sided DisplayObjects

Another common use for Flash Player 10’s 3D capabilities is to create double sided sprites or planes, than can be rotated to reveal alternate content on either side. Again, I would have expected this to be a new property of DisplayObjectContainer, so that front and back could be passed to it as DisplayObjects. This isn’t the case though, so it requires some minimal work around.

Lee Brimelow demonstrated one approach to this, which was to monitor the rotation of a DisplayObject, and swap the visible faces accordingly (Jesse Freeman also used this technique). This works great in many cases, however it doesn’t give the desired results when the parent DisplayObject is off centre and therefore subject to the perspective of the 3D environment. It also means that if you are rotating the DisplayObject around multiple axis then the calculations required get slightly more complex.

The Solution

As ever, the wise and prolific Senocular came to the rescue. He described a technique whereby 3 points are placed on the surface of the DisplayObject, and by determining the winding direction of these points when translated from 3D to 2D space; one is able to determine which way the DisplayObject is facing. Because this is relative to the perspective used to produce this transformation, the result is true to the point of view of the user.

Senocular published his helper class which will return a Boolean value indicating whether a given DisplayObject is front facing or not, and so I wish to stress that all credit is his for his solution and wonderfully clear explanation of the process.

I did feel that it might be beneficial though to wrap this up, along with other functionality, into a class which solved the double sided DisplayObject issue, and so I introduce the PaperSprite class (I chose to call it PaperSprite so that it isn’t confused with Sprite3D if you use another 3D library).

How to use PaperSprite

Creating a two sided Plane using PaperSprite is simple:

import soulwire.display.PaperSprite;

var mySprite:PaperSprite = new PaperSprite();

// Any type of DisplayObject can be used for each face
mySprite.front = new Sprite();
mySprite.back = new Bitmap( myImage );

So you can use any DisplayObject for the front and reverse sides; MovieClips, Sprites, Bitmaps, Shapes, Loaders etc. These can also be set via the constructor…

Another property of PaperSprite is whether is updates it’s display automatically. If true, once the PaperSprite is added to the stage it will update its visible face autonomously, however if you have a more general render loop where you’re controlling your 3D scene from, you can set this to false and then call the update method when calculating the visible face is required:

Thanks to Jesse Freeman for suggesting a method to limit unnecessary calls to the update method. By overriding the setters for the position and rotation properties and using stage.invalidate(), then listening for the render event (Event.RENDER), the calculations needed to determine the visible face of the PaperSprite need only be made if one or more properties have been changed and if the PaperSprite needs to be rendered. Therefore there is no longer a need for autoUpdate, everything is handled internally by the PaperSprite class. Cheers Jesse! :)

…The other useful feature is the pivot property. By default, a PaperSprite will pivot (rotate around) its centre point, which will be automatically calculated for each face depending on its respective dimensions. Alternatively, if you want the PaperSprite to rotate around the top left corner, bottom right, or indeed any point on its surface, you can set the pivot property, which is a Point object and requires 2 normalised values, representing the fraction of the width and height of the PaperSprite at which you want to place the pivot.

So to set the PaperSprite to rotate around its centre (this is set by default) you can use:

mySprite.pivot = new Point( 0.5, 0.5 );

And its Top Left corner:

mySprite.pivot = new Point( 0.0, 0.0 );

And its top centre:

mySprite.pivot = new Point( 0.5, 0.0 );

And so on!

You can also access its isFrontFacing property at any time:

if ( mySprite.isFrontFacing )
{
	doSomethingGood();
}

An finally, removing the front or back face from its display list (or indeed not defining one at all) will cause the PaperSprite to display the available face at all times, essentially giving it the default behaviour of a one sided Sprite.

Download the PaperSprite class

Again, this is really just a glorified augmentation of Senocular’s solution to finding the front facing side of a DisplayObject in 3D. All credit should go to him for creating such an elegant solution. I built this class in order to simplify its use and provide the extra functionality needed to quickly start working with a two-sided DisplayObject.

Related posts:

1. Fit DisplayObject into Rectangle
2. Flipping you the Boid
3. AS3 Webcam Motion Tracking

It was a rainy Sunday and I found myself playing around with perlin noise for a couple of hours.

The idea was to create a simple flow field using the BitmapData perlinNoise method, and use this to influence the movement of autonomous agents. There is no complex vector math or steering behaviours, just some simple logic for how a particle or agent reacts to the current pixel it is positioned over.

Despite its simplicity, by changing parameters such as the level of offset per frame or the x and y frequency, it is possible to create some interesting effects which resemble the movement of flocking birds, shoals of fish or indeed more fittingly for this time of year; panicking Christmas shoppers desperately hunting for gifts.

At each update step, an agent will grab the value of the pixel it is on top of and alter its trajectory based on the brightness of the pixel. Throw in some variance relating to how strongly an individual agent will react to this value and some interesting patterns begin to emerge.

The basic logic I’m using in this demo is as follows:

// Get the pixel the agent is over
var pixel:uint = noiseBitmap.getPixel( agent.x, agent.y );

// Find the brightness of the pixel
var brightness:Number = pixel / 0xFFFFFF;

// Set the angle and the speed according to brightness
var speed:Number = brightness * agent.speed;
var angle:Number = brightness * 360 * Math.PI / 180;

// Update the agent's position / rotation
agent.x += Math.cos( angle ) * speed;
agent.y += Math.sin( angle ) * speed;
agent.rotation = brightness * 360;

Feel free to download the source code and start tweaking the many variables. Adding further life to the agents by introducing collision or allowing additional individual traits beyond simply speed and wander might be one option worth exploring.

Have fun!…

Related posts:

1. Flipping you the Boid
2. AS3 Webcam Motion Tracking
3. Double Sided 3D Plane in FP10

My previous post explained and provided a very simple method for extracting colours from a BitmapData image, by averaging the colours in specific areas. This can have several applications, for example it features in a large amount of prototypes for the update to my Motion Tracking engine. However, if you want to create an accurate and representative colour palette from an image it has several flaws, the most obvious being that by averaging colours, you are actually removing or diluting the striking but perhaps less frequent colours in the image – the very colours which often make an image’s colour palette so exiting!

So, if we’re to extract an exciting and more representative palette from an image, we need a more intelligent algorithm; one which takes into account what makes a colour palette interesting – the contrasts and juxtapositions of colours within the image.

After some experimentation I arrived at the following solution. I have posted the code bellow, but I think it’s important to understand how it works and why each step is taken, so without further ado, here be my approach to calculating the colour palette of a BitmapData image…

Reducing the colours

We’re going to be performing quite a few operations on the colours in our input image, so for the sake of performance it’s wise to reduce the colours in the image as much as possible without losing the overall look and feel of the palette.

Nicolas Barradeau has done some really cool things with this, and his Color Depth Change script is really worth checking out. We can actually enhance this technique slightly by omitting the two loops which cycle through every pixel in the image and use paletteMap. The result is good enough for our needs and substantially quicker. I’m also glad I could use the BitmapData paletteMap method, because up until now I hadn’t ever had a use for it :)

I’ve found that reducing the image’s colour palette to 64 colours works well performance wise and still looks great, but you might want to use anything up to 256.

I won’t go into how paletteMap works, and the docs aren’t very helpful on this matter so I suggest you check out Google if you want a bit more information. In short, we create a new index of colours for the image by creating three arrays, one for each channel (red, green and blue) and pass these values to the paletteMap method, which takes care of the rest at blisteringly fast speeds. What we are left with is a dithered version of our original image, allowing us to analyse it’s pixels in a much speedier manner.

public static function reduceColours( source:BitmapData, colours:int = 16 ):void
{
	var Ra:Array = new Array(256);
	var Ga:Array = new Array(256);
	var Ba:Array = new Array(256);

	var n:Number = 256 / ( colours / 3 );

	for (var i:int = 0; i < 256; i++)
	{
		Ba[i] = Math.floor(i / n) * n;
		Ga[i] = Ba[i] << 8;
		Ra[i] = Ga[i] << 8;
	}

	source.paletteMap( source, source.rect, new Point(), Ra, Ga, Ba );
}

Note: at this stage, if you’re working with a very large image, it’s advisable to draw the image to a smaller BitmapData using a scaled Matrix, as we will be looping over its pixels and so a 2000 x 2000 image is likely to make your CPU rather pissed off at you!

Indexing the colours

The next step is to build an index of the colours in the image. By index, I mean that we want a list of unique colours and a tally for each telling us how many times they occur in the image. We can do this by using an Object or a Dictionary, which uses the colour values as a key and a counter as the value.

public static function indexColours( source:BitmapData, sort:Boolean = true ):Array
{

	var n:Object = {};
	var a:Array = [];
	var p:int;

	for (var x:int = 0; x < source.width; x++)
	{
		for (var y:int = 0; y < source.height; y++)
		{
			p = source.getPixel(x, y);
			n[p] ? n[p]++ : n[p] = 1;
		}
	}

	for (var c:String in n)
	{
		a.push ( { colour:c, count:n[c] } );
	}

	function byCount( a:Object, b:Object ):int
	{
		if ( a.count > b.count ) return 1;
		if ( a.count < b.count ) return -1;
		return 0;
	}

	return a.sort( byCount, Array.DESCENDING );
}

We are left with an Object containing all the colours in the image, grouped by their colour value and with a reference to how frequently they appear. We can then use a for in loop to create an array from the Objects properties.

Finally, we need to sort the resulting Array, essentially by popularity, so that the colours which appear most frequently within the image are at the start of the Array. We can do this using the Array’s sort method, and passing a custom function which compares each colour’s count or frequency to that of the other colours. Now we can discard the count value, and what we are left with is a correctly ordered array of unsigned integers representing our entire spectrum of colours.

Finding unique, contrasting colours

At this point, we could simply take the first values from the array and call this our palette, after all if we wanted, say, a 16 colour palette then we know that the first 16 colours in our Array are the most frequently occurring colours in our source image. The problem with this is that, as with most things in life, just because something is popular doesn’t necessarily make it good! Take a landscape photograph for example; there may be a beautiful collection of flowers with striking colours in the foreground, and a burning orange and pink sunset descending over the horizon, yet the first group of colours in our Array will likely be a host of different shades of green from the hills and fields – not exactly what we were hoping for!

So we need an algorithm which can discern between colours and tell us which of our colours have a sufficient variance, therefore producing a palette which fairly represents the broader spectrum of colours in our image.

The best solution that I found was to add up the square of the red, green and blue components of two colours and compare their results. If they are sufficiently different (as determined by a variable tolerance) then we have a match.

public static function similar( colour1:uint, colour2:uint, tolerance:Number = 0.01 ):Boolean
{
	var RGB1:Object = Hex24ToRGB( colour1 );
	var RGB2:Object = Hex24ToRGB( colour2 );

	tolerance = tolerance * ( 255 * 255 * 3 ) << 0;

	var distance:Number = 0;

	distance += Math.pow( RGB1.red - RGB2.red, 2 );
	distance += Math.pow( RGB1.green - RGB2.green, 2 );
	distance += Math.pow( RGB1.blue - RGB2.blue, 2 );

	return distance <= tolerance;
}

To apply this idea, we need to create an empty Array which will eventually become our results. We then begin to loop through our colours, taking the first and comparing it to subsequent colours until we find one which is sufficiently different. We then push this colour into our results Array, and then continue searching, this time comparing each colour in our initial Array to each colour in our growing Array of unique colours. Once we have reached our predefined quota of colours, or the end of the Array, we have our beautiful colour palette!

public static function different( colour:uint, colours:Array, tolerance:Number = 0.01 ):Boolean
{
	for (var i:int = 0; i < colours.length; i++)
	{
		if ( similar( colour, colours[i], tolerance ) )
		{
			return false;
		}
	}
	return true;
}

public static function uniqueColours( colours:Array, maximum:int, tolerance:Number = 0.01 ):Array
{
	var unique:Array = [];

	for (var i:int = 0; i < colours.length && unique.length < maximum; i++)
	{
		if ( different( colours[i], unique, tolerance ) )
		{
			unique.push( colours[i] );
		}
	}

	return unique;
}

Putting it all together

I’ve packaged the above technique, as well as the averageColour and averageColours methods from my last post into a ColourUtils class. Most of the methods detailed above, despite being necessary steps in finding an image’s colour palette, are useful in their own right and so I have separated them out into a documented collection of static methods.

You can download the ColourUtils class and also some examples of how to use it:

Possible Enhancements

I have a few ideas for possible enhancements to the algorithm, and would like to hear yours if you have any. Currently, these are my intentions for the next revision:

• Integration of the Müller formula so that a single colour from a group of similar colours is chosen not by its frequency but by its suitability for the extracted palette – i.e. a colour with greater tonal variation is preferred by the algorithm much the same way as it would be by the eye.
• Palette colour sorting by hue, saturation or brightness.
• A faster way of discarding unwanted colours, preferably before looping through the colour index, resulting in much better performance.

Related posts:

1. BitmapData Average Colours
2. AS3 Webcam Motion Tracking
3. Fit DisplayObject into Rectangle

If you want a very simple way of extracting a colour palette from an image, one technique would be to average the colour values within specific areas. Averaging colour values is almost identical to averaging numbers, except with the added initial step of finding the red, green and blue components of the colour. To do this we can use bitwise operators, in this case bitwise shift, to perform fast operations on each bit inside the unsigned integer returned by getPixel or getPixel32. If you want to know more about bitwise operators, Moock has written a detailed and, as ever, very clear article on where, when and why to use bitewise operations. You can read it here.

So once you’re familiar with how to shift the bits of an integer, you can easily get the RGB values from a 24 bit hexadecimal by moving the bits to the right by a certain amount using the bitwise right shift operator (>>).

For example, getting the red, green and blue values from a 24 bit integer would look like:

var colour:uint = 0x33CC99;

var R:Number = colour >> 16 & 0xFF;
var G:Number = colour >> 8  & 0xFF;
var B:Number = colour & 0xFF;

And finding the alpha, red, green and blue values of a 32 bit integer:

var colour:uint = 0xFF33CC99;

var A:Number = colour >> 24 & 0xFF;
var R:Number = colour >> 16 & 0xFF;
var G:Number = colour >> 8 & 0xFF;
var B:Number = colour & 0xFF;

So we know that by shifting the bits to the right, we can effectively break an unsigned integer representing a colour into its specific components, but what about changing these components back into an RGB or ARGB value? Well, it’s simply a matter of shifting the bits in the other direction, using the bitwise left shift operator, which looks like this << (two less than operators next to each other).

So using this bitwise shift left operator, you can take your RGB or ARGB values and cram the little buggers back into a usable format, with a little help from bitwise OR:

// RGB
var colourRBG:uint = (R << 16 | G << 8 | B);

// ARGB
var colourARBG:uint = (A << 24 | R << 16 | G << 8 | B);

Right, so we know how to take an unsigned integer, for example one that’s been returned from getPixel or getPixel32 and break it down into its RGB or ARGB components – and then for the sake of universal harmony, put it back together again and release it back into its 24 or 32bit world with all its binary friends.

So now, finding the average colour within a set of colours is simply a question of averaging the red, green and blue values of all the colours, and then turning these averages into a new colour.

The method bellow does just this, by looping through the pixels in a BitmapData object, adding up all of the red, green and blue values, dividing them by the total number of pixels and then creating a new colour from the results.

public static function averageColour( source:BitmapData ):uint
{
	var red:Number = 0;
	var green:Number = 0;
	var blue:Number = 0;

	var count:Number = 0;
	var pixel:Number;

	for (var x:int = 0; x < source.width; x++)
	{
		for (var y:int = 0; y < source.height; y++)
		{
			pixel = source.getPixel(x, y);

			red += pixel >> 16 & 0xFF;
			green += pixel >> 8 & 0xFF;
			blue += pixel & 0xFF;

			count++
		}
	}

	red /= count;
	green /= count;
	blue /= count;

	return red << 16 | green << 8 | blue;
}

So taking this a step further; say we want an array of 64 colours from an image, we can break the input image down into 64 chunks and find the averages of each. The easiest way of doing this is to build a grid, copy the pixels in each cell and use the averageColour function to return the average colour value of that cell.

public static function averageColours( source:BitmapData, colours:int ):Array
{
	var averages:Array = new Array();
	var columns:int = Math.round( Math.sqrt( colours ) );

	var row:int = 0;
	var col:int = 0;

	var x:int = 0;
	var y:int = 0;

	var w:int = Math.round( source.width / columns );
	var h:int = Math.round( source.height / columns );

	for (var i:int = 0; i < colours; i++)
	{
		var rect:Rectangle = new Rectangle( x, y, w, h );

		var box:BitmapData = new BitmapData( w, h, false );
		box.copyPixels( source, rect, new Point() );

		averages.push( averageColour( box ) );
		box.dispose();

		col = i % columns;

		x = w * col;
		y = h * row;

		if ( col == columns - 1 ) row++;
	}

	return averages;
}

In terms of building an accurate colour palette from an image, this isn’t a great method, mainly because averaging the colours doesn’t give you a fair representation of the range of colours in the image. The palette returned can often be somewhat muddy; however it is still a technique that I’ve found a use for on a surprisingly regular basis and so thought it worth sharing. Becoming familiar with bitwise operators is also a useful exercise and so, if they aren’t already part of your life, I suggest you begin to woo them with your nerdy charm.

Related posts:

1. BitmapData Colour Palette
2. AS3 Webcam Motion Tracking
3. AS3 Shuffle Grid Class