April 20262 weeks

Mosaic

Rebuilding photographs out of tiles that follow the image's flow.

A solo project, from the flow-field math to the interface.

ReactTypeScriptViteWeb WorkersCanvas 2D

Starting With Mosaics

I've always liked mosaics, especially Roman and Byzantine ones. Up close a face is just thousands of little tiles, and the artist sets those tiles in curving rows called opus vermiculatum that follow the shape of whatever they're depicting. The rows themselves do a lot of the work. I wanted to see if I could get a browser to build a photo the same way, so I made Mosaic. You drop in an image and it rebuilds it out of small tiles that follow the flow of the picture.

It all runs in the browser with React and TypeScript. The heavy image processing happens in a Web Worker so the interface stays responsive while it works.

From a Photograph to a Flow Field

Before it places any tiles, Mosaic works out which way the image flows. It builds a structure tensor from the image gradients: at each pixel it looks at the direction things are changing and how strongly the surrounding pixels agree on that direction. Smooth that field out and you get streamlines that stay calm and parallel across a flat sky but curve tightly around an eye or a fold of cloth.

The original painted portrait of a woman in a red-and-blue headwrap — Left: the original image. Right: the flow field Mosaic pulls out of it. Every tile ends up following these lines.

The same portrait reduced to thousands of fine white flow lines that curve around the face, headwrap folds, and the concentric rings of the earring — Left: the original image. Right: the flow field Mosaic pulls out of it. Every tile ends up following these lines.

A Palette, Not a Picture

A real mosaic is made from a limited set of tiles, so Mosaic starts by choosing its box of colors. It runs k-means clustering in Lab color space, which measures how different two colors look to the eye instead of comparing raw RGB values, and reduces the whole photo to a small palette. Each tile is later painted with the closest palette color to the area beneath it. A bit of adjustable randomness occasionally reaches for the second or third closest color, which keeps large flat regions from turning into one dead block of a single tone.

The portrait rendered in only about fourteen flat colors, showing the reduced palette Mosaic extracted from it — The same portrait cut down to a fourteen-color palette, which is the set of tiles the mosaic gets built from.

Weaving the Rivers

This is where the flow field and the palette come together. Mosaic traces evenly spaced streamlines across the flow field, using an approach based on the Jobard-Lefer method, and then walks along each one dropping a tile at a time. Neighboring lines negotiate their spacing so the gaps between tiles stay even, even where the flow pinches together or fans out, and every tile gets a small random nudge so no two are exactly alike. Busy, detailed areas like a face get smaller tiles, and calm areas like a background get larger ones, which is roughly how a person would ration the work too.

The finished mosaic of the portrait: tiles curve around the face and headwrap, with noticeably finer tiles rendering the eyes and lips and larger tiles in the background — The finished result. The tiles shrink around the eyes and lips where there is more detail to hold.

Any Picture, Any Style

Because everything is derived from the image itself, Mosaic never needs to know what it is looking at. A still life fans its background tiles out around the fruit. Van Gogh's Starry Night, which is close to a flow field already, turns its swirling sky into spirals of tiles that trace the brushstrokes.

Van Gogh's Starry Night rebuilt as a mosaic, its swirling sky rendered as spiraling rings of tiles — The same process on two very different images.

A still life of gourds rebuilt as a mosaic, background tiles fanning outward around the fruit — The same process on two very different images.

Tuned in the Browser

Mosaic is built to be played with. The whole pipeline (adjust, flow field, palette, weave) runs in that Web Worker and reports its progress back to the interface, and every control is debounced so an image re-weaves almost as fast as you can drag a slider. You can switch between four views to watch each stage (Source, Palette, Flow, and Mosaic), change the tile size and shape, widen the gaps, resize the palette, calm down or stir up the flow, add irregularity, then pan around, zoom in, re-roll with a new random seed, and export the result as a full-resolution PNG.