How a Machine Sees an Image
Pixels, channels, and the grid behind every picture. The single highest-leverage four hours of the whole track β because you cannot be surgical about something you cannot address.
1.1 Why this is the opener
Because you cannot be surgical about something you cannot address.
Right now an image is, to you, a thing you look at. Beautiful, or not. On-brand, or not. Reading well at 96 px, or not. Those judgments are excellent and this track will not touch them.
But there is a second thing an image is: a rectangular grid of numbers. Not "kind of like" a grid of numbers. It is one β that's the entire storage format, and the picture you perceive is what happens when a screen obediently lights up according to those numbers. Once you can see the grid behind the picture, every operation in every tool you have ever used stops being a magic button and becomes an arithmetic you could describe.
Everything else in this track is downstream of today. Compression throws numbers away. Filters multiply neighbourhoods of numbers. Colour spaces reinterpret the numbers. Diffusion models guess numbers. Video is numbers with a time axis; audio is numbers with only a time axis. One idea, five phases.
1.2 The grid
Open DB-01 Pixel Explorer. Load cherries. Push the zoom slider.
At 1Γ it's a cherry symbol. At 4Γ it's a slightly soft cherry symbol. At 12Γ a blue grid appears. At 32Γ the cherry is gone and you are looking at coloured squares. At 34Γ each square starts printing its own hexadecimal value.
Nothing was destroyed on the way up. The squares were always what was there. The cherry is an effect that happens in your visual system when the squares get small enough.
π¨ D1 β The Grid (live: drag the slider)
Same file, four times, at increasing zoom. The rightmost panel is what the computer has.
The three numbers that define the grid
| Term | What it is | On cherries.png |
|---|---|---|
| Width Γ Height | How many cells across and down | 2000 Γ 2000 in the master |
| Pixel count | width Γ height β how many cells total | 4,000,000 |
| Channels | How many numbers per cell | 4 (R, G, B, A) |
So the cherries master holds 16,000,000 numbers. Each is a whole number from 0 to 255. That's it. That's the file. Everything else β the shine on the fruit, the leaf, the fact that it reads as cherries β is emergent.
The Pixel Explorer prints these live. Watch the "distinct colours in view" figure while you move the resolution slider: a 256Γ256 view of the cherry contains somewhere around 20β30 thousand distinct colours. A human would say "it's red with a green leaf." The machine has thirty thousand opinions about it.
1.3 One pixel, opened up
Hover any pixel in the Pixel Explorer and look at the readout panel. You get six lines. Here is what each one means and why you'd ever care.
π¨ D2 β Four Numbers per Pixel (live: pick a colour)
| Line | What it is | When you actually use it |
|---|---|---|
hex #C4283A | The three colour channels in base-16. C4 = 196, 28 = 40, 3A = 58. | Talking to developers and web/CSS. It's just RGB in disguise. |
rgba 196, 40, 58, 255 | Red, Green, Blue, Alpha β each 0β255. Literally the bytes in the file. | The ground truth. Everything else is computed from here. |
hsv 353Β°, 80%, 77% | Hue (wheel position), Saturation (colourfulness), Value (brightness). | Shifting colour while keeping the material. "Same but green" is a hue change and nothing else. |
lab L 43.6, a 60.4, b 29.5 | Perceptual space. L = how bright it looks, a = greenβred, b = blueβyellow. | Judging whether two colours look the same; grading brightness without wrecking hue. How brand-colour accuracy (ΞE) is measured properly. Deep dive in M2. |
alpha 255 (100%) | How much this pixel exists. 255 = fully opaque, 0 = invisible, in between = partially there. | Everything about cut-outs. Β§1.6 is entirely about this number. |
offset 41232 of 262144 | Where the pixel physically sits in the byte array. | Rarely β but it makes the point concrete. |
Note the arithmetic in that last row, because it is the whole idea of the module in one line:
byteIndex = (y Γ width + x) Γ 4
A pixel has an address. You can name it. That is what "surgical" means, formally: an operation applied to a named set of addresses rather than to the whole file.
1.4 Channels: the picture is a stack of grey pictures
An RGB image is not one colour image. It's three greyscale images stacked, each recording "how much of this one primary is at this cell," plus a fourth recording "how much of this cell exists at all."
π¨ D3 β The Channel Split (live, real THG symbol)
Composite Β· R plate Β· G plate Β· B plate Β· A plate. Nobody drew those four grey images β they fall out of the physics.
Do this on the cherry and something clicks: the red channel almost looks like a photo of the cherries with the leaf removed, and the green channel almost looks like a photo of the leaf. Nobody drew those. They fall out of the fact that red things have high R values and green things have high G values.
This is not a curiosity. It is a working technique:
| Trick | What you do | Why it works |
|---|---|---|
| Free mask from a channel | Take the R channel of a red symbol as a starting selection | Red objects are already bright in R, everything else is dark β most of the way to a mask, at zero effort |
| Diagnose a colour cast | Look at the three channels' histograms | If B is lifted at the bottom everywhere, someone's shadows are blue |
| Spot channel damage | Compare channels after compression | JPEG chroma subsampling damages colour channels far more than luminance β visible here first |
| Understand edge fringing | Isolate R vs G at an edge | The channels don't always agree at the edge. That disagreement is the fringe. |
cherries, the leaf averages roughly R 99, G 161 and the fruit R 207, G 66. So the G plate alone does not cleanly separate them β the fruit still has meaningful green in it, and a plain G threshold will grab parts of the fruit's lit side. What actually separates them is the difference between channels (G β R is strongly positive on the leaf and strongly negative on the fruit). That is the honest version, and it is the seed of what M6 calls chroma-distance masking: channel arithmetic, not a single channel.The fourth plate is different
R, G and B are colour. A (alpha) is not a colour at all β it's a per-pixel statement about existence. It doesn't affect what the pixel looks like; it affects how much the pixel participates when it's placed over something else. It is the single most misunderstood number in game-art delivery, so it gets its own section.
1.5 Bit depth: how finely the machine is allowed to count
Push the Bits slider in the Pixel Explorer from 8 down to 3. Watch the cherry's highlight.
It bands. It breaks into visible steps of flat colour. And crucially: nothing was blurred, nothing was resized, no compression was applied. You only reduced how many distinct values each channel is allowed to hold.
π¨ D4 β Bit depth staircase (live: the gradient behind every slot banner)
Banding is not a rendering bug. It's arithmetic running out of room.
| Bits/channel | Levels | Total colours | Where it shows up |
|---|---|---|---|
| 1 | 2 | 8 | Pure line art, hard mattes (1 bit per channel β 2Β³ = 8 colours. Not to be confused with a 1-bit image, which is 2 colours) |
| 4 | 16 | 4,096 | Visible banding in any gradient (GIF is actually 8-bit indexed β a 256-entry palette, a different mechanism) |
| 8 | 256 | 16,777,216 | The standard. PNG, JPEG, WebP, every screen you deliver to |
| 10 | 1,024 | ~1.07 billion | HDR video; the reason HDR gradients are clean |
| 16 | 65,536 | ~2.8 Γ 10ΒΉβ΄ | Master files. Not for delivery β for headroom |
| 32 (float) | effectively continuous | β | VFX/compositing intermediates; can hold values above "white" |
The rule that matters: 8 bits is plenty for displaying an image, and often not enough for editing one.
Here's why. Every grade, curve, brightness lift and colour shift is arithmetic on those integers. Push a dark region up and you might map values 10β14 onto values 40β90 β five original levels stretched across fifty. The gaps between them are empty, and they show as bands. The original had the detail; the arithmetic couldn't preserve it because there were no in-between numbers to use. Do the same on a 16-bit master and there are 256Γ more intermediate values to survive the stretch.
1.5b The numbers are not light β the one thing that makes everything else make sense
Here is a fact that looks like trivia and is actually load-bearing for the rest of the track.
You would assume that 128 is half as much light as 255. It is not. 128 emits about 22% of the light of 255. Mid-grey on your screen is nowhere near half-brightness.
The reason: the 0β255 numbers in a PNG are gamma-encoded, not linear. Roughly, light β (value/255)2.2. This is deliberate and it is good engineering β human vision is far more sensitive to differences in dark tones than bright ones, so spending the limited 256 codes evenly across perceived brightness (rather than physical light) is what makes 8-bit look acceptable at all. An 8-bit linear image would band horribly in the shadows.
Concretely: average black (0) and white (255).
ββNaive:
(0 + 255)/2 = 128.
ββLinear-correct: linearise β average β re-encode = 188.
That is not a rounding difference. That is a visibly different grey.
watermelon, drag resolution down to 16 px, and flip between Linear light (correct) and sRGB values (naive). The naive version is measurably darker and its bright rind detail dies faster. Every tool you use makes this choice on your behalf, usually silently β Photoshop's "Blend RGB Colors Using Gamma", ffmpeg's scaler flags, and every game engine's texture import settings are all this one switch.Three consequences worth carrying into the next modules:
- Thin bright lines vanish when downscaled in naive sRGB β exactly the failure mode for the WILD lettering and for specular highlights.
- A 50% opacity layer is not a perceptual halfway point, which is why cross-fades often look like they dip in the middle.
- "L" in LAB is the perceptual axis β it is doing the same job gamma encoding does, but properly and by design. That is why M2 tells you to grade brightness in LAB.
One clarification so it does not confuse you later: gamma is about what the numbers mean; bit depth (Β§1.5) is about how many numbers you get. Independent dials, and gamma is the reason 8 bits is survivable at all.
1.6 Alpha: the number that decides whether your cut-out looks professional
Open DB-02 Alpha & Matte Lab. Load cherries. Set the backdrop to White. Look at the edge. Now set it to Near-black. Then turn on Broken (unmatted fringe).
There it is. The halo.
The one equation
There is exactly one equation behind every composite in every tool ever made:
out = src Γ Ξ± + bg Γ (1 β Ξ±)
- Ξ± = 1.0 β
out = src. You see only the symbol. - Ξ± = 0.0 β
out = bg. You see only the reel behind it. - Ξ± = 0.5 β half and half. A blend.
π¨ D5 β The edge is a blend, not a line (live: change the backdrop)
A run of Ξ± values across one edge: 1.00 1.00 0.97 0.71 0.34 0.06 0.00. Identical data, two backdrops, two completely different edges.
Why halos happen (and how to specify the fix)
An artist composes a symbol against a white studio background. The soft edge pixels physically contain white mixed into them β that's what antialiasing over white means. When the cut-out is exported without correctly removing that contribution, each edge pixel stores something like "70% cherry-red + 30% white" as if it were the pure colour of the object, with Ξ± = 0.7 alongside it.
Place that over white and it looks perfect β the white it's carrying matches the white it lands on. Place it over your #0d1117 reel and the white it's carrying is suddenly wrong by a mile. You get a bright rind: the halo.
Say which halo you have, though: a bright rind = background contamination left in the edge, or premultiplied data being read as straight. A dark rind = straight data being read as premultiplied, i.e. Ξ± applied twice. Same corner of the pipeline, opposite fix.
1-bit vs 8-bit mattes
Push the Alpha threshold slider up in the Matte Lab. Every partial pixel snaps to fully-there or fully-gone. The quality report flips from 8-bit (soft) to 1-bit (hard), and the verdict badge turns red.
| Symptom | 1-bit (hard) matte | 8-bit (soft) matte |
|---|---|---|
| Edge appearance at 100% | Stair-stepped, jagged | Smooth |
| Edge at 25% (mobile lobby) | Crunchy, "cheap" | Clean |
| During a spin/scale animation | Crawls and shimmers β the staircase moves | Stable |
| Over a light backdrop | Acceptable-ish | Clean |
| Over a dark backdrop | Obvious | Clean |
| Where it comes from | "Remove background" with a threshold; magic wand without feather; some automated cut-out tools | Proper alpha-aware export |
The animation row is the one that costs money. A hard matte looks nearly fine on a static mock-up and then visibly crawls the moment the reel spins β which is after approval, in QA, on someone else's schedule.
The number to remember β and it is not a percentage. Semi-Ξ± as a % of the canvas is a trap: it depends on how much empty padding the file has and on how big the shape is, so the same artwork exported two ways gives two different numbers. The portable metric is mean edge width = semi-Ξ± pixels Γ· perimeter pixels, which DB-02 computes live:
| Mean edge width | Verdict | What it means |
|---|---|---|
| 0 px (0% semi-Ξ±) | β hard-cut | 1-bit matte. Will alias and crawl on every spin. |
| ~1β2 px | β healthy | Normal antialiasing. This is what a clean export looks like at any resolution. |
| > ~4 px | β soft | A real glow, soft shadow or feathered edge is baked into the matte. Often deliberate in slot art β but know it is there, because it eats into the silhouette. |
cherries reports about 20% semi-Ξ± β alarming until you see the edge width is ~10 px, because the artwork has a genuine soft drop-shadow/glow baked in. That is not a broken matte; it is a designed one. plum sits near 4% / ~3.8 px, wild near 8% / ~4.6 px, and ruby is 0% β a fully opaque logo with no cut-out at all. Four real files, four completely different matte profiles: this is exactly why you measure instead of assuming.1.7 Resolution: how many buckets you chop the world into
Back in DB-01. Move the Resolution slider from 256 down to 8. At 128 the cherry is still obviously a cherry. At 64 it's a red blob with a green tick. At 32 it's an abstract mark. At 16 it's four coloured squares.
First: the point where meaning dies is a real, findable number. For the cherry on a dark reel it's somewhere in the 40β56 px range β below that, the leaf stops being a leaf. That number is not an opinion, and it should be a line in a spec: "minimum rendered size for the Katarina symbol set: 56px; below that, use the simplified variant." Most studios never establish this and just discover it in a complaint.
Second: resolution and bit depth are independent.
π¨ D6 β Two dials, two failures (live: both sliders, same symbol)
Four cells: correct Β· low-res (blocky) Β· low-depth (banded) Β· both. Name the cell, name the fix.
| Dial | What it controls | Failure looks like |
|---|---|---|
| Resolution | How many places you can put a value | Blockiness, lost detail, mush |
| Bit depth | How many values each place can hold | Banding, posterisation, dead gradients |
1.8 A quiet preview of the whole track
One idea to plant now, because it makes the rest of the track feel inevitable rather than long.
| Image | Audio | |
|---|---|---|
| Resolution (samples across space) | = | Sample rate (samples across time) |
| Bit depth (levels per sample) | = | Bit depth (levels per sample) |
| Channels (R, G, B, A) | = | Channels (L, R, 5.1) |
| The pixel grid | = | The waveform |
| Frequency content (fine detail vs flat areas) | = | Spectrum (treble vs bass) |
| JPEG: discard high frequencies the eye won't miss | = | MP3: discard frequencies the ear won't miss |
| Nyquist in space: detail finer than 2 px per cycle cannot be represented β and aliases if you don't filter it out first | = | Nyquist in time: frequencies above half the sample rate cannot be represented β and alias down as false tones |
You are not learning three subjects. You are learning one idea, applied to space, to time, and to both.
Video, when we get there in M13βM14, is simply the image idea with a time axis bolted on β and it borrows the compression trick from both columns.
The Nyquist row is the sharpest of these. It is the same theorem, and it explains a thing you have already seen: when a finely-textured symbol is downscaled without a proper filter, the texture does not just soften β it turns into a coarse, crawling moirΓ© pattern. That is aliasing, and it is the visual twin of the buzzy artefacts you get sampling audio too slowly. Both are fixed the same way: filter out what the new grid cannot hold before you resample. It is exactly why DB-01 area-averages instead of point-sampling.
π οΈ Hands-on task (45 min)
The job: produce an Asset Spec Card for two real THG symbols. This is a document you could actually start requiring from suppliers.
Asset A β katarina-symbols/cherries.png (2000Γ2000 RGBA master)
Asset B β katarina-symbols/wilds_WS.png (1094Γ776 RGBA β the WILD lettering symbol)
Both are loaded in DB-01 and DB-02 as cherries and wild.
| Field | How to get it |
|---|---|
| Grid | width Γ height, and total pixel count |
| Numbers in the file | pixels Γ 4 |
| Channel report | Which channel carries the most information? Isolate each in DB-01. Is any channel nearly the mask? |
| Matte class | 1-bit or 8-bit? Read the DB-02 quality report. |
| Matte profile | From DB-02: semi-Ξ± % and mean edge width. Is the edge ~1β2 px (plain antialiasing) or much wider (a baked-in glow/shadow)? Which, and is it intentional? |
| Halo risk | Toggle DB-02 backdrop white β near-black. Anything appear? |
| Bit-depth verdict | In DB-01, at what bit depth does this asset visibly break? |
| Minimum legible size | In DB-01, drop the resolution until the symbol stops reading. Record the number. |
| The one change you'd demand | One sentence, in machine language, not description. |
Then answer the real question: the two assets need different treatment. Say why, in terms of what you measured β not in terms of taste.
Expected findings (open only after you've done it)
- cherries is a compact, high-saturation, mostly-red object. Its R channel is close to a usable mask by itself. Semi-Ξ± will be modest. It stays legible surprisingly small because its silhouette is simple and its contrast against a dark reel is enormous. Its gradients (the specular on the fruit) break first under low bit depth.
- wilds_WS is a lettering asset, and that changes everything: it is perimeter-heavy (much more edge per unit area, so a higher semi-Ξ± % is normal and correct), its legibility floor is set by stroke width, not overall size β so it dies at a much larger size than the cherry β and it is the asset class most damaged by hard mattes, by aggressive compression (M4), and by anything latent-space (M10).
- The general rule: symbols and lettering are different asset classes with different minimum sizes, different matte tolerances and different format rules. Treating them the same is the source of a surprising share of delivery problems.
Deliverable: one page, two cards, and the comparison paragraph. Paste it into the Notion Hands-on Log.
π Module 1 Quiz
Eight questions. Aim for 6/8 before moving to M2. Answers are hidden until you open them.
Answer
512 Γ 512 = 262,144 pixels Γ 4 channels = 1,048,576 numbers. At 8 bits (1 byte) each = 1,048,576 bytes β 1.0 MB raw. The PNG on disk will be far smaller β that's compression (M4) β but this is what the machine actually works with in memory.
rgba(196, 40, 58, 102). The reel behind it is #0d1117 = rgb(13, 17, 23). What colour does the screen actually display? Show the arithmetic for the red channel at least.Answer
Ξ± = 102/255 = 0.4. Using out = src Γ Ξ± + bg Γ (1 β Ξ±):
R: 196 Γ 0.4 + 13 Γ 0.6 = 78.4 + 7.8 = 86.2 β 86
G: 40 Γ 0.4 + 17 Γ 0.6 = 16.0 + 10.2 = 26.2 β 26
B: 58 Γ 0.4 + 23 Γ 0.6 = 23.2 + 13.8 = 37.0 β 37
Displayed: rgb(86, 26, 37) β a dark maroon, nothing like the stored rgb(196,40,58). Over a white backdrop the same pixel would display as roughly rgb(231, 169, 176) β a pale pink. Same stored pixel, two completely different appearances. That is why edge pixels are where cut-outs go wrong.
Answer
The semi-transparent edge pixels are carrying the studio background colour (white) mixed into their stored RGB β a premultiply/matting error. Over white it's invisible because the contamination matches the destination; over a dark reel it's exposed. Instruction: "The matte is contaminated β un-premultiply the edge against white and re-export with straight (unassociated) alpha." Not "the edges look dirty, please fix."
Answer
File B. File A's 0.0% semi-Ξ± means a 1-bit hard matte β a stair-stepped edge that crawls and shimmers on every frame of the animation. File B's 2.1% is a healthy antialiased matte and it's already much closer to display size. Resolution is not quality; File A has more pixels and a worse matte, and the matte wins.
Answer
No, the original wasn't damaged. 8 bits gives 256 levels per channel; a dark gradient might only occupy levels ~8β30 β about 22 distinct values. Brightening stretches those 22 across a much wider output range with nothing to put in the gaps, so the steps become visible. The information to fill them never existed in an 8-bit delivery. Policy: masters archived at 16-bit; 8-bit files are final deliveries; any downstream grading must be done against the master, not the delivery. (Alternatively: deliver a pre-graded variant, or dither the gradient.)
Answer
Look at the individual channels first (DB-01, isolate mode) and compare them. The leaf is much brighter in G than in R; the fruit is the reverse. So the starting point is not βthreshold Gβ β measured on the real file the fruit still carries plenty of green β it is G minus R, which is strongly positive on the leaf and negative on the fruit. Starting from channel arithmetic and refining beats hand-selecting almost every time. The principle: the image often already contains the mask you want, or a subtraction away from it; check before you build one.
Answer
(a) β resolution Β· (b) β bit depth Β· (c) β matte class (1-bit) Β· (d) β matting/premultiply error. Four independent dials, four distinct symptoms, four different fixes. Naming the right one is the entire skill of this module.
Answer
Legibility floor is set by the smallest meaningful feature, not by overall size β for the cherry that's the leaf (a large, high-contrast blob), but for the WILD symbol it's the stroke width of the letterforms, and when a stroke drops below ~2px it can no longer be represented and the letters blur into a grey bar. They need two different minimum-size rules, and probably a simplified lettering variant for small placements.
β Completion checklist
- Read Β§1.1 β Β§1.8 (including Β§1.5b β gamma is the one people skip and regret)
- Played with DB-01 Pixel Explorer β zoomed past 34Γ to see hex values, dropped bit depth to 3 and watched banding appear, isolated each channel
- Played with DB-02 Alpha & Matte Lab β flipped Broken/Correct with a white fringe on a dark backdrop, pushed the threshold to make a 1-bit matte, over-feathered until the verdict went yellow
- Completed the Asset Spec Card hands-on for both
cherriesandwild - Scored β₯ 6/8 on the quiz
- Logged at least one "aha" or one question in the Notion Questions & Aha's page β not optional; it's how the track gets calibrated to you
β back to the hub