Inside Matrix Desktop's 5-Pass Metal Rendering Pipeline

Matrix Desktop renders film-accurate digital rain as a live macOS desktop wallpaper. At full quality it targets 60 fps with a Metal compute pipeline, then reduces work under battery or thermal pressure. This article walks through how cell data flows from CPU simulation to pixel output, how bloom creates the phosphor glow, how a palette LUT drives the color system, and how the app handles power and multiple displays.

The Big Picture: CPU Simulation to GPU Rendering

The architecture splits cleanly into two halves. The CPU runs a state machine (RainSimulation) at a base 20 ticks per second that determines which glyphs are visible, how bright each cell is, and where deletion strings are erasing content. The GPU takes that cell data and turns it into pixels through a 5-pass flat-mode compute pipeline.

Each MatrixRenderer instance owns one simulation, one glyph atlas, and a set of triple-buffered Metal buffers. There is one renderer per connected display, and each renderer selects the GPU attached to its screen.

Cell Data: The CPU-GPU Interface

The simulation produces a combined array of CellData structs for the configured layer grids. Each struct is 8 bytes and laid out to match the Metal shader exactly:

Copystruct CellData {
    var charIndex: UInt16      // Index into glyph atlas
    var brightness: Float16    // 0.0 to ~2.5 (HDR cursors)
    var flags: UInt8           // Changing, cursor, cross-dissolve
    var age: UInt8             // Frames since last glyph change
    var prevCharIndex: UInt16  // Previous glyph for cross-dissolve
}

The prevCharIndex field enables the cross-dissolve. Every cell carries its own cycle age, so glyph changes are staggered across the field rather than synchronized; each change blends old and new glyphs for 40 milliseconds on Authentic, one 25-fps source frame, independently of render frame rate.

Style-Selected Layers

The shared renderer supports three layer slots, and a uniform mask chooses which participate. Since 1.5.1 Authentic enables all three planes, like the layered presets, on its own film-measured lattice; the mask remains so a preset can opt out of a plane without duplicating the rendering pipeline.

The Glyph Atlas

The app ships a deterministic 868,352-byte R8 coverage resource containing 53 independent 128 × 128 slices. GlyphAtlas validates its fixed SHA-256, loads it into a texture2d_array, and generates eight mip levels independently for every slice. The inventory contains:

• 30 half-width katakana, mirrored horizontally (matching the film's convention)
• The kanji character for "day" (日)
• 10 Arabic numerals, some mirrored (2 and 5 horizontal, 3 and 6 vertical)
• Z plus 11 symbols

Transforms and small deterministic geometry variations are baked into the canonical resource offline. The shader selects the array slice from charIndex and chooses an explicit mip level from the glyph's native pixel footprint, preventing neighboring-glyph bleed.

Pass 1: Main Composite

The first compute pass runs at the native drawable resolution. Each thread maps to one output pixel. For every enabled layer, the shader:

1. Maps the pixel into the layer's logical grid cell
2. Skips empty cells and samples the selected glyph-array slice at an explicit mip level
3. Blends prevCharIndex and charIndex during each cell's staggered mutation
4. Applies glyph scale, X-only width correction, sharpness, and cell brightness
5. Accumulates body luminance in red and cursor luminance in green

Keeping body and cursor luminance separate lets Cursor Color and cursor isolation survive scene rendering and bloom instead of being collapsed into the body palette.

Copyfloat2 channels = float2(0.0);
for (uint layer = 0; layer < 3; layer++) {
    if ((uniforms.activeLayerMask & (1u << layer)) == 0) continue;
    CellData cell = cells[cellIndex(layer, gid)];
    float brightness = sampleGlyphArray(cell, gid) * float(cell.brightness);
    channels[cell.isCursor ? 1 : 0] += brightness;
}
scene.write(float4(channels, 0.0, 1.0), gid);

The output goes to an intermediate full-resolution texture (not directly to the drawable). This texture feeds into both the bloom pipeline and the final composite pass.

Pass 2: Bloom Downsample

The bloom pass extracts the brightest pixels from the scene and wraps them in a soft glow. This is what makes cursor cells and high-brightness glyphs feel like they are emitting light rather than just being painted white.

The downsample target uses ceil(width / 4) by ceil(height / 4) dimensions. Each thread averages the valid pixels in one 4 × 4 source block, then applies the bloom threshold. Isolated cursors retain separate red/green bloom channels; legacy non-isolated styles threshold their combined luminance once.

Copyfloat2 sum = float2(0.0);
uint sampleCount = 0;
uint2 base = gid * 4;
for (uint y = 0; y < 4; y++) {
    for (uint x = 0; x < 4; x++) {
        uint2 source = base + uint2(x, y);
        if (insideScene(source)) { sum += scene.read(source).rg; sampleCount++; }
    }
}
float2 average = sum / float(max(sampleCount, 1u));
float2 bloomChannels = thresholdChannels(average, uniforms);
bloom.write(float4(bloomChannels, 0.0, 1.0), gid);

Passes 3-4: Separable Gaussian Blur

A direct 2D Gaussian blur at radius 13 would require sampling 27x27 = 729 pixels per thread. That is expensive even on a quarter-resolution texture. The standard trick is a separable filter: split the 2D blur into two 1D passes. A horizontal pass blurs each row, then a vertical pass blurs each column of the already-horizontally-blurred result. This drops the sample count from 729 to 54 (27 + 27) per pixel while producing an identical result.

Both passes use the same normalized radius-13 Gaussian weights stored as shader constants. There are 27 taps (13 on each side plus the center), with 14 unique symmetric weights. Each pass filters both body and cursor channels and renormalizes at texture edges.

Copyfloat2 sum = float2(0.0);
float weightSum = 0.0;
for (int offset = -13; offset <= 13; offset++) {
    if (!insideTexture(gid, offset)) continue;
    float weight = gaussianWeights[abs(offset)];
    sum += input.read(horizontalNeighbor(gid, offset)).rg * weight;
    weightSum += weight;
}
output.write(float4(sum / weightSum, 0.0, 1.0), gid);

Pass 4 is identical except it iterates over the vertical axis. The output is a quarter-resolution bloom texture with soft, wide halos around every bright pixel in the scene.

Pass 5: Final Composite

The last pass combines everything and produces the final pixel output at the native drawable resolution. This pass handles four responsibilities:

1. Scene + Bloom Merge

The shader reads the full-resolution red/green scene channels and, when bloom is enabled, bilinearly upsamples the blurred red/green bloom texture. When adaptive quality disables bloom, the shader does not sample that private texture at all.

2. Palette LUT Color Mapping

Up to this point, the red channel carries body brightness and the green channel carries cursor brightness. The final pass maps the body through a 256x1 palette lookup table (LUT). Isolated cursors use their configured RGB color; legacy non-isolated styles recombine both channels before the LUT.

Dark body values map to the left of the LUT and bright values map to the right. This keeps palette changes data-driven while allowing Cursor Color to remain independent where the preset requests isolation.

Copyfloat2 brightness = sceneChannels + bloomChannels;
float3 body = paletteLUT.sample(linearSampler,
                               float2(saturate(brightness.x), 0.5)).rgb;
float3 cursor = uniforms.cursorColor * saturate(brightness.y);
float3 color = uniforms.isolateCursor ? body + cursor
                                      : samplePalette(brightness.x + brightness.y);

This is how Matrix Desktop supports 12 presets with radically different color palettes (green, red, blue, purple, amber) without changing a single line of shader code. The CPU generates a different LUT texture for each preset and uploads it. The shader is palette-agnostic.

3. CRT Scanlines

A subtle scanline effect darkens alternating rows by a configurable amount. This simulates the horizontal gaps between phosphor rows on a CRT monitor, reinforcing the retro-digital aesthetic across display resolutions.

4. Dithering

Finally, the shader adds a small amount of blue noise dithering to break up color banding. This is especially important in the dark regions of the image where 8-bit color output would otherwise show visible stepping between nearly-black values.

Copyfloat2 sceneChannels = scene.read(gid).rg;
float2 bloomChannels = float2(0.0);
if (uniforms.bloomIntensity > 0.0) {
    bloomChannels = bloom.sample(linearSampler, normalizedUV(gid)).rg
                    * uniforms.bloomIntensity;
}
float3 color = colorBodyAndCursor(sceneChannels + bloomChannels, uniforms);
color *= scanlineMultiplier(gid, uniforms);
color += dither(gid, uniforms);
drawable.write(float4(color, 1.0), gid);

The Palette LUT System

Each of Matrix Desktop's 12 presets defines a color palette as a series of HSL gradient stops. On the CPU, these stops are interpolated into a 256-entry RGBA array and uploaded as a 256x1 Metal texture with rgba8Unorm format.

The generation process:

1. Define 2-5 HSL color stops (e.g., the Classic preset goes from black at 0.0, through deep green at 0.3, to bright green at 0.8, to white-green at 1.0)
2. Linearly interpolate between stops in HSL space to produce 256 samples
3. Convert each HSL sample to RGBA
4. Write the 256 RGBA values into a MTLTexture using replace(region:...)

When the user switches presets, only this 256x1 texture is regenerated and swapped. The entire GPU pipeline, all five passes, all shader code, stays identical. This separation of color from geometry is what makes preset switching instantaneous.

Triple Buffering and Frame Pacing

The renderer uses three sets of Metal buffers for cell data and uniforms. A DispatchSemaphore initialized to 3 gates frame submission: the CPU can prepare up to 2 frames ahead of the GPU without blocking, but if all 3 are in flight, the CPU waits.

Copyprivate let inflightSemaphore = DispatchSemaphore(value: 3)
private var bufferIndex = 0

func draw(in view: MTKView) {
    inflightSemaphore.wait()

    let buffer = cellDataBuffers[bufferIndex]
    let uniforms = uniformBuffers[bufferIndex]
    bufferIndex = (bufferIndex + 1) % 3

    // Update simulation, write to buffer
    simulation.tick()
    simulation.copyToBuffer(buffer)

    // Encode 5 compute passes...
    let commandBuffer = commandQueue.makeCommandBuffer()!
    commandBuffer.addCompletedHandler { [weak self] _ in
        self?.inflightSemaphore.signal()
    }

    // Dispatch passes, commit, present
    commandBuffer.present(drawable)
    commandBuffer.commit()
}

This pattern bounds frames in flight and prevents the CPU from overwriting a buffer that an unfinished GPU command still uses.

These are configured targets, not benchmark promises. Actual frame time and utilization vary by Mac, resolution, display count, preset, and render mode.

Adaptive Quality

A desktop wallpaper that drains battery or spins fans would be unacceptable. Matrix Desktop monitors system conditions and degrades gracefully:

Thermal Monitoring

The app reads ProcessInfo.thermalState and adjusts rendering based on the current thermal pressure:

• Nominal: Full quality. 60fps, all 5 passes, quarter-resolution bloom.
• Fair: Full quality remains active.
• Serious: 30fps and skip bloom passes (passes 2-4). The final composite renders the scene without bloom, saving three dispatch calls and their associated texture bandwidth.
• Critical: 15fps, no bloom. Minimum viable rendering to keep the rain visible without contributing to thermal pressure.

Battery Detection

When a MacBook is unplugged, the renderer uses at least the medium tier: 30 fps with bloom skipped. Returning to AC permits full quality again unless Low Power Mode or thermal state requires a lower tier.

Sleep, Screen Saver, and Lid Close

The app observes system notifications for sleep, screen saver activation, and display power state. When those paths trigger, its renderers pause and resume on wake or deactivation.

Per-Screen GPU Selection

Matrix Desktop creates one MatrixRenderer per connected screen. Each renderer asks macOS for the MTLDevice associated with that display and falls back to the system default device when none is returned.

On the supported Apple Silicon Macs, connected displays normally resolve to the integrated Metal device, while each screen still has independent window, simulation, and renderer resources.

When displays are hot-plugged (connecting or disconnecting a monitor), the app observes NSApplication.didChangeScreenParametersNotification, tears down renderers for removed screens, and creates new ones for added screens.

Why framebufferOnly Must Be False

By default, MTKView sets its drawable textures to framebufferOnly = true. This is a Metal optimization hint that tells the driver the texture will only be used as a render target for fragment shaders in a render pass. The driver can store it in a format optimized for that specific access pattern.

Matrix Desktop uses compute shaders, not render passes. The final composite kernel writes to the drawable texture directly via texture2d<float, access::write>. This is a general-purpose texture write, not a framebuffer attachment, and it requires the texture to support arbitrary write access.

Copylet mtkView = MTKView(frame: screen.frame, device: device)
mtkView.framebufferOnly = false  // Compute shaders need write access
mtkView.colorPixelFormat = .bgra8Unorm
mtkView.preferredFramesPerSecond = 60

If framebufferOnly is left at its default of true, the compute shader dispatch fails silently or crashes on some GPU architectures. This is a common pitfall when using compute-only rendering pipelines with MTKView.

Summary

Matrix Desktop's rendering pipeline separates CPU simulation from GPU rendering, keeps its image passes in compute, uses a LUT texture so palettes are data changes, and reduces quality automatically based on system conditions. Performance measurements should be reported per Mac and display setup rather than as a universal frame-time claim.