← Back to projects
Embedded Graphics & Tooling

Lyric Video Engine

A lyric video authoring tool for displays the size of a postage stamp. Feed it an LRC file and it works out the song's structure, casts each line to one of eighteen animation treatments, and renders the whole thing at the panel's real resolution — then bakes it into a compressed frame pack an ESP32 streams straight to the glass.

Embedded here: the full engine. Hit play, or click any line in the Shots tab to jump to it and override how it is staged. Load a track on the Sync tab to time lyrics against a real waveform, and draw your own animation frames under Draw. Everything renders at 128 × 64 — what you see is exactly what the hardware gets.
Lyric engine (live)
Open full page Format spec
The constraint 8,192 pixels, one bit each

No grey, no subpixels, no antialiasing. Type is composed at native resolution and scaled through EPX so caps stay readable instead of turning into staircases.

The director Structure picks the staging

Repeated lines are detected as the hook and pinned to one treatment for the whole song. Everything else is cast from syllable rate, section position, and whether the words physically fit.

The payoff 3 MB down to 312 KB

Frames are XOR-delta'd against their predecessor and run-length coded. A lyric panel barely changes between frames, which is exactly what that pair of tricks is good at.

Timing Tap it, then drag it

Load the track and stamp lines by ear while it plays, then drag markers on the waveform to tighten up. Tempo and a beat grid come from an onset envelope, so snapping lands on the actual drums.

Rendering and alignment run locally in the browser. Auto-sync downloads an on-device Whisper model on first use; audio and lyric text are not uploaded.

Tech stack

ES modules Canvas 2D Bitmap fonts Whisper / WebGPU RLE + XOR delta C99 decoder ESP32 / SSD1306

How a line gets staged

Repeated text marks the chorus. Long instrumental gaps mark section breaks. Characters-per-second, normalised across the song, gives each line an energy score. Those three signals pick a pool of treatments; a seeded hash picks from the pool and avoids repeating the previous choice. Any line you disagree with can be locked to a specific treatment, and the rest re-cast around it.

Determinism

renderFrame(t) is a pure function of t — no accumulated state, no frame-to-frame carry. Randomness comes from a PRNG re-seeded per cue on every call. Scrubbing to 1:42 produces the identical bytes the exporter writes for 1:42, which is what lets the preview be trusted as a proof of the hardware output.

Two ways onto the device

  • Frame pack — every frame baked. The firmware only decodes and blits, so playback is exact, but a song is a few hundred KB.
  • Timeline — a few KB of cues and treatment IDs. The device renders live and adapts to any panel, but has to carry the renderer.

Verification

  • The C decoder in firmware/ reproduces all 1,530 frames of the demo bit-for-bit, across four panel and packing configurations.
  • The RLE codec round-trips 400 randomised payloads plus the degenerate cases.
  • Every treatment is rendered across each cue of the demo and checked for blank or near-empty frames.

Following Spotify

Both halves sync to live playback. The browser uses PKCE, so no client secret sits in the page; the ESP32 uses a refresh token, so it never runs a browser. Both poll slowly and interpolate with the local clock in between — /v1/me/player is rate limited and its progress_ms is already stale on arrival, so between anchors the local clock is the more accurate of the two.