Subtitles for the television you already own

What it is for

One viewer, one television, and a gap that broadcast subtitling does not fill.

Italian broadcasters caption some programmes and not others. Live news, regional windows, sport, talk shows, most of daytime — these arrive with no captions at all, or with captions that lag so far behind that they describe a different sentence than the one on screen. For someone who is hard of hearing, that is the difference between watching television and watching people move their mouths.

HearAble was built for exactly that case: subtitle whatever is on, now, in Italian, on the television itself — not on a phone, not on a tablet propped against the screen, and without asking the viewer to learn anything beyond one button.

Three constraints shaped every decision that follows.

• It must not break the television. The set-top box has one terrestrial tuner. If subtitling costs a tuner, or swallows a remote-control key, or blanks the picture, the viewer loses more than they gain.
• It must run in the house. No cloud speech service, no account, no upload of what someone watches in their living room.
• It must be operable by pressing one button. The viewer presses F4. Everything else — waking the recogniser, connecting, tearing down, powering off afterwards — is the system's problem.

The greeting, as it appears on the television. Four seconds, then the first real subtitle replaces it.

The first question: where does the audio come from?

Everything downstream is ordinary. Getting clean programme audio off a closed set-top box, without cost, is the part that had to be invented.

A speech recogniser needs the audio of the programme. The obvious routes were tried and each one charged a price the project could not pay.

The demux tap

The receiver runs a Hisilicon DVB stack and exposes the standard Linux DVB API. Enigma2 holds twenty-four PID filters open on demux0 while showing a channel; HearAble's tap is the twenty-fifth. It is userspace only — no kernel module, no patched image, nothing persistent.

# the whole mechanism, in five calls
open("/dev/dvb/adapter0/demux0", O_RDWR | O_NONBLOCK)
DMX_SET_BUFFER_SIZE   1 MiB
DMX_SET_PES_FILTER    pid    = <the audio PID the viewer selected>
                      input  = DMX_IN_FRONTEND
                      output = DMX_OUT_TSDEMUX_TAP
                      flags  = DMX_IMMEDIATE_START
read()                188-byte MPEG-TS packets

Three output modes were tried on the real box rather than inferred from the header file, which mattered: DMX_OUT_TS_TAP opens successfully and delivers zero bytes — it routes to the dvr device, not to this descriptor. DMX_OUT_TAP works at 127.3 kbit/s with first data after 0.27 s. DMX_OUT_TSDEMUX_TAP works at 132.8 kbit/s with first data after 0.03 s, and won on that number.

The choices that shaped it

Four decisions explain most of what the system looks like, each with the measurement behind it — a design choice without a number is a preference.

The recogniser runs on the CPU, and there is no GPU

Nemotron 3.5 ASR Streaming 0.6B, quantised q8_0, on NeMo-Speech.cpp. A GPU is faster in raw throughput, but under pacing — fed audio at the rate audio actually arrives, which is the only mode that matters for live — the advantage collapses from 5.7× to 2.2×, because a GPU cannot overlap work it has not been given yet. A fanless N95 reaches a real-time factor of 0.4848 over thirty minutes of live television without one.

The governor, not the compiler

No compute code is tuned for this machine. The speed comes from one line of sysfs: the recogniser works in bursts of roughly 105 ms every 160, and intel_pstate in powersave never sees a burst long enough to clock up.

The publication delay is zero

The tap reads the stream entering the receiver, so it runs 64 ms ahead of the picture — measured by reading two clocks in the same instant, AUDIO_GET_PTS against the PTS of the packet the PID filter is handing over. There is nothing to compensate for, so nothing is compensated.

The transcript is portable

The same sources, built on a different architecture with a different compiler and different SIMD kernels, produce a bit-identical transcript: the same 74 words, WER 0.00 %. Nothing in the chain depends on the machine it runs on, which is why the appliance could be chosen on power and price rather than on compatibility.

How it fits together

Two machines, one private cable, two small protocols going in opposite directions.

The split is deliberate. The decoder is a closed appliance with a 4.4 kernel and no room to run a speech model; the mini PC has the compute but cannot draw on the television. So each does the one thing it can: the box taps and paints, the mini PC listens and recognises.

The decoder side

One Enigma2 plugin, about two thousand lines, doing four jobs: it taps, it relays, it paints, and it wakes the other machine.

Hardware

Octagon SF8008 V3 Supreme Combo (Hisilicon), one DVB-T2/C tuner on frontend1, one DVB-S2X on frontend0

Image

openATV 7.5.1, Linux 4.4.35, Enigma2

Plugin

/usr/lib/enigma2/python/Plugins/Extensions/HearAbleOSD, loaded at WHERE_SESSIONSTART

Listens on

:8770 — /hearable (WebSocket, subtitles in) and /control (HTTP, relay control)

Persistent state

/etc/enigma2/hearable_look.json, hearable_sync.json, hearable_wol.json

Drawing on a television

The on-screen display is an Enigma2 skin built at runtime, and almost every line of it encodes a defect that was found on a real screen.

The panel is two lines of centred white text on a black band at 85 % opacity, each line about 7.5 % of screen height, touching, sitting 7 % up from the bottom edge so it stays inside the safe area at any size. That geometry is broadcast practice, not what happened to fit: it reads as one block from across a room. The OSD canvas is 1280 × 720 even on an HD service, which is a trap worth naming — laying the band out in 1080p coordinates puts it off the bottom of the screen.

What the viewer can change

A change applies immediately: the dialog is destroyed and re-instantiated rather than requiring an Enigma2 restart, which would blank the picture.

The remote control

Three keys, and nothing else taken from Enigma2.

HearAble starts off after every boot. The state is deliberately not persisted: subtitles are something the viewer asks for, and a box that comes back from a power cut already drawing on the screen is a box that has made a decision for them.

Switching off. The version is shown here because it is the one place a viewer without a terminal can read it.

The relay: sending audio without sending the programme

An earlier design streamed the whole container — video included — and required the receiving machine to hold a bit-identical copy of the media so the decoder's play position could be recovered by matching a frame off its screen. That is unaffordable on a small machine, and unnecessary for live television.

The relay instead sends only the selected audio track, still compressed: 26 kB/s of MPEG-TS produced by an ffmpeg process outside Enigma2, so a slow or absent receiver can never stall the user interface.

• The relay connects out. The mini PC listens and announces itself in its control poll. That is what lets the stream start exactly where the viewer is, and why an absent receiver costs the decoder nothing at all: no process is started, no port is held.
• The stream says where it starts. -c copy can only seek to a container boundary, so the relay begins about 0.65 s before the position asked for. The first packets of each epoch are probed for their timestamp and the excess is discarded at the receiver — both ends know a real timestamp, so the difference is simply dropped rather than corrected through a feedback loop.
• A receiver that stops asking disappears after an 8 s TTL, and the relay stops by itself.

Waking the other machine

Pressing F4 sends a Wake-on-LAN magic packet over the private link — broadcast to 10.77.0.255 on ports 9 and 7, retried up to fifteen times at twenty-second intervals, which gives a five-minute window. Measured wake time from soft-off: 32.0 s, three cycles out of three.

The mini PC side

A fanless N95 box that nobody logs into: it wakes when asked, recognises, and switches itself off when the television stops talking.

Hardware

T9 Plus, Intel N95 — 4 cores / 4 threads, 800 MHz – 3.40 GHz, L1d 128 KiB · L2 2 MiB · L3 6 MiB, 15 GiB RAM

SIMD

sse4_2, avx, avx2, fma, f16c, bmi2, avx_vnni

OS

Debian 13 trixie, Linux 6.12.107

Model

Nemotron 3.5 ASR Streaming 0.6B, q8_0, 741 548 352 bytes, sha256 a5c435f2…f429ae

Runtime

NeMo-Speech.cpp at 4f96762, ggml, CPU only — no GPU in the machine and none wanted

Network

enp1s0 = 10.77.0.2 static, private cable to the decoder. Wi-Fi for maintenance only

What actually made it fast

No compute code was optimised for this machine. The speed came from one line of sysfs.

The four units that make it an appliance

Switching itself off

The watchdog asks two questions and needs both answers to act: is anything connected to the audio port (read from /proc/net/tcp on port 9010), and is the decoder still watching (its /api/powerstate, falling back to a ping)? With no audio session it starts a timer — twenty minutes by default, three when the decoder has gone to standby, both in /etc/default/hearable — and powers the machine down. Any returning session cancels it.

The practical consequence is that the viewer never thinks about the mini PC at all. It appears when F4 is pressed and is gone a few minutes after the television goes quiet.

The status ring

The case has an RGB ring on a CH340 serial bridge. The protocol is a five-byte frame — 0xFA, mode, brightness, speed, checksum — at 10 000 baud, with brightness and speed inverted relative to what you would guess.

Surviving the network

The single genuine defect the thirty-minute torture campaign found was in the audio receiver, and only a physically unplugged cable exposed it: a socket timeout was being swallowed with continue, so a dead connection was held forever — eighteen respawns, three half-open sockets, 80 kB of unread data. The receiver now gives up after 15 s of silence and sets SO_KEEPALIVE with a 10 s idle, 5 s interval and 3 probes.

It also subtitles what you play on the box

A film from the receiver's own storage goes down the same chain — no extra component, no second code path.

Live television is the hard case and the one this page has been about. But the relay is a second reader of whatever the decoder is playing: the demux for broadcast, the file for a file. Either way it forwards only the selected audio track, compressed, from ahead of the viewer's position. The receiver sees source_type: file instead of dvb in the control response and carries on.

FILE_STALE_SUBTITLES: 0 across the run — nothing from before a pause, a seek or a change of file was ever painted afterwards. An earlier regression at v1.2.0 had already closed the same path.

Measured with the film on the box's USB storage (/media/hdd). An NVMe or internal drive is the same path as far as HearAble is concerned — it reads whatever Enigma2 is playing and never touches the storage itself — but that has not been measured and is not claimed here.

The two protocols

Deliberately small, deliberately separate, and each with one rule that shapes everything else.

Subtitles: hearable-enigma2/1

A persistent WebSocket carrying one JSON object per text frame. The producer connects out; the renderer listens. No TLS and no authentication — the private cable is the trust boundary.

Never send a cumulative transcript. A subtitle_state carries the two display lines and nothing else; the validator rejects a running transcript outright, because sending one makes traffic and memory quadratic in session length. And the producer never waits. Acknowledgements are observational — a slow renderer costs subtitle freshness, never backpressure into the recogniser. There is exactly one pending state, and a newer one replaces it.

Audio: hearable-audio-relay/1

Two channels in opposite directions, so that a slow receiver can never stall Enigma2.

The control request doubles as the announcement and the heartbeat: the receiver says where it wants the audio, how much head start it wants, and how far ahead of the viewer it has measured itself to be. Those last two are different numbers and must not be confused — one is a request, the other a measurement.

Where the second goes

The subtitle arrives about a second after the word is spoken. Almost all of it is the recogniser deciding, and that is a choice rather than an accident.

Measured end to end: p50 1.072 s between the audio being consumed and the subtitle state being emitted (n = 1461, p05 0.893 s). Reconstructed from the settings in use:

More than half of that second is spent not showing something that might be wrong. The stabiliser waits for a word to be confirmed, and the unconfirmed tail is not displayed at all. A faster subtitle that rewrites itself in front of the reader is worse than a slower one that does not, and for a reader who depends on the text rather than glancing at it, it is much worse.

What is not finished, and what was never measured

Recorded as gaps rather than quietly omitted, because a missing measurement that looks like a result is the most expensive thing in this project.

What has been answered

• Wake-on-LAN survives a mains cut. Everything previously known about waking the mini PC was measured from soft-off, with the power supply keeping the network card alive. With the power physically pulled from both machines, the mini PC still came up 6.4 minutes after the decoder — when F4 was pressed — with the clocks of the two machines agreeing to within a second and the power button untouched.
• The demux tap does not cost a tuner. Thirty cross-multiplex zaps, thirty successes, no lost services.
• The pipeline survives being attacked. Nineteen injected faults, nineteen unattended recoveries; thirty minutes of deliberate interference with zero lost chunks and zero stale subtitles.

How this was built

By one person, with a model as development assistant, and gated at every phase by tests and measurements.

HearAble is by Nunzio Raciti, built with Claude (Anthropic's Opus) as a development assistant. He set the goal and the constraints, made every design decision, owns the hardware, and did the testing that mattered most; Claude wrote code, tests and measurement tools to his direction.

The division is visible in the findings. Several defects of consequence were found not by a test but by the person using the real thing:

Nothing advanced on an opinion

246 automated tests, all runnable without a set-top box: the wire protocol, the receiver-side renderer and its epoch and sequence rejection, the stabiliser, the commit rules, the configuration loaders, the clock discipline, and the version consistency between package and plugin.

Purpose-built gates for everything that needs real hardware — the LAN gate, the live session, the UI lifecycle, boot cycles, Wake-on-LAN, the idle timer, the file player, a nineteen-case failure matrix. Each produces a verdict and a machine-readable block rather than a log someone has to interpret.

Campaign reports with every line labelled MEASURED, DERIVED or NOT_MEASURED, so a gap can never be read as a result.

Verdicts allowed to say no. Two investigations were closed by the numbers that closed them, and a gate producing too little evidence reports NON_MISURATO rather than passing on nothing.