IAMF / Eclipsa Audio (experimental)
The open immersive format from AOMedia, played through Omniphony. Decoding is solid; the Matroska track and the IEC 61937 carrier are our own inventions. Here's exactly where the edges are.
IAMF (Immersive Audio Model and Formats) is AOMedia’s open, royalty-free format for immersive sound: channel beds, ambisonic scenes and, since version 2.0, objects. Eclipsa Audio is the name it goes by on consumer gear and on YouTube. An open spec with free decoders is exactly the kind of thing we like to play with, so Omniphony can render it.
It works, and you can listen to it today. But two of the three pieces involved are things we made up ourselves, because the standards for them don’t exist yet. This page says which parts are solid, which parts are homemade, and what you can and can’t build on top of them.
The short version
| Piece | State | Standard? |
|---|---|---|
| Decoding IAMF in the engine | Works, ships in 0.7.0 on Linux, Windows and macOS | Yes: the IAMF spec, checked against the libiamf conformance vectors |
IAMF in Matroska (.mka / .mkv, A_IAMF track) |
Experimental, plays in mpv-omniphony | No. Our own draft mapping |
| IAMF over IEC 61937 (S/PDIF-style bursts) | Experimental, Linux + PipeWire only | No. A private data type we picked |
If you only remember one thing: files and streams made with the two homemade pieces are read by Omniphony and nothing else. Keep your originals.
Decoding: the solid part
The IAMF decoder lives in the IAMF decoder bridge (harletty_iamf_bridge), which the release zips and installers ship next to the other bridges. It’s built on iamf-rs, with object support (IAMF v2.0) added on our side.
- Channel-based and ambisonic mixes come out as a 7.1.4 bed. That bed lands on your rig like any other fixed-channel source: Studio’s Fixed-channel sources panel has an IAMF family of its own (Sphere, Room or Manual placement; see Speaker layout).
- Objects (IAMF v2.0) come through as real objects, with their positions and movements, and go through the render backend like any other object. Content that actually uses v2.0 objects is still rare.
- Not everything decodes yet. The libiamf conformance vectors are our yardstick, and at our last sweep a few still failed: some scalable multi-layer channel layouts, expanded layouts, and streams whose second mix is invalid. If a stream refuses to play, that’s the likely reason, and an issue with the stream attached is welcome.
The decoder is the part we’d trust. The question is how the stream gets to it.
IAMF in Matroska: our own track definition
The official home of IAMF is MP4 (ISO-BMFF). Matroska has no IAMF mapping: there’s an open request for one (matroska-specification#940), and no answer yet. We needed IAMF in .mka/.mkv for our own use, so we wrote a draft: docs/iamf-matroska-mapping.md, codec ID A_IAMF, currently at v0.2.
What that means in practice:
- It’s our proposal, not a standard. AOMedia and the Matroska working group (CELLAR) haven’t endorsed it. We’ve modelled it on the AV1 mapping and on IAMF’s own MP4 boxes so it has a fair chance of looking like whatever gets standardised, but there’s no guarantee the final mapping will match it.
- Only mpv-omniphony plays it, with
--ad=orenderand the IAMF bridge installed. Plain mpv, VLC, Kodi, FFmpeg, your TV or your streaming box will not recognise the track.mkvmergepasses it through a remux untouched, and that’s about it for third-party support. - It can change. It already did: v0.1 → v0.2 changed how seek points and the end trim are written, and some early files had to be re-muxed. A future change could mean re-muxing again.
- Nothing outside Omniphony writes it. Our converter,
scripts/iamf2mka.py, takes IAMF from an MP4 (fragmented or not) or a standalone.iamfand writes an audio-onlyA_IAMFtrack.mkvmergecan then put it next to a video. The OBUs are copied untouched, and--extractturns the track back into a standalone.iamf, byte for byte except for the temporal delimiters.
python3 scripts/iamf2mka.py input.mp4 audio.mka
mpv --ad=orender audio.mka
IAMF in MP4 doesn’t play directly in the player today: FFmpeg, which mpv uses to read MP4, splits an IAMF stream into one stream per substream, and the engine wants it whole. Convert it to Matroska first. The standalone engine, orender, also reads a raw .iamf stream (offline render, or its input pipe), which is handy for tests, though there’s no A/V sync there.
IAMF over IEC 61937: our own carrier
TrueHD, E-AC-3 and DTS can travel as IEC 61937 bursts, the format S/PDIF and HDMI passthrough use: compressed data disguised as PCM. That disguise is useful, because the bursts inherit the PCM link’s clock. IAMF has no IEC 61937 data type: the IEC hasn’t assigned one. So we made a private one.
Why bother? The setup we’re after is one machine playing the film (mpv, video) and another running the renderer next to the DAC, joined by a synchronous audio link (netjack2). Sending the IAMF bitstream rather than rendered PCM means the renderer can sit on the audio machine, and the bursts ride on that link’s clock, so there’s one clock for the whole chain and no drift.
What exists, all in 0.7.0:
- Sender: mpv-omniphony’s
--ad=iamf-iec61937wraps each IAMF unit into a burst on a PCM-shaped carrier (8 channels by default,--ad-iamf-iec61937-channels, at the track’s own sample rate). - Receiver: the renderer’s PipeWire input as a carrier sink,
render.live_input.carrier_channels: Nin the config, or the carrier switch in Studio’s Audio input panel. - The format itself is written down in
docs/iamf-transport-plan.md.
And the limits, which are many:
- Private, homemade, and only Omniphony understands it. The data type value (
0x1D) is one we chose; we couldn’t read the IEC’s table of assigned values (it’s paywalled), so we can’t promise it doesn’t collide with something, and it may move. - No AV receiver, TV or soundbar will ever decode it. It won’t cross HDMI through a TV or an AVR either: nothing advertises IAMF passthrough, and an eARC hop mutes unknown types. It’s for a link between two machines you control, both running Omniphony’s stack.
- Linux and PipeWire only for live use.
--ao=pcmcan also write the carrier to a file or a pipe. - Never point it at real speakers. On a loudspeaker those words are full-scale noise. That’s why mpv refuses to send the carrier to the default sink: you have to name the carrier sink with
--audio-device=pipewire/<sink>. - Bit-exact or nothing. No volume, no mixing, no resampling anywhere on the path. A 44.1 kHz track won’t cross a 48 kHz link.
- Not proven on the setup it was built for. On one machine’s PipeWire graph the live render is byte-identical to the offline render of the same stream. The two-machine netjack2 link, its channel plan and its lip-sync delay haven’t been validated end-to-end yet.
So when is this usable?
Yes, go ahead:
- You have IAMF content (your own encodes, MP4 files you’re allowed to use) and want to hear it on your speakers or your headphones through Omniphony. Convert it to
.mka, play it with mpv-omniphony on the machine that renders. That path is the one we use ourselves. - You want to poke at the Matroska mapping, try it on your files, or argue about it in #940. That’s what a draft is for.
- You run Linux on both ends, like tinkering with PipeWire and netjack2, and want to help prove the carrier on a real two-machine link. We’d love the reports.
No, don’t:
- Don’t convert a library to
A_IAMFand throw away the MP4 or.iamfit came from. The.mkais a working copy for one player, in a format that may change. - Don’t share
A_IAMFfiles expecting other people’s players to read them, unless they run mpv-omniphony too. - Don’t plan an AVR, a TV or any HDMI gear into an IAMF chain built on our carrier. It can’t work, and that won’t change unless the IEC assigns IAMF a real data type and devices adopt it.
- Don’t expect certification of any kind: this is an independent implementation, not affiliated with AOMedia or with the Eclipsa Audio programme.
When the real standards land (an official Matroska mapping, an IEC data type), we’ll follow them and say so here. Until then, it’s an experiment with the edges clearly marked. Come break it.