Binaural for headphones

Not everyone has a ceiling full of speakers. Flip Omniphony into binaural mode and the entire scene collapses into a plain stereo pair for your headphones — every bed channel and every object filtered through a head-related transfer function (HRTF), with interaural time delay, room reflections, and live head tracking so sounds stay put when you turn your head.

This is a separate output stage, not a render backend. When it’s on, the whole VBAP / speaker chain is bypassed and each channel is rendered straight to two channels (FL/FR). Your speaker layout is ignored while you’re in headphones mode.

Hear it in ten seconds

The bundled demo is the binaural stage — no media file, no speakers, no proprietary decoder:

cd omniphony-renderer
./scripts/demo.sh            # a source sweeps around your head, in stereo

It renders the rotating 7.1.4 demo bed through the headphone stage to your default output. No audio device? ./scripts/demo.sh file pipes the same raw stereo to ffplay.

What happens to each sound

Per channel, per block, the stage runs roughly:

position → rotate by your head pose → (azimuth, elevation, distance) → air-absorption low-pass → distance gain → per-ear time delay (ITD) → HRIR convolution → early room reflections → shared late-reverb tail → mix into [L, R]

The reflections and reverb aren’t decoration — they’re what pulls the sound out of your head. A bone-dry HRTF render almost always sounds like it’s between your ears; a little room puts it back in the world.

Turning it on

There’s no CLI flag — binaural lives entirely in config (and over OSC). The relevant keys sit under render.binaural:

render:
  binaural:
    output_mode: binaural      # "speaker" (default) or "binaural"
    hrir_source: saf           # which HRTF to use (see below)
    unit_scale_m: 3.0          # metres per scene unit — sets apparent distance
    reflections:
      enabled: true
      level: 0.4
    reverb:
      enabled: true
      level: 0.2
      rt60_s: 0.3

In Studio, the same thing is the Headphones panel: a speaker/headphones toggle up top, then the HRTF picker and all the room controls. Flipping it there is live and saved to config.

Playing films? In mpv-omniphony, ad_orender fixes the channel count when the decoder starts, so binaural has to be active in config before playback begins — switch modes, then (re)start mpv. It’s also worth setting audio-buffer=0.05 in mpv.conf so head movements track tightly.

Choosing an HRTF

The HRTF is the “shape of your ears” filter, set by render.binaural.hrir_source:

Here’s the thing about generic HRTFs: they rarely give you good up/down. Elevation cues are spectral notches carved by your pinna, and a stranger’s ears won’t match. If height feels flat or the image sits too high, go HRTF shopping — Studio’s HRTF panel has a Browse… button that pulls SOFA measurements straight from sofacoustics.org so you can audition different subjects until one clicks. (Omniphony never redistributes HRTF data; it ships only the openly-licensed KEMAR set.)

Head tracking

Static binaural is fine; tracked binaural is the magic — the scene stays fixed in the room while your head turns. Omniphony takes head orientation over OSC from any source you point at it.

Set an address and format under render.binaural.head_tracking (an empty address disables tracking):

render:
  binaural:
    head_tracking:
      osc_address: /gamerotationvector
      format: auto        # auto | quat | rotvec | euler

• auto figures it out: 4 floats → quaternion [x, y, z, w], 3 floats → rotation vector.
• quat [x, y, z, w], rotvec [x, y, z], euler [yaw, pitch, roll] in degrees.

Pose arrives on the renderer’s OSC port (UDP 9000 by default), so make sure your firewall lets that in. In Studio the Head tracking sub-panel gives you a Recenter button (whatever way you’re facing becomes “front”), a smoothing slider, an invert toggle, and a live yaw / pitch / roll readout — and the 3D head in the scene turns with you.

Where the orientation comes from

• A phone — Sensors2OSC on Android. Enable its Game Rotation Vector sensor (gyro + accelerometer, no magnetometer drift) and aim it at your-machine:9000.

• A dedicated tracker — the nx-tracker-osc CLI reads a Bluetooth head tracker and emits the same /gamerotationvector feed:

  nxosc run --profile omniphony --osc-address /gamerotationvector --osc-target 127.0.0.1:9000

  (It can also drive an IEM SceneRotator with --profile scenerotator.)

Under the hood, over OSC

Everything above is reachable from any OSC client — /omniphony/control/output_mode, /omniphony/control/binaural/hrir_source, the /head/tracking/* controls, and /omniphony/control/head/recenter. Going the other way, the engine streams head pose back on /omniphony/state/head_pose (w x y z, ~30 Hz) so a visualizer can follow along with low latency, plus a fuller binaural block inside /omniphony/state/renderer. See the OSC protocol guide for the shape of that contract.