Skip to main content

13. Hand detection

An EGO1GS can detect left and right hands while recording and save boxes as a separate channel in the same .mcap, providing a capture-time position reference for checking framing and selecting data. Requires EGO1GS firmware 1.1.3+; EGO1 does not support it.

Detection is off by default. Enable it when needed; when off, detection does not run or consume compute.

What it outputs​

  • One box per detected hand, labeled left or right.
  • Results share the video's timeline. Opening the .mcap directly in Foxglove overlays boxes: left green, right cyan. visio-display shows boxes only when forwarding a live stream to Foxglove.
  • The app's live preview also shows hand boxes on the right-eye image when enabled.

Turning it on and off​

  • Connect the app, tap the gear → Device settings → Hand detection → Detect hands in the image. Unsupported devices (including EGO1) or older firmware do not show the switch.
  • The setting is remembered per device, including across reboots, until turned off.
  • During recording, switching off applies immediately; switching on waits until the current recording ends. If detection cannot start, the switch returns to off.

Voice notice: hands out of view​

During recording, the device announces left hand, right hand, or both hands out of view. A single missing hand triggers one notice, with another when the situation changes. If both hands are absent, notices repeat periodically until a hand returns. There are no notices when idle. Volume and Chinese / English language follow the device's voice settings.

Tips​

  • Detection is intended for the wearer's hands; other people's hands are not guaranteed to be ignored.
  • Low light, full occlusion, or only a small visible part of a hand can prevent detection. This does not affect recording.
  • Enable it for capture tasks that need hand annotations; leave it off for ordinary recording.

Under the hood​

Availability​

Hand detection is part of the standard EGO1GS firmware since 1.1.3, and the app switch exists since app 1.1.3 (it is shown only on devices that support the feature). The detector runs on the headset's own neural accelerator — no cloud round-trip and no tethered computer — alongside recording. The wide 157° field of view is what keeps both hands in frame through a full reach envelope, which is the precondition for the detector being useful. Accuracy, throughput and inference-budget figures are not published yet; treat the feature as a capture-time quality aid and don't build acceptance criteria on numbers we haven't given you. EGO1 has no detector.

Data format​

The annotations channel, how to overlay it in a viewer, and how to read it in Python are in 7.5 Hand-detection data.