7.5 Hand-detection data

When hand detection is switched on, an EGO1GS records its detections as a separate channel inside the same .mcap, providing a capture-time quality reference. This page is the engineering view of that channel; the operator view is chapter 13.

The channel

Topic	Schema	Contents
/ego/camera/0/annotations	foxglove.ImageAnnotations (embedded in the file)	One box per detected hand, labeled left / right

• Coordinates are in camera 0's image (the right eye), and the messages share camera 0's timeline. That is what lets Foxglove overlay the boxes on the video.
• Each box carries a left / right label. Detection targets the wearer's hands, but other people's hands are not guaranteed to be ignored.
• The channel is absent when detection is off, and on EGO1 (which has no detector). Consumers must tolerate a missing channel.

Overlaying in a viewer

• Foxglove Studio: open the recording, add an Image panel on /ego/camera/0, and enable /ego/camera/0/annotations in the panel's annotation topics. The shipped Ego-Foxglove-Layout.json already has the image panels.
• visio-display: boxes are supported only when forwarding a live stream to Foxglove. For recorded boxes, open the .mcap directly in Foxglove (left green, right cyan).

Reading it in Python

The schema is embedded in the file, so resolve the message class from the channel rather than importing a fixed type:

from visio_schema import read_mcap, message_class

for msg, channel in read_mcap("capture.mcap"):
    if channel.topic == "/ego/camera/0/annotations":
        ann = message_class(channel.schema_name)()
        ann.ParseFromString(msg.payload)
        print(msg.log_time, ann)   # boxes with left / right labels, in camera-0 pixels

Match a detection to a video frame by timestamp (both run on the monotonic clock — see 7.2 Timestamps & Unix time); the nearest earlier camera-0 frame is the one the detection was computed on.

Availability

Hand detection is a standard EGO1GS feature since firmware 1.1.3, toggled in the app since app 1.1.3. It ships off. Accuracy and throughput figures are not published yet; treat the boxes as a capture-time quality aid rather than ground truth. See 13. Hand detection.