Skip to main content

Metadata Model

This section is normative.

The Metadata Model defines the structured descriptions of bodies, sensors, devices, cameras, robots, objects, and retargeting relationships used throughout the MIND Standard. Metadata enables correct interpretation, alignment, calibration, and fusion of multimodal human, robotic, biosensing, and agent-control data.

Metadata as a Distinct Category​

Metadata objects MUST NOT be treated as modalities or streams.

Metadata describes what a stream represents—not the data itself.

Metadata types include:

  • SkeletonProfile
  • CameraProfile
  • DeviceProfile
  • BiosensorProfile
  • RobotModelProfile
  • ObjectProfile
  • RetargetMap

Each metadata object MUST be a standalone entity referenced by streams or events.

Metadata Identifiers​

Metadata identifiers MUST follow the same structured, namespaced, versioned format as modalities:

<namespace>.<family>/<name>@<version>

Valid examples:

MIND.skeleton/HumanDefault@1.0.0
MIND.camera/Pinhole@1.0.0
LAB_X.biosensor/EEG64Channel@2.1.0
ROBOT_CO.robot/6DOFManipulator@3.2.5

Unknown namespaces MUST be preserved.

Canonical Metadata Families​

Metadata MUST belong to one of the following canonical families:

  • skeleton/
  • camera/
  • device/
  • biosensor/
  • robot/
  • object/
  • retarget/

Additional families MAY be added via metadata extensions.

Metadata Requirements for Streams​

Every stream MUST declare at least one metadata reference via:

stream.metadata_refs: [metadata_id]

Examples:

  • SkeletalPose → SkeletonProfile
  • Pose2D → CameraProfile
  • EEG → BiosensorProfile
  • IMURaw → DeviceProfile
  • AgentCommand → RobotModelProfile

A stream without required metadata MUST invalidate the Container.

Hierarchical Metadata Composition​

Metadata objects MAY contain hierarchical subcomponents.

Examples:

  • A SkeletonProfile containing separate hand/face models
  • A CameraRigProfile containing multiple cameras with transforms
  • A BiosensorRig containing multiple sensors
  • A RobotModel composed of links, joints, and end effectors

Hierarchy MUST form a Directed Acyclic Graph (DAG).

ASCII example:

RobotModel
 ├── Link 1
 ├── Link 2
 │     └── Joint A
 ├── EndEffector
 └── CalibrationProfile

Calibration Transforms​

Metadata MUST support calibration transforms, including:

  • local coordinate frame definitions
  • parent-child transforms
  • device-to-world transforms
  • camera intrinsics/extrinsics
  • robot joint offsets
  • biosensor placement orientations

Transforms MUST follow the canonical coordinate system (Section 4 & 5).

Temporal Validity​

Metadata MAY specify:

valid_from
valid_to

If present, a stream referring to metadata outside its validity range MUST be considered invalid.

Examples:

  • Recalibrated camera after frame 10000
  • EEG electrodes shifted after minute 12
  • Robot tool-change at timestamp T

Metadata Referencing Streams​

Metadata MAY reference streams when needed.

Examples:

  • CameraProfile referencing calibration frames
  • RobotModel referencing encoder calibration logs
  • BiosensorProfile referencing baseline recordings

These references MUST resolve strictly.

Metadata Extensions​

All metadata types MUST support the extensions field:

extensions: {
   "LAB_X.customCalib": {...}
}

Extensions MUST NOT invalidate core schemas.

Summary​

The metadata model provides:

  • namespaced, versioned metadata types
  • required metadata for streams
  • hierarchical profiles
  • calibration transforms
  • temporal validity
  • cross-references
  • extensibility

Metadata is foundational for interpreting pose, bio, CV, XR, and robotics data in MIND.