NWB Export

The jabs-cli convert-to-nwb command converts a JABS pose estimation HDF5 file to NWB (Neurodata Without Borders) format using the ndx-pose extension (plus ndx-jabs for segmentation contours).

pip install "jabs-behavior-classifier[nwb]"

The extra adds pynwb, ndx-pose, ndx-jabs, and ndx-multisubjects as dependencies.

Two output modes are available. Choose the mode based on how the files will be used:

Command usage

jabs-cli convert-to-nwb INPUT_PATH OUTPUT [OPTIONS]

Examples

# One NWB file per identity (default; recommended for DANDI upload)
jabs-cli convert-to-nwb session_pose_est_v6.h5 session.nwb --subjects subjects.json

# A single multi-subject file holding every identity
jabs-cli convert-to-nwb session_pose_est_v6.h5 session.nwb --subjects subjects.json --multisubject

# Also set session start time and experimenter
jabs-cli convert-to-nwb session_pose_est_v6.h5 session.nwb \
    --subjects subjects.json --session-metadata session.json

# Leave segmentation contours out of a pose file that has them
jabs-cli convert-to-nwb session_pose_est_v6.h5 session.nwb \
    --subjects subjects.json --no-segmentation

--subjects appears in every example because the conversion fails without it; see Subjects JSON format.

Subjects JSON format

Pass a JSON file to --subjects to attach per-animal biological metadata to the NWB output. Keys are identity names: use external IDs from the pose file when present (e.g. "mouse_a"), or subject_1, subject_2, … (1-based) when the pose file has no external IDs. subject_1 is identity index 0.

An optional name field renames the identity — see Naming identities.

{
  "subject_1": {
    "subject_id": "M123",
    "sex": "M",
    "species": "Mus musculus",
    "age": "P70D",
    "genotype": "WT",
    "strain": "C57BL/6J"
  },
  "subject_2": {
    "subject_id": "M124",
    "sex": "F",
    "species": "Mus musculus",
    "age": "P72D",
    "genotype": "Shank3+/-",
    "strain": "C57BL/6J"
  }
}

housing_id and test_chamber are generic JAX study identifiers. cfi, cfi_norm, pll and diet are specific to the JAX aging study. All are optional. When any aging field is present the file's NWBFile.subject is written as an ndx-jabs JaxAgingSubject; when only housing_id or test_chamber is present it is a JaxStudySubject; otherwise it is a plain Subject. None and blank values are left out, and a non-numeric cfi, cfi_norm or pll is rejected. In multisubject mode these fields are not added to the SubjectsTable; they are kept only in the jabs_metadata copy of the subject. The aging study's batch and tester are session-level fields; see Session metadata JSON format.

In per-identity mode (the default), subject metadata is written to both the standard NWBFile.subject field and the jabs_metadata scratch field. In multisubject mode it is written to a SubjectsTable (one row per subject) and to jabs_metadata (see below).

The validation above is a pre-flight check that mirrors nwbinspector's subject checks, which DANDI treats as blocking. It is not a substitute for the real thing - run nwbinspector against the converted files before uploading:

nwbinspector session_subject_1.nwb --config dandi

Naming identities

A pose file with no external identities leaves its animals called subject_1, subject_2, … . That name is not cosmetic — it names the PoseEstimation container, the per-identity output file ({stem}_{identity_name}.nwb) and the bounding box series. subject_id does not change any of them; it only labels the Subject.

The name field supplies the external identity such a pose file is missing. A pose file that already carries external_identities normally keeps them, and needs none of this.

To name an identity, give its entry a name:

{
  "subject_1": {
    "name": "NV1-B2A",
    "subject_id": "M123",
    "species": "Mus musculus",
    "sex": "M",
    "age": "P70D"
  }
}

That writes session_NV1-B2A.nwb holding an NV1-B2A container, whose Subject is M123. Use the same name and subject_id if you want them to agree.

• The key still identifies which identity you mean, so it stays subject_1 (or the pose file's external ID) even though the entry renames it.
• Renaming some identities and not others is fine; the rest keep the names they had.
• Names must be unique, and are sanitized for use as container names — any character that is not alphanumeric, _ or - becomes _, so NV1/B2A is stored as NV1_B2A and a warning says so.
• Whatever it resolves to is recorded as that identity's external_ids entry in jabs_metadata, which is the field the reader restores identity names from. Identities you leave unnamed keep their subject_N placeholder, and that placeholder is what lands in external_ids for them.
• A pose file that already has external identities can be renamed the same way, keyed by the existing external ID, but normally you would leave those names alone.

Session metadata JSON format

Pass a JSON file to --session-metadata to set NWB session-level fields. This is the primary way to specify session_start_time, which is not currently stored in JABS pose files and otherwise defaults to the time the export was run.

{
  "session_start_time": "2024-03-15T10:30:00-05:00",
  "experimenter": ["Jane Smith", "John Doe"],
  "lab": "Kumar Lab",
  "institution": "The Jackson Laboratory",
  "experiment_description": "Open field test",
  "session_id": "session_001",
  "keywords": ["open field", "mouse", "behavior"]
}

All fields are optional. Unknown keys are ignored with a warning.

Publishing to the DANDI archive

JABS NWB output is intended for the EMBER archive — the data archive for the NIH BRAIN Initiative's Brain Behavior Quantification and Synchronization (BBQS) program. EMBER is operationally distinct from DANDI but runs on DANDI infrastructure at dandi.emberarchive.org, so DANDI's validation rules apply.

Uploads are validated with nwbinspector, and any CRITICAL finding blocks the upload. The guidance below is about clearing that gate.

Requirements at a glance

1. Prepare the metadata files

Write a subjects.json covering every identity in the pose file (see Subjects JSON format), and a session.json with at least session_start_time (see Session metadata JSON format). Pose files do not record a session start time, so without it the export falls back to the time the conversion ran, which is not the recording time.

2. Convert

jabs-cli convert-to-nwb session_pose_est_v6.h5 session.nwb \
    --subjects subjects.json \
    --session-metadata session.json

The converter validates subject metadata before writing anything. If a required field is missing or malformed it reports every problem across every identity at once and writes no files:

Error: Subject metadata required by the DANDI archive is missing or malformed:
  subject_1: species is missing; sex is missing; age or date_of_birth is missing
  subject_2: sex 'male' must be one of 'M', 'F', 'O', 'U'

A --subjects key that matches no identity is reported as a warning naming the valid identity names, which is the usual explanation for an identity that looks like it was given metadata but reports it as missing.

3. Link the videos

To publish the videos the poses were estimated from, the NWB files must point to them: DANDI accepts a video only when an NWB file references it, and the converted pose files do not. Videos uploaded on their own would be disconnected from their sessions.

jabs-cli link-nwb-video /path/to/nwb_files /path/to/videos

Each <name>.nwb in the first directory is matched with the <name>.mp4 of the same name in the second. The two directories can be different or the same directory; subdirectories are not searched. For each match the command adds an ImageSeries named video to acquisition whose external_file is the path to the video, relative to the NWB file. The video is referenced, not copied or changed, and the pose data in the NWB file is not touched.

• Before linking, the video's frame count and frame rate are checked against the pose data. A video that disagrees is reported and its NWB file is left unchanged. Use --no-verify to skip the check.
• A file that already links to its video is skipped, so the command is safe to re-run.
• Use --dry-run to see what would be linked without modifying any file.
• The exit status is 1 if any file could not be linked, for example because its video is missing.
• Requires pynwb 4.0 or newer.

The relative path is resolved against the directory of each NWB file, both by dandi organize and by nwbinspector, which reports a CRITICAL finding for a link that does not resolve. Link the files once the NWB and video directories are where they will stay until the next step, because moving either one afterwards breaks the links.

Then let DANDI gather the files into the dandiset. --update-external-file-paths is what makes dandi organize follow the links:

dandi organize -d /path/to/dandiset --files-mode copy \
    --update-external-file-paths --media-files-mode copy /path/to/nwb_files

This copies each video into the dandiset beside its NWB file and rewrites the link inside the NWB file to point at that copy. Each NWB file gets its own copy of the video, and dandi organize adds an _image suffix to the names of NWB files that contain an ImageSeries.

4. Validate locally before uploading

JABS's pre-flight check mirrors nwbinspector's subject rules, but it is not a substitute for running the real validator — it checks subject metadata, not the rest of the file. Run nwbinspector on each output before you upload:

pip install nwbinspector
nwbinspector session_subject_1.nwb --config dandi

A clean result reports no CRITICAL findings. See Expected warnings for the ones that are safe to ignore.

5. Upload

Follow the DANDI upload documentation, pointing the DANDI CLI at the EMBER instance rather than the main archive. Upload all per-identity files from a session together — the JABS reader needs the full set of siblings to reassemble the session (see Reading per-identity files).

What JABS checks, and what it does not

Passing the JABS check is necessary but not sufficient — always run nwbinspector before uploading.

Expected warnings

These appear in a normal, acceptable export and do not block upload:

• Missing session-level metadata, when --session-metadata is omitted: check_experimenter_exists, check_institution, check_keywords, check_experiment_description, and a missing subject description. All are best-practice suggestions rather than blocking findings.

Limitations for archive submission

• --multisubject output cannot be validated. The file depends on the ndx-multisubjects extension, and nwbinspector fails to read it, so the archive cannot validate it. It also does not populate NWBFile.subject, which DANDI's required subject checks read. Use the default per-identity mode for anything destined for the archive.
• Segmentation data is not exported. See Data not exported.

Output modes

Per-identity files (default)

One NWB file is written per animal. The OUTPUT path is used as a naming template; files are written as {output_stem}_{identity_name}.nwb in the same directory. The OUTPUT path itself is not created.

session_subject_1.nwb   ← identity 0 + all objects
session_subject_2.nwb   ← identity 1 + all objects
session_subject_3.nwb   ← identity 2 + all objects

This is the most standard output. Each file contains exactly one animal, so NWBFile.subject is populated with that animal's biological metadata from --subjects. Any standard NWB tool — including the DANDI archive — can read the subject field directly without knowing anything about JABS.

Identity names in the filenames come from external_ids in the pose file (sanitized for filesystem compatibility), or from the name field in --subjects when the pose file has none — see Naming identities — falling back to subject_1, subject_2, … when neither supplies one. Static and dynamic objects are written to every per-identity file identically, since they are session-level data.

Reading per-identity files

The JABS reader re-assembles per-identity files transparently. Point it at any one sibling file; it detects the per_identity_files flag in jabs_metadata, finds all siblings, and merges them into a single result with all identities in their original order.

Multisubject single file (--multisubject)

All identities from the recording session are written into a single, self-contained NWB file at OUTPUT, using the ndx-multisubjects extension.

session.nwb
  └── all identities + all objects + a SubjectsTable listing every subject

The file is an NdxMultiSubjectsNWBFile (a drop-in NWBFile subclass). Because standard NWB's NWBFile.subject only holds one subject, multiple subjects are instead described by a SubjectsTable (one row per animal) stored in acquisition. The pose data itself is laid out in processing/behavior exactly as in a per-identity file, and the full, lossless JABS round-trip still rides on the jabs_metadata scratch field.

This mode is intended for sharing a whole session as one artifact. Reading it back with the JABS reader returns all identities directly (no sibling files are involved).

Subject metadata by mode

jabs_metadata.subjects always carries the full dict for all identities, in both modes. This makes each per-identity file self-contained: the JABS reader can recover complete subject metadata from any sibling without loading the others. In multisubject mode the SubjectsTable is provided for standard NWB / DANDI consumers; JABS itself recovers subject metadata from jabs_metadata.

NWB file structure

For the full format specification — including all field definitions, jabs_metadata keys, and worked examples for static and dynamic objects — see File Formats — NWB Pose File.

The layout below shows a multisubject file with two animal identities, two static objects (corners, lixit), and one dynamic object (fecal_boli).

NdxMultiSubjectsNWBFile
├── acquisition/
│   └── SubjectsTable                      [DynamicTable] multisubject mode only — one row per subject
├── processing/
│   └── behavior/                          [ProcessingModule]
│       ├── Skeletons/                     [Skeletons container]
│       │   ├── subject/                   Skeleton — animal keypoints + edges
│       │   ├── corners/                   Skeleton — static object (4 nodes)
│       │   ├── lixit/                     Skeleton — static object (1 or 3 nodes)
│       │   └── fecal_boli/                Skeleton — dynamic object (max_count nodes)
│       │
│       ├── subject_1/                     [PoseEstimation] animal identity 0
│       │   ├── nose/                      [PoseEstimationSeries] num_frames timestamps
│       │   ├── left_ear/
│       │   └── ...
│       │
│       ├── subject_2/                     [PoseEstimation] animal identity 1
│       │   ├── nose/
│       │   └── ...
│       │
│       ├── corners/                       [PoseEstimation] static object
│       │   ├── corners_0/                 [PoseEstimationSeries] 1 timestamp
│       │   ├── corners_1/
│       │   ├── corners_2/
│       │   └── corners_3/
│       │
│       ├── lixit/                         [PoseEstimation] static object
│       │   └── lixit_0/                   [PoseEstimationSeries] 1 timestamp
│       │
│       ├── fecal_boli/                    [PoseEstimation] dynamic object
│       │   ├── fecal_boli_0/              [PoseEstimationSeries] n_predictions timestamps
│       │   ├── fecal_boli_1/
│       │   └── ...
│       │
│       ├── jabs_identity_mask             [TimeSeries] uint8 identity presence mask
│       ├── jabs_bounding_boxes_subject_1  [TimeSeries] optional, one per identity
│       └── jabs_bounding_boxes_subject_2  [TimeSeries] optional, one per identity
│
└── scratch/
    └── jabs_metadata/                     [ScratchData] JSON string (see below)

A per-identity file (the default) uses a plain NWBFile and the same processing/behavior layout, except: - The root is a standard NWBFile; there is no SubjectsTable - NWBFile.subject is populated (when subject metadata is provided) - Only one animal identity container is present - jabs_identity_mask / jabs_bounding_boxes_<identity> cover that identity only

Animal pose

Each animal identity is a PoseEstimation container in processing/behavior. The container name is the sanitized external ID from the pose file, or subject_1, subject_2, … (1-based) when no external IDs are available.

A single Skeleton named subject is shared by all animal identities and stored in the Skeletons container.

PoseEstimationSeries fields (per keypoint)

Identity mask

jabs_identity_mask is a TimeSeries that records whether each identity is present in each frame.

Bounding boxes (optional)

When the pose file contains bounding box data, one TimeSeries per identity is written with the name jabs_bounding_boxes_{identity_name}.

Format: [[upper_left_x, upper_left_y], [lower_right_x, lower_right_y]] in pixels.

Segmentation contours (optional)

When the pose file contains instance segmentation (v6 and later, and only when the file was generated with segmentation), one ContourSeries per identity is written to the behavior processing module, named jabs_segmentation_contours_{identity_name}. It sits next to that identity's PoseEstimation, so the contours, the keypoints, and the subject all describe the same animal.

ContourSeries comes from the ndx-jabs extension rather than ndx-pose, because contours are an intermediate product of JABS's own shape features, not a representation other pose tools consume. The extension's spec is embedded in every file that uses it, so readers without ndx-jabs installed can still open the file.

An animal needs more than one contour on a frame when its outline has a hole (it curls around a gap) or when an occluder splits it into disjoint parts. Because different frames need different numbers of contours and vertices, data is padded out to the largest of each with -1, and vertex_count is what says how much of each slot is real. Read it rather than scanning for the padding value:

n = contour_series.vertex_count[frame, slot]
if n:  # 0 means this slot holds no contour on this frame
    vertices = contour_series.data[frame, slot, :n, :]

is_external has no meaning where vertex_count is 0.

The contour dataset is chunked along the frame axis and gzip compressed. It is mostly padding, which compresses well: roughly 40 MB per identity per hour of 30fps video.

Segmentation presence is independent of jabs_identity_mask: an identity can have contours on a frame where its keypoints were not resolved, because JABS assigns contours from longterm_seg_id and keypoints from instance_embed_id. Use vertex_count to decide whether an identity has an outline on a given frame.

Pass --no-segmentation to leave the contours out of a pose file that has them. jabs_metadata.has_segmentation records whether they were written.

A pose file that has contours but no seg_external_flag dataset is exported without them, and a warning is logged. is_external is required and cannot say "unknown", so exporting such a file would mean guessing which contours are holes. JABS-pose writes the two datasets together, so this should only affect files produced some other way.

Identity names must not collide with names JABS generates in the same container: an identity cannot be called jabs_identity_mask, and in a --multisubject file one identity cannot be named jabs_segmentation_contours_<another identity> or jabs_bounding_boxes_<another identity>. The export fails before writing anything if they do.

Static objects

Static objects are fixed-position spatial landmarks that do not move during a session. They are read from static_objects/ in JABS pose HDF5 files (pose format v5+).

Common static objects:

Each static object is a PoseEstimation container with one PoseEstimationSeries per keypoint and a dedicated Skeleton. Nodes are named {object_name}_{i} (zero-indexed).

The constant (x, y) value is written at two timestamps spanning the session — the first and last frame — giving each series data shape (2, 2). A single-timestamp series would have shape (1, 2), whose non-time axis is longer than its time axis, which nwbinspector's check_data_orientation flags regardless of the data being genuinely static. Repeating the value at both ends leaves it unchanged while keeping the export clean.

Dynamic objects

Dynamic objects are objects whose position or count may change over time. Unlike animal pose, predictions are made only for a sparse subset of frames. Dynamic objects are available from JABS pose format v7+.

Each dynamic object is a PoseEstimation container with n_predictions irregular timestamps. One PoseEstimationSeries is written per instance slot × keypoint combination.

Instance slot occupancy is encoded in the confidence field:

• confidence = 1.0 — slot is occupied at this prediction
• confidence = 0.0 — slot is unoccupied; coordinate values are padding and must be ignored

Node naming:

jabs_metadata scratch field

Every JABS NWB file contains a ScratchData object named jabs_metadata in the NWB scratch space. Its data field is a JSON string carrying JABS-specific metadata needed for a lossless round-trip. Standard NWB fields alone are insufficient because pynwb returns PoseEstimationSeries in alphabetical order from HDF5, which would otherwise scramble the keypoint ordering.

Tools that do not use the JABS reader can parse this JSON directly to recover identity ordering, subject metadata, and object classification. (Keypoint ordering is not stored here; the JABS reader restores it from the canonical keypoint index.)

Keys

Example — multisubject file

{
  "format_version": 1,
  "multisubject": true,
  "identity_names": ["subject_1", "subject_2"],
  "num_identities": 2,
  "cm_per_pixel": 0.043,
  "external_ids": null,
  "subjects": {
    "subject_1": {
      "subject_id": "M123",
      "sex": "M",
      "species": "Mus musculus",
      "age": "P70D",
      "genotype": "WT",
      "strain": "C57BL/6J",
      "weight": null,
      "description": null
    },
    "subject_2": {
      "subject_id": "M124",
      "sex": "F",
      "species": "Mus musculus",
      "age": "P72D",
      "genotype": "Shank3+/-",
      "strain": "C57BL/6J",
      "weight": null,
      "description": null
    }
  },
  "metadata": {
    "source_file": "/data/session_pose_est_v7.h5",
    "pose_format_version": 7,
    "source_file_hash": "a3f1c8..."
  },
  "has_segmentation": true,
  "static_object_names": ["corners", "lixit"],
  "dynamic_object_names": ["fecal_boli"],
  "dynamic_object_shapes": {
    "fecal_boli": [3, 1]
  }
}

Coordinate system

All coordinates in JABS NWB files use the following convention:

This applies to animal keypoints, static object points, and dynamic object points. NWB files always store coordinates in (x, y) order.

Data not exported

Instance segmentation contours (poseest/seg_data) are exported; see Segmentation contours above. The segmentation bookkeeping datasets JABS uses to assign those contours to identities are not, because the export resolves them into per-identity contours. See File Formats — Data not exported to NWB for the full list of omitted fields.