codec4AI

Rethinking Video Storage for AI Training: Breaking Cross-Frame Dependencies for Efficient Random Access

An efficient alternative to LeRobot v3 video storage1.45× loading throughput 58.7% less media payload DDRA (H.265), at comparable reconstruction quality.
Dependency-Aware Sparse Decoding: Reconstructs only requested frames and their required references, skipping unrelated reconstruction without transcoding source videos.
Dependency-Decoupled Random Access: DDRA restricts each interior frame to two bounding intra-coded anchors, retaining inter-frame prediction with at most three reconstructed frames per target.
arXiv ↗PDF ↗Code ↗Documentation ↗
codec4AI illustration of dual anchors and sparse video decoding
codec4AI
Zetian Song1,† Qize Yu1,† Yue Chen1 Bohan Zhou1 Bowen Ping1 Yuqi Ye1 Haocheng Tang1 Haotian Zhang1 Qizhe Wang1 Shiyu Huang2 Jiaqi Zhang1,* Siwei Ma1 Wen Gao1
  • 1Peking University
  • 2Tsinghua University

† Equal contribution    * Corresponding author

Abstract

Storage–throughput trade-off: DDRA H.265 and H.264 deliver higher loading throughput with less media payload than LeRobot v3.
Enlarge ↗

Large-scale AI training increasingly relies on massive video datasets with random temporal sampling patterns. However, existing video storage paradigms are optimized for sequential playback and employ sophisticated inter-frame reference structures to maximize compression efficiency. When applied to the random-sampling workloads of AI training, these paradigms face substantial decoding-efficiency bottlenecks. Workarounds such as independent JPEG frames and short groups of pictures (GOPs) reduce decoding work at the cost of larger storage and transfer volumes. In this paper, we identify two sources of decode amplification: sequential reconstruction of unrelated frames and reconstruction of the reference chains required by sampled targets. We address both sources by co-designing the decoding schedule and video reference structure. Dependency-aware sparse decoding reconstructs only requested frames and their required references, skipping unrelated reconstruction without transcoding the source videos. Optional offline Dependency-Decoupled Random Access (DDRA) encoding further reduces reference reconstruction by restricting each interior frame to two bounding intra-coded anchors. Together, sparse decoding and DDRA retain inter-frame prediction while bounding reconstruction to at most three frames per requested target, independent of the intra period. Experiments on a large-scale robotics corpus show that, at comparable reconstruction quality, our method achieves 1.45× the loading throughput of LeRobot v3 with 58.7% less media payload.

Method

Codec–loader co-design: optional offline DDRA encoding and online dependency-aware sparse decoding deliver requested RGB frames.
Codec–loader co-design for video-based training.Enlarge ↗

Results

Original figures & tables
Storage–throughput trade-off Exact expected decode amplification Training performance under compression Bandwidth-constrained loading Influence of intra-period Access and resource sensitivity Batch-loading efficiency Cumulative component ablation Reconstruction-quality calibration Numerical training results

Citation

@misc{song2026rethinkingvideostorageai,
      title={Rethinking Video Storage for AI Training: Breaking Cross-Frame Dependencies for Efficient Random Access},
      author={Zetian Song and Qize Yu and Yue Chen and Bohan Zhou and Bowen Ping and Yuqi Ye and Haocheng Tang and Haotian Zhang and Qizhe Wang and Shiyu Huang and Jiaqi Zhang and Siwei Ma and Wen Gao},
      year={2026},
      eprint={2610.33079},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2610.33079},
}

Open original ↗