Event-stream Dynamics Priors for Low-latency Visual Representation Learning
Keywords:
Neuromorphic vision, Representation learning, Event-stream processing, Continuous-time networksAbstract
The rapid evolution of neuromorphic engineering has introduced event-based cameras as a revolutionary alternative to traditional frame-based visual sensors. Unlike standard cameras that capture full images at fixed intervals, event cameras respond asynchronously to logarithmic changes in per-pixel illumination, yielding a sparse, continuous-time data stream with microsecond temporal resolution and exceptionally high dynamic range. Despite these hardware advantages, extracting meaningful representations from such asynchronous streams with minimal latency remains a profound challenge for modern computer vision architectures. Standard deep learning frameworks, optimized for dense, synchronous tensor operations, inherently compromise the latency benefits of event streams by requiring artificial temporal accumulation or framing. This paper introduces the concept of event-stream dynamics priors as a fundamental inductive bias to guide low-latency visual representation learning. By embedding physical kinematic principles directly into the architecture and optimization objectives of continuous-time neural networks, we enable the model to infer latent spatiotemporal states instantly upon the arrival of individual events. We present a comprehensive theoretical framework for these dynamics priors, detailing their integration into asynchronous learning pipelines. Extensive experimental analysis across multiple neuromorphic datasets demonstrates that integrating dynamics priors significantly reduces the time-to-accuracy metric, allowing networks to converge to robust representations at sub-millisecond latencies. The findings suggest a paradigm shift in processing asynchronous visual data, moving away from purely data-driven feature extraction towards physics-informed continuous representation learning.
References
1. Yang, Y., & Zhang, S. (2025). Pixel-wise calibration for a multi-focus microscopic 3D imaging system.Optics and Lasers in Engineering,194, 109127.
2. Fang, S., Shen, I., Wang, Y., Tsai, Y. H., Yang, Y., Zhou, S., ... & Yang, M. H. (2025). Meshllm: Empowering large language models to progressively understand and generate 3d mesh. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 14061-14072).
3. Li, Y. K., Yu, Y. Z., Liu, Y. L., & Gou, C. (2022, October). MS-GCN: Multi-stream graph convolution network for driver head pose estimation. In 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC) (pp. 3819-3824). IEEE.
4. Wang, J., Xu, Y., Li, Y., Khargonekar, P., & Faruque, M. A. A. (2026). CRUISE: Vision-Language Model-Guided Uncertainty-Aware Cross-Modal Sensor Fusion for Robust Autonomous Driving. arXiv preprint arXiv:2608.09202.
5. Cong, P., Xu, Y., Ren, Y., Zhang, J., Xu, L., Wang, J., ... & Ma, Y. (2023, June). Weakly supervised 3d multi-person pose estimation for large-scale scenes based on monocular camera and single lidar. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 37, No. 1, pp. 461-469).
6. Meng, X., & Lin, J. (2026). CARTN: Counterfactual Audit and Repair for Trust-Aware Multi-Signal Streaming LiDAR Segmentation. Information Fusion, 104809.
7. Chang Liu, Tianjiao Jing, Chengcheng Ma, XuanQi Zhou, Zhengxuan Lian, Qin Jin, Hongliang Yuan, and Shi-Sheng Huang. 2025. EmoDiffTalk:Emotion-aware
Diffusion for Editable 3D Gaussian Talking Head. CoRR abs/2512.05991 (2025).
arXiv:2512.05991 doi:10.48550/ARXIV.2512.05991
8. Qu, W., Shao, Y., Meng, L., Huang, X., & Xiao, L. (2024, June). A conditional denoising diffusion probabilistic model for point cloud upsampling. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 20786-20795). IEEE.
9. Chen, Z., Feng, Y., Zhang, Y., Liu, J., Zhu, C., & Chen, A. (2022). An accurate and convenient method of vehicle spatiotemporal distribution recognition based on computer vision. Sensors, 22(17), 6437.
10. Huang, Y., Thede, L., Mancini, M., Xu, W., & Akata, Z. (2026). Structural Pruning of Large Vision Language Models: A Comprehensive Study on Pruning Dynamics, Recovery, and Data Efficiency. International Journal of Computer Vision, 134(6), 313.
11. Simonyan, K., & Zisserman, A. (2014). Two-stream convolutional networks for action recognition in videos. In Advances in Neural Information Processing Systems.
12. Li, Z., He, Z., Pan, J., Lin, S., & Peng, Y. (2026). E2Pano: Learning Event-to-Panorama Image Reconstruction. arXiv preprint arXiv:2608.00694.
13. Ye, Z., Ma, S., Yang, J., Yang, X., Yang, Y., Gong, Z., ... & Wang, H. (2026, April). Object fidelity diffusion for remote sensing image generation. In International Conference on Learning Representations (Vol. 2026, pp. 3258-3278).
14. Li, H., Wang, Y., Huang, T., Huang, H., Wang, H., & Chu, X. (2025, October). Ld-rps: Zero-shot unified image restoration via latent diffusion recurrent posterior sampling. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 13684-13694). IEEE.
15. Li, Y., Tan, G., & Gou, C. (2024). Cascaded iterative transformer for jointly predicting facial landmark, occlusion probability and head pose. International Journal of Computer Vision, 132(4), 1242-1257.
16. Zhang, W., Huang, M., Zhou, Y., Zhang, J., Yu, J., Wang, J., & Xu, L. (2024). Both2hands: Inferring 3d hands from both text prompts and body dynamics. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 2393-2404).
17. Z. Wang, R. Yuan, Z. Geng, H. Li, X. Qu, X. Li, S. Chen,
H. Fu, R. B. Dannenberg, and K. Zhang, ‘‘Singing timbre popularity
assessment based on multimodal large foundation model,’’ in Proc.
33rd ACM Int. Conf. Multimedia, Oct. 2025, pp. 12227–12236, doi:
10.1145/3746027.3758148.
18. Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., & Joulin, A. (2020). Unsupervised learning of visual features by contrasting cluster assignments. In Advances in Neural Information Processing Systems.
19. Tian, Y., & Yang, Z. (2026, May). SAEC: scene-aware enhanced edge-cloud collaborative industrial vision inspection with multimodal LLM. In ICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 19747-19751). IEEE.
20. Wang, Y., Yu, H., Tang, J., Gao, Y., Wang, X., Yue, Y., & Yang, Y. (2026). FilterGS: Traversal-Free Parallel Filtering and Adaptive Shrinking for Large-Scale LoD 3D Gaussian Splatting. arXiv preprint arXiv:2603.23891.
21. Yang, Z., Liang, B., & Ji, W. (2021). An intelligent end–edge–cloud architecture for visual IoT-assisted healthcare systems. IEEE internet of things journal, 8(23), 16779-16786.
22. Liang Mi, Weijun Wang, Wenming Tu, Qingfeng He, Rui Kong, Xinyu Fang, Yazhu Dong, Yikang Zhang, Yuanchun Li, Meng Li, Haipeng Dai, Guihai Chen, and Yunxin Liu. 2025. Empower Vision Applications with LoRA LMM. In ACM European Conference on Computer Systems. 261–277.
23. Chen, L., Wang, J., Mortlock, T., Khargonekar, P., & Al Faruque, M. A. (2025, June). Hyperdimensional uncertainty quantification for multimodal uncertainty fusion in autonomous vehicles perception. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 22306-22316). IEEE.
24. Qu, W., Wang, J., Gong, Y., Huang, X., & Xiao, L. (2025, June). An end-to-end robust point cloud semantic segmentation network with single-step conditional diffusion models. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 27325-27335). IEEE.
25. Wang, Y., Li, F. W., & Wang, J. (2026, May). Motion-Guided Semantic Alignment with Negative Prompts for Zero-Shot Video Action Recognition. In ICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 10712-10716). IEEE.
26. Fan, J., Liang, Y., Liu, L., Huang, S., & Zhang, L. (2023). RCA-NOC: Relative contrastive alignment for novel object captioning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 15510–15520). IEEE. https://doi.org/10.1109/ICCV51070.2023.01422
27. Lyu, S., Mao, S., & Ma, Y. (2025). Unified Long Video Inpainting and Outpainting via Overlapping High-Order Co-Denoising. arXiv preprint arXiv:2511.03272.
28. Li, Q., Liu, Z., Luo, W., Luo, T., & Hou, C. (2026). Correcting Visual Blur Induced by Attention Distraction to Reduce Hallucinations: Algorithm and Theory. arXiv preprint arXiv:2605.24602.
29. Liu, Y., Cai, Q., Wang, C., Yang, J., Fan, H., Dong, J., & Chen, S. (2024). Geometry-enhanced attentive multi-view stereo for challenging matching scenarios. IEEE Transactions on Circuits and Systems for Video Technology, 34(8), 7401-7416.
30. Hao, Y., Xu, M., Ye, C., Qin, J., Lu, S., Qin, Y., & Han, X. (2026). LoFA: Learning to Predict Personalized Prior for Fast Adaptation of Visual Generative Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 21986-21996).
31. Yang, Z., Hu, D., Guo, Q., Zuo, L., & Ji, W. (2023). Visual E 2 C: AI-driven visual end-edge-cloud architecture for 6G in low-carbon smart cities. IEEE Wireless Communications, 30(3), 204-210.
32. Yang, D., Ai, H., Liu, J., & He, B. (2022). Absolute scale estimation for underwater monocular visual odometry based on 2-D imaging sonar. Measurement, 190, 110665.
33. Li, Y., Wang, S., & Tan, G. (2025). ID-NeRF: Indirect diffusion-guided neural radiance fields for generalizable view synthesis. Expert Systems with Applications, 266, 126068.
34. Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., & Joulin, A. (2021). Emerging properties in self-supervised vision transformers. In Proceedings of ICCV.
35. Pei, Z., & Wang, S. (2023, July). Dynamics-inspired neuromorphic visual representation learning. In International Conference on Machine Learning (pp. 27521-27541). PMLR.
36. He, H., Sun, Z., Fan, J., & Mok, P. Y. (2025). TiPGAN: High-quality tileable textures synthesis with intrinsic priors for cloth digitization applications. Computer-Aided Design, 183, 103866.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Nicholas Ho, Ka-Lun Shum
This work is licensed under a Creative Commons Attribution 4.0 International License.
