Autonomous aerial robot swarms promise transformative applications, from planetary exploration to search and rescue in complex environments. However, navigating these swarms efficiently in unknown and cluttered spaces without bulky sensors, heavy computation or constant communication between robots remains a major research problem. This paper introduces an end-to-end approach that combines deep learning with first-principles physics through differentiable simulation to enable autonomous navigation by several aerial robots through complex environments at high speed. Our approach directly optimizes a neural network control policy by backpropagating loss gradients through the robot simulation using a simple point-mass physics model. Despite this simplicity, our method excels in both multi-agent and single-agent applications. In multi-agent scenarios, our system demonstrates self-organized behaviour, which enables autonomous coordination without communication or centralized planning. In single-agent scenarios, our system achieved a 90% success rate in navigating through complex unknown environments and demonstrated enhanced robustness compared to previous state-of-the-art approaches. Our system can operate without state estimation and adapt to dynamic obstacles. In real-world forest environments, it navigates at speeds of up to 20 m/s, doubling the speed of previous imitation-learning-based solutions. Notably, all these capabilities are deployed on a budget-friendly US$21 computer, which costs less than 5% of the GPU-equipped board used in existing systems.
自主空中机器人集群技术有望诸多行业应用中带来革命性改变。然而，如何在未知杂乱空间中实现高效集群导航，同时避免依赖高负载传感器、高耗能机载计算或机器人间的持续通信，仍是亟待解决的关键科学难题。本文提出一种端到端解决方案，通过可微分模拟将深度学习与第一性原理物理建模相结合，实现了多架空中机器人在复杂环境中的高速自主导航。我们的方法采用简化的质点物理模型，通过模拟器反向传播损失梯度直接优化神经网络控制策略。尽管模型简单，该方法在单机与集群场景中均表现卓越：在集群应用中，系统展现出无需通信或集中规划的自组织行为；在单机任务中，系统在未知复杂环境中的导航成功率高达90%，其鲁棒性显著优于现有最优方案。该系统无需状态估计模块即可运行，并能自适应动态障碍物。在真实森林环境中，其导航速度高达20米/秒，达到同类模仿学习方案的两倍。尤为突出的是，所有功能均部署在单价仅150元（不足现有GPU系统5%成本）的微型计算机上实现。
@article{zhang2025learning,
title={Learning vision-based agile flight via differentiable physics},
author={Zhang, Yuang and Hu, Yu and Song, Yunlong and Zou, Danping and Lin, Weiyao},
journal={Nature Machine Intelligence},
pages={1--13},
year={2025},
publisher={Nature Publishing Group}
}This project is supported by the National Natural Science Foundation of China (Grant Nos 62325109 and U21B2013 to W.L. and Grant No. 62073214 to D.Z.). We thank SJTU SEIEE ⋅ G60 Yun Zhi AI Innovation and Application Research Center for indoor experiment support, J. Li for initial efforts in the multi-agent experiments, and L. Zhang and F. Yu for helping with the experiments and the valuable discussions. Further detailed information about the project can be contacted via henryhuyu@sjtu.edu.cn.
本项目由国家自然科学基金资助（项目编号：62325109和U21B2013资助林巍峣.，项目编号：62073214资助邹丹平）。感谢上海交通大学电子信息与电气工程学院·G60云智人工智能创新与应用研究中心上海松江测试场的实验支持，感谢李嘉晔在多智能体实验中的前期工作，感谢张临佐和郁枫在实验操作和宝贵讨论中提供的帮助。更多项目细节可咨询henryhuyu@sjtu.edu.cn.