Agentic AI for Safety-critical Multi-drone Systems: Challenges and Opportunities

Computer Science > Artificial Intelligence
[Submitted on 19 Aug 2026]
Title:Agentic AI for Safety-critical Multi-drone Systems: Challenges and Opportunities
View PDF HTML (experimental)Abstract:Multi-drone systems are increasingly positioned for safety-critical missions such as search and rescue (SAR) and critical infrastructure monitoring. Yet, real-world adoption remains constrained not only by autonomy performance, but by the difficulty of integrating agentic behavior into professional work: operators must understand, trust, and govern automation under uncertainty, time pressure, and accountability. This position paper synthesizes the ambitions and lessons from two ongoing efforts: NAMUR, which explores LLM-supported robot control in SAR and firefighting contexts, and PERSIST, which explores persistent drone operations for monitoring and security at critical infrastructure sites. We argue that agentic AI should be approached as a socio-technical design problem, where interfaces, oversight mechanisms, and evaluation practices are as critical as algorithms. We outline a human-centered, participatory, and iterative research approach aimed at uncovering stakeholder needs, shaping agent capabilities through successive prototypes, and producing transferable proof-of-concept systems and evaluation strategies for other safety-critical contexts.
Current browse context:
cs.AI
References & Citations
Loading...
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Connected Papers (What is Connected Papers?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
alphaXiv (What is alphaXiv?)
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Hugging Face (What is Huggingface?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

* Multi-drone systems are positioned for safety-critical missions such as search and rescue (SAR) and critical infrastructure monitoring.
* Real-world adoption is constrained by the difficulty of integrating agentic behavior into professional work.
* Operators require understanding, trust, and governance of automation under uncertainty, time pressure, and accountability.
* The paper synthesizes efforts from NAMUR, which explores LLM-supported robot control in SAR and firefighting contexts.
* The paper synthesizes efforts from PERSIST, which explores persistent drone operations for monitoring and security at critical infrastructure sites.
* Agentic AI should be approached as a socio-technical design problem.
* Interfaces, oversight mechanisms, and evaluation practices are considered as critical as algorithms.
* A human-centered, participatory, and iterative research approach is proposed.
* The goal of the research is to uncover stakeholder needs and shape agent capabilities through successive prototypes.

Multi-drone systems are increasingly used for safety-critical missions like search and rescue (SAR) and monitoring critical infrastructure. Real-world adoption is hindered by the difficulty of integrating agentic behavior into professional work, specifically regarding operator understanding, trust, and governance amidst uncertainty, time pressure, and accountability. This position paper synthesizes lessons from two ongoing efforts: NAMUR, which explores LLM-supported robot control in SAR and firefighting, and PERSIST, which investigates persistent drone operations for monitoring security at critical infrastructure sites. The paper argues that agentic AI should be treated as a socio-technical design problem where oversight mechanisms are as important as algorithms. It proposes a human-centered, iterative research approach to uncover stakeholder needs and shape agent capabilities through prototyping to create transferable proof-of-concept systems.

The framing positions agentic AI not merely as an algorithmic advancement but as a challenge in socio-technical design, implicitly shifting focus from technical performance to human integration and control within high-stakes environments. This mirrors the recognized gap between AI capability demonstrations and real-world operational deployment. The synthesis of NAMUR (SAR/firefighting) and PERSIST (infrastructure monitoring) suggests a pattern where safety criticality necessitates explicit mechanisms for human oversight; the risk is not just technical failure but the delegation of responsibility in ambiguous scenarios. The proposed methodology—participatory, iterative prototyping—addresses the systemic inertia often found in engineering projects by prioritizing stakeholder needs upfront. The implication is that trust and accountability are prerequisites for utility, suggesting that current safety paradigms often fail to account for dynamic human-agent interaction under pressure. The challenge lies in operationalizing "socio-technical design" rigorously; how do established governance structures translate into executable agentic interfaces that genuinely reflect operator intent while managing the inherent uncertainty of autonomous action? What specific evaluation metrics can bridge the gap between algorithmic success and perceived human safety during these iterative development cycles?

Agentic AI for Safety-critical Multi-drone Systems: Challenges and Opportunities

Facts Only

Executive Summary

Full Take

Agentic AI for Safety-critical Multi-drone Systems: Challenges and Opportunities

Facts Only

Executive Summary

Full Take