TECoSA Seminar – Runtime Assurance in AI-Based Autonomy: Between Learning and Monitoring 

Presenter: Hazem Torfah

Abstract: The deployment of autonomous systems in complex environments has increased significantly in recent years, with many relying on machine learning (ML) components for critical decision-making tasks. However, ML models are inherently brittle and prone to failures that can compromise system safety. This highlights the need for a systematic methodology to identify the conditions under which autonomy pipeline components may fail and how to detect such failures at runtime.

This talk introduces a safety assurance approach for autonomous systems based on runtime verification. Runtime verification involves extracting information from a system at runtime and evaluating it to determine whether an execution satisfies or violates a given safety property. We will present recent methods for learning and monitoring the conditions under which autonomous systems operate correctly. Specifically, we will discuss approaches for learning runtime monitors that can detect deviations from safe and expected system behavior, while remaining robust to uncertainties in both the environment and AI-based components.

Zoom-link to the seminar : https://kth-se.zoom.us/j/66857695267