On the Evaluation Measures for Machine Learning Algorithms for Safety-critical Systems

Research Area: Uncategorized Year: 2019
Type of Publication: In Proceedings Keywords: Machine learning, Algorithms, Performance Metrics, Safety Measures, Safety-critical Systems
Authors: Mohamad Gharib; Andrea Bondavalli
Book title: 15th European Dependable Computing Conference (EDCC)
Pages: 141-144
Address: Napels
ISBN: 978-1-7281-3929-6
The ability of Machine Learning (ML) algorithms to learn and work with incomplete knowledge has motivated many system manufacturers to include such algorithms in their products. However, some of these systems can be described as Safety-Critical Systems (SCS) since their failure may cause injury or even death to humans. Therefore, the performance of ML algorithms with respect to the safety requirements of such systems must be evaluated before they are used in their operational environment. Although there exist several measures that can be used for evaluating the performance of ML algorithms, most of these measures focus mainly on some properties of interest in the domains where they were developed. For example, Recall, Precision and F-Factor are, usually, used in Information Retrieval (IR) domain, and they mainly focus on correct predictions with less emphasis on incorrect predictions, which are very important in SCS. Accordingly, such measures need to be tuned to fit the needs for evaluating the safe performance of ML algorithms. This position paper presents the authors' view on the inadequacy of existing measures, and it proposes a new set of measures to be used for the evaluation of the safe performance of ML algorithms.
Full text: CR-EDCC19.pdf

Resilient Computing Lab, 2011

Joomla - Realizzazione siti web