Security risks of machine learning systems and the usefulness of their classification by failure mode approach

Abstract

Machine learning is an inductive methodology which automatically extracts the relationship among data from huge amount of input/output samples. Recently machine learning systems have been implemented deeply in social infrastructure. However, machine learning models have specific vulnerabilities and relevant security risks that conventional systems do not have. Overall picture of those security risks has not been clarified sufficiently, and there has been no consensus about the taxonomy of the vulnerabilities and the security risks. This paper clarifies the source of the difficulty of machine learning security and finds the usefulness of the failure mode approach to capture the security risks comprehensively. Based on this approach, we describe the primary vulnerabilities and related attacks by classifying them on three axes, i.e. (1) presence or absence of an attack, (2) location of vulnerabilities, and (3) functional characteristics to be lost. Lastly, we consider the points for future utilization of machine learning in society.