Ethical Risks of Intelligent Dialogue Systems from View of European Trends on AI

Abstract

AI technology brings major transformation to the human society, while it may conflict with the values (ethics) of the human society, depending on how it is used. With this background, a number of “AI principles” have been developed as a code to AI ethics by major countries and institutions.

Intelligent dialogue systems (IDSs), as a part of AI technologies, will play an important role as an interface of "human-machine (AI system)" in the future, and therefore, developers and users of IDSs need to be aware and manage the risk of ethical aspects of IDSs, including those related to "human-machine" relationship.

The purpose of this paper is to clarify the possible ethical risks (including legal risks) of IDSs through analysis of AI regulation in Europe, which leads the discussion on AI ethics in the world, with its reference of its social acceptance and cultural background regarding “human-machine” relationship.

For that purpose, this paper first shows the direction of future technological development of the IDSs, and then identifies the characteristics of AI principles of Europe, which may be affected by its cultural background related to the "human-machine" relationship. Then, through analysis of the recently proposed European AI Act with considering future technological development of IDSs and its characteristics, this paper clarifies the possible ethical risks for the future development and practical application of IDSs are not only those related to shared human rights such as fairness/non-discrimination and privacy but also those related to the cultural differences on the view of “human-machine” relationship.