Statistical Modelling, Exploration, and Society

Abstract

Statistical modeling and exploration (or optimization) are two essential components in modern data science. Statistical modeling uses empirical knowledge (training data) to model the target system and considers the uncertainties as random noise. Exploration uses prior knowledge to model the target system, represented as a combination of boundary conditions and objective functions, and the unexplored regions are considered as uncertainties. In real-world scenarios, the objective function, which represents people's values, often becomes the major source of uncertainty. In this paper, we reexamine various approaches in data science from the perspective of "how to represent knowledge and address uncertainty" and discuss the challenges of applying them in our society. We also touch upon non-technical approaches such as agile development and TEAL organizations as methods for dealing with uncertainties in societal values.