Despite the explosive growth of activity in the field of Earth System data assimilation over the past decade or so, there remains a substantial gap between theory and practice. The present article attempts to bridge this gap by exposing some of the central concepts of estimation theory and connecting them with current and future data assimilation approaches. Estimation theory provides a broad and natural mathematical foundation for data assimilation science. Stochastic-dynamic modeling and stochastic observation modeling are described first. Optimality criteria for linear and nonlinear state estimation problems are then explored, leading to conditional-mean estimation procedures such as the Kalman filter and some of its generalizations, and to conditional/mode estimation procedures such as variational methods. A detailed derivation of the Kalman filter is given to illustrate the role of key probabilistic concepts and assumptions. Extensions of the Kalman filter to nonlinear observation operators and to non-Gaussian errors are then described. In a simple illustrative example, rigorous treatment of representativeness error and model error is highlighted in finite-dimensional estimation procedures for continuum dynamics and observations of the continuum state.