Atmospheric observations consist of a mixture of in situ, visual, and remotely sensed observations. These provide an extensive database for research and numerical weather prediction. However, significant data deficiencies still exist, and new observing systems are continually being proposed. Observing system experiments (OSE's) are conducted to assess the usefulness of different types of existing atmospheric observations. Observing system simulation experiments (OSSE's) are conducted to evaluate the potential impact of proposed observing systems, as well as to determine tradeoffs in their design, and to evaluate data assimilation methodology. This paper contains a review of the development of the global atmospheric observing system, a description of the principal types of data, an overview of OSE and OSSE methodology, and results from recent experiments to evaluate the relative utility of the principal atmospheric observing systems and the potential for new observing systems. These experiments show the critical contributions being made by both conventional and space-based observations, and indicate considerable potential for future satellite observing systems to improve data assimilation.
For many years, merchant ships and the naval fleets of various countries have been the major source of data over and in the open ocean. Oceanographic research experiments and process studies in the field have also contributed to the climatological data bases for the global ocean, but, for the most part, these have been limited in duration and extent. However, over the last 10 years under the auspices of the World Climate Research Program and the International Geosphere Biosphere Program the role of the oceans in global and climate change has taken on increased significance. This has created a need for a considerably improved understanding of the seasonal, interannual, decadal and longer time-scale variability of the physical and biogeochemical attributes of the global ocean. As a result, over the past 10 years several major international field programs have been implemented and have had a tremendous impact on the number of in situ observations obtained for the global ocean. The Tropical Ocean Global Atmosphere (TOGA) program, the World Ocean Circulation Experiment (WOCE), and the Joint Global Ocean Flux Study (JGOFS) were designed with observational, modeling, and process study components aimed at analyzing different aspects of the ocean's role in the coupled climate system. In parallel with the field programs, continuous space-based observations of sea surface temperature, sea surface topography, and sea surface winds spanning nearly a decade or longer have become a reality. During this same time period, numerical ocean models and computational power have advanced to the point where the oceanographic observations, both in situ and remotely sensed, can be assimilated into numerical ocean models in order to provide a four-dimensional (x-y-z-t) depiction of the evolving state of the global ocean.
The basic governing equations and the dynamical framework used in large-scale atmospheric models are briefly reviewed including a presentation of commonly used vertical coordinates. An overview of dynamical and physical adjustment mechanisms in the atmosphere with relatively short time scales and their possible modifications in models are then presented. The overview includes mutual adjustment of velocity and mass fields, instabilities under which the adjustment does not take place, and the effect of horizontal and vertical computational modes on the adjustment. The physical adjustment mechanisms in models, those due to PBL and condensation processes in particular, are also discussed.
The need for unified notation in atmospheric and oceanic data assimilation arises from the field's rapid theoretical expansion and the desire to translate it into practical applications. Self-consistent notation is proposed that bridges sequential and variational methods, on the one hand, and operational usage, on the other. Over various other mottoes for this risky endeavor, the authors selected: "When I use a word, " Humpty Dumpty said, in rather a scornful voice tone, "it means just what I choose it to mean - neither more more less. "Lewis Carroll, 1871.
Assimilation of meteorological or oceanographical observations can be described as the process through which all the available information is used in order to estimate as accurately as possible the state of the atmospheric or oceanic flow. The available information essentially consists of the observations proper, and of the Physical laws which govern the evolution of the flow. The latter are available in practice under the form of a numerical model. The existing assimilation algorithms can be described as either sequential or variational. The links between these algorithms and the theory of statistical estimation are discussed. The performances of present algorithms, and the perspectives for future development, are also briefly discussed.
Variational methods are introduced as particular algorithms to solve linear estimation problems in the presence of a dynamics i.e. data assimilation. They are compared from the algebraic and algorithmic viewpoint to optimal interpolation and the Kalman filter.
There are several successful techniques for removing relatively high-frequency "noise" due to propagating inertial-gravitational waves in model forecasts. Not all behavior that is sometimes considered noise is necessarily spurious, however, including that due to tidal forcing. Published evidence of the natural existence of high-frequency behavior within numerical weather prediction and general circulation models is briefly reviewed and some additional evidence with yet another model is introduced. In this new model, the existence of local diabatic balances is also examined. Imposing balances or filters at scales /here high-frequency oscillations are naturally and realistically occurring causes errors which may become more significant as we demand more accurate models and atmospheric analyses. The intention of this paper is to stimulate further research into the naturally occuring high-frequency behavior in numerical models, as well as in the atmosphere, for the purpose of improving both models and atmospheric analyses.
Advanced data assimilation becomes extremely complicated and challenging when used with strongly nonlinear models. Several previous works have reported various problems when applying existing popular data assimilation techniques with strongly nonlinear dynamics. Common for these techniques is that they can all be considered as extensions to methods which have proven to work well with linear dynamics. This paper shows that a weak constraint variational formulation for the Lorenz model, where the full model state in space and time is considered as control variables, can be minimized using a gradient descent method. It is further shown that the weak constraint formulation removes some of the previous reported problems associated to the predictability limit of nonlinear models when strong constraint formulations are used. Further, by using a gradient descent method, problems associated to the use of an approximate tangent linear model when solving the Euler-Lagrange equations or when the extended Kalman filter is used, are eliminated, since the solution is found without integration of any dynamical equations. The method works well with reasonable data coverage and quality of the measurements, however, with poorer data coverage a statistical minimization method, simulated annealing, may be used to search for the global minimum.
A model of the evolution of the state of the ocean or of the atmosphere makes sense in and only in presence of data. Any sensitivity analysis have to take into account the data. In this paper we will show the general principle of sensitivity analysis and the use of the adjoint model in order to derive the sensitivity. In presence of data this analysis must be carried out not on the model itself but on the optimality system. Two examples are shown, firstly we study the sensitivity of a variable governed by a simple ordinary differential equation, a realization being imperfectly observed, with respect to one of the parameter of the governing equation. In a second example we will estimate the sensitivity with respect to surface observations of retrieved kinetic energy of the bottom layer in a quasigeostrophic model of oceanic circulation. Both examples clearly show that a correct sensitivity have to take into account the second order terms.
Despite the explosive growth of activity in the field of Earth System data assimilation over the past decade or so, there remains a substantial gap between theory and practice. The present article attempts to bridge this gap by exposing some of the central concepts of estimation theory and connecting them with current and future data assimilation approaches. Estimation theory provides a broad and natural mathematical foundation for data assimilation science. Stochastic-dynamic modeling and stochastic observation modeling are described first. Optimality criteria for linear and nonlinear state estimation problems are then explored, leading to conditional-mean estimation procedures such as the Kalman filter and some of its generalizations, and to conditional/mode estimation procedures such as variational methods. A detailed derivation of the Kalman filter is given to illustrate the role of key probabilistic concepts and assumptions. Extensions of the Kalman filter to nonlinear observation operators and to non-Gaussian errors are then described. In a simple illustrative example, rigorous treatment of representativeness error and model error is highlighted in finite-dimensional estimation procedures for continuum dynamics and observations of the continuum state.
What: Estimate the state of a fluid system - the atmosphere or oceans - from incomplete and inaccurate observations, with the help of dynamical models. When: After the observations have been made and before making a numerical forecast of the system. If the evolution of the system over some finite time is to be evaluated - i.e., if interested in climate rather than prediction --- sequential estimation proceeds by scanning through the observations over the interval, forward and back. How: Admit, that the dynamical model of the system isn't perfect either. Assign relative weights to the current observations and to the model forecast, based on past observations, that are inversely proportional t() their respective error variances. Yes, but: To compute the forecast errors is computationally expensive. So what: Compromise! The thrust of this review is to illustrate some smart ways of (i) near-optimal, but computationally still feasible implementation of the extended Kalman filter (EKF), while using (ii) the EKF for observing system design, as well as for estimating (iii) the state and parameters of (iv) unstable and strongly nonlinear systems, including (v) the coupled ocean-atmosphere system.
Two data assimilation procedures for real time storm surge forecasting based on Kalman filtering are described. To reduce the computational burden of the Kalman filter, a time invariant filter approximation is suggested first. This filter is computed using the Chandrasekhar-type algorithm. The resulting data assimilation procedure has been used for storm surge forecasting on a routine basis for a number of years in The Netherlands and in Denmark. The results of these operational systems are discussed in detail. Finally also a new efficient algorithm for time varying Kalman filtering problems is introduced and applied to storm surge forecasting.
This paper provides an overview of atmospheric data assimilation. It is shown bow data assimilation developed historically from the requirement to provide initial conditions for numerical weather prediction. The basic concepts of atmospheric data assimilation are discussed, starting with the scalar case, and progressing through three dimensional spatial analysis to the full four dimensional problem. The most advanced algorithms (4DVAR and the Kalman filter) are introduced briefly and their relation to the simpler algorithms explored. The control of undesirable high frequency oscillations is sketched. The present state of atmospheric data assimilation is discussed and possible future developments are suggested.
This paper considers some of the problems that arise concerning the assimilation of remotely-sensed observations into numerical weather prediction models, including the options and compromises that have to be faced in deciding whether and how to pre-process the observations prior to assimilation. These issues are discussed first in general and then in relation to some specific observation types: TOVS radiances, satellite winds and cloud imagery, SSM/I data, scatterometer data and radio occultation data. Results of recent work in these areas at ECMWF are summarized or referenced.
The United Kingdom Meteorological Office is developing a variational assimilation for its Unified Model forecast system, which contains a grid-point model, run operationally in global, regional, and mesoscale configurations. Key characteristics of the design are: ·development path from 3-dimensional to 4-dimensional scheme ·global and limited area configurations ·variational analysis of perturbations ·carefully designed, well conditioned 'background' term This paper describes the variational scheme, with some example results from a simple 2-dimensional variational analysis which has been developed as a prototype.
Many developments have occurred in the large scale Numerical Weather Prediction (NWP) systems since the beginning of their operational use (end of the sixties). They have been pushed by scientific developments in data assimilation and modelling aspects as well as by the rapid progress in computer technology. After a quick description of the evolution of the operational data assimilation schemes since 1970, a review of the current state of the art in meteorological data assimilation is made. This review is limited to operational NWP performed for global modelling. Then the main algorithmic trends are outlined, and one tries to guess what operational data assimilation is likely to be in NWP centres in 2000 and beyond.
This paper describes significant changes to the U.S. National Centers for Environmental Prediction (NCEP, formerly NMC, the National Meteorological Center) global data assimilation system that were made operational in January, 1995. The emphasis is on changes to the 3D-variational analysis, which has been running operationally since June, 1991. The changes include additions of new data types, modifications to the background error, and addition of a weak constraint on divergence tendency. A uniform improvement in fit of the 6 hour forecast guess to all observation types was observed in parallel tests over a period of 9 months. Most of the improvement is believed to be due to the addition of the divergence tendency constraint. The improved performance extends to medium range forecasts, as measured by anomaly correlation scores for geopotential height. Current research activities related to global and regional data assimilation at NCEP are also briefly described.
The present status of development of data assimilation techniques for high resolution limited area models is reviewed and various candidates for a new generation of a data assimilation system are compared. It is concluded that data assimilation based on 3-dimensional or 4-dimensional variational techniques is the most promising approach. Further investigations are needed, however, to find out whether variational data assimilation is feasible over "limited" model integration areas or whether global/hemispheric model integration areas have to be introduced for high resolution limited area data assimilation purposes. The main problem in this connection is the rapid propagation of forecast errors on a global or at least hemispheric atmospheric scale within a time-scale of a few days.
This paper describes the methods used to produce cloud-drift winds (CDWs), concentrating, in particular, on their generation from sequential Geostationary Meteorological Satellite (GMS) imagery. It discusses the estimation of these motion vectors from both infrared (IR) and visible imagery at high spatial and temporal resolution and also records their accuracy and utility. The paper then discusses the assimilation of CDW data for numerical weather prediction (NWP). It does this by looking at studies, both in the Australian Region and over a larger domain, showing the impact of CDWs on operational NWP using current conventional data assimilation techniques. Subsequent to this, the use of CDWs is examined in the context of tropical cyclone motion prediction, where intermittent assimilation, nudging and the use of a full variational technique are contrasted, using examples from the Tropical Cyclone Motion-90 (TCM-90) experiment in the tropical North-West Pacific and by examining the impact of hourly CDWs in the Australian region. It was found that the high spatial and temporal resolution winds clearly have the potential to improve the accuracy of NWP, however, full exploitation of their information content appears to require appropriate assimilation techniques such as the variational method employed here.
Mesoscale data assimilation systems are being run at a number of operational and research centres as the demand for accurate mesoscale forecasts increases. In this paper, a review of some major issues and common practices in mesoscale data assimilation are considered with an emphasis on operational applications. Attention is focussed on the generation of an adequate initial state using a variety of conventional and non-conventional data sources and techniques.
Methods used to estimate the state of the ocean and its temporal variations with special focus on the mesoscale (the eddy scale) are examined and illustrated. There are two broad categories of methods, although mixed approaches can be found. The "statistical methods" are first discussed. It is shown that these methods are robust and can help assimilate the routinely available ocean observations (typically, satellite altimetry and surface temperature), with the help of order-reduction methods which are still open to debate, especially as far as the inference of the deep circulation is concerned. Among the possible simplifications, recent developments around the adaptive filter and representer approaches attempt to address the facts that oceanic eddy-resolving problems are usually of very high dimension, and that the knowledge of process and observational noise statistics is poor. The sophisticated "variational methods" lack an error theory, but are more flexible in efficiently assimilating various data kinds, and can help study the sensitivity to various controls. Examples of strategies using both types of methods in sequence are also given.
Satellite altimeter data from both ERS-1 and TOPEX/POSEIDON have been assimilated into a 1/8° eddy-resolving model of the Pacific Ocean north of 20°S. We present strong evidence of significant oceanographic impact from this assimilation by direct comparison with two independent sets of observations, tide gauge sea level data and frontal locations from satellite infrared (IR) imagery. The assimilation of altimeter data provides marked and highly significant improvement in the accuracy of the model output in representing observed oceanic variations over most of the North Pacific basin. Additionally, it gives improved model accuracy in locating mesoscale eddies and meandering frontal structure in non-deterministic oceanographic regions like the Kuroshio-Kuroshio Extension. In regions where the ocean has a strong deterministic response to wind forcing, such as the equatorial region and along the American Pacific coast, the model has a high correlation with observed sea level variations before assimilation and the improvements are smaller. They are also small along the coast of Japan due to the importance of flow instabilities in this region and their relatively short time and space scales.
It is commonplace to begin talks on this topic by noting that oceanographic data are too scarce and sparse to provide complete initial and boundary conditions for large-scale ocean models. Even considering the availability of remotely-sensed data such as radar altimetry from the TOPEX and ERS-1 satellites, a glance at a map of available subsurface data should convince most observers that this is still the case. Data are still too sparse for comprehensive treatment of interannual to interdecadal climate change through the use of models, since the new data sets have not been around for very long. In view of the dearth of data, we must note that the overall picture is changing rapidly. Recently, there have been a number of large scale ocean analysis and prediction efforts, some of which now run on an operational or at least quasi-operational basis, most notably the model based analyses of the tropical oceans. These programs are modeled on numerical weather prediction. Aside from the success of the global tide models, assimilation of data in the tropics, in support of prediction and analysis of seasonal to interannual climate change, is probably the area of large scale ocean modeling and data assimilation in which the most progress has been made. Climate change is a problem which is particularly suited to advanced data assimilation methods. Linear models are useful, and the linear theory can be exploited. For the most part, the data are sufficiently sparse that implementation of advanced methods is worthwhile. As an example of a large scale data assimilation experiment with a recent extensive data set, we present results of a tropical ocean experiment in which the Kalman filter was used to assimilate three years of altimetric data from Geosat into a coarsely resolved linearized long wave shallow water model. Since nonlinear processes dominate the local dynamic signal outside the tropics, subsurface dynamical quantities cannot be reliably inferred from surface height anomalies. Because of its potential for large scale synoptic coverage of the deep ocean, acoustic travel time data should be a natural complement to satellite altimetry. Satellite data give us vertical integrals associated with thermodynamic and dynamic processes, while acoustic travel times provide horizontal integrals from which dynamics of the deep ocean can be inferred. Linearized analysis indicates that detailed information can be retrieved by means of data assimilation from integral sources of data such as acoustic travel times. Static analysis of tomographic data without data assimilation cannot provide nearly so much detail. It can be shown that integrated quantities along the edges and diagonals of a simple square array combined with a linearized quasigeostrophic model is an observable system, down to scales much shorter than the dimensions of the array. Nonlinearities complicate the picture, but the linear results, along with a few preliminary numerical experiments give us cause for optimism.
A Primitive Equation Ocean General Circulation Model (PE OGCM) in a global configuration similar to that used in coupled ocean-atmosphere models is fitted to climatological data using the adjoint method. The ultimate objective is the use of data assimilation for the improvement of the ocean component of coupled models, and for the calculation of initial conditions for initializing coupled model integrations. We argue that ocean models that are used for coupled climate studies are an especially appropriate target for data assimilation using the adjoint method.
An ocean data assimilation system developed for climate monitoring at the Japan Meteorological Agency (JMA) is described. The system consists of an ocean general circulation model (OGCM) and a subsurface temperature analysis scheme using optimal interpolation. The analyzed temperatures are continuously assimilated into the wind-driven OGCM. The atmospheric forcing is obtained from the operational numerical weather prediction system. The dynamical ocean model helps synthesize information in the forcing and data, neither of which is complete for analyzing subsurface structures. The temporary and spatially continuous estimates given by the system aid the operational long-range forecasters to monitor climatically important phenomena such as El Nino-Southern Oscillation more closely on intraseasonal to interannual time scales. The introduction of temperature data significantly improves the quality of the wind-forced simulation of subsurface thermal fields. However, the model's inability to reproduce the Equatorial Undercurrent with sufficient strength appears to limit the impact of data on the current field. Improvements in the model and in the data assimilation scheme and introduction of other types of observational data are required for further development of the system.
As part of the International Symposium on Assimilation of Observations in Meteorology and Oceanography, a panel discussion was held on the O1997, Meteorological Society of Japan evening of 15 March 1995. The purpose of this panel discussion was to focus on what the next major areas of research in data assimilation should be. The panelists had five minutes each for short presentations and this was followed by an open discussion. The Appendix summarizes the opinions of the panelists, and of a few additional well-known practitioners of data assimilation, on major areas requiring development in the future. Several themes emerged repeatedly among the individual statements (see also "Open questions" in Ghil, 1997): 1) the need to include "errors of the day" in data assimilation of unstable systems such as the atmosphere, and perhaps the coupled ocean-atmosphere (Anderson, Cohn, Courtier, Kalnay, Parrish, Purser); 2) the need to perform observational impact studies to determine the most cost-effective observing systems (Busalacchi, Schlatter); 3) the need to improve the models to ensure that they have attractors not too different from nature (Bennett, Busalacchi, Parrish). The statement "the principal influence of the assimilated data (in tropical ocean data assimilation) is to eliminate systematic biases in the model temperature fields" (Busalacchi, 1996) would suggest that the improvement of the ocean models is, at this time, more important than any assimilation technique. For the coupled models it is of paramount importance that the ocean forcings (e.g., wind Stress) also be close to the real forcings (Anderson); 4) the need to tackle new areas of data assimilation (mesoscale, land surface, coupled ocean atmosphere) (Anderson, Sato, Schlatter), and new types of observations (Derber); 5) the need to develop advanced data assimilation systems which are computationally affordable (Cohn, Lorenc); 6) the need to estimate analysis errors and to test the validity of these estimates with data (Bennett, Busalacchi, Cohn, Purser); 7) the need for methods that do not smear small-scale structures (Bennett, Purser). During the discussion, several additional points were brought up: Prof. Y. Sasaki (U. Oklahoma) showed how new observing systems (such as NEXRAD) allow the possibility of data assimilation for such small-scale phenomena as microbursts. G.Evensen (NERSC, Norway) asked what could be the role of data assimilation in small countries. One answer offered by J. Cramm (Forecast Systems Lab-oratory, NOAA): a local-area analysis and predication system (like NCEP's LAPS). E. Brin (Goddard, NASA) asked whether it was possible to use reanalyses for studies of climate change. Answer by A. Lorenc (UKMO): basically no. (One exception is the use of optimal averaging within the reanalysis processing, which improves the estimate of the areal average from the observations, and provides an estimate of its error).