Minimisation Techniques 1 Assimilation Algorithms: Minimisation Techniques Yannick Trémolet ECMWF Data Assimilation Training Course March 2006

Minimisation Techniques 2 4D Variational Data Assimilation

Minimisation Techniques 3 Incremental 4D-Var

Minimisation Techniques 4 Incremental 4D-Var

Minimisation Techniques 5 Incremental 4D-Var

Minimisation Techniques 6 The Outer Iterations

Minimisation Techniques 7 The Inner Iterations

Minimisation Techniques 8 Minimisation: Newton method

Minimisation Techniques 9 Minimisation: Newton method

Minimisation Techniques 10 Minimisation: Quasi-Newton method

Minimisation Techniques 11 Minimisation: Quasi-Newton method

Minimisation Techniques 12 Limited Memory Quasi-Newton

Minimisation Techniques 13 Minimisation: Steepest Descent

Minimisation Techniques 14 Minimisation: Steepest Descent The first step fully minimizes the function in the descent direction, but this is undone by subsequent steps. We want to avoid this.

Minimisation Techniques 15 Minimisation: Conjugate Gradient

Minimisation Techniques 16 Conjugate Gradient Convergence

Minimisation Techniques 17 Preconditioning

Minimisation Techniques 18 4D-Var Preconditioning

Minimisation Techniques 19 A case of poor convergence

Minimisation Techniques 20 Theoretical example

Minimisation Techniques 21 Theoretical example

Minimisation Techniques 22 A case of poor convergence

Minimisation Techniques 23 Hessian Preconditioning

Minimisation Techniques 24 Hessian Eigenvectors Preconditioning

Minimisation Techniques 25 4D-Var Eigenvalues Eigenvalue N =1 1 = = Preconditioning reduces the condition number k= 1 / N from to

Minimisation Techniques 26 Conjugate Gradient and Lanczos Algorithm

Minimisation Techniques 27 Lanczos Algorithm

Minimisation Techniques 28 Lanczos Algorithm

Minimisation Techniques 29 Conjugate Gradient and Lanczos Algorithm

Minimisation Techniques 30 Superlinear Convergence

Minimisation Techniques 31 Rounding Error

Minimisation Techniques 32 CG Cost function reduction Quasi-Newton with inexact line searches Quasi-Newton with exact line searches Conjugate Gradient without orthogonalisation Conjugate Gradient with orthogonalisation

Minimisation Techniques 33 CG Gradient norm reduction Quasi-Newton with inexact line searches Quasi-Newton with exact line searches Conjugate Gradient without orthogonalisation Conjugate Gradient with orthogonalisation

Minimisation Techniques 34 4D-Var Cost function reduction Variational Quality Control

Minimisation Techniques 35 4D-Var gradient norm reduction Convergence is roughly twice as fast with Hessian preconditioning.

Minimisation Techniques 36 CG reduction of norm of gradient 0.05

Minimisation Techniques 37 Spectrum of preconditioned Hessian

Minimisation Techniques 38 RMS of T analysis increments

Minimisation Techniques 39 4D-Var Convergence

Minimisation Techniques 40 4D-Var Convergence

Minimisation Techniques 41 Summary

Minimisation Techniques 42