Stochastic filters (ensemble Kalman filters)¶
Summary¶
File: src/estimators/ensembles.py. EnsembleEstimator is the concrete intermediate base; leaf classes only implement _analysis_kernel(Af, d, Cdd).
Key attributes: m (ensemble size), std_phi, std_alpha, regularization_factor
| Class | Filter type | Reference |
|---|---|---|
EnKF |
Stochastic EnKF (perturbed observations) | Evensen (2003) |
EnSRKF |
Deterministic square-root EnKF | Tippett et al. (2003) |
rBA_EnKF |
Regularized bias-aware EnKF, weight γ | Nóvoa, Racca & Magri (2023) |
Covariance inflation (src/estimators/inflation.py, Evensen 2009 Chap. 15): fixed
multiplicative inflation via inflation_factor. Applied factors are logged in
estimator.inflation_history.
romda.estimators.EnsembleEstimator(parent_model, parent_bias=None, **kwargs)
¶
Bases: Estimator
Abstract base for ensemble-based estimators (EnKF, EnSRKF, rBA-EnKF).
Owns the model, ensemble, and bias. Subclasses only need to implement
_analysis_kernel(Af, d, Cdd, **kwargs) -> Aa.
Attributes:
| Name | Type | Description |
|---|---|---|
m |
int
|
Number of ensemble members. |
std_phi |
float
|
Fractional std for initial state perturbations. |
std_alpha |
float or dict
|
Std (or |
distribution_phi |
str
|
Sampling distribution for state members ("normal" or "uniform"). |
distribution_alpha |
str
|
Sampling distribution for parameter members. |
ensure_mean_at_init |
bool
|
Force one member to equal the ensemble mean at initialisation. |
ensemble_psi0 |
ndarray or None
|
Pre-built initial ensemble; bypasses generation if provided. |
activate_parameter_estimation |
bool
|
Whether to include parameter rows in the analysis update. |
regularization_factor |
float
|
Bias-regularisation weight (used by rBA-EnKF; ignored otherwise). |
Initialise ensemble estimator.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
parent_model
|
Model instance or Model subclass
|
If a class is passed, remaining kwargs are forwarded to its
constructor (e.g. |
required |
parent_bias
|
Bias instance, Bias subclass, or None
|
|
None
|
**kwargs
|
Ensemble config keys ( |
{}
|
Source code in src/estimators/ensembles.py
101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 | |
m
property
¶
Ensemble size.
inflation_history
property
¶
Namedtuple with fields times and factors of each applied inflation.
get_observables(Nt=1, **kwargs)
¶
Return ensemble observables, bias-corrected if applicable.
Source code in src/estimators/ensembles.py
154 155 156 157 158 159 160 161 162 163 164 | |
get_observable_hist(Nt=0)
¶
Return (y_unbiased, y_model) history, interpolating bias if needed.
Source code in src/estimators/ensembles.py
166 167 168 169 170 171 172 173 174 175 176 177 178 | |
analysis_step(d, Cdd, return_analysis=False)
¶
Bayesian analysis step.
Builds the augmented forecast ensemble, delegates the filter update to
_analysis_kernel, applies inflation, validates parameters, and
updates the model history in-place.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
d
|
ndarray(Nq)
|
Observation vector. |
required |
Cdd
|
ndarray(Nq, Nq)
|
Observation noise covariance. |
required |
return_analysis
|
bool
|
If True, return the analysed state array. |
False
|
Source code in src/estimators/ensembles.py
196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 | |
reshape_ensemble(m=None, reset=True)
¶
Re-perturb the current ensemble around its mean.
Source code in src/estimators/ensembles.py
333 334 335 336 337 338 339 340 341 342 343 344 | |
inflate(A, rho, d=None, additive=True)
staticmethod
¶
Deprecated alias for romda.estimators.inflation.multiplicative_inflation.
Source code in src/estimators/ensembles.py
346 347 348 349 350 351 352 353 354 | |
has_valid_spread(A, tol=1e-06)
staticmethod
¶
Return True if ensemble spread is non-degenerate.
Source code in src/estimators/ensembles.py
356 357 358 359 | |
has_valid_params(A_alpha, alpha_limits_matrix, get_deltas=False)
staticmethod
¶
Check whether all parameter ensemble members lie within bounds.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
A_alpha
|
(Na, m)
|
|
required |
alpha_limits_matrix
|
(2, Na, 1) rows are [lower_bounds, upper_bounds]
|
|
required |
Returns:
| Name | Type | Description |
|---|---|---|
is_physical |
bool
|
|
idx_alpha |
list of out-of-bounds parameter indices
|
|
d_alpha |
allowed values array (only populated if get_deltas=True)
|
|
Source code in src/estimators/ensembles.py
361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 | |
visualize_history(**kwargs)
¶
Plot observable and parameter histories.
Source code in src/estimators/ensembles.py
480 481 482 483 484 485 486 | |
visualize_state(**kwargs)
¶
Plot ensemble state distributions.
Source code in src/estimators/ensembles.py
488 489 490 491 492 | |
romda.estimators.EnKF(parent_model, parent_bias=None, **kwargs)
¶
Bases: EnsembleEstimator
Stochastic Ensemble Kalman Filter (perturbed-observation variant).
Each ensemble member assimilates a randomly perturbed copy of the observation, \(\mathbf{d}_j \sim \mathcal{N}(\mathbf{d}, \mathbf{C}_{dd})\). Writing \(\boldsymbol{\Psi}^\mathrm{f} = \mathbf{A}^\mathrm{f} - \overline{\mathbf{A}^\mathrm{f}}\) for the mean-subtracted forecast ensemble and \(\mathbf{S} = \mathbf{M}\boldsymbol{\Psi}^\mathrm{f}\):
Notes
Algebraically equivalent to the textbook Kalman-gain form \(\mathbf{K} = \mathbf{C}_{\psi\psi}\mathbf{C}_{yy}^{-1}\) with \(\mathbf{C}_{\psi\psi} = \boldsymbol{\Psi}^\mathrm{f}\mathbf{S}^\mathrm{T}/(m{-}1)\), \(\mathbf{C}_{yy} = \mathbf{S}\mathbf{S}^\mathrm{T}/(m{-}1) + \mathbf{C}_{dd}\), but rearranged to avoid forming the covariance matrices explicitly. The implementation equivalently expresses the update as a member-space transform \(\mathbf{A}^\mathrm{a} = \mathbf{A}^\mathrm{f}(\mathbb{I}_m + \mathbf{X})\), \(\mathbf{X} = \mathbf{S}^\mathrm{T}\mathbf{C}^{-1}(\mathbf{D}-\mathbf{M}\mathbf{A}^\mathrm{f})\), which coincides with the equation above because \(\mathbf{S}\) has zero column-sum.
References
Evensen (2003). The ensemble Kalman filter: theoretical formulation and practical implementation. Ocean Dynamics, 53, 343-367, Eq. (9.27).
Source code in src/estimators/ensembles.py
101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 | |
romda.estimators.EnSRKF(parent_model, parent_bias=None, **kwargs)
¶
Bases: EnsembleEstimator
Ensemble Square-Root Kalman Filter (no observation perturbations).
Updates the ensemble mean with the Kalman gain and transforms the ensemble deviations with a symmetric square-root transform, so no stochastic observation perturbations are needed. Writing \(\boldsymbol{\Psi}^\mathrm{f}\) for the mean-subtracted forecast ensemble and \(\mathbf{S} = \mathbf{M}\boldsymbol{\Psi}^\mathrm{f}\):
with \(\mathbf{T}^{1/2}\) the symmetric square root of \(\mathbf{T}\) (computed via its eigendecomposition, since \(\mathbf{T}\) is symmetric).
References
Tippett, Anderson, Bishop, Hamill & Whitaker (2003). Ensemble square root filters. Mon. Wea. Rev., 131, 1485-1490.
Source code in src/estimators/ensembles.py
101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 | |
romda.estimators.rBA_EnKF(parent_model, parent_bias=None, gamma=1.0, **kwargs)
¶
Bases: EnsembleEstimator
Regularized bias-aware ensemble Kalman filter (r-EnKF).
Extends the stochastic EnKF with an explicit correction for the (estimated)
observation bias \(\mathbf{b}\) and its Jacobian
\(\mathbf{J} = \mathrm{d}\mathbf{b}/\mathrm{d}(\mathbf{M}\boldsymbol{\psi})\),
weighted by the regularization factor \(\gamma \ge 0\) (\(\gamma=0\) recovers the
standard EnKF). Writing \(\mathbf{Y} = \mathbf{M}\mathbf{A}^\mathrm{f} + \mathbf{B}\)
for the bias-corrected forecast observables, and \(\mathbf{C}_{\psi q}\),
\(\mathbf{C}_{qq}\) for the (sample) forecast cross- and auto-covariances of the
state and the mapped observables:
where \(\mathbf{D}\) is the perturbed-observation ensemble and \(\mathbf{b}\) the current (mean) bias estimate, broadcast over the ensemble. The bias covariance is fixed to \(\mathbf{C}_{bb} = \mathbf{C}_{dd}\), so the weight \(\mathbf{W} = \mathbf{C}_{dd}\mathbf{C}_{bb}^{-1}\) of the reference paper reduces to the identity and is omitted.
Notes
\(\mathbf{C}_{\psi q}\) and \(\mathbf{C}_{qq}\) here are the sample cross- and
auto-covariances (divided by \(m-1\)), unlike the unnormalized cross-moments used
by the legacy romda.legacy.data_assimilation.rBA_EnKF implementation of the same
filter. If observations are biased (Bias.biased_observations), \(\mathbf{d}\) is
shifted by the mean bias-innovation gap before the update. During the first
start_bias analysis steps the plain EnKF update is applied instead
(bias-blind warm-up window).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
gamma
|
float
|
Bias-regularization factor (default 1.0), aliased as |
1.0
|
References
Nóvoa, Racca & Magri (2023). Inferring unknown unknowns: Regularized bias-aware ensemble Kalman filter. Comput. Methods Appl. Mech. Eng., 418, 116502. DOI: 10.1016/j.cma.2023.116502.
Source code in src/estimators/ensembles.py
697 698 699 700 701 702 703 704 705 706 | |