Operational Data Checking

An automatic data checking system has been implemented at ECMWF to trigger warnings in case of anomalies affecting the quality or the availability of the data assimilated by the model. Selected statistical parameters (number of observations, bias correction, and mean bias-corrected background, analysis departures and probability of gross error) are checked against an expected range. An appropriate alert message (including a time series plot) is generated if statistics are outside the specified ranges. A severity level (slight, considerable, severe) is assigned to each message depending on how far statistics are from the expected values. Two kinds of ranges are used by the automatic checking: Soft and Hard limits. Soft limits are updated automatically using statistics from the last twenty days (extremes are excluded during this process). Hard limits are adjusted manually when required.

Currently, the automatic checking is limited to data passing through the minimisation process (including VarBC passive data). It's being applied twice a day to the delayed cut-off 4D-VAR cycles (00 and 12) and four times a day to the early delivery assimilation system (00,06,12 and 18).

The alarm system results are based on feedback information from the ECMWF data assimilation and consequently reflect the ECMWF data usage. Although most warnings are related to data problems, they may on occasion also be related to model issues. Alerts are provided for information on an as-is basis and may be withdrawn or changed without notice. ECMWF accepts no responsibility for actions taken on the basis of these results.

Delayed cut-off 4DVAR at 2020-07-10 12:00

iasi
METOP-A IASI RADIANCES CHANNEL 286 : out of range
Considerable: obs_number=3807.0000, expected range: 3055.5725 3624.8981
METOP-A IASI RADIANCES CHANNEL 290 : out of range
Considerable: obs_number=3720.0000, expected range: 2948.9079 3523.0921
METOP-A IASI RADIANCES CHANNEL 292 : out of range
Considerable: obs_number=3903.0000, expected range: 3191.3331 3710.0787
METOP-A IASI RADIANCES CHANNEL 308 : out of range
Considerable: obs_number=3523.0000, expected range: 2712.5285 3302.0597
METOP-A IASI RADIANCES CHANNEL 379 : out of range
Considerable: obs_number=3575.0000, expected range: 2795.1433 3333.2097
METOP-A IASI RADIANCES CHANNEL 389 : out of range
Considerable: obs_number=3626.0000, expected range: 2811.6810 3412.7896
cris
NPP CRIS RADIANCES CHANNEL 665 : out of range
Considerable: obs_number=979.0000, expected range: 601.4668 898.0626
NPP CRIS RADIANCES CHANNEL 115 : out of range
Considerable: stdev_fgdepar=0.1128, expected range: 0.0827 0.1013
saphir
All MEGHA-TROPIQUES SAPHIR ALLSKY ALL CHANNELS Global : all data missing
Severely_persistent: obs_number=0, expected range: 431.72 16887.26
mwhs2
FY-3C MWHS2 ALLSKY RADIANCES (11 channels affected) : out of range
Considerable: obs_number=1071.0000, expected range: 4819.0541 13657.1682
gpsro
All KOMPSAT-5 GPSRO SETTING ALL HEIGHTS Global : all data missing
Severely_persistent: obs_number=0, expected range: 286.49 1219.04
resat
NOAA-19 SBUV OZONE, TOP PRESSURE LAYER 20-21 (HPA) : out of range
25 times in the last 10 days for at least one item
Slight: stdev_fgdepar=0.0128, expected range: 0.0074 0.0113
Slight: avg_fgdepar=-0.0431, expected range: -0.0404 -0.0333
NOAA-19 SBUV OZONE, TOP PRESSURE LAYER 30-31 (HPA) : out of range
7 times in the last 10 days for at least one item
Slight: stdev_fgdepar=0.0241, expected range: 0.0090 0.0209
Radiosondes
LAND TEMP TEMPERATURE (K) 61660 PRESSURE LAYER 0-300 (HPA) : out of range
14 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=-14.3348, expected range: -39.4027 29.8450
Severely_persistent: rms_fgdepar=20.4878, expected range: -41.3676 57.0059
HIGH RESO LAND TEMP TEMPERATURE (K) (2 pressure layers affected) : missing
3 times in the last 10 days for at least one item
Severely_persistent: obs_number=0.0000, expected range: 22.2372 355.5054
Severely_persistent: obs_number=0.0000, expected range: 0.0000 653.7140
Severe: obs_number=0.0000, expected range: 70.4436 586.1140
Severe: obs_number=0.0000, expected range: 70.6139 125.3861
HIGH RESO SEA TEMP TEMPERATURE (K) (3 pressure layers affected) : missing
7 times in the last 10 days for at least one item
Severely_persistent: obs_number=0.0000, expected range: 0.0000 818.3312
HIGH RESO LAND TEMP TEMPERATURE (K) (3 pressure layers affected) : missing
4 times in the last 10 days for at least one item
Severe: obs_number=0.0000, expected range: 0.0000 1019.3335
HIGH RESO LAND TEMP Q (G/KG) PRESSURE LAYER 0-300 (HPA) : missing
2 times in the last 10 days for at least one item
Severe: obs_number=0.0000, expected range: 28.0313 276.4013
Severe: obs_number=0.0000, expected range: 0.0000 515.2055
BUFR TEMP DESCENT Q (G/KG) PRESSURE LAYER 900-1200 (HPA) : missing
Severe: obs_number=0.0000, expected range: 0.0000 287.1855
amdar
WIGOS AMDAR WIND VECTOR DIFF (M/S) 4HSSSNBA PRESSURE LAYER 300-600 (HP : out of range
Severe: rms_fgdepar=16.8060, expected range: 0.0000 7.0571
WIGOS AMDAR WIND VECTOR DIFF (M/S) J1QUOUJA PRESSURE LAYER 300-600 (HP : out of range
Severe: rms_fgdepar=16.5311, expected range: 0.0000 6.5639
Severe: avg_pge=0.8428, expected range: -1.4121 0.7500
WIGOS AMDAR TEMPERATURE (K) EU0758 PRESSURE LAYER 0-300 (HPA) : out of range
1 times in the last 10 days for at least one item
Severe: avg_pge=0.8487, expected range: 0.7655 0.7500
WIGOS AMDAR Q (G/KG) 3OXWUWJA (2 pressure layers affected) : out of range
Severe: avg_fgdepar=-6.2558, expected range: -4.7401 2.6954
Severe: rms_fgdepar=6.2640, expected range: 0.0000 7.0178
WIGOS AMDAR Q (G/KG) 3OXWUWJA PRESSURE LAYER 900-1200 (HPA) : out of range
1 times in the last 10 days for at least one item
Severe: avg_pge=0.8243, expected range: -0.3300 0.5421
pilot
BUFR LAND PILOT WIND VECTOR DIFF (M/S) PRESSURE LAYER 0-300 (HPA) : out of range
Considerable: obs_number=74.0000, expected range: 97.0093 142.9907
synop
MANUAL LAND SYNOP RH2M (%) 42776 : out of range
8 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=-50.0363, expected range: -69.8652 -16.2103
Severely_persistent: rms_fgdepar=50.0363, expected range: 16.2103 69.8652
Severely_persistent: avg_pge=0.9998, expected range: 0.0234 0.7500
BUFR LAND SYNOP SURFACE PRESSURE (PA) 12215 : out of range
17 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=653.4304, expected range: -1565.8160 1335.8239
Severely_persistent: rms_fgdepar=657.0346, expected range: 0.0000 1315.3511
AUTOMATIC LAND SYNOP SURFACE PRESSURE (PA) 89799 : out of range
3 times in the last 10 days for at least one item
Severe: avg_fgdepar=-542.0768, expected range: -194.4120 243.5278
Severe: rms_fgdepar=542.1243, expected range: 0.0000 212.3890
Severe: avg_pge=0.8753, expected range: -0.0458 0.0560
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 41678 : out of range
1 times in the last 10 days for at least one item
Severe: rms_fgdepar=330.7252, expected range: 0.0000 149.0058
Severe: avg_pge=0.8033, expected range: -1.0640 0.7500
MANUAL LAND SYNOP RH2M (%) 63740 : out of range
Severe: rms_fgdepar=36.2069, expected range: 0.0000 17.4382
MANUAL LAND SYNOP RH2M (%) 65501 : out of range
Severe: rms_fgdepar=32.7968, expected range: 0.0000 18.4561
MANUAL LAND SYNOP RH2M (%) 38933 : out of range
4 times in the last 10 days for at least one item
Severe: avg_pge=0.9166, expected range: -0.9181 0.7500
MANUAL LAND SYNOP RH2M (%) 42174 : out of range
2 times in the last 10 days for at least one item
Severe: rms_fgdepar=47.9303, expected range: 0.0000 31.7828
Severe: avg_pge=0.9979, expected range: -1.6241 0.7500
BUFR LAND SYNOP SURFACE PRESSURE (PA) 72327 : out of range
4 times in the last 10 days for at least one item
Severe: avg_pge=0.8638, expected range: 0.8534 0.7500
BUFR LAND SYNOP SURFACE PRESSURE (PA) 41678 : out of range
1 times in the last 10 days for at least one item
Severe: rms_fgdepar=330.7252, expected range: 0.0000 144.3332
BUFR LAND SYNOP RH2M (%) 52533 : out of range
Severe: avg_pge=0.8231, expected range: -0.2002 0.3045
BUFR LAND SYNOP RH2M (%) 72525 : out of range
Severe: rms_fgdepar=35.1446, expected range: 0.5003 13.1014
MANUAL LAND SYNOP RH2M (%) : out of range
Considerable: obs_number=86.0000, expected range: 117.8814 148.1186
ship
BUFR SHIP SYNOP WIND VECTOR DIFF (M/S) EUCDE37 : out of range
6 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=20.0422, expected range: -0.6391 35.4385
Severely_persistent: rms_fgdepar=20.0422, expected range: 6.6327 29.9352
Severely_persistent: avg_pge=0.9141, expected range: 0.0352 0.7500
All BUFR SHIP SYNOP WMO BLOCK 41 : all data missing
Severely_persistent: obs_number=0, expected range: 283.28 326.15
All BUFR SHIP SYNOP WMO BLOCK 42 : all data missing
Severely_persistent: obs_number=0, expected range: 233.75 356.82
All BUFR SHIP SYNOP WMO BLOCK 51 : all data missing
Severely_persistent: obs_number=0, expected range: 75.07 125.07
SHIP SURFACE PRESSURE (PA) TBWUK76 : out of range
Severe: avg_fgdepar=815.5994, expected range: -318.2224 171.9985
Severe: rms_fgdepar=815.5994, expected range: 0.0000 393.8522
BUFR SHIP SYNOP WIND VECTOR DIFF (M/S) DGWE2 : out of range
Severe: avg_fgdepar=23.1435, expected range: -1.7387 8.2543
Severe: rms_fgdepar=23.1435, expected range: 0.0000 8.2543
Severe: avg_pge=0.9257, expected range: -1.2116 0.7500
BUFR SHIP SYNOP WIND VECTOR DIFF (M/S) V7WS4 : out of range
Severe: avg_pge=0.8945, expected range: -1.2016 0.7500
BUFR SHIP SYNOP WIND VECTOR DIFF (M/S) WADN : out of range
Severe: avg_pge=0.8752, expected range: -1.4776 0.7500
BUFR SHIP SYNOP SURFACE PRESSURE (PA) OWJG2 : out of range
3 times in the last 10 days for at least one item
Severe: avg_fgdepar=786.8749, expected range: -987.1692 793.5325
Severe: rms_fgdepar=786.8749, expected range: 0.0000 620.2380
Severe: avg_pge=0.8708, expected range: -0.6800 0.7500
BUFR SHIP SYNOP SURFACE PRESSURE (PA) OUOW2 : out of range
3 times in the last 10 days for at least one item
Severe: avg_pge=0.8207, expected range: -1.0670 0.7500
BUFR SHIP SYNOP SURFACE PRESSURE (PA) TBWUK76 : out of range
Severe: avg_fgdepar=815.5994, expected range: -318.2224 171.9985
Severe: rms_fgdepar=815.5994, expected range: 0.0000 393.8522
Severe: avg_pge=0.8670, expected range: -0.9717 0.7500
buoy
DRIFTING BUOYS SURFACE PRESSURE (PA) 1601531 : out of range
1 times in the last 10 days for at least one item
Severe: rms_fgdepar=522.5353, expected range: 0.0000 98.4235
DRIFTING BUOYS SURFACE PRESSURE (PA) 5501594 : out of range
1 times in the last 10 days for at least one item
Severe: avg_fgdepar=-970.3555, expected range: -157.3943 222.6596
Severe: rms_fgdepar=973.4397, expected range: 0.0000 271.8287
Severe: avg_pge=0.9380, expected range: -0.4111 0.5113
DRIFTING BUOYS SURFACE PRESSURE (PA) 2501647 : out of range
3 times in the last 10 days for at least one item
Severe: avg_pge=0.8420, expected range: -0.8405 0.7500
DRIFTING BUOYS SURFACE PRESSURE (PA) 1501748 : out of range
Severe: rms_fgdepar=372.6088, expected range: 0.0000 118.5621
DRIFTING BUOYS SURFACE PRESSURE (PA) 1701529 : out of range
4 times in the last 10 days for at least one item
Severe: avg_pge=0.8701, expected range: -0.6914 0.7500
DRIFTING BUOYS SURFACE PRESSURE (PA) 1401759 : out of range
Severe: rms_fgdepar=445.7425, expected range: 0.0000 112.7419
metar
METAR SURFACE PRESSURE (PA) LTCR : out of range
8 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=-529.5408, expected range: -604.2184 -291.6893
Severely_persistent: rms_fgdepar=529.9451, expected range: 293.9480 604.3851
METAR SURFACE PRESSURE (PA) SBKG : out of range
Severe: rms_fgdepar=423.5694, expected range: 0.0000 120.4706
snow
BUFR LAND SYNOP SNOW (CM) 01035 : out of range
38 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=84.1487, expected range: 79.1755 130.6294
Severely_persistent: rms_fgdepar=84.1505, expected range: 79.1975 130.6175
t2m
AUTOMATIC LAND SYNOP 2M TEMPERATURE (K) 89577 : out of range
13 times in the last 10 days for at least one item
Severely_persistent: avg_est_pge=0.8146, expected range: 0.5426 0.9853
MANUAL LAND SYNOP 2M TEMPERATURE (K) 89606 : out of range
8 times in the last 10 days for at least one item
Severely_persistent: avg_est_pge=0.8015, expected range: 0.1339 1.1808
MANUAL LAND SYNOP 2M TEMPERATURE (K) 89009 : out of range
8 times in the last 10 days for at least one item
Severely_persistent: avg_est_pge=0.8209, expected range: 0.1761 1.1361
BUFR LAND SYNOP 2M TEMPERATURE (K) 27415 : out of range
30 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=-16.7066, expected range: -22.1185 -11.9201
Severely_persistent: rms_fgdepar=16.7066, expected range: 12.0781 22.0324
BUFR LAND SYNOP 2M TEMPERATURE (K) 36026 : out of range
14 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=-18.7155, expected range: -30.4200 17.8441
Severely_persistent: rms_fgdepar=18.7155, expected range: 0.0000 28.8074
BUFR LAND SYNOP 2M TEMPERATURE (K) 89577 : out of range
12 times in the last 10 days for at least one item
Severely_persistent: avg_est_pge=0.8063, expected range: 0.5555 0.9768
AUTOMATIC LAND SYNOP 2M TEMPERATURE (K) 68817 : out of range
2 times in the last 10 days for at least one item
Severe: avg_fgdepar=12.0320, expected range: -1.9514 6.0212
Severe: rms_fgdepar=12.1429, expected range: 0.0000 6.1810
AUTOMATIC LAND SYNOP 2M TEMPERATURE (K) 11858 : out of range
1 times in the last 10 days for at least one item
Severe: avg_fgdepar=-16.9218, expected range: -8.7025 4.3811
Severe: rms_fgdepar=16.9218, expected range: 0.0000 12.8374
Severe: avg_est_pge=0.8310, expected range: -0.3895 0.8624
MANUAL LAND SYNOP 2M TEMPERATURE (K) 42056 : out of range
Severe: rms_fgdepar=11.2761, expected range: 0.2972 4.7022