Operational Data Checking

An automatic data checking system has been implemented at ECMWF to trigger warnings in case of anomalies affecting the quality or the availability of the data assimilated by the model. Selected statistical parameters (number of observations, bias correction, and mean bias-corrected background, analysis departures and probability of gross error) are checked against an expected range. An appropriate alert message (including a time series plot) is generated if statistics are outside the specified ranges. A severity level (slight, considerable, severe) is assigned to each message depending on how far statistics are from the expected values. Two kinds of ranges are used by the automatic checking: Soft and Hard limits. Soft limits are updated automatically using statistics from the last twenty days (extremes are excluded during this process). Hard limits are adjusted manually when required.

Currently, the automatic checking is limited to data passing through the minimisation process (including VarBC passive data). It's being applied twice a day to the delayed cut-off 4D-VAR cycles (00 and 12) and four times a day to the early delivery assimilation system (00,06,12 and 18).

The alarm system results are based on feedback information from the ECMWF data assimilation and consequently reflect the ECMWF data usage. Although most warnings are related to data problems, they may on occasion also be related to model issues. Alerts are provided for information on an as-is basis and may be withdrawn or changed without notice. ECMWF accepts no responsibility for actions taken on the basis of these results.

Delayed cut-off 4DVAR at 2020-07-08 00:00

amsua
NOAA-19 AMSUA RADIANCES (8 channels affected) : out of range
Considerable: obs_number=6663.0000, expected range: 7994.3419 8701.7831
cris
NOAA20 CRIS RADIANCES CHANNEL 1189 : out of range
Considerable: avg_biascorr=-0.3052, expected range: -0.4381 -0.3513
saphir
All MEGHA-TROPIQUES SAPHIR ALLSKY ALL CHANNELS Global : all data missing
Severe: obs_number=0, expected range: 50.80 16607.27
gpsro
All KOMPSAT-5 GPSRO SETTING ALL HEIGHTS Global : all data missing
Severe: obs_number=0, expected range: 302.72 1328.81
SPIRE LEMUR 3U GPSRO RISING (16 impact heights affected) : out of range
Severe: stdev_norm_fgdepar=0.7900, expected range: 0.5862 0.6848
SPIRE LEMUR 3U GPSRO RISING (14 impact heights affected) : out of range
Severe: avg_norm_fgdepar=-0.2543, expected range: -0.1238 0.0153
SPIRE LEMUR 3U GPSRO SETTING (15 impact heights affected) : out of range
Considerable: avg_norm_fgdepar=0.0769, expected range: -0.1183 0.0078
SPIRE LEMUR 3U GPSRO SETTING (11 impact heights affected) : out of range
Slight: stdev_norm_fgdepar=0.7363, expected range: 0.5677 0.6925
resat
NOAA-19 SBUV OZONE, TOP PRESSURE LAYER 1-2 (HPA) : out of range
13 times in the last 10 days for at least one item
Severely_persistent: stdev_fgdepar=0.0230, expected range: 0.0023 0.0131
NOAA-19 SBUV OZONE, TOP PRESSURE LAYER 20-21 (HPA) : out of range
18 times in the last 10 days for at least one item
Slight: stdev_fgdepar=0.0128, expected range: 0.0074 0.0113
Slight: avg_fgdepar=-0.0429, expected range: -0.0404 -0.0333
Radiosondes
LAND TEMP Q (G/KG) 37259 PRESSURE LAYER 900-1200 (HPA) : out of range
7 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=13.3950, expected range: -6.6483 11.3326
Severely_persistent: rms_fgdepar=13.3950, expected range: 0.0000 11.5102
Severely_persistent: avg_pge=0.9970, expected range: -0.1300 0.1709
HIGH RESO LAND TEMP TEMPERATURE (K) (2 pressure layers affected) : missing
2 times in the last 10 days for at least one item
Severely_persistent: obs_number=0.0000, expected range: 92.8962 220.4371
Severely_persistent: obs_number=0.0000, expected range: 71.6180 361.6321
Severe: obs_number=0.0000, expected range: 43.3719 370.4001
Severe: obs_number=0.0000, expected range: 43.3719 370.4001
All HIGH RESO LAND TEMP ALL LAYERS WMO BLOCK 37 : all data missing
Severely_persistent: obs_number=0, expected range: 27.73 195.87
All HIGH RESO LAND TEMP ALL LAYERS South pole : all data missing
Severely_persistent: obs_number=0, expected range: 48.72 363.28
LAND TEMP TEMPERATURE (K) PRESSURE LAYER 0-300 (HPA) : out of range
3 times in the last 10 days for at least one item
Severe: rms_fgdepar=7.5115, expected range: 0.8669 1.7660
Severe: rms_fgdepar=3.2881, expected range: 1.4505 2.2787
Severe: avg_fgdepar=-3.2339, expected range: -0.3006 0.4042
Severe: rms_fgdepar=10.0572, expected range: 1.1133 1.8977
LAND TEMP TEMPERATURE (K) 61660 PRESSURE LAYER 0-300 (HPA) : out of range
4 times in the last 10 days for at least one item
Severe: avg_fgdepar=-14.6632, expected range: -1.1839 1.3122
Severe: rms_fgdepar=21.4178, expected range: 0.2794 2.4915
LAND TEMP Q (G/KG) 98618 PRESSURE LAYER 300-600 (HPA) : out of range
Severe: rms_fgdepar=2.0761, expected range: 0.0000 1.1139
HIGH RESO LAND TEMP TEMPERATURE (K) 07761 PRESSURE LAYER 0-300 (HPA) : out of range
Severe: avg_fgdepar=15.1522, expected range: -0.0679 1.0925
Severe: rms_fgdepar=15.2449, expected range: 0.2882 1.8989
Severe: avg_pge=0.8428, expected range: -0.0322 0.0449
HIGH RESO LAND TEMP TEMPERATURE (K) PRESSURE LAYER 0-300 (HPA) : out of range
Severe: rms_fgdepar=4.1454, expected range: 0.5667 1.1520
HIGH RESO LAND TEMP TEMPERATURE (K) PRESSURE LAYER 900-1200 (HPA) : out of range
Severe: avg_fgdepar=-4.7230, expected range: -1.2803 1.2630
HIGH RESO LAND TEMP TEMPERATURE (K) (3 pressure layers affected) : missing
2 times in the last 10 days for at least one item
Severe: obs_number=0.0000, expected range: 77.0127 1226.5799
HIGH RESO LAND TEMP Q (G/KG) (2 pressure layers affected) : out of range
Severe: obs_number=341.0000, expected range: 88.9822 105.0178
All HIGH RESO LAND TEMP ALL LAYERS WMO BLOCK 96 : all data missing
Severe: obs_number=0, expected range: 0.00 487.06
All HIGH RESO LAND TEMP ALL LAYERS Indian Ocean Tropica : all data missing
Severe: obs_number=0, expected range: 0.00 487.06
LAND TEMP WIND VECTOR DIFF (M/S) PRESSURE LAYER 0-300 (HPA) : out of range
Considerable: obs_number=507.0000, expected range: 219.6989 405.5511
HIGH RESO LAND TEMP Q (G/KG) (3 pressure layers affected) : out of range
1 times in the last 10 days for at least one item
Considerable: obs_number=86.0000, expected range: 57.6863 69.5637
synop
AUTOMATIC LAND SYNOP SURFACE PRESSURE (PA) 68754 : out of range
18 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=-718.2966, expected range: -1193.8160 1364.5108
Severely_persistent: rms_fgdepar=718.6789, expected range: -963.7617 1282.2766
Severely_persistent: avg_pge=0.9405, expected range: -1.3326 0.7500
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 68192 : out of range
10 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=-924.2887, expected range: -663.1547 828.2428
Severely_persistent: rms_fgdepar=936.5148, expected range: -639.0577 817.3178
Severely_persistent: avg_pge=0.9353, expected range: -1.6441 0.7500
BUFR LAND SYNOP SURFACE PRESSURE (PA) 12215 : out of range
9 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=-638.0477, expected range: -194.9884 297.2595
Severely_persistent: rms_fgdepar=638.9883, expected range: 0.0000 328.8619
Severely_persistent: avg_pge=0.9401, expected range: -1.2091 0.7500
BUFR LAND SYNOP RH2M (%) 94451 : out of range
7 times in the last 10 days for at least one item
Severely_persistent: avg_pge=0.9920, expected range: -2.1620 0.7500
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 43083 : out of range
2 times in the last 10 days for at least one item
Severe: avg_pge=0.8042, expected range: -1.0453 0.7500
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 40634 : out of range
4 times in the last 10 days for at least one item
Severe: avg_pge=0.8399, expected range: -0.8186 0.7500
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 84036 : out of range
2 times in the last 10 days for at least one item
Severe: avg_pge=0.8009, expected range: -0.9595 0.7500
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 42937 : out of range
Severe: avg_fgdepar=-1144.3370, expected range: -528.2161 461.9655
Severe: rms_fgdepar=1144.3370, expected range: 0.0000 451.1627
Severe: avg_pge=0.9483, expected range: -1.0767 0.7500
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 62318 : out of range
Severe: rms_fgdepar=549.4071, expected range: 0.0000 63.6299
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 41759 : out of range
Severe: rms_fgdepar=449.5639, expected range: 0.0000 186.1910
Severe: avg_pge=0.8758, expected range: -0.4423 0.6011
MANUAL LAND SYNOP RH2M (%) 76040 : out of range
4 times in the last 10 days for at least one item
Severe: avg_pge=0.9972, expected range: -0.8839 0.7500
MANUAL LAND SYNOP RH2M (%) 68098 : out of range
3 times in the last 10 days for at least one item
Severe: avg_fgdepar=54.6237, expected range: -70.7854 95.8115
Severe: rms_fgdepar=54.6237, expected range: -70.7854 95.8115
Severe: avg_pge=0.8658, expected range: -0.1395 0.1771
BUFR LAND SYNOP SURFACE PRESSURE (PA) 27402 : out of range
Severe: rms_fgdepar=359.9435, expected range: 0.0000 124.5131
BUFR LAND SYNOP SURFACE PRESSURE (PA) 23862 : out of range
Severe: rms_fgdepar=302.7051, expected range: 0.0000 171.8158
BUFR LAND SYNOP SURFACE PRESSURE (PA) 27215 : out of range
Severe: rms_fgdepar=306.3276, expected range: 0.0000 74.6688
BUFR LAND SYNOP SURFACE PRESSURE (PA) 71446 : out of range
Severe: rms_fgdepar=308.3483, expected range: 0.0000 123.7841
BUFR LAND SYNOP SURFACE PRESSURE (PA) 26499 : out of range
Severe: rms_fgdepar=364.6606, expected range: 0.0000 61.3545
BUFR LAND SYNOP RH2M (%) 72520 : out of range
Severe: rms_fgdepar=34.1446, expected range: 0.0000 13.3752
BUFR LAND SYNOP RH2M (%) 71301 : out of range
1 times in the last 10 days for at least one item
Severe: avg_pge=0.8369, expected range: -1.3960 0.7500
BUFR LAND SYNOP RH2M (%) 27523 : out of range
Severe: rms_fgdepar=34.3633, expected range: 0.0000 17.9579
BUFR LAND SYNOP RH2M (%) 60025 : out of range
Severe: avg_pge=0.8454, expected range: -0.0332 0.0525
MANUAL LAND SYNOP RH2M (%) : out of range
Considerable: obs_number=184.0000, expected range: 135.6697 161.2192
Considerable: obs_number=96.0000, expected range: 72.0618 77.7382
BUFR LAND SYNOP SURFACE PRESSURE (PA) : out of range
1 times in the last 10 days for at least one item
Considerable: obs_number=123.0000, expected range: 101.8413 114.6587
ship
BUFR SHIP SYNOP WIND VECTOR DIFF (M/S) AUBC : out of range
9 times in the last 10 days for at least one item
Severely_persistent: avg_pge=0.9055, expected range: 0.7518 0.7500
BUFR SHIP SYNOP SURFACE PRESSURE (PA) OZBY2 : out of range
9 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=871.9222, expected range: -496.6979 1183.9979
Severely_persistent: rms_fgdepar=871.9222, expected range: 0.0000 1215.5042
Severely_persistent: avg_pge=0.8829, expected range: -0.8314 0.7500
BUFR SHIP SYNOP SURFACE PRESSURE (PA) V7FF3 : out of range
8 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=1090.4679, expected range: 704.8700 1209.4837
Severely_persistent: rms_fgdepar=1090.4679, expected range: 704.8700 1209.4837
Severely_persistent: avg_pge=0.9044, expected range: 0.8726 0.7500
BUFR SHIP SYNOP SURFACE PRESSURE (PA) 9V5245 : out of range
9 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=754.6965, expected range: 331.9350 1438.1791
Severely_persistent: rms_fgdepar=754.6965, expected range: 331.9350 1438.1791
Severely_persistent: avg_pge=0.8637, expected range: 0.8020 0.7500
SHIP SURFACE PRESSURE (PA) OZBY2 : out of range
4 times in the last 10 days for at least one item
Severe: avg_fgdepar=871.9222, expected range: -589.6374 1198.0849
Severe: rms_fgdepar=871.9222, expected range: 0.0000 1218.5133
SHIP SURFACE PRESSURE (PA) V7FF3 : out of range
4 times in the last 10 days for at least one item
Severe: avg_fgdepar=1090.4679, expected range: 782.0845 1135.4842
Severe: rms_fgdepar=1090.4679, expected range: 782.0845 1135.4842
SHIP SURFACE PRESSURE (PA) 9V5245 : out of range
4 times in the last 10 days for at least one item
Severe: avg_fgdepar=754.6965, expected range: 357.5357 1446.5925
Severe: rms_fgdepar=754.6965, expected range: 357.5357 1446.5925
BUFR SHIP SYNOP WIND VECTOR DIFF (M/S) A8UC5 : out of range
1 times in the last 10 days for at least one item
Severe: avg_fgdepar=21.4536, expected range: -4.1283 16.6929
Severe: rms_fgdepar=21.4536, expected range: 0.0000 16.6929
Severe: avg_pge=0.9142, expected range: -0.4243 0.7500
BUFR SHIP SYNOP WIND VECTOR DIFF (M/S) A8IE8 : out of range
Severe: avg_pge=0.8012, expected range: -0.9217 0.7500
BUFR SHIP SYNOP SURFACE PRESSURE (PA) OUOW2 : out of range
Severe: avg_fgdepar=-565.5551, expected range: -246.2796 619.2241
Severe: rms_fgdepar=565.5657, expected range: 0.0000 626.3060
Severe: avg_pge=0.8419, expected range: -1.0080 0.7500
BUFR SHIP SYNOP SURFACE PRESSURE (PA) WMCU : out of range
Severe: rms_fgdepar=643.9425, expected range: 0.0000 146.0973
All BUFR SHIP SYNOP WMO BLOCK 41 : all data missing
Severe: obs_number=0, expected range: 283.76 329.49
All BUFR SHIP SYNOP WMO BLOCK 42 : all data missing
Severe: obs_number=0, expected range: 242.14 352.24
All BUFR SHIP SYNOP WMO BLOCK 51 : all data missing
Severe: obs_number=0, expected range: 82.42 122.58
BUFR SHIP SYNOP SURFACE PRESSURE (PA) : out of range
1 times in the last 10 days for at least one item
Considerable: obs_number=3902.0000, expected range: 5591.1725 6673.0775
Considerable: obs_number=755.0000, expected range: 1941.2843 2543.5907
Considerable: obs_number=95.0000, expected range: 940.8133 1161.3117
buoy
DRIFTING BUOYS SURFACE PRESSURE (PA) 5601595 : out of range
10 times in the last 10 days for at least one item
Severely_persistent: avg_pge=0.8076, expected range: -1.0574 0.7500
MOORED BUOYS SURFACE PRESSURE (PA) : out of range
2 times in the last 10 days for at least one item
Severe: obs_number=2705.0000, expected range: 935.1901 1078.8099
Severe: obs_number=836.0000, expected range: 478.4713 517.0287
Severe: obs_number=1347.0000, expected range: 152.5863 162.0453
Severe: obs_number=87.0000, expected range: 141.5863 151.0453
Severe: obs_number=433.0000, expected range: 101.2059 112.3496
Severe: obs_number=490.0000, expected range: 137.5236 159.2542
Severe: obs_number=106.0000, expected range: 83.9159 87.7507
Considerable: obs_number=355.0000, expected range: 427.4238 550.3262
Considerable: obs_number=178.0000, expected range: 206.1446 249.2672
All MOORED BUOYS WMO BLOCK 32 : all data missing
Severe: obs_number=0, expected range: 29.95 138.17
All MOORED BUOYS South West Pacific : all data missing
Severe: obs_number=0, expected range: 99.00 145.75
DRIFTING BUOYS SURFACE PRESSURE (PA) : out of range
Severe: obs_number=8052.0000, expected range: 8902.7854 9085.4646
Considerable: obs_number=1292.0000, expected range: 1409.6763 1604.5737
Considerable: obs_number=693.0000, expected range: 761.9922 842.6328
Considerable: obs_number=916.0000, expected range: 988.8577 1085.2599
DRIFTING BUOYS SURFACE PRESSURE (PA) 7101533 : out of range
1 times in the last 10 days for at least one item
Severe: rms_fgdepar=461.0212, expected range: 0.0000 234.6602
metar
METAR SURFACE PRESSURE (PA) OIBH : out of range
5 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=-578.9062, expected range: -746.0061 -89.2510
Severely_persistent: rms_fgdepar=580.3111, expected range: 96.9288 745.5976
Severely_persistent: avg_pge=0.8144, expected range: 0.1368 0.7500
METAR SURFACE PRESSURE (PA) GMMB : out of range
1 times in the last 10 days for at least one item
Severe: rms_fgdepar=347.7309, expected range: 0.0000 147.7409
METAR SURFACE PRESSURE (PA) SAOC : out of range
Severe: rms_fgdepar=333.3492, expected range: 0.0000 130.5308
METAR SURFACE PRESSURE (PA) EGYE : out of range
1 times in the last 10 days for at least one item
Severe: rms_fgdepar=445.2726, expected range: 0.0000 158.1351
METAR SURFACE PRESSURE (PA) OPPI : out of range
Severe: rms_fgdepar=465.0896, expected range: 0.0000 234.7352
Severe: avg_pge=0.8154, expected range: -0.5485 0.6319
snow
AUTOMATIC LAND SYNOP SNOW (CM) 11316 : out of range
8 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=55.7375, expected range: 45.9077 147.2201
Severely_persistent: rms_fgdepar=55.7375, expected range: 45.9077 147.2201
BUFR LAND SYNOP SNOW (CM) 01035 : out of range
32 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=91.4582, expected range: 92.7699 129.6738
Severely_persistent: rms_fgdepar=91.4586, expected range: 92.7718 129.6766
BUFR LAND SYNOP SNOW (CM) 11316 : out of range
8 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=55.7367, expected range: 45.9192 147.2012
Severely_persistent: rms_fgdepar=55.7367, expected range: 45.9192 147.2012
AUTOMATIC LAND SYNOP SNOW (CM) 01035 : out of range
4 times in the last 10 days for at least one item
Severe: avg_fgdepar=91.1148, expected range: 94.9485 127.0320
Severe: rms_fgdepar=91.1148, expected range: 94.9485 127.0320
t2m
MANUAL LAND SYNOP 2M TEMPERATURE (K) 89009 : out of range
5 times in the last 10 days for at least one item
Severely_persistent: avg_est_pge=0.8097, expected range: 0.0936 1.1449
BUFR LAND SYNOP 2M TEMPERATURE (K) 27415 : out of range
28 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=-18.8508, expected range: -21.6339 -12.6566
Severely_persistent: rms_fgdepar=18.8508, expected range: 12.8562 21.5114
BUFR LAND SYNOP 2M TEMPERATURE (K) 36026 : out of range
12 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=-18.4224, expected range: -24.7342 15.9088
Severely_persistent: rms_fgdepar=18.4224, expected range: 0.0000 24.4935
AUTOMATIC LAND SYNOP 2M TEMPERATURE (K) 68380 : out of range
Severe: avg_fgdepar=18.3068, expected range: -6.7874 8.2305
Severe: rms_fgdepar=18.3603, expected range: 0.0000 7.4324
Severe: avg_est_pge=0.8808, expected range: -0.3908 0.8313
MANUAL LAND SYNOP 2M TEMPERATURE (K) 87405 : out of range
3 times in the last 10 days for at least one item
Severe: avg_fgdepar=13.8608, expected range: -5.0449 14.5654
Severe: rms_fgdepar=13.8608, expected range: 0.0000 13.1363
Severe: avg_est_pge=0.8487, expected range: 0.1348 1.0275
MANUAL LAND SYNOP 2M TEMPERATURE (K) 89606 : out of range
3 times in the last 10 days for at least one item
Severe: avg_est_pge=0.8215, expected range: 0.1449 1.1196
METAR 2M TEMPERATURE (K) KMWC : out of range
1 times in the last 10 days for at least one item
Severe: rms_fgdepar=12.3432, expected range: 0.3103 5.6099
METAR 2M TEMPERATURE (K) EGNR : out of range
Severe: rms_fgdepar=12.9963, expected range: 0.1072 2.6075
BUFR LAND SYNOP 2M TEMPERATURE (K) 68380 : out of range
Severe: avg_fgdepar=19.6651, expected range: -6.8344 9.2935
Severe: rms_fgdepar=19.6651, expected range: 0.0000 7.6514
BUFR LAND SYNOP 2M TEMPERATURE (K) 87405 : out of range
2 times in the last 10 days for at least one item
Severe: avg_fgdepar=13.8108, expected range: -5.1840 15.2665
Severe: rms_fgdepar=13.8108, expected range: 0.0000 14.0800
Severe: avg_est_pge=0.8482, expected range: 0.1162 1.0393