Operational Data Checking

An automatic data checking system has been implemented at ECMWF to trigger warnings in case of anomalies affecting the quality or the availability of the data assimilated by the model. Selected statistical parameters (number of observations, bias correction, and mean bias-corrected background, analysis departures and probability of gross error) are checked against an expected range. An appropriate alert message (including a time series plot) is generated if statistics are outside the specified ranges. A severity level (slight, considerable, severe) is assigned to each message depending on how far statistics are from the expected values. Two kinds of ranges are used by the automatic checking: Soft and Hard limits. Soft limits are updated automatically using statistics from the last twenty days (extremes are excluded during this process). Hard limits are adjusted manually when required.

Currently, the automatic checking is limited to data passing through the minimisation process (including VarBC passive data). It's being applied twice a day to the delayed cut-off 4D-VAR cycles (00 and 12) and four times a day to the early delivery assimilation system (00,06,12 and 18).

The alarm system results are based on feedback information from the ECMWF data assimilation and consequently reflect the ECMWF data usage. Although most warnings are related to data problems, they may on occasion also be related to model issues. Alerts are provided for information on an as-is basis and may be withdrawn or changed without notice. ECMWF accepts no responsibility for actions taken on the basis of these results.

Delayed cut-off 4DVAR at 2020-07-09 00:00

amsua
METOP-C AMSUA RADIANCES CHANNEL 5 : out of range
Considerable: avg_biascorr=-0.1147, expected range: -0.0949 -0.0678
cris
NPP CRIS RADIANCES (74 channels affected) : out of range
Severe: obs_number=19070.0000, expected range: 21999.3232 22655.6768
NOAA20 CRIS RADIANCES (89 channels affected) : out of range
Severe: obs_number=19742.0000, expected range: 22838.9595 23198.7905
saphir
All MEGHA-TROPIQUES SAPHIR ALLSKY ALL CHANNELS Global : all data missing
Severely_persistent: obs_number=0, expected range: 128.84 16866.85
mwhs2
FY-3D MWHS2 ALLSKY RADIANCES CHANNEL 2 : out of range
Slight: obs_number=52023.0000, expected range: 55436.9666 57477.6584
FY-3D MWHS2 ALLSKY RADIANCES CHANNEL 3 : out of range
Slight: obs_number=52023.0000, expected range: 55440.4894 57478.6356
FY-3D MWHS2 ALLSKY RADIANCES CHANNEL 4 : out of range
Slight: obs_number=52024.0000, expected range: 55438.8203 57479.1797
atms
NPP ATMS RADIANCES (12 channels affected) : out of range
Considerable: obs_number=20278.0000, expected range: 21411.7171 24835.9079
NOAA20 ATMS RADIANCES (12 channels affected) : out of range
Considerable: obs_number=20326.0000, expected range: 21600.5101 25613.8649
gpsro
All KOMPSAT-5 GPSRO SETTING ALL HEIGHTS Global : all data missing
Severe: obs_number=0, expected range: 286.49 1273.56
SPIRE LEMUR 3U GPSRO RISING HEIGHT 9 : out of range
Considerable: stdev_norm_fgdepar=1.6734, expected range: 1.3746 1.5718
COSMIC2-E5 GPSRO RISING HEIGHT 11 : out of range
Slight: stdev_norm_fgdepar=1.4277, expected range: 0.9484 1.2951
Atmospheric Motion Vector
NPP VIIRS AMV IR WIND VECTOR DIFF (M/S) PRESSURE LAYER 300-600 (HPA) : out of range
Considerable: obs_number=1473.0000, expected range: 2527.3705 3281.8795
NOAA20 VIIRS-AMV IR WIND VECTOR DIFF (M/S) PRESSURE LAYER 300-600 (HPA : out of range
Considerable: obs_number=1755.0000, expected range: 2437.7741 3662.7259
resat
NOAA-19 SBUV OZONE, TOP PRESSURE LAYER 20-21 (HPA) : out of range
21 times in the last 10 days for at least one item
Severely_persistent: stdev_fgdepar=0.0150, expected range: 0.0074 0.0113
NOAA-19 SBUV OZONE, TOP PRESSURE LAYER 1-2 (HPA) : out of range
15 times in the last 10 days for at least one item
Severely_persistent: stdev_fgdepar=0.0231, expected range: 0.0023 0.0131
Radiosondes
LAND TEMP TEMPERATURE (K) 61660 PRESSURE LAYER 0-300 (HPA) : out of range
8 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=-12.7650, expected range: -57.4705 48.9144
Severely_persistent: rms_fgdepar=20.1065, expected range: -52.1404 64.4277
HIGH RESO LAND TEMP WIND VECTOR DIFF (M/S) (2 pressure layers affected : missing
4 times in the last 10 days for at least one item
Severely_persistent: obs_number=0.0000, expected range: 59.2013 375.4745
Severe: obs_number=0.0000, expected range: 0.0000 443.1011
HIGH RESO LAND TEMP TEMPERATURE (K) (2 pressure layers affected) : missing
6 times in the last 10 days for at least one item
Severely_persistent: obs_number=0.0000, expected range: 104.5025 205.4975
All HIGH RESO LAND TEMP ALL LAYERS WMO BLOCK 37 : all data missing
Severely_persistent: obs_number=0, expected range: 27.73 195.87
All HIGH RESO LAND TEMP ALL LAYERS South pole : all data missing
Severely_persistent: obs_number=0, expected range: 38.43 382.57
LAND TEMP WIND VECTOR DIFF (M/S) 31873 PRESSURE LAYER 300-600 (HPA) : out of range
Severe: rms_fgdepar=16.5937, expected range: 0.0000 6.1845
LAND TEMP WIND VECTOR DIFF (M/S) 34247 PRESSURE LAYER 900-1200 (HPA) : out of range
Severe: avg_pge=0.8459, expected range: -0.7167 0.7500
LAND TEMP Q (G/KG) 41256 PRESSURE LAYER 900-1200 (HPA) : out of range
Severe: rms_fgdepar=6.5995, expected range: 0.0000 7.5373
HIGH RESO LAND TEMP TEMPERATURE (K) 27713 PRESSURE LAYER 0-300 (HPA) : out of range
Severe: rms_fgdepar=10.0293, expected range: 0.2319 2.0732
Severe: avg_pge=0.8122, expected range: -0.0523 0.0682
HIGH RESO LAND TEMP TEMPERATURE (K) PRESSURE LAYER 0-300 (HPA) : out of range
Severe: rms_fgdepar=4.1244, expected range: 0.2040 1.9845
HIGH RESO LAND TEMP Q (G/KG) 74389 PRESSURE LAYER 900-1200 (HPA) : out of range
Severe: rms_fgdepar=4.2839, expected range: 0.0000 2.0046
HIGH RESO LAND TEMP Q (G/KG) (3 pressure layers affected) : out of range
2 times in the last 10 days for at least one item
Severe: obs_number=124.0000, expected range: 58.0717 68.6783
HIGH RESO LAND TEMP Q (G/KG) PRESSURE LAYER 600-900 (HPA) : out of range
Severe: rms_fgdepar=1.5616, expected range: 0.5444 0.9764
HIGH RESO LAND TEMP Q (G/KG) PRESSURE LAYER 900-1200 (HPA) : out of range
1 times in the last 10 days for at least one item
Severe: rms_fgdepar=2.9093, expected range: 0.2898 1.7369
Severe: obs_number=0.0000, expected range: 29.2843 262.1443
HIGH RESO LAND TEMP Q (G/KG) PRESSURE LAYER 0-300 (HPA) : missing
1 times in the last 10 days for at least one item
Severe: obs_number=0.0000, expected range: 53.4627 72.8706
All HIGH RESO LAND TEMP ALL LAYERS WMO BLOCK 15 : all data missing
Severe: obs_number=0, expected range: 0.00 321.96
All HIGH RESO LAND TEMP ALL LAYERS WMO BLOCK 63 : all data missing
Severe: obs_number=0, expected range: 41.31 178.37
LAND TEMP WIND VECTOR DIFF (M/S) PRESSURE LAYER 0-300 (HPA) : out of range
Considerable: obs_number=338.0000, expected range: 400.3246 516.1754
HIGH RESO LAND TEMP Q (G/KG) PRESSURE LAYER 300-600 (HPA) : out of range
Considerable: obs_number=115.0000, expected range: 165.4215 263.8285
HIGH RESO LAND TEMP Q (G/KG) (2 pressure layers affected) : out of range
1 times in the last 10 days for at least one item
Considerable: obs_number=139.0000, expected range: 89.1118 104.6382
BUFR TEMP DESCENT WIND VECTOR DIFF (M/S) PRESSURE LAYER 0-300 (HPA) : out of range
Considerable: obs_number=1052.0000, expected range: 2073.8663 3842.7051
amdar
WIGOS AMDAR WIND VECTOR DIFF (M/S) QXKXR2JA PRESSURE LAYER 900-1200 (H : out of range
Severe: rms_fgdepar=16.6069, expected range: 0.0000 7.9703
pilot
LAND PILOT WIND VECTOR DIFF (M/S) 48480 PRESSURE LAYER 0-300 (HPA) : out of range
1 times in the last 10 days for at least one item
Severe: rms_fgdepar=24.3549, expected range: 0.0000 29.2755
LAND PILOT WIND VECTOR DIFF (M/S) 48551 PRESSURE LAYER 0-300 (HPA) : out of range
Severe: rms_fgdepar=21.4196, expected range: 0.0000 30.9326
synop
AUTOMATIC LAND SYNOP RH2M (%) 35182 : out of range
14 times in the last 10 days for at least one item
Severely_persistent: avg_pge=0.9810, expected range: -1.1362 0.7500
MANUAL LAND SYNOP RH2M (%) 76040 : out of range
5 times in the last 10 days for at least one item
Severely_persistent: avg_pge=1.0000, expected range: -1.0457 0.7500
BUFR LAND SYNOP RH2M (%) 94451 : out of range
8 times in the last 10 days for at least one item
Severely_persistent: avg_pge=0.9715, expected range: -2.1220 0.7500
AUTOMATIC LAND SYNOP RH2M (%) 40550 : out of range
Severe: rms_fgdepar=31.4360, expected range: 0.0000 11.9277
AUTOMATIC LAND SYNOP RH2M (%) 40594 : out of range
Severe: rms_fgdepar=33.6932, expected range: 0.0000 17.9793
MANUAL LAND SYNOP RH2M (%) 60390 : out of range
Severe: avg_pge=0.9417, expected range: -0.0654 0.0868
MANUAL LAND SYNOP RH2M (%) 76571 : out of range
Severe: rms_fgdepar=37.9618, expected range: 0.0000 25.7219
Severe: avg_pge=0.8254, expected range: -2.1814 0.7500
MANUAL LAND SYNOP RH2M (%) 38933 : out of range
2 times in the last 10 days for at least one item
Severe: avg_pge=0.9665, expected range: -0.7461 0.7500
MANUAL LAND SYNOP RH2M (%) 36859 : out of range
4 times in the last 10 days for at least one item
Severe: avg_pge=0.9414, expected range: -0.2554 0.3656
MANUAL LAND SYNOP RH2M (%) : out of range
2 times in the last 10 days for at least one item
Severe: obs_number=96.0000, expected range: 73.1072 76.6428
Considerable: obs_number=183.0000, expected range: 134.7808 160.3303
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 42441 : out of range
1 times in the last 10 days for at least one item
Severe: avg_fgdepar=-538.7565, expected range: -283.2433 249.7731
Severe: rms_fgdepar=538.7565, expected range: -392.9430 576.5830
Severe: avg_pge=0.8944, expected range: -1.4038 0.7500
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 84036 : out of range
3 times in the last 10 days for at least one item
Severe: avg_pge=0.8096, expected range: -0.9595 0.7500
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 43359 : out of range
Severe: rms_fgdepar=315.5310, expected range: 0.0000 232.8242
Severe: avg_pge=0.8285, expected range: -0.9870 0.7500
BUFR LAND SYNOP RH2M (%) 27752 : out of range
Severe: rms_fgdepar=31.0533, expected range: 0.0000 16.0544
BUFR LAND SYNOP RH2M (%) 72533 : out of range
Severe: rms_fgdepar=41.9258, expected range: 0.0000 20.5687
BUFR LAND SYNOP RH2M (%) 60253 : out of range
4 times in the last 10 days for at least one item
Severe: avg_pge=0.8630, expected range: -1.9449 0.7500
BUFR LAND SYNOP RH2M (%) 72526 : out of range
Severe: avg_pge=0.8838, expected range: -0.0906 0.1395
BUFR LAND SYNOP RH2M (%) 71398 : out of range
Severe: avg_pge=0.9486, expected range: -0.0052 0.0130
BUFR LAND SYNOP RH2M (%) 71309 : out of range
Severe: rms_fgdepar=44.4205, expected range: 0.0000 33.9854
Severe: avg_pge=0.9984, expected range: -0.0072 0.0163
BUFR LAND SYNOP SURFACE PRESSURE (PA) 72552 : out of range
Severe: rms_fgdepar=372.7877, expected range: 0.0000 204.4915
BUFR LAND SYNOP SURFACE PRESSURE (PA) 63624 : out of range
2 times in the last 10 days for at least one item
Severe: avg_fgdepar=-541.7291, expected range: -97.2751 31.8096
Severe: rms_fgdepar=753.1218, expected range: 0.0000 92.1671
BUFR LAND SYNOP SURFACE PRESSURE (PA) 94215 : out of range
Severe: avg_fgdepar=720.0181, expected range: -103.2913 28.2822
Severe: rms_fgdepar=720.0181, expected range: 0.0000 103.2913
Severe: avg_pge=0.9153, expected range: -0.5623 0.6426
BUFR LAND SYNOP SURFACE PRESSURE (PA) 94645 : out of range
Severe: rms_fgdepar=673.1641, expected range: 0.0000 133.8380
ship
BUFR SHIP SYNOP WIND VECTOR DIFF (M/S) OZDQ2 : out of range
17 times in the last 10 days for at least one item
Severely_persistent: avg_pge=0.8727, expected range: 0.2134 0.7500
BUFR SHIP SYNOP SURFACE PRESSURE (PA) C6DQ4 : out of range
5 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=-837.1545, expected range: -1367.4205 938.1511
Severely_persistent: rms_fgdepar=837.1545, expected range: 0.0000 1577.2326
Severely_persistent: avg_pge=0.8901, expected range: -1.3856 0.7500
SHIP SURFACE PRESSURE (PA) C6DQ4 : out of range
2 times in the last 10 days for at least one item
Severe: avg_fgdepar=-837.1545, expected range: -1539.9920 1272.8573
Severe: rms_fgdepar=837.1545, expected range: 0.0000 1567.4621
SHIP SURFACE PRESSURE (PA) KGTZ : out of range
Severe: rms_fgdepar=314.6821, expected range: 0.0000 142.4017
BUFR SHIP SYNOP WIND VECTOR DIFF (M/S) WYQ4356 : out of range
Severe: rms_fgdepar=18.4101, expected range: 0.0000 6.7974
Severe: avg_pge=0.8981, expected range: -0.8189 0.7500
BUFR SHIP SYNOP WIND VECTOR DIFF (M/S) EUCDE37 : out of range
Severe: avg_fgdepar=20.1400, expected range: -1.5355 35.5520
Severe: rms_fgdepar=20.1400, expected range: 6.1036 29.9340
Severe: avg_pge=0.9144, expected range: -0.0084 0.7500
BUFR SHIP SYNOP SURFACE PRESSURE (PA) DHPG2 : out of range
Severe: rms_fgdepar=365.9500, expected range: 0.0000 89.8372
BUFR SHIP SYNOP SURFACE PRESSURE (PA) 3FXM3 : out of range
Severe: avg_fgdepar=733.8496, expected range: -98.6168 603.9940
Severe: rms_fgdepar=733.8496, expected range: 0.0000 610.3849
Severe: avg_pge=0.8595, expected range: -0.4898 0.7500
BUFR SHIP SYNOP SURFACE PRESSURE (PA) VRPT5 : out of range
Severe: avg_fgdepar=-511.6382, expected range: -432.9382 307.7246
Severe: rms_fgdepar=567.8616, expected range: 0.0000 390.7862
All BUFR SHIP SYNOP WMO BLOCK 41 : all data missing
Severe: obs_number=0, expected range: 232.57 373.43
All BUFR SHIP SYNOP WMO BLOCK 42 : all data missing
Severe: obs_number=0, expected range: 170.52 411.36
All BUFR SHIP SYNOP WMO BLOCK 51 : all data missing
Severe: obs_number=0, expected range: 63.05 136.33
BUFR SHIP SYNOP SURFACE PRESSURE (PA) : out of range
3 times in the last 10 days for at least one item
Considerable: obs_number=3942.0000, expected range: 5332.3911 6887.2339
Considerable: obs_number=753.0000, expected range: 1769.8510 2687.6490
Considerable: obs_number=77.0000, expected range: 768.6580 1305.8420
buoy
DRIBU SURFACE PRESSURE (PA) 23491 : out of range
1 times in the last 10 days for at least one item
Severe: avg_pge=0.8370, expected range: -0.2898 0.7500
MOORED BUOYS SURFACE PRESSURE (PA) : out of range
3 times in the last 10 days for at least one item
Severe: obs_number=2598.0000, expected range: 880.4111 1117.5889
Severe: obs_number=1292.0000, expected range: 120.9017 190.6538
Severe: obs_number=392.0000, expected range: 101.0848 112.3597
Considerable: obs_number=816.0000, expected range: 458.0350 533.4944
Considerable: obs_number=458.0000, expected range: 110.5561 181.5550
DRIFTING BUOYS SURFACE PRESSURE (PA) 5401537 : out of range
1 times in the last 10 days for at least one item
Severe: rms_fgdepar=370.4770, expected range: 5.7085 68.6452
DRIFTING BUOYS SURFACE PRESSURE (PA) 2501647 : out of range
Severe: rms_fgdepar=500.9153, expected range: 0.0000 101.2308
Severe: avg_pge=0.8588, expected range: -0.2629 0.3937
DRIFTING BUOYS SURFACE PRESSURE (PA) 1701529 : out of range
Severe: rms_fgdepar=382.0763, expected range: 0.0000 104.3722
metar
METAR SURFACE PRESSURE (PA) LGKR : out of range
Severe: rms_fgdepar=346.0489, expected range: 0.0000 141.1775
AUTOMATIC METAR SURFACE PRESSURE (PA) KMHK : out of range
Severe: rms_fgdepar=313.9643, expected range: 0.0000 171.2461
snow
AUTOMATIC LAND SYNOP SNOW (CM) 01035 : out of range
6 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=86.1445, expected range: 84.0986 131.6608
Severely_persistent: rms_fgdepar=86.1445, expected range: 84.0986 131.6608
BUFR LAND SYNOP SNOW (CM) 01035 : out of range
36 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=88.9901, expected range: 85.0505 131.3870
Severely_persistent: rms_fgdepar=89.0328, expected range: 85.0526 131.3893
BUFR LAND SYNOP SNOW (CM) : out of range
Severe: rms_fgdepar=63.5356, expected range: 2.1883 3.9680
t2m
AUTOMATIC LAND SYNOP 2M TEMPERATURE (K) 68380 : out of range
6 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=14.8064, expected range: -6.3719 9.6487
Severely_persistent: rms_fgdepar=15.0813, expected range: 0.0000 11.0614
Severely_persistent: avg_est_pge=0.8497, expected range: -0.3808 0.8335
MANUAL LAND SYNOP 2M TEMPERATURE (K) 89606 : out of range
5 times in the last 10 days for at least one item
Severely_persistent: avg_est_pge=0.8119, expected range: 0.1346 1.1391
BUFR LAND SYNOP 2M TEMPERATURE (K) 27415 : out of range
28 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=-14.1955, expected range: -21.6160 -12.6677
Severely_persistent: rms_fgdepar=14.1973, expected range: 12.8676 21.4933
BUFR LAND SYNOP 2M TEMPERATURE (K) 36026 : out of range
12 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=-17.3584, expected range: -29.2716 17.1917
Severely_persistent: rms_fgdepar=17.3584, expected range: 0.0000 27.7882
AUTOMATIC LAND SYNOP 2M TEMPERATURE (K) 68920 : out of range
Severe: rms_fgdepar=11.2493, expected range: 0.0000 3.1835
MANUAL LAND SYNOP 2M TEMPERATURE (K) 87405 : out of range
3 times in the last 10 days for at least one item
Severe: avg_fgdepar=14.6680, expected range: -4.3486 13.1549
Severe: rms_fgdepar=14.6680, expected range: 0.0000 12.1500
Severe: avg_est_pge=0.8567, expected range: 0.1343 0.9743
METAR 2M TEMPERATURE (K) KOCF : out of range
Severe: rms_fgdepar=12.7790, expected range: 0.0000 5.9190
METAR 2M TEMPERATURE (K) KHOP : out of range
Severe: avg_fgdepar=-18.1919, expected range: -11.0573 8.7093
Severe: rms_fgdepar=18.1919, expected range: 0.0000 11.9030
METAR 2M TEMPERATURE (K) EGNR : out of range
1 times in the last 10 days for at least one item
Severe: avg_fgdepar=12.2627, expected range: -1.2779 2.8429
Severe: rms_fgdepar=14.0219, expected range: 0.0000 2.5382
METAR 2M TEMPERATURE (K) KMYR : out of range
Severe: avg_fgdepar=-15.6426, expected range: -1.4997 5.1534
Severe: rms_fgdepar=15.6426, expected range: 0.0000 4.8850
BUFR LAND SYNOP 2M TEMPERATURE (K) 68380 : out of range
4 times in the last 10 days for at least one item
Severe: avg_fgdepar=16.4172, expected range: -5.3152 9.5985
Severe: rms_fgdepar=16.5357, expected range: 0.0000 11.7201
BUFR LAND SYNOP 2M TEMPERATURE (K) 87405 : out of range
3 times in the last 10 days for at least one item
Severe: avg_fgdepar=14.6180, expected range: -4.5166 13.8954
Severe: rms_fgdepar=14.6180, expected range: 0.0000 13.2695
Severe: avg_est_pge=0.8563, expected range: 0.1344 0.9752
rh2m
MANUAL LAND SYNOP RH2M (%) 41255 : out of range
Severe: rms_fgdepar=30.5417, expected range: 0.0000 15.5061
MANUAL LAND SYNOP RH2M (%) 60417 : out of range
Severe: rms_fgdepar=36.4674, expected range: 0.0000 18.3311