Operational Data Checking

An automatic data checking system has been implemented at ECMWF to trigger warnings in case of anomalies affecting the quality or the availability of the data assimilated by the model. Selected statistical parameters (number of observations, bias correction, and mean bias-corrected background, analysis departures and probability of gross error) are checked against an expected range. An appropriate alert message (including a time series plot) is generated if statistics are outside the specified ranges. A severity level (slight, considerable, severe) is assigned to each message depending on how far statistics are from the expected values. Two kinds of ranges are used by the automatic checking: Soft and Hard limits. Soft limits are updated automatically using statistics from the last twenty days (extremes are excluded during this process). Hard limits are adjusted manually when required.

Currently, the automatic checking is limited to data passing through the minimisation process (including VarBC passive data). It's being applied twice a day to the delayed cut-off 4D-VAR cycles (00 and 12) and four times a day to the early delivery assimilation system (00,06,12 and 18).

The alarm system results are based on feedback information from the ECMWF data assimilation and consequently reflect the ECMWF data usage. Although most warnings are related to data problems, they may on occasion also be related to model issues. Alerts are provided for information on an as-is basis and may be withdrawn or changed without notice. ECMWF accepts no responsibility for actions taken on the basis of these results.

Delayed cut-off 4DVAR at 2020-05-28 00:00

AMSUA Pacific RARS
NOAA-19 AMSUA RADIANCES CHANNEL 4 : out of range
Severe: stdev_fgdepar=0.2509, expected range: 0.0000 0.0097
iasi
METOP-C IASI RADIANCES CHANNEL 258 : out of range
Considerable: avg_biascorr=-0.3362, expected range: -0.3008 -0.2470
gpsro
All FY-3C GNOS RISING ALL HEIGHTS Global : all data missing
Severe: obs_number=0, expected range: 467.59 2312.99
TERRASAR-X GPSRO SETTING (7 impact heights affected) : out of range
Considerable: obs_number=332.0000, expected range: 510.8632 1128.1796
COSMIC2-E5 GPSRO SETTING HEIGHT 17 : out of range
Considerable: avg_norm_fgdepar=0.4352, expected range: -0.0194 0.2482
COSMIC2-E5 GPSRO SETTING (24 impact heights affected) : out of range
Considerable: obs_number=240.0000, expected range: 1287.1945 2792.1805
COSMIC2-E6 GPSRO SETTING (14 impact heights affected) : out of range
Considerable: obs_number=1069.0000, expected range: 1359.7157 2308.4093
Atmospheric Motion Vector
HIMAWARI-8 AMV CLOUDY WV2 WIND VECTOR DIFF (M/S) PRESSURE LAYER 300-60 : out of range
Considerable: obs_number=2730.0000, expected range: 1572.2482 2430.7518
METOP-C AMV IR WIND VECTOR DIFF (M/S) PRESSURE LAYER 300-600 (HPA) : out of range
Considerable: obs_number=573.0000, expected range: 267.6539 498.3148
GOES-17 AMV IR1 WIND VECTOR DIFF (M/S) PRESSURE LAYER 0-300 (HPA) : out of range
1 times in the last 10 days for at least one item
Considerable: obs_number=4058.0000, expected range: 1056.7083 2979.7917
NPP VIIRS AMV IR WIND VECTOR DIFF (M/S) PRESSURE LAYER 300-600 (HPA) : out of range
Slight: stdev_press=73.5859, expected range: 64.3776 70.7285
Radiosondes
LAND TEMP Q (G/KG) 41217 PRESSURE LAYER 600-900 (HPA) : out of range
Severe: rms_fgdepar=3.7086, expected range: 0.0527 2.1036
LAND TEMP WIND VECTOR DIFF (M/S) PRESSURE LAYER 0-300 (HPA) : missing
Severe: obs_number=0.0000, expected range: 0.0000 383.7707
HIGH RESO LAND TEMP Q (G/KG) 47778 PRESSURE LAYER 900-1200 (HPA) : out of range
Severe: rms_fgdepar=2.6560, expected range: 0.0000 1.3132
HIGH RESO SEA TEMP WIND VECTOR DIFF (M/S) PRESSURE LAYER 900-1200 (HPA : out of range
Considerable: rms_fgdepar=9.1541, expected range: 0.1975 5.4364
amdar
WIGOS AMDAR WIND VECTOR DIFF (M/S) Y0A0JHBA PRESSURE LAYER 0-300 (HPA) : out of range
Severe: rms_fgdepar=22.6042, expected range: 0.0000 10.7449
Severe: avg_fgdepar=20.7466, expected range: -2.7778 10.1714
prof
All EUROPEAN WIND PROFILER ALL LAYERS West pacific Tropica : all data missing
Severe: obs_number=0, expected range: 251.86 778.89
All EUROPEAN WIND PROFILER ALL LAYERS South West Pacific : all data missing
Severe: obs_number=0, expected range: 251.86 778.89
EUROPEAN WIND PROFILER VECTOR DIFF (M/S) (2 pressure layers affected) : out of range
4 times in the last 10 days for at least one item
Considerable: obs_number=1081.0000, expected range: 1913.4186 2522.8314
synop
MANUAL LAND SYNOP RH2M (%) 42770 : out of range
21 times in the last 10 days for at least one item
Severely_persistent: avg_pge=0.8209, expected range: -0.5230 0.7500
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 42809 : out of range
1 times in the last 10 days for at least one item
Severe: rms_fgdepar=326.3892, expected range: 0.0000 119.0779
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 42398 : out of range
3 times in the last 10 days for at least one item
Severe: avg_pge=0.8460, expected range: -1.4505 0.7500
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 42807 : out of range
Severe: rms_fgdepar=314.1576, expected range: 0.0000 126.3627
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 41696 : out of range
1 times in the last 10 days for at least one item
Severe: avg_pge=0.8769, expected range: -0.6239 0.7500
Severe: rms_fgdepar=651.2093, expected range: 0.0000 178.4885
Severe: avg_fgdepar=650.3672, expected range: -158.9824 170.2150
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 42176 : out of range
3 times in the last 10 days for at least one item
Severe: avg_pge=0.8201, expected range: -1.5231 0.7500
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 42262 : out of range
Severe: avg_pge=0.8100, expected range: -0.8182 0.7500
Severe: rms_fgdepar=324.5254, expected range: 0.0000 183.6664
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 40911 : out of range
Severe: rms_fgdepar=472.8087, expected range: 0.0000 116.1944
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 40990 : out of range
Severe: rms_fgdepar=561.9565, expected range: 0.0000 81.5789
MANUAL LAND SYNOP RH2M (%) 40938 : out of range
Severe: rms_fgdepar=32.9716, expected range: 0.0000 14.0592
Severe: rms_fgdepar=32.0373, expected range: 0.0000 18.2280
MANUAL LAND SYNOP RH2M (%) 42577 : out of range
1 times in the last 10 days for at least one item
Severe: avg_pge=0.8919, expected range: -0.8405 0.7500
MANUAL LAND SYNOP RH2M (%) 42177 : out of range
2 times in the last 10 days for at least one item
Severe: avg_pge=0.9876, expected range: -1.7701 0.7500
MANUAL LAND SYNOP RH2M (%) 40016 : out of range
1 times in the last 10 days for at least one item
Severe: avg_pge=0.9783, expected range: -1.8772 0.7500
MANUAL LAND SYNOP RH2M (%) 76775 : out of range
2 times in the last 10 days for at least one item
Severe: avg_pge=0.9913, expected range: -1.5606 0.7500
Severe: rms_fgdepar=54.0896, expected range: 0.0000 73.3469
Severe: avg_fgdepar=54.0896, expected range: -16.3689 73.3469
MANUAL LAND SYNOP RH2M (%) 42393 : out of range
Severe: rms_fgdepar=35.6600, expected range: 0.0000 20.2759
MANUAL LAND SYNOP RH2M (%) 61043 : out of range
1 times in the last 10 days for at least one item
Severe: avg_pge=0.9141, expected range: -0.0013 0.0081
Severe: rms_fgdepar=43.8301, expected range: 0.0000 22.1508
MANUAL LAND SYNOP RH2M (%) 67791 : out of range
Severe: avg_pge=0.9995, expected range: -0.0053 0.0145
Severe: rms_fgdepar=53.4969, expected range: 0.0000 39.0941
Severe: avg_fgdepar=-53.4969, expected range: -45.8168 24.5513
MANUAL LAND SYNOP RH2M (%) 64751 : out of range
3 times in the last 10 days for at least one item
Severe: avg_pge=0.9089, expected range: -0.0814 0.1226
MANUAL LAND SYNOP RH2M (%) 64700 : out of range
Severe: avg_pge=0.9906, expected range: -0.0963 0.1348
BUFR LAND SYNOP SURFACE PRESSURE (PA) 48952 : out of range
Severe: avg_pge=0.8128, expected range: -0.5333 0.7500
Severe: rms_fgdepar=348.0535, expected range: 43.5197 166.6621
BUFR LAND SYNOP SURFACE PRESSURE (PA) 22602 : out of range
Severe: avg_pge=0.8684, expected range: -0.2251 0.2555
Severe: rms_fgdepar=596.8624, expected range: 0.0000 97.3506
Severe: avg_fgdepar=-528.4433, expected range: -97.8242 74.3338
BUFR LAND SYNOP SURFACE PRESSURE (PA) 48921 : out of range
Severe: avg_pge=0.8442, expected range: -0.6579 0.7500
Severe: rms_fgdepar=371.6532, expected range: 0.0000 184.3466
BUFR LAND SYNOP RH2M (%) 72636 : out of range
3 times in the last 10 days for at least one item
Severe: avg_pge=0.9177, expected range: -1.1903 0.7500
BUFR LAND SYNOP RH2M (%) 48064 : out of range
Severe: rms_fgdepar=34.9254, expected range: 0.0000 19.9151
MANUAL LAND SYNOP SURFACE PRESSURE (PA) : out of range
Considerable: obs_number=119.0000, expected range: 129.8638 149.1362
Considerable: rms_fgdepar=151.3058, expected range: 42.6551 109.2707
ship
BUFR SHIP SYNOP SURFACE PRESSURE (PA) J8QX8 : out of range
6 times in the last 10 days for at least one item
Severely_persistent: avg_pge=0.8776, expected range: -1.4930 0.7500
Severely_persistent: rms_fgdepar=829.3512, expected range: 0.0000 1160.1491
Severely_persistent: avg_fgdepar=829.3512, expected range: -719.5194 1154.1410
BUFR SHIP SYNOP SURFACE PRESSURE (PA) VRQH5 : out of range
9 times in the last 10 days for at least one item
Severely_persistent: avg_pge=0.8818, expected range: -0.9936 0.7500
Severely_persistent: rms_fgdepar=923.4099, expected range: 0.0000 1304.9121
Severely_persistent: avg_fgdepar=923.3630, expected range: -1396.8477 1083.4952
BUFR SHIP SYNOP SURFACE PRESSURE (PA) C6AX3 : out of range
8 times in the last 10 days for at least one item
Severely_persistent: avg_pge=0.8354, expected range: 0.7101 0.7500
Severely_persistent: rms_fgdepar=555.0933, expected range: 259.0676 656.1634
Severely_persistent: avg_fgdepar=-546.5672, expected range: -644.4482 -265.9464
BUFR SHIP SYNOP SURFACE PRESSURE (PA) VRDJ3 : out of range
24 times in the last 10 days for at least one item
Severely_persistent: avg_pge=0.8231, expected range: 0.7379 0.7500
BUFR SHIP SYNOP SURFACE PRESSURE (PA) WGEB : out of range
Severe: avg_pge=0.9029, expected range: -0.8396 0.7500
Severe: rms_fgdepar=913.1929, expected range: 0.0000 371.7414
Severe: avg_fgdepar=-913.1929, expected range: -137.4896 371.7414
BUFR SHIP SYNOP SURFACE PRESSURE (PA) TFEA : out of range
Severe: rms_fgdepar=558.0253, expected range: -349.8334 518.5502
BUFR SHIP SYNOP SURFACE PRESSURE (PA) WDE9586 : out of range
Severe: avg_fgdepar=665.6650, expected range: -444.2323 470.1982
Severe: rms_fgdepar=728.0356, expected range: 0.0000 569.7115
BUFR SHIP SYNOP SURFACE PRESSURE (PA) VRQK3 : out of range
2 times in the last 10 days for at least one item
Severe: rms_fgdepar=576.6075, expected range: 351.2903 828.9327
Severe: avg_fgdepar=-568.8631, expected range: -827.7871 -362.6625
BUFR SHIP SYNOP SURFACE PRESSURE (PA) 9V9450 : out of range
Severe: rms_fgdepar=527.7589, expected range: 173.3057 591.9670
Severe: avg_fgdepar=518.1488, expected range: 174.6229 588.6183
BUFR SHIP SYNOP WIND VECTOR DIFF (M/S) TBWUK48 : out of range
2 times in the last 10 days for at least one item
Severe: avg_pge=0.8798, expected range: -0.5739 0.7500
BUFR SHIP SYNOP WIND VECTOR DIFF (M/S) C6AX3 : out of range
1 times in the last 10 days for at least one item
Severe: avg_pge=0.8766, expected range: -0.9467 0.7500
buoy
DRIFTING BUOYS SURFACE PRESSURE (PA) 1501629 : out of range
7 times in the last 10 days for at least one item
Severely_persistent: avg_pge=0.8195, expected range: -1.2565 0.7500
DRIFTING BUOYS SURFACE PRESSURE (PA) 5601634 : out of range
1 times in the last 10 days for at least one item
Severe: avg_pge=0.8504, expected range: -0.0379 0.0511
Severe: rms_fgdepar=426.6073, expected range: 0.0000 84.9577
metar
METAR SURFACE PRESSURE (PA) OPPG : out of range
Severe: rms_fgdepar=401.1165, expected range: 0.0000 225.3986
METAR SURFACE PRESSURE (PA) FZIC : out of range
Severe: avg_pge=0.8894, expected range: -0.0768 0.0901
Severe: rms_fgdepar=924.9897, expected range: 0.0000 208.1008
Severe: avg_fgdepar=924.9897, expected range: -126.8849 187.2891
METAR SURFACE PRESSURE (PA) FZNA : out of range
Severe: avg_pge=0.8416, expected range: -0.4048 0.4876
Severe: rms_fgdepar=787.3078, expected range: 0.0000 534.8443
Severe: avg_fgdepar=-787.3078, expected range: -291.6725 376.2868
snow
AUTOMATIC LAND SYNOP SNOW (CM) 01132 : out of range
14 times in the last 10 days for at least one item
Severely_persistent: rms_fgdepar=71.7618, expected range: 68.2647 77.2099
Severely_persistent: avg_fgdepar=71.7618, expected range: 68.2647 77.2099
MANUAL LAND SYNOP SNOW (CM) 11138 : out of range
12 times in the last 10 days for at least one item
Severely_persistent: rms_fgdepar=71.4951, expected range: 67.6442 98.6515
Severely_persistent: avg_fgdepar=71.4951, expected range: 67.6442 98.6515
BUFR LAND SYNOP SNOW (CM) 11138 : out of range
14 times in the last 10 days for at least one item
Severely_persistent: rms_fgdepar=72.2209, expected range: 69.3391 99.1696
Severely_persistent: avg_fgdepar=72.2209, expected range: 69.3391 99.1696
BUFR LAND SYNOP SNOW (CM) 01008 : out of range
38 times in the last 10 days for at least one item
Severely_persistent: rms_fgdepar=69.1029, expected range: 68.5719 130.3710
Severely_persistent: avg_fgdepar=-69.1021, expected range: -130.3715 -68.5705
BUFR LAND SYNOP SNOW (CM) 01132 : out of range
36 times in the last 10 days for at least one item
Severely_persistent: rms_fgdepar=71.8607, expected range: 70.8516 75.9142
Severely_persistent: avg_fgdepar=71.8604, expected range: 70.8531 75.9099
AUTOMATIC LAND SYNOP SNOW (CM) : out of range
1 times in the last 10 days for at least one item
Severe: rms_fgdepar=87.4772, expected range: 0.0000 12.4947
Severe: rms_fgdepar=88.1374, expected range: -9.0117 15.5607
MANUAL LAND SYNOP SNOW (CM) 20107 : out of range
Severe: rms_fgdepar=16.4191, expected range: 4.9633 11.1994
MANUAL LAND SYNOP SNOW (CM) 22602 : out of range
Severe: rms_fgdepar=29.0000, expected range: -0.0029 0.0041
ROCOB TEMP SNOW (CM) 64301455 : out of range
Severe: rms_fgdepar=31.7467, expected range: 0.0936 13.7244
BUFR LAND SYNOP SNOW (CM) 01435 : out of range
4 times in the last 10 days for at least one item
Severe: rms_fgdepar=108.2597, expected range: 95.2084 117.8776
Severe: avg_fgdepar=108.2592, expected range: 95.1995 117.8783
BUFR LAND SYNOP SNOW (CM) 01361 : out of range
Severe: rms_fgdepar=107.6318, expected range: 96.1156 109.3881
Severe: avg_fgdepar=107.6298, expected range: 96.1110 109.3885
t2m
BUFR LAND SYNOP 2M TEMPERATURE (K) 27415 : out of range
6 times in the last 10 days for at least one item
Severely_persistent: rms_fgdepar=13.3056, expected range: 0.0000 15.3082
Severely_persistent: avg_fgdepar=-12.8171, expected range: -14.8020 0.1626
AUTOMATIC LAND SYNOP 2M TEMPERATURE (K) 71319 : out of range
2 times in the last 10 days for at least one item
Severe: rms_fgdepar=10.2868, expected range: 0.1671 4.6585
METAR 2M TEMPERATURE (K) KRBD : out of range
Severe: rms_fgdepar=10.5120, expected range: 0.2452 4.7903
rh2m
METAR RH2M (%) DRRT : out of range
Severe: rms_fgdepar=33.5833, expected range: 0.0000 18.5419