Operational Data Checking

An automatic data checking system has been implemented at ECMWF to trigger warnings in case of anomalies affecting the quality or the availability of the data assimilated by the model. Selected statistical parameters (number of observations, bias correction, and mean bias-corrected background, analysis departures and probability of gross error) are checked against an expected range. An appropriate alert message (including a time series plot) is generated if statistics are outside the specified ranges. A severity level (slight, considerable, severe) is assigned to each message depending on how far statistics are from the expected values. Two kinds of ranges are used by the automatic checking: Soft and Hard limits. Soft limits are updated automatically using statistics from the last twenty days (extremes are excluded during this process). Hard limits are adjusted manually when required.

Currently, the automatic checking is limited to data passing through the minimisation process (including VarBC passive data). It's being applied twice a day to the delayed cut-off 4D-VAR cycles (00 and 12) and four times a day to the early delivery assimilation system (00,06,12 and 18).

The alarm system results are based on feedback information from the ECMWF data assimilation and consequently reflect the ECMWF data usage. Although most warnings are related to data problems, they may on occasion also be related to model issues. Alerts are provided for information on an as-is basis and may be withdrawn or changed without notice. ECMWF accepts no responsibility for actions taken on the basis of these results.

Delayed cut-off 4DVAR at 2020-08-06 12:00

cris
NPP CRIS RADIANCES CHANNEL 109 : out of range
7 times in the last 10 days for at least one item
Severely_persistent: avg_biascorr=0.1606, expected range: 0.1401 0.1515
NPP CRIS RADIANCES CHANNEL 106 : out of range
8 times in the last 10 days for at least one item
Severely_persistent: avg_biascorr=0.2381, expected range: 0.2019 0.2195
NPP CRIS RADIANCES CHANNEL 107 : out of range
14 times in the last 10 days for at least one item
Severely_persistent: avg_biascorr=0.1865, expected range: 0.1453 0.1561
NPP CRIS RADIANCES CHANNEL 108 : out of range
14 times in the last 10 days for at least one item
Severely_persistent: avg_biascorr=0.1458, expected range: 0.1054 0.1190
NPP CRIS RADIANCES CHANNEL 119 : out of range
10 times in the last 10 days for at least one item
Severely_persistent: avg_biascorr=0.1902, expected range: 0.1505 0.1678
NPP CRIS RADIANCES CHANNEL 137 : out of range
6 times in the last 10 days for at least one item
Severely_persistent: avg_biascorr=0.0422, expected range: 0.0012 0.0223
NPP CRIS RADIANCES (11 channels affected) : out of range
Considerable: avg_biascorr=0.0793, expected range: 0.0660 0.0744
NOAA20 CRIS RADIANCES (107 channels affected) : out of range
Considerable: obs_number=9532.0000, expected range: 19449.9686 26291.0213
airs
AQUA AIRS RADIANCES (51 channels affected) : out of range
Considerable: obs_number=16192.0000, expected range: 17866.7066 19845.9992
gpsro
All COSMIC2-E5 GPSRO RISING ALL HEIGHTS Global : all data missing
Severely_persistent: obs_number=0, expected range: 268.24 3255.42
Radiosondes
LAND TEMP TEMPERATURE (K) 61660 PRESSURE LAYER 0-300 (HPA) : out of range
32 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=-15.5475, expected range: -18.8946 -11.4216
Severely_persistent: rms_fgdepar=22.5763, expected range: 18.5054 23.6981
HIGH RESO SEA TEMP TEMPERATURE (K) (3 pressure layers affected) : missing
11 times in the last 10 days for at least one item
Severely_persistent: obs_number=0.0000, expected range: 26.8608 674.4370
LAND TEMP TEMPERATURE (K) 24688 PRESSURE LAYER 900-1200 (HPA) : out of range
3 times in the last 10 days for at least one item
Severe: rms_fgdepar=7.9161, expected range: 0.0000 4.8853
Severe: avg_pge=0.9022, expected range: -0.9339 0.7500
LAND TEMP TEMPERATURE (K) PRESSURE LAYER 0-300 (HPA) : out of range
Severe: rms_fgdepar=4.2940, expected range: 0.5556 2.6882
LAND TEMP WIND VECTOR DIFF (M/S) 71603 PRESSURE LAYER 300-600 (HPA) : out of range
Severe: rms_fgdepar=26.6093, expected range: 0.2921 4.8180
HIGH RESO LAND TEMP TEMPERATURE (K) 53915 PRESSURE LAYER 0-300 (HPA) : out of range
Severe: rms_fgdepar=6.5103, expected range: 0.0000 2.5681
HIGH RESO LAND TEMP TEMPERATURE (K) (2 pressure layers affected) : missing
1 times in the last 10 days for at least one item
Severe: obs_number=0.0000, expected range: 156.0685 216.7315
Severe: obs_number=0.0000, expected range: 3.2606 399.1394
HIGH RESO LAND TEMP Q (G/KG) (2 pressure layers affected) : out of range
Considerable: obs_number=93.0000, expected range: 176.9257 196.0743
amdar
AMDAR TEMPERATURE (K) CNFKQP PRESSURE LAYER 600-900 (HPA) : out of range
13 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=-11.0254, expected range: -23.1514 8.4446
Severely_persistent: rms_fgdepar=12.0318, expected range: 0.0000 24.5772
WIGOS AMDAR WIND VECTOR DIFF (M/S) DIJX1DJA PRESSURE LAYER 0-300 (HPA) : out of range
8 times in the last 10 days for at least one item
Severely_persistent: rms_fgdepar=20.0842, expected range: -41.0597 61.2856
Severely_persistent: avg_pge=0.8485, expected range: -1.5736 0.7500
AMDAR WIND VECTOR DIFF (M/S) BR970734 PRESSURE LAYER 0-300 (HPA) : out of range
Severe: rms_fgdepar=21.0062, expected range: 3.8950 38.4607
AMDAR WIND VECTOR DIFF (M/S) BR998497 PRESSURE LAYER 0-300 (HPA) : out of range
Severe: avg_fgdepar=20.6781, expected range: 1.0578 27.2199
Severe: rms_fgdepar=22.0191, expected range: 0.0000 30.8225
Severe: avg_pge=0.8087, expected range: 0.0682 0.7500
WIGOS AMDAR WIND VECTOR DIFF (M/S) VJ3H1DBA (2 pressure layers affecte : out of range
3 times in the last 10 days for at least one item
Severe: rms_fgdepar=19.2049, expected range: 0.2853 6.2928
WIGOS AMDAR WIND VECTOR DIFF (M/S) BTSX1DZA PRESSURE LAYER 0-300 (HPA) : out of range
3 times in the last 10 days for at least one item
Severe: avg_fgdepar=21.5828, expected range: -26.8342 42.9440
Severe: rms_fgdepar=21.8207, expected range: 0.0000 45.1933
pilot
LAND PILOT WIND VECTOR DIFF (M/S) 97260 PRESSURE LAYER 900-1200 (HPA) : out of range
9 times in the last 10 days for at least one item
Severely_persistent: avg_pge=0.8464, expected range: -0.5049 0.7500
synop
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 43012 : out of range
14 times in the last 10 days for at least one item
Severely_persistent: avg_pge=0.8122, expected range: 0.2161 0.7500
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 43029 : out of range
15 times in the last 10 days for at least one item
Severely_persistent: avg_pge=0.8426, expected range: -0.5979 0.7500
MANUAL LAND SYNOP RH2M (%) 76040 : out of range
6 times in the last 10 days for at least one item
Severely_persistent: avg_pge=0.9928, expected range: -1.0630 0.7500
BUFR LAND SYNOP SURFACE PRESSURE (PA) 91682 : out of range
11 times in the last 10 days for at least one item
Severely_persistent: avg_pge=0.8345, expected range: -0.2538 0.7500
BUFR LAND SYNOP SURFACE PRESSURE (PA) 55664 : out of range
25 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=-596.8036, expected range: -1807.6923 994.3341
Severely_persistent: rms_fgdepar=598.6190, expected range: 0.0000 1916.3000
Severely_persistent: avg_pge=0.8767, expected range: -1.5668 0.7500
BUFR LAND SYNOP SURFACE PRESSURE (PA) 72494 : out of range
11 times in the last 10 days for at least one item
Severely_persistent: avg_pge=0.8061, expected range: 0.7765 0.7500
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 43181 : out of range
Severe: avg_fgdepar=886.3573, expected range: -227.6520 90.2664
Severe: rms_fgdepar=886.7494, expected range: 0.0000 225.3883
Severe: avg_pge=0.9400, expected range: -0.3350 0.4724
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 13591 : out of range
3 times in the last 10 days for at least one item
Severe: rms_fgdepar=386.6908, expected range: 0.0000 76.1693
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 87163 : out of range
Severe: rms_fgdepar=322.6278, expected range: 0.0000 90.5074
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 13493 : out of range
1 times in the last 10 days for at least one item
Severe: rms_fgdepar=328.3581, expected range: 0.0000 113.7034
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 82915 : out of range
Severe: avg_pge=0.8003, expected range: -0.6865 0.7500
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 91425 : out of range
2 times in the last 10 days for at least one item
Severe: rms_fgdepar=375.7935, expected range: 0.0000 157.0351
Severe: avg_pge=0.8544, expected range: -0.5096 0.6002
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 42255 : out of range
3 times in the last 10 days for at least one item
Severe: avg_fgdepar=502.3784, expected range: -717.2051 665.5354
Severe: rms_fgdepar=502.3784, expected range: 0.0000 526.8130
Severe: avg_pge=0.9032, expected range: -1.4069 0.7500
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 83916 : out of range
3 times in the last 10 days for at least one item
Severe: avg_pge=0.8183, expected range: -1.7229 0.7500
MANUAL LAND SYNOP SURFACE PRESSURE (PA) 42577 : out of range
3 times in the last 10 days for at least one item
Severe: rms_fgdepar=467.0191, expected range: 0.0000 368.9136
Severe: avg_pge=0.8546, expected range: -1.3225 0.7500
MANUAL LAND SYNOP RH2M (%) 35321 : out of range
Severe: rms_fgdepar=39.2524, expected range: 0.0000 16.3741
MANUAL LAND SYNOP RH2M (%) 83363 : out of range
2 times in the last 10 days for at least one item
Severe: avg_pge=0.9311, expected range: -0.8626 0.7500
BUFR LAND SYNOP SURFACE PRESSURE (PA) 43181 : out of range
Severe: avg_fgdepar=886.3573, expected range: -227.6520 90.2664
Severe: rms_fgdepar=886.7494, expected range: 0.0000 225.3883
BUFR LAND SYNOP SURFACE PRESSURE (PA) 40582 : out of range
Severe: avg_fgdepar=-515.7724, expected range: -47.3252 54.9385
Severe: rms_fgdepar=516.0483, expected range: 0.0000 65.9976
Severe: avg_pge=0.8983, expected range: -0.0226 0.0256
BUFR LAND SYNOP SURFACE PRESSURE (PA) 13493 : out of range
Severe: rms_fgdepar=328.3581, expected range: 0.0000 103.1570
BUFR LAND SYNOP RH2M (%) 23174 : out of range
Severe: avg_pge=0.9832, expected range: -1.0741 0.7500
BUFR LAND SYNOP RH2M (%) 16618 : out of range
Severe: rms_fgdepar=31.4800, expected range: 0.0000 16.7059
BUFR LAND SYNOP RH2M (%) 24738 : out of range
1 times in the last 10 days for at least one item
Severe: avg_pge=0.9356, expected range: -0.0480 0.0692
BUFR LAND SYNOP RH2M (%) 16648 : out of range
Severe: rms_fgdepar=41.8404, expected range: 0.0000 16.4812
ship
BUFR SHIP SYNOP SURFACE PRESSURE (PA) 3FVW3 : out of range
7 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=-561.3655, expected range: -909.9554 -163.9169
Severely_persistent: rms_fgdepar=561.3655, expected range: 182.5608 897.5984
BUFR SHIP SYNOP WIND VECTOR DIFF (M/S) ASES01 : out of range
5 times in the last 10 days for at least one item
Severely_persistent: avg_pge=0.8657, expected range: -0.3550 0.7500
BUFR SHIP SYNOP SURFACE PRESSURE (PA) AMOUK49 : out of range
3 times in the last 10 days for at least one item
Severe: avg_pge=0.8131, expected range: -0.7744 0.7500
BUFR SHIP SYNOP WIND VECTOR DIFF (M/S) VRED4 : out of range
Severe: avg_fgdepar=22.3845, expected range: -3.6898 9.9852
Severe: rms_fgdepar=22.3845, expected range: 0.0000 10.3476
Severe: avg_pge=0.9187, expected range: -0.9149 0.7500
BUFR SHIP SYNOP WIND VECTOR DIFF (M/S) VRPD9 : out of range
Severe: avg_fgdepar=20.8297, expected range: -13.2207 20.8391
Severe: rms_fgdepar=20.8297, expected range: 0.0000 24.7738
Severe: avg_pge=0.9075, expected range: -1.2733 0.7500
buoy
DRIFTING BUOYS SURFACE PRESSURE (PA) 2501647 : out of range
9 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=505.8525, expected range: -1288.0659 1081.4455
Severely_persistent: rms_fgdepar=537.1070, expected range: 0.0000 1151.3105
Severely_persistent: avg_pge=0.8514, expected range: -1.4263 0.7500
DRIFTING BUOYS SURFACE PRESSURE (PA) 1501748 : out of range
16 times in the last 10 days for at least one item
Severely_persistent: rms_fgdepar=511.3196, expected range: 0.0000 974.3928
DRIFTING BUOYS SURFACE PRESSURE (PA) 3301601 : out of range
Severe: rms_fgdepar=371.2704, expected range: 0.0000 134.6642
DRIFTING BUOYS SURFACE PRESSURE (PA) 5601521 : out of range
Severe: rms_fgdepar=346.3300, expected range: 0.0000 100.1992
metar
METAR SURFACE PRESSURE (PA) HTSO : out of range
Severe: rms_fgdepar=310.3517, expected range: 0.0000 118.6868
t2m
AUTOMATIC LAND SYNOP 2M TEMPERATURE (K) 89504 : out of range
15 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=13.4063, expected range: 4.1047 14.6870
Severely_persistent: rms_fgdepar=13.4465, expected range: 4.8149 14.4934
Severely_persistent: avg_est_pge=0.8414, expected range: 0.5631 0.9449
AUTOMATIC LAND SYNOP 2M TEMPERATURE (K) 89625 : out of range
13 times in the last 10 days for at least one item
Severely_persistent: avg_est_pge=0.8144, expected range: 0.0812 1.2610
AUTOMATIC LAND SYNOP 2M TEMPERATURE (K) 89577 : out of range
9 times in the last 10 days for at least one item
Severely_persistent: avg_est_pge=0.8016, expected range: 0.5704 0.9310
MANUAL LAND SYNOP 2M TEMPERATURE (K) 89504 : out of range
17 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=13.5763, expected range: 3.7252 15.3383
Severely_persistent: rms_fgdepar=13.6029, expected range: 4.2554 15.1719
Severely_persistent: avg_est_pge=0.8436, expected range: 0.5649 0.9547
LAND TEMP 2M TEMPERATURE (K) 72376 : out of range
7 times in the last 10 days for at least one item
Severely_persistent: avg_est_pge=0.8254, expected range: -0.4069 1.3879
LAND TEMP 2M TEMPERATURE (K) 56029 : out of range
5 times in the last 10 days for at least one item
Severely_persistent: avg_est_pge=0.8510, expected range: -0.0963 1.3072
BUFR LAND SYNOP 2M TEMPERATURE (K) 89504 : out of range
15 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=13.3801, expected range: 4.1322 14.6205
Severely_persistent: rms_fgdepar=13.4181, expected range: 4.8862 14.3828
Severely_persistent: avg_est_pge=0.8412, expected range: 0.5665 0.9422
BUFR LAND SYNOP 2M TEMPERATURE (K) 27415 : out of range
34 times in the last 10 days for at least one item
Severely_persistent: avg_fgdepar=-17.9837, expected range: -20.6925 -12.0789
Severely_persistent: rms_fgdepar=17.9837, expected range: 12.4787 21.0081
BUFR LAND SYNOP 2M TEMPERATURE (K) 89577 : out of range
11 times in the last 10 days for at least one item
Severely_persistent: avg_est_pge=0.8018, expected range: 0.5633 0.9416
AUTOMATIC LAND SYNOP 2M TEMPERATURE (K) 89734 : out of range
3 times in the last 10 days for at least one item
Severe: avg_est_pge=0.8271, expected range: 0.5099 0.9553
AUTOMATIC LAND SYNOP 2M TEMPERATURE (K) 51777 : out of range
Severe: avg_fgdepar=-12.3381, expected range: -19.2681 15.0879
Severe: rms_fgdepar=12.3381, expected range: 0.0000 13.6844
Severe: avg_est_pge=0.8048, expected range: -0.0508 1.1318
METAR 2M TEMPERATURE (K) KLUF : out of range
Severe: avg_fgdepar=-17.9572, expected range: -6.7753 6.2818
Severe: rms_fgdepar=17.9572, expected range: 0.0000 5.2687
rh2m
MANUAL LAND SYNOP RH2M (%) 41780 : out of range
Severe: rms_fgdepar=32.6359, expected range: 0.0000 15.0471
BUFR LAND SYNOP RH2M (%) 80398 : out of range
Severe: rms_fgdepar=31.4963, expected range: 0.0000 15.9573