Update README.md

README.md

@@ -27,10 +27,6 @@ are available to the public under ECMWF’s open data policy. (https://www.ecmwf
 AIFS is based on a graph neural network (GNN) encoder and decoder, and a sliding window transformer processor, 
 and is trained on ECMWF’s ERA5 re-analysis and ECMWF’s operational numerical weather prediction (NWP) analyses. 
 
-<div style="display: flex; justify-content: center;">
-  <img src="aifs_diagram.png" alt="High-level AIFS diagram" style="width: 50%;"/>
-</div>
-
 <div style="display: flex;">
   <img src="encoder_graph.jpeg" alt="Encoder graph" style="width: 50%;"/>
   <img src="decoder_graph.jpeg" alt="Decoder graph" style="width: 50%;"/>
@@ -110,6 +106,10 @@ aifs_forecast.to_xarray()
 AIFS is trained to produce 6-hour forecasts. It receives as input a representation of the atmospheric states
 at \\(t_{−6h}\\), \\(t_{0}\\), and then forecasts the state at time \\(t_{+6h}\\). 
 
+<div style="display: flex; justify-content: center;">
+  <img src="aifs_diagram.png" alt="High-level AIFS diagram" style="width: 80%;"/>
+</div>
+
 The full list of input and output fields is shown below:
 
 | Field                                                                                                                                                       | Level type                                                                   | Input/Output |
@@ -130,7 +130,7 @@ the forcing variables, like orography, are min-max normalised.
 of 260,000 steps. The LR is increased from 0 to \\(10^{-4}\\) during the first 1000 steps, then it is annealed to a minimum
 of \\(3 × 10^{-7}\\).
 - **Fine-tuning I**: The pre-training is then followed by rollout on ERA5 for the years 1979 to 2018, this time with a LR
-of \\(6 × 10^{-7}\\). As in [Lam et al. [2023]](https://doi.org/10.48550/arXiv.2212.12794) we increase the
+of \\(6 × 10^{-7}\\). As in [Lam et al. [2023]](doi: 10.21957/slk503fs2i) we increase the
 rollout every 1000 training steps up to a maximum of 72 h (12 auto-regressive steps).
 - **Fine-tuning II**: Finally, to further improve forecast performance, we fine-tune the model on operational real-time IFS NWP
 analyses. This is done via another round of rollout training, this time using IFS operational analysis data
@@ -162,39 +162,29 @@ state.
 
 <!-- This section describes the evaluation protocols and provides the results. -->
 
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-{{ testing_data | default("[More Information Needed]", true)}}
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+AIFS is evaluated against ECMWF IFS (Integrated Forecast System) for 2022. The results of such evaluation are summarized in 
+the scorecard below that compares different forecast skill measures across a range of
+variables. For verification, each system is compared against the operational ECMWF analysis from which the forecasts
+are initialised. In addition, the forecasts are compared against radiosonde observations of geopotential, temperature
+and windspeed, and SYNOP observations of 2 m temperature, 10 m wind and 24 h total precipitation. The definition
+of the metrics, such as ACC (ccaf), RMSE (rmsef) and forecast activity (standard deviation of forecast anomaly,
+sdaf) can be found in e.g Ben Bouallegue et al. ` [2024].
 
-{{ testing_factors | default("[More Information Needed]", true)}}
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-{{ testing_metrics | default("[More Information Needed]", true)}}
-
-### Results
-
-{{ results | default("[More Information Needed]", true)}}
-
-#### Summary
-
-{{ results_summary | default("", true) }}
-
-## Model Examination [optional]
+<div style="display: flex; justify-content: center;">
+  <img src="aifs_v021_scorecard.png" alt="Scorecard comparing forecast scores of AIFS versus IFS (2022)" style="width: 80%;"/>
+</div>
 
-<!-- Relevant interpretability work for the model goes here -->
 
-{{ model_examination | default("[More Information Needed]", true)}}
+. Forecasts are initialised on 00 and 12
+UTC. Shown are relative score changes as function of lead time (day 1 to 10) for northern extra-tropics (n.hem),
+southern extra-tropics (s.hem), tropics and Europe. Blue colours mark score improvements and red colours score
+degradations. Purple colours indicate an increased in standard deviation of forecast anomaly, while green colours
+indicate a reduction. Framed rectangles indicate 95% significance level. Variables are geopotential (z), temperature
+(t), wind speed (ff), mean sea level pressure (msl), 2 m temperature (2t), 10 m wind speed (10ff) and 24 hr total
+precipitation (tp). Numbers behind variable abbreviations indicate variables on pressure levels (e.g., 500 hPa), and
+suffix indicates verification against IFS NWP analyses (an) or radiosonde and SYNOP observations (ob). Scores
+shown are anomaly correlation (ccaf), SEEPS (seeps, for precipitation), RMSE (rmsef) and standard deviation of
+forecast anomaly (sdaf, see text for more explanation).
 
 ## Technical Specifications