Benchmarking
The Automated Model Benchmarking (AMBER) framework
We evaluate the ability of CLASSIC to reproduce land surface processes by comparing model outputs against quasi-observational data sets derived from remote sensing products, eddy covariance flux tower measurements (FLUXNET2015), and streamflow measurements. To summarize model performance across different statistical metrics we employ a skill score system that was originally developed by the International Land Model Benchmarking (ILAMB) framework. To tailor the approach to our needs, we have implemented ILAMB’s skill score system in an R-package called Automated Model Benchmarking (AMBER) framework.
AMBER evaluates the performance of a model through scores that range from zero to one, where increasing values imply better performance. These scores are computed for each variable in five steps: (1) computation of a statistical metric, (2) nondimensionalization, (3) conversion to unit interval, (4) spatial integration, and (5) averaging scores computed from different statistical metrics. The latter includes the bias, root-mean-square error, phase shift, interannual variability, and spatial distribution.
AMBER Resources
- AMBER code repository and manual (available on CRAN)
- AMBER outputs for all CLASSIC releases (archived on Zenodo)
Site-level benchmarking with eddy covariance flux tower sites
Meteorological and initial conditions files for select EC flux tower sites have been processed for model benchmarking and use by CLASSIC users. Example outputs from the most recent release of CLASSIC for these sites are also provided.
Resources
Sites
The following sites are included in the site-level benchmarking package. Most sites are from the FLUXNET2015 data release. If you plan on using these sites in a paper, please follow the FLUXNET2015 data policy.
Locations of the FLUXNET sites grouped by IGBF biome
| Site ID | Site Name | Latitude | Longitude | IGBF biome | DOI |
|---|---|---|---|---|---|
| AU-Tum | Tumbarumba | -35.6566 | 148.1517 | EBF | 10.18140/FLX/1440126 |
| BR-Sa1 | Santarem-Km67-Primary Forest | -2.8567 | -54.9589 | EBF | 10.18140/FLX/1440032 |
| CA-Qfo | Quebec - E. Boreal, Mature Black Spruce | 49.6925 | -74.3421 | ENF | 10.18140/FLX/1440045 |
| CA-TPD | Ontario - Turkey Point Mature Deciduous | 42.6353 | -80.5577 | DBF | 10.18140/FLX/1440112 |
| CG-Tch | Tchizalamou | -4.2892 | 11.6564 | SAV | 10.18140/FLX/1440142 |
| CN-Dan | Dangxiong | 30.4978 | 91.0664 | GRA | 10.18140/FLX/1440138 |
| CZ-BK1 | Bily Kriz forest | 49.5021 | 18.5369 | ENF | 10.18140/FLX/1440143 |
| DE-Kli | Klingenberg | 50.8931 | 13.5224 | CRO | 10.18140/FLX/1440149 |
| DE-Tha | Tharandt | 50.9624 | 13.5652 | ENF | 10.18140/FLX/1440152 |
| DK-Sor | Soroe | 55.4859 | 11.6446 | DBF | 10.18140/FLX/1440155 |
| FI-Hyy | Hyytiala | 61.8474 | 24.2948 | ENF | 10.18140/FLX/1440158 |
| FR-Fon | Fontainebleau-Barbeau | 48.4764 | 2.7801 | DBF | 10.18140/FLX/1440161 |
| FR-Pue | Puechabon | 43.7413 | 3.5957 | EBF | 10.18140/FLX/1440164 |
| ES-LgS | Laguna Seca | 37.0979 | -2.9658 | OSH | 10.18140/FLX/1440225 |
| GF-Guy | Guyaflux (French Guiana) | 5.2788 | -52.9249 | EBF | 10.18140/FLX/1440165 |
| GH-Ank | Ankasa | 5.2685 | -2.6942 | EBF | 10.18140/FLX/1440229 |
| IT-Lav | Lavarone | 45.9562 | 11.2813 | ENF | 10.18140/FLX/1440169 |
| IT-SRo | San Rossore | 43.7279 | 10.2844 | ENF | 10.18140/FLX/1440176 |
| IT-Tor | Torgnon | 45.8444 | 7.5781 | GRA | 10.18140/FLX/1440237 |
| MY-PSO | Pasoh Forest Reserve | 2.9730 | 102.3062 | EBF | 10.18140/FLX/1440240 |
| NL-Loo | Loobos | 52.1666 | 5.7436 | ENF | 10.18140/FLX/1440178 |
| PA-SPs | Sardinilla-Pasture | 9.3138 | -79.6314 | GRA | 10.18140/FLX/1440179 |
| RU-Che | Cherski | 68.6130 | 161.3414 | WET | 10.18140/FLX/1440181 |
| RU-Fyo | Fyodorovskoye | 56.4615 | 32.9221 | ENF | 10.18140/FLX/1440183 |
| RU-Ha1 | Hakasia steppe | 54.7252 | 90.0022 | GRA | 10.18140/FLX/1440184 |
| RU-SkP | Yakutsk Spasskaya Pad larch | 62.2550 | 129.1680 | DNF | 10.18140/FLX/1440243 |
| SD-Dem | Demokeya | 13.2829 | 30.4783 | SAV | 10.18140/FLX/1440186 |
| US-WCr | Willow Creek | 45.8059 | -90.0799 | DBF | 10.18140/FLX/1440095 |
| US-Wkg | Walnut Gulch Kendall Grasslands | 31.7365 | -109.9419 | GRA | 10.18140/FLX/1440096 |
| ZA-Kru | Skukuza | -25.0197 | 31.4969 | SAV | 10.18140/FLX/1440188 |
| ZM-Mon | Mongu | -15.4378 | 23.2528 | DBF | 10.18140/FLX/1440189 |