DLR-Logo -> http://www.dlr.de
DLR Portal Home | Impressum | Kontakt | English
Schriftgröße: [-] Text [+]

Comment on "Quantitative performance metrics for stratospheric-resolving chemistry-climate models" by Waugh and Eyring (2008)

Grewe, Volker und Sausen, Robert (2009) Comment on "Quantitative performance metrics for stratospheric-resolving chemistry-climate models" by Waugh and Eyring (2008). Atmospheric Chemistry and Physics, 9, Seiten 9101-9110. Copernicus. ISSN 1680-7316

[img] PDF - Nur angemeldete Benutzer - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader

Offizielle URL: http://www.atmos-chem-phys.net/9/9101/2009/


This comment focuses on the statistical limitations of a model grading, as applied by D. Waugh and V. Eyring (2008) (WE08). The grade g is calculated for a specific diagnostic, which basically relates the difference of means of model and observational data to the standard deviation in the observational dataset. We performed Monte Carlo simulations, which show that this method has the potential to lead to large 95%-confidence intervals for the grade. Moreover, the difference between two model grades often has to be very large to become statistically significant. Since the confidence intervals were not considered in detail for all diagnostics, the grading in WE08 cannot be interpreted, without further analysis. The results of the statistical tests performed in WE08 agree with our findings. However, most of those tests are based on special cases, which implicitely assume that observations are available without any errors and that the interannual variability of the observational data and the model data are equal. Without these assumptions, the 95%- confidence intervals become even larger. Hence, the case, where we assumed perfect observations (ignored errors), provides a good estimate for an upper boundary of the threshold, below that a grade becomes statistically significant. Examples have shown that the 95%-confidence interval may even span the whole grading interval [0, 1]. Without considering confidence intervals, the grades presented in WE08 do not allow to decide whether a model result significantly deviates from reality. Neither in WE08 nor in our comment it is pointed out, which of the grades presented in WE08 inhibits such kind of significant deviation. However, our analysis of the grading method demonstrates the unacceptably high potential for these grades to be insignificant. This implies that the grades given by WE08 can not be interpreted by the reader. We further show that the inclusion of confidence intervals into the grading approach is necessary, since otherwise even a perfect model may get a low grade.

Titel:Comment on "Quantitative performance metrics for stratospheric-resolving chemistry-climate models" by Waugh and Eyring (2008)
AutorenInstitution oder E-Mail-AdresseAutoren-ORCID
Datum:1 Dezember 2009
Erschienen in:Atmospheric Chemistry and Physics
Referierte Publikation:Ja
In Open Access:Ja
In ISI Web of Science:Ja
Seitenbereich:Seiten 9101-9110
Stichwörter:Climate-Chemistry Model Evaluation Grade
HGF - Forschungsbereich:Verkehr und Weltraum (alt)
HGF - Programm:Weltraum (alt)
HGF - Programmthema:W EO - Erdbeobachtung
DLR - Schwerpunkt:Weltraum
DLR - Forschungsgebiet:W EO - Erdbeobachtung
DLR - Teilgebiet (Projekt, Vorhaben):W - Vorhaben Atmosphären- und Klimaforschung (alt)
Standort: Oberpfaffenhofen
Institute & Einrichtungen:Institut für Physik der Atmosphäre > Dynamik der Atmosphäre
Hinterlegt von: Grewe, Prof. Dr. Volker
Hinterlegt am:02 Dez 2009 17:52
Letzte Änderung:23 Jul 2015 18:50

Nur für Mitarbeiter des Archivs: Kontrollseite des Eintrags

Hilfe & Kontakt
electronic library verwendet EPrints 3.3.12
Copyright © 2008-2013 Deutsches Zentrum für Luft- und Raumfahrt (DLR). Alle Rechte vorbehalten.