The International Verification Methods Workshop was held online in November 2020 and included ses... more The International Verification Methods Workshop was held online in November 2020 and included sessions on physical error characterization using process diagnostics and error tracking techniques; exploitation of data assimilation techniques in verification practices, e.g., to address representativeness issues and observation uncertainty; spatial verification methods and the Model Evaluation Tools, as unified reference verification software; and meta-verification and best practices for scores computation. The workshop reached out to diverse research communities working in the areas of high-impact weather, subseasonal to seasonal prediction, polar prediction, and sea ice and ocean prediction. This article summarizes the major outcomes of the workshop and outlines future strategic directions for verification research.
<jats:p>&amp;lt;p&amp;gt;Numerical Model Prediction (NWP) verification against stat... more <jats:p>&amp;lt;p&amp;gt;Numerical Model Prediction (NWP) verification against station measurements from a surface network is affected by sub-tile representativeness issues. Moreover, the station network is often not representative of the whole verification domain (e.g. usually coastal stations are predominant) and large unpopulated regions (such as oceans, Polar regions, deserts) are under-sampled. Verification against gridded analyses mitigate these issues, since they partially address the sub-tile representativeness, and sample homogeneously the verification domain. Moreover, gridded analyses merge station network measurements to radar and satellite retrieval estimates, in a physical coherent fashion, over the same NWP grid. Verification against own analysis, despite quite convenient, is however hampered by its dependence on the NWP background model, which renders the verification &amp;amp;#8220;incestuous&amp;amp;#8221;, further than being affected by the uncertainties introduced by retrieval algorithms and Data Assimilation (DA) procedures.&amp;lt;/p&amp;gt;&amp;lt;p&amp;gt;In this study we investigate the use of a gridded NWP own analysis for verification, by applying a mask to reduce the background model contribution. The mask weights the verification scores to account for the amounts of observations assimilated and their associated uncertainty, as estimated from DA. We illustrate the approach by using the Canadian Precipitation Analysis (CaPA), which assimilates station measurements, radar and satellite-based (IMERG) observations. The CaPA confidence (weighting) mask is dynamic and changes depending on the daily available (assimilated) observations, and on their corresponding DA error statistics; it is defined as&amp;lt;/p&amp;gt;&amp;lt;p&amp;gt;&amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160;mask = 1 - var(A-O)/var(B-O)&amp;lt;/p&amp;gt;&amp;lt;p&amp;gt;where A=analysis, B=Background, O=observations. Where the analysis is identical to the background model, the weighting mask is zero.&amp;lt;/p&amp;gt;&amp;lt;p&amp;gt;We evaluate the Canadian Regional Deterministic Prediction System (RDPS), which is the NWP system used as background model for CaPA. As expected, the verification results obtained by using the weighting mask lay between the verification results obtained verifying against the analysis over the full domain, and the results obtained verifying against station measurements.&amp;amp;#160;The effects of sub-tile representativeness are quantified by comparing verification results against station measurements to verification results against CaPA for the grid-points co-located with the stations. Finally, the comparison of the verification results against CaPA over the full domain versus the verification results against CaPA for the grid-points co-located with stations, estimates to which extent the station network is representative of the full domain.&amp;lt;/p&amp;gt;&amp;lt;p&amp;gt;The approach aims to propose a simple -yet effective- better practice for verification against own analysis.&amp;lt;/p&amp;gt;</jats:p>
The Year of Polar Prediction (YOPP) is planned for mid-2017 to mid-2019, centred on 2018. Its goa... more The Year of Polar Prediction (YOPP) is planned for mid-2017 to mid-2019, centred on 2018. Its goal is to enable a significant improvement in environmental prediction capabilities for the polar regions and beyond, by coordinating a period of intensive observing, modelling, prediction, verification, user-engagement and education activities. With a focus on time scales from hours to a season, YOPP is a major initiative of the World Meteorological Organization’s World Weather Research Programme (WWRP) and a key component of the Polar Prediction Project (PPP). YOPP is being planned and coordinated by the PPP Steering Group together with representatives from partners and other initiatives, including the World Climate Research Programme’s Polar Climate Predictability Initiative (PCPI). The objectives of YOPP are to: 1. Improve the existing polar observing system (enhanced coverage, higher-quality observations). 2. Gather additional observations through field programmes aimed at improving u...
The continuous measuring of the vertical profile of water vapor in the boundary layer using a com... more The continuous measuring of the vertical profile of water vapor in the boundary layer using a commercially available differential absorption lidar (DIAL) has only recently been made possible. Since September 2018, a new pre-production version of the Vaisala DIAL system has operated at the Iqaluit supersite (63.74°N, 68.51°W), commissioned by Environment and Climate Change Canada (ECCC) as part of the Canadian Arctic Weather Science project. This study presents its evaluation during the extremely dry conditions experienced in the Arctic by comparing it with coincident radiosonde and Raman lidar observations. Comparisons over a one year period were strongly correlated (r > 0.8 at almost all heights) and exhibited an average bias of +0.13 ± 0.01 g/kg (DIAL-sonde) and +0.18 ± 0.02 g/kg (DIAL-Raman). Larger differences exhibiting distinct artifacts were found between 250 and 400 m above ground level (AGL). The DIAL’s observations were also used to conduct a verification case study of ...
Abstract Forecast verification is a key component of a forecasting system; it provides informatio... more Abstract Forecast verification is a key component of a forecasting system; it provides information about forecast qualify to model and forecast developers and various users. This chapter provides an overview of methods relevant to sub-seasonal to seasonal (S2S) forecast verification, starting with the definition of forecast goodness and some fundamental forecast quality attributes. Next, the factors affecting the design of verification studies are presented. The recognition of uncertainties in observational data sets and the need for care in matching forecasts and observations is also discussed. A large part of the chapter is dedicated to a review of the most common deterministic and probabilistic forecast verification measures and a summary of novel spatial verification methods developed during the last two decades. Types of S2S forecasts and current verification practices are presented. The chapter concludes with a summary, challenges, and recommendations for advancing S2S verification research and practice.
Assessing the quality of precipitation forecasts requires observations, but all precipitation obs... more Assessing the quality of precipitation forecasts requires observations, but all precipitation observations have associated uncertainties making it difficult to quantify the true forecast quality. One of the largest uncertainties is due to the wind-induced undercatch of solid precipitation gauge measurements. This study discusses how this impacts the verification of precipitation forecasts for Norway for one global model [the high-resolution version of the ECMWF Integrated Forecasting System (IFS-HRES)], and one high-resolution, limited-area model [Applications of Research to Operations at Mesoscale (MEPS)]. First, the forecasts are compared with high-quality reference measurements (less undercatch) and with more simple measurement equipment commonly available (substantial undercatch) at the Haukeliseter observation site. Then the verification is extended to include all Norwegian observation sites: 1) stratified by wind speed, since calm (windy) conditions experience less (more) unde...
Verification of high-impact weather is needed by the Meteorological Centres, but how to perform i... more Verification of high-impact weather is needed by the Meteorological Centres, but how to perform it still presents many open questions, starting from which data are suitable as reference. This paper reviews new observations which can be considered for the verification of high-impact weather, and provides advice for their usage in objective verification. Two highimpact weather phenomena are considered: Thunderstorm and fog. First, a framework for the verification of high-impact weather is proposed, including the definition of forecast and observations in this context and creation of a verification set. Then, new observations showing a potential for the detection and quantification of high-impact weather are reviewed, including remote sensing datasets, products developed for nowcasting, datasets derived from telecommunication systems, data collected from citizens, reports of impacts and claim/damage reports from insurance companies. The observation characteristics which are relevant for their usage in forecast verification are also discussed. Examples of forecast evaluation and verification are then presented, highlighting the methods which can be adopted to address the issues posed by the usage of these non-conventional observations and objectively quantify the skill of a high-impact weather forecast. 1 Introduction Verification of forecasts and warnings issued for high-impact weather is increasingly needed by operational centres. The model and nowcast products used in operations to support the forecasting and warning of high-impact weather such as thunderstorm cells also need to be verified. The World Weather Research Programme (WWRP) of the World Meteorological Organisation
Bulletin of the American Meteorological Society, 2018
Recent advancements in numerical weather prediction (NWP) and the enhancement of model resolution... more Recent advancements in numerical weather prediction (NWP) and the enhancement of model resolution have created the need for more robust and informative verification methods. In response to these needs, a plethora of spatial verification approaches have been developed in the past two decades. A spatial verification method intercomparison was established in 2007 with the aim of gaining a better understanding of the abilities of the new spatial verification methods to diagnose different types of forecast errors. The project focused on prescribed errors for quantitative precipitation forecasts over the central United States. The intercomparison led to a classification of spatial verification methods and a cataloging of their diagnostic capabilities, providing useful guidance to end users, model developers, and verification scientists. A decade later, NWP systems have continued to increase in resolution, including advances in high-resolution ensembles. This article describes the setup of...
An accurate simulation of rainfall at local scale and at high temporal resolution is needed for m... more An accurate simulation of rainfall at local scale and at high temporal resolution is needed for many climate change impact studies. The statistical downscaling (SD) techniques are often used to perform this simulation taking into account their relative simplicity. One performance criterion associated to SD rainfall simulation is the spatial-temporal coherence of the precipitation field. The present work assesses the spatial coherence of daily rainfall data from observation (9 climatic stations distributed over Chaudiere watershed with records from 1961-2000) and simulations from various SD techniques. CGCM3 and HadCM3 climatic models were considered for large scale predictors necessary for the SD. In a first step, the SD outputs characteristics are evaluated in particular in terms of percentage of explained variance. In a second step, intersite correlations and principal component analysis (PCA) are used to determine the considered SD techniques efficiency to reproduce the different...
We are grateful to T. Kosatsky for his useful comments. We also acknowledge the Data Access Integ... more We are grateful to T. Kosatsky for his useful comments. We also acknowledge the Data Access Integration (DA) Team for providing the data and technical support. This project was supported by the Québec Government Fonds vert of the Action 21 of the Plan d'action 2006-2012 sur les changements climatiques (PACC) and by Health Canada. The DAI Portal (http://climat-quebec.qc.ca/CC-DEV/trunk/index. php/pages/dai) is made possible by a collaboration among the Global Environmental and Climate Change Centre (GEC3), the Adaptation and Impacts Research Division (AIRD) of Environment Canada, and the Drought Research Initiative (DRI). The Ouranos Consortium (in Quebec) provides access of the Canadian Regional Climate Model (CRCM). The authors declare they have no actual or potential competing financial interests.
Verification of forecasts and warnings of highimpact weather is needed by the meteorological cent... more Verification of forecasts and warnings of highimpact weather is needed by the meteorological centres, but how to perform it still presents many open questions, starting from which data are suitable as reference. This paper reviews new observations which can be considered for the verification of high-impact weather and provides advice for their usage in objective verification. Two high-impact weather phenomena are considered: thunderstorm and fog. First, a framework for the verification of high-impact weather is proposed, including the definition of forecast and observations in this context and creation of a verification set. Then, new observations showing a potential for the detection and quantification of high-impact weather are reviewed, including remote sensing datasets, products developed for nowcasting, datasets derived from telecommunication systems, data collected from citizens, reports of impacts and claim/damage reports from insurance companies. The observation characteristics which are relevant for their usage in forecast verification are also discussed. Examples of forecast evaluation and verification are then presented, highlighting the methods which can be adopted to address the issues posed by the usage of these non-conventional observations and objectively quantify the skill of a high-impact weather forecast.
Capulse Summary The Year of Polar Prediction in the Southern Hemisphere had a Special Observing P... more Capulse Summary The Year of Polar Prediction in the Southern Hemisphere had a Special Observing Period (SOP) during the 2018-2019 austral summer. Activities during and resulting from the Antarctic SOP are described.
As part of the second phase of the spatial forecast verification intercomparison project (ICP), d... more As part of the second phase of the spatial forecast verification intercomparison project (ICP), dubbed the Mesoscale Verification Intercomparison in Complex Terrain (MesoVICT) project, a new set of idealized test fields is prepared. This paper describes these new fields and their rationale and uses them to analyze a number of summary measures associated with distance and geometric-based approaches. The results provide guidance about how they inform about performance under various scenarios. The new case comparisons are grouped into four categories: (i) pathological situations such as when a variable is zero valued at all grid points; (ii) circular events aimed at evaluating how different methods handle contrived situations, such as equal but opposite translations, the presence of multiple events of same/different size, boundary effects, and the influence of the positioning of events in the domain; (iii) elliptical events representing simplified scenarios that mimic commonly encounte...
Doppler light detection and ranging (lidar) wind profilers have proven their capability to measur... more Doppler light detection and ranging (lidar) wind profilers have proven their capability to measure vertical wind profiles with an accuracy comparable to anemometers and radiosondes. However, most of these comparisons were performed over short time periods or at mid-latitudes. This study presents a multi-year assessment of the accuracy of Doppler lidar wind-profile measurements in the Arctic by comparing them with coincident radiosonde observations, and excellent agreement was observed. The suitability of the Doppler lidar for verification case studies of operational numerical weather prediction (NWP) models during the World Meteorological Organization’s Year of Polar Prediction is also demonstrated, by using Environment and Climate Change Canada’s (ECCC) global environmental multiscale model (GEM-2.5 km and GEM-10 km). Since 2016, identical scanning Doppler lidars were deployed at two supersites commissioned by ECCC as part of the Canadian Arctic Weather Science project. The supersi...
The International Verification Methods Workshop was held online in November 2020 and included ses... more The International Verification Methods Workshop was held online in November 2020 and included sessions on physical error characterization using process diagnostics and error tracking techniques; exploitation of data assimilation techniques in verification practices, e.g., to address representativeness issues and observation uncertainty; spatial verification methods and the Model Evaluation Tools, as unified reference verification software; and meta-verification and best practices for scores computation. The workshop reached out to diverse research communities working in the areas of high-impact weather, subseasonal to seasonal prediction, polar prediction, and sea ice and ocean prediction. This article summarizes the major outcomes of the workshop and outlines future strategic directions for verification research.
<jats:p>&amp;lt;p&amp;gt;Numerical Model Prediction (NWP) verification against stat... more <jats:p>&amp;lt;p&amp;gt;Numerical Model Prediction (NWP) verification against station measurements from a surface network is affected by sub-tile representativeness issues. Moreover, the station network is often not representative of the whole verification domain (e.g. usually coastal stations are predominant) and large unpopulated regions (such as oceans, Polar regions, deserts) are under-sampled. Verification against gridded analyses mitigate these issues, since they partially address the sub-tile representativeness, and sample homogeneously the verification domain. Moreover, gridded analyses merge station network measurements to radar and satellite retrieval estimates, in a physical coherent fashion, over the same NWP grid. Verification against own analysis, despite quite convenient, is however hampered by its dependence on the NWP background model, which renders the verification &amp;amp;#8220;incestuous&amp;amp;#8221;, further than being affected by the uncertainties introduced by retrieval algorithms and Data Assimilation (DA) procedures.&amp;lt;/p&amp;gt;&amp;lt;p&amp;gt;In this study we investigate the use of a gridded NWP own analysis for verification, by applying a mask to reduce the background model contribution. The mask weights the verification scores to account for the amounts of observations assimilated and their associated uncertainty, as estimated from DA. We illustrate the approach by using the Canadian Precipitation Analysis (CaPA), which assimilates station measurements, radar and satellite-based (IMERG) observations. The CaPA confidence (weighting) mask is dynamic and changes depending on the daily available (assimilated) observations, and on their corresponding DA error statistics; it is defined as&amp;lt;/p&amp;gt;&amp;lt;p&amp;gt;&amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160; &amp;amp;#160;mask = 1 - var(A-O)/var(B-O)&amp;lt;/p&amp;gt;&amp;lt;p&amp;gt;where A=analysis, B=Background, O=observations. Where the analysis is identical to the background model, the weighting mask is zero.&amp;lt;/p&amp;gt;&amp;lt;p&amp;gt;We evaluate the Canadian Regional Deterministic Prediction System (RDPS), which is the NWP system used as background model for CaPA. As expected, the verification results obtained by using the weighting mask lay between the verification results obtained verifying against the analysis over the full domain, and the results obtained verifying against station measurements.&amp;amp;#160;The effects of sub-tile representativeness are quantified by comparing verification results against station measurements to verification results against CaPA for the grid-points co-located with the stations. Finally, the comparison of the verification results against CaPA over the full domain versus the verification results against CaPA for the grid-points co-located with stations, estimates to which extent the station network is representative of the full domain.&amp;lt;/p&amp;gt;&amp;lt;p&amp;gt;The approach aims to propose a simple -yet effective- better practice for verification against own analysis.&amp;lt;/p&amp;gt;</jats:p>
The Year of Polar Prediction (YOPP) is planned for mid-2017 to mid-2019, centred on 2018. Its goa... more The Year of Polar Prediction (YOPP) is planned for mid-2017 to mid-2019, centred on 2018. Its goal is to enable a significant improvement in environmental prediction capabilities for the polar regions and beyond, by coordinating a period of intensive observing, modelling, prediction, verification, user-engagement and education activities. With a focus on time scales from hours to a season, YOPP is a major initiative of the World Meteorological Organization’s World Weather Research Programme (WWRP) and a key component of the Polar Prediction Project (PPP). YOPP is being planned and coordinated by the PPP Steering Group together with representatives from partners and other initiatives, including the World Climate Research Programme’s Polar Climate Predictability Initiative (PCPI). The objectives of YOPP are to: 1. Improve the existing polar observing system (enhanced coverage, higher-quality observations). 2. Gather additional observations through field programmes aimed at improving u...
The continuous measuring of the vertical profile of water vapor in the boundary layer using a com... more The continuous measuring of the vertical profile of water vapor in the boundary layer using a commercially available differential absorption lidar (DIAL) has only recently been made possible. Since September 2018, a new pre-production version of the Vaisala DIAL system has operated at the Iqaluit supersite (63.74°N, 68.51°W), commissioned by Environment and Climate Change Canada (ECCC) as part of the Canadian Arctic Weather Science project. This study presents its evaluation during the extremely dry conditions experienced in the Arctic by comparing it with coincident radiosonde and Raman lidar observations. Comparisons over a one year period were strongly correlated (r > 0.8 at almost all heights) and exhibited an average bias of +0.13 ± 0.01 g/kg (DIAL-sonde) and +0.18 ± 0.02 g/kg (DIAL-Raman). Larger differences exhibiting distinct artifacts were found between 250 and 400 m above ground level (AGL). The DIAL’s observations were also used to conduct a verification case study of ...
Abstract Forecast verification is a key component of a forecasting system; it provides informatio... more Abstract Forecast verification is a key component of a forecasting system; it provides information about forecast qualify to model and forecast developers and various users. This chapter provides an overview of methods relevant to sub-seasonal to seasonal (S2S) forecast verification, starting with the definition of forecast goodness and some fundamental forecast quality attributes. Next, the factors affecting the design of verification studies are presented. The recognition of uncertainties in observational data sets and the need for care in matching forecasts and observations is also discussed. A large part of the chapter is dedicated to a review of the most common deterministic and probabilistic forecast verification measures and a summary of novel spatial verification methods developed during the last two decades. Types of S2S forecasts and current verification practices are presented. The chapter concludes with a summary, challenges, and recommendations for advancing S2S verification research and practice.
Assessing the quality of precipitation forecasts requires observations, but all precipitation obs... more Assessing the quality of precipitation forecasts requires observations, but all precipitation observations have associated uncertainties making it difficult to quantify the true forecast quality. One of the largest uncertainties is due to the wind-induced undercatch of solid precipitation gauge measurements. This study discusses how this impacts the verification of precipitation forecasts for Norway for one global model [the high-resolution version of the ECMWF Integrated Forecasting System (IFS-HRES)], and one high-resolution, limited-area model [Applications of Research to Operations at Mesoscale (MEPS)]. First, the forecasts are compared with high-quality reference measurements (less undercatch) and with more simple measurement equipment commonly available (substantial undercatch) at the Haukeliseter observation site. Then the verification is extended to include all Norwegian observation sites: 1) stratified by wind speed, since calm (windy) conditions experience less (more) unde...
Verification of high-impact weather is needed by the Meteorological Centres, but how to perform i... more Verification of high-impact weather is needed by the Meteorological Centres, but how to perform it still presents many open questions, starting from which data are suitable as reference. This paper reviews new observations which can be considered for the verification of high-impact weather, and provides advice for their usage in objective verification. Two highimpact weather phenomena are considered: Thunderstorm and fog. First, a framework for the verification of high-impact weather is proposed, including the definition of forecast and observations in this context and creation of a verification set. Then, new observations showing a potential for the detection and quantification of high-impact weather are reviewed, including remote sensing datasets, products developed for nowcasting, datasets derived from telecommunication systems, data collected from citizens, reports of impacts and claim/damage reports from insurance companies. The observation characteristics which are relevant for their usage in forecast verification are also discussed. Examples of forecast evaluation and verification are then presented, highlighting the methods which can be adopted to address the issues posed by the usage of these non-conventional observations and objectively quantify the skill of a high-impact weather forecast. 1 Introduction Verification of forecasts and warnings issued for high-impact weather is increasingly needed by operational centres. The model and nowcast products used in operations to support the forecasting and warning of high-impact weather such as thunderstorm cells also need to be verified. The World Weather Research Programme (WWRP) of the World Meteorological Organisation
Bulletin of the American Meteorological Society, 2018
Recent advancements in numerical weather prediction (NWP) and the enhancement of model resolution... more Recent advancements in numerical weather prediction (NWP) and the enhancement of model resolution have created the need for more robust and informative verification methods. In response to these needs, a plethora of spatial verification approaches have been developed in the past two decades. A spatial verification method intercomparison was established in 2007 with the aim of gaining a better understanding of the abilities of the new spatial verification methods to diagnose different types of forecast errors. The project focused on prescribed errors for quantitative precipitation forecasts over the central United States. The intercomparison led to a classification of spatial verification methods and a cataloging of their diagnostic capabilities, providing useful guidance to end users, model developers, and verification scientists. A decade later, NWP systems have continued to increase in resolution, including advances in high-resolution ensembles. This article describes the setup of...
An accurate simulation of rainfall at local scale and at high temporal resolution is needed for m... more An accurate simulation of rainfall at local scale and at high temporal resolution is needed for many climate change impact studies. The statistical downscaling (SD) techniques are often used to perform this simulation taking into account their relative simplicity. One performance criterion associated to SD rainfall simulation is the spatial-temporal coherence of the precipitation field. The present work assesses the spatial coherence of daily rainfall data from observation (9 climatic stations distributed over Chaudiere watershed with records from 1961-2000) and simulations from various SD techniques. CGCM3 and HadCM3 climatic models were considered for large scale predictors necessary for the SD. In a first step, the SD outputs characteristics are evaluated in particular in terms of percentage of explained variance. In a second step, intersite correlations and principal component analysis (PCA) are used to determine the considered SD techniques efficiency to reproduce the different...
We are grateful to T. Kosatsky for his useful comments. We also acknowledge the Data Access Integ... more We are grateful to T. Kosatsky for his useful comments. We also acknowledge the Data Access Integration (DA) Team for providing the data and technical support. This project was supported by the Québec Government Fonds vert of the Action 21 of the Plan d'action 2006-2012 sur les changements climatiques (PACC) and by Health Canada. The DAI Portal (http://climat-quebec.qc.ca/CC-DEV/trunk/index. php/pages/dai) is made possible by a collaboration among the Global Environmental and Climate Change Centre (GEC3), the Adaptation and Impacts Research Division (AIRD) of Environment Canada, and the Drought Research Initiative (DRI). The Ouranos Consortium (in Quebec) provides access of the Canadian Regional Climate Model (CRCM). The authors declare they have no actual or potential competing financial interests.
Verification of forecasts and warnings of highimpact weather is needed by the meteorological cent... more Verification of forecasts and warnings of highimpact weather is needed by the meteorological centres, but how to perform it still presents many open questions, starting from which data are suitable as reference. This paper reviews new observations which can be considered for the verification of high-impact weather and provides advice for their usage in objective verification. Two high-impact weather phenomena are considered: thunderstorm and fog. First, a framework for the verification of high-impact weather is proposed, including the definition of forecast and observations in this context and creation of a verification set. Then, new observations showing a potential for the detection and quantification of high-impact weather are reviewed, including remote sensing datasets, products developed for nowcasting, datasets derived from telecommunication systems, data collected from citizens, reports of impacts and claim/damage reports from insurance companies. The observation characteristics which are relevant for their usage in forecast verification are also discussed. Examples of forecast evaluation and verification are then presented, highlighting the methods which can be adopted to address the issues posed by the usage of these non-conventional observations and objectively quantify the skill of a high-impact weather forecast.
Capulse Summary The Year of Polar Prediction in the Southern Hemisphere had a Special Observing P... more Capulse Summary The Year of Polar Prediction in the Southern Hemisphere had a Special Observing Period (SOP) during the 2018-2019 austral summer. Activities during and resulting from the Antarctic SOP are described.
As part of the second phase of the spatial forecast verification intercomparison project (ICP), d... more As part of the second phase of the spatial forecast verification intercomparison project (ICP), dubbed the Mesoscale Verification Intercomparison in Complex Terrain (MesoVICT) project, a new set of idealized test fields is prepared. This paper describes these new fields and their rationale and uses them to analyze a number of summary measures associated with distance and geometric-based approaches. The results provide guidance about how they inform about performance under various scenarios. The new case comparisons are grouped into four categories: (i) pathological situations such as when a variable is zero valued at all grid points; (ii) circular events aimed at evaluating how different methods handle contrived situations, such as equal but opposite translations, the presence of multiple events of same/different size, boundary effects, and the influence of the positioning of events in the domain; (iii) elliptical events representing simplified scenarios that mimic commonly encounte...
Doppler light detection and ranging (lidar) wind profilers have proven their capability to measur... more Doppler light detection and ranging (lidar) wind profilers have proven their capability to measure vertical wind profiles with an accuracy comparable to anemometers and radiosondes. However, most of these comparisons were performed over short time periods or at mid-latitudes. This study presents a multi-year assessment of the accuracy of Doppler lidar wind-profile measurements in the Arctic by comparing them with coincident radiosonde observations, and excellent agreement was observed. The suitability of the Doppler lidar for verification case studies of operational numerical weather prediction (NWP) models during the World Meteorological Organization’s Year of Polar Prediction is also demonstrated, by using Environment and Climate Change Canada’s (ECCC) global environmental multiscale model (GEM-2.5 km and GEM-10 km). Since 2016, identical scanning Doppler lidars were deployed at two supersites commissioned by ECCC as part of the Canadian Arctic Weather Science project. The supersi...
Uploads
Papers by Barbara Casati