2002 CMAQ Performance Evaluation Appraoch
Evaluation by Model Sub-region
For continental scale modeling of regional haze we expect
considerable variation in model performance by geographical
region, for example, coastal versus mountainous sites and the eastern US
(high sulfur) versus the western US (low sulfur) sites.
Therefore, it is useful to divide the model domain into different
georgraphical regions when compiling model performance metrics.
Listed below are several proposals for grouping IMPROVE sites into model performance evaluation zones:
We seek input on the draft perfomanace zones so that we
can revise the UCR model evaluation software to
genrerate performance metrics for appropriately defined sub-regions.
Important issues to consider include:
- More higly resolved sub-regional evaluation also requires greater
effort required to analyze and interpret plots.
- New analysis & visulation tools may be needed to summarize sub-regional model performance.
Stratification by Site Type
We also expect variation in model performance either by type of site, i.e.,
urban versus rural versus remote environements. Thus it might also be
useful to stratify the ambient monitoring data according to site type.
To some extent the design of the monitoring networks themselves address
the issue of site type for example,the IMPROVE sites are only located
at remote sites. For other networks such as the AQS, for example, the
sites are not stratified by type, and additional effort will be needed
to stratify these data for mdoel performance evaluations.
One possible stratification scheme of AQS sites is:
- Urban hot spots
- Aged Urban plumes
- Rural sites
- Remote sites
Evaluation by Meteorological Conditions
Currently we evaluate model performance by season: Winter, Spring, Summer and Fall.
We expect model performance within a season to vary depending on episodic
meteorology conditions. We need to determine how best to stratify by episode:
(e.g., by temperature, PBL Height, passage of frontal zones, etc.)
Therefore, it would be more useful to stratify model
performance results according to meteorology conditions. As in the case of
greater sub-regional resolution, meteorology stratification will result
in increased evaluation results, and new tools or methods will be needed to
summarize model performance.
