Home > Explore Data & Reports > Skill assessment of NOAA's Chesapeake Bay Vibrio vulnificus model


Lanerolle, L., R. Daniels, B. Zhang, and J. Jacobs. 2016. Skill assessment of NOAA's Chesapeake Bay Vibrio vulnificus model. NOAA Technical Memorandum NOS NCCOS 215. Oxford, MD. 18 pp. https://doi.org/10.7289/V5/TM-NOS-NCCOS-215

Data/Report Type:

NOAA Technical Memorandum


Since 2012, NOAA has generated model guidance for the probability of occurrence of the harmful marine bacteria Vibrio vulnificus in Chesapeake Bay. The system employs NOAAs Operational Forecast System for Chesapeake Bay (CBOFS) to force a statistical model developed by NCCOS and provides daily guidance for the entirety of the Bay. While the empirical model was validated in 2014, the system as a whole has not been assessed for model skill. In this memorandum, we report the results of a skill assessment conducted with paired observations and model predictions for the year 2011. As part of this exercise, we also evaluate model sensitivity to provide guidance on accuracy requirements for needed fields obtained from CBOFS. Model sensitivity is dependent on the salinity in which predictions are applied. Bias in modeled salinity has a more pronounced effect in lower salinity areas (< 5PSU) than high. Overall, results suggest that effective criteria for accuracy requirements from CBOFS for this product are within 1°C and 1 PSU. Predicted sea surface temperature (SST) was relatively close to requirements having an average difference between predicted and observed of 1.4°C. Modeled salinity, however, was positively biased by 2.5 PSU. This bias is a known issue with many of the operational forecast systems, and was corrected using a post-hoc approach for the Vibrio vulnificus model guidance. Predicted probability of V. vulnificus occurrence matched well with observed overall, with no apparent spatial or temporal trends in error. However, the model error slightly exceeds our criteria of < 10% RMSE (11%) when raw salinity data is used from CBOFS due to the 2.5 PSU bias. When salinity is adjusted to remove bias, as is done routinely for guidance dissemination, the criteria is met. This initial skill assessment provides evidence that the system is fully capable of providing accurate guidance given accurate input. Attention should be focused on improving CBOFS salinity estimates. Currently, post-hoc correction of salinity bias is required to meet skill criteria. The monitoring program enabling this evaluation of skill is ongoing, and subsequent skill assessments will be conducted regularly to ensure maintenance of model performance.

Note to readers with disabilities: Some scientific publications linked from this website may not conform to Section 508 accessibility standards due to the complexity of the information being presented. If you need assistance accessing this electronic content, please contact the lead/corresponding author, Primary Contact, or nccos.webcontent@noaa.gov.

Explore Similar Data/Reports


NCCOS delivers ecosystem science solutions for stewardship of the nation’s ocean and coastal resources to sustain thriving coastal communities and economies.

Stay Connected

Sign up for our quarterly newsletter or view our archives.

Follow us on Social

Listen to our Podcast

Check our our new podcast "Coastal Conversations"