Catalog-based Forecast Evaluation¶
This example shows how to evaluate a catalog-based forecasting using the Number test. This test is the simplest of the evaluations.
Define forecast properties (time horizon, spatial region, etc).
Access catalog from ComCat
Filter catalog to be consistent with the forecast properties
Apply catalog-based number test to catalog
Visualize results for catalog-based forecast
Load required libraries¶
Most of the core functionality can be imported from the top-level
csep package. Utilities are available from the
import csep from csep.core import regions, catalog_evaluations from csep.utils import datasets, time_utils
Define start and end times of forecast¶
Forecasts should define a time horizon in which they are valid. The choice is flexible for catalog-based forecasts, because the catalogs can be filtered to accommodate multiple end-times. Conceptually, these should be separate forecasts.
Define spatial and magnitude regions¶
Before we can compute the bin-wise rates we need to define a spatial region and a set of magnitude bin edges. The magnitude
bin edges # are the lower bound (inclusive) except for the last bin, which is treated as extending to infinity. We can
bind these # to the forecast object. This can also be done by passing them as keyword arguments
# Magnitude bins properties min_mw = 4.95 max_mw = 8.95 dmw = 0.1 # Create space and magnitude regions. The forecast is already filtered in space and magnitude magnitudes = regions.magnitude_bins(min_mw, max_mw, dmw) region = regions.california_relm_region() # Bind region information to the forecast (this will be used for binning of the catalogs) space_magnitude_region = regions.create_space_magnitude_region(region, magnitudes)
Load catalog forecast¶
To reduce the file size of this example, we’ve already filtered the catalogs to the appropriate magnitudes and spatial locations. The original forecast was computed for 1 year following the start date, so we still need to filter the catalog in time. We can do this by passing a list of filtering arguments to the forecast or updating the class.
By default, the forecast loads catalogs on-demand, so the filters are applied as the catalog loads. On-demand means that until we loop over the forecast in some capacity, none of the catalogs are actually loaded.
More fine-grain control and optimizations can be achieved by creating a
Obtain evaluation catalog from ComCat¶
csep.core.forecasts.CatalogForecast provides a method to compute the expected number of events in spatial cells. This
requires a region with magnitude information.
We need to filter the ComCat catalog to be consistent with the forecast. This can be done either through the ComCat API or using catalog filtering strings. Here we’ll use the Comcat API to make the data access quicker for this example. We still need to filter the observed catalog in space though.
Fetched ComCat catalog in 0.6147332191467285 seconds. Downloaded catalog from ComCat with following parameters Start Date: 1992-06-28 11:57:34.130000+00:00 End Date: 1992-07-24 18:14:36.250000+00:00 Min Latitude: 33.901 and Max Latitude: 36.698 Min Longitude: -118.067 and Max Longitude: -116.285 Min Magnitude: 4.95 Found 20 events in the ComCat catalog.
Perform number test¶
We can perform the Number test on the catalog based forecast using the observed catalog we obtained from Comcat.