.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "tutorials/catalog_forecast_evaluation.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_tutorials_catalog_forecast_evaluation.py: .. _catalog-forecast-evaluation: Catalog-based Forecast Evaluation ================================= This example shows how to evaluate a catalog-based forecasting using the Number test. This test is the simplest of the evaluations. Overview: 1. Define forecast properties (time horizon, spatial region, etc). 2. Access catalog from ComCat 3. Filter catalog to be consistent with the forecast properties 4. Apply catalog-based number test to catalog 5. Visualize results for catalog-based forecast .. GENERATED FROM PYTHON SOURCE LINES 19-24 Load required libraries ----------------------- Most of the core functionality can be imported from the top-level :mod:`csep` package. Utilities are available from the :mod:`csep.utils` subpackage. .. GENERATED FROM PYTHON SOURCE LINES 24-29 .. code-block:: Python import csep from csep.core import regions, catalog_evaluations from csep.utils import datasets, time_utils .. GENERATED FROM PYTHON SOURCE LINES 30-35 Define start and end times of forecast -------------------------------------- Forecasts should define a time horizon in which they are valid. The choice is flexible for catalog-based forecasts, because the catalogs can be filtered to accommodate multiple end-times. Conceptually, these should be separate forecasts. .. GENERATED FROM PYTHON SOURCE LINES 35-39 .. code-block:: Python start_time = time_utils.strptime_to_utc_datetime("1992-06-28 11:57:35.0") end_time = time_utils.strptime_to_utc_datetime("1992-07-28 11:57:35.0") .. GENERATED FROM PYTHON SOURCE LINES 40-47 Define spatial and magnitude regions ------------------------------------ Before we can compute the bin-wise rates we need to define a spatial region and a set of magnitude bin edges. The magnitude bin edges # are the lower bound (inclusive) except for the last bin, which is treated as extending to infinity. We can bind these # to the forecast object. This can also be done by passing them as keyword arguments into :func:`csep.load_catalog_forecast`. .. GENERATED FROM PYTHON SOURCE LINES 47-60 .. code-block:: Python # Magnitude bins properties min_mw = 4.95 max_mw = 8.95 dmw = 0.1 # Create space and magnitude regions. The forecast is already filtered in space and magnitude magnitudes = regions.magnitude_bins(min_mw, max_mw, dmw) region = regions.california_relm_region() # Bind region information to the forecast (this will be used for binning of the catalogs) space_magnitude_region = regions.create_space_magnitude_region(region, magnitudes) .. GENERATED FROM PYTHON SOURCE LINES 61-72 Load catalog forecast --------------------- To reduce the file size of this example, we've already filtered the catalogs to the appropriate magnitudes and spatial locations. The original forecast was computed for 1 year following the start date, so we still need to filter the catalog in time. We can do this by passing a list of filtering arguments to the forecast or updating the class. By default, the forecast loads catalogs on-demand, so the filters are applied as the catalog loads. On-demand means that until we loop over the forecast in some capacity, none of the catalogs are actually loaded. More fine-grain control and optimizations can be achieved by creating a :class:`csep.core.forecasts.CatalogForecast` directly. .. GENERATED FROM PYTHON SOURCE LINES 72-81 .. code-block:: Python forecast = csep.load_catalog_forecast(datasets.ucerf3_ascii_format_landers_fname, start_time = start_time, end_time = end_time, region = space_magnitude_region, apply_filters = True) # Assign filters to forecast forecast.filters = [f'origin_time >= {forecast.start_epoch}', f'origin_time < {forecast.end_epoch}'] .. GENERATED FROM PYTHON SOURCE LINES 82-91 Obtain evaluation catalog from ComCat ------------------------------------- The :class:`csep.core.forecasts.CatalogForecast` provides a method to compute the expected number of events in spatial cells. This requires a region with magnitude information. We need to filter the ComCat catalog to be consistent with the forecast. This can be done either through the ComCat API or using catalog filtering strings. Here we'll use the ComCat API to make the data access quicker for this example. We still need to filter the observed catalog in space though. .. GENERATED FROM PYTHON SOURCE LINES 91-102 .. code-block:: Python # Obtain Comcat catalog and filter to region. comcat_catalog = csep.query_comcat(start_time, end_time, min_magnitude=forecast.min_magnitude) # Filter observed catalog using the same region as the forecast comcat_catalog = comcat_catalog.filter_spatial(forecast.region) print(comcat_catalog) # Plot the catalog comcat_catalog.plot() .. image-sg:: /tutorials/images/sphx_glr_catalog_forecast_evaluation_001.png :alt: catalog forecast evaluation :srcset: /tutorials/images/sphx_glr_catalog_forecast_evaluation_001.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-script-out .. code-block:: none Fetched ComCat catalog in 0.6656863689422607 seconds. Downloaded catalog from ComCat with following parameters Start Date: 1992-06-28 12:00:45+00:00 End Date: 1992-07-24 18:14:36.250000+00:00 Min Latitude: 33.901 and Max Latitude: 36.705 Min Longitude: -118.067 and Max Longitude: -116.285 Min Magnitude: 4.95 Found 19 events in the ComCat catalog. Name: None Start Date: 1992-06-28 12:00:45+00:00 End Date: 1992-07-24 18:14:36.250000+00:00 Latitude: (33.901, 36.705) Longitude: (-118.067, -116.285) Min Mw: 4.95 Max Mw: 6.3 Event Count: 19 .. GENERATED FROM PYTHON SOURCE LINES 103-107 Perform number test ------------------- We can perform the Number test on the catalog based forecast using the observed catalog we obtained from Comcat. .. GENERATED FROM PYTHON SOURCE LINES 107-110 .. code-block:: Python number_test_result = catalog_evaluations.number_test(forecast, comcat_catalog) .. rst-class:: sphx-glr-script-out .. code-block:: none Processed 1 catalogs in 0.0011861324310302734 seconds Processed 2 catalogs in 0.0016164779663085938 seconds Processed 3 catalogs in 0.0020356178283691406 seconds Processed 4 catalogs in 0.0023474693298339844 seconds Processed 5 catalogs in 0.002604246139526367 seconds Processed 6 catalogs in 0.002966642379760742 seconds Processed 7 catalogs in 0.0032396316528320312 seconds Processed 8 catalogs in 0.003611326217651367 seconds Processed 9 catalogs in 0.004532337188720703 seconds Processed 10 catalogs in 0.004876375198364258 seconds Processed 20 catalogs in 0.008562564849853516 seconds Processed 30 catalogs in 0.012447834014892578 seconds Processed 40 catalogs in 0.016121864318847656 seconds Processed 50 catalogs in 0.0202786922454834 seconds Processed 60 catalogs in 0.02376842498779297 seconds Processed 70 catalogs in 0.027079105377197266 seconds Processed 80 catalogs in 0.03068685531616211 seconds Processed 90 catalogs in 0.03446841239929199 seconds Processed 100 catalogs in 0.037847042083740234 seconds Processed 200 catalogs in 0.07080221176147461 seconds Processed 300 catalogs in 0.10667037963867188 seconds Processed 400 catalogs in 0.1414473056793213 seconds Processed 500 catalogs in 0.2045900821685791 seconds Processed 600 catalogs in 0.23855948448181152 seconds Processed 700 catalogs in 0.27303290367126465 seconds Processed 800 catalogs in 0.3395709991455078 seconds Processed 900 catalogs in 0.3728957176208496 seconds Processed 1000 catalogs in 0.40724658966064453 seconds Processed 2000 catalogs in 0.8882627487182617 seconds Processed 3000 catalogs in 1.3292644023895264 seconds Processed 4000 catalogs in 1.7945597171783447 seconds Processed 5000 catalogs in 2.2449986934661865 seconds Processed 6000 catalogs in 2.688971996307373 seconds Processed 7000 catalogs in 3.161001443862915 seconds Processed 8000 catalogs in 3.5757346153259277 seconds Processed 9000 catalogs in 4.072112083435059 seconds Processed 10000 catalogs in 4.516096115112305 seconds .. GENERATED FROM PYTHON SOURCE LINES 111-115 Plot number test result ----------------------- We can create a simple visualization of the number test from the evaluation result class. .. GENERATED FROM PYTHON SOURCE LINES 115-116 .. code-block:: Python ax = number_test_result.plot(show=True) .. image-sg:: /tutorials/images/sphx_glr_catalog_forecast_evaluation_002.png :alt: Number Test :srcset: /tutorials/images/sphx_glr_catalog_forecast_evaluation_002.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 9.913 seconds) .. _sphx_glr_download_tutorials_catalog_forecast_evaluation.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: catalog_forecast_evaluation.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: catalog_forecast_evaluation.py ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_