{
"cells": [
{
"cell_type": "markdown",
"id": "2148ff56-339e-4831-bcb3-03a36c4738ed",
"metadata": {},
"source": [
"# Extended source sensitivity calculator\n",
"\n",
"This notebook shows how to compute a extended source sensitivity for a specific model. In the current status of cosipy, we are limited by the available responses, i.e at 511, 1157, 1173, 1333 and 1809 keV and a coarse continuum. Keep in mind that due to the coarseness of the current responses, the sensitivity might be underestimated. \n",
"\n",
"The collaboration is currently developping a neural network approach for the response to use with an unbinned analysis. This will allow us to fully exploit COSI power. "
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "2d780467-ec91-481b-90a8-353f9db61ac8",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"15:21:22 WARNING The naima package is not available. Models that depend on it will not be functions.py:43\n",
" available \n",
"\n"
],
"text/plain": [
"\u001b[38;5;46m15:21:22\u001b[0m\u001b[38;5;46m \u001b[0m\u001b[38;5;134mWARNING \u001b[0m \u001b[1;38;5;251m The naima package is not available. Models that depend on it will not be \u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b]8;id=38245;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/astromodels/functions/functions_1D/functions.py\u001b\\\u001b[2mfunctions.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=820159;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/astromodels/functions/functions_1D/functions.py#43\u001b\\\u001b[2m43\u001b[0m\u001b]8;;\u001b\\\n",
"\u001b[38;5;46m \u001b[0m \u001b[1;38;5;251mavailable \u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b[2m \u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
" WARNING The GSL library or the pygsl wrapper cannot be loaded. Models that depend on it functions.py:65\n",
" will not be available. \n",
"\n"
],
"text/plain": [
"\u001b[38;5;46m \u001b[0m\u001b[38;5;46m \u001b[0m\u001b[38;5;134mWARNING \u001b[0m \u001b[1;38;5;251m The GSL library or the pygsl wrapper cannot be loaded. Models that depend on it \u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b]8;id=167690;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/astromodels/functions/functions_1D/functions.py\u001b\\\u001b[2mfunctions.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=307444;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/astromodels/functions/functions_1D/functions.py#65\u001b\\\u001b[2m65\u001b[0m\u001b]8;;\u001b\\\n",
"\u001b[38;5;46m \u001b[0m \u001b[1;38;5;251mwill not be available. \u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b[2m \u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
" WARNING The ebltable package is not available. Models that depend on it will not be absorption.py:33\n",
" available \n",
"\n"
],
"text/plain": [
"\u001b[38;5;46m \u001b[0m\u001b[38;5;46m \u001b[0m\u001b[38;5;134mWARNING \u001b[0m \u001b[1;38;5;251m The ebltable package is not available. Models that depend on it will not be \u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b]8;id=94778;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/astromodels/functions/functions_1D/absorption.py\u001b\\\u001b[2mabsorption.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=555992;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/astromodels/functions/functions_1D/absorption.py#33\u001b\\\u001b[2m33\u001b[0m\u001b]8;;\u001b\\\n",
"\u001b[38;5;46m \u001b[0m \u001b[1;38;5;251mavailable \u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b[2m \u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"15:21:23 INFO Starting 3ML! __init__.py:44\n",
"\n"
],
"text/plain": [
"\u001b[38;5;46m15:21:23\u001b[0m\u001b[38;5;46m \u001b[0m\u001b[38;5;49mINFO \u001b[0m \u001b[1;38;5;251m Starting 3ML! \u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b]8;id=488426;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/threeML/__init__.py\u001b\\\u001b[2m__init__.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=110256;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/threeML/__init__.py#44\u001b\\\u001b[2m44\u001b[0m\u001b]8;;\u001b\\\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
" WARNING WARNINGs here are NOT errors __init__.py:45\n",
"\n"
],
"text/plain": [
"\u001b[38;5;46m \u001b[0m\u001b[38;5;46m \u001b[0m\u001b[38;5;134mWARNING \u001b[0m \u001b[1;38;5;251m WARNINGs here are \u001b[0m\u001b[1;31mNOT\u001b[0m\u001b[1;38;5;251m errors \u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b]8;id=250620;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/threeML/__init__.py\u001b\\\u001b[2m__init__.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=335980;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/threeML/__init__.py#45\u001b\\\u001b[2m45\u001b[0m\u001b]8;;\u001b\\\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
" WARNING but are inform you about optional packages that can be installed __init__.py:46\n",
"\n"
],
"text/plain": [
"\u001b[38;5;46m \u001b[0m\u001b[38;5;46m \u001b[0m\u001b[38;5;134mWARNING \u001b[0m \u001b[1;38;5;251m but are inform you about optional packages that can be installed \u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b]8;id=119419;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/threeML/__init__.py\u001b\\\u001b[2m__init__.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=882949;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/threeML/__init__.py#46\u001b\\\u001b[2m46\u001b[0m\u001b]8;;\u001b\\\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
" WARNING to disable these messages, turn off start_warning in your config file __init__.py:47\n",
"\n"
],
"text/plain": [
"\u001b[38;5;46m \u001b[0m\u001b[38;5;46m \u001b[0m\u001b[38;5;134mWARNING \u001b[0m \u001b[1;38;5;251m \u001b[0m\u001b[1;31m to disable these messages, turn off start_warning in your config file\u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b]8;id=470243;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/threeML/__init__.py\u001b\\\u001b[2m__init__.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=750564;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/threeML/__init__.py#47\u001b\\\u001b[2m47\u001b[0m\u001b]8;;\u001b\\\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
" WARNING no display variable set. using backend for graphics without display (agg) __init__.py:53\n",
"\n"
],
"text/plain": [
"\u001b[38;5;46m \u001b[0m\u001b[38;5;46m \u001b[0m\u001b[38;5;134mWARNING \u001b[0m \u001b[1;38;5;251m no display variable set. using backend for graphics without display \u001b[0m\u001b[1;38;5;251m(\u001b[0m\u001b[1;38;5;251magg\u001b[0m\u001b[1;38;5;251m)\u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b]8;id=889165;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/threeML/__init__.py\u001b\\\u001b[2m__init__.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=80745;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/threeML/__init__.py#53\u001b\\\u001b[2m53\u001b[0m\u001b]8;;\u001b\\\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
" WARNING ROOT minimizer not available minimization.py:1208\n",
"\n"
],
"text/plain": [
"\u001b[38;5;46m \u001b[0m\u001b[38;5;46m \u001b[0m\u001b[38;5;134mWARNING \u001b[0m \u001b[1;38;5;251m ROOT minimizer not available \u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b]8;id=500782;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/threeML/minimizer/minimization.py\u001b\\\u001b[2mminimization.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=17242;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/threeML/minimizer/minimization.py#1208\u001b\\\u001b[2m1208\u001b[0m\u001b]8;;\u001b\\\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
" WARNING Multinest minimizer not available minimization.py:1218\n",
"\n"
],
"text/plain": [
"\u001b[38;5;46m \u001b[0m\u001b[38;5;46m \u001b[0m\u001b[38;5;134mWARNING \u001b[0m \u001b[1;38;5;251m Multinest minimizer not available \u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b]8;id=365338;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/threeML/minimizer/minimization.py\u001b\\\u001b[2mminimization.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=530513;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/threeML/minimizer/minimization.py#1218\u001b\\\u001b[2m1218\u001b[0m\u001b]8;;\u001b\\\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
" WARNING PyGMO is not available minimization.py:1228\n",
"\n"
],
"text/plain": [
"\u001b[38;5;46m \u001b[0m\u001b[38;5;46m \u001b[0m\u001b[38;5;134mWARNING \u001b[0m \u001b[1;38;5;251m PyGMO is not available \u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b]8;id=792143;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/threeML/minimizer/minimization.py\u001b\\\u001b[2mminimization.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=484923;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/threeML/minimizer/minimization.py#1228\u001b\\\u001b[2m1228\u001b[0m\u001b]8;;\u001b\\\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"15:21:24 WARNING Could not import plugin FermiLATLike.py. Do you have the relative instrument __init__.py:126\n",
" software installed and configured? \n",
"\n"
],
"text/plain": [
"\u001b[38;5;46m15:21:24\u001b[0m\u001b[38;5;46m \u001b[0m\u001b[38;5;134mWARNING \u001b[0m \u001b[1;38;5;251m Could not import plugin FermiLATLike.py. Do you have the relative instrument \u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b]8;id=195716;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/threeML/__init__.py\u001b\\\u001b[2m__init__.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=861023;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/threeML/__init__.py#126\u001b\\\u001b[2m126\u001b[0m\u001b]8;;\u001b\\\n",
"\u001b[38;5;46m \u001b[0m \u001b[1;38;5;251msoftware installed and configured? \u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b[2m \u001b[0m\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
" WARNING No fermitools installed lat_transient_builder.py:44\n",
"\n"
],
"text/plain": [
"\u001b[38;5;46m \u001b[0m\u001b[38;5;46m \u001b[0m\u001b[38;5;134mWARNING \u001b[0m \u001b[1;38;5;251m No fermitools installed \u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b]8;id=339303;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/threeML/utils/data_builders/fermi/lat_transient_builder.py\u001b\\\u001b[2mlat_transient_builder.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=2719;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/threeML/utils/data_builders/fermi/lat_transient_builder.py#44\u001b\\\u001b[2m44\u001b[0m\u001b]8;;\u001b\\\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"ename": "ImportError",
"evalue": "cannot import name 'SpatialTemplate_2D_Healpix' from 'cosipy.threeml.custom_functions' (/uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/cosipy/threeml/custom_functions.py)",
"output_type": "error",
"traceback": [
"\u001b[31m---------------------------------------------------------------------------\u001b[39m",
"\u001b[31mImportError\u001b[39m Traceback (most recent call last)",
"\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[1]\u001b[39m\u001b[32m, line 27\u001b[39m\n\u001b[32m 25\u001b[39m \u001b[38;5;28;01mfrom\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34;01mcosipy\u001b[39;00m\u001b[34;01m.\u001b[39;00m\u001b[34;01mdata_io\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;28;01mimport\u001b[39;00m EmCDSBinnedData\n\u001b[32m 26\u001b[39m \u001b[38;5;28;01mfrom\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34;01mcosipy\u001b[39;00m\u001b[34;01m.\u001b[39;00m\u001b[34;01msource_injector\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;28;01mimport\u001b[39;00m SourceInjector\n\u001b[32m---> \u001b[39m\u001b[32m27\u001b[39m \u001b[38;5;28;01mfrom\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34;01mcosipy\u001b[39;00m\u001b[34;01m.\u001b[39;00m\u001b[34;01mthreeml\u001b[39;00m\u001b[34;01m.\u001b[39;00m\u001b[34;01mcustom_functions\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;28;01mimport\u001b[39;00m SpatialTemplate_2D_Healpix\n\u001b[32m 28\u001b[39m \u001b[38;5;28;01mfrom\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34;01mhistpy\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;28;01mimport\u001b[39;00m Histogram\n\u001b[32m 29\u001b[39m \u001b[38;5;28;01mfrom\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34;01mmhealpy\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;28;01mimport\u001b[39;00m HealpixMap\n",
"\u001b[31mImportError\u001b[39m: cannot import name 'SpatialTemplate_2D_Healpix' from 'cosipy.threeml.custom_functions' (/uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/cosipy/threeml/custom_functions.py)"
]
}
],
"source": [
"import os\n",
"import logging\n",
"import sys\n",
"\n",
"\n",
"\n",
"logger = logging.getLogger('cosipy')\n",
"logger.setLevel(logging.INFO)\n",
"logger.addHandler(logging.StreamHandler(sys.stdout))\n",
"\n",
"os.environ[\"OMP_NUM_THREADS\"] = \"1\"\n",
"os.environ[\"MKL_NUM_THREADS\"] = \"1\"\n",
"os.environ[\"NUMEXPR_NUM_THREADS\"] = \"1\"\n",
"\n",
"\n",
"from cosipy.spacecraftfile import SpacecraftHistory\n",
"from cosipy.response.FullDetectorResponse import FullDetectorResponse\n",
"from cosipy.response import PointSourceResponse,ExtendedSourceResponse\n",
"from cosipy.util import fetch_wasabi_file\n",
"\n",
"from cosipy.statistics import PoissonLikelihood\n",
"from cosipy.background_estimation import FreeNormBinnedBackground\n",
"from cosipy.interfaces import ThreeMLPluginInterface\n",
"from cosipy.response import BinnedThreeMLModelFolding, BinnedInstrumentResponse, BinnedThreeMLExtendedSourceResponse\n",
"from cosipy.data_io import EmCDSBinnedData\n",
"from cosipy.source_injector import SourceInjector\n",
"from cosipy.threeml.custom_functions import SpatialTemplate_2D_Healpix\n",
"from histpy import Histogram\n",
"from mhealpy import HealpixMap\n",
"\n",
"from scoords import SpacecraftFrame\n",
"\n",
"from astropy.time import Time\n",
"import astropy.units as u\n",
"from astropy.coordinates import SkyCoord, Galactic\n",
"\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"\n",
"from threeML import *\n",
"from astromodels import *\n",
"\n",
"from pathlib import Path\n",
"\n",
"%matplotlib inline"
]
},
{
"cell_type": "markdown",
"id": "dbe965ab-12fb-40a8-82a5-a4a19510b6e0",
"metadata": {
"jp-MarkdownHeadingCollapsed": true
},
"source": [
"## Download the data"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "6b1a4a9d-78cd-4f46-9329-baf31f2c6349",
"metadata": {},
"outputs": [],
"source": [
"data_path = Path(\"\") # /path/to/files. Current dir by default"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d5b68117-177b-4553-b62e-361bd7d456cd",
"metadata": {},
"outputs": [],
"source": [
"#Download the Orientation file\n",
"fetch_wasabi_file('COSI-SMEX/develop/Data/Orientation/DC3_final_530km_3_month_with_slew_15sbins_GalacticEarth_SAA.fits', checksum = 'e86df2407eb052cf0c1db4a8e7598727')"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "7d720757-2f47-471e-baa7-ebf9495a2477",
"metadata": {
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Downloading cosi-pipeline-public/COSI-SMEX/DC3/Data/Responses/extended_source_response/extended_source_response_Al26_merged.h5\n"
]
},
{
"ename": "ClientError",
"evalue": "An error occurred (404) when calling the HeadObject operation: Not Found",
"output_type": "error",
"traceback": [
"\u001b[31m---------------------------------------------------------------------------\u001b[39m",
"\u001b[31mClientError\u001b[39m Traceback (most recent call last)",
"\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[3]\u001b[39m\u001b[32m, line 17\u001b[39m\n\u001b[32m 1\u001b[39m \u001b[38;5;66;03m#Download the extended sourde response\u001b[39;00m\n\u001b[32m 2\u001b[39m \u001b[38;5;66;03m#Choose the response depending on your model\u001b[39;00m\n\u001b[32m 3\u001b[39m \n\u001b[32m (...)\u001b[39m\u001b[32m 15\u001b[39m \n\u001b[32m 16\u001b[39m \u001b[38;5;66;03m#Model for a 1809 keV line\u001b[39;00m\n\u001b[32m---> \u001b[39m\u001b[32m17\u001b[39m \u001b[43mfetch_wasabi_file\u001b[49m\u001b[43m(\u001b[49m\u001b[33;43m'\u001b[39;49m\u001b[33;43mCOSI-SMEX/DC3/Data/Responses/extended_source_response/extended_source_response_Al26_merged.h5\u001b[39;49m\u001b[33;43m'\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43mchecksum\u001b[49m\u001b[43m \u001b[49m\u001b[43m=\u001b[49m\u001b[43m \u001b[49m\u001b[33;43m'\u001b[39;49m\u001b[33;43m218d088159c6e11d0ddecc03ca9fa399\u001b[39;49m\u001b[33;43m'\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[32m 20\u001b[39m \u001b[38;5;66;03m#Model for an other line or continuum source\u001b[39;00m\n\u001b[32m 21\u001b[39m \u001b[38;5;66;03m#fetch_wasabi_file('COSI-SMEX/DC3/Data/Responses/extended_source_response/extended_source_response_continuum_merged.h5',checksum = '16fe005d3ab924ad98322b6579aabf2a')\u001b[39;00m\n",
"\u001b[36mFile \u001b[39m\u001b[32m~/software/COSItools/cosipy_interfaces/cosipy/util/data_fetching.py:245\u001b[39m, in \u001b[36mfetch_wasabi_file\u001b[39m\u001b[34m(file, output, overwrite, unzip, unzip_output, checksum, bucket, endpoint, access_key, secret_key)\u001b[39m\n\u001b[32m 241\u001b[39m \u001b[38;5;66;03m# We'll get here only if:\u001b[39;00m\n\u001b[32m 242\u001b[39m \u001b[38;5;66;03m# - The output file doesn't exist\u001b[39;00m\n\u001b[32m 243\u001b[39m \u001b[38;5;66;03m# - The output file exists, has the wrong checksum/ETag, and overwrite = True\u001b[39;00m\n\u001b[32m 244\u001b[39m logger.info(\u001b[33mf\u001b[39m\u001b[33m\"\u001b[39m\u001b[33mDownloading \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mbucket\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m/\u001b[39m\u001b[38;5;132;01m{\u001b[39;00mfile\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m\"\u001b[39m)\n\u001b[32m--> \u001b[39m\u001b[32m245\u001b[39m \u001b[43ms3\u001b[49m\u001b[43m.\u001b[49m\u001b[43mdownload_file\u001b[49m\u001b[43m(\u001b[49m\u001b[43mBucket\u001b[49m\u001b[43m=\u001b[49m\u001b[43mbucket\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mKey\u001b[49m\u001b[43m=\u001b[49m\u001b[43mfile\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mFilename\u001b[49m\u001b[43m=\u001b[49m\u001b[43moutput\u001b[49m\u001b[43m)\u001b[49m\n",
"\u001b[36mFile \u001b[39m\u001b[32m~/software/COSItools/cosipy_interfaces/Pyenv/lib/python3.11/site-packages/botocore/context.py:123\u001b[39m, in \u001b[36mwith_current_context..decorator..wrapper\u001b[39m\u001b[34m(*args, **kwargs)\u001b[39m\n\u001b[32m 121\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m hook:\n\u001b[32m 122\u001b[39m hook()\n\u001b[32m--> \u001b[39m\u001b[32m123\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfunc\u001b[49m\u001b[43m(\u001b[49m\u001b[43m*\u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n",
"\u001b[36mFile \u001b[39m\u001b[32m~/software/COSItools/cosipy_interfaces/Pyenv/lib/python3.11/site-packages/boto3/s3/inject.py:223\u001b[39m, in \u001b[36mdownload_file\u001b[39m\u001b[34m(self, Bucket, Key, Filename, ExtraArgs, Callback, Config)\u001b[39m\n\u001b[32m 188\u001b[39m \u001b[38;5;250m\u001b[39m\u001b[33;03m\"\"\"Download an S3 object to a file.\u001b[39;00m\n\u001b[32m 189\u001b[39m \n\u001b[32m 190\u001b[39m \u001b[33;03mUsage::\u001b[39;00m\n\u001b[32m (...)\u001b[39m\u001b[32m 220\u001b[39m \u001b[33;03m transfer.\u001b[39;00m\n\u001b[32m 221\u001b[39m \u001b[33;03m\"\"\"\u001b[39;00m\n\u001b[32m 222\u001b[39m \u001b[38;5;28;01mwith\u001b[39;00m S3Transfer(\u001b[38;5;28mself\u001b[39m, Config) \u001b[38;5;28;01mas\u001b[39;00m transfer:\n\u001b[32m--> \u001b[39m\u001b[32m223\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mtransfer\u001b[49m\u001b[43m.\u001b[49m\u001b[43mdownload_file\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 224\u001b[39m \u001b[43m \u001b[49m\u001b[43mbucket\u001b[49m\u001b[43m=\u001b[49m\u001b[43mBucket\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 225\u001b[39m \u001b[43m \u001b[49m\u001b[43mkey\u001b[49m\u001b[43m=\u001b[49m\u001b[43mKey\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 226\u001b[39m \u001b[43m \u001b[49m\u001b[43mfilename\u001b[49m\u001b[43m=\u001b[49m\u001b[43mFilename\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 227\u001b[39m \u001b[43m \u001b[49m\u001b[43mextra_args\u001b[49m\u001b[43m=\u001b[49m\u001b[43mExtraArgs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 228\u001b[39m \u001b[43m \u001b[49m\u001b[43mcallback\u001b[49m\u001b[43m=\u001b[49m\u001b[43mCallback\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 229\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n",
"\u001b[36mFile \u001b[39m\u001b[32m~/software/COSItools/cosipy_interfaces/Pyenv/lib/python3.11/site-packages/boto3/s3/transfer.py:484\u001b[39m, in \u001b[36mS3Transfer.download_file\u001b[39m\u001b[34m(self, bucket, key, filename, extra_args, callback)\u001b[39m\n\u001b[32m 480\u001b[39m future = \u001b[38;5;28mself\u001b[39m._manager.download(\n\u001b[32m 481\u001b[39m bucket, key, filename, extra_args, subscribers\n\u001b[32m 482\u001b[39m )\n\u001b[32m 483\u001b[39m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[32m--> \u001b[39m\u001b[32m484\u001b[39m \u001b[43mfuture\u001b[49m\u001b[43m.\u001b[49m\u001b[43mresult\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 485\u001b[39m \u001b[38;5;66;03m# This is for backwards compatibility where when retries are\u001b[39;00m\n\u001b[32m 486\u001b[39m \u001b[38;5;66;03m# exceeded we need to throw the same error from boto3 instead of\u001b[39;00m\n\u001b[32m 487\u001b[39m \u001b[38;5;66;03m# s3transfer's built in RetriesExceededError as current users are\u001b[39;00m\n\u001b[32m 488\u001b[39m \u001b[38;5;66;03m# catching the boto3 one instead of the s3transfer exception to do\u001b[39;00m\n\u001b[32m 489\u001b[39m \u001b[38;5;66;03m# their own retries.\u001b[39;00m\n\u001b[32m 490\u001b[39m \u001b[38;5;28;01mexcept\u001b[39;00m S3TransferRetriesExceededError \u001b[38;5;28;01mas\u001b[39;00m e:\n",
"\u001b[36mFile \u001b[39m\u001b[32m~/software/COSItools/cosipy_interfaces/Pyenv/lib/python3.11/site-packages/s3transfer/futures.py:111\u001b[39m, in \u001b[36mTransferFuture.result\u001b[39m\u001b[34m(self)\u001b[39m\n\u001b[32m 106\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34mresult\u001b[39m(\u001b[38;5;28mself\u001b[39m):\n\u001b[32m 107\u001b[39m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[32m 108\u001b[39m \u001b[38;5;66;03m# Usually the result() method blocks until the transfer is done,\u001b[39;00m\n\u001b[32m 109\u001b[39m \u001b[38;5;66;03m# however if a KeyboardInterrupt is raised we want want to exit\u001b[39;00m\n\u001b[32m 110\u001b[39m \u001b[38;5;66;03m# out of this and propagate the exception.\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m111\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m_coordinator\u001b[49m\u001b[43m.\u001b[49m\u001b[43mresult\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 112\u001b[39m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mKeyboardInterrupt\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m e:\n\u001b[32m 113\u001b[39m \u001b[38;5;28mself\u001b[39m.cancel()\n",
"\u001b[36mFile \u001b[39m\u001b[32m~/software/COSItools/cosipy_interfaces/Pyenv/lib/python3.11/site-packages/s3transfer/futures.py:287\u001b[39m, in \u001b[36mTransferCoordinator.result\u001b[39m\u001b[34m(self)\u001b[39m\n\u001b[32m 284\u001b[39m \u001b[38;5;66;03m# Once done waiting, raise an exception if present or return the\u001b[39;00m\n\u001b[32m 285\u001b[39m \u001b[38;5;66;03m# final result.\u001b[39;00m\n\u001b[32m 286\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m._exception:\n\u001b[32m--> \u001b[39m\u001b[32m287\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;28mself\u001b[39m._exception\n\u001b[32m 288\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m._result\n",
"\u001b[36mFile \u001b[39m\u001b[32m~/software/COSItools/cosipy_interfaces/Pyenv/lib/python3.11/site-packages/s3transfer/tasks.py:272\u001b[39m, in \u001b[36mSubmissionTask._main\u001b[39m\u001b[34m(self, transfer_future, **kwargs)\u001b[39m\n\u001b[32m 268\u001b[39m \u001b[38;5;28mself\u001b[39m._transfer_coordinator.set_status_to_running()\n\u001b[32m 270\u001b[39m \u001b[38;5;66;03m# Call the submit method to start submitting tasks to execute the\u001b[39;00m\n\u001b[32m 271\u001b[39m \u001b[38;5;66;03m# transfer.\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m272\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m_submit\u001b[49m\u001b[43m(\u001b[49m\u001b[43mtransfer_future\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtransfer_future\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 273\u001b[39m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mBaseException\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m e:\n\u001b[32m 274\u001b[39m \u001b[38;5;66;03m# If there was an exception raised during the submission of task\u001b[39;00m\n\u001b[32m 275\u001b[39m \u001b[38;5;66;03m# there is a chance that the final task that signals if a transfer\u001b[39;00m\n\u001b[32m (...)\u001b[39m\u001b[32m 284\u001b[39m \n\u001b[32m 285\u001b[39m \u001b[38;5;66;03m# Set the exception, that caused the process to fail.\u001b[39;00m\n\u001b[32m 286\u001b[39m \u001b[38;5;28mself\u001b[39m._log_and_set_exception(e)\n",
"\u001b[36mFile \u001b[39m\u001b[32m~/software/COSItools/cosipy_interfaces/Pyenv/lib/python3.11/site-packages/s3transfer/download.py:359\u001b[39m, in \u001b[36mDownloadSubmissionTask._submit\u001b[39m\u001b[34m(self, client, config, osutil, request_executor, io_executor, transfer_future, bandwidth_limiter)\u001b[39m\n\u001b[32m 329\u001b[39m \u001b[38;5;250m\u001b[39m\u001b[33;03m\"\"\"\u001b[39;00m\n\u001b[32m 330\u001b[39m \u001b[33;03m:param client: The client associated with the transfer manager\u001b[39;00m\n\u001b[32m 331\u001b[39m \n\u001b[32m (...)\u001b[39m\u001b[32m 353\u001b[39m \u001b[33;03m downloading streams\u001b[39;00m\n\u001b[32m 354\u001b[39m \u001b[33;03m\"\"\"\u001b[39;00m\n\u001b[32m 355\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m (\n\u001b[32m 356\u001b[39m transfer_future.meta.size \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[32m 357\u001b[39m \u001b[38;5;129;01mor\u001b[39;00m transfer_future.meta.etag \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[32m 358\u001b[39m ):\n\u001b[32m--> \u001b[39m\u001b[32m359\u001b[39m response = \u001b[43mclient\u001b[49m\u001b[43m.\u001b[49m\u001b[43mhead_object\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 360\u001b[39m \u001b[43m \u001b[49m\u001b[43mBucket\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtransfer_future\u001b[49m\u001b[43m.\u001b[49m\u001b[43mmeta\u001b[49m\u001b[43m.\u001b[49m\u001b[43mcall_args\u001b[49m\u001b[43m.\u001b[49m\u001b[43mbucket\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 361\u001b[39m \u001b[43m \u001b[49m\u001b[43mKey\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtransfer_future\u001b[49m\u001b[43m.\u001b[49m\u001b[43mmeta\u001b[49m\u001b[43m.\u001b[49m\u001b[43mcall_args\u001b[49m\u001b[43m.\u001b[49m\u001b[43mkey\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 362\u001b[39m \u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mtransfer_future\u001b[49m\u001b[43m.\u001b[49m\u001b[43mmeta\u001b[49m\u001b[43m.\u001b[49m\u001b[43mcall_args\u001b[49m\u001b[43m.\u001b[49m\u001b[43mextra_args\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 363\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 364\u001b[39m \u001b[38;5;66;03m# If a size was not provided figure out the size for the\u001b[39;00m\n\u001b[32m 365\u001b[39m \u001b[38;5;66;03m# user.\u001b[39;00m\n\u001b[32m 366\u001b[39m transfer_future.meta.provide_transfer_size(\n\u001b[32m 367\u001b[39m response[\u001b[33m'\u001b[39m\u001b[33mContentLength\u001b[39m\u001b[33m'\u001b[39m]\n\u001b[32m 368\u001b[39m )\n",
"\u001b[36mFile \u001b[39m\u001b[32m~/software/COSItools/cosipy_interfaces/Pyenv/lib/python3.11/site-packages/botocore/client.py:602\u001b[39m, in \u001b[36mClientCreator._create_api_method.._api_call\u001b[39m\u001b[34m(self, *args, **kwargs)\u001b[39m\n\u001b[32m 598\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mTypeError\u001b[39;00m(\n\u001b[32m 599\u001b[39m \u001b[33mf\u001b[39m\u001b[33m\"\u001b[39m\u001b[38;5;132;01m{\u001b[39;00mpy_operation_name\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m() only accepts keyword arguments.\u001b[39m\u001b[33m\"\u001b[39m\n\u001b[32m 600\u001b[39m )\n\u001b[32m 601\u001b[39m \u001b[38;5;66;03m# The \"self\" in this scope is referring to the BaseClient.\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m602\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m_make_api_call\u001b[49m\u001b[43m(\u001b[49m\u001b[43moperation_name\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n",
"\u001b[36mFile \u001b[39m\u001b[32m~/software/COSItools/cosipy_interfaces/Pyenv/lib/python3.11/site-packages/botocore/context.py:123\u001b[39m, in \u001b[36mwith_current_context..decorator..wrapper\u001b[39m\u001b[34m(*args, **kwargs)\u001b[39m\n\u001b[32m 121\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m hook:\n\u001b[32m 122\u001b[39m hook()\n\u001b[32m--> \u001b[39m\u001b[32m123\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfunc\u001b[49m\u001b[43m(\u001b[49m\u001b[43m*\u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n",
"\u001b[36mFile \u001b[39m\u001b[32m~/software/COSItools/cosipy_interfaces/Pyenv/lib/python3.11/site-packages/botocore/client.py:1078\u001b[39m, in \u001b[36mBaseClient._make_api_call\u001b[39m\u001b[34m(self, operation_name, api_params)\u001b[39m\n\u001b[32m 1074\u001b[39m error_code = request_context.get(\n\u001b[32m 1075\u001b[39m \u001b[33m'\u001b[39m\u001b[33merror_code_override\u001b[39m\u001b[33m'\u001b[39m\n\u001b[32m 1076\u001b[39m ) \u001b[38;5;129;01mor\u001b[39;00m error_info.get(\u001b[33m\"\u001b[39m\u001b[33mCode\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m 1077\u001b[39m error_class = \u001b[38;5;28mself\u001b[39m.exceptions.from_code(error_code)\n\u001b[32m-> \u001b[39m\u001b[32m1078\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m error_class(parsed_response, operation_name)\n\u001b[32m 1079\u001b[39m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[32m 1080\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m parsed_response\n",
"\u001b[31mClientError\u001b[39m: An error occurred (404) when calling the HeadObject operation: Not Found"
]
}
],
"source": [
"#Download the extended sourde response\n",
"#Choose the response depending on your model\n",
"\n",
"#Model for a 511 keV line\n",
"#fetch_wasabi_file('COSI-SMEX/DC3/Data/Responses/extended_source_response/extended_source_response_511_merged.h5.gz', unzip=True,checksum = 'afcf551d3a8f800a268bef1490d0d48a')\n",
"\n",
"#Model for a 1157 keV line\n",
"#fetch_wasabi_file('COSI-SMEX/DC3/Data/Responses/extended_source_response/extended_source_response_Ti44_merged.h5.gz',unzip=True,checksum = '7ac6e1d9651ae33cccf94102f5356d9e')\n",
"\n",
"#Model for a 1173 keV line\n",
"#fetch_wasabi_file('COSI-SMEX/DC3/Data/Responses/extended_source_response/extended_source_response_Fe60_low_merged.h5.gz', unzip=True ,checksum = '2562f2ac0be44a60ea2d1adcea8e3b4a')\n",
"\n",
"#Model for a 1333 keV line\n",
"#fetch_wasabi_file('COSI-SMEX/DC3/Data/Responses/extended_source_response/extended_source_response_Fe60_high_merged.h5.gz', unzip=True ,checksum = '037217db2e8ca8a9a74a247e8fab9eb3')\n",
"\n",
"#Model for a 1809 keV line\n",
"fetch_wasabi_file('COSI-SMEX/DC3/Data/Responses/extended_source_response/extended_source_response_Al26_merged.h5.gz',unzip=True,checksum = '218d088159c6e11d0ddecc03ca9fa399')\n",
"\n",
"\n",
"#Model for an other line or continuum source\n",
"#fetch_wasabi_file('COSI-SMEX/DC3/Data/Responses/extended_source_response/extended_source_response_continuum_merged.h5.gz' ,unzip=True,checksum = 'a3a4a57acfbfe00378507441fa7a5891')\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8056e551-7058-48dd-b6b0-505391bd5153",
"metadata": {},
"outputs": [],
"source": [
"#Download the background model \n",
"#Choose the file depending on your model\n",
"\n",
"#Model for a 511 keV line\n",
"#fetch_wasabi_file('COSI-SMEX/DC4/Data/Backgrounds/gal_DC4_bkg_511_binned_smoothed_fwhm_10.0deg.hdf5', checksum = '1b437be9b4d9e85a230dfe4f4d2ca67e')\n",
"\n",
"#Model for a 1157 keV line\n",
"#fetch_wasabi_file('COSI-SMEX/DC4/Data/Backgrounds/gal_DC4_bkg_Ti44_binned_smoothed_fwhm_10.0deg.hdf5',checksum = 'bd62bfde46628a6e68434a6c68c7167d')\n",
"\n",
"#Model for a 1173 keV line\n",
"#fetch_wasabi_file('COSI-SMEX/DC4/Data/Backgrounds/gal_DC4_bkg_Fe60_low_binned_smoothed_fwhm_10.0deg.hdf5',checksum = '57f972b0a1c2ece4626c85ced3cc050f')\n",
"\n",
"#Model for a 1333 keV line\n",
"#fetch_wasabi_file('COSI-SMEX/DC4/Data/Backgrounds/gal_DC4_bkg_Fe60_high_binned_smoothed_fwhm_10.0deg.hdf5',checksum = 'db56a9fd9d8fc37cb03fba5e907a80c3')\n",
"\n",
"#Model for a 1809 keV line\n",
"fetch_wasabi_file('COSI-SMEX/DC4/Data/Backgrounds/gal_DC4_bkg_Al26_binned_smoothed_fwhm_10.0deg.hdf5',checksum = 'bd6a32f8abee5b989b87149111487faa')\n",
"\n",
"\n",
"#Model for an other line or continuum source\n",
"#fetch_wasabi_file('COSI-SMEX/DC4/Data/Backgrounds/gal_DC4_bkg_continuum_binned_smoothed_fwhm_10.0deg.hdf5',checksum = '1acceb619b0d7e43eea8cb29ffe74279')\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "cd9ed8aa-bb64-48cb-bce2-1845936b4faa",
"metadata": {},
"source": [
"## Define your source model\n",
"\n",
"For an extended source we need to define a spectrum shape and a spatial shape : \n",
"\n",
"### Spectrum Shape\n",
"For this example we are using a Gaussian but other functions can be used (see https://astromodels.readthedocs.io/en/latest/notebooks/function_list.html#Included-1D-Functions) \n",
"\n",
"Additionaly you can also provide your own spectrum template from a file. See Template (Table) Models in :\n",
"https://threeml.readthedocs.io/en/stable/notebooks/spectral_models.html\n",
"\n",
"Threeml expect the spectrum shape to have units of cm2/s/keV"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "20b2a4fa-3584-4918-84a5-ce38ba06b098",
"metadata": {},
"outputs": [],
"source": [
"F = 3e-5 / u.cm / u.cm / u.s \n",
"mu = 1809*u.keV\n",
"sigma = 0.3*u.keV\n",
"spectrum = Gaussian()\n",
"spectrum.F.value = F.value\n",
"spectrum.F.unit = F.unit\n",
"spectrum.mu.value = mu.value\n",
"spectrum.mu.unit = mu.unit\n",
"spectrum.sigma.value = sigma.value\n",
"spectrum.sigma.unit = sigma.unit\n",
"\n",
"# Set spectral parameters for fitting:\n",
"spectrum.F.free = True\n",
"spectrum.mu.free = False\n",
"spectrum.sigma.free = False\n",
"\n",
"#set limit for the param\n",
"spectrum.F.min_value = 0\n",
"spectrum.F.max_value = 1\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "60650681-5f3a-4f43-9edc-8d81170f90e1",
"metadata": {},
"source": [
"### Spatial shape\n",
"\n",
"For the spatial shape you need to provide a Healpix map in a fits file. The map should be normalized to the pixel area so that sum(map)*pixelarea = 1 /sr"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "67b5a1ed-1d74-4cb7-8a87-4d7ed03c0efe",
"metadata": {},
"outputs": [],
"source": [
"#For this example we will create here the spatial shape model but the fits file containing the spatial distribution can come\n",
"#from any source (ex : CLUMPY) \n",
"skymap = HealpixMap(nside = 16, scheme = \"ring\", dtype = float, coordsys='G')\n",
"skymap[:] = 1 #fill the map with 1, so no particular spatial distribution\n",
"\n",
"\n",
"#normalized the map to 1/sr\n",
"pix_area = skymap.pixarea().value\n",
"skymap[:] = skymap[:]/(np.sum(skymap)*pix_area)\n",
"\n",
"\n",
"#save the map example to a fits file\n",
"skymap.write_map(\"HealpixMap_example.fits\",overwrite=True)\n",
"\n",
"#declare the spatial shape\n",
"spatial_shape = SpatialTemplate_2D_Healpix(fits_file=\"HealpixMap_example.fits\")\n"
]
},
{
"cell_type": "markdown",
"id": "82513f39-cc7f-4883-a5cf-5f64a4e53d38",
"metadata": {},
"source": [
"## Extended source model\n",
"Now that we declare our spectral shape and spatial shape, we can create our extended source model"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "fb001ad1-ba23-4fbe-8176-0cf9fe95dc7a",
"metadata": {},
"outputs": [],
"source": [
"source = ExtendedSource('source', spectral_shape=spectrum, spatial_shape=spatial_shape)\n",
"\n",
"\n",
"model = Model(source) # Model with single source. If we had multiple sources, we would do Model(source1, source2, ...)\n"
]
},
{
"cell_type": "markdown",
"id": "d27be424-4525-476c-afc7-cb3610cbff98",
"metadata": {},
"source": [
"## Open the Response, Orientation and background files"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "5e7d1a4b-13ca-400c-8357-b6a24992c93c",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: XDG_CACHE_HOME is set to '/localscratch/.cache/sgallego', but the default location, /uni-mainz.de/homes/sgallego/.astropy/cache, already exists, and takes precedence. This environment variable will be ignored. [astropy.config.paths]\n",
"WARNING: XDG_CACHE_HOME is set to '/localscratch/.cache/sgallego', but the default location, /uni-mainz.de/homes/sgallego/.astropy/cache, already exists, and takes precedence. This environment variable will be ignored. [astropy.config.paths]\n"
]
}
],
"source": [
"#open the Response file\n",
"\n",
"#replace the background and response file by the one you want to use\n",
"extendedsourceresponse = \"extended_source_response_Al26_merged.h5\"\n",
"bkgfile = \"gal_DC4_bkg_Al26_binned_smoothed_fwhm_10.0deg.hdf5\"\n",
"sc_orientation_path = \"DC3_final_530km_3_month_with_slew_15sbins_GalacticEarth_SAA.fits\"\n",
"\n",
"#open the extended source response\n",
"dr = ExtendedSourceResponse.open(data_path / extendedsourceresponse )\n",
"\n",
"#Create the extended source injector \n",
"injector = SourceInjector(response_path=data_path / extendedsourceresponse, response_frame = \"galactic\")\n",
"\n",
"#open the background model \n",
"bg_model = Histogram.open(data_path / bkgfile)\n",
"\n",
"#open the Orientation file\n",
"sc_orientation = SpacecraftHistory.open(data_path / sc_orientation_path)"
]
},
{
"cell_type": "markdown",
"id": "dcf349e9-4640-4c11-99ed-df9431ece3e7",
"metadata": {},
"source": [
"## Compute the sensitivity\n",
"\n",
"The Likelihood (LH) fit is ran hundred times (more would be better but depending on how fast it is you could increase this number).\n",
"Each time a new signal dataset is created by convolving the model with the response and applying some poisson statistic (also called the source injector). Similarly for the background, a dataset is created by taking a poisson sample of the background model.\n",
"\n",
"For each iteration the LH ratio test is computed : \n",
"\n",
"$$\n",
"\\text{LHtest} = -2 * (L_0 - L_1)\n",
"$$\n",
"\n",
"$L_0$ being the LH value for the background only hypothesis and $L_1$ for background+signal . Assuming the Wilk's theorem is applying, the LH ratio test should follow a $\\chi^2$ distribution with dof = nb of parameters. The median value of the LH ratio test distribution should tell us if COSI is sensitive or not to this flux value.\n",
"\n",
"For our example we have only one parameter we test : the signal strength . So for a 3 $\\sigma$ sensitivity we want to have the median superior or equal to 9.\n",
"\n",
"Keep in mind we are testing only 3 months of observations but we will scale the flux later to 2 years of observation if it is already sensitive after 3 months."
]
},
{
"cell_type": "markdown",
"id": "c1340a05-538e-4c97-b096-12f3e6a26a8e",
"metadata": {},
"source": [
"⚠ !! WARNING !! ⚠ In this example we used a predifined Gaussian function from Threeml so the parameter value is `F` and is call with `spectrum.F.value` . If you changed the spectrum shape model, please change in the code below `F` with the new name of your parameter. To know it, you can use `model.display()`"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "eddea6f5-bf2a-4c47-a15e-0fc883f44744",
"metadata": {},
"outputs": [],
"source": [
"#Calculate the response once \n",
"\n",
"#create signal dataset\n",
"signal = injector.inject_model(model = model, make_spectrum_plot = False ,data_save_path = None)#,fluctuate=True)\n",
"\n",
"#create a bkg dataset from the bkg model\n",
"bkg_poissonsamp = Histogram(bg_model.axes)\n",
"bkg_poissonsamp[:] += np.random.poisson(lam = bg_model.to_dense()[:])\n",
"\n",
"#combine the data + the bck like we would get for real data\n",
"data = EmCDSBinnedData(signal.project('Em', 'Phi', 'PsiChi') + bkg_poissonsamp.project('Em', 'Phi', 'PsiChi'))\n",
"\n",
"#declare the extended source response\n",
"esr = BinnedThreeMLExtendedSourceResponse(data = data,\n",
" precomputed_psr = dr,\n",
" polarization_axis = dr.axes['Pol'] if 'Pol' in dr.axes.labels else None)\n",
"response = BinnedThreeMLModelFolding(data = data, extended_source_response = esr)\n"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "8d8cf924-67ba-4b67-b8fa-24b0f4ada9d9",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"ITERATION 0\n"
]
},
{
"data": {
"text/html": [
"13:45:49 WARNING External parameter Background already exist in the model. Overwriting it... model.py:593\n",
"\n"
],
"text/plain": [
"\u001b[38;5;46m13:45:49\u001b[0m\u001b[38;5;46m \u001b[0m\u001b[38;5;134mWARNING \u001b[0m \u001b[1;38;5;251m External parameter Background already exist in the model. Overwriting it\u001b[0m\u001b[1;33m...\u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b]8;id=476485;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/astromodels/core/model.py\u001b\\\u001b[2mmodel.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=959805;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/astromodels/core/model.py#593\u001b\\\u001b[2m593\u001b[0m\u001b]8;;\u001b\\\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"13:45:49 INFO set the minimizer to minuit joint_likelihood.py:1017\n",
"\n"
],
"text/plain": [
"\u001b[38;5;46m13:45:49\u001b[0m\u001b[38;5;46m \u001b[0m\u001b[38;5;49mINFO \u001b[0m \u001b[1;38;5;251m set the minimizer to minuit \u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b]8;id=990961;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/threeML/classicMLE/joint_likelihood.py\u001b\\\u001b[2mjoint_likelihood.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=789203;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/threeML/classicMLE/joint_likelihood.py#1017\u001b\\\u001b[2m1017\u001b[0m\u001b]8;;\u001b\\\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"ITERATION 1\n"
]
},
{
"data": {
"text/html": [
"13:46:50 WARNING External parameter Background already exist in the model. Overwriting it... model.py:593\n",
"\n"
],
"text/plain": [
"\u001b[38;5;46m13:46:50\u001b[0m\u001b[38;5;46m \u001b[0m\u001b[38;5;134mWARNING \u001b[0m \u001b[1;38;5;251m External parameter Background already exist in the model. Overwriting it\u001b[0m\u001b[1;33m...\u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b]8;id=116475;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/astromodels/core/model.py\u001b\\\u001b[2mmodel.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=582445;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/astromodels/core/model.py#593\u001b\\\u001b[2m593\u001b[0m\u001b]8;;\u001b\\\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"13:46:50 INFO set the minimizer to minuit joint_likelihood.py:1017\n",
"\n"
],
"text/plain": [
"\u001b[38;5;46m13:46:50\u001b[0m\u001b[38;5;46m \u001b[0m\u001b[38;5;49mINFO \u001b[0m \u001b[1;38;5;251m set the minimizer to minuit \u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b]8;id=745403;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/threeML/classicMLE/joint_likelihood.py\u001b\\\u001b[2mjoint_likelihood.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=94671;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/threeML/classicMLE/joint_likelihood.py#1017\u001b\\\u001b[2m1017\u001b[0m\u001b]8;;\u001b\\\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"ITERATION 2\n"
]
},
{
"data": {
"text/html": [
"13:47:52 WARNING External parameter Background already exist in the model. Overwriting it... model.py:593\n",
"\n"
],
"text/plain": [
"\u001b[38;5;46m13:47:52\u001b[0m\u001b[38;5;46m \u001b[0m\u001b[38;5;134mWARNING \u001b[0m \u001b[1;38;5;251m External parameter Background already exist in the model. Overwriting it\u001b[0m\u001b[1;33m...\u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b]8;id=135532;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/astromodels/core/model.py\u001b\\\u001b[2mmodel.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=773056;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/astromodels/core/model.py#593\u001b\\\u001b[2m593\u001b[0m\u001b]8;;\u001b\\\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"13:47:52 INFO set the minimizer to minuit joint_likelihood.py:1017\n",
"\n"
],
"text/plain": [
"\u001b[38;5;46m13:47:52\u001b[0m\u001b[38;5;46m \u001b[0m\u001b[38;5;49mINFO \u001b[0m \u001b[1;38;5;251m set the minimizer to minuit \u001b[0m\u001b[1;38;5;251m \u001b[0m\u001b]8;id=164682;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/threeML/classicMLE/joint_likelihood.py\u001b\\\u001b[2mjoint_likelihood.py\u001b[0m\u001b]8;;\u001b\\\u001b[2m:\u001b[0m\u001b]8;id=481407;file:///uni-mainz.de/homes/sgallego/software/COSItools/cosipy_DC4/Pyenv_DC4/lib/python3.13/site-packages/threeML/classicMLE/joint_likelihood.py#1017\u001b\\\u001b[2m1017\u001b[0m\u001b]8;;\u001b\\\n"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"\n",
"# We start with 100 but higher would be better\n",
"# To adjust depending on how fast the LH fit is\n",
"# You can also run a script in parallel and save the likelihood ratio test instead of using a for loop\n",
"\n",
"nbiter = 100 # put this number to 100\n",
"\n",
"LHratiotest = []\n",
"\n",
"for i in range(nbiter) :\n",
"\n",
" print(f\"ITERATION {i}\")\n",
" \n",
" #create a signal dataset with the source injector\n",
" spectrum.F.value = F.value\n",
" spectrum.F.free = True\n",
" signal = injector.inject_model(model = model, make_spectrum_plot = False ,data_save_path = None)#,fluctuate=True)\n",
"\n",
" \n",
" #create a bkg dataset from the bkg model\n",
" bkg_poissonsamp = Histogram(bg_model.axes)\n",
" bkg_poissonsamp[:] += np.random.poisson(lam = bg_model.to_dense()[:])\n",
"\n",
" # Set background parameter, which is used to fit the amplitude of the background, and instantiate the COSI 3ML plugin\n",
" # here we define only one bkg model but we could add more\n",
" bkg_dist = {\"Background\":bg_model.project('Em', 'Phi', 'PsiChi')}\n",
" \n",
" # Workaround to avoid inf values. Out bkg should be smooth, but currently it's not.\n",
" for bckfile in bkg_dist.keys() :\n",
" bkg_dist[bckfile] += sys.float_info.min\n",
"\n",
" #combine the data + the bck like we would get for real data\n",
" newdata = EmCDSBinnedData(signal.project('Em', 'Phi', 'PsiChi') \n",
" + bkg_poissonsamp.project('Em', 'Phi', 'PsiChi')\n",
" )\n",
" bkg = FreeNormBinnedBackground(bkg_dist,\n",
" sc_history=sc_orientation,\n",
" copy = True)\n",
"\n",
" #change the data of the extended source response\n",
" esr._data = newdata\n",
" \n",
"\n",
" ##====\n",
"\n",
"\n",
"\n",
" like_fun = PoissonLikelihood(newdata, response, bkg)\n",
"\n",
" cosi = ThreeMLPluginInterface('cosi',\n",
" like_fun,\n",
" response,\n",
" bkg)\n",
"\n",
" # Nuisance parameter guess, bounds, etc.\n",
" cosi.bkg_parameter['Background'] = Parameter(\"Background\", # background parameter\n",
" 1, # initial value of parameter\n",
" min_value=0, # minimum value of parameter\n",
" max_value=10, # maximum value of parameter\n",
" delta=0.05, # initial step used by fitting engine\n",
" unit = u.Hz\n",
" )\n",
" \n",
" # Set the source Model\n",
" cosi.set_model(model)\n",
"\n",
"\n",
" plugins = DataList(cosi) # If we had multiple instruments, we would do e.g. DataList(cosi, lat, hawc, ...)\n",
"\n",
" like = JointLikelihood(model, plugins, verbose = False)\n",
"\n",
" #do the fit\n",
" like.fit(compute_covariance = False,quiet=True)\n",
"\n",
" #get L1\n",
" L1 = cosi.get_log_like()\n",
"\n",
" spectrum.F.value = 0 #null hypothesis (no signal)\n",
" spectrum.F.free = False\n",
" \n",
" #do the fit again\n",
" like.fit(compute_covariance = False,quiet=True)\n",
" \n",
" #Get L0\n",
" L0 = cosi.get_log_like()\n",
" \n",
" #save LH ratio test\n",
" LHratiotest.append( -2 * (L0-L1) )\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "77f74ceb-cd5f-4c8d-8653-e40d2be0ecb0",
"metadata": {},
"source": [
"## Plot the LH ratio test\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "0a064e2e-fa2a-43d3-a49a-f0f52c818608",
"metadata": {},
"source": [
"When testing high flux value for your model, some \"islands\" in the TS distribution could appears (peaks with very high TS). This happens when a signal event lands in a bin where the estimated background was originally 0, and then filled with an arbitrary floor value. This is not a problem with the sensitivity estimation algorithm as this doesn't affect significantly the median of a TS distribution of a flux near threshold, so the estimated sensitivity is still sound. \n",
"See this issue for more informations : https://github.com/cositools/cosipy/issues/598"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "02a9f721-b864-453a-8935-32e461b09830",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"COSI is 3 sigma sensitive to 3e-05 ph/cm2/s. You can continue to the next cell\n"
]
}
],
"source": [
"if np.median(LHratiotest) <= 9 :\n",
" print(f\"COSI is not sensitive to this model with a flux of {F.value} ph/cm2/s. Please try a higher flux\")\n",
"else :\n",
" print(f\"COSI is 3 sigma sensitive to {F.value} ph/cm2/s. You can continue to the next cell\")\n",
"\n",
"fig,ax = plt.subplots()\n",
"ax.hist(LHratiotest,bins=30)\n",
"ax.axvline(np.median(LHratiotest),label=f\"median {np.median(LHratiotest):.3f}\",color=\"r\",linestyle=\"--\")\n",
"ax.legend()\n",
"ax.set_xlabel(\"LH ratio test\")\n",
"ax.set_ylabel(\"counts\")\n",
"ax.set_title(f\"Model - flux {F.value} ph/cm2/s\")\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "155e30d4-cbc2-4fc0-92bd-03d20339f1f0",
"metadata": {},
"source": [
"## Extrapolate the sensitivity to 2 years\n",
"\n",
"DC4 simulations only includes 3 months of orbit. To get the sensitivity after 2 years (nominal life time of COSI) as a first approximation we can scale the flux by $\\frac{1}{\\sqrt{T}}$ "
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "fbf1883a-1633-4986-9dcc-fd17d021e041",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"3 sigma sensitivity after 2 years of COSI observation : 1.0606601717798212e-05\n"
]
}
],
"source": [
"# 3 * 8 = 24 months = 2 years \n",
"\n",
"sensi_2y = F.value/np.sqrt(8)\n",
"\n",
"print(f\"3 sigma sensitivity after 2 years of COSI observation : {sensi_2y}\")\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.13.5"
}
},
"nbformat": 4,
"nbformat_minor": 5
}