Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

General Query #169

Open
ajaykhoj opened this issue May 17, 2022 · 3 comments
Open

General Query #169

ajaykhoj opened this issue May 17, 2022 · 3 comments

Comments

@ajaykhoj
Copy link

Hello

This is not an issue. I just want to know that when we prepare to analyze a source we should keep in mind what we want to do and set different parameters accordingly. Suppose I want to analyze a source and for that I have downloaded the required fits files from FSSC archive with 20 degree search radius. Now I have fixed the ROI equal to 20 degrees and want to accept all sources with TS equal to 25 and no sources with TS less than 25, lie within the ROI. Now when I ran command "enrico_sed" for unbinned likelihood analysis, I got some error while computing/creating the exposure map.

************************************************************

*** 7 gtexpmap --- Make an exposure map

************************************************************

time -p gtexpmap evfile=/home/ajay/enrico/5S0716+714_Full_roi_20_degree/S50716+714_TestofS50716+714_FRONTBACK_MkTime.fits evtable="EVENTS" scfile=/home/ajay/enrico/5S0716+714_Full_roi_20_degree/L220516023160FD19599823_SC00.fits sctable="SC_DATA" expcube=/home/ajay/enrico/5S0716+714_Full_roi_20_degree/S50716+714_TestofS50716+714_FRONTBACK_ltCube.fits outfile=/home/ajay/enrico/5S0716+714_Full_roi_20_degree/S50716+714_TestofS50716+714_FRONTBACK_ExpMap.fits irfs="P8R3_SOURCE_V3" evtype=3 srcrad=35.0 nlong=120 nlat=120 nenergies=35 submap=no nlongmin=0 nlongmax=0 nlatmin=0 nlatmax=0 chatter=2 clobber=yes debug=no gui=no mode="ql"
The exposure maps generated by this tool are meant
to be used for unbinned likelihood analysis only.
Do not use them for binned analyses.
Using evtype=3 (i.e., FRONT/BACK irfs)
Caught St13runtime_error at the top level: RoiCuts::sortCuts:
[Message] Warning: An error ocurred, retrying ...
time -p gtexpmap evfile=/home/ajay/enrico/5S0716+714_Full_roi_20_degree/S50716+714_TestofS50716+714_FRONTBACK_MkTime.fits evtable="EVENTS" scfile=/home/ajay/enrico/5S0716+714_Full_roi_20_degree/L220516023160FD19599823_SC00.fits sctable="SC_DATA" expcube=/home/ajay/enrico/5S0716+714_Full_roi_20_degree/S50716+714_TestofS50716+714_FRONTBACK_ltCube.fits outfile=/home/ajay/enrico/5S0716+714_Full_roi_20_degree/S50716+714_TestofS50716+714_FRONTBACK_ExpMap.fits irfs="P8R3_SOURCE_V3" evtype=3 srcrad=35.0 nlong=120 nlat=120 nenergies=35 submap=no nlongmin=0 nlongmax=0 nlatmin=0 nlatmax=0 chatter=2 clobber=yes debug=no gui=no mode="ql"
The exposure maps generated by this tool are meant
to be used for unbinned likelihood analysis only.
Do not use them for binned analyses.
Using evtype=3 (i.e., FRONT/BACK irfs)
Caught St13runtime_error at the top level: RoiCuts::sortCuts:
[Message] Warning: An error ocurred, retrying ...
time -p gtexpmap evfile=/home/ajay/enrico/5S0716+714_Full_roi_20_degree/S50716+714_TestofS50716+714_FRONTBACK_MkTime.fits evtable="EVENTS" scfile=/home/ajay/enrico/5S0716+714_Full_roi_20_degree/L220516023160FD19599823_SC00.fits sctable="SC_DATA" expcube=/home/ajay/enrico/5S0716+714_Full_roi_20_degree/S50716+714_TestofS50716+714_FRONTBACK_ltCube.fits outfile=/home/ajay/enrico/5S0716+714_Full_roi_20_degree/S50716+714_TestofS50716+714_FRONTBACK_ExpMap.fits irfs="P8R3_SOURCE_V3" evtype=3 srcrad=35.0 nlong=120 nlat=120 nenergies=35 submap=no nlongmin=0 nlongmax=0 nlatmin=0 nlatmax=0 chatter=2 clobber=yes debug=no gui=no mode="ql"
The exposure maps generated by this tool are meant
to be used for unbinned likelihood analysis only.
Do not use them for binned analyses.
Using evtype=3 (i.e., FRONT/BACK irfs)
Caught St13runtime_error at the top level: RoiCuts::sortCuts:
[Message] Warning: An error ocurred, retrying ...
time -p gtexpmap evfile=/home/ajay/enrico/5S0716+714_Full_roi_20_degree/S50716+714_TestofS50716+714_FRONTBACK_MkTime.fits evtable="EVENTS" scfile=/home/ajay/enrico/5S0716+714_Full_roi_20_degree/L220516023160FD19599823_SC00.fits sctable="SC_DATA" expcube=/home/ajay/enrico/5S0716+714_Full_roi_20_degree/S50716+714_TestofS50716+714_FRONTBACK_ltCube.fits outfile=/home/ajay/enrico/5S0716+714_Full_roi_20_degree/S50716+714_TestofS50716+714_FRONTBACK_ExpMap.fits irfs="P8R3_SOURCE_V3" evtype=3 srcrad=35.0 nlong=120 nlat=120 nenergies=35 submap=no nlongmin=0 nlongmax=0 nlatmin=0 nlatmax=0 chatter=2 clobber=yes debug=no gui=no mode="ql"
The exposure maps generated by this tool are meant
to be used for unbinned likelihood analysis only.
Do not use them for binned analyses.
Using evtype=3 (i.e., FRONT/BACK irfs)
Caught St13runtime_error at the top level: RoiCuts::sortCuts:
[Message] Warning: An error ocurred, retrying ...
time -p gtexpmap evfile=/home/ajay/enrico/5S0716+714_Full_roi_20_degree/S50716+714_TestofS50716+714_FRONTBACK_MkTime.fits evtable="EVENTS" scfile=/home/ajay/enrico/5S0716+714_Full_roi_20_degree/L220516023160FD19599823_SC00.fits sctable="SC_DATA" expcube=/home/ajay/enrico/5S0716+714_Full_roi_20_degree/S50716+714_TestofS50716+714_FRONTBACK_ltCube.fits outfile=/home/ajay/enrico/5S0716+714_Full_roi_20_degree/S50716+714_TestofS50716+714_FRONTBACK_ExpMap.fits irfs="P8R3_SOURCE_V3" evtype=3 srcrad=35.0 nlong=120 nlat=120 nenergies=35 submap=no nlongmin=0 nlongmax=0 nlatmin=0 nlatmax=0 chatter=2 clobber=yes debug=no gui=no mode="ql"
The exposure maps generated by this tool are meant
to be used for unbinned likelihood analysis only.
Do not use them for binned analyses.
Using evtype=3 (i.e., FRONT/BACK irfs)
Caught St13runtime_error at the top level: RoiCuts::sortCuts:
[Message] Warning: An error ocurred, retrying ...

*** Error [Message]: An error ocurred and could not be recovered. Exiting! ***

==================================================================================================
Now I need your help to solve this.
And can you also explain , when i want to set ROI =20 degree, what should be the value of parameters in Model section of config file and should i set "roicut=yes" in analysis section of config file to eliminate this error.

can you explain what does mean of "max_radius" in model section of .conf file ?

.conf file is :-

Default config and validation file for Enrico.

Besides providing default options it is also used

to check the user config file.

Options without default options here must be

specified in the user config file

Folder where the output will be stored

out = /home/ajay/enrico/5S0716+714_Full_roi_20_degree

verbosity

verbose = yes
#Overwrite existing output files. Use with caution
clobber = yes
#Submit the job to a cluster?
Submit = no

[target]
name = S50716+714
ra = 110.473
dec = 71.3434
redshift = 0.0
spectrum = PowerLaw2
ebl_model = 4
# 0=Kneiske, 1=Primack05, 2=Kneiske_HighUV, 3=Stecker05, 4=Franceschini, 5=Finke, 6=Gilmore, 10=Dominguez11
# full list http://www-glast.stanford.edu/cgi-bin/viewcvs/celestialSources/eblAtten/eblAtten/EblAtten.h?view=markup
fit_tau = no

[space]
xref = 110.473
yref = 71.3434
rad = 20.0
binsz = 0.1
coordsys = CEL
proj = AIT
phibins = 0

[file]
spacecraft = /home/ajay/enrico/5S0716+714_Full_roi_20_degree/L220516023160FD19599823_SC00.fits
event = /home/ajay/enrico/5S0716+714_Full_roi_20_degree/data.list
xml = /home/ajay/enrico/5S0716+714_Full_roi_20_degree/S50716+714_PowerLaw2_model.xml
tag = TestofS50716+714
# keep track of the parent config during nested operations (e.g. energy/time bins)
parent_config = ""

[time]
tmin = 239557417.0
tmax = 271093418.0
file = ""
type = MET

[energy]
emin = 100.0
emax = 300000.0
enumbins_per_decade = 10
decorrelation_energy = no

[event]
irfs = CALDB
evclass = 128
evtype = 3

[analysis]
zmax = 90
# General analysis options
likelihood = unbinned
# Correct energy dispersion?
EnergyDispersion = yes
ComputeDiffrsp = yes
roicut = no
# cuts applied during the first coarse bin selection:
evtroicuts = yes
evttimecuts = yes
filter = (DATA_QUAL>0)&&(LAT_CONFIG==1)
# should we keep the srcmaps for all sources in the srcMap file?
# this should speed up future fits, at the cost of disk space.
keep_all_srcmaps = no

[environ]
# Analysis environment configuration
# Can also be done via shell environment variables
FERMI_DATA_DIR = ""
FERMI_CATALOG_DIR = ""
FERMI_CATALOG = ""
FERMI_DIFFUSE_DIR = ""
FERMI_PREPROCESSED_DIR = ""

[fitting]
optimizer = NEWMINUIT
ftol = 1e-06
# if source_ts < min_source_TS in the fit, the source is removed from the model
min_source_TS = 1.0

[model]
# The following options determine the xml model
diffuse_gal_dir = ""
diffuse_iso_dir = ""
diffuse_gal = ""
diffuse_iso = ""
# user points sources for diffuse catalog sources
# freeze spectral parameters for weak and far away sources:
min_significance_catalog = 4.0
min_significance_free = 25.0
max_radius = 5.0
# Add all source in a radius of analyse ROI + this radius
max_roi = 20.0
# Add gaussian noise to the spectral parameters of free sources (may help during the fit)
parameters_noise = 0.0

[Spectrum]
#Generates fits files or not?
FitsGeneration = yes
#Generates plots (SED, model map)
ResultPlots = yes
#Plot (update) the parent config related plots during energy bin production
ResultParentPlots = no
#Freeze the spectral index of the source. Has no implication if 0 (Left free to vary)
FrozenSpectralIndex = 0.0
#Write covariance matrix to csv file
WriteCovMatrix = yes
#Write residuals to csv file
WriteResiduals = yes

[UpperLimit]
#Assumed Spectral index
SpectralIndex = 2.0
# UL method could be Profile or Integral (provided by the fermi collaboration).
#Poisson is base on the Feldman-Cousins method for low signal. Only 95 % yet available
Method = Profile
envelope = no
#Compute an UL if the TS of the sources is <TSlimit
TSlimit = 25.0
# Confidence level for the Ul computation
cl = 0.95

[LightCurve]
#Generates fits files or not?
FitsGeneration = yes
#Number of points for the LC, bin size is 7 days
NLCbin = 53
#Index for the power law. Left free to vary if 0
SpectralIndex = 2.0
MakeConfFile = yes
#Bayesian blocks
BayesianBlocks = no
#Compute Variability index as in the 2FGL.
ComputeVarIndex = yes
#Compute an UL if the TS of the sources is <TSLightCurve
TSLightCurve = 25.0
#Generates control plots
DiagnosticPlots = yes
# Plot (update) the parent config related plots during time bin production
DiagnosticParentPlots = no

[BayesianBlocks]
#Generates fits files or not?
FitsGeneration = yes
#False alarm probability
p0 = 0.01
#Index for the power law. Left free to vary if 0
SpectralIndex = 2.0
MakeConfFile = yes
#Compute an UL if the TS of the sources is <TSLightCurve
TSLightCurve = 25.0
#Generates control plots
DiagnosticPlots = yes
# Plot (update) the parent config related plots during time bin production
DiagnosticParentPlots = no

[FoldedLC]
#Number of bins for the orbitally folded LC
NLCbin = 10
#Epoch of phase=0 in MJD, equal to tmin is 0
epoch = 0.0
#Orbital period in days
Period = 0.0

[AppLC]
#Generates fits files or not?
FitsGeneration = yes
#Spectral index for the exposure calculation
index = 2.0
#Apperture radius
rad = 1.0
#Number of bins
NLCbin = 10
#bin form data or frozen bin size
binsFromData = no
#minimum energy for the apperture lightcurve, if -1 use the one define in energy section
emin = -1.0
emax = -1.0

[Ebin]
#Generates fits files or not?
FitsGeneration = yes
NumEnergyBins = 15
#Compute an UL if the TS of the sources is <TSEnergyBins
TSEnergyBins = 9.0
# How to generate the Ebins
# - logE: log scaled bins
# - TS: similar TS per bin
# - mix: half-way between log and TS.
# - [ E1, E2, E3, E4, ..., EN ]: custom binning.
# - default is logE
DistEbins = logE

[TSMap]
#Re-fit before computing the TS map
Re-Fit = no
#Numbers of pixel in x and y
npix = 10
#Remove or not the target from the model
RemoveTarget = yes
#Generate the TS map pixel by pixel or by grouping the pixels by row.
#(reduce the numbers of jobs but each job is longer)
method = row

[findsrc]
#Generates fits files or not?
FitsGeneration = yes
#Reoptimize before
Refit = yes

[srcprob]
#Generates fits files or not?
FitsGeneration = yes
#radius for the computation
rad = 1.0
# list of sources
srclist = ""
# number of photons to print
numberPhoton = 10

[Contours]
parname1 = Prefactor
parname2 = Index

[ComponentAnalysis]
# EUnBinned to the energy (MeV) where
# binned and unbinned analysis should be split
# disabled if -1
EUnBinned = -1.0
# Divide the analysis in FrontBack classes
FrontBack = no
# Divide the analysis in PSF classes
PSF = no
# Divide the analysis in EDISP classes
EDISP = no
# Divide the analysis in EDISP classes
FGL4 = no

[ObservationSimulation]
infile = Simmodel.xml
srclist = source_names.txt

@ajaykhoj
Copy link
Author

Hi,

I tried to solve this problem, i couldn't. First i downloaded the data with 30 degree radius and then choose 20 degree as region of interest (ROI). Then i ran the programme(enrico_sed ...) , but it killed in the middle, i don't know, why it has been stopped. C an someone help? Any help is much appreciated. Thanks

what i did, i'm mentioning here
-> Data(with 30 degree radius)
-> Choose ROI 20 degree (set "rad=20" in Space section of .conf file )
-> what should be the value of "min_significance_catalog, min_significance_free, max_radius, max_roi " ? in model section
[model]
# The following options determine the xml model
diffuse_gal_dir = ""
diffuse_iso_dir = ""
diffuse_gal = ""
diffuse_iso = ""
# user points sources for diffuse catalog sources
# freeze spectral parameters for weak and far away sources:
min_significance_catalog = 4.0
min_significance_free = 25.0
max_radius = 5.0
# Add all source in a radius of analyse ROI + this radius
max_roi = 10.0
# Add gaussian noise to the spectral parameters of free sources (may help during the fit)
parameters_noise = 0.0

@ajaykhoj
Copy link
Author

Hello,

I am getting an error while doing unbinned likelihood analysis.

time -p gtmodel srcmaps=/home/ajay/enrico/PKS0346-27/PKS0346-27_TestofPKS_FRONTBACK_PowerLaw2_srcMap.fits srcmdl=/home/ajay/enrico/PKS0346-27/PKS0346-27_PowerLaw2_TestofPKS_out.xml outfile=/home/ajay/enrico/PKS0346-27/PKS0346-27_TestofPKS_FRONTBACK_PowerLaw2_ModelMap.fits irfs="P8R3_SOURCE_V3" evtype="INDEF" expcube=/home/ajay/enrico/PKS0346-27/PKS0346-27_TestofPKS_FRONTBACK_ltCube.fits bexpmap=/home/ajay/enrico/PKS0346-27/PKS0346-27_TestofPKS_FRONTBACK_BinnedMap.fits convol=yes resample=yes rfactor=2 outtype="CMAP" psfcorr=yes phased_expmap=none edisp_bins=0 chatter=2 clobber=yes debug=no gui=no mode="ql"
Parameter srcmaps: file "/home/ajay/enrico/PKS0346-27/PKS0346-27_TestofPKS_FRONTBACK_PowerLaw2_srcMap.fits" is not accessible in mode r.
Caught N5hoops12ApeExceptionE at the top level: Exception while querying for file name parameter srcmaps for component gtmodel; Ape exception code 43 (at /usr/share/miniconda/conda-bld/fermitools_1611158415999/work/hoops/build/debianstretch/sid-x86_64-64bit-gcc75-Optimized/src/hoops_ape.cxx: 565)
[Message] Warning: An error ocurred, retrying ...
time -p gtmodel srcmaps=/home/ajay/enrico/PKS0346-27/PKS0346-27_TestofPKS_FRONTBACK_PowerLaw2_srcMap.fits srcmdl=/home/ajay/enrico/PKS0346-27/PKS0346-27_PowerLaw2_TestofPKS_out.xml outfile=/home/ajay/enrico/PKS0346-27/PKS0346-27_TestofPKS_FRONTBACK_PowerLaw2_ModelMap.fits irfs="P8R3_SOURCE_V3" evtype="INDEF" expcube=/home/ajay/enrico/PKS0346-27/PKS0346-27_TestofPKS_FRONTBACK_ltCube.fits bexpmap=/home/ajay/enrico/PKS0346-27/PKS0346-27_TestofPKS_FRONTBACK_BinnedMap.fits convol=yes resample=yes rfactor=2 outtype="CMAP" psfcorr=yes phased_expmap=none edisp_bins=0 chatter=2 clobber=yes debug=no gui=no mode="ql"
Parameter srcmaps: file "/home/ajay/enrico/PKS0346-27/PKS0346-27_TestofPKS_FRONTBACK_PowerLaw2_srcMap.fits" is not accessible in mode r.
Caught N5hoops12ApeExceptionE at the top level: Exception while querying for file name parameter srcmaps for component gtmodel; Ape exception code 43 (at /usr/share/miniconda/conda-bld/fermitools_1611158415999/work/hoops/build/debianstretch/sid-x86_64-64bit-gcc75-Optimized/src/hoops_ape.cxx: 565)
[Message] Warning: An error ocurred, retrying ...
time -p gtmodel srcmaps=/home/ajay/enrico/PKS0346-27/PKS0346-27_TestofPKS_FRONTBACK_PowerLaw2_srcMap.fits srcmdl=/home/ajay/enrico/PKS0346-27/PKS0346-27_PowerLaw2_TestofPKS_out.xml outfile=/home/ajay/enrico/PKS0346-27/PKS0346-27_TestofPKS_FRONTBACK_PowerLaw2_ModelMap.fits irfs="P8R3_SOURCE_V3" evtype="INDEF" expcube=/home/ajay/enrico/PKS0346-27/PKS0346-27_TestofPKS_FRONTBACK_ltCube.fits bexpmap=/home/ajay/enrico/PKS0346-27/PKS0346-27_TestofPKS_FRONTBACK_BinnedMap.fits convol=yes resample=yes rfactor=2 outtype="CMAP" psfcorr=yes phased_expmap=none edisp_bins=0 chatter=2 clobber=yes debug=no gui=no mode="ql"
Parameter srcmaps: file "/home/ajay/enrico/PKS0346-27/PKS0346-27_TestofPKS_FRONTBACK_PowerLaw2_srcMap.fits" is not accessible in mode r.
Caught N5hoops12ApeExceptionE at the top level: Exception while querying for file name parameter srcmaps for component gtmodel; Ape exception code 43 (at /usr/share/miniconda/conda-bld/fermitools_1611158415999/work/hoops/build/debianstretch/sid-x86_64-64bit-gcc75-Optimized/src/hoops_ape.cxx: 565)
[Message] Warning: An error ocurred, retrying ...
time -p gtmodel srcmaps=/home/ajay/enrico/PKS0346-27/PKS0346-27_TestofPKS_FRONTBACK_PowerLaw2_srcMap.fits srcmdl=/home/ajay/enrico/PKS0346-27/PKS0346-27_PowerLaw2_TestofPKS_out.xml outfile=/home/ajay/enrico/PKS0346-27/PKS0346-27_TestofPKS_FRONTBACK_PowerLaw2_ModelMap.fits irfs="P8R3_SOURCE_V3" evtype="INDEF" expcube=/home/ajay/enrico/PKS0346-27/PKS0346-27_TestofPKS_FRONTBACK_ltCube.fits bexpmap=/home/ajay/enrico/PKS0346-27/PKS0346-27_TestofPKS_FRONTBACK_BinnedMap.fits convol=yes resample=yes rfactor=2 outtype="CMAP" psfcorr=yes phased_expmap=none edisp_bins=0 chatter=2 clobber=yes debug=no gui=no mode="ql"
Parameter srcmaps: file "/home/ajay/enrico/PKS0346-27/PKS0346-27_TestofPKS_FRONTBACK_PowerLaw2_srcMap.fits" is not accessible in mode r.
Caught N5hoops12ApeExceptionE at the top level: Exception while querying for file name parameter srcmaps for component gtmodel; Ape exception code 43 (at /usr/share/miniconda/conda-bld/fermitools_1611158415999/work/hoops/build/debianstretch/sid-x86_64-64bit-gcc75-Optimized/src/hoops_ape.cxx: 565)
[Message] Warning: An error ocurred, retrying ...
time -p gtmodel srcmaps=/home/ajay/enrico/PKS0346-27/PKS0346-27_TestofPKS_FRONTBACK_PowerLaw2_srcMap.fits srcmdl=/home/ajay/enrico/PKS0346-27/PKS0346-27_PowerLaw2_TestofPKS_out.xml outfile=/home/ajay/enrico/PKS0346-27/PKS0346-27_TestofPKS_FRONTBACK_PowerLaw2_ModelMap.fits irfs="P8R3_SOURCE_V3" evtype="INDEF" expcube=/home/ajay/enrico/PKS0346-27/PKS0346-27_TestofPKS_FRONTBACK_ltCube.fits bexpmap=/home/ajay/enrico/PKS0346-27/PKS0346-27_TestofPKS_FRONTBACK_BinnedMap.fits convol=yes resample=yes rfactor=2 outtype="CMAP" psfcorr=yes phased_expmap=none edisp_bins=0 chatter=2 clobber=yes debug=no gui=no mode="ql"
Parameter srcmaps: file "/home/ajay/enrico/PKS0346-27/PKS0346-27_TestofPKS_FRONTBACK_PowerLaw2_srcMap.fits" is not accessible in mode r.
Caught N5hoops12ApeExceptionE at the top level: Exception while querying for file name parameter srcmaps for component gtmodel; Ape exception code 43 (at /usr/share/miniconda/conda-bld/fermitools_1611158415999/work/hoops/build/debianstretch/sid-x86_64-64bit-gcc75-Optimized/src/hoops_ape.cxx: 565)
[Message] Warning: An error ocurred, retrying ...

*** Error [Message]: An error ocurred and could not be recovered. Exiting! ***

(fermi2.0.8) ajay@ajay:~/enrico/PKS0346-27$ enrico_sed PKS.conf

any help would be much appreciated.

@mireianievas
Copy link
Contributor

Does this exist? /home/ajay/enrico/PKS0346-27/PKS0346-27_TestofPKS_FRONTBACK_PowerLaw2_srcMap.fits ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants