Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Acceptance is not properly propagated during MapDatasetOnOff stacking #5245

Open
registerrier opened this issue May 3, 2024 · 1 comment · May be fixed by #5270
Open

Acceptance is not properly propagated during MapDatasetOnOff stacking #5245

registerrier opened this issue May 3, 2024 · 1 comment · May be fixed by #5270
Assignees
Labels
Milestone

Comments

@registerrier
Copy link
Contributor

Gammapy version
v1.0 and beyond

Bug description
The ring background analysis that is demonstrated in the corresponding tutorial is incorrect. The OFF is correlated twice (one with the RingBackgroundMaker and the other with the ExcessMapEstimator because the latter use incorrectly the default value correlate_off=True. The estimated significance is therefore artificially large.

Wile using instead correlate_off=False removes the double OFF counting, it creates residual features where the acceptance is discontinuous (mostly because of FoV masking).

Expected behavior
ExcessMapEstimator should provide images without artifacts using the proper correlate_off=False and the tutorial should be updated accordingly.

To Reproduce
Modify the ring background tutorial to include an offset-max cut and add the correct option to compute Li-Ma excess and significance map correlate=False.

# Cutout size (for the run-wise event selection)
config.datasets.geom.selection.offset_max = 3.0 * u.deg
config.datasets.safe_mask.parameters = {"offset_max" : 1.5*u.deg}
config.datasets.safe_mask.methods = ["offset-max"]

You will obtain this figure:
download-5

Other information
Any other information you think will be useful for us to fix the issue can go here.

@registerrier registerrier added this to the 1.2.1 milestone May 3, 2024
@registerrier
Copy link
Contributor Author

registerrier commented May 3, 2024

The issue most likely comes from the way we compute stacked acceptances.

Reminder, for excess calculation with ON-OFF data:

  • for a single dataset we have $N_{excess}$ and $\alpha$:
    $$N_{excess} = N_{ON} - \alpha N_{OFF} $$
  • with $\alpha = \frac{Acc_{ON}}{Acc_{OFF}}$ the ratio of ON and OFF acceptances
  • to guarantee correct number of excess when stacking one has to compute an $N_{OFF}$-weighted $\alpha$, such that:

$$ \sum_i N_{excess,i} = \sum_i N_{ON,i} - \bar{\alpha} \sum_i{N_{OFF,i}}$$

  • with:
    $$\bar{\alpha} = \frac{\sum_i \alpha_i N_{OFF,i}}{\sum_i N_{OFF,i}}$$

Usage in Gammapy

  • This principle is applied :
    • when stacking ON-OFF datasets
    • when computing the correlated maps in ExcessMapEstimator

A caveat

  • this does not say what to do with the ON and OFF acceptances, in Gammapy we assume that the stacked acceptances are:
    $$\bar{A}cc_{ON} = 1 \quad \text and \quad \bar{A}cc_{OFF} = \frac{1}{\bar{\alpha}}$$
  • This choice is problematic as it removes all information on where the events actually come from
  • We should instead stack $Acc$ to represent the sum of acceptances taking into account the safe mask:

$$ \bar{A}cc_{ON} = \sum_i Acc_{i} \times \epsilon_i $$

  • and
    $$\bar{A}cc_{OFF} = \sum_i Acc_{i}\ \epsilon_i \times \frac{\sum_i N_{OFF,i}}{\sum_i \alpha_i N_{OFF,i}}$$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant