Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add write function to FitResult #5258

Merged
merged 5 commits into from May 14, 2024
Merged

Conversation

Astro-Kirsty
Copy link
Member

@Astro-Kirsty Astro-Kirsty commented May 10, 2024

This PR is to include a write function in the FitResult and partly resolves #5154

The following is implemented
#5255 (comment)

Not sure why 'total_stat' is not saving as a value but instead like this

Copy link

codecov bot commented May 10, 2024

Codecov Report

Attention: Patch coverage is 35.00000% with 13 lines in your changes are missing coverage. Please review.

Project coverage is 75.25%. Comparing base (67efb7e) to head (9089db5).
Report is 1 commits behind head on main.

❗ Current head 9089db5 differs from pull request most recent head ce847b9. Consider uploading reports for the commit ce847b9 to get more accurate results

Files Patch % Lines
gammapy/modeling/fit.py 35.00% 13 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5258      +/-   ##
==========================================
- Coverage   75.27%   75.25%   -0.03%     
==========================================
  Files         234      234              
  Lines       35148    35168      +20     
==========================================
+ Hits        26458    26465       +7     
- Misses       8690     8703      +13     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Astro-Kirsty Astro-Kirsty marked this pull request as ready for review May 10, 2024 15:26
Copy link
Contributor

@QRemy QRemy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Astro-Kirsty, could you add a test for that ?
How do you produce the bad outpout for total_stat ?
I tried with the analysis_2 toturial and the output seems fine.

@QRemy QRemy added the feature label May 14, 2024
@QRemy QRemy added this to In progress in gammapy.modeling via automation May 14, 2024
@QRemy QRemy added this to the 1.3 milestone May 14, 2024
@QRemy QRemy requested a review from bkhelifi May 14, 2024 10:12
bkhelifi
bkhelifi previously approved these changes May 14, 2024
Copy link
Member

@bkhelifi bkhelifi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have some minor comments. This PR is globally ready to be merged! Thanks you a lot

gammapy/modeling/fit.py Show resolved Hide resolved
gammapy/modeling/fit.py Show resolved Hide resolved
@bkhelifi
Copy link
Member

Just as a comment, some information stored as data are in fact metadata, e.g.

            "optimize_result": {
                "backend": self.optimize_result.backend,
                "method": self.optimize_result.method,

I propose to fix that later on a separated PR about metadata handling.

@Astro-Kirsty
Copy link
Member Author

Thanks @Astro-Kirsty, could you add a test for that ? How do you produce the bad outpout for total_stat ? I tried with the analysis_2 toturial and the output seems fine.

Using this code snippet

The print is fine, but then the output in the yaml file is not

Signed-off-by: Astro-Kirsty <AstroKirsty@gmail.com>
Signed-off-by: Astro-Kirsty <AstroKirsty@gmail.com>
Signed-off-by: Astro-Kirsty <AstroKirsty@gmail.com>
gammapy/modeling/fit.py Outdated Show resolved Hide resolved
QRemy and others added 2 commits May 14, 2024 14:41
Signed-off-by: Quentin Remy <quentin.remy@mpi-hd.mpg.de>
Copy link
Contributor

@QRemy QRemy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Astro-Kirsty, the yaml file written by the test seems ok.

@QRemy QRemy merged commit fe0f3cf into gammapy:main May 14, 2024
14 of 15 checks passed
gammapy.modeling automation moved this from In progress to Done May 14, 2024
Copy link
Contributor

@registerrier registerrier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thnaks @Astro-Kirsty . My review is a bit late.
Some general comments:

  • What about a read method? The point of the write method is to be able to retrieve the result later.
  • I think we would need a more generic scheme to be able to handle different steps. I would suggest to implement a to_dict() method on the FitStepResult.
  • There is code duplication between this write and to_yaml methods and those implemented on Models.write and Datasets.write. Maybe we should improve gammapy.utils.scripts.write_yaml to support all these cases.

full_output=full_output, overwrite_templates=overwrite_templates
)

return {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe start with optimize_results then covariance_result and finally models.


return {
"models": models_dict,
"covariance_result": {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if covariance has not been run?

@registerrier
Copy link
Contributor

Just as a comment, some information stored as data are in fact metadata, e.g.

            "optimize_result": {
                "backend": self.optimize_result.backend,
                "method": self.optimize_result.method,

I propose to fix that later on a separated PR about metadata handling.

FitResult is not meant to be a data product. I don't think we should try to have a scheme of data vs metadata here.

@QRemy
Copy link
Contributor

QRemy commented May 14, 2024

  • What about a read method? The point of the write method is to be able to retrieve the result later.

@registerrier Do we really need a read method for objects that are only used for diagnostics and not configuration ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Development

Successfully merging this pull request may close these issues.

Exposing FitResult ?
4 participants