Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(system): Added new method to save the system to a given folder. #23

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
13 changes: 13 additions & 0 deletions docs/how_tos/save_system.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# How to save the system

## Exporting the entire contents of the system

If the user wants to createa folder that containts all the information of the
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If the user wants to createa folder that containts all the information of the
If the user wants to create a folder that contains all the information of the

system `infrasys` provides a simple method `system.save("my_folder")` that
creates a folder (if it does not exists) and save all the contents of the
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
creates a folder (if it does not exists) and save all the contents of the
creates a folder (if it does not exist) and save all the contents of the

system including the `system.to_json()` and the time series arrow files and
sqlite.

To archive the system into a zip file, the user can use `system.save("my_folder",
zip=True)`. This will create a zip folder of the contents of `my_folder` and
delete the folder once the archive is done.
53 changes: 53 additions & 0 deletions src/infrasys/system.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
"""Defines a System"""

import json
import shutil
from operator import itemgetter
from collections import defaultdict
from datetime import datetime
Expand Down Expand Up @@ -284,6 +285,58 @@ def from_dict(
logger.info("Deserialized system {}", system.label)
return system

def save(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need a System.load classmethod that constructs a system from a directory or zip file? I know it's not a big deal to have the user call System.from_json. We could essentially make save/load be the main ways of operating and leave the JSON details opaque. I could do the work in a separate PR.

self,
fpath: Path | str,
filename: str = "system.json",
zip: bool = False,
overwrite: bool = False,
) -> None:
"""Save the contents of a system and the Time series in a single directory.

By default, this method creates the user specified folder using the
`to_json` method. If user sets `zip = True`, we create the folder of
the user (if it does not exists), zip it to the same location specified
and delete the folder.

Parameters
----------
fpath : Path | str
Filepath to write the contents of the system.
zip : bool
Set to True if you want to archive to a zip file.
filename: str
Name of the sytem to serialize. Default value: "system.json".
overwrite: bool
Overwrites the system if it already exist on the fpath.

Examples
--------
>>> system.save("/home/$USER/systems/my_system")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this trick some users into thinking that $USER will expand to their username?

INFO: Wrote system data to systems/system.json
INFO: Copied time series data to systems/my_system/system_time_series

>>> system.save("/home/$USER/systems/my_system", zip=True)
INFO: Wrote system data to systems/system1.json
INFO: Copied time series data to systems/system1_time_series


See Also
--------
to_json: System serialization
"""
if isinstance(fpath, str):
fpath = Path(fpath)
self.to_json(fpath / filename, overwrite=overwrite)

if zip:
logger.info("Archiving system and time series into a single file")
_ = shutil.make_archive(str(fpath), "zip", fpath)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel uncertain about this behavior because it will zip whatever else was in the directory (and then delete it). Would it be better to require that the directory not exist and use the overwrite flag to delete it? The result would be that system.save() always produces one directory with one system and nothing else.

logger.debug("Removing {}", fpath)
shutil.rmtree(fpath)

return

def add_component(self, component: Component, **kwargs) -> None:
"""Add one component to the system.

Expand Down
18 changes: 18 additions & 0 deletions tests/test_serialization.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import json
import random
import os
from datetime import datetime, timedelta

import numpy as np
Expand Down Expand Up @@ -187,3 +188,20 @@ def test_system_with_time_series_normalization(tmp_path, in_memory):
def test_json_schema():
schema = ComponentWithPintQuantity.model_json_schema()
assert isinstance(json.loads(json.dumps(schema)), dict)


def test_system_save(tmp_path, simple_system_with_time_series):
simple_system = simple_system_with_time_series
custom_folder = "my_system"
fpath = tmp_path / custom_folder
fname = "test_system"
simple_system.save(fpath, filename=fname)
assert os.path.exists(fpath), f"Folder {fpath} was not created successfully"
assert os.path.exists(fpath / fname), f"Serialized system {fname} was not created successfully"

custom_folder = "my_system_zip"
fpath = tmp_path / custom_folder
simple_system.save(fpath, filename=fname, zip=True)
assert not os.path.exists(fpath), f"Original folder {fpath} was not deleted sucessfully."
zip_fpath = f"{fpath}.zip"
assert os.path.exists(zip_fpath), f"Zip file {zip_fpath} does not exists"