Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi recording archive #1

Draft
wants to merge 29 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
208d74b
add support for multiple recordings in archives
jhazentia Apr 27, 2023
44462f1
fix SigMFArchiveReader error
jhazentia May 1, 2023
832b731
support single or multiple sigmffiles in archive __init__()
jhazentia May 3, 2023
8d25adf
renamed archive "name" to "path", allow os.PathLike
jhazentia May 3, 2023
4f58453
Fixed bug in checking sigmffiles type
jhazentia May 3, 2023
89242c8
add test for missing name
jhazentia May 3, 2023
0c503ab
require name in SigMFFile constructor
jhazentia May 5, 2023
d234ddf
return single or list of SigMFFiles in fromarchive
jhazentia May 5, 2023
348bed8
fix some formatting, unused imports, docstrings, rename archivereader…
jhazentia May 8, 2023
b6df262
add support for collections in archives, check for path and fileobj i…
jhazentia May 11, 2023
4cfc8c2
rename collectionfile to collection
jhazentia May 12, 2023
ea4e633
make json end of file new line consistent, add support for collection…
jhazentia May 12, 2023
68c6825
add README examples for archives with multiple recordings
jhazentia May 12, 2023
454dd34
fix archive docstring, remove unneeded variables from archivereader
jhazentia May 15, 2023
af9002d
simplify SigMFCollection archive tests
jhazentia May 15, 2023
f1d108b
organize SigMFFile constructor doc string
jhazentia May 15, 2023
a631eb3
clarify different ways to do the same thing in README
jhazentia May 26, 2023
74a7b86
fix typo
jhazentia May 26, 2023
ae4c424
Merge branch 'main' of https://github.com/NTIA/sigmf-python into mult…
jhazentia May 26, 2023
93ab02b
add support for passing SigMFFile objects to SigMFCollection to impro…
jhazentia May 30, 2023
5376ece
fix SigMFCollection docstring
jhazentia Jun 1, 2023
46e7d8f
SigMFCollection set_streams() will check type for each element of met…
jhazentia Jun 1, 2023
660ba82
break up and simplify archive examples in README
jhazentia Jun 1, 2023
e2919d8
fix docstring, add ability to control pretty print JSON for archive
jhazentia Jun 1, 2023
e4e1775
update docstrings, formatting
jhazentia Jun 2, 2023
3131683
improve docstrings, remove duplicative test, add test for fromarchive…
jhazentia Jun 2, 2023
29827af
fix error message
jhazentia Jun 5, 2023
b81289b
make archives work when using folders
jhazentia Jun 6, 2023
15ca451
folders in archives are no longer created by default to maintain cons…
jhazentia Jun 8, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
113 changes: 113 additions & 0 deletions README.md
Expand Up @@ -47,6 +47,32 @@ handle.get_captures() # returns list of 'captures' dictionaries
handle.get_annotations() # returns list of all annotations
```

### Load a SigMF archive with multiple recordings
There are different ways to read an archive using `SigMFArchiveReader`
class, the `sigmffile.fromarchive()` method, and the `sigmffile.fromfile()`
method.

```python
import numpy as np
from sigmf.archivereader import SigMFArchiveReader

from sigmf.sigmffile import (fromarchive,
fromfile)
# read multirecording archives using fromarchive
sigmffiles = fromarchive("multi_recording_archive1.sigmf")
print(len(sigmffiles))

# read multirecording archives using fromfile
sigmffiles = fromfile("multi_recording_archive1.sigmf")
print(len(sigmffiles))

# read multirecording archives using archive reader with collection
reader = SigMFArchiveReader("multi_recording_archive2.sigmf")
print(len(reader))
print(reader.collection)
print(len(reader.collection.sigmffiles)) # get SigMFFiles from collection
```

### Verify SigMF dataset integrity & compliance

```bash
Expand Down Expand Up @@ -180,6 +206,93 @@ ci16_sigmffile = collection.get_SigMFFile(stream_name='example_ci16')
cf32_sigmffile = collection.get_SigMFFile(stream_name='example_cf32')
```

### Create a SigMF Archive
The `SigMFArchive` class, the `SigMFFile.archive()` method, and the
`SigMFFile.tofile()` method can all be used to create an archive.

```python
import numpy as np

from sigmf.sigmffile import (SigMFFile,
SigMFArchive)


# create data file
random_data1 = np.random.rand(128)
data1_path = "recording1.sigmf-data"
random_data1.tofile(data1_path)

# create metadata
sigmf_file_1 = SigMFFile(name='recording1')
sigmf_file_1.set_global_field("core:datatype", "rf32_le")
sigmf_file_1.add_annotation(start_index=0, length=len(random_data1))
sigmf_file_1.add_capture(start_index=0)
sigmf_file_1.set_data_file(data1_path)

# create archive using SigMFArchive
archive1 = SigMFArchive(sigmffiles=sigmf_file_1,
path="single_recording_archive1.sigmf")

# create archive using SigMFFile archive()
archive1_path = sigmf_file_1.archive(file_path="single_recording_archive2.sigmf")

# create archive using tofile
sigmf_file_1.tofile(file_path="single_recording_archive3.sigmf",
toarchive=True)
jhazentia marked this conversation as resolved.
Show resolved Hide resolved
```

### Create SigMF Archives with Multiple Recordings
Archives with collections can be created using `SigMFArchive` class,
`SigMFCollection.archive()` method, and the `SigMFCollection.tofile()` method.

```python
import numpy as np

from sigmf.sigmffile import (SigMFFile,
SigMFArchive,
SigMFCollection)


# create data files
random_data1 = np.random.rand(128)
data1_path = "recording1.sigmf-data"
random_data1.tofile(data1_path)

random_data2 = np.random.rand(128)
data2_path = "recording2.sigmf-data"
random_data2.tofile(data2_path)

# create metadata
sigmf_file_1 = SigMFFile(name='recording1')
sigmf_file_1.set_global_field("core:datatype", "rf32_le")
sigmf_file_1.add_annotation(start_index=0, length=len(random_data1))
sigmf_file_1.add_capture(start_index=0)
sigmf_file_1.set_data_file(data1_path)

sigmf_file_2 = SigMFFile(name='recording2')
sigmf_file_2.set_global_field("core:datatype", "rf32_le")
sigmf_file_2.add_annotation(start_index=0, length=len(random_data2))
sigmf_file_2.add_capture(start_index=0)
sigmf_file_2.set_data_file(data2_path)

# create collection
sigmf_file_1.tofile("recording1.sigmf-meta")
sigmf_file_2.tofile("recording2.sigmf-meta")
metafiles = ["recording1.sigmf-meta", "recording2.sigmf-meta"]
collection = SigMFCollection(metafiles=metafiles)

# create archive using SigMFArchive without collection
sigmffiles = [sigmf_file_1, sigmf_file_2]
archive3 = SigMFArchive(sigmffiles=sigmffiles,
path="multi_recording_archive1.sigmf")

# create archive using collection archive
archive3_path = collection.archive(file_path="multi_recording_archive2.sigmf")

# create archive using collection tofile
collection.tofile(file_path="multi_recording_archive3.sigmf", toarchive=True)
```

### Load a SigMF Archive and slice its data without untaring it

Since an *archive* is merely a tarball (uncompressed), and since there any many
Expand Down