Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/Measurement loop #4074

Open
wants to merge 167 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
167 commits
Select commit Hold shift + click to select a range
6b9475d
keysight issue
nulinspiratie Feb 25, 2022
8b384bf
Merge branch 'origin/master'
nulinspiratie Feb 25, 2022
bfc330e
Auto stash before merge of "master" and "origin/master"
nulinspiratie Feb 25, 2022
4faaf95
Merge branch 'feature/monitor-upgrade'
nulinspiratie Feb 25, 2022
a6acead
monitor message
nulinspiratie Mar 6, 2022
8116751
Merge branch 'feature/monitor-upgrade'
nulinspiratie Mar 6, 2022
ab523df
Merge remote-tracking branch 'origin/master'
nulinspiratie Apr 2, 2022
88757c2
fix: ensure keyboardinterrupt doesn't interrupt visa command
nulinspiratie Apr 2, 2022
16d79a6
Merge branch 'bug/qdac-keyboardinterrupt'
nulinspiratie Apr 2, 2022
ffb444b
Begun incorporating measurement loop
nulinspiratie Apr 2, 2022
89d6ed3
Begun incorporating measurement loop
nulinspiratie Apr 2, 2022
4f43aed
Added DataHandler, DatasetHandler
nulinspiratie Apr 2, 2022
60a1d8c
Fixed Sweep
nulinspiratie Apr 2, 2022
0f1cab0
Gone through entire file, but needs testing
nulinspiratie Apr 2, 2022
575ede2
creating tests for measurement loop
nulinspiratie Apr 2, 2022
1f0171b
working 1d and 2d datasets
nulinspiratie Apr 3, 2022
5755686
basic 1d and 2d tests are working!
nulinspiratie Apr 3, 2022
a2852de
initializes dataset on second iteration of first sweep
nulinspiratie Apr 3, 2022
32aac38
Merge branch 'feature/measurement-loop'
nulinspiratie Apr 4, 2022
124e550
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 4, 2022
47beddd
merge
nulinspiratie Apr 5, 2022
f7e7a4d
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 5, 2022
7e80e47
Implemented incremental adding of parameters
nulinspiratie Apr 5, 2022
8aa2742
Merge branch 'master' into feature/measurement-loop
nulinspiratie Apr 5, 2022
dbe5fcd
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 5, 2022
a029767
Merge remote-tracking branch 'origin/bug/qdac-keyboardinterrupt'
Apr 6, 2022
64468a6
fix: Update data cache
nulinspiratie Apr 6, 2022
39ccc0b
test nested measurement and no_parameter
nulinspiratie Apr 6, 2022
afc0a94
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 6, 2022
c3ff94e
Merge remote-tracking branch 'origin/master'
Apr 8, 2022
8fa68f1
Merge remote-tracking branch 'sydney-fork/feature/measurement-loop'
nulinspiratie Apr 8, 2022
3fab4c8
Merge remote-tracking branch 'origin/master'
Apr 23, 2022
785031e
Merge remote-tracking branch 'microsoft/master'
nulinspiratie Apr 23, 2022
9398f1d
Merge remote-tracking branch 'nulinspiratie/master'
May 3, 2022
c3ffca3
add Measurement.enteractions
May 3, 2022
8b49145
Add repetitionSweep
nulinspiratie Jun 20, 2022
e1ea91c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 20, 2022
877d4e3
Merge remote-tracking branch 'MorelloLab/feature/measurement-loop'
Jun 20, 2022
b4f57e1
minor bugfixes
Jun 20, 2022
dfb7085
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 20, 2022
4185e55
Merge branch 'feature/measurement-loop'
Jun 20, 2022
9326822
upgrading Sweep
nulinspiratie Jul 20, 2022
431d753
upgrading sweep
nulinspiratie Jul 29, 2022
2ab4566
Nearly finished with Sweep
nulinspiratie Aug 1, 2022
013e3aa
Started adding tests for Sweep
nulinspiratie Aug 1, 2022
e648ad6
fixed tests
nulinspiratie Aug 1, 2022
a4b1720
Merge remote-tracking branch 'external/master' into feature/measureme…
nulinspiratie Aug 1, 2022
d4e18c1
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 1, 2022
ae0ee84
Wrote ~2/3 of MeasurementLoop tutorial
nulinspiratie Aug 2, 2022
7627398
Merge remote-tracking branch 'old-qcodes/feature/measurement-loop' in…
nulinspiratie Aug 2, 2022
783c48d
finished tutorial
nulinspiratie Aug 3, 2022
06cd4d8
slightly improve sweep execution
nulinspiratie Aug 4, 2022
73cc680
Merge branch 'feature/measurement-loop'
Aug 5, 2022
f441802
Forgot to add self as sweep in Sweep.execute
Aug 5, 2022
24f5da3
fixed issue with self-referencing type-hint
Aug 5, 2022
14a2835
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 5, 2022
1afa7d8
Merge remote-tracking branch 'MorelloLab/feature/measurement-loop'
Aug 5, 2022
3b9359a
Added MeasurementLoop, Sweep to qcodes.dataset
nulinspiratie Aug 9, 2022
b661605
BaseSweep is subclass of AbstractSweep
nulinspiratie Aug 10, 2022
71795d4
fixed bugs of BaseSweep
nulinspiratie Aug 10, 2022
8f165d9
Add args to sweep.execute
nulinspiratie Aug 10, 2022
744901f
fix: all tests working again
nulinspiratie Aug 10, 2022
f6a2b09
add some basic tests
nulinspiratie Aug 10, 2022
4916697
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 10, 2022
b4b5384
add len(sweep)
nulinspiratie Aug 15, 2022
3bdda68
Merge remote-tracking branch 'MorelloLab/feature/measurement-loop'
Aug 16, 2022
384d0e1
improve database for tests
nulinspiratie Aug 21, 2022
da536ef
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 21, 2022
9d38998
Apply pep8 formatting to code
nulinspiratie Sep 13, 2022
c5180a5
Added type hints
nulinspiratie Sep 13, 2022
79c5cc9
fix public api imports
nulinspiratie Sep 13, 2022
ea1be0f
attempting fix of keysight __init__.py
nulinspiratie Sep 13, 2022
513ca14
added newline after keysight/__init__.py
nulinspiratie Sep 13, 2022
e05a82e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 13, 2022
6da094d
Merge remote-tracking branch 'QCoDeS-CPH/master' into feature/measure…
nulinspiratie Sep 13, 2022
fa85657
added _BaseParameter import
nulinspiratie Sep 13, 2022
e3eebce
Merge remote-tracking branch 'QCoDeS-Sydney/feature/measurement-loop'…
nulinspiratie Sep 13, 2022
1fcab47
Changed to double-quotes
nulinspiratie Sep 13, 2022
74ea45a
load Measurement before MeasurementLoop
nulinspiratie Sep 13, 2022
f50595d
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 13, 2022
c15cd8a
Fixing circular imports
nulinspiratie Sep 13, 2022
beb66ab
Merge remote-tracking branch 'QCoDeS-Sydney/feature/measurement-loop'…
nulinspiratie Sep 13, 2022
f984c26
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 13, 2022
30859f8
wrote all the documentation for measurement_loop.py
nulinspiratie Sep 13, 2022
0d3ae37
Merge remote-tracking branch 'QCoDeS-Sydney/feature/measurement-loop'…
nulinspiratie Sep 13, 2022
8345ad9
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 13, 2022
1fa5df8
fix typo
nulinspiratie Sep 14, 2022
c40c771
applied pep styling
nulinspiratie Sep 14, 2022
deb0a12
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 14, 2022
b9519df
fix pytest error
nulinspiratie Sep 14, 2022
927ede2
Merge branch 'feature/measurement-loop' of https://github.com/nulinsp…
nulinspiratie Sep 14, 2022
70e0896
second round of pep fixes
nulinspiratie Sep 14, 2022
369c5dc
typing fixes
nulinspiratie Sep 14, 2022
54d9297
return self in MeasurementLoop context manager
nulinspiratie Sep 14, 2022
ae10757
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 14, 2022
66a7a63
Fixed last pep issue?
nulinspiratie Sep 14, 2022
800f0f6
Fixing Keysight capitalization issue
nulinspiratie Sep 14, 2022
c671d12
Merge branch 'feature/measurement-loop' of https://github.com/nulinsp…
nulinspiratie Sep 14, 2022
f1f2067
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 14, 2022
74596eb
Delete __init__.py
nulinspiratie Sep 14, 2022
33cac93
fixing pytest issues
nulinspiratie Sep 14, 2022
d13aaf3
Merge branch 'feature/measurement-loop' of https://github.com/nulinsp…
nulinspiratie Sep 14, 2022
ae78eed
Fixing pytest issues
nulinspiratie Sep 14, 2022
93027b9
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 14, 2022
9c587e5
missing double quote
nulinspiratie Sep 14, 2022
d8eee70
Merge branch 'feature/measurement-loop' of https://github.com/nulinsp…
nulinspiratie Sep 14, 2022
e099071
allow linsweep in MeasurementLoop
nulinspiratie Sep 15, 2022
448acc0
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 15, 2022
16c7f09
fix type errors
nulinspiratie Sep 15, 2022
bad167c
Merge remote-tracking branch 'QCoDeS-Sydney/feature/measurement-loop'…
nulinspiratie Sep 15, 2022
3d85b55
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 15, 2022
b2e0537
fraction complete working for basic example
nulinspiratie Sep 20, 2022
4711f6f
round fraction_complete
nulinspiratie Sep 21, 2022
b2c1997
add test fraction complete
nulinspiratie Sep 21, 2022
258c1d4
Working progress bars
nulinspiratie Sep 21, 2022
6e057a8
Don't show progress bar by default
nulinspiratie Sep 22, 2022
85131d4
Catch any progress bar errors
nulinspiratie Sep 22, 2022
02faed5
add plot functionality to Sweep.execute
nulinspiratie Sep 22, 2022
248e41b
Perform sweep using Sweep(*args)()
nulinspiratie Sep 22, 2022
02b2cdd
Merge branch 'feature/measurement-loop-percentage' into feature/measu…
nulinspiratie Sep 23, 2022
bdd3b00
Merge remote-tracking branch 'QCoDeS-CPH/master' into feature/measure…
nulinspiratie Sep 23, 2022
06bea97
Merge remote-tracking branch 'MorelloLab/feature/measurement-loop'
nulinspiratie Sep 23, 2022
8385b9c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 23, 2022
9d83dab
override parameter.sweep
nulinspiratie Sep 23, 2022
007ec03
Merge branch 'feature/parameter-sweep'
nulinspiratie Sep 23, 2022
15029a8
mistakenly added multiple sweeps
Oct 17, 2022
997e136
Merge branch 'feature/measurement-loop' into feature/parameter-sweep
Oct 17, 2022
611e7a7
fix in parameter sweep
Oct 17, 2022
7f8468d
Merge branch 'feature/parameter-sweep'
Oct 17, 2022
2106278
Add Exception to except
Oct 18, 2022
e2d2a56
Merge branch 'master' into feature/measurement-loop
Oct 18, 2022
698e205
Merge branch 'feature/measurement-loop' into master-msmt_pc
Oct 18, 2022
9e7f4fe
Merge branch 'fix/snapshot-exception' into master-msmt_pc
Oct 18, 2022
8356937
Convert None to NaN so it doesn't raise an error
Oct 18, 2022
d8ae7b5
Merge branch 'feature/measurement-loop' into master-msmt_pc
Oct 18, 2022
cf7af50
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 18, 2022
ecea839
minor fix to masking/unmasking
Oct 18, 2022
a754ba8
Merge branch 'feature/measurement-loop' into master-msmt_pc
Oct 18, 2022
67d969c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 18, 2022
c9400b1
reverting tqdm
Oct 18, 2022
3ced8d8
Fix plot function
Oct 18, 2022
66d54c9
Merge branch 'feature/measurement-loop' into feature/parameter-sweep
Oct 18, 2022
147b983
better plotting logic
Oct 18, 2022
aaa834b
Merge branch 'feature/measurement-loop' into master-msmt_pc
Oct 18, 2022
d0126b6
Merge branch 'feature/parameter-sweep' into master-msmt_pc
Oct 18, 2022
1dca9e8
show plot after sweep
nulinspiratie Oct 28, 2022
fb2db63
Merge branch 'feature/parameter-sweep'
nulinspiratie Oct 28, 2022
ad7db8d
remove sweep stop arg
nulinspiratie Nov 8, 2022
3eecb6c
ignore num when step is given
Nov 11, 2022
dbcc850
fix step + num
Nov 13, 2022
bf61df7
include metadata
nulinspiratie Nov 15, 2022
0ad7dad
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 15, 2022
a29ff55
begun adding measurement loop array
nulinspiratie Dec 6, 2022
391b5d5
fixed mistaken test
nulinspiratie Dec 6, 2022
a7a45a9
ran through first test without errors
nulinspiratie Dec 6, 2022
a9020ef
measuring arrays works
nulinspiratie Dec 6, 2022
248c48c
Merge branch 'feature/measurement-loop-array' into feature/measuremen…
nulinspiratie Dec 6, 2022
f1f1516
don't show more than one tqdm progress bar
nulinspiratie Dec 8, 2022
f7fe1c3
minor fix for measuring arrays
nulinspiratie Dec 8, 2022
6dad7e6
bug in unmasking
nulinspiratie Dec 8, 2022
ed1cb20
Allow sweep without name
nulinspiratie Dec 8, 2022
cd950df
Fix unmasking of sweep in msmt
nulinspiratie Dec 8, 2022
4dd77cb
add Iterate
nulinspiratie Dec 8, 2022
9c34a28
Add N-dimensional array support
nulinspiratie Dec 18, 2022
b79b2b5
improvements to config
Dec 19, 2022
54aad18
verification improvements
Dec 19, 2022
6300b17
add thread functionality
Mar 7, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1,403 changes: 1,403 additions & 0 deletions docs/examples/DataSet/MeasurementLoop.ipynb

Large diffs are not rendered by default.

113 changes: 79 additions & 34 deletions qcodes/configuration/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -403,13 +403,18 @@ def describe(self, name: str) -> str:

return doc

def __getitem__(self, name: str) -> Any:
val = self.current_config
for key in name.split('.'):
if val is None:
raise KeyError(f"{name} not found in current config")
val = val[key]
return val
def __getitem__(self, name: Union[int,str]) -> Any:
if isinstance(name, int):
# Integer requested, likely a consequence of "if 'string' in config"
# Return key corresponding to index
return list(self.current_config.keys())[name]
else:
val = self.current_config
for key in name.split('.'):
if val is None:
raise KeyError(f"{name} not found in current config")
val = val[key]
return val

def __getattr__(self, name: str) -> Any:
return getattr(self.current_config, name)
Expand All @@ -422,22 +427,21 @@ def __repr__(self) -> str:
return output


class DotDict(Dict[str, Any]):
class DotDict(dict):
"""
Wrapper dict that allows to get dotted attributes

Requires keys to be strings.
"""

def __init__(self, value: Mapping[str, Any] | None = None):
exclude_from_dict = []
def __init__(self, value=None):
if value is None:
pass
else:
for key in value:
self.__setitem__(key, value[key])

def __setitem__(self, key: str, value: Any) -> None:
if '.' in key:
def __setitem__(self, key, value):
# string type must be checked, as key could be other datatype
if type(key)==str and '.' in key:
myKey, restOfKey = key.split('.', 1)
target = self.setdefault(myKey, DotDict())
target[restOfKey] = value
Expand All @@ -446,36 +450,77 @@ def __setitem__(self, key: str, value: Any) -> None:
value = DotDict(value)
dict.__setitem__(self, key, value)

def __getitem__(self, key: str) -> Any:
if '.' not in key:
def __getitem__(self, key):
if type(key) != str or '.' not in key:
return dict.__getitem__(self, key)
myKey, restOfKey = key.split('.', 1)
target = dict.__getitem__(self, myKey)
return target[restOfKey]

def __contains__(self, key: object) -> bool:
if not isinstance(key, str):
return False
if '.' not in key:
return super().__contains__(key)
def __contains__(self, key):
if not isinstance(key, str) or '.' not in key:
return dict.__contains__(self, key)
myKey, restOfKey = key.split('.', 1)
target = dict.__getitem__(self, myKey)
return restOfKey in target

def __deepcopy__(self, memo: dict[Any, Any] | None) -> DotDict:
return DotDict(copy.deepcopy(dict(self)))
if myKey not in self:
return False
else:
target = dict.__getitem__(self, myKey)
return restOfKey in target

def __getattr__(self, name: str) -> Any:
"""
Overwrite ``__getattr__`` to provide dot access
"""
return self.__getitem__(name)
def __deepcopy__(self, memo):
return DotDict(copy.deepcopy(dict(self)))

def __setattr__(self, key: str, value: Any) -> None:
"""
Overwrite ``__setattr__`` to provide dot access
# dot acces baby
def __setattr__(self, key, val):
if key in self.exclude_from_dict:
self.__dict__[key] = val
else:
self.__setitem__(key, val)

def __getattr__(self, key):
try:
return self.__getitem__(key)
except KeyError:
raise AttributeError(f'Attribute {key} not found')

def __dir__(self):
# Add keys to dir, used for auto-completion
items = super().__dir__()
items.extend(self.keys())
return items

def setdefault(self, key, default=None):
"""Set value of a key if it does not yet exist"""
d = self
if isinstance(key, str):
*parent_keys, key = key.split('.')
for subkey in parent_keys:
d = dict.setdefault(d, subkey, DotDict())

return dict.setdefault(d, key, default)

def create_dicts(self, *keys):
"""Create nested dict structure
Args:
*keys: Sequence of key strings. Empty DotDicts will be created if
each key does not yet exist
Returns:
Most inner dict, newly created if it does not yet exist
Examples:
d = DotDict()
d.create_dicts('a', 'b', 'c')
print(d.a.b.c)
>>> {}
"""
self.__setitem__(key, value)
d = self
for key in keys:
if key in self:
assert isinstance(d[key], dict)

d.setdefault(key, DotDict())
d = d[key]
return d


def update(d: dict[Any, Any], u: Mapping[Any, Any]) -> dict[Any, Any]:
Expand Down
3 changes: 3 additions & 0 deletions qcodes/dataset/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
)
from .experiment_settings import get_default_experiment_id, reset_default_experiment_id
from .legacy_import import import_dat_file
from .measurement_loop import MeasurementLoop, Sweep, Iterate
from .measurements import Measurement
from .plotting import plot_by_id, plot_dataset
from .sqlite.connection import ConnectionPlus
Expand Down Expand Up @@ -82,9 +83,11 @@
"load_from_netcdf",
"load_last_experiment",
"load_or_create_experiment",
"MeasurementLoop",
"new_data_set",
"new_experiment",
"plot_by_id",
"plot_dataset",
"reset_default_experiment_id",
"Sweep",
]
3 changes: 2 additions & 1 deletion qcodes/dataset/data_set.py
Original file line number Diff line number Diff line change
Expand Up @@ -313,11 +313,12 @@ def prepare(
shapes: Shapes | None = None,
parent_datasets: Sequence[Mapping[Any, Any]] = (),
write_in_background: bool = False,
allow_empty_dataset: bool = False,
) -> None:

self.add_snapshot(json.dumps({"station": snapshot}, cls=NumpyJSONEncoder))

if interdeps == InterDependencies_():
if interdeps == InterDependencies_() and not allow_empty_dataset:
raise RuntimeError("No parameters supplied")

self.set_interdependencies(interdeps, shapes)
Expand Down
3 changes: 2 additions & 1 deletion qcodes/dataset/data_set_in_memory.py
Original file line number Diff line number Diff line change
Expand Up @@ -401,13 +401,14 @@ def prepare(
shapes: Shapes | None = None,
parent_datasets: Sequence[Mapping[Any, Any]] = (),
write_in_background: bool = False,
allow_empty_dataset: bool = False,
) -> None:
if not self.pristine:
raise RuntimeError("Cannot prepare a dataset that is not pristine.")

self.add_snapshot(json.dumps({"station": snapshot}, cls=NumpyJSONEncoder))

if interdeps == InterDependencies_():
if interdeps == InterDependencies_() and not allow_empty_dataset:
raise RuntimeError("No parameters supplied")

self._set_interdependencies(interdeps, shapes)
Expand Down
1 change: 1 addition & 0 deletions qcodes/dataset/data_set_protocol.py
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ def prepare(
shapes: Shapes | None = None,
parent_datasets: Sequence[Mapping[Any, Any]] = (),
write_in_background: bool = False,
allow_empty_dataset: bool = False,
) -> None:
pass

Expand Down