Skip to content

Commit

Permalink
feat: Add wrapper for defining columns in TTree in ROOT (#2898)
Browse files Browse the repository at this point in the history
Adds a wrapper for defining columns in TTree in [the ROOT software
package](https://root.cern.ch/). The ntuple ROOT files in the test are
from the test of the `hadd` wrapper.

<!-- Ensure that the PR title follows conventional commit style (<type>:
<description>)-->
<!-- Possible types are here:
https://github.com/commitizen/conventional-commit-types/blob/master/index.json
-->

<!-- Add a description of your PR here-->

### QC
<!-- Make sure that you can tick the boxes below. -->

* [x] I confirm that:

For all wrappers added by this PR, 

* there is a test case which covers any introduced changes,
* `input:` and `output:` file paths in the resulting rule can be changed
arbitrarily,
* either the wrapper can only use a single core, or the example rule
contains a `threads: x` statement with `x` being a reasonable default,
* rule names in the test case are in
[snake_case](https://en.wikipedia.org/wiki/Snake_case) and somehow tell
what the rule is about or match the tools purpose or name (e.g.,
`map_reads` for a step that maps reads),
* all `environment.yaml` specifications follow [the respective best
practices](https://stackoverflow.com/a/64594513/2352071),
* the `environment.yaml` pinning has been updated by running
`snakedeploy pin-conda-envs environment.yaml` on a linux machine,
* wherever possible, command line arguments are inferred and set
automatically (e.g. based on file extensions in `input:` or `output:`),
* all fields of the example rules in the `Snakefile`s and their entries
are explained via comments (`input:`/`output:`/`params:` etc.),
* `stderr` and/or `stdout` are logged correctly (`log:`), depending on
the wrapped tool,
* temporary files are either written to a unique hidden folder in the
working directory, or (better) stored where the Python function
`tempfile.gettempdir()` points to (see
[here](https://docs.python.org/3/library/tempfile.html#tempfile.gettempdir);
this also means that using any Python `tempfile` default behavior
works),
* the `meta.yaml` contains a link to the documentation of the respective
tool or command,
* `Snakefile`s pass the linting (`snakemake --lint`),
* `Snakefile`s are formatted with
[snakefmt](https://github.com/snakemake/snakefmt),
* Python wrapper scripts are formatted with
[black](https://black.readthedocs.io).
* Conda environments use a minimal amount of channels, in recommended
ordering. E.g. for bioconda, use (conda-forge, bioconda, nodefaults, as
conda-forge should have highest priority and defaults channels are
usually not needed because most packages are in conda-forge nowadays).

---------

Co-authored-by: Anfeng Li <anfeng.li@cern.ch>
Co-authored-by: Filipe G. Vieira <1151762+fgvieira@users.noreply.github.com>
  • Loading branch information
3 people committed Apr 22, 2024
1 parent 206425b commit 48730cd
Show file tree
Hide file tree
Showing 7 changed files with 350 additions and 0 deletions.
277 changes: 277 additions & 0 deletions phys/root/define_columns/environment.linux-64.pin.txt

Large diffs are not rendered by default.

4 changes: 4 additions & 0 deletions phys/root/define_columns/environment.yaml
@@ -0,0 +1,4 @@
channels:
- conda-forge
dependencies:
- ROOT=6.30.4
15 changes: 15 additions & 0 deletions phys/root/define_columns/meta.yaml
@@ -0,0 +1,15 @@
name: "define_columns"
description: Define columns in a TTree using RDataFrame
url: https://root.cern/doc/master/classROOT_1_1RDataFrame.html
authors:
- Anfeng Li
input:
- TTree ROOT file
output:
- TTree ROOT file
params:
- input_tree_name: name of the input TTree
- output_tree_name: name of the output TTree
- branches: branches to be defined, specified in the format of [["branch_name1", "definition_expression1"], ["branch_name2", "definition_expression2"]]. If not specified, no branch will be defined, i.e. just save the TTree. (optional)
- redefine: list of branch names to be redefined. It must be defined in `params.branch` also. (optional)
- branches_to_save: list of branch names to be saved. If not specified, all branches of the input TTree will be saved. (optional)
18 changes: 18 additions & 0 deletions phys/root/define_columns/test/Snakefile
@@ -0,0 +1,18 @@
rule define_columns:
input:
"ntuple0.root",
output:
"ntuple0_output.root",
log:
"logs/define_columns/define_columns.log",
params:
input_tree_name="TestTree",
output_tree_name="TestTree",
branches=[
["p2", "px * px + py * py + pz * pz"],
["pt", "sqrt(px * px + py * py)"],
],
redefine=["pt"],
threads: 2
wrapper:
"master/phys/root/define_columns"
Binary file added phys/root/define_columns/test/ntuple0.root
Binary file not shown.
28 changes: 28 additions & 0 deletions phys/root/define_columns/wrapper.py
@@ -0,0 +1,28 @@
__author__ = "Anfeng Li"
__copyright__ = "Copyright 2024, Anfeng Li"
__email__ = "anfeng.li@cern.ch"
__license__ = "MIT"


import ROOT

ROOT.EnableImplicitMT(snakemake.threads)

redefine_list = snakemake.params.get("redefine", [])
branches = snakemake.params.get("branches", [])
branches_to_save = snakemake.params.get("branches_to_save", None)

df = ROOT.RDataFrame(snakemake.params.input_tree_name, snakemake.input[0])
for branch_name, branch_definition in branches:
if branch_name in redefine_list:
df = df.Redefine(branch_name, branch_definition)
else:
df = df.Define(branch_name, branch_definition)
if branches_to_save is not None:
df.Snapshot(
snakemake.params.output_tree_name,
snakemake.output[0],
branches_to_save,
)
else:
df.Snapshot(snakemake.params.output_tree_name, snakemake.output[0])
8 changes: 8 additions & 0 deletions test.py
Expand Up @@ -6484,6 +6484,14 @@ def test_root_hadd():
)


@skip_if_not_modified
def test_root_define_columns():
run(
"phys/root/define_columns",
["snakemake", "--cores", "2", "--use-conda", "-F"],
)


@skip_if_not_modified
def test_emu_abundance():
run(
Expand Down

0 comments on commit 48730cd

Please sign in to comment.