Refactored 2.16 release into a more reusable template

[EricLeFort]

Description

Refactors the release DAG to allow for easy swapping of versions for reuse in future releases and renames the modules to be more generic (...tf_release... instead of ...tf_2_16...).

Also changes some typing references (e.g. list instead of typing.List) to adhere to Python best practices PEP 585. This specific change will require python 3.11+ but adding from __future__ import annotations can allow python 3.7+. If we still need to support the older versions, I can update this change to include that.

Tests

Ran 2.16 release PJRT and SE keras tests, both were successful.

List links for your tests (use go/shortn-gen for any internal link):
Variety of tests of both nightly and 2.15.1 DAGS as well as from resnet and tf-keras tests:
http://shortn/_2sfrmqsyfr
http://shortn/_v5h49gyH6t
http://shortn/_MNIr72f9Z5
http://shortn/_WBsR2NNTlT
http://shortn/_MbmkG5jqOF
http://shortn/_HVtxmvAvoP
http://shortn/_XneLXMOyTn
Note: nightly tests are failing but they're failing for the same reason as they are in the dashboard.

Checklist

Before submitting this PR, please make sure (put X in square brackets):

• I have performed a self-review of my code.
• I have necessary comments in my code, particularly in hard-to-understand areas.
• I have run one-shot tests and provided workload links above if applicable.
• I have made or will make corresponding changes to the doc if needed.

[EricLeFort]

Looks like GitHub's dag checker is using Python < 3.11 so I may as well just add the 3.7+ support. Fixing.

[RissyRan]

Thanks Eric! Others look good to me!

[RissyRan]

Good point Eric! Do we need to actually monitor each release for 3-6 months? I think that's what we did last year unless the guidelines changed.

…

On Tue, Mar 19, 2024, 10:47 AM Eric Le Fort ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In dags/solutions_team/solutionsteam_tf_release_se_supported.py <#205 (comment)> : > with models.DAG( - dag_id="tf_2_16_se_nightly_supported", + dag_id=f"tf_{tf_config.MAJOR_VERSION}_{tf_config.MINOR_VERSION}_se_nightly_supported", I think we'd only ever want to have a single release in flight at any given time -- that's what we do today anyway. Although I could see a world where we leave release DAGs in place for future reference. I don't think it's a bad refactor regardless, it separates the logic of how to build a release DAG from which release to build a DAG for. — Reply to this email directly, view it on GitHub <#205 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE3Q5OTWD7AZRY2QHLJ4PELYZB2ZPAVCNFSM6AAAAABEZVPALWVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTSNBWHE3TAMZTG4> . You are receiving this because your review was requested.Message ID: ***@***.*** com>

[EricLeFort]

@RissyRan The 2.15.0 release DAG didn't even exist when I started on 2.16.0. We also only run release tests using manual triggers these days. So I'm not sure but it doesn't seem like it.

[RissyRan]

LGTM. I am not sure the current release process (if a single release still is monitored for 3-6 months, so we will have multiple release dags).

I see owner has approved. We could always come back to update variables