Deprecate device-specific GradScaler autocast API #126527

guangyey · 2024-05-17T10:23:46Z

Stack from ghstack (oldest at bottom):

Motivation

for `torch.amp.GradScaler`,

torch.cpu.amp.GradScaler(args...) is completely equivalent to torch. amp.GradScaler("cpu", args...).
torch.cuda.amp.GradScaler(args...) is completely equivalent to torch.amp.GradScaler("cuda", args...).

So, we intend to depreate them and strongly recommend developer to use torch.amp.GradScaler.

for `custom_fwd` and `custom_bwd`,

this is a good solution to make the custom function run with or without effect even in an autocast-enabled region and can be shared by other backends, like CPU and XPU.
So we generalize it to be device-agnostic and put them int torch/amp/autocast_mode.py and re-expose to torch.amp.custom_fwd and torch.amp.custom_bwd. Meanwhile, we deprecate torch.cuda.amp.custom_fwd and torch.cuda.amp.custom_bwd.

Additional Context

Add UT to cover the deprecated warning.
No need for more UTs to cover the functionality of torch.amp.custom_f/bwd, the existing UTs that previously covered the functionality of torch.cuda.amp.custom_f/bwd can cover them.
To facilitate the review, we separate these code changes to two PRs. The first PR cover torch.amp.GradScaler. The follow-up covers custom_fwd and custom_bwd.

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @penguinwu @fegin @XilunWu @wanchaol @fduwjj @wz337 @tianyu-l @wconstab @yf225 @chauhang @d4l3k @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10 @mcarilli @ptrblck @leslie-fang-intel @voznesenskym @EikanWang @Guobing-Chen @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng

pytorch-bot · 2024-05-17T10:23:49Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/126527

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 22b3690 with merge base 5fb11cd ():

FLAKY - The following job failed but was likely due to flakiness present on trunk:

inductor / cuda12.1-py3.12-gcc9-sm86 / test (inductor, 1, 1, linux.g5.4xlarge.nvidia.gpu) (gh) (similar failure)
inductor/test_aot_inductor.py::AOTInductorTestABICompatibleCuda::test_bmm_multiple_dynamic_abi_compatible_cuda

This comment was automatically generated by Dr. CI and updates every 15 minutes.

ghstack-source-id: 139fa79ce287da588c7d6d967371057c9081219c Pull Request resolved: #126527

[ghstack-poisoned]

# Motivation ## for `torch.amp.GradScaler`, - `torch.cpu.amp.GradScaler(args...)` is completely equivalent to `torch. amp.GradScaler("cpu", args...)`. - `torch.cuda.amp.GradScaler(args...)` is completely equivalent to `torch.amp.GradScaler("cuda", args...)`. So, we intend to depreate them and **strongly recommend** developer to use `torch.amp.GradScaler`. ## for `custom_fwd` and `custom_bwd`, this is a good solution to make the custom function run without effect even in an autocast-enabled region and can be shared by other backends, like CPU and XPU. So we generalize it to be device-agnostic and put them int `torch/amp/autocast_mode.py` and re-expose to `torch.amp.custom_fwd` and `torch.amp.custom_bwd`. Meanwhile, we deprecate `torch.cuda.amp.custom_fwd` and `torch.cuda.amp.custom_bwd`. # Additional Context Add UT to cover the deprecated warning. No need for more UTs to cover the functionality of `torch.amp.custom_f/bwd`, the existing UTs that previously covered the functionality of `torch.cuda.amp.custom_f/bwd` can cover them. To facilitate the review, we separate these code changes to two PRs. The first PR cover `torch.amp.GradScaler`. The follow-up covers `custom_fwd` and `custom_bwd`. cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 mcarilli ptrblck leslie-fang-intel [ghstack-poisoned]

# Motivation ## for `torch.amp.GradScaler`, - `torch.cpu.amp.GradScaler(args...)` is completely equivalent to `torch. amp.GradScaler("cpu", args...)`. - `torch.cuda.amp.GradScaler(args...)` is completely equivalent to `torch.amp.GradScaler("cuda", args...)`. So, we intend to depreate them and **strongly recommend** developer to use `torch.amp.GradScaler`. ## for `custom_fwd` and `custom_bwd`, this is a good solution to make the custom function run with or without effect even in an autocast-enabled region and can be shared by other backends, like CPU and XPU. So we generalize it to be device-agnostic and put them int `torch/amp/autocast_mode.py` and re-expose to `torch.amp.custom_fwd` and `torch.amp.custom_bwd`. Meanwhile, we deprecate `torch.cuda.amp.custom_fwd` and `torch.cuda.amp.custom_bwd`. # Additional Context Add UT to cover the deprecated warning. No need for more UTs to cover the functionality of `torch.amp.custom_f/bwd`, the existing UTs that previously covered the functionality of `torch.cuda.amp.custom_f/bwd` can cover them. To facilitate the review, we separate these code changes to two PRs. The first PR cover `torch.amp.GradScaler`. The follow-up covers `custom_fwd` and `custom_bwd`. cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 mcarilli ptrblck leslie-fang-intel [ghstack-poisoned]

janeyx99

This deprecation is desirable, but we should remove all mentions of torch.cuda.amp.GradScaler and torch.cpu.amp.GradScaler in the codebase (e.g., in the tests) and replace with the new usage.

This is one way to ensure that the recommended version of GradScaler will remain sufficiently tested too.

test/test_torch.py

janeyx99

Please also update the docs

pytorch/docs/source/notes/amp_examples.rst

Line 112 in 86ad101

.. currentmodule:: torch.cuda.amp.GradScaler

guangyey · 2024-05-22T15:34:28Z

Please also update the docs

pytorch/docs/source/notes/amp_examples.rst

Line 112 in 86ad101

.. currentmodule:: torch.cuda.amp.GradScaler

I update deprecation warning here, I think it will be exposed here https://pytorch.org/docs/stable/amp.html#gradient-scaling feature. May I know if I capture your point?

janeyx99 · 2024-05-22T17:04:23Z

@guangyey I mean that this page should also get updated to direct people to the recommended API

# Motivation ## for `torch.amp.GradScaler`, - `torch.cpu.amp.GradScaler(args...)` is completely equivalent to `torch. amp.GradScaler("cpu", args...)`. - `torch.cuda.amp.GradScaler(args...)` is completely equivalent to `torch.amp.GradScaler("cuda", args...)`. So, we intend to depreate them and **strongly recommend** developer to use `torch.amp.GradScaler`. ## for `custom_fwd` and `custom_bwd`, this is a good solution to make the custom function run with or without effect even in an autocast-enabled region and can be shared by other backends, like CPU and XPU. So we generalize it to be device-agnostic and put them int `torch/amp/autocast_mode.py` and re-expose to `torch.amp.custom_fwd` and `torch.amp.custom_bwd`. Meanwhile, we deprecate `torch.cuda.amp.custom_fwd` and `torch.cuda.amp.custom_bwd`. # Additional Context Add UT to cover the deprecated warning. No need for more UTs to cover the functionality of `torch.amp.custom_f/bwd`, the existing UTs that previously covered the functionality of `torch.cuda.amp.custom_f/bwd` can cover them. To facilitate the review, we separate these code changes to two PRs. The first PR cover `torch.amp.GradScaler`. The follow-up covers `custom_fwd` and `custom_bwd`. cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 mcarilli ptrblck leslie-fang-intel [ghstack-poisoned]

# Motivation ## for `torch.amp.GradScaler`, - `torch.cpu.amp.GradScaler(args...)` is completely equivalent to `torch. amp.GradScaler("cpu", args...)`. - `torch.cuda.amp.GradScaler(args...)` is completely equivalent to `torch.amp.GradScaler("cuda", args...)`. So, we intend to depreate them and **strongly recommend** developer to use `torch.amp.GradScaler`. ## for `custom_fwd` and `custom_bwd`, this is a good solution to make the custom function run with or without effect even in an autocast-enabled region and can be shared by other backends, like CPU and XPU. So we generalize it to be device-agnostic and put them int `torch/amp/autocast_mode.py` and re-expose to `torch.amp.custom_fwd` and `torch.amp.custom_bwd`. Meanwhile, we deprecate `torch.cuda.amp.custom_fwd` and `torch.cuda.amp.custom_bwd`. # Additional Context Add UT to cover the deprecated warning. No need for more UTs to cover the functionality of `torch.amp.custom_f/bwd`, the existing UTs that previously covered the functionality of `torch.cuda.amp.custom_f/bwd` can cover them. To facilitate the review, we separate these code changes to two PRs. The first PR cover `torch.amp.GradScaler`. The follow-up covers `custom_fwd` and `custom_bwd`. cc mrshenli pritamdamania87 zhaojuanmao satgera gqchen aazzolini osalpekar jiayisuse H-Huang kwen2501 awgu penguinwu fegin XilunWu wanchaol fduwjj wz337 tianyu-l wconstab yf225 chauhang d4l3k jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 mcarilli ptrblck leslie-fang-intel voznesenskym EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng [ghstack-poisoned]

guangyey · 2024-05-23T07:27:04Z

@guangyey I mean that this page should also get updated to direct people to the recommended API

I updated the doc. Could you help review this PR again?

# Motivation ## for `torch.amp.GradScaler`, - `torch.cpu.amp.GradScaler(args...)` is completely equivalent to `torch. amp.GradScaler("cpu", args...)`. - `torch.cuda.amp.GradScaler(args...)` is completely equivalent to `torch.amp.GradScaler("cuda", args...)`. So, we intend to depreate them and **strongly recommend** developer to use `torch.amp.GradScaler`. ## for `custom_fwd` and `custom_bwd`, this is a good solution to make the custom function run with or without effect even in an autocast-enabled region and can be shared by other backends, like CPU and XPU. So we generalize it to be device-agnostic and put them int `torch/amp/autocast_mode.py` and re-expose to `torch.amp.custom_fwd` and `torch.amp.custom_bwd`. Meanwhile, we deprecate `torch.cuda.amp.custom_fwd` and `torch.cuda.amp.custom_bwd`. # Additional Context Add UT to cover the deprecated warning. No need for more UTs to cover the functionality of `torch.amp.custom_f/bwd`, the existing UTs that previously covered the functionality of `torch.cuda.amp.custom_f/bwd` can cover them. To facilitate the review, we separate these code changes to two PRs. The first PR cover `torch.amp.GradScaler`. The follow-up covers `custom_fwd` and `custom_bwd`. cc mrshenli pritamdamania87 zhaojuanmao satgera gqchen aazzolini osalpekar jiayisuse H-Huang kwen2501 awgu penguinwu fegin XilunWu wanchaol fduwjj wz337 tianyu-l wconstab yf225 chauhang d4l3k jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 mcarilli ptrblck leslie-fang-intel voznesenskym EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng [ghstack-poisoned]

guangyey · 2024-05-23T16:07:37Z

@janeyx99 could you help review this PR again? Thanks very much~

janeyx99 · 2024-05-24T13:40:48Z

docs/source/amp.rst

-* ``torch.GradScaler("cuda", args...)`` is equivalent to ``torch.cuda.amp.GradScaler(args...)``.
-* ``torch.GradScaler("cpu", args...)`` is equivalent to ``torch.cpu.amp.GradScaler(args...)``.
+.. warning::
+    ``torch.cuda.amp.autocast(args...)`` and ``torch.cpu.amp.autocast(args...)`` will be deprecated. Please use ``torch.autocast("cuda", args...)`` or ``torch.autocast("cpu", args...)`` instead.


Include this fact here still?
For CPU, only lower precision floating point datatype of torch.bfloat16 is supported for now.

I remember that torch.float16 is already supported on CPU now, right @leslie-fang-intel

I think CPU Autocast also support torch.float16 now

janeyx99 · 2024-05-24T13:41:53Z

docs/source/amp.rst

@@ -25,12 +25,9 @@ However, :class:`torch.autocast` and :class:`torch.GradScaler` are modular, and
 As shown in the CPU example section of :class:`torch.autocast`, "automatic mixed precision training/inference" on CPU with
 datatype of ``torch.bfloat16`` only uses :class:`torch.autocast`.

-For CUDA and CPU, APIs are also provided separately:


Replace mention of torch.cpu.amp.GradScaler and cuda.amp.GradScaler with just amp.Scaler in line 22

Good catch. Updated.

janeyx99

Some last nits, thanks!

ghstack-source-id: f454a70e4fddb4af7db42d69d27b1f247004966d Pull Request resolved: #126527

guangyey · 2024-05-24T14:08:04Z

Some last nits, thanks!

Thanks for your approval. Have a nice day~

# Motivation ## for `torch.amp.GradScaler`, - `torch.cpu.amp.GradScaler(args...)` is completely equivalent to `torch. amp.GradScaler("cpu", args...)`. - `torch.cuda.amp.GradScaler(args...)` is completely equivalent to `torch.amp.GradScaler("cuda", args...)`. So, we intend to depreate them and **strongly recommend** developer to use `torch.amp.GradScaler`. ## for `custom_fwd` and `custom_bwd`, this is a good solution to make the custom function run with or without effect even in an autocast-enabled region and can be shared by other backends, like CPU and XPU. So we generalize it to be device-agnostic and put them int `torch/amp/autocast_mode.py` and re-expose to `torch.amp.custom_fwd` and `torch.amp.custom_bwd`. Meanwhile, we deprecate `torch.cuda.amp.custom_fwd` and `torch.cuda.amp.custom_bwd`. # Additional Context Add UT to cover the deprecated warning. No need for more UTs to cover the functionality of `torch.amp.custom_f/bwd`, the existing UTs that previously covered the functionality of `torch.cuda.amp.custom_f/bwd` can cover them. To facilitate the review, we separate these code changes to two PRs. The first PR cover `torch.amp.GradScaler`. The follow-up covers `custom_fwd` and `custom_bwd`. cc mrshenli pritamdamania87 zhaojuanmao satgera gqchen aazzolini osalpekar jiayisuse H-Huang kwen2501 awgu penguinwu fegin XilunWu wanchaol fduwjj wz337 tianyu-l wconstab yf225 chauhang d4l3k jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 mcarilli ptrblck leslie-fang-intel voznesenskym EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng [ghstack-poisoned]

guangyey · 2024-05-25T06:39:17Z

@pytorchbot merge

pytorchmergebot · 2024-05-25T06:41:06Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

Pull Request resolved: #126531 Approved by: https://github.com/jgong5, https://github.com/gujinghui, https://github.com/albanD, https://github.com/EikanWang ghstack dependencies: #126527

Summary: # Motivation ## for `torch.amp.GradScaler`, - `torch.cpu.amp.GradScaler(args...)` is completely equivalent to `torch. amp.GradScaler("cpu", args...)`. - `torch.cuda.amp.GradScaler(args...)` is completely equivalent to `torch.amp.GradScaler("cuda", args...)`. So, we intend to depreate them and **strongly recommend** developer to use `torch.amp.GradScaler`. ## for `custom_fwd` and `custom_bwd`, this is a good solution to make the custom function run with or without effect even in an autocast-enabled region and can be shared by other backends, like CPU and XPU. So we generalize it to be device-agnostic and put them int `torch/amp/autocast_mode.py` and re-expose to `torch.amp.custom_fwd` and `torch.amp.custom_bwd`. Meanwhile, we deprecate `torch.cuda.amp.custom_fwd` and `torch.cuda.amp.custom_bwd`. # Additional Context Add UT to cover the deprecated warning. No need for more UTs to cover the functionality of `torch.amp.custom_f/bwd`, the existing UTs that previously covered the functionality of `torch.cuda.amp.custom_f/bwd` can cover them. To facilitate the review, we separate these code changes to two PRs. The first PR cover `torch.amp.GradScaler`. The follow-up covers `custom_fwd` and `custom_bwd`. X-link: pytorch/pytorch#126527 Approved by: https://github.com/jgong5, https://github.com/gujinghui, https://github.com/janeyx99, https://github.com/EikanWang Reviewed By: PaliC Differential Revision: D57838085 fbshipit-source-id: 09a29e2535e66643d212276779605c573391666f

# Motivation ## for `torch.amp.GradScaler`, - `torch.cpu.amp.GradScaler(args...)` is completely equivalent to `torch. amp.GradScaler("cpu", args...)`. - `torch.cuda.amp.GradScaler(args...)` is completely equivalent to `torch.amp.GradScaler("cuda", args...)`. So, we intend to depreate them and **strongly recommend** developer to use `torch.amp.GradScaler`. ## for `custom_fwd` and `custom_bwd`, this is a good solution to make the custom function run with or without effect even in an autocast-enabled region and can be shared by other backends, like CPU and XPU. So we generalize it to be device-agnostic and put them int `torch/amp/autocast_mode.py` and re-expose to `torch.amp.custom_fwd` and `torch.amp.custom_bwd`. Meanwhile, we deprecate `torch.cuda.amp.custom_fwd` and `torch.cuda.amp.custom_bwd`. # Additional Context Add UT to cover the deprecated warning. No need for more UTs to cover the functionality of `torch.amp.custom_f/bwd`, the existing UTs that previously covered the functionality of `torch.cuda.amp.custom_f/bwd` can cover them. To facilitate the review, we separate these code changes to two PRs. The first PR cover `torch.amp.GradScaler`. The follow-up covers `custom_fwd` and `custom_bwd`. Pull Request resolved: pytorch#126527 Approved by: https://github.com/jgong5, https://github.com/gujinghui, https://github.com/janeyx99, https://github.com/EikanWang

Pull Request resolved: pytorch#126531 Approved by: https://github.com/jgong5, https://github.com/gujinghui, https://github.com/albanD, https://github.com/EikanWang ghstack dependencies: pytorch#126527

# Motivation ## for `torch.amp.GradScaler`, - `torch.cpu.amp.GradScaler(args...)` is completely equivalent to `torch. amp.GradScaler("cpu", args...)`. - `torch.cuda.amp.GradScaler(args...)` is completely equivalent to `torch.amp.GradScaler("cuda", args...)`. So, we intend to depreate them and **strongly recommend** developer to use `torch.amp.GradScaler`. ## for `custom_fwd` and `custom_bwd`, this is a good solution to make the custom function run with or without effect even in an autocast-enabled region and can be shared by other backends, like CPU and XPU. So we generalize it to be device-agnostic and put them int `torch/amp/autocast_mode.py` and re-expose to `torch.amp.custom_fwd` and `torch.amp.custom_bwd`. Meanwhile, we deprecate `torch.cuda.amp.custom_fwd` and `torch.cuda.amp.custom_bwd`. # Additional Context Add UT to cover the deprecated warning. No need for more UTs to cover the functionality of `torch.amp.custom_f/bwd`, the existing UTs that previously covered the functionality of `torch.cuda.amp.custom_f/bwd` can cover them. To facilitate the review, we separate these code changes to two PRs. The first PR cover `torch.amp.GradScaler`. The follow-up covers `custom_fwd` and `custom_bwd`. Pull Request resolved: #126527 Approved by: https://github.com/jgong5, https://github.com/gujinghui, https://github.com/janeyx99, https://github.com/EikanWang (cherry picked from commit c09205a)

guangyey requested a review from eqy as a code owner May 17, 2024 10:23

pytorch-bot bot added module: amp (automated mixed precision) autocast module: cpu CPU specific problem (e.g., perf, algorithm) labels May 17, 2024

guangyey added a commit that referenced this pull request May 17, 2024

Deprecate other autocast API

0434437

ghstack-source-id: 139fa79ce287da588c7d6d967371057c9081219c Pull Request resolved: #126527

guangyey added ciflow/trunk Trigger trunk jobs on your pull request ciflow/inductor labels May 17, 2024

guangyey marked this pull request as draft May 17, 2024 10:29

pytorchbot added the open source label May 17, 2024

guangyey mentioned this pull request May 17, 2024

generalize custom_fwd&custom_bwd to be device-agnostic #126531

Closed

guangyey changed the title ~~Deprecate other autocast API~~ Deprecate device-specific GradScaler autocast API May 17, 2024

guangyey added 5 commits May 17, 2024 18:11

Deprecate other autocast API

1e2a33f

[ghstack-poisoned]

guangyey marked this pull request as ready for review May 21, 2024 01:45

guangyey requested review from jgong5, EikanWang, gujinghui and albanD May 21, 2024 01:46

jgong5 approved these changes May 21, 2024

View reviewed changes

gujinghui approved these changes May 21, 2024

View reviewed changes

janeyx99 requested changes May 22, 2024

View reviewed changes

pytorch-bot bot added module: dynamo oncall: distributed Add this issue/PR to distributed oncall triage queue release notes: distributed (sharded) release notes category labels May 22, 2024

janeyx99 reviewed May 22, 2024

View reviewed changes

test/test_torch.py Show resolved Hide resolved

janeyx99 reviewed May 22, 2024

View reviewed changes

test/test_torch.py Outdated Show resolved Hide resolved

janeyx99 reviewed May 22, 2024

View reviewed changes

guangyey requested a review from janeyx99 May 22, 2024 15:54

guangyey added 3 commits May 22, 2024 22:53

guangyey added 2 commits May 23, 2024 10:11

EikanWang approved these changes May 24, 2024

View reviewed changes

janeyx99 reviewed May 24, 2024

View reviewed changes

janeyx99 approved these changes May 24, 2024

View reviewed changes

guangyey added a commit that referenced this pull request May 24, 2024

Deprecate other autocast API

f3b68b0

ghstack-source-id: f454a70e4fddb4af7db42d69d27b1f247004966d Pull Request resolved: #126527

pytorchmergebot added the merging label May 25, 2024

pytorchmergebot closed this in c09205a May 25, 2024

pytorchmergebot added Merged and removed merging labels May 25, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deprecate device-specific GradScaler autocast API #126527

Deprecate device-specific GradScaler autocast API #126527

guangyey commented May 17, 2024 •

edited

pytorch-bot bot commented May 17, 2024 •

edited

janeyx99 left a comment •

edited

janeyx99 left a comment

guangyey commented May 22, 2024

janeyx99 commented May 22, 2024

guangyey commented May 23, 2024

guangyey commented May 23, 2024

janeyx99 May 24, 2024

guangyey May 24, 2024

leslie-fang-intel May 24, 2024

janeyx99 May 24, 2024

guangyey May 24, 2024 •

edited

janeyx99 left a comment

guangyey commented May 24, 2024

guangyey commented May 25, 2024

pytorchmergebot commented May 25, 2024

Deprecate device-specific GradScaler autocast API #126527

Deprecate device-specific GradScaler autocast API #126527

Conversation

guangyey commented May 17, 2024 • edited

Motivation

for torch.amp.GradScaler,

for custom_fwd and custom_bwd,

Additional Context

pytorch-bot bot commented May 17, 2024 • edited

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/126527

✅ You can merge normally! (1 Unrelated Failure)

janeyx99 left a comment • edited

Choose a reason for hiding this comment

janeyx99 left a comment

Choose a reason for hiding this comment

guangyey commented May 22, 2024

janeyx99 commented May 22, 2024

guangyey commented May 23, 2024

guangyey commented May 23, 2024

janeyx99 May 24, 2024

Choose a reason for hiding this comment

guangyey May 24, 2024

Choose a reason for hiding this comment

leslie-fang-intel May 24, 2024

Choose a reason for hiding this comment

janeyx99 May 24, 2024

Choose a reason for hiding this comment

guangyey May 24, 2024 • edited

Choose a reason for hiding this comment

janeyx99 left a comment

Choose a reason for hiding this comment

guangyey commented May 24, 2024

guangyey commented May 25, 2024

pytorchmergebot commented May 25, 2024

Merge started

guangyey commented May 17, 2024 •

edited

for `torch.amp.GradScaler`,

for `custom_fwd` and `custom_bwd`,

pytorch-bot bot commented May 17, 2024 •

edited

janeyx99 left a comment •

edited

guangyey May 24, 2024 •

edited