Creates Patch #2720
Open
Creates Patch #2720
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This creates the Patch concept along with some start of usages. There is a more specialized ParamPatch for the standard parameter additive patches and a Scaled, Basic, and LoRA implementation. The patches can be created directly, by comparing models, and from gradients. This is an initial step. Following this, there are a few pieces of work that could be considered: 1. DJL Serving Python engine specific patch implementation 2. LoRA for full training 3. Make BasicParamPatch from Optimizer (including gradients, momentum, and lr) Additionally, I included some changes to the IntegrationTest. I ran into the dumb issue where I made the tests private which makes them unable to run from IntegrationTest. Worse, the exceptions had no cause and therefore they wouldn't print any message or run to give feedback through println or logger. It still runs fine in IntelliJ too, making this issue only show up through gradle. After this change, it would provide a clear exception message which makes this easy to debug in the future.
Codecov ReportPatch coverage:
❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the GitHub App Integration for your organization. Read more. Additional details and impacted files@@ Coverage Diff @@
## master #2720 +/- ##
============================================
+ Coverage 72.08% 72.13% +0.04%
- Complexity 5126 7107 +1981
============================================
Files 473 706 +233
Lines 21970 31599 +9629
Branches 2351 3265 +914
============================================
+ Hits 15838 22795 +6957
- Misses 4925 7252 +2327
- Partials 1207 1552 +345
☔ View full report in Codecov by Sentry. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This creates the Patch concept along with some start of usages. There is a more specialized ParamPatch for the standard parameter additive patches and a Scaled, Basic, and LoRA implementation. The patches can be created directly, by comparing models, and from gradients.
This is an initial step. Following this, there are a few pieces of work that could be considered:
Additionally, I included some changes to the IntegrationTest. I ran into the
dumb issue where I made the tests private which makes them unable to run from
IntegrationTest. Worse, the exceptions had no cause and therefore they wouldn't
print any message or run to give feedback through println or logger. It still
runs fine in IntelliJ too, making this issue only show up through gradle. After
this change, it would provide a clear exception message which makes this easy to
debug in the future.