Add support for HGQ proxy model #914
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
calad0i
force-pushed
the
HGQ-integration
branch
from
7bd57b7
to
e7aef01
Compare
calad0i
force-pushed
the
HGQ-integration
branch
2 times, most recently
from
5301b4d
to
97f4eb2
Compare
calad0i
force-pushed
the
HGQ-integration
branch
from
1430729
to
b29cfdf
Compare
jmitrevs
added
please test
calad0i
force-pushed
the
HGQ-integration
branch
2 times, most recently
from
c14e299
to
a24bfc2
Compare
jmitrevs
added
please test
latency pooling overhaul vivado latency pooling overhaul vitis latency pooling overhaul, fix comment fix boundry cond fix syn issues latency pooling overhaul Fix pooling accum_t autoset & avoid global override [pre-commit.ci] auto fixes from pre-commit hooks better way to get inp layer name fix for vitis / input_t fetch torch padding fix avoid name dup in torch api test rm pooling precision override in favor of fastmachinelearning#855
Support UnaryLUT w/ vivado&io_parallel hotfix format support vivado_stream fix syn issue fix syn issue2 Revert unnecessary "fix syn issue1/2" Revert "fix syn issue2" This reverts commit af3c347. Revert "fix syn issue" This reverts commit 532cb9d. rm redundant pipeline pragma unary lut vitis fix unary lut vitis fix - leftover
calad0i
force-pushed
the
HGQ-integration
branch
from
f3d7fb5
to
2644786
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Requires #973 and
python>=3.10now.Description
This PR adds support for a special layer type
FixedPointQuantizerused in calad0i/HGQ/hls4ml-integration to facilitate conversion from HGQ model to hls4ml model graph. ModelGraph converted from proxy-model is meant to be bit-accurate, untilfp32(ortf32) cannot emulate the required precision during tensorflow model inference.The layer has three major functions:
fp32can represent the required precision, the proxy model can be used to emulate hls models' output, with or without overflows withSATorWRAP(other overflow modes not tested).In the future, the proxy model mechanism can also be used with Qkeras models, but this part is not yet implemented.
#912 could still break bit-accuracy for pooling layer with
io_stream. Temporary fix available at #917 (included), superseded by #855 when available.New (09/01/24):
Rebased to current master and squashed commits for basic HGQ proxy (also supports QKeras partially) support.
Allowing converting any unary activation function to a LUT, ensuring bit-accuracy for all unary activation functions. Performance in resource and timing may be better or worse depending on the input data format.
TODO
If the python version for hls4ml bumps to
>=3.10, we can make HGQ an optional dependency for hls4ml and re-use tests from there. We can remove the re-defining ofFixedPointQuantizerin hls4ml also.Independent of this PR,
Conv2Din Quartus with3x3filter size withio_parallelseems to be broken at the moment, some tests are expected to fail at the moment. The cause seems to be related to the winograd implementation.This PR dependents on #887 #906 #907 #908 #909 #911All merged now.Type of change
Tests
test/pytest/test_proxy_model.pyTest Configuration:
Requires models add in fastmachinelearning/example-models#11.
Checklist
pre-commiton the files I edited or added.