New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support incremental state for E2E ASR experiments #173
base: 3.0.0-dev
Are you sure you want to change the base?
Conversation
change weight-c-init from Optional[float] = None to float = 1.0
normalize_after: bool, optional | ||
if True, apply post layer normalization; otherwise apply pre layer normalization | ||
(default=True). | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would you mind adding the doc string of the right_window
also? Just wanted to make sure the legal range of this parameter.
experiments/e2e_asr/extension.py
Outdated
state["saved_query"] = query_not_ready | ||
state["saved_key"] = k | ||
state["saved_value"] = v | ||
incremental_state["attn_state"] = state |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When there are no ready queries, would it be more efficient to just return the empty tensor here?
experiments/e2e_asr/extension.py
Outdated
input: Tensor, | ||
mask_pad: Optional[Tensor] = None, | ||
attn_mask: Optional[Tensor] = None, | ||
incremental_state: Optional[Dict[str, Dict[str, Optional[Tensor]]]] = None, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you change data structure of incremental_state
to Optional[Dict[str, Optional[Tensor]]
? Seems the SRUCell does use the incremental_state
and I should write some glue codes inheriting from the fair_seq's incremental class, and assign & maintain a layer specific hash as the key to this dictionary, in stead of attn_state
experiments/e2e_asr/extension.py
Outdated
c0: Optional[Tensor] = None, | ||
mask_pad: Optional[Tensor] = None, | ||
attn_mask: Optional[Tensor] = None, | ||
incremental_state: Optional[Dict[str, Dict[str, Optional[Tensor]]]] = None, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same^
No description provided.