You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
defget_peft_state_maybe_zero_3(named_params, bias):
ifbias=="none": # no bias mode, only returns lora weightsto_return= {k: tfork, tinnamed_paramsif"lora_"ink}
# Filters and returns only the parameters that include "lora_" in their names.elifbias=="all": # all bias mode, return all biasesto_return= {k: tfork, tinnamed_paramsif"lora_"inkor"bias"ink}
# Filters and returns parameters that include either "lora_" or "bias" in their names.elifbias=="lora_only": # return biases only from lorato_return= {}
maybe_lora_bias= {}
lora_bias_names=set()
fork, tinnamed_params:
if"lora_"ink:
to_return[k] =t# store lora weightbias_name=k.split("lora_")[0] +"bias"# store lora module's namelora_bias_names.add(bias_name)
elif"bias"ink:
maybe_lora_bias[k] =t# temporally store all biases# ----------------------------------------------------------fork, tinmaybe_lora_bias:
ifbias_nameinlora_bias_names: # check the namesto_return[bias_name] =t# -----------------------------------------------------------else:
raiseNotImplementedErrorto_return= {k: maybe_zero_3(v, ignore_status=True) fork, vinto_return.items()}
returnto_return
The code inside the #---# box, shouldn't it be
fork, tinmaybe_lora_bias:
ifkinlora_bias_names: # check the namesto_return[k] =t
or
forbias_name, tinmaybe_lora_bias:
ifbias_nameinlora_bias_names: # check the namesto_return[bias_name] =t
The text was updated successfully, but these errors were encountered:
Question
The code inside the #---# box, shouldn't it be
or
The text was updated successfully, but these errors were encountered: