Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deepsparse.TextGeneration doesn't accept trust_remote_code as an arg anymore #1639

Open
mgoin opened this issue Mar 26, 2024 · 1 comment
Open
Assignees
Labels
bug Something isn't working

Comments

@mgoin
Copy link
Member

mgoin commented Mar 26, 2024

The legacy text-generation pipeline supported trust_remote_code as an argument, which was inherited from the Transformers base pipeline here:

trust_remote_code: bool = False,

This was useful for models with custom code related to their tokenizer: #1176

Testing code:

from deepsparse import TextGeneration
pipeline = TextGeneration(model="hf:mgoin/llama2.c-stories15M-quant-ds", trust_remote_code=True)
print(pipeline("hello").generations[0].text)

Output:

Fetching 7 files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00, 28042.15it/s]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mgoin/code/deepsparse/src/deepsparse/pipeline.py", line 724, in text_generation_pipeline
    return Pipeline.create("text_generation", **kwargs)
  File "/home/mgoin/code/deepsparse/src/deepsparse/pipeline.py", line 148, in create
    raise e
  File "/home/mgoin/code/deepsparse/src/deepsparse/pipeline.py", line 140, in create
    pipeline = Operator.create(task=task, **new_kwargs)
  File "/home/mgoin/code/deepsparse/src/deepsparse/operators/operator.py", line 137, in create
    return operator_constructor(**kwargs)
  File "/home/mgoin/code/deepsparse/src/deepsparse/transformers/pipelines/text_generation/pipeline.py", line 175, in __init__
    single_engine_operator = NLEngineOperator(
  File "/home/mgoin/code/deepsparse/src/deepsparse/transformers/pipelines/text_generation/nl_engine_operator.py", line 132, in __init__
    super().__init__(**kwargs)
TypeError: EngineOperator.__init__() got an unexpected keyword argument 'trust_remote_code'
@mgoin mgoin added the bug Something isn't working label Mar 26, 2024
@mgoin
Copy link
Member Author

mgoin commented Apr 24, 2024

Hey @bfineran could we get this addressed?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants