Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow easier programmatic use by extracting code from translate.py #1107

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

AmitMY
Copy link

@AmitMY AmitMY commented Mar 26, 2024

Running sockeye models programmatically is desired (serving models) but is difficult.

You have to know what you are doing, and how to initialize the inference Translator class, among all other classes.
This was done by you already in translate.py, the CLI tool you made for translation, and so I was thinking to extract the code from the translate.py functionality and allow programatic access.

This PR makes it possible to run sockeye code by using the same arguments as the CLI tool, thus allowing easy use and reuse of the translator.

from sockeye.translate import parse_translation_arguments, load_translator_from_args
from sockeye.inference import make_input_from_factored_string

args = parse_translation_arguments(["-m", model_path])
translator = load_translator_from_args(args, True)
strings = ["my|PRP name|NN is|VBZ John|NN"]
inputs = [inference.make_input_from_factored_string(sentence_id=i,
                                                    factored_string=s,
                                                    translator=translator)
          for i, s in enumerate(strings)]
outputs = translator.translate(inputs)

Pull Request Checklist

  • Changes are complete (if posting work-in-progress code, prefix your pull request title with '[WIP]'
    until you can check this box.
  • Unit tests pass (pytest)
no, but i suspect it is because i am using a mac
  • System tests pass (pytest test/system)
  • Passed code style checking (./style-check.sh)
************* Module sockeye.layers
sockeye/layers.py:766:28: E1136: Value 'self.weight.unsqueeze(0)' is unsubscriptable (unsubscriptable-object)
pylint: Command line or configuration file:1: UserWarning: 'Exception' is not a proper value for the 'overgeneral-exceptions' option. Use fully qualified name (maybe 'builtins.Exception' ?) instead. This will cease to be checked at runtime when the configuration upgrader is released.
************* Module test.unit.test_deepspeed
test/unit/test_deepspeed.py:27:4: E0401: Unable to import 'deepspeed' (import-error)
./style-check.sh: line 14: mypy: command not found
  • You have considered writing a test
  • Updated major/minor version in sockeye/__init__.py. Major version bump if this is a backwards incompatible change.
  • Updated CHANGELOG.md

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@AmitMY
Copy link
Author

AmitMY commented May 17, 2024

This is very useful for me, in multiple projects now.
any chance this gets in the main repo?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant