Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[xh]Optimize prompt for block params generation in HuggingFace #4474

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

matrixstone
Copy link
Collaborator

Description

For the task to generate different block params: block_type, pipeline_type, action_code, etc based on block description,
this PR optimizes it by split one prompt to multiple small prompt for individual prompt and run them in parallel.
Since hugging face charge LLM instance by time not number of request, this change does not affect money spending.

However because we can tailer and focus on prompt for each different param, I got 100% correctness with 10 examples I tried: https://docs.google.com/spreadsheets/d/1-fongvJv0lrtNp06wbnDqxmPNp66lCBJ-8TVKuftDds/edit#gid=0
Previously it is 70%.

How Has This Been Tested?

  1. Tested with commands. Result recorded in https://docs.google.com/spreadsheets/d/1-fongvJv0lrtNp06wbnDqxmPNp66lCBJ-8TVKuftDds/edit#gid=0
  2. Tested with Mage UI. Screenshot:
    WX20240127-121814@2x

Checklist

  • The PR is tagged with proper labels (bug, enhancement, feature, documentation)
  • I have performed a self-review of my own code
  • I have added unit tests that prove my fix is effective or that my feature works
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • If new documentation has been added, relative paths have been added to the appropriate section of docs/mint.json

cc:
@wangxiaoyou1993 @dy46

@matrixstone matrixstone added the feature New feature or request label Jan 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant