Default off_prompt=False on BaseTool
#757
Conversation
collindutter
force-pushed
the
refactor/task-memory-default
branch
5 times, most recently
from
f30a0ed
to
8e7b22d
Compare
| @@ -56,7 +56,7 @@ mongo_driver = AzureMongoDbVectorStoreDriver( | |||
|
|
|||
| loader = Agent( | |||
| tools=[ | |||
| WebScraper() | |||
| WebScraper(off_prompt=True), | |||
There was a problem hiding this comment.
That's the nifty part about this example, it's a separate Agent using the same shared Task/Meta Memory instances. So this is correct.
| @@ -103,7 +103,7 @@ agent = Agent() | |||
| agent.add_task( | |||
| ToolkitTask( | |||
| "Load https://www.griptape.ai, summarize it, and store it in a file called griptape.txt", | |||
| tools=[WebScraper(), FileManager(), TaskMemoryClient(off_prompt=False)] | |||
| tools=[WebScraper(off_prompt=True), FileManager(off_prompt=True), TaskMemoryClient(off_prompt=True)] | |||
There was a problem hiding this comment.
Should be False for TaskMemoryClient.
There was a problem hiding this comment.
FileManager will read directly from Task Memory
|
|
||
| ## Off Prompt | ||
|
|
||
| You can enable or disable Task Memory on a Tool with the `off_prompt` parameter. When `off_prompt` is set to `True`, the Tool will store its output in Task Memory. When `off_prompt` is set to `False`, the Tool will return its output directly to the LLM. |
There was a problem hiding this comment.
If it's mandatory to determine whether or not TaskMemory should be true or false, should we say "You must enable or disable..."?
|
|
||
| Note that in this example GPT-4 _never_ saw the contents of the page, only that it was stored in Task Memory. | ||
|
|
||
| !!! info |
There was a problem hiding this comment.
This didn't display properly in the preview
There was a problem hiding this comment.
I think this will only work on the doc preview, not Github.
|
|
||
| * **Security requirements**: many organizations don't want data to leave their cloud for regulatory and security reasons. | ||
| * **Long textual content**: when textual content returned by tools can't fit in the token limit, it's often useful to perform operations on it in a separate process, not in the main LLM. | ||
| * **Non-textual content**: tools can generate images, videos, PDFs, and other non-textual content that can be stored in memory and acted upon later by other tools. | ||
| * **Long textual content**: when textual content returned by Tools can't fit in the token limit, it's often useful to perform operations on it in a separate process, not in the main LLM. |
There was a problem hiding this comment.
Should probably not say "process" since it's still the same system process. Can't think of a better word though right now.
There was a problem hiding this comment.
suggestions: operation, thread, or action?
| [04/26/24 13:07:16] ERROR Subtask ecbb788d9830491ab72a8a2bbef5fb0a | ||
| Invalid action JSON: Or({Literal("name", description=""): 'Calculator', Literal("path", description="Can be used for computing simple | ||
| numerical or algebraic calculations in Python"): 'calculate', Literal("input", description=""): {'values': Schema({Literal("expression", | ||
| description="Arithmetic expression parsable in pure Python. Single line only. Don't use variables. Don't use any imports or external | ||
| libraries"): <class 'str'>})}, Literal("tag", description="Unique tag name for action execution."): <class 'str'>}) did not validate {'name': | ||
| 'Memory', 'path': 'get', 'input': {'memory_name': 'TaskMemory', 'artifact_namespace': '6be74c5128024c0588eb9bee1fdb9aa5'}, 'tag': | ||
| 'get_sqrt_result'} | ||
| Key 'name' error: | ||
| 'Calculator' does not match 'Memory' |
There was a problem hiding this comment.
Should we truncate this as well and just say "errors are generated" or something?
There was a problem hiding this comment.
We should probably leave the actual error message the customer would see in the example, but fine with truncating any of the middle.
There was a problem hiding this comment.
I think it's important to show a full error message, I already truncated quite a bit of other output.
| ...Output truncated for brevity... | ||
| ``` | ||
|
|
||
| When we set `off_prompt` to `True`, the Agent does not function as expected, even generating an error. This is because because the Calculator output is being stored in Task Memory but the Agent has no way to access it. To fix this, we need the `TaskMemoryClient`. |
| @@ -1,89 +1,320 @@ | |||
| ## Overview | |||
| Task Memory augments activity inputs and outputs with storage capabilities. It's a way to keep any data generated by tools off prompt but still allow LLMs to operate on that data remotely. This is useful in the following scenarios: | |||
|
|
|||
| Task Memory is a powerful feature of Griptape that allows you to control where the data returned by Tools is stored. This is useful in the following scenarios: | |||
There was a problem hiding this comment.
How does "the data returned by Tools" relate to the data returned by the LLM? It may be useful to add context in relation to the model as that is what most new users will be familiar with.
There was a problem hiding this comment.
Not sure I totally follow the question, can you please elaborate a bit more on this?
|
|
||
| * **Security requirements**: many organizations don't want data to leave their cloud for regulatory and security reasons. | ||
| * **Long textual content**: when textual content returned by tools can't fit in the token limit, it's often useful to perform operations on it in a separate process, not in the main LLM. | ||
| * **Non-textual content**: tools can generate images, videos, PDFs, and other non-textual content that can be stored in memory and acted upon later by other tools. | ||
| * **Long textual content**: when textual content returned by Tools can't fit in the token limit, it's often useful to perform operations on it in a separate process, not in the main LLM. |
There was a problem hiding this comment.
suggestions: operation, thread, or action?
|
|
||
| You can enable or disable Task Memory on a Tool with the `off_prompt` parameter. When `off_prompt` is set to `True`, the Tool will store its output in Task Memory. When `off_prompt` is set to `False`, the Tool will return its output directly to the LLM. | ||
|
|
||
| By default, all Tools have `off_prompt` as a required parameter, forcing you to consider whether you'd like to use Task Memory or not. Lets look at a simple example where `off_prompt` is set to `False`: |
There was a problem hiding this comment.
nit: I would put the info about the default up in the previous paragraph as it puts all the base information a customer needs up front.
You can enable or disable Task Memory on a Tool with the
off_promptparameter. By default, all Tools haveoff_promptas a required parameter, forcing you to consider whether you'd like to use Task Memory or not. Whenoff_promptis set toTrue...
| [04/26/24 13:07:16] ERROR Subtask ecbb788d9830491ab72a8a2bbef5fb0a | ||
| Invalid action JSON: Or({Literal("name", description=""): 'Calculator', Literal("path", description="Can be used for computing simple | ||
| numerical or algebraic calculations in Python"): 'calculate', Literal("input", description=""): {'values': Schema({Literal("expression", | ||
| description="Arithmetic expression parsable in pure Python. Single line only. Don't use variables. Don't use any imports or external | ||
| libraries"): <class 'str'>})}, Literal("tag", description="Unique tag name for action execution."): <class 'str'>}) did not validate {'name': | ||
| 'Memory', 'path': 'get', 'input': {'memory_name': 'TaskMemory', 'artifact_namespace': '6be74c5128024c0588eb9bee1fdb9aa5'}, 'tag': | ||
| 'get_sqrt_result'} | ||
| Key 'name' error: | ||
| 'Calculator' does not match 'Memory' |
There was a problem hiding this comment.
We should probably leave the actual error message the customer would see in the example, but fine with truncating any of the middle.
| Output: The square root of 12345 is approximately 111.108. | ||
| ``` | ||
|
|
||
| Note that on the `TaskMemoryClient` we've set `off_prompt` to `False` so that the results of the query can be returned directly to the LLM. If we had kept it as `True`, the results would have been stored back Task Memory which would've put us back to square one. |
There was a problem hiding this comment.
nit: I'd put this comment up above the example as it was my first question when reading the example and was then confused until I got here.
| agent = Agent( | ||
| tools=[ | ||
| WebScraper(off_prompt=True), | ||
| TaskMemoryClient(off_prompt=False), |
There was a problem hiding this comment.
We may have covered this already in a separate conversation, but what is the value of TaskMemoryClient having off_prompt at all? would it every be true?
There was a problem hiding this comment.
Yes, the Sensitive Data example has it set to true. Will add more details.
|
|
||
| Here is an example where we use GPT-4 to orchestrate the Tools and store the data in Task Memory, and a self-hosted Llama-2 model to summarize the raw content. | ||
|
|
||
| ```python |
There was a problem hiding this comment.
This example may be good to copy into our Recipes section of our docs as well. I could see this being useful for people at a top-level example of the power of Griptape vs. needing to find it buried in our docs.
| Note that in this example GPT-4 _never_ saw the contents of the page, only that it was stored in Task Memory. | ||
|
|
||
| !!! info | ||
| In this instance of using `FileManager` to save a file to disk, it does not matter what the value of `off_prompt` is set to. This Tool Activity returns an `InfoArtifact` which Griptape will not store in Task Memory. Griptape will only store `TextArtifact`'s, `BlobArtifact`'s, and `ListArtifact`'s in Task Memory. |
There was a problem hiding this comment.
This may be good to call out with a section header like ## When using FileManager or ## Notes on InfoArtifacts.
|
|
||
| ## Tools That Use Task Memory | ||
|
|
||
| As seen in the previous example, certain Tools are designed to read directly from Task Memory. This means that you can use these Tools to interact with the data stored in Task Memory without needing to pass it through the LLM. |
There was a problem hiding this comment.
Does this mean a user does not need to specify off_prompt=true when using these tools?
There was a problem hiding this comment.
Added more details.
collindutter
force-pushed
the
refactor/task-memory-default
branch
from
a918832
to
f4f44b4
Compare
collindutter
force-pushed
the
refactor/task-memory-default
branch
from
f4f44b4
to
b0055b8
Compare
collindutter
force-pushed
the
refactor/default-configs
branch
from
2ff3632
to
30bf66a
Compare
collindutter
force-pushed
the
refactor/default-configs
branch
2 times, most recently
from
e0e0adb
to
8f14657
Compare
collindutter
force-pushed
the
refactor/default-configs
branch
from
b664e94
to
fc427c7
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅ 📢 Thoughts on this report? Let us know! |
collindutter
force-pushed
the
refactor/task-memory-default
branch
from
0729ecb
to
78d58cd
Compare
There was a problem hiding this comment.
The TaskMemory docs look great and have some awesome examples.
I am questioning the over-arching change of removing the off_prompt default for the rest of this PR. I feel like we should default to something as removing the default makes the barrier to getting started w/ Griptape that much more complicated for customers.
No description provided.