Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Copilot No. Series] Number series copilot #764

Open
wants to merge 48 commits into
base: main
Choose a base branch
from

Conversation

DmitryKatson
Copy link
Contributor

@DmitryKatson DmitryKatson commented Mar 16, 2024

Summary

Introducing the first [draft] version of a "Number Series Copilot" for Business Central, an innovative feature designed to simplify the creation of number series through natural language commands. This allows users to effortlessly set up number series for specific entities like customers, vendors, or even across all entities, by simply describing their needs or specifying a format example.

Current version is intent to create number series for NEW company, where no Number Series exist yet.

Number.Series.Copilot.-.New.company.Scenario.mp4

This flow was tested on the Set up numbers series for the new company like user prompts

Please follow this diagram (green flow), to get the idea of the ongoing process.

diagram-export-16-03-2024-13_39_19

Key ideas:

  • Azure OpenAI Tools calling is used to understand the user intent
  • AL is used to gather data from BC and build tool response
  • The new company flow requires about 120 different number series to be generated. Nor GPT-4, not GPT 3.5 turbo (latest) where possible to generate accurate json array with 1 go
  • Thus, instead of 1 LLM call to build 120 number series, the tool response is been chunked, to generate not more than 10 number series in one go
  • All LLM generations results (12 in total, each requires 5-9 sec of GPT 3.5 time) then been combined in 1 answer, which is presented to the user

Setup:

  • Use No. Series with Copilot Setup to set up Azure Open AI connection and prompts
  • Prompts should be provided separately (using other channels) due to security reasons
  • Model used: gpt-35-turbo-1106
image

Work Item(s)

Fixes #659
Fixes AB#503148

…ut AI implementation business logic)

Added new app components and capabilities for the No. Series Copilot app, including codeunits, page extensions, pages, and tables, to introduce AI features for generating number series. Additionally, new registers and installers were included to handle capability registration and app installation. This enhances the app's functionality and extensibility.
Update page and table numbers as there was overflow with the Base Application.
- Temporary adding Number Series Copilot Setup table and page to store aoai secrets, system prompt and functions prompt in the isolated storage.
- Add Jupyter Notebook patterns to gitignore.
- Updating codeunit, page, and table objects numbers.
The code refactor improves tool retrieval, adds tool ID handling, and updates UI elements. The changes aim to enhance the No. Series Copilot tool functionality and user experience. These alterations are crucial for integrating Azure Key Vault in future.

Include tool calls to the chat history

- Add support for tool call IDs in chat history, improving traceability and accountability for tool call results. The changes enable associating tool call IDs with tool messages in the chat history, enhancing the clarity and transparency of the interactions. This update aligns with the need to provide comprehensive context for tool call responses within the chat history.
- Introduce 'AddToolMessage' method and modify existing methods to capture tool-related data for chat messages.
- Add 'HistoryToolCallIds' list to save tool call IDs for messages.
- Create 'IsToolsList' method to check if a message contains tool-related information.
- Update 'AOAIChatRoles.Enum.al' to define a new 'tool' chat role.

These changes enable the tracking and differentiation of tool call results in chat history.
Enable support for returning valid JSON object as chat completion, adding guidelines for JSON production, and checking compatibility for JSON response format.
Improve json output format specification declaration and [TEMPORARY] implement no. series number limitations to avoid reaching token limit and timeout when dealing with many tables.
Consolidates tool invocation logic by using a dedicated function and refines prompt formats for better user experience. Also updates existing patterns to reflect more accurate numbering conventions. These changes aim to streamline tool usage and enhance prompt clarity.
Consolidate tool retrieval methods; refactor output format instructions.
Improved modularity and maintainability.
Add new Json module to provide tools for working with JSON data, including reading, writing, and parsing JSON. Introduces codeunit for initializing JSON array and object, retrieving element counts, fetching objects by index, and getting values for specified record fields. Implements methods to interact with JSON data using .NET types.

This is partly ported from Base Application codeunit 5459 "Json management"
Additionally, the visibility of response text on the No. Series Proposal Page is dynamically managed, depending on json response validation
Tools message parsing into `function name`, `arguments` and `tool call id`
Replace direct parsing with dedicated method for improved maintainability and readability. This enhances code structure and sets the stage for future extensibility. No related issues.
…f the tool response exceeds token limit.

Ensure secure handling of sensitive data in chat messages by swapping text types with secure text types. Implement chunking of tool responses to avoid exceeding token limits, resulting in improved prompt generation for new number series. Indirectly address potential security vulnerabilities and enhance system functionality.
Change the dictionary to standardize chunk processing of tables in order to avoid hallucinations and effectively handle large volumes. This ensures accurate and complete model responses, enhancing overall functionality. The refactor also introduces new helper functions to process tool completions and validate generated number series.
…nd output examples from setup

Introduce methods to retrieve tool 1 patterns and output examples from setup. Also, add labels and separators for better formatting. Improve error handling and optimize code readability.
Improved retrieval of tool 1 general instructions and output format for better modularity and maintainability.
…dle duplicate No. Series codes

This commit refactors JSON manipulation by separating common JSON operations into a separate file and introduces new functionality to handle duplicate No. Series codes using random character generation. This change improves data integrity and ensures uniqueness of No. Series codes. It adds methods to replace or add properties in JSON objects and modify JSON arrays as necessary.
Replace text prompts for tool limitations, code guidelines, description guidelines, number guidelines, output examples, and output format prompts. Add validation for property lengths in generating number series.
Refactor JSON handling methods to return JSON objects and arrays as text, enhancing readability and modularity across the codebase, paving the way for simplified integration with other modules. This change sets the stage for easier maintenance and future expansions of the JSON-related functionalities. No related issues are involved.
Consolidate actions into groups and modify captions for clarity and consistency. Update labels for better clarity and usability. Improve organization and usability of the prompt guide.
@DmitryKatson
Copy link
Contributor Author

@darjoo i found a bug in the prompt dialog rendering. In the content area if the field has dynamic visibility, like here IsResponseTextVisible, when it has value and visible, the value is not shown in the screen. If the visibility is static, all fine.

@DmitryKatson per our documentation, Dynamic Visibility only works on groups, parts, and actions. It does not work on fields.

The workaround is to wrap the field in a group with showCaption = false, though this has other visual implications that may not be desired.

image

Thanks! That worked

@DmitryKatson
Copy link
Contributor Author

@JesperSchulz @darjoo
Just finished this PR. I think it's ready for review.

  • Added new UI components: text placeholder and prompt guides
  • Made possible to edit copilot proposals, before saving
  • Refactored code: spitted big No. Series Copilot Impl. into tool calling modules
  • Refined prompts, for better results and less retries. (Still need some way to pass to you)

In general, it works as I expected: user can create new and modify existing no series with AI :)

new-modify-no-series-with-prompt-guides.mov

@JesperSchulz
Copy link
Contributor

@JesperSchulz @darjoo Just finished this PR. I think it's ready for review.

  • Added new UI components: text placeholder and prompt guides
  • Made possible to edit copilot proposals, before saving
  • Refactored code: spitted big No. Series Copilot Impl. into tool calling modules
  • Refined prompts, for better results and less retries. (Still need some way to pass to you)

In general, it works as I expected: user can create new and modify existing no series with AI :)

new-modify-no-series-with-prompt-guides.mov

Wow, we better get busy then! But as discussed, having created the PR is just the very beginning! Now all the fun begins to live up to our responsible AI rules 🙂 We'll be in touch! In very close touch 😉

…e" instead of "Set up" for uniformity and clearer user understanding.
@DmitryKatson
Copy link
Contributor Author

Not sure if i did everything correctly. But due to the latest Tools Calling Implementation refactoring, this PR needs to be reimplemented.

- Update codeunit number and merge with existing AOAIOperationResponse
…odule changes

Remove redundant tool parsing method, move functionality to the corresponding implementation. Centralize tool parsing code logic in the "AOAIToolsImpl" codeunit and remove the "AOAIChatMessages" implementation. This consolidation streamlines maintenance and enforces a single source of truth for tool-related operations.
…hance AI tool usage

Incorporate GetNoSeriesSuggestions method to improve the AI tool usage for generating number series suggestions in NoSeriesCopilot. This update aims to streamline the process and increase the accuracy of generating number series proposals using the copilot capability.
@DmitryKatson
Copy link
Contributor Author

DmitryKatson commented Apr 19, 2024

@darjoo @JesperSchulz Finished the refactoring, to make it compatible with latest AOAI module changes. Ready for review.

Also refined a bit of UI and Prompt. As I understood correctly Prompting page group is not visible in docker env.

@@ -0,0 +1,63 @@
pageextension 324 "No. Series Ext" extends "No. Series"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the note of namespaces: we need those ;-)

codeunit 324 "No. Series Copilot Impl."
{
var
IncorrectCompletionErr: Label 'Incorrect completion. The property %1 is empty';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Placeholders must have a comment.

SystemPromptTxt: SecretText;
CompletePromptTokenCount: Integer;
Completion: Text;
TokenCountImpl: Codeunit "AOAI Token";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should order all of your variables properly, or CodeCop will be on your neck ;-)

local procedure GenerateNoSeries(SystemPromptTxt: SecretText; InputText: Text): Text
var
AzureOpenAI: Codeunit "Azure OpenAi";
AOAIDeployments: Codeunit "AOAI Deployments";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused. You should enable the CodeCops to get all of minor issues sorted out. Won't mention further of those.

Adjust code to allow for dynamic removal of multiple text parts, requires for better promptin building.
@DmitryKatson
Copy link
Contributor Author

1 min 20 sec.
This time you need now to configure No Series with Copilot for the whole new company.

ns-copilot-speed.mp4

@JesperSchulz JesperSchulz added the Integration GitHub request for Integration area label Apr 22, 2024
@DmitryKatson DmitryKatson changed the title [Copilot No. Series] [DRAFT] Number series copilot [Copilot No. Series] Number series copilot Apr 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
AL: Business Foundation From Fork Pull request is coming from a fork Integration GitHub request for Integration area
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BC Idea]: Number Series Copilot
8 participants