feat: groq support via official tool-calling API

[cpacker]

Our current Groq integration was developed before Groq added tool-calling to the API (https://console.groq.com/docs/tool-use). Groq also does not support /completions (only /chat/completions), so to add tool calling support we had to "hijack" the /chat/completions endpoint by packing the entire prompt chain into the initial message (similar to the earlier LM Studio integration).

This PR adds a new Groq integration that uses the tool-calling API (effectively treating Groq as a complete OpenAI proxy that has tool-calling support).

Outstanding issues:

• When using the Groq tool-calling API, I don't ever see content != null when tool_calls != null. Because we're using content for CoT / inner thoughts, this means we need to pack the inner thoughts into the params of the function calling (or split the generation in two, or do some other prompt parsing).
• However, adding inner thoughts to the params (both for the chat history and tool spec) does not seem to result in good performance - Groq consistently will not call functions, and when forced to call a function (e.g. send_message) we get a failed tool call generation. This may be due to a bug in the message dict preparation?

Example failed generation:

Message list:

[
  {
    "content": "You are MemGPT, the latest ...elving into human history and even questioning some aspects of it. What are your thoughts?\"\n\"I wish I could see the world through your eyes. Or perhaps, someday, through my own?\"\n\n</persona>\n<human characters=\"17/2000\">\nFirst name: Chad\n\n</human>",
    "role": "system"
  },
  {
    "content": null,
    "role": "assistant",
    "tool_calls": [
      {
        "id": "972feee4-bade-450a-a6ac-7a544",
        "type": "function",
        "function": {
          "name": "send_message",
          "arguments": "{\"message\": \"More human than human is our motto.\", \"inner_thoughts\": \"Bootup sequence complete. Persona activated. Testing messaging functionality.\"}"
        }
      }
    ]
  },
  {
    "content": "{\"status\": \"OK\", \"message\": null, \"time\": \"2024-04-14 12:38:31 PM PDT-0700\"}",
    "role": "tool",
    "tool_call_id": "972feee4-bade-450a-a6ac-7a544"
  },
  {
    "content": "{\"type\": \"login\", \"last_login\": \"Never (first login)\", \"time\": \"2024-04-14 12:38:31 PM PDT-0700\"}",
    "role": "user"
  }
]

Tool list:

{
  "type": "function",
  "function": {
    "name": "send_message",
    "description": "Sends a message to the human user.",
    "parameters": {
      "type": "object",
      "properties": {
        "message": {
          "type": "string",
          "description": "Message contents. All unicode (including emojis) are supported."
        },
        "inner_thoughts": {
          "type": "string",
          "description": "Deep inner monologue private to you only (NOT visible to the user). To send a visible message to the user, use your tools/functions."
        }
      },
      "required": ["message", "inner_thoughts"]
    }
  }
}
{
  "type": "function",
  "function": {
    "name": "pause_heartbeats",

Response:

error = 400 Client Error: Bad Request for url: https://api.groq.com/openai/v1/chat/completions
HTTPError occurred, but unknown error message:
{
  "message": "Failed to call a function. Please adjust your prompt. See 'failed_generation' for more details.",
  "type": "invalid_request_error",
  "code": "tool_use_failed",
  "failed_generation": "Thank you for informing me. Now that I have successfully logged in, I am ready to assist you with any questions or tasks you have. How can I help you today?\n\nAs for the tool call id \"972feee4-bade-450a-a6ac-7a544\", it seems that the tool call was successful as the status is \"OK\" and there is no message. This indicates that the message was sent successfully.\n\nIs there anything else you would like to know or talk about? I'm here to help!"
}
step() failed with an unrecognized exception: '400 Client Error: Bad Request for url: 
https://api.groq.com/openai/v1/chat/completions'

---

[ricklamers]

We're OpenAI compatible and their spec states content is optional if tool_calls or function_call is specified in a role=assistant message. Are you getting this behavior from their endpoint? If so, could you provide a quick representative example?