Evaluate third party models

[thelinuxkid]

Code Pull Requests

Please provide the following:

• a clear explanation of what your code does
• if applicable, a reference to an issue
• a reproducible test for your PR (code, config and data sample)

Documentation Pull Requests

Note that the documentation HTML files are in docs/ while the Markdown sources are in mkdocs/docs.

If you are proposing a modification to the documentation you should change only the Markdown files.

api.md is automatically generated from the docstrings in the code, so if you want to change something in that file, first modify ludwig/api.py docstring, then run mkdocs/code_docs_autogen.py, which will create mkdocs/docs/api.md .

[github-actions]

Unit Test Results

0 files   -   6  0 suites   - 6   0s ⏱️ - 39m 11s
0 tests  - 31  0 ✔️  - 26  0 💤  -   5  0 ❌ ±0 
0 runs   - 82  0 ✔️  - 66  0 💤  - 16  0 ❌ ±0 

Results for commit e0fc119. ± Comparison against base commit ccd8d82.

♻️ This comment has been updated with latest results.

[arnavgarg1]

Hey @thelinuxkid - Super excited to see this draft PR come along! Let us know if there's anything we can help you with, and also when we should start taking a look at the PR to review it!

[thelinuxkid]

Hey @thelinuxkid - Super excited to see this draft PR come along! Let us know if there's anything we can help you with, and also when we should start taking a look at the PR to review it!

Thanks @arnavgarg1 I should have something for review tonight

[thelinuxkid]

Hey @thelinuxkid - Super excited to see this draft PR come along! Let us know if there's anything we can help you with, and also when we should start taking a look at the PR to review it!

Thanks @arnavgarg1 I should have something for review tonight

Hi @arnavgarg1 Everything is in place but I'm having a hard time figuring out how to update the metrics since there's no access to the OpenAI model's prediction metadata. Would love to get your input on this. Thanks!!