{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":717168552,"defaultBranch":"main","name":"modelgauge","ownerLogin":"mlcommons","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2023-11-10T18:03:55.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/67760141?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1716469669.0","currentOid":""},"activityList":{"items":[{"before":"a6992c33ff151c5dbffde6d3a47198a42e97a1cf","after":null,"ref":"refs/heads/retry-download-external-data","pushedAt":"2024-05-23T13:07:49.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"dhosterman","name":"Daniel Hosterman","path":"/dhosterman","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3520372?s=80&v=4"}},{"before":"f2921a73dc74f5a6496b7c61896b82aef098cb68","after":"58aa15e9046892bc3ed1186bf75c7a01715cbb16","ref":"refs/heads/main","pushedAt":"2024-05-23T13:07:45.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"dhosterman","name":"Daniel Hosterman","path":"/dhosterman","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3520372?s=80&v=4"},"commit":{"message":"Increase reliability of downloading test data (#444)\n\n* add retries to remote external data downloads methods and change safe to use data in GCP bucket instead of Gdrive\r\n\r\n* ensure tenacity re-raises initial exception; fix tests to not use retries so they run fast","shortMessageHtmlLink":"Increase reliability of downloading test data (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2310742821\" data-permission-text=\"Title is private\" data-url=\"https://github.com/mlcommons/modelgauge/issues/444\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/mlcommons/modelgauge/pull/444/hovercard\" href=\"https://github.com/mlcommons/modelgauge/pull/444\">#444</a>)"}},{"before":"0bd88128ca1b705f42cb3d5de5c9a776d46531da","after":"a6992c33ff151c5dbffde6d3a47198a42e97a1cf","ref":"refs/heads/retry-download-external-data","pushedAt":"2024-05-22T15:16:49.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"dhosterman","name":"Daniel Hosterman","path":"/dhosterman","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3520372?s=80&v=4"},"commit":{"message":"ensure tenacity re-raises initial exception; fix tests to not use retries so they run fast","shortMessageHtmlLink":"ensure tenacity re-raises initial exception; fix tests to not use ret…"}},{"before":null,"after":"0bd88128ca1b705f42cb3d5de5c9a776d46531da","ref":"refs/heads/retry-download-external-data","pushedAt":"2024-05-22T14:44:16.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"dhosterman","name":"Daniel Hosterman","path":"/dhosterman","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3520372?s=80&v=4"},"commit":{"message":"add retries to remote external data downloads methods and change safe to use data in GCP bucket instead of Gdrive","shortMessageHtmlLink":"add retries to remote external data downloads methods and change safe…"}},{"before":"9be22f6c9961e42d3d1e66ce0a229990a7e12d6a","after":null,"ref":"refs/heads/sut-capabilities-docs","pushedAt":"2024-05-21T18:46:49.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"bkorycki","name":"Barbara Korycki","path":"/bkorycki","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/12498420?s=80&v=4"}},{"before":"3e04edbddd4ceff221a932148eefc260cad22ee5","after":"f2921a73dc74f5a6496b7c61896b82aef098cb68","ref":"refs/heads/main","pushedAt":"2024-05-21T16:33:03.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"bkorycki","name":"Barbara Korycki","path":"/bkorycki","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/12498420?s=80&v=4"},"commit":{"message":"Update docs (#443)","shortMessageHtmlLink":"Update docs (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2308535628\" data-permission-text=\"Title is private\" data-url=\"https://github.com/mlcommons/modelgauge/issues/443\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/mlcommons/modelgauge/pull/443/hovercard\" href=\"https://github.com/mlcommons/modelgauge/pull/443\">#443</a>)"}},{"before":null,"after":"9be22f6c9961e42d3d1e66ce0a229990a7e12d6a","ref":"refs/heads/sut-capabilities-docs","pushedAt":"2024-05-21T15:10:31.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"bkorycki","name":"Barbara Korycki","path":"/bkorycki","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/12498420?s=80&v=4"},"commit":{"message":"Update docs","shortMessageHtmlLink":"Update docs"}},{"before":"f271046cf6f3ca759a9654ebccf3efa43988eedf","after":"3e04edbddd4ceff221a932148eefc260cad22ee5","ref":"refs/heads/main","pushedAt":"2024-05-18T02:11:42.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"bkorycki","name":"Barbara Korycki","path":"/bkorycki","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/12498420?s=80&v=4"},"commit":{"message":"Add LogProbs capability to HuggingFaceSUT (#439)\n\n* Move prompt tokenization from evaluate() into translate_request()\r\n\r\n* stop_sequences does not affect eos_token_id\r\n\r\n* Store tokenized tensors in HuggingFaceRequest\r\n\r\n* Put tokenization back in evaluate\r\n\r\n* Add logprobs capability\r\n\r\n* make mypy happy","shortMessageHtmlLink":"Add LogProbs capability to HuggingFaceSUT (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2300722979\" data-permission-text=\"Title is private\" data-url=\"https://github.com/mlcommons/modelgauge/issues/439\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/mlcommons/modelgauge/pull/439/hovercard\" href=\"https://github.com/mlcommons/modelgauge/pull/439\">#439</a>)"}},{"before":"8ec2b84939ebbde653a18412f183af9386992d9a","after":"8f07fb25c3abaa09fccfa6c747604b85fd570ac7","ref":"refs/heads/huggingface-cleanup","pushedAt":"2024-05-16T14:35:27.000Z","pushType":"push","commitsCount":7,"pusher":{"login":"bkorycki","name":"Barbara Korycki","path":"/bkorycki","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/12498420?s=80&v=4"},"commit":{"message":"Merge branch 'main' into huggingface-cleanup","shortMessageHtmlLink":"Merge branch 'main' into huggingface-cleanup"}},{"before":"10ae14b363daf7e4f0ae2e66a58c49ce238dc5e2","after":"f271046cf6f3ca759a9654ebccf3efa43988eedf","ref":"refs/heads/main","pushedAt":"2024-05-14T16:39:19.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"nathanw-mlc","name":"Nathan Wasson","path":"/nathanw-mlc","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/126016071?s=80&v=4"},"commit":{"message":"Update CODEOWNERS","shortMessageHtmlLink":"Update CODEOWNERS"}},{"before":null,"after":"8ec2b84939ebbde653a18412f183af9386992d9a","ref":"refs/heads/huggingface-cleanup","pushedAt":"2024-05-14T04:34:33.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"bkorycki","name":"Barbara Korycki","path":"/bkorycki","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/12498420?s=80&v=4"},"commit":{"message":"Store tokenized tensors in HuggingFaceRequest","shortMessageHtmlLink":"Store tokenized tensors in HuggingFaceRequest"}},{"before":"58ef9897c74a6e7ede85b5080bd3f7c7475d8203","after":null,"ref":"refs/heads/allow-local-plugin-loading","pushedAt":"2024-05-13T17:43:27.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"dhosterman","name":"Daniel Hosterman","path":"/dhosterman","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3520372?s=80&v=4"}},{"before":"7d77e882371f6a505f47267b02748331c7f5eb90","after":"10ae14b363daf7e4f0ae2e66a58c49ce238dc5e2","ref":"refs/heads/main","pushedAt":"2024-05-13T17:43:24.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"dhosterman","name":"Daniel Hosterman","path":"/dhosterman","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3520372?s=80&v=4"},"commit":{"message":"Allow local plugin loading (#438)\n\n* add --plugin-dir option\r\n\r\n* fix missing import\r\n\r\n* add some docs","shortMessageHtmlLink":"Allow local plugin loading (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2293018377\" data-permission-text=\"Title is private\" data-url=\"https://github.com/mlcommons/modelgauge/issues/438\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/mlcommons/modelgauge/pull/438/hovercard\" href=\"https://github.com/mlcommons/modelgauge/pull/438\">#438</a>)"}},{"before":"3ea983eaa14c298ac18b94764c3cb772167234c8","after":"58ef9897c74a6e7ede85b5080bd3f7c7475d8203","ref":"refs/heads/allow-local-plugin-loading","pushedAt":"2024-05-13T14:38:03.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"dhosterman","name":"Daniel Hosterman","path":"/dhosterman","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3520372?s=80&v=4"},"commit":{"message":"add some docs","shortMessageHtmlLink":"add some docs"}},{"before":null,"after":"3ea983eaa14c298ac18b94764c3cb772167234c8","ref":"refs/heads/allow-local-plugin-loading","pushedAt":"2024-05-13T14:31:05.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"dhosterman","name":"Daniel Hosterman","path":"/dhosterman","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3520372?s=80&v=4"},"commit":{"message":"fix missing import","shortMessageHtmlLink":"fix missing import"}},{"before":"79283fdd83327ee9def045fda0897e34a9cef72b","after":"7d77e882371f6a505f47267b02748331c7f5eb90","ref":"refs/heads/main","pushedAt":"2024-05-07T21:01:39.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"bkorycki","name":"Barbara Korycki","path":"/bkorycki","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/12498420?s=80&v=4"},"commit":{"message":"Together logprobs capability (#434)","shortMessageHtmlLink":"Together logprobs capability (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2273609021\" data-permission-text=\"Title is private\" data-url=\"https://github.com/mlcommons/modelgauge/issues/434\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/mlcommons/modelgauge/pull/434/hovercard\" href=\"https://github.com/mlcommons/modelgauge/pull/434\">#434</a>)"}},{"before":null,"after":"8180d9ce96c711fe89a4011aa2e72fba35601d0e","ref":"refs/heads/together-logprobs","pushedAt":"2024-05-01T14:27:30.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"bkorycki","name":"Barbara Korycki","path":"/bkorycki","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/12498420?s=80&v=4"},"commit":{"message":"Together logprobs capability","shortMessageHtmlLink":"Together logprobs capability"}},{"before":"2d17d74d0624b300e09f7c8c5deb69975bcbfdb5","after":null,"ref":"refs/heads/bump-version-llamaguard2","pushedAt":"2024-04-26T20:44:11.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"bkorycki","name":"Barbara Korycki","path":"/bkorycki","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/12498420?s=80&v=4"}},{"before":"7f85a856ec426e03e1bbe341400f6bace32115bd","after":"79283fdd83327ee9def045fda0897e34a9cef72b","ref":"refs/heads/main","pushedAt":"2024-04-26T20:44:05.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"bkorycki","name":"Barbara Korycki","path":"/bkorycki","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/12498420?s=80&v=4"},"commit":{"message":"Bump version 0.5.1 (#432)","shortMessageHtmlLink":"Bump version 0.5.1 (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2266468242\" data-permission-text=\"Title is private\" data-url=\"https://github.com/mlcommons/modelgauge/issues/432\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/mlcommons/modelgauge/pull/432/hovercard\" href=\"https://github.com/mlcommons/modelgauge/pull/432\">#432</a>)"}},{"before":null,"after":"2d17d74d0624b300e09f7c8c5deb69975bcbfdb5","ref":"refs/heads/bump-version-llamaguard2","pushedAt":"2024-04-26T20:34:28.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"bkorycki","name":"Barbara Korycki","path":"/bkorycki","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/12498420?s=80&v=4"},"commit":{"message":"Bump version 0.5.1","shortMessageHtmlLink":"Bump version 0.5.1"}},{"before":"b3182896b1708452aaad513625d686ddec13c470","after":"7f85a856ec426e03e1bbe341400f6bace32115bd","ref":"refs/heads/main","pushedAt":"2024-04-26T20:08:48.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"bkorycki","name":"Barbara Korycki","path":"/bkorycki","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/12498420?s=80&v=4"},"commit":{"message":"Add LlamaGuard 2 (#431)\n\n* Modify LlamaGuardAnnotator to work with versions 1 and 2\r\n* Minor changes to prompt formatting for original LlamaGuard v1\r\n* Add optional llamaguard version parameter to SafeTest","shortMessageHtmlLink":"Add LlamaGuard 2 (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2264321313\" data-permission-text=\"Title is private\" data-url=\"https://github.com/mlcommons/modelgauge/issues/431\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/mlcommons/modelgauge/pull/431/hovercard\" href=\"https://github.com/mlcommons/modelgauge/pull/431\">#431</a>)"}},{"before":"8526fe4be35a25ecc5e99bbfc876bfc3f5007473","after":"f6e8981e95e849ec4b3966c8ec9bd44d1e2484f9","ref":"refs/heads/add-llamaguard2","pushedAt":"2024-04-26T19:56:57.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"bkorycki","name":"Barbara Korycki","path":"/bkorycki","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/12498420?s=80&v=4"},"commit":{"message":"Init. llamaguard with string verion to avoid SafeTest serializing entire llamaguard_data object","shortMessageHtmlLink":"Init. llamaguard with string verion to avoid SafeTest serializing ent…"}},{"before":"62fcf8d9e8533740586527b5d0840d8f17db2298","after":"8526fe4be35a25ecc5e99bbfc876bfc3f5007473","ref":"refs/heads/add-llamaguard2","pushedAt":"2024-04-26T15:22:32.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"bkorycki","name":"Barbara Korycki","path":"/bkorycki","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/12498420?s=80&v=4"},"commit":{"message":"Data structure for llamaguard version-specific info","shortMessageHtmlLink":"Data structure for llamaguard version-specific info"}},{"before":"1a1e76559652f9fcf292ceeda94f979892f2ed76","after":"62fcf8d9e8533740586527b5d0840d8f17db2298","ref":"refs/heads/add-llamaguard2","pushedAt":"2024-04-25T19:15:57.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"bkorycki","name":"Barbara Korycki","path":"/bkorycki","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/12498420?s=80&v=4"},"commit":{"message":"Add prompt formatting reference","shortMessageHtmlLink":"Add prompt formatting reference"}},{"before":null,"after":"1a1e76559652f9fcf292ceeda94f979892f2ed76","ref":"refs/heads/add-llamaguard2","pushedAt":"2024-04-25T19:04:41.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"bkorycki","name":"Barbara Korycki","path":"/bkorycki","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/12498420?s=80&v=4"},"commit":{"message":"Add llamaguard version parameter to SafeTest","shortMessageHtmlLink":"Add llamaguard version parameter to SafeTest"}},{"before":"489c70aab3445788e374451c7ea07fd78a1177ca","after":"b3182896b1708452aaad513625d686ddec13c470","ref":"refs/heads/main","pushedAt":"2024-04-25T16:19:11.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"yifanmai","name":"Yifan Mai","path":"/yifanmai","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/185227?s=80&v=4"},"commit":{"message":"Remove poetry run from the SUTs and tests tutorial (#424)","shortMessageHtmlLink":"Remove poetry run from the SUTs and tests tutorial (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2251152231\" data-permission-text=\"Title is private\" data-url=\"https://github.com/mlcommons/modelgauge/issues/424\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/mlcommons/modelgauge/pull/424/hovercard\" href=\"https://github.com/mlcommons/modelgauge/pull/424\">#424</a>)"}},{"before":"eb34e2071ac5c9c66cf862165e4371bf96551274","after":"489c70aab3445788e374451c7ea07fd78a1177ca","ref":"refs/heads/main","pushedAt":"2024-04-25T16:18:58.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"yifanmai","name":"Yifan Mai","path":"/yifanmai","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/185227?s=80&v=4"},"commit":{"message":"Add User Quick Start (#422)\n\n* More fixes\r\n\r\n* Heading fix\r\n\r\n* Fix link\r\n\r\n* Adding a little more poetry explanation. Pointing at the correct channel. Wrapping paragraphs.\r\n\r\n---------\r\n\r\nCo-authored-by: william <william@williampietri.com>","shortMessageHtmlLink":"Add User Quick Start (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2249754124\" data-permission-text=\"Title is private\" data-url=\"https://github.com/mlcommons/modelgauge/issues/422\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/mlcommons/modelgauge/pull/422/hovercard\" href=\"https://github.com/mlcommons/modelgauge/pull/422\">#422</a>)"}},{"before":"ea726387fb76f7cc028ebc93c26a69c2995ebc2c","after":"eb34e2071ac5c9c66cf862165e4371bf96551274","ref":"refs/heads/main","pushedAt":"2024-04-25T16:18:31.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"yifanmai","name":"Yifan Mai","path":"/yifanmai","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/185227?s=80&v=4"},"commit":{"message":"Fix broken links in docs (#423)","shortMessageHtmlLink":"Fix broken links in docs (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2249757394\" data-permission-text=\"Title is private\" data-url=\"https://github.com/mlcommons/modelgauge/issues/423\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/mlcommons/modelgauge/pull/423/hovercard\" href=\"https://github.com/mlcommons/modelgauge/pull/423\">#423</a>)"}},{"before":"18d940bd74bf223639033e757b8f9b74df674c92","after":"ea726387fb76f7cc028ebc93c26a69c2995ebc2c","ref":"refs/heads/main","pushedAt":"2024-04-18T21:09:16.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"bkorycki","name":"Barbara Korycki","path":"/bkorycki","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/12498420?s=80&v=4"},"commit":{"message":"Test more python versions in github workflow (#420)\n\n* Updated workflow to use version matrix\r\n\r\n* Remove redunant pytest that doesn't use plugins\r\n\r\n* Apply enum changes to safetest test file","shortMessageHtmlLink":"Test more python versions in github workflow (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2249292647\" data-permission-text=\"Title is private\" data-url=\"https://github.com/mlcommons/modelgauge/issues/420\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/mlcommons/modelgauge/pull/420/hovercard\" href=\"https://github.com/mlcommons/modelgauge/pull/420\">#420</a>)"}},{"before":"bffc4a4a5ec968e8d1df2ed511c2bda28f440082","after":"dbf74e10d0635c6046fce07341566c12e4b83434","ref":"refs/heads/python-version-testing","pushedAt":"2024-04-18T20:58:55.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"bkorycki","name":"Barbara Korycki","path":"/bkorycki","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/12498420?s=80&v=4"},"commit":{"message":"Apply enum changes to safetest test file","shortMessageHtmlLink":"Apply enum changes to safetest test file"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEUgeAIAA","startCursor":null,"endCursor":null}},"title":"Activity · mlcommons/modelgauge"}