{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":323651234,"defaultBranch":"main","name":"gpt-neox","ownerLogin":"EleutherAI","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2020-12-22T14:37:54.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/68924597?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1715028442.0","currentOid":""},"activityList":{"items":[{"before":null,"after":"af91a7bfe8fce52fa30675e2e0fdc8aadbc8a46f","ref":"refs/heads/dependabot/pip/requirements/jinja2-3.1.4","pushedAt":"2024-05-06T20:47:22.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"dependabot[bot]","name":null,"path":"/apps/dependabot","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/29110?s=80&v=4"},"commit":{"message":"Bump jinja2 from 3.1.3 to 3.1.4 in /requirements\n\nBumps [jinja2](https://github.com/pallets/jinja) from 3.1.3 to 3.1.4.\n- [Release notes](https://github.com/pallets/jinja/releases)\n- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)\n- [Commits](https://github.com/pallets/jinja/compare/3.1.3...3.1.4)\n\n---\nupdated-dependencies:\n- dependency-name: jinja2\n dependency-type: direct:production\n...\n\nSigned-off-by: dependabot[bot] ","shortMessageHtmlLink":"Bump jinja2 from 3.1.3 to 3.1.4 in /requirements"}},{"before":"2e769fbd04fbdfe24ce51a780517a6608565e3e4","after":"43ccefe6a3e083b6a15e8a3884fa7f17fab0aca9","ref":"refs/heads/infinite_lr","pushedAt":"2024-05-06T17:16:00.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"Update NeoXArgs docs automatically","shortMessageHtmlLink":"Update NeoXArgs docs automatically"}},{"before":"b6eefceb4d3f6be7faae6563b3bf8766b06d3db4","after":"2e769fbd04fbdfe24ce51a780517a6608565e3e4","ref":"refs/heads/infinite_lr","pushedAt":"2024-05-06T17:15:45.000Z","pushType":"push","commitsCount":8,"pusher":{"login":"Quentin-Anthony","name":"Quentin Anthony","path":"/Quentin-Anthony","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10281105?s=80&v=4"},"commit":{"message":"Merge branch 'main' into infinite_lr","shortMessageHtmlLink":"Merge branch 'main' into infinite_lr"}},{"before":"3388c515ae051b275468d86a20fae2987e4da144","after":"e987126f51f01368ddc1a0a38d17fda6659eaa7d","ref":"refs/heads/dmoe_integration","pushedAt":"2024-05-06T03:57:10.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"Update NeoXArgs docs automatically","shortMessageHtmlLink":"Update NeoXArgs docs automatically"}},{"before":"9c66895a701e72d7fbbc87a4abbb993e857ba1fe","after":"3388c515ae051b275468d86a20fae2987e4da144","ref":"refs/heads/dmoe_integration","pushedAt":"2024-05-06T03:34:34.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"DayOfThePenguin","name":"Colin","path":"/DayOfThePenguin","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8410369?s=80&v=4"},"commit":{"message":"feat: update args, configs, and requirements","shortMessageHtmlLink":"feat: update args, configs, and requirements"}},{"before":"52a20010e3bf4c141f9214bb081544b4350c85a6","after":"9c66895a701e72d7fbbc87a4abbb993e857ba1fe","ref":"refs/heads/dmoe_integration","pushedAt":"2024-05-06T03:27:08.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"DayOfThePenguin","name":"Colin","path":"/DayOfThePenguin","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8410369?s=80&v=4"},"commit":{"message":"feat: clean up megablocks-based DMoE implementation","shortMessageHtmlLink":"feat: clean up megablocks-based DMoE implementation"}},{"before":"4bc667031d82e5690882ef16be9edad09d469a8d","after":"52a20010e3bf4c141f9214bb081544b4350c85a6","ref":"refs/heads/dmoe_integration","pushedAt":"2024-05-06T03:07:41.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"DayOfThePenguin","name":"Colin","path":"/DayOfThePenguin","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8410369?s=80&v=4"},"commit":{"message":"feat: ensure only the right architectures get built vs all of them for 11.x","shortMessageHtmlLink":"feat: ensure only the right architectures get built vs all of them fo…"}},{"before":null,"after":"4bc667031d82e5690882ef16be9edad09d469a8d","ref":"refs/heads/dmoe_integration","pushedAt":"2024-05-06T03:05:03.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"DayOfThePenguin","name":"Colin","path":"/DayOfThePenguin","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8410369?s=80&v=4"},"commit":{"message":"add rwkv support (#1198)\n\n* add rwkv support\r\n\r\n* Update init_functions.py\r\n\r\n* rwkv model files\r\n\r\n* configs\r\n\r\n* kernels\r\n\r\n* Cleanup\r\n\r\n* Update 760M.yml\r\n\r\n* remove preffn and mishglu\r\n\r\n* Update NeoXArgs docs automatically\r\n\r\n* Add RWKV parallelism assertions\r\n\r\n* Update NeoXArgs docs automatically\r\n\r\n* pre-commit and config cleanup\r\n\r\n* Update NeoXArgs docs automatically\r\n\r\n* rwkv logging\r\n\r\n* Update NeoXArgs docs automatically\r\n\r\n* Add rwkv version dirname, make hdim 3.5x\r\n\r\n* pre-commit\r\n\r\n* Update NeoXArgs docs automatically\r\n\r\n* fix bug and set batch size to 32\r\n\r\n* Update NeoXArgs docs automatically\r\n\r\n---------\r\n\r\nCo-authored-by: Quentin Anthony \r\nCo-authored-by: github-actions ","shortMessageHtmlLink":"add rwkv support (#1198)"}},{"before":"dd0138e8cf14024879beab599b8ca7efdc56445f","after":null,"ref":"refs/heads/rwkv","pushedAt":"2024-05-06T00:10:31.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"Quentin-Anthony","name":"Quentin Anthony","path":"/Quentin-Anthony","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10281105?s=80&v=4"}},{"before":"c8149592d936c3a23ecff4c0092d33bd6c64fab5","after":"4bc667031d82e5690882ef16be9edad09d469a8d","ref":"refs/heads/main","pushedAt":"2024-05-06T00:10:30.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"Quentin-Anthony","name":"Quentin Anthony","path":"/Quentin-Anthony","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10281105?s=80&v=4"},"commit":{"message":"add rwkv support (#1198)\n\n* add rwkv support\r\n\r\n* Update init_functions.py\r\n\r\n* rwkv model files\r\n\r\n* configs\r\n\r\n* kernels\r\n\r\n* Cleanup\r\n\r\n* Update 760M.yml\r\n\r\n* remove preffn and mishglu\r\n\r\n* Update NeoXArgs docs automatically\r\n\r\n* Add RWKV parallelism assertions\r\n\r\n* Update NeoXArgs docs automatically\r\n\r\n* pre-commit and config cleanup\r\n\r\n* Update NeoXArgs docs automatically\r\n\r\n* rwkv logging\r\n\r\n* Update NeoXArgs docs automatically\r\n\r\n* Add rwkv version dirname, make hdim 3.5x\r\n\r\n* pre-commit\r\n\r\n* Update NeoXArgs docs automatically\r\n\r\n* fix bug and set batch size to 32\r\n\r\n* Update NeoXArgs docs automatically\r\n\r\n---------\r\n\r\nCo-authored-by: Quentin Anthony \r\nCo-authored-by: github-actions ","shortMessageHtmlLink":"add rwkv support (#1198)"}},{"before":"6fb840e9ecdda4ba69034722d4a6cd4a040834cf","after":"dd0138e8cf14024879beab599b8ca7efdc56445f","ref":"refs/heads/rwkv","pushedAt":"2024-05-05T20:00:36.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"Update NeoXArgs docs automatically","shortMessageHtmlLink":"Update NeoXArgs docs automatically"}},{"before":"8f60a43192b472eb0dd6e898e17e91a7c989f2ee","after":"6fb840e9ecdda4ba69034722d4a6cd4a040834cf","ref":"refs/heads/rwkv","pushedAt":"2024-05-05T20:00:23.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"jahatef","name":"Jacob Hatef","path":"/jahatef","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/74274091?s=80&v=4"},"commit":{"message":"fix bug and set batch size to 32","shortMessageHtmlLink":"fix bug and set batch size to 32"}},{"before":"916c88357fdbee5107574da156585addd17b31bb","after":"c8149592d936c3a23ecff4c0092d33bd6c64fab5","ref":"refs/heads/main","pushedAt":"2024-05-04T18:52:36.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"Quentin-Anthony","name":"Quentin Anthony","path":"/Quentin-Anthony","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10281105?s=80&v=4"},"commit":{"message":"Fix bug in tools/ckpts/convert_neox_to_hf.py for setting intermediate_size (#1209)\n\nIn tools/ckpts/convert_neox_to_hf.py, for neox architecture the 'intermediate_size'\r\nargument is not explicitly set, so it defaults to 24576 from:\r\n\r\nhttps://github.com/huggingface/transformers/blob/9fe3f585bb4ea29f209dc705d269fbe292e1128f/src/transformers/models/gpt_neox/configuration_gpt_neox.py#L48\r\n\r\nProposed solution: set intermediate-size to 4 * hidden-size","shortMessageHtmlLink":"Fix bug in tools/ckpts/convert_neox_to_hf.py for setting intermediate…"}},{"before":"06e5f0c2fe3b61d54496c0545ce4afbe4099120c","after":"916c88357fdbee5107574da156585addd17b31bb","ref":"refs/heads/main","pushedAt":"2024-05-04T18:25:27.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"Quentin-Anthony","name":"Quentin Anthony","path":"/Quentin-Anthony","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10281105?s=80&v=4"},"commit":{"message":"Add megablocks dropless MoE (#1192)\n\n* Add megablocks dropless MoE\r\n\r\n* pre-commit\r\n\r\n---------\r\n\r\nCo-authored-by: Yang Zhang \r\nCo-authored-by: Quentin Anthony ","shortMessageHtmlLink":"Add megablocks dropless MoE (#1192)"}},{"before":"921c41a5daeb5a74760c3dde270ecf491efaa189","after":"8f60a43192b472eb0dd6e898e17e91a7c989f2ee","ref":"refs/heads/rwkv","pushedAt":"2024-05-04T17:44:55.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"Update NeoXArgs docs automatically","shortMessageHtmlLink":"Update NeoXArgs docs automatically"}},{"before":"a599ac7c48eb2a0f2bd7d73070e5d550d33569db","after":"921c41a5daeb5a74760c3dde270ecf491efaa189","ref":"refs/heads/rwkv","pushedAt":"2024-05-04T17:44:41.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"Quentin-Anthony","name":"Quentin Anthony","path":"/Quentin-Anthony","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10281105?s=80&v=4"},"commit":{"message":"pre-commit","shortMessageHtmlLink":"pre-commit"}},{"before":"11036630c589bba76257f668249ff519acd314e2","after":"a599ac7c48eb2a0f2bd7d73070e5d550d33569db","ref":"refs/heads/rwkv","pushedAt":"2024-05-04T17:15:10.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"Update NeoXArgs docs automatically","shortMessageHtmlLink":"Update NeoXArgs docs automatically"}},{"before":"330a80215d765530d828c70f2abe1d2a450b69bd","after":"11036630c589bba76257f668249ff519acd314e2","ref":"refs/heads/rwkv","pushedAt":"2024-05-04T17:14:54.000Z","pushType":"push","commitsCount":5,"pusher":{"login":"Quentin-Anthony","name":"Quentin Anthony","path":"/Quentin-Anthony","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10281105?s=80&v=4"},"commit":{"message":"Merge branch 'main' into rwkv","shortMessageHtmlLink":"Merge branch 'main' into rwkv"}},{"before":"3cf13eca777cc195d36bf551921cd69ceb747c45","after":null,"ref":"refs/heads/jaimemcc-intel/ci-composite-cpu-tests","pushedAt":"2024-05-04T17:14:27.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"Quentin-Anthony","name":"Quentin Anthony","path":"/Quentin-Anthony","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10281105?s=80&v=4"}},{"before":"9d9d7c8f5255fe261386bbcf0be6705644a982aa","after":"06e5f0c2fe3b61d54496c0545ce4afbe4099120c","ref":"refs/heads/main","pushedAt":"2024-05-04T17:14:26.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"Quentin-Anthony","name":"Quentin Anthony","path":"/Quentin-Anthony","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10281105?s=80&v=4"},"commit":{"message":"Jaimemcc intel/ci composite cpu tests (#1205)\n\n* split PR and CPU tests into separate work; adjust references to env variables in workflow\r\n\r\n* tweaking to pull compose file from CPU test dir\r\n\r\n* adding post-cleanup for portability; adding workflow_dispatch to test\r\n\r\n* fixing mapping\r\n\r\n* forgot shell declaration in composite run\r\n\r\n* make sure all steps run even if first CPU tests fail\r\n\r\n* adding workflow dispatch to manually call workflow; removing httpserver\r\n\r\n* Update NeoXArgs docs automatically\r\n\r\n* Update NeoXArgs docs automatically\r\n\r\n---------\r\n\r\nCo-authored-by: github-actions \r\nCo-authored-by: Quentin Anthony ","shortMessageHtmlLink":"Jaimemcc intel/ci composite cpu tests (#1205)"}},{"before":"47c93fba9cebc4f1efdf22105c319e9ee38b4e8f","after":"3cf13eca777cc195d36bf551921cd69ceb747c45","ref":"refs/heads/jaimemcc-intel/ci-composite-cpu-tests","pushedAt":"2024-05-04T17:13:58.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"Update NeoXArgs docs automatically","shortMessageHtmlLink":"Update NeoXArgs docs automatically"}},{"before":"c3b0adc20b7f598ac5cae338e3ec8658dfee917a","after":"47c93fba9cebc4f1efdf22105c319e9ee38b4e8f","ref":"refs/heads/jaimemcc-intel/ci-composite-cpu-tests","pushedAt":"2024-05-04T17:13:45.000Z","pushType":"push","commitsCount":4,"pusher":{"login":"Quentin-Anthony","name":"Quentin Anthony","path":"/Quentin-Anthony","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10281105?s=80&v=4"},"commit":{"message":"Merge branch 'main' into jaimemcc-intel/ci-composite-cpu-tests","shortMessageHtmlLink":"Merge branch 'main' into jaimemcc-intel/ci-composite-cpu-tests"}},{"before":"675205f4f9800f1b3fffee1e5a24c18ab2088d86","after":null,"ref":"refs/heads/dependabot/pip/requirements/transformers-4.38.0","pushedAt":"2024-05-04T17:13:25.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"Quentin-Anthony","name":"Quentin Anthony","path":"/Quentin-Anthony","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10281105?s=80&v=4"}},{"before":"838d5bfc3c5b0c03c9037c5fe72e4fa8ebf00c6a","after":"9d9d7c8f5255fe261386bbcf0be6705644a982aa","ref":"refs/heads/main","pushedAt":"2024-05-04T17:13:24.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"Quentin-Anthony","name":"Quentin Anthony","path":"/Quentin-Anthony","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10281105?s=80&v=4"},"commit":{"message":"Bump transformers from 4.36.0 to 4.38.0 in /requirements (#1199)\n\nBumps [transformers](https://github.com/huggingface/transformers) from 4.36.0 to 4.38.0.\r\n- [Release notes](https://github.com/huggingface/transformers/releases)\r\n- [Commits](https://github.com/huggingface/transformers/compare/v4.36.0...v4.38.0)\r\n\r\n---\r\nupdated-dependencies:\r\n- dependency-name: transformers\r\n dependency-type: direct:production\r\n...\r\n\r\nSigned-off-by: dependabot[bot] \r\nCo-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>","shortMessageHtmlLink":"Bump transformers from 4.36.0 to 4.38.0 in /requirements (#1199)"}},{"before":"c0af56368a717268fe9a514108224ada7c743d58","after":"330a80215d765530d828c70f2abe1d2a450b69bd","ref":"refs/heads/rwkv","pushedAt":"2024-05-03T14:41:33.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"jahatef","name":"Jacob Hatef","path":"/jahatef","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/74274091?s=80&v=4"},"commit":{"message":"Merge branch 'rwkv' of https://github.com/EleutherAI/gpt-neox into rwkv","shortMessageHtmlLink":"Merge branch 'rwkv' of https://github.com/EleutherAI/gpt-neox into rwkv"}},{"before":"a3ca0f2b8f8c3f3e3671c7ccbcc1b140003d6334","after":"43c1a5337553cff0b40b2fa31937eef6f6f14d83","ref":"refs/heads/sparse-is-enough","pushedAt":"2024-05-02T20:08:33.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"haileyschoelkopf","name":"Hailey Schoelkopf","path":"/haileyschoelkopf","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/65563625?s=80&v=4"},"commit":{"message":"push code","shortMessageHtmlLink":"push code"}},{"before":"6a8ad71ad6b14fa369480643b4586ec819a9f62b","after":"485cad4c320fe7eaddf992083ec3bbc15bf713b7","ref":"refs/heads/rework-mup","pushedAt":"2024-05-02T16:38:56.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"Update NeoXArgs docs automatically","shortMessageHtmlLink":"Update NeoXArgs docs automatically"}},{"before":"9dd583b072ab5b0f37650a5d54bd463745e64a73","after":"6a8ad71ad6b14fa369480643b4586ec819a9f62b","ref":"refs/heads/rework-mup","pushedAt":"2024-05-02T16:38:41.000Z","pushType":"push","commitsCount":3,"pusher":{"login":"lintangsutawika","name":"Lintang Sutawika","path":"/lintangsutawika","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/5774558?s=80&v=4"},"commit":{"message":"Merge branch 'main' into rework-mup","shortMessageHtmlLink":"Merge branch 'main' into rework-mup"}},{"before":"6fe55f4b5c6c8ab1d0eacd7abecc254a17ff4c79","after":"9dd583b072ab5b0f37650a5d54bd463745e64a73","ref":"refs/heads/rework-mup","pushedAt":"2024-05-02T16:38:11.000Z","pushType":"push","commitsCount":14,"pusher":{"login":"lintangsutawika","name":"Lintang Sutawika","path":"/lintangsutawika","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/5774558?s=80&v=4"},"commit":{"message":"Merge branch 'rework-mup' of https://github.com/EleutherAI/gpt-neox into rework-mup","shortMessageHtmlLink":"Merge branch 'rework-mup' of https://github.com/EleutherAI/gpt-neox i…"}},{"before":"b8decc7a577a8ba5c7e94335cca134179cc45b87","after":"a3ca0f2b8f8c3f3e3671c7ccbcc1b140003d6334","ref":"refs/heads/sparse-is-enough","pushedAt":"2024-04-26T19:32:10.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"haileyschoelkopf","name":"Hailey Schoelkopf","path":"/haileyschoelkopf","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/65563625?s=80&v=4"},"commit":{"message":"first pass at something which might work","shortMessageHtmlLink":"first pass at something which might work"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEQu1maQA","startCursor":null,"endCursor":null}},"title":"Activity · EleutherAI/gpt-neox"}