{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":653567769,"defaultBranch":"main","name":"xFasterTransformer","ownerLogin":"intel","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2023-06-14T09:52:28.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/17888862?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1715734432.0","currentOid":""},"activityList":{"items":[{"before":"ca7f6dddf3c6fe1f8736157dc281683d98fd9a70","after":"970e5dec93572f50b45e402c5f93d3470edaf28b","ref":"refs/heads/main","pushedAt":"2024-05-27T02:13:01.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"abenmao","name":"Meng,Chen","path":"/abenmao","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/29789552?s=80&v=4"},"commit":{"message":"[COMM] Fix bugs of core dump && hang when running cross nodes (#423)","shortMessageHtmlLink":"[COMM] Fix bugs of core dump &amp;&amp; hang when running cross nodes (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2314975821\" data-permission-text=\"Title is private\" data-url=\"https://github.com/intel/xFasterTransformer/issues/423\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/intel/xFasterTransformer/pull/423/hovercard\" href=\"https://github.com/intel/xFasterTransformer/pull/423\">#423</a>)"}},{"before":"6d19ba1521c501750a3790adcac5c14a68edc4ff","after":"ca7f6dddf3c6fe1f8736157dc281683d98fd9a70","ref":"refs/heads/main","pushedAt":"2024-05-23T00:50:42.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"pujiang2018","name":null,"path":"/pujiang2018","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/43156995?s=80&v=4"},"commit":{"message":"[Dependency] Update torch to 2.3.0. (#416)","shortMessageHtmlLink":"[Dependency] Update torch to 2.3.0. (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2309378612\" data-permission-text=\"Title is private\" data-url=\"https://github.com/intel/xFasterTransformer/issues/416\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/intel/xFasterTransformer/pull/416/hovercard\" href=\"https://github.com/intel/xFasterTransformer/pull/416\">#416</a>)"}},{"before":"89690c4835797ca0e9702cb46b114ca91cb1eca0","after":"6d19ba1521c501750a3790adcac5c14a68edc4ff","ref":"refs/heads/main","pushedAt":"2024-05-22T00:55:02.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"Duyi-Wang","name":null,"path":"/Duyi-Wang","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/83744562?s=80&v=4"},"commit":{"message":"[Benchmark] Add platform options. Support real model. (#409)","shortMessageHtmlLink":"[Benchmark] Add platform options. Support real model. (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2305361940\" data-permission-text=\"Title is private\" data-url=\"https://github.com/intel/xFasterTransformer/issues/409\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/intel/xFasterTransformer/pull/409/hovercard\" href=\"https://github.com/intel/xFasterTransformer/pull/409\">#409</a>)"}},{"before":"6c723b55af92e09815f8a1c5b0ea5f37d59a28b6","after":"89690c4835797ca0e9702cb46b114ca91cb1eca0","ref":"refs/heads/main","pushedAt":"2024-05-21T07:29:51.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"changqi1","name":"Changqing Li","path":"/changqi1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/33472713?s=80&v=4"},"commit":{"message":"[Kenrel] Add FP16 LLaMA YARN rotary_embedding. (#412)\n\n* [Kenrel] Add FP16 LLaMA YARN rotary_embedding.\r\n\r\n* add LlamaYaRNScaledRotaryEmbedding for CB.","shortMessageHtmlLink":"[Kenrel] Add FP16 LLaMA YARN rotary_embedding. (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2307405660\" data-permission-text=\"Title is private\" data-url=\"https://github.com/intel/xFasterTransformer/issues/412\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/intel/xFasterTransformer/pull/412/hovercard\" href=\"https://github.com/intel/xFasterTransformer/pull/412\">#412</a>)"}},{"before":"2c49c4500b2c77fab07ef89d74329cdd3734ef05","after":"6c723b55af92e09815f8a1c5b0ea5f37d59a28b6","ref":"refs/heads/main","pushedAt":"2024-05-20T15:21:59.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"changqi1","name":"Changqing Li","path":"/changqi1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/33472713?s=80&v=4"},"commit":{"message":"[Kernel] Add FP16 rmsnorm and rope kernels. (#408)\n\n* [Kernel] Add FP16 rmsnorm and rope kernels.\r\n\r\n* Reduce qwen rope kernels.","shortMessageHtmlLink":"[Kernel] Add FP16 rmsnorm and rope kernels. (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2305283481\" data-permission-text=\"Title is private\" data-url=\"https://github.com/intel/xFasterTransformer/issues/408\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/intel/xFasterTransformer/pull/408/hovercard\" href=\"https://github.com/intel/xFasterTransformer/pull/408\">#408</a>)"}},{"before":"ce7431bae4c4a4abefd18f31822c59c123d3d425","after":"2c49c4500b2c77fab07ef89d74329cdd3734ef05","ref":"refs/heads/main","pushedAt":"2024-05-20T09:20:07.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"changqi1","name":"Changqing Li","path":"/changqi1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/33472713?s=80&v=4"},"commit":{"message":"[xDNN] Release v1.5.0. (#410)","shortMessageHtmlLink":"[xDNN] Release v1.5.0. (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2305369431\" data-permission-text=\"Title is private\" data-url=\"https://github.com/intel/xFasterTransformer/issues/410\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/intel/xFasterTransformer/pull/410/hovercard\" href=\"https://github.com/intel/xFasterTransformer/pull/410\">#410</a>)"}},{"before":"94f751e14c46d1c04b876ab2e45207b859a3fcd1","after":"ce7431bae4c4a4abefd18f31822c59c123d3d425","ref":"refs/heads/main","pushedAt":"2024-05-20T08:07:54.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"abenmao","name":"Meng,Chen","path":"/abenmao","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/29789552?s=80&v=4"},"commit":{"message":"Baichuan1/2 full-link bf16 support (#407)","shortMessageHtmlLink":"Baichuan1/2 full-link bf16 support (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2305194294\" data-permission-text=\"Title is private\" data-url=\"https://github.com/intel/xFasterTransformer/issues/407\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/intel/xFasterTransformer/pull/407/hovercard\" href=\"https://github.com/intel/xFasterTransformer/pull/407\">#407</a>)"}},{"before":"ea5546a5e6637b18ac5fed785e4dda6dc58011c1","after":"94f751e14c46d1c04b876ab2e45207b859a3fcd1","ref":"refs/heads/main","pushedAt":"2024-05-20T05:23:56.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"Duyi-Wang","name":null,"path":"/Duyi-Wang","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/83744562?s=80&v=4"},"commit":{"message":"[Example] Fix incorrect tensor dimension with latest interface (#406)","shortMessageHtmlLink":"[Example] Fix incorrect tensor dimension with latest interface (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2305089538\" data-permission-text=\"Title is private\" data-url=\"https://github.com/intel/xFasterTransformer/issues/406\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/intel/xFasterTransformer/pull/406/hovercard\" href=\"https://github.com/intel/xFasterTransformer/pull/406\">#406</a>)"}},{"before":"93b1981ed4f547c313414d1e415bf19a77faa36f","after":"ea5546a5e6637b18ac5fed785e4dda6dc58011c1","ref":"refs/heads/main","pushedAt":"2024-05-20T04:54:52.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"pujiang2018","name":null,"path":"/pujiang2018","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/43156995?s=80&v=4"},"commit":{"message":"[Bug] fix incorrect input offset computing (#405)","shortMessageHtmlLink":"[Bug] fix incorrect input offset computing (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2304920100\" data-permission-text=\"Title is private\" data-url=\"https://github.com/intel/xFasterTransformer/issues/405\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/intel/xFasterTransformer/pull/405/hovercard\" href=\"https://github.com/intel/xFasterTransformer/pull/405\">#405</a>)"}},{"before":"4aa7d0053987c285976a430844bd76b8d18ec832","after":"93b1981ed4f547c313414d1e415bf19a77faa36f","ref":"refs/heads/main","pushedAt":"2024-05-20T01:23:59.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"Duyi-Wang","name":null,"path":"/Duyi-Wang","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/83744562?s=80&v=4"},"commit":{"message":"[Interface] Support List[int] and List[List[int]] for set_input_sb. (#404)","shortMessageHtmlLink":"[Interface] Support List[int] and List[List[int]] for set_input_sb. (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2302139979\" data-permission-text=\"Title is private\" data-url=\"https://github.com/intel/xFasterTransformer/issues/404\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/intel/xFasterTransformer/pull/404/hovercard\" href=\"https://github.com/intel/xFasterTransformer/pull/404\">#…</a>"}},{"before":"c07dc84de58a6b53b413750b05dea0d87fc0992c","after":"4aa7d0053987c285976a430844bd76b8d18ec832","ref":"refs/heads/main","pushedAt":"2024-05-17T09:35:59.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"abenmao","name":"Meng,Chen","path":"/abenmao","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/29789552?s=80&v=4"},"commit":{"message":"[Layers] Add alibiSlopes Attn && Flash Attn for CB. (#402)","shortMessageHtmlLink":"[Layers] Add alibiSlopes Attn &amp;&amp; Flash Attn for CB. (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2299932064\" data-permission-text=\"Title is private\" data-url=\"https://github.com/intel/xFasterTransformer/issues/402\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/intel/xFasterTransformer/pull/402/hovercard\" href=\"https://github.com/intel/xFasterTransformer/pull/402\">#402</a>)"}},{"before":"f0c6941b3865316351ad5947e777a74e279e3060","after":"c07dc84de58a6b53b413750b05dea0d87fc0992c","ref":"refs/heads/main","pushedAt":"2024-05-17T06:56:33.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"Duyi-Wang","name":null,"path":"/Duyi-Wang","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/83744562?s=80&v=4"},"commit":{"message":"[Example] Add demo of offline continuous batching (#401)","shortMessageHtmlLink":"[Example] Add demo of offline continuous batching (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2299838975\" data-permission-text=\"Title is private\" data-url=\"https://github.com/intel/xFasterTransformer/issues/401\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/intel/xFasterTransformer/pull/401/hovercard\" href=\"https://github.com/intel/xFasterTransformer/pull/401\">#401</a>)"}},{"before":"f9836305919c063903db6dd66cfb524aaae9bd9f","after":"f0c6941b3865316351ad5947e777a74e279e3060","ref":"refs/heads/main","pushedAt":"2024-05-16T05:09:43.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"Duyi-Wang","name":null,"path":"/Duyi-Wang","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/83744562?s=80&v=4"},"commit":{"message":"[Interface] Change return shape of forward_cb. (#400)","shortMessageHtmlLink":"[Interface] Change return shape of forward_cb. (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2299238940\" data-permission-text=\"Title is private\" data-url=\"https://github.com/intel/xFasterTransformer/issues/400\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/intel/xFasterTransformer/pull/400/hovercard\" href=\"https://github.com/intel/xFasterTransformer/pull/400\">#400</a>)"}},{"before":"7a113f25f51fe4d48229d266b560a661450dbe9c","after":"f9836305919c063903db6dd66cfb524aaae9bd9f","ref":"refs/heads/main","pushedAt":"2024-05-15T06:50:45.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"Duyi-Wang","name":null,"path":"/Duyi-Wang","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/83744562?s=80&v=4"},"commit":{"message":"[KVCache] Remove FP32 data type. (#399)","shortMessageHtmlLink":"[KVCache] Remove FP32 data type. (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2296944792\" data-permission-text=\"Title is private\" data-url=\"https://github.com/intel/xFasterTransformer/issues/399\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/intel/xFasterTransformer/pull/399/hovercard\" href=\"https://github.com/intel/xFasterTransformer/pull/399\">#399</a>)"}},{"before":"d4e391156284f277c888636c5404557476f7c4c5","after":"7a113f25f51fe4d48229d266b560a661450dbe9c","ref":"refs/heads/main","pushedAt":"2024-05-15T05:06:49.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"pujiang2018","name":null,"path":"/pujiang2018","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/43156995?s=80&v=4"},"commit":{"message":"[Framework] Continuous Batching Support (#357)","shortMessageHtmlLink":"[Framework] Continuous Batching Support (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2268301602\" data-permission-text=\"Title is private\" data-url=\"https://github.com/intel/xFasterTransformer/issues/357\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/intel/xFasterTransformer/pull/357/hovercard\" href=\"https://github.com/intel/xFasterTransformer/pull/357\">#357</a>)"}},{"before":"af0aae879cb1f13fe440b0db71e2bf8ddcbb0386","after":"cef27bc44887d9695231ec3941d327d4aabc2f45","ref":"refs/heads/cb_dev","pushedAt":"2024-05-15T01:31:05.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"Duyi-Wang","name":null,"path":"/Duyi-Wang","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/83744562?s=80&v=4"},"commit":{"message":"[Build] Fix build issue.","shortMessageHtmlLink":"[Build] Fix build issue."}},{"before":"90d74f9aa4ce319c5fc2ed55bc4ff25b4f04e260","after":"af0aae879cb1f13fe440b0db71e2bf8ddcbb0386","ref":"refs/heads/cb_dev","pushedAt":"2024-05-15T01:19:32.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"Duyi-Wang","name":null,"path":"/Duyi-Wang","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/83744562?s=80&v=4"},"commit":{"message":"[Example] Reactivate the old path.","shortMessageHtmlLink":"[Example] Reactivate the old path."}},{"before":"7fddd22f5f6c9fc141f63868d0d62791f893e8ba","after":null,"ref":"refs/heads/fix_GQA_issue","pushedAt":"2024-05-15T00:53:52.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"marvin-Yu","name":"marvinYu","path":"/marvin-Yu","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19548774?s=80&v=4"}},{"before":"9cb8ec4bc7961e019965918d1217d0ed9eff910c","after":"90d74f9aa4ce319c5fc2ed55bc4ff25b4f04e260","ref":"refs/heads/cb_dev","pushedAt":"2024-05-15T00:53:05.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"Duyi-Wang","name":null,"path":"/Duyi-Wang","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/83744562?s=80&v=4"},"commit":{"message":"[Interface] Add python api for continuous batching. (#398)","shortMessageHtmlLink":"[Interface] Add python api for continuous batching. (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2294750604\" data-permission-text=\"Title is private\" data-url=\"https://github.com/intel/xFasterTransformer/issues/398\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/intel/xFasterTransformer/pull/398/hovercard\" href=\"https://github.com/intel/xFasterTransformer/pull/398\">#398</a>)"}},{"before":"d3901da39700ee8aa0c74015bed79d7ceb88e3e5","after":"dd18f22eaa8e52458265537b958ccd00fc16d020","ref":"refs/heads/eval/with_opencompass","pushedAt":"2024-05-14T06:29:54.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"marvin-Yu","name":"marvinYu","path":"/marvin-Yu","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19548774?s=80&v=4"},"commit":{"message":"[Eval] Add eval test with opencompass.","shortMessageHtmlLink":"[Eval] Add eval test with opencompass."}},{"before":"bff98bf3f047964c30bb134867a5e6f6b2c60d1e","after":"d4e391156284f277c888636c5404557476f7c4c5","ref":"refs/heads/main","pushedAt":"2024-05-13T08:34:06.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"changqi1","name":"Changqing Li","path":"/changqi1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/33472713?s=80&v=4"},"commit":{"message":"[API] Optimize API Impl. (#396)\n\n* [API] Refactor API Impl.\r\n\r\n* format code\r\n\r\n* add activtion for MLP.","shortMessageHtmlLink":"[API] Optimize API Impl. (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2292063379\" data-permission-text=\"Title is private\" data-url=\"https://github.com/intel/xFasterTransformer/issues/396\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/intel/xFasterTransformer/pull/396/hovercard\" href=\"https://github.com/intel/xFasterTransformer/pull/396\">#396</a>)"}},{"before":"999cf3b6d3533671dcf67b0c296297abb3a72879","after":"9cb8ec4bc7961e019965918d1217d0ed9eff910c","ref":"refs/heads/cb_dev","pushedAt":"2024-05-13T08:13:33.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"changqi1","name":"Changqing Li","path":"/changqi1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/33472713?s=80&v=4"},"commit":{"message":"[Layer] Better method to reinterpret KV cache (#397)\n\n* [Common] Add sequenceMeta, sequenceGroup and sequenecePool. (#343)\r\n\r\n* merge batchSize and seqLen into one in TokenEembedding\r\n\r\n* merge batchSize and seqLen into one in TokenEembedding (#350)\r\n\r\n* [Common] Move Martix into xft namespace. (#351)\r\n\r\n* remove unsed function in DecoderLayer\r\n\r\n* [Layer] Remove unused functions in Decoder layer (#353)\r\n\r\n* fix compile error of embeddingForward\r\n\r\n* [Model] Fix compile error of embeddingForward in YaRNLlama (#358)\r\n\r\n* [Common] Add sampling params into group seq. (#356)\r\n\r\n* remove DecoderContext in computeSoftmax\r\n\r\n* [Util] Remove DecoderContext in computeSoftmax (#362)\r\n\r\n* [Common] Refactor sequence.h. (#363)\r\n\r\n* [kernels] refactor flash attention for continuous batching (#361)\r\n\r\n* [models] Add attnMeta for continuous batching (#364)\r\n\r\n* [Layers] fix build error (#365)\r\n\r\n* [Model] add interface for seq meta. (#366)\r\n\r\n* refactor resize function in DecoderContext to support CB, and qkScores member removed\r\n\r\n* [Common] Modify resize() in DecoderContext to support  (#367)\r\n\r\n* add some code to CommonDecoder::forward()\r\n\r\n* SequenceMeta refactor\r\n\r\n* [Model] New CommonDecoder::forward impl. skeleton (#369)\r\n\r\n* new KVCacheMgr supporting CB\r\n\r\n* fix typo & set default prefixId to -1 in addSequence()\r\n\r\n* [Common] New KVCacheMgr to support CB (#371)\r\n\r\n* [Sampling] Add repetition penalty for new seq type. (#373)\r\n\r\n* New foward to support CB (CommonDecoder->DecoderBlock->DecoderLayer->Attention/MLP)\r\n\r\n* add todo\r\n\r\n* [Sampling] Add greedy search for cb path. (#376)\r\n\r\n* logic issue fix\r\n\r\n* code fix to make new forward work\r\n\r\n* add maxSeqLen limitation\r\n\r\n* cross attention impl. for CB\r\n\r\n* DecoderContext::resize fix\r\n\r\n* correct the output of the new forward\r\n\r\n* add cb_check\r\n\r\n* fix incorrect buffer size calculation\r\n\r\n* 2 sequences -> 3 sequences\r\n\r\n* better method to prepare KV cache\r\n\r\n---------\r\n\r\nCo-authored-by: Changqing Li <changqing.li@intel.com>\r\nCo-authored-by: Duyi-Wang <duyi.wang@intel.com>\r\nCo-authored-by: Meng,Chen <chen.meng@intel.com>","shortMessageHtmlLink":"[Layer] Better method to reinterpret KV cache (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2292124223\" data-permission-text=\"Title is private\" data-url=\"https://github.com/intel/xFasterTransformer/issues/397\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/intel/xFasterTransformer/pull/397/hovercard\" href=\"https://github.com/intel/xFasterTransformer/pull/397\">#397</a>)"}},{"before":"13e55c85825f86bebd507dc1b081cfd3396c64da","after":"37e7ab1e01faf7a4701c2220c76014ff7392772f","ref":"refs/heads/pujiang/feature/cb_dev","pushedAt":"2024-05-13T08:04:47.000Z","pushType":"push","commitsCount":3,"pusher":{"login":"pujiang2018","name":null,"path":"/pujiang2018","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/43156995?s=80&v=4"},"commit":{"message":"Merge commit '999cf3b6d3533671dcf67b0c296297abb3a72879' into pujiang/feature/cb_dev","shortMessageHtmlLink":"Merge commit '999cf3b6d3533671dcf67b0c296297abb3a72879' into pujiang/…"}},{"before":"9e07c8dca351c213f9d0ed2657778d58a668298e","after":"13e55c85825f86bebd507dc1b081cfd3396c64da","ref":"refs/heads/pujiang/feature/cb_dev","pushedAt":"2024-05-13T08:03:51.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"pujiang2018","name":null,"path":"/pujiang2018","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/43156995?s=80&v=4"},"commit":{"message":"better method to prepare KV cache","shortMessageHtmlLink":"better method to prepare KV cache"}},{"before":"09bf49bf3adac1dec9d35c6fefc2a1e3479d6d9a","after":null,"ref":"refs/heads/bugfix/common/types","pushedAt":"2024-05-13T07:46:52.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"Duyi-Wang","name":null,"path":"/Duyi-Wang","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/83744562?s=80&v=4"}},{"before":"49539097abecb1e470bd20e9768634a03e25ebbf","after":"999cf3b6d3533671dcf67b0c296297abb3a72879","ref":"refs/heads/cb_dev","pushedAt":"2024-05-13T07:45:20.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"Duyi-Wang","name":null,"path":"/Duyi-Wang","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/83744562?s=80&v=4"},"commit":{"message":"[Model] Check maxLen should be [input len, model max len]. (#394)","shortMessageHtmlLink":"[Model] Check maxLen should be [input len, model max len]. (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2292016828\" data-permission-text=\"Title is private\" data-url=\"https://github.com/intel/xFasterTransformer/issues/394\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/intel/xFasterTransformer/pull/394/hovercard\" href=\"https://github.com/intel/xFasterTransformer/pull/394\">#394</a>)"}},{"before":"cf81b38b5272689d10a995e413f449d26670dd5a","after":"bff98bf3f047964c30bb134867a5e6f6b2c60d1e","ref":"refs/heads/main","pushedAt":"2024-05-13T07:28:24.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"changqi1","name":"Changqing Li","path":"/changqi1","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/33472713?s=80&v=4"},"commit":{"message":"[API] Add LLaMA decoder API. (#386)\n\n* [API] Add LLaMA decoder API.\r\n\r\n* float16_t kv cache\r\n\r\n* Add activation type.\r\n\r\n* Add norm type.\r\n\r\n* comments","shortMessageHtmlLink":"[API] Add LLaMA decoder API. (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2290681928\" data-permission-text=\"Title is private\" data-url=\"https://github.com/intel/xFasterTransformer/issues/386\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/intel/xFasterTransformer/pull/386/hovercard\" href=\"https://github.com/intel/xFasterTransformer/pull/386\">#386</a>)"}},{"before":"6e31d1bd4f717c50a3a1b30017cecbfff7367832","after":"49539097abecb1e470bd20e9768634a03e25ebbf","ref":"refs/heads/cb_dev","pushedAt":"2024-05-13T06:40:29.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"Duyi-Wang","name":null,"path":"/Duyi-Wang","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/83744562?s=80&v=4"},"commit":{"message":"[Example] More check in C++ continuous batching example (#393)","shortMessageHtmlLink":"[Example] More check in C++ continuous batching example (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2291951434\" data-permission-text=\"Title is private\" data-url=\"https://github.com/intel/xFasterTransformer/issues/393\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/intel/xFasterTransformer/pull/393/hovercard\" href=\"https://github.com/intel/xFasterTransformer/pull/393\">#393</a>)"}},{"before":"06699b874b588b1baba58f4d0f122e546522a1f9","after":"9e07c8dca351c213f9d0ed2657778d58a668298e","ref":"refs/heads/pujiang/feature/cb_dev","pushedAt":"2024-05-13T06:36:37.000Z","pushType":"push","commitsCount":4,"pusher":{"login":"pujiang2018","name":null,"path":"/pujiang2018","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/43156995?s=80&v=4"},"commit":{"message":"2 sequences -> 3 sequences","shortMessageHtmlLink":"2 sequences -&gt; 3 sequences"}},{"before":"3a0989294d5b79fb8fc89d4c3c89a3c234e24dde","after":"6e31d1bd4f717c50a3a1b30017cecbfff7367832","ref":"refs/heads/cb_dev","pushedAt":"2024-05-13T06:09:33.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"pujiang2018","name":null,"path":"/pujiang2018","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/43156995?s=80&v=4"},"commit":{"message":"[Example] Fix continuous batching C++ example. (#392)","shortMessageHtmlLink":"[Example] Fix continuous batching C++ example. (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2291898021\" data-permission-text=\"Title is private\" data-url=\"https://github.com/intel/xFasterTransformer/issues/392\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/intel/xFasterTransformer/pull/392/hovercard\" href=\"https://github.com/intel/xFasterTransformer/pull/392\">#392</a>)"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEVG4gGQA","startCursor":null,"endCursor":null}},"title":"Activity · intel/xFasterTransformer"}