Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: add cube_torch module ,for pytorch train speed, wait add performance test result #2662

Closed
wants to merge 2,223 commits into from

Conversation

awzhgw
Copy link
Member

@awzhgw awzhgw commented Sep 27, 2023

This is a new feature. The main purpose of this feature is to accelerate the training performance of pytorch on cubefs. Please refer to: https://github.com/cubefs/cubefs/blob/cube_torch/aimagic/cube_torch/Readme.md

@netlify
Copy link

netlify bot commented Sep 27, 2023

Deploy Preview for cubefs-check ready!

Name Link
🔨 Latest commit a402c9f
🔍 Latest deploy log https://app.netlify.com/sites/cubefs-check/deploys/663200094a656900091ec773
😎 Deploy Preview https://deploy-preview-2662--cubefs-check.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@wenjia322 wenjia322 force-pushed the cube_torch branch 2 times, most recently from 63681f9 to 34c5b71 Compare September 27, 2023 14:42
@awzhgw awzhgw changed the title feature: add cube_torch module ,for pytorch train speed feature: add cube_torch module ,for pytorch train speed, wait add performance test result Oct 10, 2023
@awzhgw awzhgw force-pushed the cube_torch branch 4 times, most recently from 66324d1 to f21d472 Compare October 11, 2023 09:56
@awzhgw awzhgw force-pushed the cube_torch branch 5 times, most recently from e97fafd to 32a763f Compare October 26, 2023 07:58
Copy link

codecov bot commented Nov 2, 2023

Codecov Report

Attention: Patch coverage is 0% with 84 lines in your changes are missing coverage. Please review.

Project coverage is 42.00%. Comparing base (f7e7235) to head (a23d8f2).
Report is 1 commits behind head on master.

❗ Current head a23d8f2 differs from pull request most recent head 5765cc8. Consider uploading reports for the commit 5765cc8 to get more accurate results

Files Patch % Lines
proto/mount_options.go 0.00% 74 Missing ⚠️
util/iputil/ip.go 0.00% 6 Missing ⚠️
proto/fs_proto.go 0.00% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2662      +/-   ##
==========================================
+ Coverage   41.91%   42.00%   +0.08%     
==========================================
  Files         581      574       -7     
  Lines      119800   117963    -1837     
==========================================
- Hits        50218    49547     -671     
+ Misses      65271    64154    -1117     
+ Partials     4311     4262      -49     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

xuxihao1 and others added 14 commits November 23, 2023 14:50
Signed-off-by: xuxihao1 <xuxihao3@jd.com>
… opcode type and limiter context

Signed-off-by: xuxihao1 <xuxihao3@jd.com>
Signed-off-by: zhangchuanqing1 <zhangchuanqing1@jd.com>
Signed-off-by: zhangchuanqing1 <zhangchuanqing1@jd.com>
Signed-off-by: xuxihao1 <xuxihao3@jd.com>
Signed-off-by: xuxihao1 <xuxihao3@jd.com>
Signed-off-by: xuxihao1 <xuxihao3@jd.com>
Signed-off-by: xuxihao1 <xuxihao3@jd.com>
bjguoweilong and others added 25 commits February 27, 2024 16:38
…-27 16:38:03

修复block check worker 编译失败问题
Signed-off-by: wuwenjia1 <wuwenjia5@jd.com>
…-02-29 14:49:13

优化mark delete时获取data partition视图的方式,避免频繁分配内存
Signed-off-by: xuxihao1 <xuxihao3@jd.com>
…on 2024-02-29 14:57:49

更新dp视图改为分批更新
…l on 2024-02-29 14:59:02

do not verify inode on extent delete
…on 2024-03-01 14:45:34

增加bad request请求的监控key
Signed-off-by: xuxihao1 <xuxihao3@jd.com>
…5:14:54

fix: update access time of eh.conn
…5 15:15:43

[online_0701] Optimize IO operations for storage engine
Signed-off-by: bjguoweilong <guoweilong@jd.com>
Signed-off-by: bjguoweilong <guoweilong@jd.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants