-
Notifications
You must be signed in to change notification settings - Fork 654
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
缓存目录使用混乱,MODELSCOPE_CACHE环境变量在不同位置的作用不一致 #845
Comments
感谢整理,我们看一下 |
wenmengzhou
assigned liuyhwangyh and zzhangpurdue and unassigned wangxingjun778 and tastelikefeet
Apr 27, 2024
Merged
liuyhwangyh
added a commit
that referenced
this issue
May 23, 2024
liuyhwangyh
pushed a commit
that referenced
this issue
May 23, 2024
wenmengzhou
pushed a commit
that referenced
this issue
May 24, 2024
* fix #845 Supports resumption of downloads from breakpoints, optimized download progress bar, finer display granularity, better experience under low bandwidth, and added function of downloading specified directories. * restore push to hub * fix merge issue * fix ut issue --------- Co-authored-by: mulin.lyh <mulin.lyh@taobao.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the feature
(已经
pip install -U modelscope
更新到最新版,目前版本号是:modelscope==1.14.0
)modelscope项目里目前默认的缓存目录或者说主目录是:
~/.cache/modelscope
,但是当通过设置环境变量,在这个非默认目录下加载或者下载模型时,就会遇到和默认目录不一致的问题。比如希望把目标目录改成:
/data0/modelscope
,希望通过pipeline加载的模型damo/nlp_raner_named-entity-recognition_chinese-base-ecom
,以及当前在任何目录下都没有这份模型文件。在代码中加载模型的语句如下:
~/.cache/modelscope/ast_indexer
,下载时的临时目录~/.cache/modelscope/hub/temp
,最终模型文件被下载到~/.cache/modelscope/hub/damo/nlp_raner_named-entity-recognition_chinese-base-ecom
MS_CACHE_HOME=/data0/modelscope
,这个环境变量完全不起作用,仍然在~/.cache/modelscope
目录下载文件MODELSCOPE_CACHE=/data0/modelscope
,则import modelscope
时会创建索引文件/data0/modelscope/ast_indexer
,同时下载模型文件的临时目录会使用/data0/modelscope/temp
,最终模型文件被下载到/data0/modelscope/damo/nlp_raner_named-entity-recognition_chinese-base-ecom
。这时/data0/modelscope/hub
目录永远不会被使用。MODELSCOPE_CACHE=~/.cache/modelscope
,则import modelscope
时会创建索引文件~/.cache/modelscope/ast_indexer
,同时下载模型文件的临时目录会使用~/.cache/modelscope/temp
,最终模型文件被下载到~/.cache/modelscope/damo/nlp_raner_named-entity-recognition_chinese-base-ecom
。这时~/.cache/modelscope/hub
目录永远不会被使用。Motivation
统一环境变量
MS_CACHE_HOME
和MODELSCOPE_CACHE
的作用。当把缓存目录设置到非默认位置时,规范下载行为在hub
子目录进行。The text was updated successfully, but these errors were encountered: