[DNM] Proposal for Adding local model loading and automatically dependency installation in webui #91
Conversation
Signed-off-by: Xue, Chendi <chendi.xue@intel.com>
|
cc @KepingYan |
| self.ssh_connect[index].load_system_host_keys() | ||
| self.ssh_connect[index].set_missing_host_key_policy(paramiko.RejectPolicy()) | ||
| #self.ssh_connect[index].load_system_host_keys() | ||
| self.ssh_connect[index].set_missing_host_key_policy(paramiko.AutoAddPolicy()) |
There was a problem hiding this comment.
We can't use AutoAddPolicy because of code scan issue.
There was a problem hiding this comment.
@KepingYan , I see, thanks for the clarification, I'll make the change
| @@ -794,23 +852,24 @@ def set_rag_default_path(self, selector, rag_path): | |||
|
|
|||
| def _init_ui(self): | |||
| mark_alive = None | |||
| private_key = paramiko.Ed25519Key.from_private_key_file("/root/.ssh/id_ed25519") | |||
There was a problem hiding this comment.
Why use a custom path to set private key? self.ssh_connect[index].load_system_host_keys() can load the content of '~/.ssh/known_hosts' by default.
There was a problem hiding this comment.
For some reason, the id_rsa decoder is reporting error from my env, I found similiar report on google.
Using a path provides flexibility for user to decide if they want to use ED25519 or RSA. I believe in some "very Healthy" system, self.ssh_connect[index].load_system_host_keys() may also works, but not in my system...
So if you can provides this automatic option + a user defined path, it will greatly prevent user hard code to make web_ui working.
There was a problem hiding this comment.
OK, I see. I'm not sure about the format of your known_hosts, could you verify this method in your env?
For example, the node ip is 10.1.0.133, then run ssh-keyscan 10.1.0.133 >> ~/.ssh/known_hosts. And then modify 10.1.0.133 to [10.1.0.133]:22 like this:
If this doesn't work, it really needs a fallback method.
| lib_list = ["httpcore", "httpx", "paramiko", "urllib3", "markdown_it", "matplotlib"] | ||
| for lib in lib_list: | ||
| logging.getLogger(lib).setLevel(logging.ERROR) |
There was a problem hiding this comment.
If this is to solve the DEBUG log problem, I think it can be solved after #101 is merged.
| else: | ||
| raise ValueError(f"{package_or_list} with type of {type(package_or_list)} is not supported.") | ||
|
|
||
| check_availability_and_install(['gradio==3.36.1', 'gradio_client==0.7.3', 'langchain==0.1.4', 'langchain-community==0.0.16', 'lz4', 'sentence-transformers==2.2.2', 'pyrecdp']) |
There was a problem hiding this comment.
There is a script in #101 to install the UI environment, maybe we don't need to check it in code anymore. And we have to use pip install 'git+https://github.com/intel/e2eAIOK.git#egg=pyrecdp&subdirectory=RecDP' to install pyrecdp, pip install pyrecdp does not contain the latest code and will change the ray version.
Collecting ray==2.7.1 (from pyrecdp)
Using cached ray-2.7.1-cp39-cp39-manylinux2014_x86_64.whl.metadata (13 kB)
|
Hi, @KepingYan and @carsonwang , I don't meant to propose a merge in the PR. |
TODO: If it makes sense, we can add configuration for User to decide private key path