Skip to content
This repository has been archived by the owner on Nov 14, 2023. It is now read-only.

gpu usage #228

Open
Eeonum opened this issue Nov 21, 2021 · 3 comments
Open

gpu usage #228

Eeonum opened this issue Nov 21, 2021 · 3 comments

Comments

@Eeonum
Copy link

Eeonum commented Nov 21, 2021

Hi I am getting this error when I set use_gpu=True.

(scheduler +6s) Tip: use ray status to view detailed cluster status. To disable these messages, set RAY_SCHEDULER_EVENTS=0.
(scheduler +6s) Error: No available node types can fulfill resource request {'CPU': 1.0, 'GPU': 1.0}. Add suitable node types to this cluster to resolve this issue.
(scheduler +41s) Error: No available node types can fulfill resource request {'GPU': 1.0, 'CPU': 1.0}. Add suitable node types to this cluster to resolve this issue.
Ignore this message if the cluster is autoscaling. You asked for 1.0 cpu and 1.0 gpu per trial, but the cluster only has 10.0 cpu and 0 gpu. Stop the tuning job and adjust the resources requested per trial (possibly via resources_per_trial or via num_workers for rllib) and/or add more resources to your Ray runtime.
(scheduler +1m16s) Error: No available node types can fulfill resource request {'GPU': 1.0, 'CPU': 1.0}. Add suitable node types to this cluster to resolve this issue.
(scheduler +1m51s) Error: No available node types can fulfill resource request {'CPU': 1.0, 'GPU': 1.0}. Add suitable node types to this cluster to resolve this issue.
Ignore this message if the cluster is autoscaling. You asked for 1.0 cpu and 1.0 gpu per trial, but the cluster only has 10.0 cpu and 0 gpu. Stop the tuning job and adjust the resources requested per trial (possibly via resources_per_trial or via num_workers for rllib) and/or add more resources to your Ray runtime.
(scheduler +2m26s) Error: No available node types can fulfill resource request {'CPU': 1.0, 'GPU': 1.0}. Add suitable node types to this cluster to resolve this issue.
(scheduler +3m1s) Error: No available node types can fulfill resource request {'CPU': 1.0, 'GPU': 1.0}. Add suitable node types to this cluster to resolve this issue.
Ignore this message if the cluster is autoscaling. You asked for 1.0 cpu and 1.0 gpu per trial, but the cluster only has 10.0 cpu and 0 gpu. Stop the tuning job and adjust the resources requested per trial (possibly via resources_per_trial or via num_workers for rllib) and/or add more resources to your Ray runtime.
(scheduler +3m36s) Error: No available node types can fulfill resource request {'GPU': 1.0, 'CPU': 1.0}. Add suitable node types to this cluster to resolve this issue.

@Eeonum
Copy link
Author

Eeonum commented Nov 21, 2021

was trying to run the following script:

`"""
An example training a CatBoostClassifier, performing
randomized search using TuneSearchCV.
"""

from tune_sklearn import TuneSearchCV
from sklearn import datasets
from sklearn.model_selection import train_test_split
from catboost import CatBoostClassifier

digits = datasets.load_digits()
x = digits.data
y = digits.target
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=.2)

params = {
"learning_rate": [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9],
"depth": [3, 4, 5],
}

catboostclf = CatBoostClassifier(n_estimators=50, )

digit_search = TuneSearchCV(
catboostclf,
param_distributions=params,
n_trials=3,
early_stopping=True,
use_gpu=True # Commented out for testing on github actions,
# but this is how you would use gpu
)

digit_search.fit(x_train, y_train)
print(digit_search.best_params_)
print(digit_search.cv_results_)`

@papelypluma
Copy link

hi @Eeonum by any chance I'd like to ask whether you were able to address this issue on your end. I'm trying to run a similar one, but I keep getting the same error. Thanks!

@HoBeedzc
Copy link

HoBeedzc commented Aug 1, 2023

hi, @Eeonum by any chance I'd like to ask whether you could address this issue on your end. I'm trying to run a similar one but keep getting the same error. Thanks!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants