You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am encountering issues with SQLite in my distributed Dask setup, where I use multiple nodes for job submissions. I suspect that SQLite's limitations with read-write locks in a distributed environment are causing these issues. Based on what I've read, transitioning to PostgreSQL or possibly utilizing journal file storage seems like viable solutions. Could you confirm if this is the recommended approach for a multi-node setup?
My Setup and Problem:
I am implementing a backward greedy feature selection algorithm involving two nested loops. In the inner loop, I temporarily remove each feature to evaluate its impact via a small set of Optuna trials. After identifying the most valuable features, I proceed to a final optimization phase.
Questions:
Multiple Studies vs. Single Study:
Would it be more appropriate to encapsulate all operations under one study, or can I create multiple studies for each iteration of my feature selection loop? I am inclined to use multiple studies to keep trials organized and manageable.
Consistent Trial Outcomes:
Occasionally, I observe that some trials yield identical performance results. This is unexpected and I'm unsure why this might be happening. Is there a common cause for such occurrences in Optuna that I should look into?
Optuna with PostgreSQL:
Are there specific configuration details or best practices for integrating Optuna with PostgreSQL in a distributed Dask environment? Any examples or documentation would be greatly appreciated.
Thx for your support and the great work on Optuna.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Hello,
I am encountering issues with SQLite in my distributed Dask setup, where I use multiple nodes for job submissions. I suspect that SQLite's limitations with read-write locks in a distributed environment are causing these issues. Based on what I've read, transitioning to PostgreSQL or possibly utilizing journal file storage seems like viable solutions. Could you confirm if this is the recommended approach for a multi-node setup?
My Setup and Problem:
I am implementing a backward greedy feature selection algorithm involving two nested loops. In the inner loop, I temporarily remove each feature to evaluate its impact via a small set of Optuna trials. After identifying the most valuable features, I proceed to a final optimization phase.
Questions:
Multiple Studies vs. Single Study:
Consistent Trial Outcomes:
Optuna with PostgreSQL:
Thx for your support and the great work on Optuna.
Best regards,
Ademord
Beta Was this translation helpful? Give feedback.
All reactions