Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OSError: [Errno 24] Too many open files #4

Open
Hocnonsense opened this issue Jan 17, 2023 · 0 comments
Open

OSError: [Errno 24] Too many open files #4

Hocnonsense opened this issue Jan 17, 2023 · 0 comments

Comments

@Hocnonsense
Copy link

First of all, thanks for your repository!

I'm trying to use this repository to estimate the temperature optima of my genomes. However, when I run it with 12 threads for about 2000 genomes, this software broken down after finished about 40 genomes.
This is the error message:

/data/home/lenghao/software/miniconda3/envs/tome/lib/python3.6/site-packages/sklearn/base.py:253: UserWarning: Trying to unpickle estimator SVR from version 0.19.2 when using version 0.20.3. This might lead to breaking code or invalid results. Use at your own risk.
  UserWarning)
Traceback (most recent call last):
  File "/data/home/lenghao/software/miniconda3/envs/tome/bin/tome", line 33, in <module>
    sys.exit(load_entry_point('tome', 'console_scripts', 'tome')())
  File "/data/home/lenghao/software/Tome-master/tome/tome.py", line 106, in main
    else: predOGT.main(args)
  File "/data/home/lenghao/software/Tome-master/tome/core/predOGT.py", line 199, in main
    pred_ogt = predict(os.path.join(indir,name),model,means,stds,features,args.threads)
  File "/data/home/lenghao/software/Tome-master/tome/core/predOGT.py", line 170, in predict
    dimers_fq = get_dimer_frequency(fasta_file,p)
  File "/data/home/lenghao/software/Tome-master/tome/core/predOGT.py", line 156, in get_dimer_frequency
    dimers = count_dimer(fasta_file,p)
  File "/data/home/lenghao/software/Tome-master/tome/core/predOGT.py", line 151, in count_dimer
    results = Pool(num_cpus).map(do_count, seqs)
  File "/data/home/lenghao/software/miniconda3/envs/tome/lib/python3.6/multiprocessing/context.py", line 119, in Pool
    context=self.get_context())
  File "/data/home/lenghao/software/miniconda3/envs/tome/lib/python3.6/multiprocessing/pool.py", line 174, in __init__
    self._repopulate_pool()
  File "/data/home/lenghao/software/miniconda3/envs/tome/lib/python3.6/multiprocessing/pool.py", line 239, in _repopulate_pool
    w.start()
  File "/data/home/lenghao/software/miniconda3/envs/tome/lib/python3.6/multiprocessing/process.py", line 105, in start
    self._popen = self._Popen(self)
  File "/data/home/lenghao/software/miniconda3/envs/tome/lib/python3.6/multiprocessing/context.py", line 277, in _Popen
    return Popen(process_obj)
  File "/data/home/lenghao/software/miniconda3/envs/tome/lib/python3.6/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/data/home/lenghao/software/miniconda3/envs/tome/lib/python3.6/multiprocessing/popen_fork.py", line 65, in _launch
    parent_r, child_w = os.pipe()
OSError: [Errno 24] Too many open files
/data/home/lenghao/software/miniconda3/envs/tome/lib/python3.6/site-packages/sklearn/base.py:253: UserWarning: Trying to unpickle estimator SVR from version 0.19.2 when using version 0.20.3. This might lead to breaking code or invalid results. Use at your own risk.
  UserWarning)
Traceback (most recent call last):
  File "/data/home/lenghao/software/miniconda3/envs/tome/bin/tome", line 33, in <module>
    sys.exit(load_entry_point('tome', 'console_scripts', 'tome')())
  File "/data/home/lenghao/software/Tome-master/tome/tome.py", line 106, in main
    else: predOGT.main(args)
  File "/data/home/lenghao/software/Tome-master/tome/core/predOGT.py", line 199, in main
    pred_ogt = predict(os.path.join(indir,name),model,means,stds,features,args.threads)
  File "/data/home/lenghao/software/Tome-master/tome/core/predOGT.py", line 170, in predict
    dimers_fq = get_dimer_frequency(fasta_file,p)
  File "/data/home/lenghao/software/Tome-master/tome/core/predOGT.py", line 156, in get_dimer_frequency
    dimers = count_dimer(fasta_file,p)
  File "/data/home/lenghao/software/Tome-master/tome/core/predOGT.py", line 151, in count_dimer
    results = Pool(num_cpus).map(do_count, seqs)
  File "/data/home/lenghao/software/miniconda3/envs/tome/lib/python3.6/multiprocessing/context.py", line 119, in Pool
    context=self.get_context())
  File "/data/home/lenghao/software/miniconda3/envs/tome/lib/python3.6/multiprocessing/pool.py", line 174, in __init__
    self._repopulate_pool()
  File "/data/home/lenghao/software/miniconda3/envs/tome/lib/python3.6/multiprocessing/pool.py", line 239, in _repopulate_pool
    w.start()
  File "/data/home/lenghao/software/miniconda3/envs/tome/lib/python3.6/multiprocessing/process.py", line 105, in start
    self._popen = self._Popen(self)
  File "/data/home/lenghao/software/miniconda3/envs/tome/lib/python3.6/multiprocessing/context.py", line 277, in _Popen
    return Popen(process_obj)
  File "/data/home/lenghao/software/miniconda3/envs/tome/lib/python3.6/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/data/home/lenghao/software/miniconda3/envs/tome/lib/python3.6/multiprocessing/popen_fork.py", line 65, in _launch
    parent_r, child_w = os.pipe()
OSError: [Errno 24] Too many open files

This seems to be caused by opening too many files without closing it.
Do you have any idea about this problem and to solve it?
Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant