Running Error

[qiuhanauto]

Hi, thank you for opening this great project!
When I tried to run the code, I encountered the following error prompt:

Process Process-2:
Traceback (most recent call last):
  File "/home/han/miniconda3/envs/uncle-slam/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/han/miniconda3/envs/uncle-slam/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/han/os/UncLe-SLAM-main/src/uncle_slam.py", line 265, in mapping
    self.mapper.run()
  File "/home/han/os/UncLe-SLAM-main/src/mapper.py", line 735, in run
    _ = self.optimize_map(
  File "/home/han/os/UncLe-SLAM-main/src/mapper.py", line 578, in optimize_map
    ret = self.renderer.render_batch_ray(
  File "/home/han/os/UncLe-SLAM-main/src/utils/renderer.py", line 238, in render_batch_ray
    raw, var = self.eval_points(pointsf, d_feats, c_feats, decoders, c, stage, device)
  File "/home/han/os/UncLe-SLAM-main/src/utils/renderer.py", line 73, in eval_points
    ret, var = decoders(pi, c_grid=c, stage=stage, d_feats=d_feats, c_feats=c_feats)
  File "/home/han/miniconda3/envs/uncle-slam/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/han/os/UncLe-SLAM-main/src/conv_onet/models/decoder.py", line 496, in forward
    middle_occ = self.middle_decoder(p, c_grid)
  File "/home/han/miniconda3/envs/uncle-slam/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/han/os/UncLe-SLAM-main/src/conv_onet/models/decoder.py", line 237, in forward
    embedded_pts = self.embedder(p)
  File "/home/han/miniconda3/envs/uncle-slam/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/han/os/UncLe-SLAM-main/src/conv_onet/models/decoder.py", line 30, in forward
    x = x @ self._B.to(x.device)
RuntimeError: CUDA error: CUBLAS_STATUS_INVALID_VALUE when calling `cublasSgemm( handle, opa, opb, m, n, k, &alpha, a, lda, b, ldb, &beta, c, ldc)`

Can you give some advice on how to solve the problem? Sincerely looking forward to your reply!

[kev-in-ta]

Hmm, I dug around online and saw that the error could be caused by a number of different issues (what a descriptive error 🙃).

The two possible causes that stand-out to me are memory issues or some kind of cuda version mismatch possibly? Perhaps check if your installed cuda version matches your pytorch installation.

Some resources:
https://discuss.pytorch.org/t/runtimeerror-cuda-error-cublas-status-invalid-value-when-calling-cublassgemm-handle-opa-opb-m-n-k-alpha-a-lda-b-ldb-beta-c-ldc/124544
https://stackoverflow.com/questions/66600362/runtimeerror-cuda-error-cublas-status-execution-failed-when-calling-cublassge
https://stackoverflow.com/questions/68383634/cuda-error-cublas-status-invalid-value-error-when-training-bert-model-using-hug

[eriksandstroem]

Hi @qiuhanauto,
Were you able to resolve the issue?

[qiuhanauto]

Hi @eriksandstroem , I still haven't solved this problem, following the advice given by the author, checking revealed that the problem is not caused by the memory or the cuda version. BTW, I've been following your work recently, thanks for releasing the Point-SLAM code!