Cannot use GPU on tensorflow and display the game at the same time ?

[Lledune]

Is this a normal behavior ?
When I start the environment and display it, it works fine at first.
But as soon as I load tensorflow (which is using my GPU) i get the error :
"ViZDoomErrorException: Could not initialize SDL video:
No available video device"

So I'm guessing that tensorflow is blocking teh access to my GPU or something ? Sorry I'm really new to using GPU's for neural networks and I don't know if that behavior is intended or if I'm doing something wrong.

I'm using python 3.8.3 on Linux mint 20.

[Miffyli]

Hmm I have not had trouble using CUDA-based libraries (Theano, Tensorflow, PyTorch) with ViZDoom in the past like this. I do not think it should lock out GPU this way. Could you include full code to replicate the issue so I can try it on my linux machines, as well as tensorflow version you are using?

[mihahauke]

Same here, never head problems with running it on gpu with theano, tf1 or pytorch. If I am not mistaken tensorflow is kind of greedy if it comes to reserving memory but it has never caused this kind of behavior for me. Are you using TF 2?

[Miffyli]

@mihahauke TF "fixed" that issue in the latest 1.x and in 2.x versions, granted I have not tried TF2 with ViZDoom on Linux yet.

[mihahauke]

@Miffyli good to know. Though I moved to Pytorch after tf1.12.

[Lledune]

@mihahauke Yes I'm using TF 2.3.1

@Miffyli Here's the full code, if tensorflow is commented like this, it works, but if i load it then i get the error.

#import tensorflow as tf      # Deep Learning library
import numpy as np           # Handle matrices
from vizdoom import *        # Doom Environment

import random                # Handling random number generation
import time                  # Handling time calculation
from skimage import transform# Help us to preprocess the frames

from collections import deque# Ordered collection with ends
import matplotlib.pyplot as plt # Display graphs

import warnings # This ignore all the warning messages that are normally printed during the training because of skiimage
warnings.filterwarnings('ignore')

basiccfg = "/home/lucien/anaconda3/lib/python3.8/site-packages/vizdoom/scenarios/basic.cfg"
basicwad = "/home/lucien/anaconda3/lib/python3.8/site-packages/vizdoom/scenarios/basic.wad"


"""
Here we create our environment
"""
def create_environment():
    game = DoomGame()
    
    game.load_config("/home/lucien/anaconda3/lib/python3.8/site-packages/vizdoom/scenarios/basic.cfg")
    game.set_doom_scenario_path("/home/lucien/anaconda3/lib/python3.8/site-packages/vizdoom/scenarios/basic.wad")
    
    game.init()
    
    # Here our possible actions
    left = [1, 0, 0]
    right = [0, 1, 0]
    shoot = [0, 0, 1]
    possible_actions = [left, right, shoot]
    
    return game, possible_actions
       
"""
Here we performing random action to test the environment
"""
def test_environment():
    game = DoomGame()
    game.load_config("/home/lucien/anaconda3/lib/python3.8/site-packages/vizdoom/scenarios/basic.cfg")
    game.set_doom_scenario_path("/home/lucien/anaconda3/lib/python3.8/site-packages/vizdoom/scenarios/basic.wad")
    game.init()
    shoot = [0, 0, 1]
    left = [1, 0, 0]
    right = [0, 1, 0]
    actions = [shoot, left, right]

    episodes = 10
    for i in range(episodes):
        game.new_episode()
        while not game.is_episode_finished():
            state = game.get_state()
            img = state.screen_buffer
            misc = state.game_variables
            action = random.choice(actions)
            print(action)
            reward = game.make_action(action)
            print ("\treward:", reward)
            time.sleep(0.02)
        print ("Result:", game.get_total_reward())
        time.sleep(2)
    game.close()

test_environment()

[Miffyli]

I was able to run the code with and without tensorflow import as expected (Python 3.8.5 [conda env], Ubuntu 20.04 [KDE Neon], Tensorflow 2.3.1 [tried with and without GPU], ViZDoom from current master). Seems like the issue is somewhere else :/

[Lledune]

Damn it ... Do you have any idea what i can search for ?

[Miffyli]

Some random suggestions:

• Try other similar environments to see if error persists (e.g. Obstacle tower)
• Try other libraries (e.g. Tensorflow 1.x, PyTorch with cuda. In PyTorch you might need to move something to cuda side to trigger loading of the libraries, e.g. foo = torch.rand(5).cuda()).
• Avoid visualizing the game and instead record it to record files (see examples for recording demos) or recording as video (see e.g. this wrapper and VideoRecorder it uses).

Other than this I do not know what to look for as the error seems rather arbitrary (it should not happen).

[Maxwell2017]

Damn it ... Do you have any idea what i can search for ?

Are you running the test sample on the server?