minigpt4_video_demo Llama-2-7b-chat-hf

[Mikes95]

I'm trying to run demo code. I cloned the repository and installed all the dependencies. I downloaded video_llama_checkpoint_last.pth and placed it in the checkpoints folder. When I run python minigpt4_video_demo.py --ckpt /miniGPT4/MiniGPT4-video/checkpoints/video_llama_checkpoint_last.pth --cfg-path test_configs/llama2_test_config.yaml, I receive this error:

OSError: Can't load tokenizer for 'meta-llama/Llama-2-7b-chat-hf'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'meta-llama/Llama-2-7b-chat-hf' is the correct path to a directory containing all relevant files for a LlamaTokenizer.

How can I fix this?
Is there also a demo available purely in code, without using Streamlit for an HTML interface?

[KerolosAtef]

Hello @Mikes95 ,
thank you for your interest in our work.
for 'meta-llama/Llama-2-7b-chat-hf' this repo is gated model in huggingface, first try grant the access by accepting the license and make sure you have this sentence in your huggingface account

then generate a token from huggingface tokens and add these 2 lines of code in the beginning of the demo file

from huggingface_hub import login
login(token="your generated token")

for the pure python code without UI, you can try the inference file minigpt4_video_inference.py

[Mikes95]

Thank you for the prompt response. I was also wondering how easy it is to perform fine-tuning on my own dataset and whether there is more detailed documentation than the README on how to format the videos and their corresponding texts. Thank you in advance.

[KerolosAtef]

Hello @Mikes95 , thank you your nice question, I added a readme file for the steps about how to add a new dataset here Custom_training