Added a REST api service with fastapi for simple inferencing.

[w4hns1nn]

PR type

• New feature REST API.

Description

Added a REST service with fastapi. By starting the server with "python rvc_fastapy.py" the REST service is launched under localhost:8080. The API is well documented and easy to use.

Screenshot

[Kamikadashi]

This significantly speeds up the inference process compared to the currently available API.

[fumiama]

This significantly speeds up the inference process compared to the currently available API.

I agree with you. Need @RVC-Boss to decide whether merge or not.

[iotgopher]

this would be a great addition - please @RVC-Boss can you help merging?

[joshlatham]

Hi, I have cloned this solution to test it out but I am not able to successfully process using the api. No matter what I put as the value for input_path or input_file, I am getting "detail": "infer() missing 1 required positional argument: 'input'" or {"detail":[{"loc":["body","input_file"],"msg":"field required","type":"value_error.missing"}]}. I've tried both the endpoints local and non local.

[w4hns1nn]

Hey, I already made some changes there to improve and added also pure rest inference points without local paths, what works even better. I did not know how ho update the pull request with git though

[Kamikadashi]

Hey, I already made some changes there to improve and added also pure rest inference points without local paths, what works even better. I did not know how ho update the pull request with git though

The last changes I see are dated three weeks ago. Did you make any other changes?

[iotgopher]

@w4hns1nn maybe you can create a new PR?

[Kamikadashi]

Hi, I have cloned this solution to test it out but I am not able to successfully process using the api. No matter what I put as the value for input_path or input_file, I am getting "detail": "infer() missing 1 required positional argument: 'input'" or {"detail":[{"loc":["body","input_file"],"msg":"field required","type":"value_error.missing"}]}. I've tried both the endpoints local and non local.

def send_request(file_path):
    
    url = "http://localhost:8001/voice2voice"
    params = {
        "model_name": "Emilia.pth",
        "index_path": "None",
        "f0up_key": -1,
        "f0method": "rmvpe",
        "index_rate": 0.66,
        "device": "cuda",
        "is_half": "False",
        "filter_radius": 3,
        "resample_sr": 0,
        "rms_mix_rate": 0.25,
        "protect": 0.33
    }
    with open(file_path, "rb") as f:
        files = {"input_file": f}
        response = session.post(url, files=files, params=params, stream=True)
    print(f"Status code: {response.status_code}")

    # Save the response as a .wav file in the same directory as the script
    dir_path = os.path.dirname(os.path.realpath(__file__))
    output_file_path = os.path.join(dir_path, "output.wav")
    with open(output_file_path, "wb") as f:
        for chunk in response.iter_content(chunk_size=8192):
            f.write(chunk)

    return output_file_path

or

def send_request(file_data):
    url = "http://localhost:8001/voice2voice"
    params = {
        "model_name": "Emilia.pth",
        "index_path": "None",
        "f0up_key": -1,
        "f0method": "rmvpe",
        "index_rate": 0.66,
        "device": "cuda",
        "is_half": "False",
        "filter_radius": 3,
        "resample_sr": 0,
        "rms_mix_rate": 0.25,
        "protect": 0.33
    }
    files = {"input_file": file_data}
    response = session.post(url, files=files, params=params, stream=True)

    # Return the response content as a byte stream
    response_content = response.content

    return response_content

[fumiama]

Hey, I already made some changes there to improve and added also pure rest inference points without local paths, what works even better. I did not know how ho update the pull request with git though

Just edit your fork https://github.com/w4hns1nn/Retrieval-based-Voice-Conversion-FastAPI/tree/main and it will update to this PR automatically.

[markyfsun]

Could you please also add a response_format param?

[markyfsun]

Could you also add a response_format parameter here to decide whether to response in wav or mp3 format?
Like this:

def infer(
        input: Union[str, bytes],  # filepath or raw bytes
        model_name: str,
        response_format: str = 'wav',  # new parameter for specifying the output format
        index_path: str = None,
        f0up_key: int = 0,
        f0method: str = "crepe",
        index_rate: float = 0.66,
        device: str = None,
        is_half: bool = False,
        filter_radius: int = 3,
        resample_sr: int = 0,
        rms_mix_rate: float = 1,
        protect: float = 0.33,
        **kwargs
    ):
    if response_format.lower() not in ['wav', 'mp3', 'flac', 'ogg', 'aac']:
        raise ValueError(f"Unsupported response_format: {response_format}. Supported formats: wav, mp3, flac, ogg, aac")
    model_name = model_name.replace(".pth", "")

    if index_path is None:
        index_path = os.path.join("logs", model_name, f"added_IVF1254_Flat_nprobe_1_{model_name}_v2.index")
        if not os.path.exists(index_path):
            index_path = None
            # raise ValueError(f"autinferred index_path {index_path} does not exist. Please provide a valid index_path")

    vc = model_cache.load_model(model_name, device=device, is_half=is_half)

    _, wav_opt = vc.vc_single(
        sid=0,
        input_audio_path=input,
        f0_up_key=f0up_key,
        f0_file=None,
        f0_method=f0method,
        file_index=index_path,
        file_index2=None,
        index_rate=index_rate,
        filter_radius=filter_radius,
        resample_sr=resample_sr,
        rms_mix_rate=rms_mix_rate,
        protect=protect
    )

    # using virtual file to be able to return it as response
    wf = BytesIO()

    # Check if the desired response format is valid and supported


    # Save the output in the desired format
    sf.write(wf, wav_opt[1], wav_opt[0], format=response_format.upper())

    # Make sure to seek the beginning of the BytesIO object before returning it
    wf.seek(0)

    return wf

[markyfsun]

The idea is from OpenAI TTS API, which allows user to decide response format

[markyfsun]

Some models do not come with index files. If fails to find index_path, how about leaving it to None instead of raising an Exception?

[Kamikadashi]

The current version works without an index file just fine, just set "index_path": "None"

[w4hns1nn]

Could you please also add a response_format param?

I will do, in a few weeks

[Tps-F]

API is under development in this repository.
https://github.com/RVC-Project/Retrieval-based-Voice-Conversion

The base branch was changed.

[iielse]

Sorry, I'm new to using git. REST api service with fastapi This function has not been merged into the main or dev branch at present, right? Because I have not seen the existence of the rvc_fastapi.py file on main or dev.

[fumiama]

Yes, it is closed by a mistake. I will re-open it.

[iielse]

Today I performed a request test on the function of this code and it works normally.

--> END POST (1286684-byte body)
<-- 200 OK http://localhost:8001/voice2voice?model_name=qiqi-jp.pth&index_path=log%2Fadded_IVF985_Flat_nprobe_1_qiqi-jp_v2.index&f0up_key=0&f0method=rmvpe&index_rate=1.0&device=cuda%3A0&is_half=false&filter_radius=3&resample_sr=0&rms_mix_rate=1&protect=0.33 (643ms)
date: Thu, 14 Mar 2024 06:41:40 GMT
server: uvicorn
content-disposition: attachment; filename=rvc.wav
content-type: audio/wav
Transfer-Encoding: chunked

<-- END HTTP (binary 641324-byte body omitted)
File saved successfully! D:\audioApiOut.wav
cost time: 4152ms

[Kamikadashi]

It’s baffling to me that this hasn’t been merged into the main branch after all this time. Without this API, RVC is literally unusable for my use case, and I doubt I’m the only one.

[ouarrtaha]

i was playing with this branch, it was useful, but didn't u take in consideration concurrency, how we can deal with multiple requests, if u have multiple simultaneous requests will significally slow down inference time, can we do something like batching...?