Does enc-dec model support inflight bathing?

[Oldpan]

I found that trt-llm-dev supports paged KV cache for encoder-decoder models (like Nougat). Does the encoder-decoder model work with inflight batching? Or can we use other batching methods other than static batching to improve performance? Thanks

[schetlur-nv]

@Oldpan for now we only support static batching for encoder-decoder models. But we plan inflight batching support in the coming weeks. Please keep an eye out. Feel free to reopen the issue if you have more questions.