make graph capture apis easy to understand

[wangyems]

Split the current API (try_graph_with_max_batch_size) into use_graph_capture() and set_max_batch_size() so user can ramp in a comfortable pace.

[yufenglee]

I don’t see the change in c API

[yufenglee]

And there is a conflict

[yufenglee]

And we may not call the API and use default in the model_generate and phi3_qa examples

[wangyems]

I don’t see the change in c API

do we expose c API to users?

[wangyems]

And we may not call the API and use default in the model_generate and phi3_qa examples

I think we can make user aware that we have the API to manipulate max_bs?

[yufenglee]

I don’t see the change in c API

do we expose c API to users?

yes, we have C API:

onnxruntime-genai/src/ort_genai_c.h

Line 154 in 3296782

OGA_EXPORT OgaResult* OGA_API_CALL OgaGeneratorParamsTryGraphCaptureWithMaxBatchSize(OgaGeneratorParams* generator_params, int32_t max_batch_size);

[yufenglee]

I think it is better to just rename the python API to try_graph_capture_with_max_batch_size. C and C# language already use this name and they are right.

Splitting the function into 2 doesn't look natural. The SetMaxBatchSize is just for graph capture, but the name doesn't reflect that. And we need to call 2 functions for one atomic operation.