[BUG] API KEY is not extracted from environment variables during connection creation with promptflow 1.10.1

[AlexTeslenko]

Describe the bug
If Azure OpenAI API KEY is set as environment variable, and in connection yaml file it is set to get this value from environment variables, the value is not set during connection creation and it leads to failed running flow.

How To Reproduce the bug

1. Set AOAI_API_KEY as environment variable
2. Create connection yaml file with api_key taken from env variables:

$schema: 
  https://azuremlschemas.azureedge.net/promptflow/latest/AzureOpenAIConnection.schema.json
type: azure_open_ai
name: aoai
module: promptflow.connections
api_base: https://teslenkoopenairesource.openai.azure.com/
api_key: ${env:AOAI_API_KEY}
api_type: azure
api_version: 2023-03-15-preview

3.Run pf connection create --file /connections/aoai.yaml
4. Run flow with set connection.

Expected behavior
WIth promptlow==1.9.0 everything works fine. Conection can be created and used where api_key is taken from environment variable.

Also everything works if set api_key explicitly:
pf connection create --file /connections/aoai.yaml --set api_key=$AOAI_API_KEY

Running Information(please complete the following information):

• Promptflow Package Version: 1.10.1
• Operating System: Ubuntu 20.04, MacOs Sonoma 14.4.1
• Python Version using: 3.9.11

[brynn-code]

@AlexTeslenko I just test 'pf flow test' and 'pf flow serve' with 1.9.0 and 1.10.1, we only commit this work for flow serve, and flow serve works well, while flow test doesn't work. Could you please elaborate more about your scenario, which command works fine before and broken afterwards?

[AlexTeslenko]

@brynn-code thank you for checking this issue. I had this problem when I created docker image using promtflow and tried to run created image.

[brynn-code]

@AlexTeslenko Sounds like you are using the image by 'pf flow build'. When running 'pf flow build', the connection yaml will be re-extracted with an environment variable reference using default rule, so it could be different from your environment variable name in the original yaml. The name rule was expected as <connection_name>_, for example this is the api_key from the build result:

Could you please check the connection yaml under <your output path>/connections to see the env var name reference?