Behaviour of asyncapi-cli optimize is inconsistent

[davidpiendl]

Hi together,

I observe a strange behaviour of running the cli on Ubuntu 22 and @asyncapi/cli/0.50.0 linux-x64 node-v18.16.1.

First I bundle the main YAML with references from external local files where every schema and message is described in our system. The bundle works like we expect it and validate shows a lot of unused references, which we expect to be removed by the optimize command. But it somehow removes a lot more than expected.

The structure after bundle looks like following:

asyncapi: 2.6.0
...
channels:
  topic:
    description: ...
    parameters:
      clientId: ...
    subscribe:
      summary: ...
      operationId: ...
      message:
        oneOf:
          - $ref: '#/components/messages/message1'
          - $ref: '#/components/messages/message2'
          - $ref: '#/components/messages/message3'
components:
  messages:
      message1:
        messageId: message1
        name: message1
        title: ...
        summary: ...
        contentType: application/json
        payload:
          $ref: '#/components/schemas/message1'
      message2:
        messageId: message2
        name: message2
        title: ...
        summary: ...
        contentType: application/json
        payload:
          $ref: '#/components/schemas/message2'
      message3:
        messageId: message3
        name: message3
        title: ...
        summary: ...
        contentType: application/json
        payload:
          $ref: '#/components/schemas/message3'
  schemas:
    ...

The optimize now also removes every message in components/messages except for the last one in the "oneOf" in channels, even if the references are there and used (and validate didnt detect unused references there). If I change the order of the message refs in channels, I can reproduce this and always the last message is available after optimize. Did someone also observe this behaviour or am I misunderstanding the optimize command?

Thanks in advance, David

[github-actions]

Welcome to AsyncAPI. Thanks a lot for reporting your first issue. Please check out our contributors guide and the instructions about a basic recommended setup useful for opening a pull request.
Keep in mind there are also other channels you can use to interact with AsyncAPI community. For more details check out this issue.

[KhudaDad414]

Hi @davidpiendl, thanks for reporting the issue.
I tried to replicate this issue with these steps but It works as expected:
Original AsyncAPI file:

asyncapi: 2.6.0
info:
  title: Title
  version: 0.0.1
  description: Description
channels:
  topic:
    description: ''
    parameters:
      clientId: ''
    subscribe:
      summary: ''
      operationId: 'ddfdf'
      message:
        oneOf:
          - $ref: '#/components/messages/message1'
          - $ref: '#/components/messages/message2'
          - $ref: '#/components/messages/message3'
components:
  messages:
    message1:
      messageId: message1
      name: message1
      title: ''
      summary: ''
      contentType: application/json
      payload:
        type: object
    message2:
      messageId: message2
      name: message2
      title: ''
      summary: ''
      contentType: application/json
      payload:
        type: object
    message3:
      messageId: message3
      name: message3
      title: ''
      summary: ''
      contentType: application/json
      payload:
        type: object
    message4:
      messageId: message4
      name: message4
      title: ''
      summary: ''
      contentType: application/json
      payload:
        type: object

Based on your description this is what I expected to see:

asyncapi: 2.6.0
info:
  title: Title
  version: 0.0.1
  description: Description
channels:
  topic:
    description: ''
    parameters:
      clientId: ''
    subscribe:
      summary: ''
      operationId: 'ddfdf'
      message:
        oneOf:
          - $ref: '#/components/messages/message1'
          - $ref: '#/components/messages/message2'
          - $ref: '#/components/messages/message3'
components:
  messages:
    message3:
      messageId: message3
      name: message3
      title: ''
      summary: ''
      contentType: application/json
      payload:
        type: object

this is what I got which is expected:

asyncapi: 2.6.0
info:
  title: Title
  version: 0.0.1
  description: Description
channels:
  topic:
    description: ''
    parameters:
      clientId: ''
    subscribe:
      summary: ''
      operationId: ddfdf
      message:
        oneOf:
          - $ref: '#/components/messages/message1'
          - $ref: '#/components/messages/message2'
          - $ref: '#/components/messages/message3'
components:
  messages:
    message1:
      messageId: message1
      name: message1
      title: ''
      summary: ''
      contentType: application/json
      payload:
        type: object
    message2:
      messageId: message2
      name: message2
      title: ''
      summary: ''
      contentType: application/json
      payload:
        type: object
    message3:
      messageId: message3
      name: message3
      title: ''
      summary: ''
      contentType: application/json
      payload:
        type: object

can you provide another example or am I misunderstanding your issue?

[davidpiendl]

Yes you got it right, I experience the complete removement of messages except message3 in this example. If I change the order in message/oneOf, always the last item will stay in the optimized yaml. I doublechecked correct ids/names and cannot see what I am doing wrong here.

The validate command before is completely correct and shows me exactly which messages and schemas are unused.

asyncapi optimize doc.yaml --no-tty -o=new-file

then messes it completely up and the validation afterwards of course shows missing references, because the messages are removed from components. I experience this in my local VM using Ubuntu 22 / node-v18.16.1 and also executing it in a Dockercontainer using debian:11-slim and asyncapi-cli v0.48.8

I experience this with our asyncapi documentation as well as with the example here in the ticket. Any hints?

[KhudaDad414]

@davidpiendl can you try the latest version of CLI to see if the issue persists? the current version is 0.50.2 and also since you are there can you also check if your cli is using the latest version of optimizer (0.1.19)?

[davidpiendl]

@KhudaDad414 I tried it with v0.51.2 and it seems at least that the messages aren't removed anymore in the components->messages section. But what I still experience is, that the unnecessary and not used components->schemas are still there. Our document also references the payload of the message like in the example below.

asyncapi: 2.6.0
info:
  title: Title
  version: 0.0.1
  description: Description
channels:
  topic:
    description: ''
    parameters:
      clientId: ''
    subscribe:
      summary: ''
      operationId: ddfdf
      message:
        oneOf:
          - $ref: '#/components/messages/message1'
components:
  messages:
    message1:
      messageId: message1
      name: message1
      title: ''
      summary: ''
      contentType: application/json
      payload:
        $ref: '#/components/schemas/message_1'
  schemas:
    message_1:
      type: object
      ...

It renders perfectly in the preview, but the optimize command removes the message_1 schema even if its referenced. Am I using the $ref wrongly on the payload?

[alexanderschapelt]

I also did some tests with this minimal spec (I know this produces warnings, but they are not relevant for the issue):

asyncapi: 2.6.0
info:
  title: Some Title
  version: 'v1'
channels:
  a-channel:
    subscribe:
      message:
        oneOf:
          - $ref: '#/components/messages/a-message'
components:
  messages:
    a-message:
      payload:
          $ref: '#/components/schemas/a-message'
  schemas:
    a-message:
      type: object
      properties:
        a-property:
          $ref: '#/components/schemas/a-property'
    a-property:
      type: string
      minLength: 1
    unused-property:
      type: string
      minLength: 2

my expectation would be this (just remove #/components/schemas/unused-property):

asyncapi: 2.6.0
info:
  title: Some Title
  version: 'v1'
channels:
  a-channel:
    subscribe:
      message:
        oneOf:
          - $ref: '#/components/messages/a-message'
components:
  messages:
    a-message:
      payload:
          $ref: '#/components/schemas/a-message'
  schemas:
    a-message:
      type: object
      properties:
        a-property:
          $ref: '#/components/schemas/a-property'
    a-property:
      type: string
      minLength: 1

however, the actual result is this:

asyncapi: 2.6.0
info:
  title: Some Title
  version: v1
channels:
  a-channel:
    subscribe:
      message:
        oneOf:
          - $ref: '#/components/messages/a-message'
components:
  messages:
    a-message:
      payload:
        $ref: '#/components/schemas/a-message'

strangely, asyncapi validate points out that only components.schemas.unused-property is potentially unused. and now the funny part begins. If I remove the oneOf like this:

asyncapi: 2.6.0
info:
  title: Some Title
  version: 'v1'
channels:
  a-channel:
    subscribe:
      message:
        $ref: '#/components/messages/a-message'
components:
  messages:
    a-message:
      payload:
          $ref: '#/components/schemas/a-message'
  schemas:
    a-message:
      type: object
      properties:
        a-property:
          $ref: '#/components/schemas/a-property'
    a-property:
      type: string
      minLength: 1
    unused-property:
      type: string
      minLength: 2

everything works just like I would expect it. Maybe this helps here :)

[KhudaDad414]

@davidpiendl I see the issue. Actually, it's a problem with how the optimizer handles components. it will be solved at #193

@derberg can you please move this issue to optimizer? 🙇

[github-actions]

Welcome to AsyncAPI. Thanks a lot for reporting your first issue. Please check out our contributors guide and the instructions about a basic recommended setup useful for opening a pull request.
Keep in mind there are also other channels you can use to interact with AsyncAPI community. For more details check out this issue.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity 😴

It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.

There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.

Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.

Thank you for your patience ❤️

[davidpiendl]

@KhudaDad414 Tested it with the release 0.3.0 and now I got the expected result. Thank you very much. Do you have a rough feeling when it will be included in the cli release?

[KhudaDad414]

@davidpiendl we are going to release v1 of Optimizer soon. after that version is released then it will be included in CLI.