Feature: Add Support for AsyncAPI v2/v3 as Source Definition

[ivangsa]

Adding support for AsyncAPI definitions would extend the types of workflows that can be described with Arazzo, as many organizations are now combining Request/Response APIs (OpenAPI) with Event-Driven architectures (AsyncAPI).

The latest version of AsyncAPI is v3, but since v2 is still the most widely adopted version, the intention is to provide support for both versions as feasible, and prioritize v3 only when supporting both becomes impractical.

Required Changes to Arazzo Specification

• Add asyncapi as a source definition type.
• Extend the meaning of workflow.step.operationId to include pointing to an AsyncAPI operationId (either v2 or v3):
  • AsyncAPI v2 operationId
  • AsyncAPI v3 operationId
• Extend the meaning of workflow.step.parameters.in:
  • header should also include AsyncAPI message headers.
  • path should include AsyncAPI channel.parameters, as they are similar to OpenAPI path parameters (or consider adding a channel value for disambiguation).
• Runtime expressions already include support for $message.body and $message.header this would be used for context in successCriterias

Additional Considerations for Broker-Intermediated APIs

While AsyncAPI and OpenAPI are similar, and it’s tempting to use the operationId in AsyncAPI, there is a key difference: broker-mediated APIs are reciprocal (publishing/sending on one side is subscribing/receiving on the other).

• Sometimes, an API might only document one side’s perspective. For example:
  • A service that publishes events might document its "publish" operations.
  • However, it might not document the "subscribe" operation that consumers would use to receive these events (consumers just use the "publish" operation documentation and tooling).
• Teams and organizations may model this symmetry differently, and sometimes the "inverse" operation may not be defined at all.

To support this, we would need an alternative way to reference the reciprocal operation.

Additional Changes to Arazzo Specification

When the desired operationId is not documented in the existing AsyncAPI, we can support referencing the channel instead of the operation and specify the reciprocal action (send|receive).

• Add workflow.step.channel as a fixed field pointing to an AsyncAPI channel:
  • AsyncAPI v2 channels
  • AsyncAPI v3 channels
• Add workflow.step.action with supported values send|receive.

The combination of channel+action would be mutually exclusive with operationId, with operationId being the preferred option when available.

Meaning of AsyncAPI Steps

Send/Publish

A workflow step pointing to a Send/Publish operation will send a message with the workflow.step.requestBody.

workflow.step.parameters will also be used to populate AsyncAPI message.headers and channel.parameters.

The only meaningful successCriteria is "OK, message accepted," so it may be omitted.

This action is non-blocking and has no outputs; it will immediately continue to the next step, either onSuccess or onFailure.

This is an example of a workflow step sending a message:

  - stepId: "send_order_received"
    operationId: "$sourceDescriptions.OrdersAsyncAPI.onOrderEvent"
    requestBody:
      contentType: "application/json"
      payload: |-
        {
            "id": "{$steps.place_order.outputs.orderId}",
            "orderTime": "{$steps.place_order.outputs.order.orderTime}",
            "status": "RECEIVED",
            "customerDetails": {$steps.fetch_customer_details.outputs.customer},
            "restaurantDetails": {$steps.fetch_restaurant_details.outputs.restaurant},
            "orderItems": [
            {$inputs.menuItem}
            ]
        }
    onSuccess:
    - name: "goto_wait_for_downstream"
      type: "goto"
      stepId: "wait_RestaurantsStatusChannel_accepted"

Receive/Subscribe

This is a blocking action: the workflow will wait until successCriteria is met or it times out.

In the following example, in the wait_KitchenOrdersStatusChannel_accepted step, the workflow would "wait until a message is received where the message’s body contains a customerOrderId that matches the orderId of the previous step’s output."

  - stepId: "wait_KitchenOrdersStatusChannel_accepted"
    channel: "$sourceDescriptions.RestaurantsOpenAPI.KitchenOrdersStatusChannel"
    action: "receive"
    successCriteria:
    - condition: "$message.body.customerOrderId == $steps.place_order.outputs.orderId"

This step can include outputs using the contexts $message.body and $message.header

Support for Parallel Invocation: Forking and Joining

When using non-blocking steps, there may be a need for parallel invocation, involving forking and joining.

Parallel Invocation and Looping can also be applied to OpenAPI Request/Response steps.

Because of the complexity of this, we could address this in a separate issue for further discussion.

[frankkilcommins]

@ivangsa thank you for the very thoughtful proposal. Below I'll aim to process the proposal, add comments/remarks, seek clarification, or acknowledge the suggestion.

General Assumptions:

The latest version of AsyncAPI is v3, but since v2 is still the most widely adopted version, the intention is to provide support for both versions as feasible, and prioritize v3 only when supporting both becomes impractical.

Acknowledge - we should strive to support both unless we prove that impractical.

High-level changes to the Arazzo Specification:

Add asyncapi as a source definition type.

Acknowledge - this is as expected.

Extend the meaning of workflow.step.operationId to include pointing to an AsyncAPI operationId (either v2 or v3)

Acknowledge - this is as expected.

Extend the meaning of workflow.step.parameters.in:

• header should also include AsyncAPI message headers.

Acknowledge - this is as expected.

Extend the meaning of workflow.step.parameters.in:

• path should include AsyncAPI channel.parameters, as they are similar to OpenAPI path parameters (or consider adding a channel value for disambiguation).

I have a preference for a new channel value to be explicit but others are welcome to share their preferences too and we'll work towards one of these options.

Runtime expressions already include support for $message.body and $message.header this would be used for context in successCriterias

Acknowledge. These are being removed in 1.0.1 for hygiene purposes, so this would involve adding them back in and reestablishing the text and construct of the Runtime Expressions to support messages.

Dealing with reciprocal pub/sub scenarios when only one side is documented

When the desired operationId is not documented in the existing AsyncAPI, we can support referencing the channel instead of the operation and specify the reciprocal action (send|receive).
Add workflow.step.channel as a fixed field pointing to an AsyncAPI channel

To follow the pattern used for OpenAPI, this might be more applicable as mutually exclusive construct of a channelId (v3) and channelPath (v2)

Add workflow.step.action with supported values send|receive

Do we have any opinions on the choice of send|receive v publish|subscribe in this context? Are there pros/cons to the nomenclature choice?

The combination of channel+action would be mutually exclusive with operationId, with operationId being the preferred option when available.

Acknowledge - if we introduce this (or channelId / channelPath) they would be mutually exclusive with operationId and we'd clarify recommendations.

Remark: Generally in Arazzo we have a goal to be as deterministic as possible and to not make assumptions, we rely on the underlying API descriptions to express what is possible. If we are to go down a path allowing workflows to be described across omitted parts of an AsyncAPI description are we asking ourselves of trouble in the long run. Will this cause challenges for tooling w.r.t. validations/parsing OR is it an undisputed given that the channels always express both the ability to send and receive? I would like some further input from the AsyncAPI community/core team on this. Albeit laborious to enforce authors/providers to document both sides of the coin, perhaps that's the only way we can be sure what's possible. Will we also have to make assumptions on message schemas / parameters etc?

Step and Messages - Send/Publish

A workflow step pointing to a Send/Publish operation will send a message with the workflow.step.requestBody.

Acknowledge - this is as expected.

workflow.step.parameters will also be used to populate AsyncAPI message.headers and channel.parameters.

Acknowledge - this is as expected.

The only meaningful successCriteria is "OK, message accepted," so it may be omitted.

It would be good practice to specify the criteria and assuming a HTTP status code will be return, it can be set (especially if you want to handle onSuccess or onFailure flows)

This action is non-blocking and has no outputs; it will immediately continue to the next step, either onSuccess or onFailure.

I assume if no onSuccess is described, then the next sequential step is what we continue with.

Step and Messages - Receive/Subscribe

What is described makes sense but I am wondering would we benefit from an ability to set a wait/timeout?

[ivangsa]

Since most of the replies are just acknowledgments, here are the two main items:

Do we have any opinions on the choice of send|receive v publish|subscribe in this context? Are there pros/cons to the nomenclature choice?

I would prefer send|receive since it is the terminology used in the latest AsyncAPI v3. This keeps it simple and consistent. (For workflows, I would choose wait instead of receive because that better reflects what a workflow does. However, I believe it would be wiser to stick to send|receive.)

Remark: Generally in Arazzo we have a goal to be as deterministic as possible and to not make assumptions, we rely on the underlying API descriptions to express what is possible. If we are to go down a path allowing workflows to be described across omitted parts of an AsyncAPI description are we asking ourselves of trouble in the long run.

I understand your concern. This is a long-standing source of discussion and confusion for broker-based APIs, and there are many differing approaches and opinions: documenting both ends, documenting only from the provider's point of view, or only from the consumer's point of view.

And it varies wildly how different teams and orgs manage this... This is why I propose supporting both approaches.

we rely on the underlying API descriptions to express what is possible

Even if both ends are described in their respective APIs, these APIs don’t necessarily describe "what is possible"; they simply describe what an specific application is doing.

For instance, consider an "Order Updates Channel." The existence of an API documenting the "publish" operation does not necessarily mean any other application should be publishing to, or writing a workflow for, that channel.

---

• The official recomended approach from the core-team for AsyncAPI v3 is for each application to document all the operations the application perform on each channel, so following this recommendation both ends would be documented.

This means that if a new application wants to consume a message that is already documented, it should write a new AsyncAPI definition to document that operation (which is something internal, is not an actual API) and it requires crossing a $ref to the original definition, which may be authenticated.

This approach mixes "public API information" (what others can do with your API) with internal information (what your application does with other apps' APIs). This undermines the use of API definitions for self-service and discovery. And there is no standardized way to differentiate between public APIs and internal ones.

This is problematic, that's why there are different options and approaches

• Also with AsyncAPI v3 there is also the "official" possibility to define only channels and messages (operations being optional) so an organization may choose to have only channels described with AsyncAPI

---

OR is it an undisputed given that the channels always express both the ability to send and receive?

I don't think this is an "undisputed given", I believe in some cases there are teams/orgs that publish messages to a channel that can not be used to subscribe... (ex. publishing to a message broker via an http bridge, or an internal process dispatches messages from one channel to another) Althought I never seen this personaly, I think in these particular cases there would be an API definition for the "reciprocal" action otherwise it would not be discoverable/usable.. (what would be the use for a channel that can only be publishe or subscribed?)

---

My worry is that if we do only support one way we may be leaving valid options/appraches behind.