AsyncAPI is gaining traction in the ecosystem of API tools. It solves an important problem: it provides a convenient way of describing the interface of event-driven systems independently of the underlying technology. With AsyncAPI, evented systems can be treated as any other API product: a productizable and reusable, self-describing building block encapsulating some set of data and capabilities. AsyncAPI also provides an essential component required to develop tooling taking advantage of a way of describing a standard interface for the asynchronous API in order to support the full development cycle of the evented system.
The downside of the rise of AsyncAPI is that it introduces yet another specification and industry standard in an already fragmented API landscape: OAS, GraphQL, gRPC, OData, RAML and now AsyncAPI all provide solutions to understand through metadata the interface of modern APIs and they do it with different degrees of overlap. This can sometimes lead to confusion about the relationship between all these technologies and when to use each of them.
At MuleSoft, we have long embraced a multi-spec, metadata-driven approach to API interfaces. We developed our open-source parsing and metadata framework, AMF, to support the API development lifecycle of our customers and their API supply chain, no matter what specific technologies they want to use.
Our approach is based on defining a formal model that captures the semantics of the different specification languages. We achieve that by mapping elements of the specification syntax to a well-defined taxonomy of concepts in a modeling vocabulary. Then, when we parse API definitions in the specification languages, we generate a graph of metadata where each node and edge is mapped to one of the concepts in the modeling vocabulary. We capture in a standard way the data model and semantics of the information provided by the API definition document.
If two specs overlap in their meaning, we map them to the same common concepts. This allows us to identify the overlap between the different specification languages.
In our framework, we use well-tested W3C standards (e.g. JSON-LD, SHACL) that give us a solid foundation to build sophisticated tooling on top of the graph of metadata. The use of standards also gives us access to an array of battle-proven industry tools to work with the API’s metadata, like Amazon Neptune for metadata storage.
So when we need to support a new type of spec, the first question we ask ourselves is, what are the semantics of this API specification language? In other words, what type of APIs is it trying to describe? Answering this question is not always easy. Industry-driven specification languages usually lack well-defined semantics and can be ambiguous. They usually provide some grammar and schema information about the syntax of the specification but just some informal textual description of the meaning of those elements. The task of understanding and formally capturing the underlying meaning of the spec is sometimes a confusing one.
OpenAPI (OAS, before Swagger) is an interesting example. It has been described as a ‘web API language’ and many people think of OAS as a standard way of specifying ‘RESTful’ APIs.
When we added support for OAS, we modeled it as a ‘web RPC’ language. What we mean by this is that OAS is not describing canonical REST APIs (resources, hypermedia, HATEOAS), although you can define conventions to store a REST model in OAS. Rather, it is describing a classic RPC mechanism that uses HTTP as the transport layer.
At the core of our RPC model for OAS, there is a list of endpoints bound to HTTP URLs exposing operations bound to HTTP methods, expecting and returning HTTP encoded parameters and payloads.
Additionally, JSON Schema is the usual way of describing the data structures being exchanged in the messages. We modeled these schemas using SHACL, the standard W3C language for data shapes, with some extensions
This web RPC model has allowed us to model other ‘RESTful’ specification languages like RAML using the same common underlying vocabulary.
When we started looking at AsyncAPI, the same question came to mind: what are the actual semantics of AsyncAPI? What type of systems is AsyncAPI trying to describe?
AsyncAPI introduces a key concept in the way it makes it possible to define evented systems. It defines a conceptual boundary around the system, delimiting and encapsulating the internal implementation and the external set of clients consuming and producing information.
These clients interact through a well-defined interface accessible through multiple channels, exposing two asynchronous operations: publish and subscribe. The semantics of these operations are not clearly defined but they can be understood as asynchronous, non-blocking write and read operations.
This boundary makes it possible to look at the evented system as similar to traditional client-server architectures of synchronous RESTful APIs as defined by OAS.
This simple model is also incredibly useful in order to treat the evented system as an atomic component that can be reused and provides a clear contract with the exterior, unlocking a lot of productization and lifecycle use cases around the evented system.
The second interesting aspect of AsyncAPI when compared to traditional synchronous HTTP based RPC APIs is that AsyncAPI needs to support multiple transports. There are many technologies that can satisfy the simple interface introduced by AsyncAPI. Many of these transports introduce complex application level protocols, but, in the same way OAS uses HTTP as a transport without reusing the application level semantics of the protocol, AsyncAPI can abstract these details and still provide value through the abstract asynchronous operations described in the specification.
However, in order to actually invoke the operations from the external consumer point of view, the abstract operations must be bound to some actual transport component. AsyncAPI bindings capture how to bind that abstract element to the concrete protocol information required to publish and subscribe to the events.
From this point of view, we model AsyncAPI as an asynchronous RPC API mechanism with multi-transport bindings. This allows us to encode AsyncAPI specifications in the same graph of metadata with the common semantics we are generating for RAML/OAS, with some minor adjustments:
- AsyncAPI channels are mapped to RPC endpoints in our common model
- Asynchronous operations have been added to the synchronous operations already supported in each endpoint
- Events are described using the same schema language as synchronous payloads but attached to asynchronous operations
- Transport bindings were added to key elements in the RPC model that becomes an abstract RPC model with concrete bindings
- HTTP details in the synchronous case have been pushed to the common bindings mechanism
Reusing the same common semantics allowed us to explain a simple way what role AsyncAPI plays in your evented API lifecycle and, most importantly, to reuse the same tooling and technologies we are already using working with RESTful APIs specified in RAML/OAS for AsyncAPI (e.g. linting). This opens interesting interoperability between synchronous and asynchronous APIs.
One example of this interoperability is generating wrappers for GraphQL that support synchronous and asynchronous RPC operations, through GraphQL subscriptions, from existing RAML/OAS and Async APIs.
Finally, it is important to note that the simplicity and straight-forward nature of AsyncAPI (as in the case of OAS) also means that many relevant details of the API interface cannot be expressed in these specification languages.
In the same way that is impossible to describe pagination in OAS in a standard way, AsyncAPI does not provide a mechanism to describe whether a particular payload is a command or a regular event, or if an asynchronous operation is part of a pub-sub pattern or it fan-outs as part of a task distribution push-pull pattern.
As both specifications mature, support for these additional patterns could be added on top of the core API model. In our case, one of the benefits of using a common formal model independent of the specification syntax is that we can enrich the model with this kind of extended semantics and make them available for our tooling. Additional pagination semantics can then be expressed in the target specification using and use the standard extension mechanism provided by the specification language (e.g. extensions in OAS and AsyncAPI).
Summing up, AsyncAPI is a great mechanism to productize evented systems in the same way OAS has made it possible to treat API as products in the case of ‘RESTful’ APIs. It has the potential to unlock a new generation of tooling that moves the mindset of the API community in a direction where the distinctions between synchronous APIs and evented systems start to disappear. Our way of treating both types of APIs using a common model that can be consumed by a common set of tools and can be extended to fill gaps in the current specifications is further confirmation of how both types of APIs can be unified in a more general way of designing, cataloguing, discovering, and managing APIs.
AsyncAPI and OpenAPI: an API Modeling Approach was originally published in Salesforce Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.