Apply product guidelines to our event models
Symphony’s engineering team is globally distributed, and we want each member, regardless of location, to feel empowered to build the “next cool features.” Our first intent has been to apply a product strategy to our event models, consolidating them into an accessible catalog of data sets on top of which all teams can build.
Our developers already publish OpenAPI specs to document their HTTP APIs, which we use for our public facing documentation. We naturally decided to extend the practice to prescribe them to publish schemas for all their events. The most commonly used types of schemas are JSON schema, Protobuf or Avro. We decided not to enforce a particular type of schema to let teams decide on what is most achievable for their use cases. As a result, consumer services must be able to find out how incoming events are encoded. However, we do request that teams favor formats with compact binary serialization, such as Protobuf or Avro, to be cost effective. When using JSON, we only pair it with compression in scenarios where latency is not critical and volume does not force us to optimize CPU consumption. We provide schemas to support common fields expected in most schemas for standardization.
Along with the event schemas, we have devised a standard documentation to share other practical aspects around the events. For instance, this could include which services are known to produce or consume the events, if consumers cache data in some way, which topic the event is published on, what the objectives are in terms of latency, throughput, availability, and durability, what the encoding scheme is, or if events are compressed or encrypted. This documentation is used by teams trying to consume existing data or, when an incident occurs, to conduct a quick assessment of the impacted areas and guide incident recovery actions.