Adopting event sourcing

Customers have a rich history of practices and knowledge that serves as the foundation for their projects and influences most of their decisions. Oftentimes, suggesting solutions veering away from such lore is met with understandable suspicion. In such instances it is crucial to clearly convey why it is worth preferring different ways to solve problems over well-established ones. This article tells a story of such a case, in which the solution of a problem required challenging the status quo of well-established practices. Care was needed to explain the limits of existing implementations, and to ensure that all use cases dear to the customer would have been possible with a different approach.

The challenge

Let's briefly define the challenge and then search for a possible solution. A customer wanted to develop a software, called DAN, with the primary task of calculating the combination of data delivered by multiple data sources. Each data source provides incremental information about entities over time, and DAN is tasked with assembling these fragments to form a complete representation of each entity.

All information processed by DAN is delivered through a proprietary message bus, exposing DAN to typical challenges in message-based architectures such as out-of-order deliveries, missing deliveries, repeated deliveries, etc.

Initial solution and its soundness

Initially, due to the background and the expertise of the customer, the first implementation of DAN relied on a relational model using PostgreSQL as the database. This relational model is normalized using tables augmented with hand-crafted bi-temporal functionality. The database functions like an append-only log: after a message is received, the latest partial result is duplicated, and the duplicate is combined with information from the message received. All rows of the previous partial result are left untouched and just marked as obsolete, ensuring that only one valid partial result exists at a given time. Keeping all these partial results is crucial for DAN to provide auditing functionality in case of misuse.

On the surface, this approach is legitimate and provides all the core functionality that the customer demands. Why challenging this solution then?

What if a missed delivery is eventually sent later?

Getting a missed delivery at a later time is, in fact, an out-of-order delivery. A relational model makes these deliveries challenging: we have retained partial results that may have been constructed with an incorrect sequence of messages.

Preventing this problem is impossible: it requires advanced knowledge of the complete sequence of messages, which, by definition, is infinite (or as long as the lifespan of a citizen).

Recovery from this scenario is non-trivial. We have to rebuild all incorrect partial results previously computed by consuming messages which are more current than the incoming one. To do that, we discard all incorrect partial results, then we evaluate again, in logical order, all messages that originated from the partial results discarded. This solution implies storing all messages delivered to DAN, as it would be impossible to recompute any partial result without the source message. Indeed, this sounds familiar.

How is this related to event sourcing?

The idea of event sourcing is to capture changes to the application state as a sequence of events, then replay the whole sequence on-demand to rebuild the state. Event sourcing has an interesting feature: eventual consistency. This system feature ensures that, if no new changes are made to a piece of data, reading that piece of data will, over time, return the last updated value.

Eventual consistency trivializes the support of out-of-order deliveries. The results are dynamically computed on-the-fly. Therefore, as new messages arrive, any subsequent result computation will implicitly incorporate, in logical order, all new and old messages.

Proposing event sourcing

Understandably, proposing event sourcing to the customer over a relational model was met with reasonable scepticism. After all, we are challenging a well-understood and ubiquitous solution with an entirely different approach - one that does not even store the actual application state, complicating building reports and precluding freely querying the application data. Such features are no hard requirements, but de facto standard functionality valued by the customer.

In these situations, it is crucial to clearly communicate all technical difficulties stemming from a solution. The approach chosen for this was to describe in prose an instance of the problem and then prepare a simulation by mimicking database tables using Microsoft Excel. Once the problem is understood, another simulation is used to demonstrate how the problem is automatically solved using event sourcing.

Discussing and comparing event sourcing

Another delicate step was required to ensure the customer would believe in event sourcing by honestly comparing it to the previous solution. It is important to remember that one can easily appear as a «snake oil salesmen» trying to squeeze a few extra hours more into the customer’s invoice.

For this project, a document describing event sourcing and relational models served this purpose. It compared them and highlighting any missing features that could potentially trump adoption. It is important to never judge any solution, we have to neutrally provide data and information to fuel discussion and feedback. All the time spent answering questions and clearing doubts is an investment in trust and supports the solution.

Accepting event sourcing

To sum up, we must accept the fact that sometimes a customer’s policies might forbid a certain solution. This may sound like an arbitrary imposition (after all, it is), but we must respect the technological and organizational culture behind it.

Luckily, this was not the case. Event sourcing could offer our customer a simpler and more maintainable mechanism to keep track of all information in DAN. Indeed, extra development resources was involved to build a traditionally queryable reporting database. As this is a non-core requirement, it was deemed an acceptable trade-off.

To conclude, consulting the customer by objectively presenting the facts ( including non-beneficial ones) and transparently highlighting the pros and cons, has proven to be a winning strategy to build trust and promote innovation. Such a collaborative environment is ideal for creating and proposing solutions that simplify development, reduce costs and boost the developer’s confidence.