I'm not a big fan of TV murder mysteries for two reasons. The first is that you must watch for hours (sometimes weeks) before the killer's fatal mistake gets revealed. The second is that the ending is often predictable. When this happens, your time and effort get wasted. 

And so it is with enterprise architecture. Sometimes you wait months for your shiny new applications to reach production, only to discover a fatal flaw in the design, leading to a very unsatisfactory result.

One such fatal flaw is over-rotating on Kafka. So many businesses do this that Forrester recently called it out. In the paper "Event-Driven Architecture & Design: Five Big Mistakes and Five Best Practices", the number three biggest mistake was the automatic assumption that you should use Kafka for everything to do with events.

An excellent example of this is using Kafka to implement 'event sourcing.' Event sourcing is a design pattern where applications use events to store and rehydrate an object's state. It's a fabulous pattern for building high-performance data-rich event-driven applications. But just because this pattern includes the word 'event' in the title, this doesn't make it a great use case for Kafka ‒ quite the opposite. 

Kafka is designed and optimized for event streaming. It's not intended or optimized for event sourcing. If you fail to take this into account when developing your applications, you'll face some uncomfortable issues further down the line.

The first problem you'll encounter is poor developer productivity. By forcing (or allowing) software engineers to use the wrong middleware, you'll spend more time compensating for poor choices than you will writing business logic. And because 'time is money,' that's an expensive mistake to make. If you insist on going down the route, you'd better make sure that your project's budget and delivery date have lots of contingency built-in. 

The second problem you'll face will be the terrible operational experience. You'll have to build and maintain complex Kafka configurations ‒ ones where minor glitches can have a considerable impact on your data flow. You'll face some nightmarish troubleshooting scenarios where you'll need to trace problems across multiple systems, logs, topics, and databases. And, of course, you'll wrestle with complex performance & scaling challenges as the use of your application grows and the number of events increases.

Don't get me wrong. You can integrate Kafka with event-sourcing solutions just fine - it's an excellent way of streaming event notifications around your business. But for event sourcing itself? That's best left to a dedicated event-sourcing platform ‒ one that's optimized for high-performance and takes care of the needs of both developers and operators.

Consider, for example, Axon Server Enterprise ‒ a dedicated production-ready event sourcing platform. Installed and configured in minutes and with minimum hassle, connecting clients is super easy. Axon Server Enterprise automatically understands how to store and route events with no complex configuration required. And because it's a purpose-built event-sourcing platform, there will be no nasty surprises waiting for you in production as you scale up and scale out. 

For developers, the open-source Axon Framework takes away much of the toil of producing production-ready event-sourcing code. It provides all the components you need, simple annotations, and integrates seamlessly with Java & Spring Boot. 

These AxonIQ tools ‒ combined with techniques like Domain-Driven Design (DDD) and Command Query Responsibility Separation (CQRS) ‒ modernize more than just your applications. They transform your whole approach to writing software! Just look at these quotes from some of our existing customers:

"With Axon Server, we don't need to think about storing events, or optimizing performance, or routing, or queries or any of these challenges."

"For the first time, we've encountered a minimal gap between IT and the business. The code is finally expressing what is happening at the business level."

Marc van Andel - Dutch Agency for Land Registry (Kadaster)

"The AxonIQ approach brought us closer together. We were able to start deploying applications faster and more reliably. It's changed our reputation with the business."

"Axon had everything we needed. It provided an excellent way to get developers comfortable implementing a domain-driven design." 

Andy Whitaker - Consultant, US Public Sector Organizations

These customers ‒ and others like them ‒ testify that choosing a tailor-made event-sourcing platform has helped them deliver faster. And when combined with the techniques mentioned above, the transformation effect is even more significant. Adopting this approach enables people, teams, even whole businesses to deliver better results to their users, in less time, forever.

The moral of the story is this: don't force your developers to use Kafka for everything. Taking that approach might just kill your event-sourcing project. And when the project failure police start looking for the culprit; you'll be the one left holding the knife!

Related story:

Axon and Kafka - How does Axon compare to Apache Kafka? - For developers and technical architects. Covers the specific differences between Axon and Kafka when implementing event-sourcing solutions.

Subscribe to blog notifications