Remember when the Web first caught on?
One thing I remember is people saying “yeah, it is pretty cool, but, you know, it is stateless.”
As most of what I heard on this issue was from enterprise software vendors, with all the bias that could entail, I should have taken what I was told with a grain of salt. The first big problem these folks saw with the Web was its statelessness, which made it far different from the synchronously connect clients and servers (at that time, Java servers) they were used to.
The first problem the Web presented, as the enterprise crew saw it was to connect the Web to the database, which – no question – was a transactional relational database.
The first response to the problem was CGI, which quickly faded, but which, alas, still shows up in a URL address window every once and a while.
There soon followed JMS message queues, Web Services and SOA, and then REST and AJAX. Then the Web turned into the Cloud and NoSQL, Kafka, container-based microservices and Kubernetes orchestration hit the beach.
These thoughts were like an elusive butterfly in my mind as I ventured recently by Amtrak to cover the Strata Data Conference at the Jacob Javits Center on the Hudson River in New York City. Kafka certainly did seem to be a common theme in the schema presenters displayed in the technical sessions, and the Kubernetes sessions were overflow.
At the event I had a chance to speak with Jay Kreps, CEO and co-founder of Confluent, and one of the creators of the aforementioned Kafka publish-and-subscribe messaging bus. He told me that event processing is gaining a commanding presence on the scene. You could say it’s “what’s happening.”
All along, the general trend toward event-based microservices architectures has overlapped with new distributed data architectures, which are edging toward becoming mainstream competitors to traditional enterprise data warehouses.
You could think of Kubernetes as the other key part of the middleware replacement story that is going on today, he said. At the same time, he added, “Nothing in the enterprise goes away overnight.”
There is more from my conversation with Krebs in the latest episode of the Talking Data Podcast, including a conversation on the new architecture with expert analyst Mike Matchett of Small World Big Data and the ruminations of the author as the train to New York drew near Penn Station for Strata. – Jack Vaughan