Acessing Event Hubs with Confluent Kafka Library

A while ago, I was involved in a project that needed to push messages to a Kafka topic. I found that while the .NET code required for that implementation was relatively straight-forward – thanks to the Confluent’s .NET client for Kafka –  configuring the server was a nightmare. The IT team at the client site was supposed to get the kafka cluster sorted and dragged the issue for a month or so. And I understood why when I tried to setup a cluster myself – configuring a kafka cluster is not a walk in the park. If only we had a managed, one click solution to implement a event streaming solution, based on kafka protocol… 😀

When Microsoft announced last month Event Hubs support for the Kafka protocol – I thought that a great way to prove that this was really interoperable, was to use part of the original code I wrote and see if I could connect to Event Hubs without any significant changes. And I was pleasantly surprised! The only changes required was some additions to the producer/consumer configuration. This post shows how I managed to get this working, and show one of the main gotchas I found along the way. Continue reading “Acessing Event Hubs with Confluent Kafka Library”