# Dekaf

> High-performance, pure C# Apache Kafka client library for .NET 10+. A native, zero-allocation implementation with no interop overhead or JVM dependency.


## Dekaf

- [Introduction](https://thomhurst.github.io/Dekaf/docs.md): Dekaf is a high-performance, pure C# Apache Kafka client for .NET, implemented from the wire protocol up with no librdkafka or JVM dependency.
- [Replica log directories](https://thomhurst.github.io/Dekaf/docs/admin/replica-log-directories.md): Query current and future Kafka log directories for selected replicas.
- [Streams group management](https://thomhurst.github.io/Dekaf/docs/admin/streams-group-management.md): List, alter, and delete Kafka Streams group offsets and groups.
- [Topic identifiers](https://thomhurst.github.io/Dekaf/docs/admin/topic-identifiers.md): Describe and delete Kafka topics by stable topic ID.
- [Transaction remediation](https://thomhurst.github.io/Dekaf/docs/admin/transaction-remediation.md): Inspect, fence, terminate, or abort Kafka transactions safely.
- [Public API Compatibility](https://thomhurst.github.io/Dekaf/docs/api-compatibility.md): How Dekaf locks its public API surface with analyzer baselines, how to validate a change locally, and how to update a baseline.
- [Benchmark Results](https://thomhurst.github.io/Dekaf/docs/benchmarks.md): Dekaf versus Confluent.Kafka on throughput, latency, and allocations, measured with BenchmarkDotNet in automated runs from main.
- [Compatibility](https://thomhurst.github.io/Dekaf/docs/compatibility.md): Which .NET targets each Dekaf package supports, the netstandard2.0 restore baseline, and the blocker categories tracked against it.
- [Compression](https://thomhurst.github.io/Dekaf/docs/compression.md): The available compression codecs, how to install and enable batch compression, and how to pick one for your throughput and CPU budget.
- [Client DNS Lookup](https://thomhurst.github.io/Dekaf/docs/configuration/client-dns-lookup.md): How Dekaf resolves broker hostnames before connecting, the available DNS lookup modes, and the bootstrap resolution deadline.
- [Migrate Confluent configuration](https://thomhurst.github.io/Dekaf/docs/configuration/confluent-migration.md): Translate existing Confluent-style configuration to Dekaf, including Aspire setup, a settings compatibility matrix, and CoreWCF Kafka migration.
- [Consumer Options](https://thomhurst.github.io/Dekaf/docs/configuration/consumer-options.md): Complete reference for every consumer setting, covering connection, group coordination, offsets, fetch and session tuning, subscription, and rebalancing.
- [Configuration Presets](https://thomhurst.github.io/Dekaf/docs/configuration/presets.md): Ready-made producer and consumer configurations for common scenarios, how to override individual values, and how to define your own preset.
- [Producer Options](https://thomhurst.github.io/Dekaf/docs/configuration/producer-options.md): Complete reference for every producer setting, covering connection, delivery guarantees, batching, compression, partitioning, transactions, and security.
- [Consumer Basics](https://thomhurst.github.io/Dekaf/docs/consumer/basics.md): Create a consumer, subscribe to topics, and process records with await foreach, including auto offset reset, error handling, and graceful shutdown.
- [Consumer Groups](https://thomhurst.github.io/Dekaf/docs/consumer/consumer-groups.md): Share partitions across consumer instances, covering rebalancing, static membership, session and heartbeat tuning, and how to scale group members.
- [Dead Letter Queues and Retry Topics](https://thomhurst.github.io/Dekaf/docs/consumer/dead-letter-queues.md): Park repeatedly failing records in a companion topic instead of stalling the consumer, with tiered retry topics, failure counting, and custom routing.
- [Delivery Semantics](https://thomhurst.github.io/Dekaf/docs/consumer/delivery-semantics.md): The definitive reference for what happens to a message when processing fails, how Dekaf's default compares to other clients, and how to choose a guarantee.
- [Pre-deserialization filtering and routing](https://thomhurst.github.io/Dekaf/docs/consumer/filtering-and-routing.md): Reject records or choose a deserializer from raw keys, values, and headers before typed deserialization.
- [LINQ Extensions](https://thomhurst.github.io/Dekaf/docs/consumer/linq-extensions.md): LINQ-style Where, Select, Take, Batch, and ForEachAsync operators over consumed messages, and why filtering skips records permanently.
- [Manual Partition Assignment](https://thomhurst.github.io/Dekaf/docs/consumer/manual-assignment.md): Assign specific partitions directly instead of joining a consumer group, with manual offset management, seeking, and incremental assignment.
- [Offset Management](https://thomhurst.github.io/Dekaf/docs/consumer/offset-management.md): Offset commit modes, commit strategies, seeking, and inspecting committed offsets, so you can choose the right delivery guarantee.
- [Partitioned Async Processing](https://thomhurst.github.io/Dekaf/docs/consumer/partitioned-processing-api.md): RunPartitionedAsync keeps per-key work ordered while processing partitions in parallel, covering commit semantics, backpressure, and error policy.
- [Share Consumers (KIP-932)](https://thomhurst.github.io/Dekaf/docs/consumer/share-consumers.md): Queue-style consumption with share groups (KIP-932): record-level acknowledgement, Accept/Release/Reject, implicit vs explicit modes, and acquisition lock renewal.
- [Dependency Injection](https://thomhurst.github.io/Dekaf/docs/dependency-injection.md): Register Dekaf producers and consumers in the ASP.NET Core container, including multiple clients, global interceptors, appsettings binding, and NativeAOT.
- [Getting Started](https://thomhurst.github.io/Dekaf/docs/getting-started.md): Install Dekaf, run Kafka locally, and write your first producer and consumer.
- [Hosted Consumer Services](https://thomhurst.github.io/Dekaf/docs/hosted-services.md): KafkaConsumerService runs a consumer for your application's lifetime, handling scoped dependencies, failure handling, and shutdown behaviour.
- [Migrate from Confluent.Kafka](https://thomhurst.github.io/Dekaf/docs/migrating-from-confluent-kafka.md): Move a .NET application from Confluent.Kafka to Dekaf, including producers, consumers, delivery semantics, serializers, transactions, and configuration.
- [Observability (OpenTelemetry)](https://thomhurst.github.io/Dekaf/docs/observability.md): Dekaf's built-in ActivitySource and Meter for OpenTelemetry tracing and metrics, plus broker-side telemetry (KIP-714). Zero cost when nothing is listening.
- [Performance](https://thomhurst.github.io/Dekaf/docs/performance.md): What makes Dekaf fast, how to tune batching and compression for your workload, what throughput to expect, and the common mistakes that cost you.
- [Producer Basics](https://thomhurst.github.io/Dekaf/docs/producer/basics.md): Create a producer and send messages, covering acks and delivery guarantees, batching, compression, the idempotent producer, and ordering.
- [Batch Production](https://thomhurst.github.io/Dekaf/docs/producer/batch-production.md): ProduceAllAsync sends many messages concurrently without the pitfalls of awaiting ValueTask in a loop, including partial success handling.
- [Fire-and-Forget Production](https://thomhurst.github.io/Dekaf/docs/producer/fire-and-forget.md): FireAsync enqueues records without waiting for broker acknowledgment, how it differs from ProduceAsync, and how to still guarantee delivery.
- [Message Headers](https://thomhurst.github.io/Dekaf/docs/producer/headers.md): Attach metadata such as trace IDs and message types to records, read headers on the consumer side, and understand their performance cost.
- [Transactional Outbox](https://thomhurst.github.io/Dekaf/docs/producer/outbox.md): At-least-once publishing from your database to Kafka without distributed transactions, covering schema, ordering, deduplication, tuning, and custom stores.
- [Partitioning](https://thomhurst.github.io/Dekaf/docs/producer/partitioning.md): How keys map to partitions, the available partitioner types, explicit partition assignment, and the ordering guarantees each choice gives you.
- [Topic-Specific Producers](https://thomhurst.github.io/Dekaf/docs/producer/topic-producer.md): ITopicProducer binds to a single topic at construction so you skip the topic argument on every call, with disposal semantics and DI wiring.
- [Transactions](https://thomhurst.github.io/Dekaf/docs/producer/transactions.md): Atomic writes across partitions and topics for exactly-once semantics, covering consume-transform-produce, isolation levels, timeouts, and error handling.
- [OAuth / OAUTHBEARER](https://thomhurst.github.io/Dekaf/docs/security/oauth.md): OAUTHBEARER authentication with custom token providers, JWT-bearer and client assertion grants, Azure AD, AWS MSK IAM, and token refresh.
- [SASL Authentication](https://thomhurst.github.io/Dekaf/docs/security/sasl.md): SASL/PLAIN, SCRAM, GSSAPI (Kerberos), and AWS_MSK_IAM authentication for Dekaf, including Confluent Cloud setup and securing credentials.
- [TLS Encryption](https://thomhurst.github.io/Dekaf/docs/security/tls.md): Encrypt broker connections with TLS, supply a custom CA certificate, configure mutual TLS, and troubleshoot common handshake failures.
- [Built-in Serializers](https://thomhurst.github.io/Dekaf/docs/serialization/built-in.md): The serializers Dekaf selects automatically for strings, byte arrays, ReadOnlyMemory, integers, and Guid, and how to specify them explicitly.
- [Custom Serializers](https://thomhurst.github.io/Dekaf/docs/serialization/custom.md): Implement ISerializer and IDeserializer, write a zero-allocation serializer, use SerializationContext, and handle nulls and async serdes.
- [JSON Serialization](https://thomhurst.github.io/Dekaf/docs/serialization/json.md): System.Text.Json serialization for Dekaf, with custom JsonSerializerOptions, polymorphic types, NativeAOT support, and error handling.
- [Routing Serdes](https://thomhurst.github.io/Dekaf/docs/serialization/routing.md): Route heterogeneous records by topic or by schema ID without allocating in steady state, on both the produce and consume sides.
- [Schema Registry](https://thomhurst.github.io/Dekaf/docs/serialization/schema-registry.md): Confluent Schema Registry with Avro and Protobuf, JSON Schema validation, and client-side field-level encryption via AWS, Azure, GCP, Alibaba Cloud, or Vault KMS.
- [24-hour soak tests](https://thomhurst.github.io/Dekaf/docs/soak-tests.md): The 24-hour mixed producer and consumer soak run, the failure gates it enforces, and how to run it locally.
- [Stress Test Results](https://thomhurst.github.io/Dekaf/docs/stress-tests.md): Long-running stress tests comparing sustained performance between Dekaf and Confluent.Kafka under real-world load.
- [Testing](https://thomhurst.github.io/Dekaf/docs/testing.md): Dekaf.Testing provides in-memory producers and consumers for fast unit tests, plus DI wiring, admin and offset fakes, and NativeAOT smoke validation.
- [Deterministic fault injection](https://thomhurst.github.io/Dekaf/docs/testing/fault-injection.md): Script deterministic producer, consumer, group, transaction, Admin, and Share Consumer failures with Dekaf.Testing.