Rack-Awareness on nanta - Data Engineering

Rack-Awareness on nanta - Data Engineeringhttps://nanta-data.dev/en/tags/rack-awareness/Recent content in Rack-Awareness on nanta - Data EngineeringHugo -- gohugo.ioen© 2026 nantaFri, 27 Feb 2026 00:00:00 +0000Kafka Rack Awareness and Spark: Not Supported Yethttps://nanta-data.dev/en/posts/kafka-rack-awareness-spark/Fri, 27 Feb 2026 00:00:00 +0000https://nanta-data.dev/en/posts/kafka-rack-awareness-spark/We tried to apply Kafka rack awareness to Spark jobs to reduce cross-AZ network costs. Getting the AZ information was solved easily via AWS IMDS, but Spark itself doesn’t support rack-aware Kafka partition assignment. The related Jira ticket is open but the PR was closed.