How do I use allow.auto.create.topics config in consumer?

[ajinkyawebshar]

I have this sample consumer where I have passed this additional config to create topic if doesn't exist.

class TestConsumer < Racecar::Consumer
  subscribes_to "HFDM.Scout", additional_config: { "allow.auto.create.topics" => true }

  def initialize
  end

  def process(message)
    puts "Received message: #{message.value}"
  end
end

I have even set the same config in config/racecar.yml

# These config values will be shared by all environments but can be overridden.
common: &common
  client_id: "kalfka_test_app"

development:
  <<: *common
  brokers:
    - b-1.kafkaqa.abcdef.z10.kafka.us-west-2.amazonaws.com:9092
    - b-2.kafkaqa.abcdef.z10.kafka.us-west-2.amazonaws.com:9092
  security_protocol: plaintext
  consumer:
    - allow.auto.create.topics=true
  producer:
    - allow.auto.create.topics=true
  group_id: Scout.Group

We have also enabled allow.auto.create.topics in Amazon MSK.

When I run the consumer it fails with following error..

 bundle exec racecar TestConsumer
=> Starting Racecar consumer TestConsumer...
=> Detected Rails, booting application...
=> Wrooooom!
=> Ctrl-C to shutdown consumer
E, [2024-05-16T09:13:07.238232 #747] ERROR -- : (try 1/10): Error for topic subscription #<struct Racecar::Consumer::Subscription topic="HFDM.Scout", start_from_beginning=true, max_bytes_per_partition=1048576, additional_config={"allow.auto.create.topics"=>true}>: Broker: Unknown topic or partition (unknown_topic_or_part)