[QUESTION] bootstrap_servers="kafka:9092" doesn't work in my docker-compose, why? #1078

Buronn · 2022-05-21T22:00:37Z

Buronn
May 21, 2022

I can connect with kafka-python to the kafka service, but aiokafka doesn't give me that solution.

My docker-compose.yml

version: '3.5'
services:
  data_base:
    image: postgres:10.5
    container_name: postgres
    ports:
      - '5432:5432'
    environment:
      - POSTGRES_DB=postgres
      - POSTGRES_USER=myuser
      - POSTGRES_PASSWORD=mypassword
  api:
    container_name: api-login
    restart: always
    build:
      context: ./api/
      dockerfile: Dockerfile
    environment:
      POSTGRES_USER: myuser
      POSTGRES_PASSWORD: mypassword
      POSTGRES_DB: postgres
      POSTGRES_HOST: postgres
      POSTGRES_PORT: 5432
      FLASK_ENV: development
      KAFKA_ADVERTISED_HOST_NAME: kafka
    ports:
      - "3000:80"
    volumes:
      - ./api:/usr/src/app
    depends_on:
      - "data_base"
      - "kafka"
  zookeeper:
    image: 'bitnami/zookeeper:latest'
    ports:
      - '2181:2181'
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
  kafka:
    image: 'bitnami/kafka:latest'
    ports:
      - '9092:9092'
    environment:
      - KAFKA_BROKER_ID=1
      - KAFKA_CFG_LISTENERS=PLAINTEXT://:9092
      - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9092
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
      - ALLOW_ANONYMOUS_LOGIN=yes

The connection is made by this:

from aiokafka import AIOKafkaProducer
import asyncio

async def send_one(msg):
    producer = AIOKafkaProducer(
        bootstrap_servers="kafka:9092",
        loop=asyncio.get_event_loop()
    )
    await producer.start()
    try:
        await producer.send_and_wait("login", msg.encode('utf-8'))
    finally:
        await producer.stop()

And the response by calling this send_one function is:

api            | Traceback (most recent call last):
api            |   File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2088, in __call__
api            |     return self.wsgi_app(environ, start_response)
api            |   File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2073, in wsgi_app
api            |     response = self.handle_exception(e)
api            |   File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2070, in wsgi_app
api            |     response = self.full_dispatch_request()
api            |   File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1515, in full_dispatch_request
api            |     rv = self.handle_user_exception(e)
api            |   File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1513, in full_dispatch_request
api            |     rv = self.dispatch_request()
api            |   File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1499, in dispatch_request
api            |     return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
api            |   File "/usr/src/app/app.py", line 51, in login
api            |     asyncio.run(send_one(data['user']))
api            |   File "/usr/local/lib/python3.9/asyncio/runners.py", line 44, in run
api            |     return loop.run_until_complete(main)
api            |   File "/usr/local/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
api            |     return future.result()
api            |   File "/usr/src/app/app.py", line 17, in send_one
api            |     await producer.start()
api            |   File "/usr/local/lib/python3.9/site-packages/aiokafka/producer/producer.py", line 296, in start
api            |     await self.client.bootstrap()
api            |   File "/usr/local/lib/python3.9/site-packages/aiokafka/client.py", line 249, in bootstrap
api            |     raise KafkaConnectionError(
api            | kafka.errors.KafkaConnectionError: KafkaConnectionError: Unable to bootstrap from [('kafka', 9092, <AddressFamily.AF_UNSPEC: 0>)]

And sometimes, I get this error too
kafka.errors.KafkaConnectionError: KafkaConnectionError: No connection to node with id 1

If you could help me to solve the error, I would really appreciate it.
Thanks!

FadingFog · 2023-03-24T17:06:30Z

FadingFog
Mar 24, 2023

Try this and connect to kafka:9093

  kafka:
    image: bitnami/kafka
    environment:
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CLIENT:PLAINTEXT,EXTERNAL:PLAINTEXT
      - KAFKA_CFG_LISTENERS=CLIENT://:9092,EXTERNAL://:9093
      - KAFKA_CFG_ADVERTISED_LISTENERS=CLIENT://kafka:9092,EXTERNAL://kafka:9093
      - KAFKA_INTER_BROKER_LISTENER_NAME=CLIENT
    expose:
      - "${KAFKA_PORT:-9092}"
      - "${KAFKA_PORT:-9093}"
    depends_on:
      - zookeeper

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[QUESTION] bootstrap_servers="kafka:9092" doesn't work in my docker-compose, why? #1078

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 1 comment

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

[QUESTION] bootstrap_servers="kafka:9092" doesn't work in my docker-compose, why? #1078

Uh oh!

Uh oh!

Buronn May 21, 2022

Replies: 1 comment

Uh oh!

FadingFog Mar 24, 2023

Buronn
May 21, 2022

FadingFog
Mar 24, 2023