Data Engineering

Data Engineering/Kafka

[kafka] 카프카 토픽 생성하는 방법

docker-compose.yml version: '3' services: # wurstmeister/zookeeper:3.4.6 zookeeper: hostname: zookeeper container_name: zookeeper image: wurstmeister/zookeeper:3.4.6 ports: - "2181:2181" # wurstmeister/kafka:2.13-2.8.1 kafka: hostname: kafka container_name: kafka image: wurstmeister/kafka:2.13-2.8.1 environment: KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092 KAFKA_LISTENERS: PLAINTEXT://..

Data Engineering/Kafka

[kafka] 카프카 클러스터를 GUI(kafdrop)으로 구축하기

docker-compose.yml version: '2' services: # wurstmeister/zookeeper:3.4.6 zookeeper: hostname: zookeeper container_name: zookeeper image: wurstmeister/zookeeper:3.4.6 ports: - "2181:2181" # wurstmeister/kafka:2.13-2.8.1 - broker 1 kafka1: hostname: kafka1 container_name: kafka1 image: wurstmeister/kafka:2.13-2.8.1 environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 KAFKA_ADVER..

Data Engineering/Kafka

[kafka] 주키퍼 카프카를 도커컴포즈로 구축하는 방법

docker-compose.yml version: '3' services: # wurstmeister/zookeeper:3.4.6 zookeeper: hostname: zookeeper container_name: zookeeper image: wurstmeister/zookeeper:3.4.6 ports: - "2181:2181" # wurstmeister/kafka:2.13-2.8.1 kafka: hostname: kafka container_name: kafka image: wurstmeister/kafka:2.13-2.8.1 environment: KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092 KAFKA_LISTENERS: PLAINTEXT://..

Data Engineering/Kafka

[kafka] 주키퍼에서 카프카 토픽 확인하는 방법

1. 도커 컴포즈 파일 작성 docker-compose.yml version: '3' services: # wurstmeister/zookeeper:3.4.6 zookeeper: hostname: zookeeper container_name: zookeeper image: wurstmeister/zookeeper:3.4.6 ports: - "2181:2181" # wurstmeister/kafka:2.13-2.8.1 kafka: hostname: kafka container_name: kafka image: wurstmeister/kafka:2.13-2.8.1 environment: KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092 KAFKA_LISTENE..

Data Engineering/Kafka

[kafka] 카프카에서 토픽 삭제하는 방법

1. 도커 컴포즈 파일을 작성한다. docker-compse.yml version: '3' services: # wurstmeister/zookeeper:3.4.6 zookeeper: hostname: zookeeper container_name: zookeeper image: wurstmeister/zookeeper:3.4.6 ports: - "2181:2181" # wurstmeister/kafka:2.13-2.8.1 kafka: hostname: kafka container_name: kafka image: wurstmeister/kafka:2.13-2.8.1 environment: KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092 KAFKA_LIST..

Data Engineering/Kafka

[kafka] 카프카에서 토픽 생성하는 방법

1. 도커 컴포즈 파일 작성 docker-compose.yml 파일을 작성한다 version: '3' services: # wurstmeister/zookeeper:3.4.6 zookeeper: hostname: zookeeper container_name: zookeeper image: wurstmeister/zookeeper:3.4.6 ports: - "2181:2181" # wurstmeister/kafka:2.13-2.8.1 kafka: hostname: kafka container_name: kafka image: wurstmeister/kafka:2.13-2.8.1 environment: KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092 KAFK..

Data Engineering/Kafka

[kafka] 도커의 각 컨테이너의 접속할 때 현재의 사용자계정을 확인하는 방법 whoami

컨테이너 확인을 위해 도커컴포즈 파일을 만든다 docker-compose.yml version: '3' services: # zookeeper 3.4.6 zookeeper: hostname: zookeeper container_name: zookeeper image: wurstmeister/zookeeper:3.4.6 ports: - "2181:2181" # kafka 2.13-2.8.1 kafka: hostname: kafka container_name: kafka image: wurstmeister/kafka:2.13-2.8.1 environment: KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092 KAFKA_LISTENERS: PLAINTEXT://..

Data Engineering/Kafka

[kafka] 카프카 클러스터끼리 네트워크 연결 통신상태 확인하는 방법

도커컴포즈 파일을 작성해보자 docker-compose.yml version: '2' services: # wurstmeister/zookeeper:3.4.6 zookeeper: hostname: zookeeper container_name: zookeeper image: wurstmeister/zookeeper:3.4.6 ports: - "2181:2181" # wurstmeister/kafka:2.13-2.8.1 - broker 1 kafka1: hostname: kafka1 container_name: kafka1 image: wurstmeister/kafka:2.13-2.8.1 environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: zookeeper:..

Data Engineering/Kafka

[kafka] 도커컴포즈로 멀티노드 카프카 만드는 방법 (docker compose, kafka, zookeeper)

먼저 docker-compose.yml 파일을 작성한다. docker-compose.yml version: '2' services: # wurstmeister/zookeeper:3.4.6 zookeeper: hostname: zookeeper container_name: zookeeper image: wurstmeister/zookeeper:3.4.6 ports: - "2181:2181" # wurstmeister/kafka:2.13-2.8.1 - broker 1 kafka1: hostname: kafka1 container_name: kafka1 image: wurstmeister/kafka:2.13-2.8.1 environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CON..

Data Engineering/Kafka

[kafka] 도커컴포즈로 싱글노드 카프카 만드는 방법 (docker compose, single, kafka, zookeeper)

1. 도커 컴포즈 파일 만들기 docker-compose.yml version: '2' services: # zookeeper 3.4.6 zookeeper: hostname: zookeeper container_name: zookeeper image: wurstmeister/zookeeper:3.4.6 ports: - "2181:2181" # # kafka 2.13-2.8.1 kafka: hostname: kafka container_name: kafka image: wurstmeister/kafka:2.13-2.8.1 environment: KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092 KAFKA_LISTENERS: PLAINTEXT://0.0.0.0..

박경태
'Data Engineering' 카테고리의 글 목록 (20 Page)