Data Engineering/Kafka
docker-compose.yml version: '2.1' volumes: zookeeper-1-data: zookeeper-1-datalog: zookeeper-1-logs: zookeeper-2-data: zookeeper-2-datalog: zookeeper-2-logs: zookeeper-3-data: zookeeper-3-datalog: zookeeper-3-logs: kafka-1-data: kafka-1-secrets: kafka-2-data: kafka-2-secrets: kafka-3-data: kafka-3-secrets: services: zookeeper-1: hostname: zookeeper-1 container_name: zookeeper-1 image: zookeeper:3..
Data Engineering/Kafka
kafka-topics --create --bootstrap-server kafka-1:9092,kafka-2:9093,kafka-3:9094 --replication-factor 3 --partitions 1 --topic mytopic-3-1 kafka-topics --create --bootstrap-server kafka-1:9092,kafka-2:9093,kafka-3:9094 --replication-factor 3 --partitions 3 --topic mytopic-3-3 kafka-topics --list --bootstrap-server kafka-1:9092,kafka-2:9093,kafka-3:9094 kafka-console-producer --broker-list kafka-1..
Data Engineering/Kafka
kafka-topics --list --bootstrap-server kafka-1:9092,kafka-2:9093,kafka-3:9094 kafka-topics --create --bootstrap-server kafka-1:9092,kafka-2:9093,kafka-3:9094 --replication-factor 1 --partitions 1 --topic mytopic-1-1 kafka-topics --create --bootstrap-server kafka-1:9092,kafka-2:9093,kafka-3:9094 --replication-factor 2 --partitions 2 --topic mytopic-2-2 kafka-topics --create --bootstrap-server kaf..
Data Engineering/Kafka
카프카 브로커가 3개인데 레플리카 펙터가 4개인 토픽을 만들 때 발생하는 에러이다. [appuser@kafka-1 ~]$ kafka-topics --create --bootstrap-server kafka-1:9092,kafka-2:9093,kafka-3:9094 --replication-factor 1 --partitions 1 --topic mytopic-1-1 Created topic mytopic-1-1. [appuser@kafka-1 ~]$ kafka-topics --create --bootstrap-server kafka-1:9092,kafka-2:9093,kafka-3:9094 --replication-factor 2 --partitions 2 --topic mytopic-2-2 Create..
Data Engineering/Kafka
docker-compose.yml version: '2.1' services: zookeeper-1: hostname: zookeeper-1 container_name: zookeeper-1 image: zookeeper:3.6 restart: always ports: - 2181:2181 environment: ZOO_MY_ID: 1 ZOO_SERVERS: server.1=zookeeper-1:2888:3888;2181 server.2=zookeeper-2:2888:3888;2181 server.3=zookeeper-3:2888:3888;2181 volumes: - type: bind source: ./data/zk-cluster/zookeeper-1/data target: /data read_only..
Data Engineering/Kafka
docker-compose.yml version: '2.1' services: zookeeper-1: hostname: zookeeper-1 container_name: zookeeper-1 image: zookeeper:3.6 restart: always ports: - 2181:2181 environment: ZOO_MY_ID: 1 ZOO_SERVERS: server.1=zookeeper-1:2888:3888;2181 server.2=zookeeper-2:2888:3888;2181 server.3=zookeeper-3:2888:3888;2181 volumes: - type: bind source: ./data/zk-cluster/zookeeper-1/data target: /data read_only..
Data Engineering/Kafka
docker exec zookeeper-1 zkCli.sh ls /zookeeper docker exec zookeeper-2 zkCli.sh ls /zookeeper docker exec zookeeper-3 zkCli.sh ls /zookeeper
Data Engineering/Kafka
docker-compose.yml version: '2.1' services: zookeeper-1: hostname: zookeeper-1 container_name: zookeeper-1 image: zookeeper:3.6 restart: always ports: - 2181:2181 environment: ZOO_MY_ID: 1 ZOO_SERVERS: server.1=zookeeper-1:2888:3888;2181 server.2=zookeeper-2:2888:3888;2181 server.3=zookeeper-3:2888:3888;2181 volumes: - type: bind source: ./data/zk-cluster/zookeeper-1/data target: /data read_only..
Data Engineering/Kafka
docker-compose.yml version: '2.1' services: zookeeper-1: hostname: zookeeper-1 container_name: zookeeper-1 image: zookeeper:3.6 restart: always ports: - 2181:2181 environment: ZOO_MY_ID: 1 ZOO_SERVERS: server.1=zookeeper-1:2888:3888;2181 server.2=zookeeper-2:2888:3888;2181 server.3=zookeeper-3:2888:3888;2181 volumes: - type: bind source: ./data/zk-cluster/zookeeper-1/data target: /data read_only..
Data Engineering/Kafka
docker-compose.yml version: '2.1' services: zookeeper-1: hostname: zookeeper-1 container_name: zookeeper-1 image: zookeeper:3.6 restart: always ports: - 2181:2181 environment: ZOO_MY_ID: 1 ZOO_SERVERS: server.1=zookeeper-1:2888:3888;2181 server.2=zookeeper-2:2888:3888;2181 server.3=zookeeper-3:2888:3888;2181 volumes: - type: bind source: ./data/zk-cluster/zookeeper-1/data target: /data read_only..