![]()
Data Engineering/Kafka
토픽 만들기 kafka-topics --create --bootstrap-server kafka-1:9092,kafka-2:9093,kafka-3:9094 --replication-factor 3 --partitions 3 --topic performance-test --record-size 2000 으로 프로듀서 테스트 해보기 kafka-producer-perf-test --topic performance-test --throughput -1 --num-records 1 --record-size 2000 --producer-props ack=1 bootstrap.servers=kafka-1:9092,kafka-2:9093,kafka-3:9094 토픽 읽기 kafka-console-consumer --b..
![]()
Data Engineering/Kafka
토픽 생성 kafka-topics --create --bootstrap-server kafka-1:9092,kafka-2:9093,kafka-3:9094 --replication-factor 3 --partitions 3 --topic performance-test 프로듀서 테스트 kafka-producer-perf-test --topic performance-test --throughput -1 --num-records 100000 --record-size 2000 --producer-props ack=1 bootstrap.servers=kafka-1:9092,kafka-2:9093,kafka-3:9094 컨슈머 테스트 kafka-consumer-perf-test --topic performance-t..
![]()
Data Engineering/Kafka
토픽생성 kafka-topics --create --bootstrap-server kafka-1:9092,kafka-2:9093,kafka-3:9094 --replication-factor 3 --partitions 3 --topic performance-test 프로듀서 성능 테스트 kafka-producer-perf-test --topic performance-test --throughput -1 --num-records 100000 --record-size 2000 --producer-props ack=1 bootstrap.servers=kafka-1:9092,kafka-2:9093,kafka-3:9094
![]()
Data Engineering/Kafka
카프카 실행 전 docker-compose.yml version: '2.1' volumes: zookeeper-1-data: zookeeper-1-datalog: zookeeper-1-logs: zookeeper-2-data: zookeeper-2-datalog: zookeeper-2-logs: zookeeper-3-data: zookeeper-3-datalog: zookeeper-3-logs: kafka-1-data: kafka-1-secrets: kafka-2-data: kafka-2-secrets: kafka-3-data: kafka-3-secrets: services: zookeeper-1: hostname: zookeeper-1 container_name: zookeeper-1 image: zo..
![]()
Data Engineering/Kafka
docker-compose.yml version: '2.1' volumes: zookeeper-1-data: zookeeper-1-datalog: zookeeper-1-logs: zookeeper-2-data: zookeeper-2-datalog: zookeeper-2-logs: zookeeper-3-data: zookeeper-3-datalog: zookeeper-3-logs: kafka-1-data: kafka-1-secrets: kafka-2-data: kafka-2-secrets: kafka-3-data: kafka-3-secrets: services: zookeeper-1: hostname: zookeeper-1 container_name: zookeeper-1 image: zookeeper:3..
![]()
Data Engineering/Kafka
kafka-topics --create --bootstrap-server kafka-1:9092,kafka-2:9093,kafka-3:9094 --replication-factor 3 --partitions 1 --topic mytopic-3-1 kafka-topics --create --bootstrap-server kafka-1:9092,kafka-2:9093,kafka-3:9094 --replication-factor 3 --partitions 3 --topic mytopic-3-3 kafka-topics --list --bootstrap-server kafka-1:9092,kafka-2:9093,kafka-3:9094 kafka-console-producer --broker-list kafka-1..
![]()
Data Engineering/Kafka
kafka-topics --list --bootstrap-server kafka-1:9092,kafka-2:9093,kafka-3:9094 kafka-topics --create --bootstrap-server kafka-1:9092,kafka-2:9093,kafka-3:9094 --replication-factor 1 --partitions 1 --topic mytopic-1-1 kafka-topics --create --bootstrap-server kafka-1:9092,kafka-2:9093,kafka-3:9094 --replication-factor 2 --partitions 2 --topic mytopic-2-2 kafka-topics --create --bootstrap-server kaf..
![]()
Data Engineering/Kafka
카프카 브로커가 3개인데 레플리카 펙터가 4개인 토픽을 만들 때 발생하는 에러이다. [appuser@kafka-1 ~]$ kafka-topics --create --bootstrap-server kafka-1:9092,kafka-2:9093,kafka-3:9094 --replication-factor 1 --partitions 1 --topic mytopic-1-1 Created topic mytopic-1-1. [appuser@kafka-1 ~]$ kafka-topics --create --bootstrap-server kafka-1:9092,kafka-2:9093,kafka-3:9094 --replication-factor 2 --partitions 2 --topic mytopic-2-2 Create..
![]()
Data Engineering/Kafka
docker-compose.yml version: '2.1' services: zookeeper-1: hostname: zookeeper-1 container_name: zookeeper-1 image: zookeeper:3.6 restart: always ports: - 2181:2181 environment: ZOO_MY_ID: 1 ZOO_SERVERS: server.1=zookeeper-1:2888:3888;2181 server.2=zookeeper-2:2888:3888;2181 server.3=zookeeper-3:2888:3888;2181 volumes: - type: bind source: ./data/zk-cluster/zookeeper-1/data target: /data read_only..
![]()
Data Engineering/Kafka
docker-compose.yml version: '2.1' services: zookeeper-1: hostname: zookeeper-1 container_name: zookeeper-1 image: zookeeper:3.6 restart: always ports: - 2181:2181 environment: ZOO_MY_ID: 1 ZOO_SERVERS: server.1=zookeeper-1:2888:3888;2181 server.2=zookeeper-2:2888:3888;2181 server.3=zookeeper-3:2888:3888;2181 volumes: - type: bind source: ./data/zk-cluster/zookeeper-1/data target: /data read_only..
