데이터 엔지니어

Data Engineering/Kafka

[kafka] kafka cluster 구성 후 토픽 생성시의 현상 확인해보기

docker-compose.yml version: '2.1' services: zookeeper-1: hostname: zookeeper-1 container_name: zookeeper-1 image: zookeeper:3.6 restart: always ports: - 2181:2181 environment: ZOO_MY_ID: 1 ZOO_SERVERS: server.1=zookeeper-1:2888:3888;2181 server.2=zookeeper-2:2888:3888;2181 server.3=zookeeper-3:2888:3888;2181 volumes: - type: bind source: ./data/zk-cluster/zookeeper-1/data target: /data read_only..

Operating System/Network

[Sourcetree] Windows11 에 소스트리 설치하는 방법

https://www.sourcetreeapp.com/ Sourcetree | Free Git GUI for Mac and Windows A Git GUI that offers a visual representation of your repositories. Sourcetree is a free Git client for Windows and Mac. www.sourcetreeapp.com

Operating System/Docker

[Docker] executor failed running [/bin/sh -c apt-get update]: exit code: 100 에러

도커이미지 빌드시 실패한 이력이 있어서 빌드가 안됨 빌드 옵션에 --no-cache 사용할 것

Data Engineering/Kafka

[Kafka] 카프카 클러스터 만들기 (confluentinc/cp-kafka:7.1.0)

docker-compose.yml version: '2.1' services: zookeeper-1: hostname: zookeeper-1 container_name: zookeeper-1 image: zookeeper:3.6 restart: always ports: - 2181:2181 environment: ZOO_MY_ID: 1 ZOO_SERVERS: server.1=zookeeper-1:2888:3888;2181 server.2=zookeeper-2:2888:3888;2181 server.3=zookeeper-3:2888:3888;2181 volumes: - type: bind source: ./data/zk-cluster/zookeeper-1/data target: /data read_only..

Data Engineering/Kafka

[Zookeeper] 주키퍼에 특정 znode에 어떤 znode가 들어있는지 확인하는 방법

docker exec zookeeper-1 zkCli.sh ls /zookeeper docker exec zookeeper-2 zkCli.sh ls /zookeeper docker exec zookeeper-3 zkCli.sh ls /zookeeper

Data Engineering/Kafka

[Zookeeper] 주키퍼에 어떤 znode가 들어있는지 확인하는 방법

docker-compose.yml version: '2.1' services: zookeeper-1: hostname: zookeeper-1 container_name: zookeeper-1 image: zookeeper:3.6 restart: always ports: - 2181:2181 environment: ZOO_MY_ID: 1 ZOO_SERVERS: server.1=zookeeper-1:2888:3888;2181 server.2=zookeeper-2:2888:3888;2181 server.3=zookeeper-3:2888:3888;2181 volumes: - type: bind source: ./data/zk-cluster/zookeeper-1/data target: /data read_only..

Data Engineering/Kafka

[Zookeeper] 주키퍼 마스터 찾는 방법

docker-compose.yml version: '2.1' services: zookeeper-1: hostname: zookeeper-1 container_name: zookeeper-1 image: zookeeper:3.6 restart: always ports: - 2181:2181 environment: ZOO_MY_ID: 1 ZOO_SERVERS: server.1=zookeeper-1:2888:3888;2181 server.2=zookeeper-2:2888:3888;2181 server.3=zookeeper-3:2888:3888;2181 volumes: - type: bind source: ./data/zk-cluster/zookeeper-1/data target: /data read_only..

Data Engineering/Kafka

[Zookeeper] Zookeeper cluster + Zookeeper UI

docker-compose.yml version: '2.1' services: zookeeper-1: hostname: zookeeper-1 container_name: zookeeper-1 image: zookeeper:3.6 restart: always ports: - 2181:2181 environment: ZOO_MY_ID: 1 ZOO_SERVERS: server.1=zookeeper-1:2888:3888;2181 server.2=zookeeper-2:2888:3888;2181 server.3=zookeeper-3:2888:3888;2181 volumes: - type: bind source: ./data/zk-cluster/zookeeper-1/data target: /data read_only..

Data Engineering/Kafka

[Zookeeper] Zookeeper Cluster (docker-compose)

파일구조 docker-compose.yml 파일 version: '2.1' services: zookeeper-1: hostname: zookeeper-1 container_name: zookeeper-1 image: zookeeper:3.6 restart: always ports: - 2181:2181 environment: ZOO_MY_ID: 1 ZOO_SERVERS: server.1=zookeeper-1:2888:3888;2181 server.2=zookeeper-2:2888:3888;2181 server.3=zookeeper-3:2888:3888;2181 volumes: - type: bind source: ./data/zk-cluster/zookeeper-1/data target: /data r..

Data Engineering/Spark

[Spark] Docker, failed: port is already allocated

Creating zookeeper-navigator ... done Creating spark-master-1 ... done Creating spark-master-2 ... done WARNING: The "spark-slave" service specifies a port on the host. If multiple containers for this service are created on a single host, the port will clash. Creating zeppelin ... Creating 3_spark-cluster-zookeeper_spark-slave_1 ... Creating 3_spark-cluster-zookeeper_spark-slave_1 ... error Crea..

박경태
'분류 전체보기' 카테고리의 글 목록 (61 Page)