【生产级实践】Docker部署配置Hadoop3.x + HBase2.x实现真正分布式集群环境

以下是一个使用Docker部署Hadoop 3.x和HBase 2.x的示例配置。请注意，这仅是一个配置样例，您需要根据自己的需求进行相应的修改。

1. 创建docker-compose.yml文件:

version: '3'
services:
  hbase-master:
    image: dajobe/hbase
    container_name: hbase-master
    ports:
      - "16010:16010"
    environment:
      - HBASE_CONF_zookeeper_sessiontimeout=20000
    command: start-master.sh
 
  hbase-regionserver:
    image: dajobe/hbase
    container_name: hbase-regionserver
    ports:
      - "16020:16020"
      - "16030:16030"
    environment:
      - HBASE_CONF_zookeeper_sessiontimeout=20000
    command: start-regionserver.sh
    depends_on:
      - hbase-master
 
  zookeeper:
    image: zookeeper:3.5
    container_name: zookeeper
    ports:
      - "2181:2181"
 
  hadoop-namenode:
    image: bde2020/hadoop-namenode:3.0.0-hadoop3.2.1-java8
    container_name: hadoop-namenode
    ports:
      - "9870:9870"
    environment:
      - CLUSTER_NAME=test
    command: ["hadoop-daemon.sh", "start", "namenode"]
 
  hadoop-datanode:
    image: bde2020/hadoop-datanode:3.0.0-hadoop3.2.1-java8
    container_name: hadoop-datanode
    environment:
      - CLUSTER_NAME=test
    command: ["hadoop-daemon.sh", "start", "datanode"]
    depends_on:
      - hadoop-namenode
 
  hdfs-journalnode:
    image: bde2020/hadoop-journalnode:3.0.0-hadoop3.2.1-java8
    container_name: hdfs-journalnode
    command: ["hadoop-daemon.sh", "start", "journalnode"]
    depends_on:
      - hadoop-namenode
 
  hbase-thrift:
    image: dajobe/hbase
    container_name: hbase-thrift
    ports:
      - "9095:9095"
    environment:
      - HBASE_CONF_zookeeper_sessiontimeout=20000
    command: start-thrift.sh
    depends_on:
      - hbase-master
      - hbase-regionserver
      - zookeeper
 
  hbase-rest:
    image: dajobe/hbase
    container_name: hbase-rest
    ports:
      - "8080:8080"
    environment:
      - HBASE_CONF_zookeeper_sessiontimeout=20000
    command: start-rest.sh
    depends_on:
      - hbase-master
      - hbase-regionserver
      - zookeeper

2. 在含有该docker-compose.yml文件的目录中运行以下命令来启动集群:

docker-compose up -d

这个配置定义了一个由Hadoop HDFS、HBase、Zookeeper和Thrift服务组成的分布式环境。它将相关的服务运行在Docker容器中，并通过Docker网络连接它们。您可以根据需要调整配置，例如，增加或减少DataNode或JournalNode的数量，或者指定不同的Hadoop和HBase版本。