Skip to content

This tool can easily make / build an emr cluster edge node / client node / gateway node

Notifications You must be signed in to change notification settings

bluishglc/emr-edgenode-maker

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 

Repository files navigation

脚本使用方法

关于该脚本的详细介绍,请参考作者博客文章:如何制作一个AWS EMR客户端节点( Client Node / Edge Node / Gateway Node )

检出后需要先给shell文件make-emr-edge-node.sh添加可执行属性:

chmod a+x make-emr-edge-node.sh

然后可以通过help选项查看使用方式:

sudo ./make-emr-edge-node.sh help

==================  [ MAKE EMR EDGE NODE SCRIPTS ] USAGE  ==================

# 说明:初始化(只需执行一次), 安装基础软件包,emr基础app, 配置emr repo, 创建hadoop用户, 创建必要文件夹
./make-emr-edge-node.sh init [PEM_FILE_PATH] [MASTER_NODE_IP]

# 说明:制作hadoop客户端
./make-emr-edge-node.sh make-hadoop-client [PEM_FILE_PATH] [MASTER_NODE_IP]

# 说明:制作spark客户端
./make-emr-edge-node.sh make-spark-client [PEM_FILE_PATH] [MASTER_NODE_IP]

# 说明:制作hive客户端
./make-emr-edge-node.sh make-hive-client [PEM_FILE_PATH] [MASTER_NODE_IP]

# 说明:制作hbase客户端
./make-emr-edge-node.sh make-hbase-client [PEM_FILE_PATH] [MASTER_NODE_IP]

# 说明:制作oozie客户端
./make-emr-edge-node.sh make-oozie-client [PEM_FILE_PATH] [MASTER_NODE_IP]

# 说明:制作hudi客户端
./make-emr-edge-node.sh make-hudi-client

# 说明:制作sqoop客户端
./make-emr-edge-node.sh make-sqoop-client

关于脚本的使用方式,如下几点需要注意:

  1. 由于脚本中大量操作需要使用root权限,所以必须以root用户或sudo方式运行脚本,否则脚本会给出错误提示
  2. 命令行中只涉及两个参数:
  • [PEM_FILE_PATH] :你的AWS账号的PEM文件,执行脚本需要使用该秘钥文件登入Master节点拷贝配置文件;
  • [MASTER_NODE_IP]:你的Master节点IP或域名
  1. 在执行make-*-client操作前,需要先执行init操作(仅一次即可)

About

This tool can easily make / build an emr cluster edge node / client node / gateway node

Topics

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages