阿里云/centos7配置hadoop-3.0（单机模式）

修改系统hostname

1.修改network文件

vi /etc/sysconfig/network
//增加如下两行
NETWORKING=yes
HOSTNAME=node01

2.修改hosts

vi /etc/hosts
//增加一行
127.0.0.1   localhost localhost.node01

3.修改hostname文件

vi /etc/hostname
//内容改为
node01

完成后重启服务器

安装jdk

1.下载解压

下载地址jdk1.8.0-201

解压

tar -zxvf jdk-8u201-linux-x64.tar.gz

2.配置环境变量

vi /etc/profile
//增加两行
export JAVA_HOME=/root/jdk1.8.0_201
export PATH=PATH:JAVA_HOME/bin

配置好后输入代码，显示如下图即为成功

source /etc/profile
java -version

安装hadoop

1.下载解压hadoop

地址hadoop3.0.2

然后解压

tar -zxvf hadoop-3.0.2.tar.gz

2.配置文件

进入hadoop目录/etc/hadoop,配置以下配置文件

1.配置hadoop-env.sh

vi hadoop-env.sh

修改 JAVA_HOME路径为java 安装路径

2.配置core-site.xml

vi core-site.xml

.配置默认采用的文件系统。

fs.defaultFS
hdfs://node01:9000/


hadoop.tmp.dir
/home/thousfeet/app/hadoop-3.0.0/data/

3.配置hdfs-site.xml

vi hdfs-site.xml
//内容为

dfs.replication
1

4.配置 mapred-site.xml

vi mapred-site.xml



mapreduce.framework.name
yarn

5.配置 yarn-site.xml

vi yarn-site.xml


yarn.resourcemanager.hostname
node01


yarn.nodemanager.aux-services
mapreduce_shuffle

6.配置环境变量

vi /etc/profile
//加入一行
export HADOOP_HOME=/root/hadoop-3.0.2
//修改PATH为
export PATH=PATH:JAVA_HOME/bin:HADOOP_HOME/bin:HADOOP_HOME/sbin
//更新文件
source /etc/profile

SSH免密登陆

ssh-keygen -t rsa
cd .ssh
touch authorized_keys
chmod 600 authorized_keys
cat id_rsa.pub >> authorized_keys

完成

然后尝试能否成功登陆

ssh node01

启动Hadoop

1.初始化

hadoop namenode -format

2.启动

start-dfs.sh
start-yarn.sh

ERROR: Attempting to launch hdfs namenode as root
ERROR: but there is no HDFS_NAMENODE_USER defined. Aborting launch.
Starting datanodes
ERROR: Attempting to launch hdfs datanode as root
ERROR: but there is no HDFS_DATANODE_USER defined. Aborting launch.
Starting secondary namenodes [localhost.localdomain]
ERROR: Attempting to launch hdfs secondarynamenode as root
ERROR: but there is no HDFS_SECONDARYNAMENODE_USER defined. Aborting launch.

HDFS_DATANODE_USER=root
HDFS_DATANODE_SECURE_USER=hdfs
HDFS_NAMENODE_USER=root
HDFS_SECONDARYNAMENODE_USER=root

Starting resourcemanager
ERROR: Attempting to launch yarn resourcemanager as root
ERROR: but there is no YARN_RESOURCEMANAGER_USER defined. Aborting launch.

YARN_RESOURCEMANAGER_USER=root
HADOOP_SECURE_DN_USER=yarn
YARN_NODEMANAGER_USER=root

完成后输入jps 查看进程

完成

打开http://你的服务器ip:9870/dfshealth.html#tab-overview

至此安装成功