操作系统 centos7
java verson ： jdk1.8.0_151
hadoop version ： hadoop-2.8.1
1、把安装包放到 opt文件夹下面。并进行解压。
cd /opt
tar -zxvf hadoop-2.8.1
2、配置jdk、hadoop文件路径
vi /etc/profile
export JAVA_HOME=/opt/jdk1.8.0_151
export JRE_HOME=${JAVA_HOME}/jre
export HADOOP_HOME=/opt/hadoop-2.8.1
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH:${HADOOP_HOME}/bin
3、进入 etc/hadoop 目录下进行配置文件设置
修改指定运行的java路径地址
hadoop-env.sh
# The java implementation to use.
# export JAVA_HOME=${JAVA_HOME}
export JAVA_HOME=/opt/jdk1.8.0_151core-site.xml 默认的配置文件
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000/</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/hadoop/hadoop-2.8.1/tmp</value>
</property>
hdfs-site.xml
<property>
<name>dfs.replication</name>
<value>1</value>
</property>mapred-site.xml
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>yarn-site.xml
<!-- <property>
<name>yarn.resourcemanager.address</name>
<value>localhost:9000</value>
</property> -->
<property>
<name>yarn.resourcemanager.hostname</name>
<value>localhost</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>hadoop namenode -format
4、启动
cd sbin
./start-dfs.sh
./start-yarn.sh
5、校验查看
jps
6、浏览器地址访问
http://localhost:8088/cluster
http://localhost:50070
7、向dfs中上传文件
hadoop fs -put jdk-8u131-linux-x64.tar.gz hdfs://localhost:9000/devtools
8、下载文件
hadoop fs -get hdfs://localhost:9000/jdk-8u131-linux-x64.tar.gz
9、运行mapreduce程序
hadoop jar hadoop-map
hadoop jar hadoop-mapreduce-examples-2.8.3.jar wordcount /article /artout //计算文本中词汇的个数
hadoop jar hadoop-mapreduce-examples-2.8.3.jar pi 5 5 //计算圆周率
10\免密登录
$ ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa $ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys $ chmod 0600 ~/.ssh/authorized_keys
ssh localhost
11\开放端口
firewall-cmd --zone=public --add-port=6379/tcp --permanent
