Windows运行Hadoop时报错：Could not locate executable null\bin\winutils.exe in the Hadoop binaries

从标题报错中就可以看到，不能定位到winutils.exe，在hadoop中

这个错误一般是windos下运行报错的，linux下不需要
winnutils.exe，hadoop.dll

解决： 只需要下载对应的winnutils.exe 文件，hadoop.dll 文件 放进hadoop bin目录下即可，除此之外，还需要配置path环境变量

1、地址 https://github.com/cdarlint/winutils
找到对应hadoop版本，下载。

将下载的文件，放到bin目录下，即可

2、配置一下环境变量

本教程在windows10环境下安装配置hadoop+hbase+spark。

一、下载压缩包：

1、JavaJDK，1.8以上。

2、Hadoop2.8.3，下载地址：http://archive.apache.org/dist/hadoop/core/

Hadoop2.8.3windows环境下的第三方包：winutils，

https://github.com/steveloughran/winutils，如果报nativeio异常，可以将此包解压至hadoop的bin目录下。

3、Spark2.2.0，下载地址：http://archive.apache.org/dist/spark/spark-2.2.0/

4、选择spark-2.2.0-bin-hadoop2.7点击下载。

二、安装配置

1、安装jdk并配置环境变量。

2、解压缩hadoop，配置环境变量。

右键单击我的电脑 –>属性 –>高级环境变量配置 –>高级选项卡 –>环境变量 –> 单击新建HADOOP_HOME，如下图

3、修改hadoop配置文件

1）修改core-site.xml配置文件，位置在hadoop目录下etc/hadoop/core-site.xml，添加如下配置：

fs.defaultFS

hdfs://localhost:9000

2）修改hdfs-site.xml配置文件，位置在hadoop目录下etc/hadoop/hdfs-site.xml，添加如下配置：

dfs.replication

1

dfs.namenode.name.dir

/E:/geotrellis/hadoop-2.8.3/data/dfs/namenode

dfs.datanode.data.dir

/E:/geotrellis/hadoop-2.8.3/data/dfs/datanode

注意value里的目录，我这里使用全路径配置。

根据上面的配置，在hadoop目录下创建namenode、datanode节点存储目录。如下：

3）重命名mapred-site.xml.template为mapred-site.xml，位置同上添加配置：

mapreduce.framework.name

yarn

4）配置yarn-site.xml，位置同上

yarn.nodemanager.aux-services

mapreduce_shuffle

yarn.nodemanager.aux-services.mapreduce.shuffle.class

org.apache.hadoop.mapred.ShuffleHandler

至此，hadoop配置完成，hdfs负责分布式存储，mapreduce负责分布式计算,yarn负责资源调度，基本的（伪）分布式环境初步建立。

三、测试使用

1、格式化namenode的hdfs目录

cmd命令下输入执行:hdfs namenode –format

2、启动hadoop，定位到sbin目录，如下输入

3、启动所有hadoop服务，输入start-all

这时候会弹出4个窗口，输入jps查看进程情况，如下：

输入：http://localhost:8088/ 查看hadoop的启动情况

4、测试和使用

1）创建hdfs的目录，如下：

Hadoop fs –mkdir /user

Hadoop fs –mkdir /user/input

2）上传文件

Hadoop fs –put e:/test.txt /user/input

Txt文本内容如下，输入：

Hadoop fs –cat /user/input/test.txt

3）词频计算

hadoop jar E:/geotrellis/hadoop-2.8.3/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.8.3.jar wordcount/user/input/ /user/output

4）查看运行结果，如下：

windows本地配置hadoop运行环境，可以帮助我们更好进行开发或者hadoop学习。
本文介绍了如何搭建一个hadooop伪运行环境。

一、下载安装包并配置环境变量

下载hadoop binary安装安装包：
https://hadoop.apache.org/releases.html
下载完后配置环境变量：
HADOOP_HOME=D:\hadoop-2.6.0
path里面添加：%HADOOP_HOME%/bin

二、修改hadoop配置文件

进入hadoop安装目录D:\hadoop-2.6.0\etc\hadoop，修改D:\hadoop-2.6.0\etc\hadoop下面的
core-site.xml

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
  <property>
    <name>fs.defaultFS</name>
    <value>hdfs://localhost:9000/</value>
  </property>
  <property>
    <name>io.native.lib.available</name>
    <value>false</value>
  </property>
  <property>
    <name>hadoop.native.lib</name>
    <value>false</value>
  </property>
  <property>
    <name>io.compression.codecs</name>
    <value>org.apache.hadoop.io.compress.GzipCodec,
           org.apache.hadoop.io.compress.DefaultCodec,
           com.hadoop.compression.lzo.LzoCodec,
           com.hadoop.compression.lzo.LzopCodec,
           org.apache.hadoop.io.compress.BZip2Codec,
           org.apache.hadoop.io.compress.SnappyCodec
        </value>
</property>
<property>
    <name>io.compression.codec.lzo.class</name>
    <value>com.hadoop.compression.lzo.LzoCodec</value>
</property>
</configuration>

hdfs-site.xml

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
   <property>  
		   <name>dfs.replication</name>  
			<value>1</value>  
   </property>
   <property>
			<name>dfs.namenode.name.dir</name>  
		   <value>file:///D:/Hadoop/namenode</value>  
   </property>
   <property>
		   <name>dfs.datanode.data.dir</name>  
		   <value>file:///D:/Hadoop/datanode</value>  
   </property>
</configuration>

mapred-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
    <property>
        <name>mapred.compress.map.output</name>
        <value>true</value>
    </property>
    <property>
        <name>mapred.map.output.compression.codec</name>
        <value>com.hadoop.compression.lzo.LzoCodec</value>
    </property> 
    <property> 
        <name>mapred.child.env</name> 
        <value>LD_LIBRARY_PATH=</value>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
    <property>
        <name>mapred.compress.map.output</name>
        <value>true</value>
    </property>
    <property>
        <name>mapred.map.output.compression.codec</name>
        <value>com.hadoop.compression.lzo.LzoCodec</value>
    </property> 
    <property> 
        <name>mapred.child.env</name> 
        <value>LD_LIBRARY_PATH=D:\hadoop-2.7.3-win64\lib</value> 
    </property>
</configuration>

然后到bin目录下执行：
hdfs namenode -format
再到sbin目录下执行：
start-all.cmd

三、安装完后验证

浏览器输入：http://localhost:8088可以打开管理界面
命令行输入：

D:\hadoop-2.6.0\sbin>hadoop fs -ls /
Found 2 items
drwxr-xr-x   - HP supergroup          0 2019-06-19 10:28 /data
drwxr-xr-x   - HP supergroup          0 2019-06-19 11:00 /input

D:\hadoop-2.6.0\sbin>hadoop fs -mkdir /data1

D:\hadoop-2.6.0\sbin>hadoop fs -ls /
Found 3 items
drwxr-xr-x   - HP supergroup          0 2019-06-19 10:28 /data
drwxr-xr-x   - HP supergroup          0 2019-06-19 11:20 /data1
drwxr-xr-x   - HP supergroup          0 2019-06-19 11:00 /input