http://wiki.apache.org/hadoop/CouldOnlyBeReplicatedTo

Too Many Open Files

You can see this on Linux machines in client-side applications, server code or even in test runs.

It is caused by per-process limits on the number of files that a single user/process can have open, which was introduced inthe 2.6.27 kernel. The default value, 128, was chosen because "that should be enough".

In Hadoop, it isn't. To fix this log in/su/ssh as root and edit/etc/sysctl.conf

add the line

fs.epoll.max_user_instances = 20480

Then reboot. Different numbers may be chosen.

There is an immediate shortcut: echo 2048 > /proc/sys/fs/epoll/max_user_instances . This setting will be lost on the next reboot, but is handy for trying out different values.

limits.conf

Another limit on the number of files open may be the file /etc/security/limits.conf

It has a setting on the number of files a user or group may have,nofile.

To set this, as root edit /etc/security/limits.conf

and add a line such as

*               soft    nofile            20480
*               hard    nofile            20480

Then restart terminal to take effect.

To see the current/default limits, run the command ulimit -a. This should print something like

core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 20
file size               (blocks, -f) unlimited
pending signals                 (-i) 16382
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 4096
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) unlimited
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

You can dynamically up the limits until the next reboot with the same command. Specifically

ulimit -n 8192

The updated value can then be printed

# ulimit -n
8192

这个问题的出现原因是system对程序打开files数量的限制问题。

用 ulimit -a 命令可以查看

　　普通的设置，退出shell再次登录，一般是不生效，那就来点绝的：

到//etc/profile下，加入一行ulimit -SHn 51200 (注意里面已经有一条关于ulimit的设置)

到rc.local下，加入一行 ulimit -SHn 51200 (不知这条有没有必要，其实我还没加)

详见 ：

ulimit -S -c 0 > /dev/null 2>&1

这条语句设置了对软件资源和对core文件大小的设置

免费提供最新Linux技术教程书籍，为开源技术爱好者努力做得更多更好：https://www.linuxprobe.com/

在linux中跑java代码的过程中，发现日志报too many openfiles 的错误。

查了两天都没发现问题，感觉自己把该关闭的资源都关闭了。后来在网上查找解决方法，用：

lsof -n|awk ‘{print $2}’ |sort|uniq -c|sort -nr|more

lsof | grep pid

的命令，找到了问题所在。
解决办法是java代码里用okhttp3的addHeader，加上.addHeader(“Connection”,“close”)。之前没有加上这个，okhttp3默认的tcp连接方式是keep-alive，所以tcp会一直保持连接，就会报“too many open files”的错误。