disk io linux - CSDN
  • Linux 测试 IO 性能(磁盘读写速度)

    万次阅读 2018-01-10 15:29:52
    Linux 测试 IO 性能(磁盘读写速度) 这几天做MySQL性能测试,偌大一个公司,找几台性能测试机器都很纠结,终于协调到两台,IO的性能如何还不知道。 数据库属于IO密集型的应用,所以还是先评估下Server的IO...

    Linux 测试 IO 性能(磁盘读写速度)

    这几天做MySQL性能测试,偌大一个公司,找几台性能测试机器都很纠结,终于协调到两台,IO的性能如何还不知道。
    数据库属于IO密集型的应用,所以还是先评估下ServerIO性能,看看是否能和线上的机器匹配上。

    之前一直知道用dddevice to device)命令可以简单测试磁盘的IO读写速度,但没有深究。
    但这次做性能测试的关系,需要得到一个相对精确的值(之前的测试吃过这方面的亏,插个题外话,性能测试一定要首先确认好测试环境。)
    网上常见的方法是使用hdparmdd命令来测试,但实际的使用起来都有问题,而且测试结果总感觉有偏差,心里没底。
    于是还是安心研究了下这两个命令,也做了一些测试和分析,简单做下总结。

    用法:
    1.
    测试IO

         hdparm -t --direct /dev/sda3

         IO
    读用上面的命令测试即可,不过 hdparm 这个工具需要自己安装,而且需要root用户去执行。

    2.
    测试IO
        sync;/usr/bin/time -p bash -c "(dd if=/dev/zero of=test.dd  bs=1000K count=20000;sync)"

        dd bs=1M count=20000 if=/dev/zero of=test.dd conv=fdatasync   dd命令测试是IO的顺序写和读方式。

    3.查看文件系统块大小

       tune2fs -l /dev/sda1 | grep Block

    上面的命令行有些复杂,做下详细的解释:
    1sync 命令用来刷新文件系统的缓冲区,执行sync命令实际的作用是把内存中的数据缓冲写入到磁盘中。
             
    先执行下sync命令,是为了减少对后面测试的影响。也可以使用 echo 3 > /proc/sys/vm/drop_caches 来清除缓存。
    2time 命令用来测试命令的执行时间,shell内建还有一个time命令,我们这里使用全路径来指定使用的是非内建命令。
            -p
    选项设置时间的输出格式为POSIX缺省时间格式,单位是秒,在后面的测试小节可以看到time -p 的输出形式。
    3bash 命令 -c 选项的作用是将后面的字符串参数当作bash脚本来执行,看起来有些画蛇添足,好像直接执行也是可行的,
           
    其实不然,因为后面字符串中包含了两条命令行,而time命令需要统计这两条命令行的执行时间。
    4)小括号的意思是另起一个子进程来执行括号中的脚本,dd 这条命令行这里就不讲了,不知道的话Baidu吧。
            
    重点是sync命令,因为当dd退出时,这条命令行提交的大部分内容都在内存缓冲区(写缓存),甚至如果机器的内存相比于你提交的数据要大得多,
            
    那可能数据都在内存中,而再执行sync才能将内存中的数据写入到磁盘中,否则就成了测试内存的写速度,那显然不是你想要的结果。


    测试实践:
    在一台内存64GSAS硬盘上做了一些测试。
    测试过程中,需要关注两点,磁盘的读写速度和IO使用率,我们分别使用ioppiostat工具来观察。

    测试写速度:
    测试写入20G数据,数据量越大,测试值应该更精确。
    # sync;/usr/bin/time -p bash -c "(dd if=/dev/zero of=test.dd  bs=1M count=20000)"
    20000+0 records in
    20000+0 records out
    real 92.87
    user 0.00
    sys 18.08

    写入20000M的时间是92.87s,所以磁盘的写速度为 215.35MB/sec (20000M/92.87)
    使用iostat观测,%util 一直处于100%的状态,而iopp 显示的dd命令写入速度基本和上面计算出的值是一致的(取样时间设置为5s)
    # iostat -x 5
    Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
    sda               0.00     0.00    0.20  864.00     1.60 449004.80   519.56   143.61  163.46   1.16 100.00

    # iopp -i -k 5
    pid    rchar    wchar    syscr    syscw      rkb      wkb     cwkb command
    32566  1033216  1033216        0        0        0  1034868        0 dd

    注:命令行的当前工作目录在分区/dev/sda3上,所以test.dd这个文件也创建在这个分区上,下面的读测试会使用到这个文件。

    测试读速度:
    dd hdparm的做一个对比:
    #  hdparm -t --direct /dev/sda3
    /dev/sda3:
     Timing O_DIRECT disk reads: 2108 MB in  3.00 seconds = 702.34 MB/sec

    hdparm
    的测试结果为,3s读取了2108MB,读速度为702.34MB

    由于hdparm测试时间太短,iostatiopp取样时间调整为1s

    iostat -x 1
    Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
    sda               0.00     0.00 2752.00    1.00 1409024.00     8.00   511.82     4.64    1.69   0.35  96.10

    iopp -i -k 1
    pid    rchar    wchar    syscr    syscw      rkb      wkb     cwkb command
    32349   733184        0        0        0   733184        0        0 hdparm

    使用dd做读取测试
    # echo 3 > /proc/sys/vm/drop_caches ; /usr/bin/time -p dd if=test.dd of=/dev/null  bs=1M 
    20000+0 records in
    20000+0 records out
    real 35.69
    user 0.00
    sys 9.81

    读取了20G,读速度为560.38MB/sec

    读取会比写入要快,iostatiopp的取样时间调整为每3s
    Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
    sda             106.67     0.00 5851.00    0.33 1434256.00     2.67   245.12     1.62    0.28   0.16  93.60

    pid    rchar    wchar    syscr    syscw      rkb      wkb     cwkb command
    17004  1449984  1449984        1        1  1450760        0        0 dd

    结论:

    dd测试出的读速度和hdparm 是存在区别的,
    通过 bs 选项 设置不通的读写块大小测试(默认512字节,测试使用1M),
    可以看出 dd 的测出的速度与读写块的大小有关系,还可能受到系统中有IO读写的进程的影响。
    hdparm
    的测试原理没做深入研究,可能是和dd的测试方法存在差别,需要知道这一点。

    整体上看,IO的实际测试速度是受到很多因素影响的,包括读写的方式(随机还是顺序,hdparmdd测试是都是采用顺序读写)、缓存机制、测试的取样等等。
    所以不太可能得到一个确定的值(相同的命令行多次测试也不一样,不过差别要小些),以上的方法中读测试还是推荐使用hdparm
    以上的数据虽然存在偏差,但还是能大体分析出机器的IO性能。只是需要明确,这些测试值是在什么样的环境下获得的。

    友情链接:

    FIO测试iopshttp://elf8848.iteye.com/blog/2168876

    dd测试io: http://www.askoracle.org/linux/disk/968.html

    展开全文
  • ubuntu 下的硬盘 io 情况

    千次阅读 2013-12-05 20:52:13
    对于JAVA程序员来说,磁盘IO存在性能问题的时候可能不是那么多,频繁操作磁盘的也就是大量的日志、索引文件,甚或是JAVA写的数据库(如:hbase)等。   JAVA程序员在Linux下分析程序性能,top命令是比较常用的...

    对于JAVA程序员来说,磁盘IO存在性能问题的时候可能不是那么多,频繁操作磁盘的也就是大量的日志、索引文件,甚或是JAVA写的数据库(如:hbase)等。

     

    JAVA程序员在Linux下分析程序性能,top命令是比较常用的,比如这么一行:

     

    Cpu(s): 14.8%us,  0.8%sy,  0.0%ni, 60.3%id, 83.8%wa,  0.1%hi,  0.2%si,  0.0%st

     

    如果磁盘IO是瓶颈之一的话,那么%wa的值就会很高,那么这个时候就要分析磁盘IO了。其他字段这里就不再解释了(小提示:按1会显示你所有CPU的情况,这里这是个总体值的情况)

     

    用什么分析

     

    那么如何分析磁盘IO呢?要用到两个命令:iostat 和 sar 。目前我使用的是Gentoo系统,默认是没有这些分析工具的,需要emerge ,他们都在sysstat(app-admin/sysstat)下面,所以你只要

    emerge  app-admin/sysstat 

    就可以拥有iostat和sar分析工具。

     

    分析信息解释

     

    那么如何使用呢?

     

    iostat -d -x 10 3

    Device:  rrqm/s  wrqm/s  r/s  w/s  rsec/s  wsec/s avgrq-sz avgqu-sz  await  svctm  %util
    主要字段含义如下:

    r/s 每秒读操作数。

    w/s 每秒写操作数。

    rsec/s 每秒从设备读取的扇区数量。

    wsec/s 每秒向设备写入的扇区数量。

    avgrq-sz I/O 请求的平均扇区数。

    avgqu-sz I/O 请求的平均队列长度。

    await I/O 请求的平均等待时间,单位为毫秒。

    svctm I/O 请求的平均服务时间,单位为毫秒。

    %util 处理 I/O 请求所占用的时间的百分比,即设备利用率。


    #sar -pd 10 3
    输出的主要字段含义如下:

    DEV 正在监视的块设备

    tps 每秒钟物理设备的 I/O 传输总量

    rd_sec/s 每秒从设备读取的扇区数量

    wr_sec/s 每秒向设备写入的扇区数量

    avgrq-sz I/O 请求的平均扇区数

    avgqu-sz I/O 请求的平均队列长度

    await I/O 请求的平均等待时间,单位为毫秒

    svctm I/O 请求的平均服务时间,单位为毫秒

    %util I/O 请求所占用的时间的百分比,即设备利用率
    Unix/Linux 系统磁盘 I/O 性能监控自动化脚本示例

     

     

     

    =============================================================

    附上参考

    博客地址:http://dodo.ideation-cn.cn/?p=466

    内容:

     

    Unix/Linux 磁盘 I/O 性能监控命令

    磁盘 I/O 性能监控指标和调优方法

     

    在介绍磁盘 I/O 监控命令前,我们需要了解磁盘 I/O 性能监控的指标,以及每个指标的所揭示的磁盘某方面的性能。磁盘 I/O 性能监控的指标主要包括:

    指标 1:每秒 I/O 数(IOPS 或 tps)

    对于磁盘来说,一次磁盘的连续读或者连续写称为一次磁盘 I/O, 磁盘的 IOPS 就是每秒磁盘连续读次数和连续写次数之和。当传输小块不连续数据时,该指标有重要参考意义。

    指标 2:吞吐量(Throughput)

    指硬盘传输数据流的速度,传输数据为读出数据和写入数据的和。其单位一般为 Kbps, MB/s 等。当传输大块不连续数据的数据,该指标有重要参考作用。

    指标 3:平均 I/O 数据尺寸

    平均 I/O 数据尺寸为吞吐量除以 I/O 数目,该指标对揭示磁盘使用模式有重要意义。一般来说,如果平均 I/O 数据尺寸小于 32K,可认为磁盘使用模式以随机存取为主;如果平均每次 I/O 数据尺寸大于 32K,可认为磁盘使用模式以顺序存取为主。

    指标 4:磁盘活动时间百分比(Utilization)

    磁盘处于活动时间的百分比,即磁盘利用率,磁盘在数据传输和处理命令(如寻道)处于活动状态。磁盘利用率与资源争用程度成正比,与性能成反比。也就 是说磁盘利用率越高,资源争用就越严重,性能也就越差,响应时间就越长。一般来说,如果磁盘利用率超过 70%,应用进程将花费较长的时间等待 I/O 完成,因为绝大多数进程在等待过程中将被阻塞或休眠。

    指标 5:服务时间(Service Time)

    指磁盘读或写操作执行的时间,包括寻道,旋转时延,和数据传输等时间。其大小一般和磁盘性能有关,CPU/ 内存的负荷也会对其有影响,请求过多也会间接导致服务时间的增加。如果该值持续超过 20ms,一般可考虑会对上层应用产生影响。

    指标 6:I/O 等待队列长度(Queue Length)

    指待处理的 I/O 请求的数目,如果 I/O 请求压力持续超出磁盘处理能力,该值将增加。如果单块磁盘的队列长度持续超过 2,一般认为该磁盘存在 I/O 性能问题。需要注意的是,如果该磁盘为磁盘阵列虚拟的逻辑驱动器,需要再将该值除以组成这个逻辑驱动器的实际物理磁盘数目,以获得平均单块硬盘的 I/O 等待队列长度。

    指标 7:等待时间(Wait Time)

    指磁盘读或写操作等待执行的时间,即在队列中排队的时间。如果 I/O 请求持续超出磁盘处理能力,意味着来不及处理的 I/O 请求不得不在队列中等待较长时间。

     

    通过监控以上指标,并将这些指标数值与历史数据,经验数据以及磁盘标称值对比,必要时结合 CPU、内存、交换分区的使用状况,不难发现磁盘 I/O 潜在或已经出现的问题。但如果避免和解决这些问题呢?这就需要利用到磁盘 I/O 性能优化方面的知识和技术。限于本文主题和篇幅,仅列出一些常用的优化方法供读者参考:

    1. 调整数据布局,尽量将 I/O 请求较合理的分配到所有物理磁盘中。
    2. 对于 RAID 磁盘阵列,尽量使应用程序 I/O 等于条带尺寸或者为条带尺寸的倍数。并选取合适的 RAID 方式,如 RAID10,RAID5。
    3. 增大磁盘驱动程序的队列深度,但不要超过磁盘的处理能力,否则,部分 I/O 请求会因为丢失而重新发出,这将降低性能。
    4. 应用缓存技术减少应用存取磁盘的次数,缓存技术可应用在文件系统级别或者应用程序级别。
    5. 由于多数数据库中已包括经优化后的缓存技术,数据库 I/O 宜直接存取原始磁盘分区(raw partition)或者利用绕过文件系统缓存的 DIO 技术(direct IO)
    6. 利用内存读写带宽远比直接磁盘 I/O 操作性能优越的特点,将频繁访问的文件或数据置于内存中。
    磁盘 I/O 性能监控命令简介

    Unix/Linux 提供了两个非常有用的用于磁盘 I/O 性能监控的命令:iostat 和 sar。

    iostat 命令主要通过观察物理磁盘的活动时间以及他们的平均传输速度,监控系统输入 / 输出设备负载。根据 iostat 命令产生的报告,用户可确定一个系统配置是否平衡,并据此在物理磁盘与适配器之间更好地平衡输入 / 输出负载。iostat 工具的主要目的是通过监控磁盘的利用率,而探测到系统中的 I/O 瓶颈。不同操作系统命令格式输出格式略有不同,管理员可以通过查看用户手册来确定它的用法。

    sar 命令报告 CPU 的使用情况,I/O 以及其它系统行为。sar 命令可以收集,报告以及保存系统行为信息。以这种方式收集到的数据对于确定系统的时间周期特征和决定峰值使用时间是很有用的。但要注意的是,sar 命令自己运行时会产生相当数量的读写,因此最好在没有工作量的情况下运行 sar 统计,看看 sar 对总的统计数字有多大的影响。

    AIX 环境下,iostat 和 sar 位于文件集 bos.acct 中,是基本系统 (Base Operating System) 的一部分,在缺省安装情况下无需进行任何配置或任何额外包安装。

    Linux 环境下,iostat 和 sar 包含在 sysstat 包中。sysstat 是 Linux 系统中的常用工具包。不同的 Linux 系统或不同的硬件平台,安装包名字和具体命令可能会有微小差异。清单1为一个在 RHEL5.3 上安装 sysstat 包的例子。

    iostat -d -x 10 3

    Device:  rrqm/s  wrqm/s  r/s  w/s  rsec/s  wsec/s avgrq-sz avgqu-sz  await  svctm  %util
    主要字段含义如下:

    r/s 每秒读操作数。

    w/s 每秒写操作数。

    rsec/s 每秒从设备读取的扇区数量。

    wsec/s 每秒向设备写入的扇区数量。

    avgrq-sz I/O 请求的平均扇区数。

    avgqu-sz I/O 请求的平均队列长度。

    await I/O 请求的平均等待时间,单位为毫秒。

    svctm I/O 请求的平均服务时间,单位为毫秒。

    %util 处理 I/O 请求所占用的时间的百分比,即设备利用率。
    #sar -pd 10 3
    输出的主要字段含义如下:

    DEV 正在监视的块设备

    tps 每秒钟物理设备的 I/O 传输总量

    rd_sec/s 每秒从设备读取的扇区数量

    wr_sec/s 每秒向设备写入的扇区数量

    avgrq-sz I/O 请求的平均扇区数

    avgqu-sz I/O 请求的平均队列长度

    await I/O 请求的平均等待时间,单位为毫秒

    svctm I/O 请求的平均服务时间,单位为毫秒

    %util I/O 请求所占用的时间的百分比,即设备利用率
    Unix/Linux 系统磁盘 I/O 性能监控自动化脚本示例

    前文介绍了 AIX 与 Linux 磁盘 I/O 监控的手段,本节将结合实例介绍磁盘 I/O 监控的自动化脚本的设计和实现。

    设计思路

    1 . 监控指标

    前文介绍了多个监控指标,比如吞吐量,每秒 I/O 数,平均单次 I/O 响应时间,磁盘利用率等,用户可以根据自身系统和应用的特点选取监控指标。下文将以磁盘利用率为例进行监控脚本的设计。

    2 .监控手段

    对于 AIX, 命令“iostat -d”的输出字段“% tm_act”反映磁盘利用率;对于 Linux, 命令“iostat -d -x”的输出字段“%util”反映磁盘利用率。监控的频次可以通过“iostat”命令本身的参数指定。

    3. 告警机制

    一般来说,如果磁盘使用率长时间达到 75% 或 80%, 通常会视为磁盘较忙,一般需进行调整磁盘布局,调整应用使用分配等管理活动,如果磁盘使用率偶尔较高,只需要继续监视即可。因此告警既不能对于偶发现象频 繁告警,也不能放过持续时间较长的现象,本例的告警机制设定为“在一定时间间隔内的一定比例的监控记录超出设定的磁盘利用率门限值即告警”

    4 .日志

    要保留磁盘 I/O,分析活动,告警信息等原始信息,便于以后进行问题分析,定位。

    磁盘 I/O 分析和告警自动脚本

    结合上文设计思路,笔者开发了磁盘 I/O 分析和告警脚本 I/OAnalyzer.sh, 我们的脚本在 RHEL 5.3、SLES 11、AIX 5.3 和 AIX 6.1 上测试通过。

    脚本 IOAnalyzer.sh 的第一部分先进行了初始化,检验并处理输入参数,对未输入的参数指定默认值。

    清单 6. IOAnalyzer.sh 脚本初始化部分

    01 #!/bin/sh
    02   # ================================================================
    03   # Script Name:  IOAnalyzer.sh
    04   # Desciption:   To analyze output of 'iostat -d', send an alert to admin
    05   #              if disk utilization counter reach defined threshold
    06   # Date:         May 19, 2009
    07   #================================================================
    08  
    09 # -----------------------------------------------------------------
    10   # Function definitions...                                定义函数
    11   #----------------------------------------------------------------
    12  
    13 function usage {
    14   echo ""
    15   echo "usage: IOAnalyzer.sh -i inIostatFile [ -l outLogFile ] \
    16   [ -a outAlertFile ] [ -u dishUtil ] [ -r rateGEUtil ]"
    17   echo ""
    18   echo "For example: IOAnalyzer.sh -i /tmp/iostat.out -l /tmp/logFile \
    19   -a /tmp/aletFile -u 80 -r 70"
    20   echo "For AIX, please run 'iostat -d [ <interval> [ <count> ] \
    21   to create inIostatFile"
    22   echo "For Linux, please run 'iostat -d -x [ <interval> [ <count> ] \
    23   to create inIostatFile"
    24   exit 1
    25   }
    26  
    27 # ----------------------------------------------------------------
    28   # Process command-line arguments                          命令行参数处理
    29   # ----------------------------------------------------------------
    30   while getopts :i:l:a:u:r: opt
    31   do
    32   case "$opt" in
    33   i) inIostatFile= "$OPTARG" ;;
    34   l) outLogFile= "$OPTARG" ;;
    35   a) outAlertFile= "$OPTARG" ;;
    36   u) diskUtil= "$OPTARG" ;;
    37   r) rateGEUtil= "$OPTARG" ;;
    38   \?) usage;;
    39   esac
    40   done
    41  
    42 #----------------------------------------------------------------
    43   # Input validation                                       输入验证
    44   #----------------------------------------------------------------
    45  
    46 if [ ! -f  "$inIostatFile" ]
    47   then
    48   echo "error: invalid Augument inIostatFile in OPTION -i "
    49   usage
    50   exit 1
    51   fi
    52   #---------------------------------------------------------------
    53  
    54 # Set values, if unset                                   设置变量
    55   # ----------------------------------------------------------------
    56  
    57 outLogFile=${outLogFile:-${inIostatFile}.log}
    58   outAlertFile=${outAlertFile:-${inIostatFile}.alert}
    59   diskUtil=${diskUtil:- '80' }
    60   rateGEUtil=${rateGEUtil:- '60' }

    接下来, IOAnalyzer.sh 脚本查询日志,通过计算起止行的办法定位 IO 输出文件的待分析文本。

    清单 7. IOAnalyzer.sh 脚本定位 I/O 输出文件待分析部分

    01 # ----------------------------------------------------------------
    02   # Identify the lines to be analyzed between StartLine and Endline
    03   # 定位日志中待分析文本
    04  
    05 # ----------------------------------------------------------------
    06  
    07 if [ ! -f  "$outLogFile" ] || [ !  tail -1  "$outLogFile" | grep 'ENDLINE' ]
    08   then
    09   StartLineNum=1;
    10   else
    11   CompletedLine=` tail -1  "$outLogFile" |  grep 'ENDLINE' | \
    12   awk '{print $4}' | cut -d: -f2`
    13   StartLineNum=` expr 1 + $CompletedLine`
    14   fi
    15  
    16 eval "sed -n '${StartLineNum},\$p' $inIostatFile" > ${inIostatFile}. tail
    17  
    18 LineCount=` cat ${inIostatFile}. tail | wc -l| awk '{print $1}' `
    19   EndLineNum=` expr $LineCount + $StartLineNum`

    清单 7 中的脚本实现了按行分析上文定位的 iostat 输出,如果某行磁盘利用率小于先前定义的门限值,则在行尾标记“OK”,如果某行磁盘利用率大于等于先前定义的门限值,则在行尾标记“Alarm”。并且 脚本中对于 AIX 和 Linux 输出格式和磁盘命名的不同作了相应处理。

    清单 8. IOAnalyzer.sh 按行分析 iostat 输出

    01 # ----------------------------------------------------------------
    02   # Analyze 'iostat' output, append "Alarm" or "OK" at the end of each# line
    03  
    04 # ----------------------------------------------------------------
    05   OS=` uname `
    06   case "$OS" in
    07   AIX)
    08   diskUtilLabel= "% tm_act"
    09   diskUtilCol=2
    10   diskPrefix= "hdisk"
    11   ;;
    12   Linux)
    13   diskUtilLabel= "%util"
    14   diskUtilCol=14
    15   diskPrefix= "hd|sd"
    16   ;;
    17   *)       echo "not support $OS operating system!"
    18   exit 1;
    19   ;;
    20   esac
    21  
    22 eval " cat ${inIostatFile}. tail |  egrep '${diskPrefix}' \
    23   |  awk '{ if ( \$${diskUtilCol} * 100 < ${diskUtil} ) \
    24   {\$20 = \"OK\"; print \$1\"\t\"\$${diskUtilCol}\"\t\"\$20 } \
    25   else {\$20 = \ "Alarm\"; print \$1\"\t\"\$${diskUtilCol}\"\t\"\$20 } }'" \
    26   > ${outLogFile}.tmp

    下文脚本给出一个告警触发的例子,如果过高的磁盘利用率计数占总分析行数的比率达到或超出预定的比率,脚本会给 root 用户发一封告警邮件。

    清单 9. IOAnalyzer.sh 触发告警

    01 # ----------------------------------------------------------------
    02   # Send admin an alert if disk utilization counter reach defined
    03   # threshold
    04  
    05 # ----------------------------------------------------------------
    06   Alert= "NO"
    07   for DISK  in ` cut -f1  ${outLogFile}.tmp |  sort -u`
    08   do
    09   numAlarm=` cat ${outLogFile}.tmp |  grep "^$DISK.*Alarm$" | wc -l`
    10   numRecord=` cat ${outLogFile}.tmp |  grep "^$DISK" | wc -l`
    11   rateAlarm=` expr $numAlarm \* 100 / $numRecord`
    12   if [ $rateAlarm - ge $rateGEUtil ]; then
    13   echo "DISK:${DISK}      TIME:` date +%Y%m%d%H%M`  \
    14   RATE:${rateAlarm}      THRESHOLD:${rateGEUtil}" >> ${outAlertFile}.tmp
    15   Alert= "YES"
    16   fi
    17   done
    18   if [ $Alert=  "YES" ]; then
    19   cat ${outAlertFile}.tmp >> ${outAlertFile}
    20   mail -s  "DISK IO Alert"   root@localhost< ${outAlertFile}.tmp
    21   fi

    最后,脚本将分析活动归档,便于下次分析时定位起始行;另外,分析过程中产生的文件将被删除。

    清单 10. IOAnalyzer.sh 记录分析活动日志和清除临时文件

    01 #----------------------------------------------------------------
    02   # Clearup temporary files and logging
    03   # ----------------------------------------------------------------
    04  
    05 echo "IOSTATFILE:${inIostatFile}        TIME:` date +%Y%m%d%H%M`  \
    06   STARTLINE:${StartLineNum}       ENDLINE:${EndLineNum}   ALERT:${Alert}" \
    07  
    08 >> ${outLogFile}
    09  
    10 rm -f ${outLogFile}.tmp
    11   rm -f ${outAlertFile}.tmp
    12   rm -f ${inIostatFile}. tail
    13  
    14 exit 0

    脚本使用示例

    以下为 IOAnalyzer.sh 脚本在 AIX 上使用示例

    1 .后台执行 iostat, 并将输出重定向到文件中

    清单 11. 后台执行 iostat

    # nohup iostat -d 5 > /root/iostat.out &
    (对于 Linux,运行 iostat -d – x 5 > /root/iostat.out &)

    2 .编辑 crontab 文件,每 10 分钟运行一次 IOAnalyzer.sh 脚本,-u 70 –r 80,表示在距上次运行 IOAnalyzer.sh 至今产生的某磁盘的监控记录中的 80% 使用率达到或超过 70%,即发出告警。告警日志和分析日志可通过 IOAnalyzer.sh 的 –l –a 参数指定,本例保持默认值,即在 iostat 的输出文件所在目录产生 iostat.out.log 和 iostat.out.alert 文件。

    清单 12. 编辑 crontab

    # crontab – e
    0,10,20,30,40,50 * * * * /root/IOAnalyzer.sh -i /root /iostat.out -u 70 \
    -r 80>/tmp/iostat.out 2>&1

    3 .用户收到告警邮件,需要进一步查询历史记录时,可查看日志文件

    清单 13. 查看日志文件

    # cat /root/iostat.out.log | more
    IOSTATFILE: /root/iostat.out TIME:200905200255 STARTLINE:7220 ENDLINE:7580  ALARM:YES
    IOSTATFILE: /root/iostat.out  TIME:200905200300  STARTLINE:7581 ENDLINE:7940 ALARM:YES
    IOSTATFILE:/root/iostat.out TIME:200905200305  STARTLINE:7941 ENDLINE:8300 ALARM:YES

    [AIXn01]> cat /root/iostat.out.alert | more
    DISK:hdisk4     TIME:200905200250         RATE:84      THRESHOLD:70
    DISK:hdisk5     TIME:200905200250         RATE:84      THRESHOLD:70
    DISK:hdisk6     TIME:200905200250         RATE:84      THRESHOLD:70


    展开全文
  • linux查看磁盘io的几种方法

    千次阅读 2018-08-21 18:13:20
    linux查看磁盘io的几种方法 怎样才能快速的定位到并发高是由于磁盘io开销大呢?可以通过三种方式:  第一种:用 top 命令 中的cpu 信息观察  Top可以看到的cpu信息有:  Tasks: 29 total, 1 running, 28 ...

    原文:https://www.cnblogs.com/zhuawang/p/5053240.html

    linux查看磁盘io的几种方法

    怎样才能快速的定位到并发高是由于磁盘io开销大呢?可以通过三种方式:

      第一种:用 top 命令 中的cpu 信息观察

      Top可以看到的cpu信息有:

      Tasks: 29 total, 1 running, 28 sleeping, 0 stopped, 0 zombie

      Cpu(s): 0.3% us, 1.0% sy, 0.0% ni, 98.7% id, 0.0% wa, 0.0% hi, 0.0% si

      具体的解释如下:

      Tasks: 29 total 进程总数

      1 running 正在运行的进程数

      28 sleeping 睡眠的进程数

      0 stopped 停止的进程数

      0 zombie 僵尸进程数

      Cpu(s):

      0.3% us 用户空间占用CPU百分比

      1.0% sy 内核空间占用CPU百分比

      0.0% ni 用户进程空间内改变过优先级的进程占用CPU百分比

      98.7% id 空闲CPU百分比

      0.0% wa 等待输入输出的CPU时间百分比

      0.0% hi

      0.0% si

      0.0% wa 的百分比可以大致的体现出当前的磁盘io请求是否频繁。如果 wa的数量比较大,说明等待输入输出的的io比较多。

    展开全文
  • linux disk I/O tuning

    2019-02-20 17:10:44
    linux disk I/O tuning
                    原贴地址:http://www.lupaworld.com/441/viewspace_2106.html
                                                                                                                                                                                                                                                                                                                                                                                                                                                   
                                       

    Option

                                       
                                       

    Explanation

                                       
    linux disk I/O tuning?
     
     
               
                           
                                                   
    preface : tony.liu
                           
    很多时候我们面临系统响应很慢,通常面对自己的电脑我们可以主观的观察disk led是否频闪来判断是否disk I/O 比较的高,然而我的工作通常是面对远在千里的server,想象力通常显的力不从心,如果通过有力的数据来证明linux disk I/O,呵呵,follow me.
                           
     
                           
    Performance Tools: Disk I/O
                           

    This chapter covers performance tools that help you gauge disk I/O subsystem usage. These tools can show which disks or partitions are being used, how much I/O each disk is processing, and how long I/O requests issued to these disks are waiting to be processed.

                           

    After reading this chapter, you should be able to

                           
    •                            

      Determine the amount of total amount and type (read/write) of disk I/O on a system (vmstat).

                                 
    •                            
    •                            

      Determine which devices are servicing most of the disk I/O (vmstat, iostat, sar).

                                 
    •                            
    •                            

      Determine how effectively a particular disk is fielding I/O requests (iostat).

                                 
    •                            
    •                            

      Determine which processes are using a given set of files (lsof).

                                 
    •                        

    Introduction to Disk I/O

                           

    Before diving into performance tools, it is necessary to understand how the Linux disk I/O system is structured. Most modern Linux systems have one or more disk drives. If they are IDE drives, they are usually named hda, hdb, hdc, and so on; whereas SCSI drives are usually named sda, sdb, sdc, and so on. A disk is typically split into multiple partitions, where the name of the partition's device is created by adding the partition number to the end of the base device name. For example, the second partition on the first IDE hard drive in the system is usually labeled /dev/hda2. Each individual partition usually contains either a file system or a swap partition. These partitions are mounted into the Linux root file system, as specified in /etc/fstab. These mounted file systems contain the files that applications read to and write from.

                           

    When an application does a read or write, the Linux kernel may have a copy of the file stored into its cache or buffers and returns the requested information without ever accessing the disk. If the Linux kernel does not have a copy of the data stored in memory, however, it adds a request to the disk's I/O queue. If the Linux kernel notices that multiple requests are asking for contiguous locations on the disk, it merges them into a single big request. This merging increases overall disk performance by eliminating the seek time for the second request. When the request has been placed in the disk queue, if the disk is not currently busy, it starts to service the I/O request. If the disk is busy, the request waits in the queue until the drive is available, and then it is serviced.

                           

     

                           

    Disk I/O Performance Tools

                           

    This section examines the various disk I/O performance tools that enable you to investigate how a given application is using the disk I/O subsystem, including how heavily each disk is being used, how well the kernel's disk cache is working, and which files a particular application has "open."

                                                   

    6.2.1. vmstat (ii)

                           

    As you saw in Chapter 2, "Performance Tools: System CPU," vmstat is a great tool to give an overall view of how the system is performing. In addition to CPU and memory statistics, vmstat can provide a system-wide view of I/O performance.

                                                   
    6.2.1.1 Disk I/O Performance-Related Options and Outputs
                           

    While using vmstat to retrieve disk I/O statistics from the system, you must invoke it as follows:

                           
                            


     

                           

    Table 6-1 describes the other command-line parameters that influence the disk I/O statistics that vmstat will display.

                                                   

                           

    Table 6-1. vmstat Command-Line Options
                               
                                       

    -D

                                       
                                       

    This displays Linux I/O subsystem total statistics. This option can give you a good idea of how your I/O subsystem is being used, but it won't give statistics on individual disks. The statistics given are the totals since system boot, rather than just those that occurred between this sample and the previous sample.

                                       
                                       

    -d

                                       
                                       

    This option displays individual disk statistics at a rate of one sample per interval. The statistics are the totals since system boot, rather than just those that occurred between this sample and the previous sample.

                                       
                                       

    -p partition

                                       
                                       

    This displays performance statistics about the given partition at a rate of one sample per interval. The statistics are the totals since system boot, rather than just those that occurred between this sample and the previous sample.

                                       
                                       

    interval

                                       
                                       

    The length of time between samples.

                                       
                                       

    count

                                       
                                       

    The total number of samples to take.

                                       


     

                           

    If you run vmstat without any parameters other than [interval] and [count], it shows you the default output. This output contains three columns relevant to disk I/O performance: bo, bi, and wa. These statistics are described in Table 6-2.

                                                   

                           

    Table 6-2. vmstat I/O Statistics
                                                                   

    Statistic

                                                                                                               

    Explanation

                                                                                                           

    bo

                                                                                                               

    This indicates the number of total blocks written to disk in the previous interval. (In vmstat, block size for a disk is typically 1,024 bytes.)

                                                                                                           

    bi

                                                                                                               

    This shows the number of blocks read from the disk in the previous interval. (In vmstat, block size for a disk is typically 1,024 bytes.)

                                                                                                           

    wa

                                                                                                               

    This indicates the amount of CPU time spent waiting for I/O to complete. The rate of disk blocks written per second.

                                                                       


     

                           

    When running with the -D mode, vmstat provides statistical information about the system's disk I/O system as a whole. Information about these statistics is provided in Table 6-3. (Note that more information about these statistics is available in the Linux kernel source package, under Documentation/iostats.txt.)

                                                   

                           

    Table 6-3. vmstat Disk I/O Statistics
                                                                   

    Statistic

                                                                                                               

    Explanation

                                                                                                           

    disks

                                                                                                               

    The total number of disks in the system.

                                                                                                           

    partitions

                                                                                                               

    The total number of partitions in the system.

                                                                                                           

    total reads

                                                                                                               

    The total number of reads that have been requested.

                                                                                                           

    merged reads

                                                                                                               

    The total number of times that different reads to adjacent locations on the disk were merged to improve performance.

                                                                                                           

    read sectors

                                                                                                               

    The total number of sectors read from disk. (A sector is usually 512 bytes.)

                                                                                                           

    milli reading

                                                                                                               

    The amount of time (in ms) spent reading from the disk.

                                                                                                           

    writes

                                                                                                               

    The total number of writes that have been requested.

                                                                                                           

    merged writes

                                                                                                               

    The total number of times that different writes to adjacent locations on the disk were merged to improve performance.

                                                                                                           

    written sectors

                                                                                                               

    The total number of sectors written to disk. (A sector is usually 512 bytes.)

                                                                                                           

    milli writing

                                                                                                               

    The amount of time (in ms) spent writing to the disk.

                                                                                                           

    inprogress IO

                                                                                                               

    The total number of I/O that are currently in progress. Note that there is a bug in recent versions (v3.2) of vmstat in which this is incorrectly divided by 1,000, which almost always yields a 0.

                                                                                                           

    milli spent IO

                                                                                                               

    This is the number of milliseconds spent waiting for I/O to complete. Note that there is a bug in recent versions (v3.2) of vmstat in which this is the number of seconds spent on I/O rather than milliseconds.

                                                                       


     

                           

    The -d option of vmstat displays I/O statistics of each individual disk. These statistics are similar to those of the -D option and are described in Table 6-4.

                                                   

                           

    Table 6-4. vmstat disk I/O Statistics
                                                                   

    Statistic

                                                                                                               

    Explanation

                                                                                                           

    reads: total

                                                                                                               

    The total number of reads that have been requested.

                                                                                                           

    reads: merged

                                                                                                               

    The total number of times that different reads to adjacent locations on the disk were merged to improve performance.

                                                                                                           

    reads: sectors

                                                                                                               

    The total number of sectors read from disk.

                                                                                                           

    reads: ms

                                                                                                               

    The amount of time (in ms) spent reading from the disk.

                                                                                                           

    writes: total

                                                                                                               

    The total number of writes that have been requested for this disk.

                                                                                                           

    writes: merged

                                                                                                               

    The total number of times that different writes to adjacent locations on the disk were merged to improve performance.

                                                                                                           

    writes: sectors

                                                                                                               

    The total number of sectors written to disk. (A sector is usually 512 bytes.)

                                                                                                           

    writes: ms

                                                                                                               

    The amount of time (in ms) spent writing to the disk.

                                                                                                           

    IO: cur

                                                                                                               

    The total number of I/O that are currently in progress. Note that there is a bug in recent versions of vmstat in which this is incorrectly divided by 1,000, which almost always yields a 0.

                                                                                                           

    IO: s

                                                                                                               

    This is the number of seconds spent waiting for I/O to complete.

                                                                       


     

                           

    Finally, when asked to provide partition-specific statistics, vmstat displays those listed in Table 6-5.

                                                   

                           

    Table 6-5. vmstat partition I/O Statistics
                                                                   

    Statistic

                                                                                                               

    Explanation

                                                                                                           

    reads

                                                                                                               

    The total number of reads that have been requested for this partition.

                                                                                                           

    read sectors

                                                                                                               

    The total number of sectors read from this partition.

                                                                                                           

    writes

                                                                                                               

    The total number of writes that resulted in I/O for this partition.

                                                                                                           

    requested writes

                                                                                                               

    The total number of reads that have been requested for this partition.

                                                                       


     

                           

    The default vmstat output provides a coarse indication of system disk I/O, but a good level. The options provided by vmstat enable you to reveal more details about which device is responsible for the I/O. The primary advantage of vmstat over other I/O tools is that it is present on almost every Linux distribution.

                                                   
    6.2.1.2 Example Usage
                           

    The number of I/O statistics that vmstat can present to the Linux user has been growing with recent releases of vmstat. The examples shown in this section rely on vmstat version 3.2.0 or greater. In addition, the extended disk statistics provided by vmstat are only available on Linux systems with a kernel version greater than 2.5.70.

                           

    In the first example, shown in Listing 6.1, we are just invoking vmstat for three samples with an interval of 1 second. vmstat outputs the system-wide performance overview that we saw in Chapter 2.

                                                   
    Listing 6.1.
                           
                            


     

                           

    Listing 6.1 shows that during one of the samples, the system read 24,448 disk blocks. As mentioned previously, the block size for a disk is 1,024 bytes, so this means that the system is reading in data at about 23MB per second. We can also see that during this sample, the CPU was spending a significant portion of time waiting for I/O to complete. The CPU waits on I/O 63 percent of the time during the sample in which the disk was reading at ~23MB per second, and it waits on I/O 49 percent for the next sample, in which the disk was reading at ~19MB per second.

                           

    Next, in Listing 6.2, we ask vmstat to provide information about the I/O subsystem's performance since system boot.

                                                   
    Listing 6.2.
                           
                            


     

                           

    In Listing 6.2, vmstat provides I/O statistic totals for all the disk devices in the system. As mentioned previously, when reading and writing to a disk, the Linux kernel tries to merge requests for contiguous regions on the disk for a performance increase; vmstat reports these events as merged reads and merged writes. In this example, a large number of the reads issued to the system were merged before they were issued to the device. Although there were ~640,000 merged reads, only ~53,000 read commands were actually issued to the drives. The output also tells us that a total of 4,787,741 sectors have been read from the disk, and that since system boot, 343,552ms (or 344 seconds) were spent reading from the disk. The same statistics are available for write performance. This view of I/O statistics is a good view of the overall I/O subsystem's performance.

                           

    Although the previous example displayed I/O statistics for the entire system, the following example in Listing 6.3 shows the statistics broken down for each individual disk.

                                                   
    Listing 6.3.
                           
                            


     

                           

    Listing 6.4 shows that 60 (19,059 – 18,999) reads and 94 writes (24,795 – 24,795) have been issued to partition hde3. This view can prove particularly useful if you are trying to determine which partition of a disk is seeing the most usage.

                                                   
    Listing 6.4.
                           
                            


     

                           

    Although vmstat provides statistics about individual disks/partitions, it only provides totals rather than the rate of change during the sample. This can make it difficult to eyeball which device's statistics have changed significantly from sample to sample.

                                                   

    6.2.2. iostat

                           

    iostat is like vmstat, but it is a tool dedicated to the display of the disk I/O subsystem statistics. iostat provides a per-device and per-partition breakdown of how many blocks are written to and from a particular disk. (Blocks in iostat are usually sized at 512 bytes.) In addition, iostat can provide extensive information about how a disk is being utilized, as well as how long Linux spends waiting to submit requests to the disk.

                                                   
    6.2.2.1 Disk I/O Performance-Related Options and Outputs
                           

    iostat is invoked using the following command line:

                           
                            


     

                           

    Much like vmstat, iostat can display performance statistics at regular intervals. Different options modify the statistics that iostat displays. These options are described in Table 6-6.

                                                   

                           

    Table 6-6. iostat Command-Line Options
                                                                   

    Option

                                                                                                               

    Explanation

                                                                                                           

    -d

                                                                                                               

    This displays only information about disk I/O rather than the default display, which includes information about CPU usage as well.

                                                                                                           

    -k

                                                                                                               

    This shows statistics in kilobytes rather than blocks.

                                                                                                           

    -x

                                                                                                               

    This shows extended-performance I/O statistics.

                                                                                                           

    device

                                                                                                               

    If a device is specified, iostat shows only information about that device.

                                                                                                           

    interval

                                                                                                               

    The length of time between samples.

                                                                                                           

    count

                                                                                                               

    The total number of samples to take.

                                                                       


     

                           

    The default output of iostat displays the performance statistics described in Table 6-7.

                                                   

                           

    Table 6-7. iostat Device Statistics
                                                                   

    Statistic

                                                                                                               

    Explanation

                                                                                                           

    tps

                                                                                                               

    Transfers per second. This is the number of reads and writes to the drive/partition per second.

                                                                                                           

    Blk_read/s

                                                                                                               

    The rate of disk blocks read per second.

                                                                                                           

    Blk_wrtn/s

                                                                                                               

    The rate of disk blocks written per second.

                                                                                                           

    Blk_read

                                                                                                               

    The total number of blocks read during the interval.

                                                                                                           

    Blk_wrtn

                                                                                                               

    The total number of blocks written during the interval.

                                                                       


     

                           

    When you invoke iostat with the -x parameter, it displays extended statistics about the disk I/O subsystem. These extended statistics are described in Table 6-8.

                                                   

                           

    Table 6-8. iostat Extended Disk Statistics
                                                                   

    Statistic

                                                                                                               

    Explanation

                                                                                                           

    rrqm/s

                                                                                                               

    The number of reads merged before they were issued to the disk.

                                                                                                           

    wrqm/s

                                                                                                               

    The number of writes merged before they were issued to the disk.

                                                                                                           

    r/s

                                                                                                               

    The number of reads issued to the disk per second.

                                                                                                           

    w/s

                                                                                                               

    The number of writes issued to the disk per second.

                                                                                                           

    rsec/s

                                                                                                               

    Disk sectors read per second.

                                                                                                           

    wsec/s

                                                                                                               

    Disk sectors written per second.

                                                                                                           

    rkB/s

                                                                                                               

    Kilobytes read from disk per second.

                                                                                                           

    wkB/s

                                                                                                               

    Kilobytes written to disk per second.

                                                                                                           

    avgrq-sz

                                                                                                               

    The average size (in sectors) of disk requests.

                                                                                                           

    avgqu-sz

                                                                                                               

    The average size of the disk request queue.

                                                                                                           

    await

                                                                                                               

    The average time (in ms) for a request to be completely serviced. This average includes the time that the request was waiting in the disk's queue plus the amount of time it was serviced by the disk.

                                                                                                           

    svctm

                                                                                                               

    The average service time (in ms) for requests submitted to the disk. This indicates how long on average the disk took to complete a request. Unlike await, it does not include the amount of time spent waiting in the queue.

                                                                       


     

                           

    iostat is a helpful utility, providing the most complete view of disk I/O performance statistics of any that I have found so far. Although vmstat is present everywhere and provides some basic statistics, iostat is far more complete. If it is available and installed on your system, iostat should be the first tool to turn to when a system has a disk I/O performance problem.

                                                   
    6.2.2.2 Example Usage
                           

    Listing 6.5 shows an example iostat run while a disk benchmark is writing a test file to the file system on the /dev/hda2 partition. The first sample iostat displays is the total system average since system boot time. The second sample (and any that would follow) is the statistics from each 1-second interval.

                                                   
    Listing 6.5.
                           
                            


     

                           

    One interesting note in the preceding example is that /dev/hda3 had a small amount of activity. In the system being tested, /dev/hda3 is a swap partition. Any activity recorded from this partition is caused by the kernel swapping memory to disk. In this way, iostat provides an indirect method to determine how much disk I/O in the system is the result of swapping.

                           

    Listing 6.6 shows the extended output of iostat.

                                                   
    Listing 6.6.
                           
                            


     

                           

    In Listing 6.6, you can see that the average queue size is pretty high (~237 to 538) and, as a result, the amount of time that a request must wait (~422.44ms to 538.60ms) is much greater than the amount of time it takes to service the request (7.63ms to 11.90ms). These high average service times, along with the fact that the utilization is 100 percent, show that the disk is completely saturated.

                           

    The extended iostat output provides so many statistics that it only fits on a single line in a very wide terminal. However, this information is nearly all that you need to identify a particular disk as a bottleneck.

                                                   

    6.2.3. sar

                           

    As discussed in Chapter 2, "Performance Tools: System CPU," sar can collect the performance statistics of many different areas of the Linux system. In addition to CPU and memory statistics, it can collect information about the disk I/O subsystem.

                                                   
    6.2.3.1 Disk I/O Performance-Related Options and Outputs
                           

    When using sar to monitor disk I/O statistics, you can invoke it with the following command line:

                           
                            


     

                           

    Typically, sar displays information about the CPU usage in a system; to display disk usage statistics instead, you must use the -d option. sar can only display disk I/O statistics with a kernel version higher than 2.5.70. The statistics that it displays are described in Table 6-9.

                                                   

                           

    Table 6-9. sar Device Statistics
                                                                   

    Statistic

                                                                                                               

    Explanation

                                                                                                           

    tps

                                                                                                               

    Transfers per second. This is the number of reads and writes to the drive/partition per second.

                                                                                                           

    rd_sec/s

                                                                                                               

    Number of disk sectors read per second.

                                                                                                           

    wr_sec/s

                                                                                                               

    Number of disk sectors written per second.

                                                                       


     

                           

    The number of sectors is taken directly from the kernel, and although it is possible for it to vary, the size is usually 512 bytes.

                                                   
    6.2.3.2 Example Usage
                           

    In Listing 6.7, sar is used to collect information about the I/O of the devices on the system. sar lists the devices by their major and minor number rather than their names.

                                                   
    Listing 6.7.
                           
                            


     

                           

    sar has a limited number of disk I/O statistics when compared to iostat. However, the capability of sar to simultaneously record many different types of statistics may make up for these shortcomings.

                                                   

    6.2.4. lsof (List Open Files)

                           

    lsof provides a way to determine which processes have a particular file open. In addition to tracking down the user of a single file, lsof can display the processes using the files in a particular directory. It can also recursively search through an entire directory tree and list the processes using files in that directory tree. lsof can prove helpful when narrowing down which applications are generating I/O.

                                                   
    6.2.4.1 Disk I/O Performance-Related Options and Outputs
                           

    You can invoke lsof with the following command line to investigate which files processes have open:

                           
                            


     

                           

    Typically, lsof displays which processes are using a given file. However, by using the +d and +D options, it is possible for lsof to display this information for more than one file. Table 6-10 describes the command-line options of lsof that prove helpful when tracking down an I/O performance problem.

                                                   

                           

    Table 6-10. lsof Command-Line Options
                                                                   

    Option

                                                                                                               

    Explanation

                                                                                                           

    -r delay

                                                                                                               

    This causes lsof to output statistics every delay seconds.

                                                                                                           

    +D directory

                                                                                                               

    This causes lsof to recursively search all the files in the given directory and report on which processes are using them.

                                                                                                           

    +d directory

                                                                                                               

    This causes lsof to report on which processes are using the files in the given directory.

                                                                       


     

                           

    lsof displays the statistics described in Table 6-11 when showing which processes are using the specified files.

                                                   

                           

    Table 6-11. lsof File Statistics
                                                                   

    Statistic

                                                                                                               

    Explanation

                                                                                                           

    COMMAND

                                                                                                               

    The name of the command that has the file open.

                                                                                                           

    PID

                                                                                                               

    The PID of the command that has the file open.

                                                                                                           

    USER

                                                                                                               

    The user who has the file open.

                                                                                                           

    FD

                                                                                                               

    The file descriptor of the file, or tex for a executable, mem for a memory mapped file.

                                                                                                           

    TYPE

                                                                                                               

    The type of file. REG for a regular file.

                                                                                                           

    DEVICE

                                                                                                               

    Device number in major, minor number.

                                                                                                           

    SIZE

                                                                                                               

    The size of the file.

                                                                                                           

    NODE

                                                                                                               

    The inode of the file.

                                                                       


     

                           

    Although lsof does not show the amount and type of file access that a particular process is doing, it at least displays which processes are using a particular file.

                                                   
    6.2.4.2 Example Usage
                           

    Listing 6.8 shows lsof being run on the /usr/bin directory. This run shows which processes are accessing all of the files in /usr/bin.

                                                   
    Listing 6.8.
                           
                            


     

                           

    In particular, we can see that process 3807 is using the file /usr/bin/gnome-terminal. This file is an executable, as indicated by the txt in the FD column, and the name of the command that is using it is gnome-terminal. This makes sense; the process that is running gnome-terminal must therefore have the executable open. One interesting thing to note is that this file is on the device 3,2, which corresponds to /dev/hda2. (You can figure out the device number for all the system devices by executing ls -la /dev and looking at the output field that normally displays size.) Knowing on which device a file is located can help if you know that a particular device is the source of an I/O bottleneck. lsof provides the unique ability to trace an open file descriptor back to individual processes; although it does not show which processes are using a significant amount of I/O, it does provide a starting point.

                           

     

                           

    6.3. What's Missing?

                           

    All the disk I/O tools on Linux provide information about the utilization of a particular disk or partition. Unfortunately, after you determine that a particular disk is a bottleneck, there are no tools that enable you to figure out which process is causing all the I/O traffic.

                           

    Usually a system administrator has a good idea about what application uses the disk, but not always. Many times, for example, I have been using my Linux system when the disks started grinding for apparently no reason. I can usually run top and look for a process that might be causing the problem. By eliminating processes that I believe are not doing I/O, I can usually find the culprit. However, this requires knowledge of what the various applications are supposed to do. It is also error prone, because the guess about which processes are not causing the problem might be wrong. In addition, for a system with many users or many running applications, it is not always practical or easy to determine which application might be causing the problem. Other UNIXes support the inblk and oublk parameters to ps, which show you the amount of disk I/O issued on behalf of a particular process. Currently, the Linux kernel does not track the I/O of a process, so the ps tool has no way to gather this information.

                           

    You can use lsof to determine which processes are accessing files on a particular partition. After you list all PIDs accessing the files, you can then attach to each of the PIDs with strace and figure out which one is doing a significant amount of I/O. Although this method works, it is really a Band-Aid solution, because the number of processes accessing a partition could be large and it is time-consuming to attach and analyze the system calls of each process. This may also miss short-lived processes, and may unacceptably slow down processes when they are being traced.

                           

    This is an area where the Linux kernel could be improved. The ability to quickly track which processes are generating I/O would allow for much quicker diagnosis of I/O performance-related problems.

                           
                               
                                                                                                                

    再分享一下我老师大神的人工智能教程吧。零基础!通俗易懂!风趣幽默!还带黄段子!希望你也加入到我们人工智能的队伍中来!https://blog.csdn.net/jiangjunshow

    展开全文
  • linux 磁盘io监控

    万次阅读 2018-04-14 14:23:44
    有的时候我们需要更进一步了解磁盘io的使用情况,那么本文就是重点讲解一下如何查看linux的磁盘io信息的。 一、iostat: 1、基本用法: $iostat -d -k 1 101)参数 -d 表示,显示设备(磁盘)使用状态;-k某些...
  • 而system.diskio.iostat.busy 为: 此外该值最高曾到达两位数: scaled_float 类型解释为: 因此可判断监控告警cpu、filesystem、memeory相关字段值0.XX表示百分比为XX%,而system.diskio.iostat.busy 字段0.XX...
  • zabbix disk_io

    2019-06-17 20:39:00
    zabbix自带的“Template OS Linux”模板支持监控已挂载的磁盘空间利用率,是利用LLD(Low-level discovery)实现的,却没有对磁盘IO监控。本篇文章就介绍利用zabbix LLD功能监控磁盘IO的方法。思路:首先创建...
  • linux 磁盘IO

    2018-01-02 14:51:19
  • Linux 平台上使用的net-snmp 提供多种Agent 编写方式,较复杂的,可用C 调用其提供的API 接口来实现简单的情况,可利用shell 脚本,借助net-snmp 提供的接口脚本来实现。这些,可参 考/etc/snmp/snmpd.conf 中提供的...
  • linux下图形界面测试磁盘IO性能的源码,直接编译即可运行
  • Linux 磁盘IO性能测试工具:FIO

    千次阅读 2017-03-03 10:23:43
    1. FIO安装CentOS下:$ yum install fio2. FIO简介FIO是测试IOPS的非常好的工具,用来对硬件进行压力...FIO输出结果详细解释可参见:https://tobert.github.io/post/2014-04-17-fio-output-explained.html 3. FIO参
  • Linux中的IO优先级

    2019-10-13 17:20:59
    参考:https://blog.csdn.net/junmuzi/article/details/80189504
  • linux的cpu和磁盘io优先级设置

    千次阅读 2018-05-04 02:36:50
    Original url:https://www.52os.net/articles/reduce-cpu-io-priority.html在服务器上常常会运行一些备份脚本,当备份的文件比较...通常linux下限制cpu使用有三种方法:nice/renice : 调整进程使用cpu的优先级cpuli...
  • Linux系统 磁盘IO过高排查总结

    千次阅读 2019-07-25 03:34:23
    最近做的一个电商网站因为磁盘 I/O 过高导致访问速度奇慢,问题存在两个月有余未得到解决办法。此次排查原因的经验可以作下次问题的...iotop、pt-ioprofile等工具,都是有助于分析问题所在的应用。借助这些工具,...
  • ARC0 Encountered disk IO error 19502

    千次阅读 2013-03-03 13:24:11
    一个二级客户的一个备份库,所以平时没怎么关注,今天去处理ntpdbug的时候想到顺便看看oracle运行情况,结果一看告警日志一...ARC0: Encountered disk I/O error 19502 Fri Jun 22 22:00:22 CST 2012 ARC0: Closing lo
  • zabbix添加对linux主机磁盘io监控

    万次阅读 2015-12-04 16:49:28
    1、登录被监控机器上传扫描磁盘分区脚本disk_scan.sh 脚本内容如下: #!/bin/bash diskarray=(`cat /proc/diskstats |grep -E"\bsd[abcdefg]\b|\bxvd[abcdefg]\b"|grep -i "\b$1\b"|awk'{print $3}'|sort|uniq 2>/...
  • LINUX系统中,如果有大量读请求,默认的请求队列或许应付不过来,我们可以 动态调整请求队列数来提高效率,默认的请求队列数存放在/sys/block/xvda/queue/nr_requests 文件中,注意:/sys/block/xvda ,这里 xvda...
  • linux IO 性能优化

    万次阅读 2013-03-13 09:29:31
    由于各种的I/O负载情形各异,Linux系统中文件系统的缺省配置一般来说都比较中庸,强调普遍适用性。然而在特定应用下,这种配置往往在I/O性能方面不能达到最优。因此,如果应用对I/O性能要求较高,除了采用性能更高的...
  • 原文地址: https://www.0xffffff.org/2017/05/01/41-linux-io/ 写在前面 在开始正式的讨论前,我先抛出几个问题: 谈到磁盘时,常说的HDD磁盘和SSD磁盘最大的区别是什么?这些差异会影响我们的系统设计吗? ...
  • 最近给客户交付新系统,然后客户反应服务器硬盘使用的增量有点异常,5天用掉了3个G,由于我们做的是...最后打算监控到底是哪个进程在进行大量的IO操作,决定把这个过程中用到的一些指令记录下来。 查看线程IO状态 pi...
1 2 3 4 5 ... 20
收藏数 43,852
精华内容 17,540
热门标签
关键字:

disk io linux