精华内容
下载资源
问答
  • OS:oracle linux6.8 存储:HP EVA8100(存储过老,不支持软件安装) 1、查看多路径软件包是否安装 [root@node1 ~]# rpm -qa|grep device device-ma...

    OS:oracle linux6.8
    存储:HP EVA8100(存储过老,不支持软件安装)
    1、查看多路径软件包是否安装
    [root@node1 ~]# rpm -qa|grep device
    device-mapper-event-libs-1.02.117-7.el6.x86_64
    device-mapper-libs-1.02.117-7.el6.x86_64
    device-mapper-event-1.02.117-7.el6.x86_64
    device-mapper-persistent-data-0.6.2-0.1.rc7.el6.x86_64
    device-mapper-multipath-libs-0.4.9-93.0.1.el6.x86_64
    libimobiledevice-0.9.7-4.el6.x86_64
    device-mapper-1.02.117-7.el6.x86_64
    PackageKit-device-rebind-0.5.8-26.0.1.el6.x86_64
    device-mapper-multipath-0.4.9-93.0.1.el6.x86_64 ?   ---如果未安装则找到系统包进行安装
    设置开机自启动,检查安装包是否正常
    chkconfig –level 2345 multipathd on
    lsmod |grep dm_multipath
    2.配置multipathd 使其正常工作,编辑/etc/multipath.conf,开放如下内容:

    defaults {
           udev_dir                /dev
           polling_interval        5
           path_grouping_policy    failover
           getuid_callout          "/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/%n"
           prio                    const
           path_checker            directio
           rr_min_io               1000
           rr_weight               uniform
           failback                manual
           no_path_retry           fail
           user_friendly_names     yes
    }
    blacklist {
            devnode "^hd[a-z]"
            devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
    }
    multipaths {
            multipath {
                    wwid                    3600508b4000c3bcf0000e00005630000
                       alias                   asm-diskb
        
    #               path_grouping_policy    multibus
    #               path_checker            readsector0
    #               path_selector           "round-robin 0"
    #               failback                manual
    #               rr_weight               priorities
    #               no_path_retry           5
    #       }
            multipath {
                    wwid                    3600508b4000c3bcf0000e00005660000
                    alias                   asm-diskc
    devices {
    device {
    vendor "iSCSI-Enterprise"
    product "Virtual disk"
    path_grouping_policy multibus
    getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
    path_checker readsector0
    path_selector "round-robin 0"
    }
    }

    如果没有multipath文件,可以执行下列命令试试
    [root@node1 multipath]# mpathconf --enable --with_multipathd y
    [root@node1 multipath]# multipath -v$ |grep uid
    [root@node1 multipath]# multipath -v3|grep 3600

    3、重启multipath服务
    [root@node1 etc]# service multipathd restart
    ok
    Stopping multipathd daemon:                                [  OK  ]
    Starting multipathd daemon:                                [  OK  ]
    4.[root@node1 mapper]# multipath -ll
    asm-diskb (3600508b4000c3bcf0000e00005630000) dm-0 HP,HSV210
    size=5.0G features='1 queue_if_no_path' hwhandler='0' wp=rw
    |-+- policy='round-robin 0' prio=50 status=active
    | |- 1:0:0:1 sdb 8:16  active ready running
    | |- 2:0:0:1 sdj 8:144 active ready running
    | |- 1:0:3:1 sdh 8:112 active ready running
    | `- 2:0:3:1 sdp 8:240 active ready running
    `-+- policy='round-robin 0' prio=10 status=enabled
      |- 1:0:1:1 sdd 8:48  active ready running
      |- 2:0:1:1 sdl 8:176 active ready running
      |- 1:0:2:1 sdf 8:80  active ready running
      `- 2:0:2:1 sdn 8:208 active ready running
    asm-diskc (3600508b4000c3bcf0000e00005660000) dm-1 HP,HSV210
    size=300G features='1 queue_if_no_path' hwhandler='0' wp=rw
    |-+- policy='round-robin 0' prio=50 status=active
    | `- 1:0:1:2 sde 8:64  active ready running
    |-+- policy='round-robin 0' prio=50 status=enabled
    | `- 1:0:2:2 sdg 8:96  active ready running
    |-+- policy='round-robin 0' prio=50 status=enabled
    | `- 2:0:1:2 sdm 8:192 active ready running
    |-+- policy='round-robin 0' prio=50 status=enabled
    | `- 2:0:2:2 sdo 8:224 active ready running
    |-+- policy='round-robin 0' prio=10 status=enabled
    | `- 1:0:0:2 sdc 8:32  active ready running
    |-+- policy='round-robin 0' prio=10 status=enabled
    | `- 1:0:3:2 sdi 8:128 active ready running
    |-+- policy='round-robin 0' prio=10 status=enabled
    | `- 2:0:0:2 sdk 8:160 active ready running
    `-+- policy='round-robin 0' prio=10 status=enabled
      `- 2:0:3:2 sdq 65:0  active ready running
     4、查看ASM磁盘组
    [root@node1 mapper]# ls -l /dev/mapper/

    total 0
    lrwxrwxrwx. 1 root root       7 Dec 29 15:52 asm-diskb -> ../dm-0
    lrwxrwxrwx. 1 root root       7 Dec 29 15:52 asm-diskc -> ../dm-1
    crw-rw----. 1 root root 10, 236 Dec 29 09:54 control
    [root@node1 mapper]# ls
    asm-diskb  asm-diskc  control
    5、udev绑定磁盘
    [root@node1 ~]# cat /etc/udev/rules.d/99-oracle-asmdevices.rules 

    ENV{DM_NAME}=="asm-diska", OWNER:="grid", GROUP:="asmadmin", MODE:="660"
    ENV{DM_NAME}=="asm-diskb", OWNER:="grid", GROUP:="asmadmin", MODE:="660"
    ENV{DM_NAME}=="asm-diskc", OWNER:="grid", GROUP:="asmadmin", MODE:="660"

    [root@node1 rules.d]# start_udev
    Starting udev:                                             [  OK  ]

    [root@node1 rules.d]# ls -l /dev/mapper/
    total 0
    lrwxrwxrwx 1 root root       7 Jul  1 15:18 asm-diska -> ../dm-2
    lrwxrwxrwx 1 root root       7 Jul  1 15:18 asm-diskb -> ../dm-3
    lrwxrwxrwx 1 root root       7 Jul  1 15:18 asm-diskc -> ../dm-4
    crw-rw---- 1 root root 10, 236 Jul  1 15:18 control
    lrwxrwxrwx 1 root root       7 Jul  1 15:18 vg_oel-lv_root -> ../dm-0
    lrwxrwxrwx 1 root root       7 Jul  1 15:18 vg_oel-lv_swap -> ../dm-1
    [root@node1 rules.d]# ls -l /dev/dm-*
    brw-rw---- 1 root disk     252, 0 Jul  1 15:18 /dev/dm-0
    brw-rw---- 1 root disk     252, 1 Jul  1 15:18 /dev/dm-1
    brw-rw---- 1 grid asmadmin 252, 2 Jul  1 15:18 /dev/dm-2
    brw-rw---- 1 grid asmadmin 252, 3 Jul  1 15:18 /dev/dm-3
    brw-rw---- 1 grid asmadmin 252, 4 Jul  1 15:18 /dev/dm-4

    在创建asm磁盘组的时候,路径选择/dev/mapper/*,可自动识别到聚合后的磁盘,虽然软连接的权限还是root,但实际磁盘已改变,asm可用。
    绑定磁盘还有一种方式:

    [root@pdmdb1 ~]# cat /etc/rc.local

    #!/bin/sh

    #

    # This script will be executed *after* all the other init scripts.

    # You can put your own initialization stuff in here if you don't

    # want to do the full Sys V style init stuff.

    touch /var/lock/subsys/local

    chown grid:asmadmin /dev/dm-0

    chown grid:asmadmin /dev/dm-1

    chmod 660 /dev/dm-0

    chmod 660 /dev/dm-1
    --配置别名已经在multipath.conf里面配置过了




    来自 “ ITPUB博客 ” ,链接:http://blog.itpub.net/29618264/viewspace-2131954/,如需转载,请注明出处,否则将追究法律责任。

    转载于:http://blog.itpub.net/29618264/viewspace-2131954/

    展开全文
  • Linux多路径multipath配置

    万次阅读 2017-09-20 12:41:13
    Linux多路径multipath配置 一、multipath在redhat 6.2中的基本配置: 1. 通过命令:lsmod |grep dm_multipath 检查是否正常安装成功。如果没有输出说明没有安装那么通过yum功能安装一下软件包:yum –y ...

    Linux下多路径multipath配置

    一、multipath在redhat 6.2中的基本配置:

    1. 通过命令:lsmod |grep dm_multipath  检查是否正常安装成功。如果没有输出说明没有安装那么通过yum功能安装一下软件包:yum –y install device-mapper device-mapper-multipath

    接着通过命令:multipath –ll 查看多路径状态查看模块是否加载成功

    [root@liujing ~]#  multipath –ll   查看多路径状态

    Mar 10 19:18:28 | /etc/multipath.conf does not exist, blacklisting all devices.

    Mar 10 19:18:28 | A sample multipath.conf file is located at

    Mar 10 19:18:28 | /usr/share/doc/device-mapper-multipath-0.4.9/multipath.conf

    Mar 10 19:18:28 | You can run /sbin/mpathconf to create or modify /etc/multipath.conf

    Mar 10 19:18:28 | DM multipath kernel driver not loaded    ----DM模块没有加载

    如果模块没有加载成功请使用下列命初始化DM,或重启系统
    ---Use the following commands to initialize and start DM for the first time:
    # modprobe dm-multipath
    # modprobe dm-round-robin
    # service multipathd start
    # multipath –v2

    初始化完了之后再通过multipath -ll命令查看是否加载成功

    [root@liujing ~]#  multipath -ll

    Mar 10 19:21:14 | /etc/multipath.conf does not exist, blacklisting all devices.

    Mar 10 19:21:14 | A sample multipath.conf file is located at

    Mar 10 19:21:14 | /usr/share/doc/device-mapper-multipath-0.4.9/multipath.conf

    Mar 10 19:21:14 | You can run /sbin/mpathconf to create or modify /etc/multipath.conf

    DM multipath kernel driver not loaded    ----这个提示没了说明DM模块已加载成功。

    从上面的提示可以看到,DM模块是成功加载,但是/etc/下没有multipath.conf 配置文件,下一步介绍如何配置multipath.conf 文件。

    2. 配置multipath:

    通过vi命令创建一个Multipath的配置文件路径是/etc/multipath.conf ,在配置文件中添加multipath正常工作的最简配置如下:

    vi /etc/multipath.conf

    blacklist {

    devnode "^sda"

    }

    defaults {

    user_friendly_names yes

    path_grouping_policy multibus

    failback immediate

    no_path_retry fail

    }

    编辑完成后保存配置,同时通过命令:

    # /etc/init.d/multipathd start #开启mulitipath服务

    如果出现无法开启服务的情况,没有提示OK的话如下:

    [root@liujing mapper]# service multipathd start

    Starting multipathd daemon:     没有提示OK

    重新开关一下服务就可以解决了。

    [root@liujing mapper]# /etc/init.d/multipathd stop

    Stopping multipathd daemon:                                [  OK  ]

    [root@localhost mapper]# /etc/init.d/multipathd start

    Starting multipathd daemon:                                [  OK  ]  -----提示OK 正常开启服务

    通过命令查看:

    [root@liujing mapper]# multipath -ll

    mpatha (360a9800064665072443469563477396c) dm-0 NETAPP,LUN    ----创建了一个lun

    size=3.5G features='0' hwhandler='0' wp=rw

    `-+- policy='round-robin 0' prio=4 status=active

    |- 1:0:0:0 sdb 8:16 active ready  running   ----多路径下的两个盘符sdb和sde.

    `- 2:0:0:0 sde 8:64 active ready  running

    目录/dev/mapper/   下多了两个文件夹mpatha 和mpathap1。

    [root@liujing mapper]# cd /dev/mapper/

    [root@liujing mapper]# ls

    control  mpatha  mpathap1

    同时fdisk –l的命令下也多了两个设备标识:

    没有配置多路径时:

    [root@liujing~]# fdisk -l

    Disk /dev/sda: 146.8 GB, 146815733760 bytes

    255 heads, 63 sectors/track, 17849 cylinders

    Units = cylinders of 16065 * 512 = 8225280 bytes

    Sector size (logical/physical): 512 bytes / 512 bytes

    I/O size (minimum/optimal): 512 bytes / 512 bytes

    Disk identifier: 0x000a6cdd

    Device Boot      Start         End      Blocks   Id  System

    /dev/sda1   *           1          26      204800   83  Linux

    Partition 1 does not end on cylinder boundary.

    /dev/sda2              26         287     2097152   82  Linux swap / Solaris

    Partition 2 does not end on cylinder boundary.

    /dev/sda3             287       17850   141071360   83  Linux

    Disk /dev/sdb: 3774 MB, 3774873600 bytes

    117 heads, 62 sectors/track, 1016 cylinders

    Units = cylinders of 7254 * 512 = 3714048 bytes

    Sector size (logical/physical): 512 bytes / 512 bytes

    I/O size (minimum/optimal): 4096 bytes / 65536 bytes

    Disk identifier: 0xac956c3a

    Device Boot      Start         End      Blocks   Id  System

    /dev/sdb1               1        1016     3685001   83  Linux

    Partition 1 does not start on physical sector boundary.

    Disk /dev/sde: 3774 MB, 3774873600 bytes

    117 heads, 62 sectors/track, 1016 cylinders

    Units = cylinders of 7254 * 512 = 3714048 bytes

    Sector size (logical/physical): 512 bytes / 512 bytes

    I/O size (minimum/optimal): 4096 bytes / 65536 bytes

    Disk identifier: 0xac956c3a

    Device Boot      Start         End      Blocks   Id  System

    /dev/sde1               1        1016     3685001   83  Linux

    Partition 1 does not start on physical sector boundary.

    两个CAN网卡获取到同一盘符:

    /dev/sde和/dev/sdb.

    配置后多了/dev/mapper/mpatha和/dev/mapper/mpathap1:

    [root@localhost mapper]# fdisk -l

    Disk /dev/sda: 146.8 GB, 146815733760 bytes

    255 heads, 63 sectors/track, 17849 cylinders

    Units = cylinders of 16065 * 512 = 8225280 bytes

    Sector size (logical/physical): 512 bytes / 512 bytes

    I/O size (minimum/optimal): 512 bytes / 512 bytes

    Disk identifier: 0x000a6cdd

    Device Boot      Start         End      Blocks   Id  System

    /dev/sda1   *           1          26      204800   83  Linux

    Partition 1 does not end on cylinder boundary.

    /dev/sda2              26         287     2097152   82  Linux swap / Solaris

    Partition 2 does not end on cylinder boundary.

    /dev/sda3             287       17850   141071360   83  Linux

    Disk /dev/sdb: 3774 MB, 3774873600 bytes

    117 heads, 62 sectors/track, 1016 cylinders

    Units = cylinders of 7254 * 512 = 3714048 bytes

    Sector size (logical/physical): 512 bytes / 512 bytes

    I/O size (minimum/optimal): 4096 bytes / 65536 bytes

    Disk identifier: 0xac956c3a

    Device Boot      Start         End      Blocks   Id  System

    /dev/sdb1               1        1016     3685001   83  Linux

    Partition 1 does not start on physical sector boundary.

    Disk /dev/sde: 3774 MB, 3774873600 bytes

    117 heads, 62 sectors/track, 1016 cylinders

    Units = cylinders of 7254 * 512 = 3714048 bytes

    Sector size (logical/physical): 512 bytes / 512 bytes

    I/O size (minimum/optimal): 4096 bytes / 65536 bytes

    Disk identifier: 0xac956c3a

    Device Boot      Start         End      Blocks   Id  System

    /dev/sde1               1        1016     3685001   83  Linux

    Partition 1 does not start on physical sector boundary.

    Disk /dev/mapper/mpatha: 3774 MB, 3774873600 bytes

    117 heads, 62 sectors/track, 1016 cylinders

    Units = cylinders of 7254 * 512 = 3714048 bytes

    Sector size (logical/physical): 512 bytes / 512 bytes

    I/O size (minimum/optimal): 4096 bytes / 65536 bytes

    Disk identifier: 0xac956c3a

    Device Boot      Start         End      Blocks   Id  System

    /dev/mapper/mpathap1               1        1016     3685001   83  Linux

    Partition 1 does not start on physical sector boundary.

    Disk /dev/mapper/mpathap1: 3773 MB, 3773441024 bytes

    255 heads, 63 sectors/track, 458 cylinders

    Units = cylinders of 16065 * 512 = 8225280 bytes

    Sector size (logical/physical): 512 bytes / 512 bytes

    I/O size (minimum/optimal): 4096 bytes / 65536 bytes

    Alignment offset: 1024 bytes

    Disk identifier: 0x00000000

    Disk /dev/mapper/mpathap1 doesn't contain a valid partition table

    # multipath -F #删除现有路径  两个新的路径就会被删除
    # multipath -v2 #格式化路径   格式化后又出现

    3. multipath磁盘的基本操作

    要对多路径软件生成的磁盘进行操作直接操作/dev/mapper/目录下的磁盘就行.

    在对多路径软件生成的磁盘进行分区之前最好运行一下pvcreate命令:

    # pvcreate /dev/mapper/mpatha

    # fdisk /dev/mapper/mpatha  分区时用这个目录/dev/mapper/mpatha

    用fdisk对多路径软件生成的磁盘进行分区保存时会有一个报错,此报错不用理会.

    # ls -l /dev/mapper/

    [root@liujing mnt]#  ls -l /dev/mapper/

    total 0

    crw-rw----. 1 root root 10, 58 Mar 10 19:10 control

    lrwxrwxrwx. 1 root root      7 Mar 10 20:28 mpatha -> ../dm-0

    lrwxrwxrwx. 1 root root      7 Mar 10 20:33 mpathap1 -> ../dm-1

    的mpathap1就是我们对multipath磁盘进行的分区

    # mkfs.ext4 /dev/mapper/mpathap1 #对mpath1p1分区格式化成ext4文件系统

    # mount /dev/mapper/mpathap1 /mnt/ #挂载mpathap1分区

    格式化和挂载时用/dev/mapper/mpathap1 

    4. 分区磁盘:

    上面有提到分区时用目录/dev/mapper/mpatha

    [root@liujing~]# fdisk /dev/mapper/mpatha

    Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel

    Building a new DOS disklabel with disk identifier 0xac956c3a.

    Changes will remain in memory only, until you decide to write them.

    After that, of course, the previous content won't be recoverable.

    Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

    WARNING: DOS-compatible mode is deprecated. It's strongly recommended to

    switch off the mode (command 'c') and change display units to

    sectors (command 'u').

    Command (m for help): n------------------------新建分区

    Command action

    e   extended

    p   primary partition (1-4)

    p-----------------------------主分区

    Partition number (1-4): 1

    First cylinder (1-1016, default 1):

    Using default value 1

    Last cylinder, +cylinders or +size{K,M,G} (1-1016, default 1016):

    Using default value 1016

    Command (m for help): w ---------------------写入列表相当于保存

    The partition table has been altered!

    Calling ioctl() to re-read partition table.

    Syncing disks.

    注:如果同一台设备的两个node挂同样的盘符,另一个盘符还需要再次写入w就行。不需要n了。

    5. 格式化:

    [root@liujing ~]# mkfs.ext4 /dev/mapper/mpathap1

    mke2fs 1.41.12 (17-May-2010)

    /dev/sdd1 alignment is offset by 1024 bytes.

    This may result in very poor performance, (re)-partitioning suggested.

    Filesystem label=

    OS type: Linux

    Block size=4096 (log=2)

    Fragment size=4096 (log=2)

    Stride=1 blocks, Stripe width=16 blocks

    230608 inodes, 921250 blocks

    46062 blocks (5.00%) reserved for the super user

    First data block=0

    Maximum filesystem blocks=943718400

    29 block groups

    32768 blocks per group, 32768 fragments per group

    7952 inodes per group

    Superblock backups stored on blocks:

    32768, 98304, 163840, 229376, 294912, 819200, 884736

    Writing inode tables: done

    Creating journal (16384 blocks): done

    Writing superblocks and filesystem accounting information: done

    This filesystem will be automatically checked every 33 mounts or

    180 days, whichever comes first.  Use tune2fs -c or -i to override.

    6. 挂载 /dev/mapper/mpathap1 到 /mnt

    [root@liujing ~]# mount  /dev/mapper/mpathap1  /mnt

    三、multipath的高级配置之前的配置都是用multipath的默认配置来完成multipath,比如映射设备的名称,multipath负载均衡的方法都是默认设置。那有没有按照我们自己定义的方法来配置multipath呢,答案是OK。

    1、multipath.conf文件的配置

    接下来的工作就是要编辑/etc/multipath.conf的配置文件

    multipath.conf主要包括blacklist、multipaths、devices三部份的配置

    blacklist配置

    blacklist {

    devnode "^sda"

    }

    Multipaths部分配置multipaths和devices两部份的配置。

    multipaths {

    multipath {

    wwid **************** #此值multipath -v3可以看到

    alias iscsi-dm0 #映射后的别名,可以随便取

    path_grouping_policy multibus #路径组策略

    path_checker tur #决定路径状态的方法

    path_selector "round-robin 0" #选择那条路径进行下一个IO操作的方法

    }

    }

    Devices部分配置

    devices {

    device {

    vendor "iSCSI-Enterprise" #厂商名称

    product "Virtual disk" #产品型号

    path_grouping_policy multibus #默认的路径组策略

    getuid_callout "/sbin/scsi_id -g -u -s /block/%n" #获得唯一设备号使用的默认程序

    prio_callout      "/sbin/acs_prio_alua %d" #获取有限级数值使用的默认程序

    path_checker readsector0 #决定路径状态的方法

    path_selector "round-robin 0" #选择那条路径进行下一个IO操作的方法

    failback        immediate #故障恢复的模式

       no_path_retry      queue #在disable queue之前系统尝试使用失效路径的次数的数值

      rr_min_io       100 #在当前的用户组中,在切换到另外一条路径之前的IO请求的数目

    }

    }

    下面是相关参数的标准文档的介绍:

    Attribute

    Description

    wwid

    Specifies the WWID of the multipath device to which themultipath attributes apply. This parameter is mandatory for this section of themultipath.conf file.

    alias

    Specifies the symbolic name for the multipath device to which themultipath attributes apply. If you are usinguser_friendly_names, do not set this value tompathn; this may conflict with an automatically assigned user friendly name and give you incorrect device node names.

    path_grouping_policy

    Specifies the default path grouping policy to apply to unspecified multipaths. Possible values include:

    failover = 1 path per priority group

    multibus = all valid paths in 1 priority group

    group_by_serial = 1 priority group per detected serial number

    group_by_prio = 1 priority group per path priority value

    group_by_node_name = 1 priority group per target node name

     

    path_selector

    Specifies the default algorithm to use in determining what path to use for the next I/O operation. Possible values include:

    round-robin 0: Loop through every path in the path group, sending the same amount of I/O to each.

    queue-length 0: Send the next bunch of I/O down the path with the least number of outstanding I/O requests.

    service-time 0: Send the next bunch of I/O down the path with the shortest estimated service time, which is determined by dividing the total size of the outstanding I/O to each path by its relative throughput.

     

    failback

    Manages path group failback.

    A value of immediate specifies immediate failback to the highest priority path group that contains active paths.

    A value of manual specifies that there should not be immediate failback but that failback can happen only with operator intervention.

    A value of followover specifies that automatic failback should be performed when the first path of a path group becomes active. This keeps a node from automatically failing back when another node requested the failover.

    A numeric value greater than zero specifies deferred failback, expressed in seconds.

     

    prio

    Specifies the default function to call to obtain a path priority value. For example, the ALUA bits in SPC-3 provide an exploitableprio value. Possible values include:

    const: Set a priority of 1 to all paths.

    emc: Generate the path priority for EMC arrays.

    alua: Generate the path priority based on the SCSI-3 ALUA settings.

    tpg_pref: Generate the path priority based on the SCSI-3 ALUA settings, using the preferred port bit.

    ontap: Generate the path priority for NetApp arrays.

    rdac: Generate the path priority for LSI/Engenio RDAC controller.

    hp_sw: Generate the path priority for Compaq/HP controller in active/standby mode.

    hds: Generate the path priority for Hitachi HDS Modular storage arrays.

     

    no_path_retry

    A numeric value for this attribute specifies the number of times the system should attempt to use a failed path before disabling queueing.

    A value of fail indicates immediate failure, without queueing.

    A value of queue indicates that queueing should not stop until the path is fixed.

     

    rr_min_io

    Specifies the number of I/O requests to route to a path before switching to the next path in the current path group. This setting is only for systems running kernels older that 2.6.31. Newer systems should userr_min_io_rq. The default value is 1000.

    rr_min_io_rq

    Specifies the number of I/O requests to route to a path before switching to the next path in the current path group, using request-based device-mapper-multipath. This setting should be used on systems running current kernels. On systems running kernels older than 2.6.31, use rr_min_io. The default value is 1.

    rr_weight

    If set to priorities, then instead of sending rr_min_iorequests to a path before calling path_selector to choose the next path, the number of requests to send is determined byrr_min_io times the path's priority, as determined by theprio function. If set to uniform, all path weights are equal.

    flush_on_last_del

    If set to yes, then multipath will disable queueing when the last path to a device has been deleted.

    在我本地的一个完整的高级配置如下:

    [root@liujing ~]# vi /etc/multipath.conf

    blacklist {

    devnode "^sda"

    }

    multipaths {

    multipath {

    wwid       360a98000646650724434697454546156

    alias      mpathb_fcoe

    path_grouping_policy    multibus

    #path_checker            "directio"

    prio                    "random"

    path_selector           "round-robin 0"

    }

    }

    devices {

    device {

    vendor     "NETAPP"

    product    "LUN"

    getuid_callout       "/lib/udev/scsi_id --whitelisted --device=/dev/%n"

    #path_checker    "directio"

    #path_selector             "round-robin 0"

    failback             immediate

    no_path_retry fail

    }

    }

    其中 wwid,vendor,product, getuid_callout这些参数可以通过:multipath -v3命令来获取。如果在/etc/multipath.conf中有设定各wwid 别名,别名会覆盖此设定。

    四、负载均衡测试:

    可以使用dd命令来对设备进行读写操作,并同时通过iostat来查看I/0状态,流量从哪个路径出去:

    DD命令:dd if=/dev/zero of=/mnt/1Gfile bs=8k count=131072    在上面我们已经把磁盘挂载在/MNT文件夹下所以我们在读写磁盘时直接对/mnt文件夹直接读写就可以了。

    如果想对磁盘重复读写可以用如下语句:

    [root@liujing ~]# for ((i=1;i<=5;i++));do dd if=/dev/zero of=/mnt/1Gfile bs=8k count=131072 2>&1|grep MB;done; ---重复读写5次这个值可以根据自己测试需求修改。

     

    另一个控制台输入iostat 2 10查看IO读写状态:

     

    可以看到sdc和sdd是两个多路径的盘符,流量均匀的负载在两条路径中,负载均衡很成功。

    五、路径冗余备份测试

    将其中一条路径的端口down掉,所有流量会直接切换到另一个路径中。

    展开全文
  • 多路径 --介绍 普通的电脑主机都是一个硬盘挂接到一个总线上,这里是一对一的关系。而到了有光纤组成的SAN环境,或者由iSCSI组成的IPSAN环境,由于主机和存储通过了光纤交换机或者块网卡及IP来连接,这样的话,...

    多路径 -- 介绍

    普通的电脑主机都是一个硬盘挂接到一个总线上,这里是一对一的关系。而到了有光纤组成的SAN环境,或者由iSCSI组成的IPSAN环境,由于主机和存储通过了光纤交换机或者多块网卡及IP来连接,这样的话,就构成了多对多的关系。也就是说,主机到存储可以有多条路径可以选择。主机到存储之间的IO由多条路径可以选择。每个主机到所对应的存储可以经过几条不同的路径,如果是同时使用的话,I/O流量如何分配?其中一条路径坏掉了,如何处理?还有在操作系统的角度来看,每条路径,操作系统会认为是一个实际存在的物理盘,但实际上只是通向同一个物理盘的不同路径而已,这样是在使用的时候,就给用户带来了困惑。多路径软件就是为了解决上面的问题应运而生的。

    多路径 -- 功能

    1. 故障的切换和恢复 
    2. IO流量的负载均衡 
    3. 磁盘的虚拟化

    • 查看当前 scsi 设备 .
    
    suse1:~ # sfdisk -l
    
    Disk /dev/sda: 5221 cylinders, 255 heads, 63 sectors/track
    Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0
    
       Device Boot Start     End   #cyls    #blocks   Id  System
    /dev/sda1          0+    261-    262-   2103296   82  Linux swap / Solaris
    /dev/sda2   *    261+   5221-   4960-  39838720   83  Linux
    /dev/sda3          0       -       0          0    0  Empty
    /dev/sda4          0       -       0          0    0  Empty
    
    suse1:~ # more /proc/partitions
    major minor  #blocks  name
    
       8        0   41943040 sda
       8        1    2103296 sda1
       8        2   39838720 sda2
    • 获取设备wwid .
    for i in `cat /proc/partitions | awk {'print $4'} |grep sd`; 
    do echo "### $i: `/lib/udev/scsi_id --whitelist  
    --replace-whitespace /dev/$i`"; done
    
    // 获取后,过滤本地磁盘及重复的wwid .
    

    多路径 -- 配置使用

    • 安装和加载多路径软件包 .
    # yum –y install device-mapper device-mapper-multipath
    # chkconfig --level 2345 multipathd on #设置成开机自启动multipathd
    # lsmod |grep dm_multipath #来检查安装是否正常
    # rpm  -ql device-mapper-multipath | less  # 生成的配置文件
    • 模块没有加载成功使用下列命初始化DM、重启系统  .
    // DM multipath kernel driver not loaded    ----这个提示说明DM模块没有加载成功 !!
    
    # modprobe dm-multipath 
    # modprobe dm-round-robin 
    # service multipathd start 
    # multipath –v2
    • 校验多路径设备 .
    # multipath -ll | grep mpath
    
    // 设备的详细信息
    
    # multipath -ll
    # dmsetup ls |sort
    
    #以下命令均可以用于校验多路径设备
    # ll /dev/disk/by-id/
    # ls -l /dev/dm-*
    # ll /dev/mapper/
    
    # /dev/dm-*下面的为动态名,即系统重启后会发生变化,此部分用于os 内部使用。
    # /dev/mapper/下面查看到的内容为持久化名称,也即是我们在mulitpatch.conf文件中定义的
    

    • multipath 基本操作命令 .
    # /etc/init.d/multipathd start #开启mulitipath服务 
    # multipath -F #删除现有路径 
    # multipath -v2 #格式化路径 
    # multipath -ll #查看多路径
    
    • 配置 multipath .

    DM模块成功加载,但是/etc/下没有multipath.conf 配置文件,通过vi命令创建一个Multipath的配置文件路径是/etc/multipath.conf ,在配置文件中添加multipath正常工作的配置如下:

    vi /etc/multipath.conf
    
    blacklist {
    devnode "^sda"
    }
    
    defaults {
    user_friendly_names yes
    path_grouping_policy multibus
    failback immediate
    no_path_retry fail
    }
    
    编辑完成后保存配置.
    
    # /etc/init.d/multipathd start  # 开启mulitipath服务

    • multipath 高级配置
    multipath.conf主要包括blacklist、multipaths、devices三部份的配置
     
    blacklist配置
    blacklist {
    devnode "^sda"
    }
    Multipaths部分配置multipaths和devices两部份的配置。
    multipaths {
    multipath {
    wwid ****************   #此值multipath -v3可以看到
    alias iscsi-dm0 #映射后的别名,可以随便取
    path_grouping_policy multibus #路径组策略
    path_checker tur    #决定路径状态的方法
    path_selector "round-robin 0" #选择那条路径进行下一个IO操作的方法
    }
    } 
    Devices部分配置
    devices {
    device {
    vendor "iSCSI-Enterprise" #厂商名称
    product "Virtual disk" #产品型号
    path_grouping_policy multibus #默认的路径组策略
    getuid_callout "/sbin/scsi_id -g -u -s /block/%n" #获得唯一设备号使用的默认程序
    prio_callout      "/sbin/acs_prio_alua %d" #获取有限级数值使用的默认程序
    path_checker readsector0 #决定路径状态的方法
    path_selector "round-robin 0" #选择那条路径进行下一个IO操作的方法
    failback        immediate #故障恢复的模式
       no_path_retry      queue #在disable queue之前系统尝试使用失效路径的次数的数值
      rr_min_io       100 #在当前的用户组中,在切换到另外一条路径之前的IO请求的数目
    }
    }
    • 完整的配置文件 .
    blacklist {
    devnode "^sda"
    }
    defaults {
    user_friendly_names no
    }
    multipaths {
    multipath {
    wwid 14945540000000000a67854c6270b4359c66c272e2f356321
    alias iscsi-dm0
    path_grouping_policy multibus
    path_checker tur
    path_selector "round-robin 0"
    }
    multipath {
    wwid 14945540000000000dcca2eda91d70b81edbcfce2357f99ee
    alias iscsi-dm1
    path_grouping_policy multibus
    path_checker tur
    path_selector "round-robin 0"
    }
    multipath {
    wwid 1494554000000000020f763489c165561101813333957ed96
    alias iscsi-dm2
    path_grouping_policy multibus
    path_checker tur
    path_selector "round-robin 0"
    }
    multipath {
    wwid 14945540000000000919ca813020a195422ba3663e1f03cc3
    alias iscsi-dm3
    path_grouping_policy multibus
    path_checker tur
    path_selector "round-robin 0"
    }
    }
    devices {
    device {
    vendor "iSCSI-Enterprise"
    product "Virtual disk"
    path_grouping_policy multibus
    getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
    path_checker readsector0
    path_selector "round-robin 0"
    }
    }
    • 获取wwid的方法 .

    默认情况下,将使用 /var/lib/multipath/bindings 内的配置设定具体每个多路径设备名,如果在/etc/multipath.conf中有设定各wwid 别名,别名会覆盖此设定.

    • 负载均衡测试 .
    # dd if=/dev/zero of=/dev/mapper/iscsi-dm1p1 
    开启另外一个终端用以下命令查看IO情况
    # iostat 10 10

     

    展开全文
  • 在主机和存储的连接过程中一般是主机——光纤交换机——存储,这就代表了主机有条路可以通往存储。...yum -y install device-mapper-multipath.x86_64 #安装多路径包 chkconfig --level 2345 multip

    在主机和存储的连接过程中一般是主机——光纤交换机——存储,这就代表了主机有多条路可以通往存储。
    在操作系统的角度来看,每条路径,操作系统会认为是一个实际存在的物理盘,但实际上,多个路径对应了同一设备

    一、软件安装

    1.检查并安装多路径软件

    yum -y install device-mapper.x86_64      #安装映射包
    yum -y install device-mapper-multipath.x86_64      #安装多路径包
    chkconfig --level 2345 multipathd on  #设置开机自启动
    

    2.查看安装是否成功

    # lsmod | grep dm_mutipath
    ---如果模块没有加载成功则初始化DM
    # modprobe dm-multipath
    # modprobe dm-round-robin
    # service multipathd start
    # multipath -v2
    --再次查看
    lsmod | grep dm_multipath
    

    说明:
    device-mapper-multipath:主要提供multipathd和multipath等工具和multipath.conf配置文件
    创建的多路径设备放在/dev/mapper中

    二、多路径配置

    1.先生成模板配置文件(免得格式写错)
    –执行下述命令生成多路径的配置文件

    # mpathconf --enable
    # ls -l /etc/multipath.conf
    

    2.查看scsi盘的wwid号

    # /sbin/scsi_id --whitelisted --replace-whitespace --device /dev/sdb
    1ATA_VBOX_HARDDISK_VBae744270-bf228329
    

    注意1:–replace-whitespace的意思是replace all whitespace by underscores**(把空格用下划线替换)**
    如果没有该选项,结果如下,可以比较一下(这样比较好记)
    在这里插入图片描述

    注意2:在通过光交联系存储的情况下会发现不同lun的wwid一致,就说明了他们其实是一种设备(通过不同路径到达),所以对应一致的wwid的两个设备可以被同一个alias识别到

    ---比如这样的,wwid号两两对应
    [root@rac1 ~]# /sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/sdb
    46000eca45006a0220054a158101aa0000
    [root@rac1 ~]# /sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/sdc
    46000eca45006a0220054a158101ab0000
    [root@rac1 ~]# /sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/sdd
    46000eca45006a0220054a158101aa0000
    [root@rac1 ~]# /sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/sde
    46000eca45006a0220054a158101ab0000
    [root@rac1 ~]# /sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/sdf
    46000eca45006a0220054a158101ac0000
    [root@rac1 ~]# /sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/sdg
    46000eca45006a0220054a158101ac0000
    

    3.编辑配置文件
    根据上面查出来的wwid号,配置对应别名,multipath.conf如下:
    在这里插入图片描述

    4.重启multipathd服务后验证设备情况

    # service multipathd restart
    # multipath -ll
    

    PS:我因为没有外挂存储条件,用本地硬盘做的测试(1v1),多路径结果查询不到映射关系,/dev/mapper也没有,应该不是操作问题

    展开全文
  • 1、启用Multipath:(1)启动multipathd服务#service multipathd start或者#/etc/init.d/multipathd start(2)修改multipath配置文件/etc/multipath.conf:a默认情况下所以的设备都在multipath的黑名单中,所以即使启动...
  • 最近,指导一位同事对oracle rac集群服务器的linux系统做多路径映射和udev绑盘,然后做ASM加盘,一共做了两次; 第一次,同事在生产端加盘,rac集群两台服务器识别到的盘符是sdw、sdx、sdy、sdz之类,经过多路径...
  • \Multipath多路径软件: device-mapper-multipath配置文件: /etc/multipath.conf1.多路径概述当服务器到某一存储设备有多条路径时,每条路径都会识别为一个单独的设备;多路径允许您将服务器节点和储存阵列间的多个...
  • 一、multipath在redhat 6.2中的基本配置: 1.通过命令:lsmod |grep dm_multipath 检查是否正常安装成功。如果没有输出说明没有安装那么通过yum功能...接着通过命令:multipath –ll查看多路径状态查看模块是否加载...
  • 1、启用Multipath:(1)启动multipathd服务 #service multipathd start 或者 #/etc/init.d/multipathd start(2)修改multipath配置文件/etc/multipath.conf: a 默认情况下所以的设备都在multipath的黑名单中,...
  • 也就是说,主机到存储可以有路径可以选择。主机到存储之间的IO由路径可以选择。既然,每个主机到所对应的存储可以经过几条不同的路径,如果是同时使用的话,I/O流量如何分配?其中一条路径...
  • Linux 安装多路径软件multipath

    千次阅读 2019-09-12 19:40:53
    Linux和AIX平台都有一部分存储产品使用操作系统自带的多路径软件,包括最常见的HP和IBM的部分存储产品,在Linux自带的多路径软件叫做multipath,这篇文章以IBM N系列存储在Linux平台的使用为例,讨论Linux平台...
  • (一)多路径管理软件的由来 在企业中,服务器与存储通常是分开放置的,服务器上的硬盘通常用来安装操作系统和应用软件,业务数据则是存储在单独的存储设备上,那么,服务器与存储是如何连接的呢?根据存储协议,经常...
  • 这时可以通过 multipath 多路径来绑定磁盘! ☀️ 安装 multipath 需提前配置 yum 源,可参考:Linux 配置本地 yum 源(6/7/8) 如果选择最小化安装的 Linux 主机,则需要手动安装 multipath,否则默认为自动安装。...
  • 红帽 Linux6 设备映射多路径 DM Multipath 配置及管理
  • 1、安装multipath工具 1 #yum–yinstalldevice-mapperdevice-mapper-multipath 2、设置开机自启动 1 #chkconfigmultipathdon 3、开启multipath 1 #mpathconf--enabl...
  • Linux和AIX平台都有一部分存储产品使用操作系统自带的多路径软件,包括最常见的HP和IBM的部分存储产品,在Linux自带的多路径软件叫做multipath,这篇文章以IBM N系列存储在Linux平台的使用为例,讨论Linux平台...
  • 1、启用Multipath:(1)启动multipathd服务#service multipathd start 或者 #/etc/init.d/multipathd start(2)修改multipath配置文件/etc/multipath.conf:a 默认情况下所以的设备都在multipath的黑名单中,所以即使...
  • Linux平台一部分存储产品使用操作系统自带的多路径软件,包括最常见的HP和IBM的部分存储产品,在Linux自带的多路径软件叫做multipath,这篇文章以HP EVA系列存储在Linux平台的使用为例,详细的在Linux平台配置和...
  • 多路径软件是为了解决单路径一对一的单点故障而研发的,实现由主机到存储的条链路通讯。如果存储服务器到主机或是交换机只有一条线路的时候,当这条线路出现故障时,整个系统就无法使用,所以使用多路径可以解决这...
  • 多路径multipath参数配置详解

    千次阅读 2020-07-24 09:33:44
    默认情况下,multipath已经支持大...这时我们可以给特定的存储定制多路径符合的策略。  配置文件的模板默认位于/usr/share/doc/device-mapper-multipath-x.x.x/multipath.conf(x为multipath的实际版本号),配置mult
  • RHEL7 多路径multipath配置

    千次阅读 2018-09-30 10:23:07
    multipath的高级配置 映射设备的名称,multipath负载均衡的方法都是默认设置。那有没有按照我们自己定义的方法来配置multipath呢,当可以。 1、multipath.conf文件的配置 接下来的工作就是要编辑/etc/multipath....

空空如也

空空如也

1 2 3 4 5 ... 20
收藏数 2,201
精华内容 880
关键字:

linux多路径multipath

linux 订阅