一般情况下大家在使用LVS的时候都很喜欢直接用DR模式，觉得DR模式的效率是最高。不过实际上DR模式在很多时候给我们带来的约束也非常大，最明显的莫过于LVS机器需要和RS机器能有一张网卡共处在一个vlan下。机房环境比较复杂的时候还用DR模式经常会受到各种的约束，比如同一个VLAN的IP都被用光了、同一个交换机下机柜没有空位了，etc。所以实际上我们也经常使用TUN模式。最近遇到一个之前使用DR模式，现在不能扩容的情况，上午就尝试在测试环境测试了一下DR模式到TUN模式的切换，整体影响和LVS主备切换的时候差不多，影响可控。
实际的背景是现在LVS1和LVS2做互备给DNS1、DNS2做负载均衡。但是因为找不到机为能和LVS机器挂同一个VLAN下，所以我现在需要把LVS的模式修改为TUN模式，以便对RS直接扩容。
机器列表：
LVS:
192.168.100.16 LVS1-slave
192.168.100.17 LVS2-slave
VIP:192.168.100.8
DNS Server:
192.168.100.18 DNS1
192.168.100.22 DNS2
192.168.100.38 DNS3(NEW)
192.168.128.29 DNS4(NEW)
keepalived原来的配置文件：
vrrp_instance dns {
!state MASTER
state BACKUP
interface bond0
lvs_sync_daemon_interface bond0
virtual_router_id 51
priority 99
advert_int 1
nopreempt
garp_master_delay 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.100.8/22 dev bond0 label bond0:1
#配置的时候主要掩码不要写错了
}
}
virtual_server 192.168.100.8 53 {
delay_loop 30
lb_algo rr
lb_kind DR
ha_suspend
persistence_timeout 0
protocol TCP
real_server 192.168.100.18 53 {
weight 100
TCP_CHECK {
connect_port 53
connect_timeout 3
nb_get_retry 3
delay_before_retry 10
}
}
real_server 192.168.100.22 53 {
weight 100
TCP_CHECK {
connect_port 53
connect_timeout 3
nb_get_retry 3
delay_before_retry 10
}
}
}
virtual_server 192.168.100.8 53 {
delay_loop 30
lb_algo rr
lb_kind DR
ha_suspend
persistence_timeout 0
omega
protocol UDP
real_server 192.168.100.18 53 {
weight 100
TCP_CHECK {
connect_port 53
connect_timeout 3
nb_get_retry 3
delay_before_retry 10
}
}
real_server 192.168.100.22 53 {
weight 100
TCP_CHECK {
connect_port 53
connect_timeout 3
nb_get_retry 3
delay_before_retry 10
}
}
}
基本的步奏：
1. 在RS服务器上把tun设备加上，直接绑定上VIP。有报错提示IP加重复了也没有关系。
#!/bin/bash
VIP='192.168.100.8'
case $1 in
start)
modprobe -r ipip
modprobe ipip
ip link set tunl0 up
ip link set tunl0 arp off
for IP in $VIP
do
NO=$((NO+1))
ip addr add $IP/32 br $IP label tunl0:$NO dev tunl0
ip route add $IP/32 dev tunl0
done
echo 1 > /proc/sys/net/ipv4/conf/tunl0/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/tunl0/arp_announce
echo 0 > /proc/sys/net/ipv4/conf/tunl0/rp_filter
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
;;
stop)
modprobe -r ipip
;;
*)
echo "$0: Usage: $0 {start|stop|status}"
exit 1
;;
esac
-
先把目前做backup状态的LVS配置文件修改掉，改成使用TUN模式。重启好keepalived后把主的服务器停掉，观察主备切换后是否正常。不出问题就可以继续把停掉的这台的配置文件也修改好重启。
-
向keeplived里把其他要扩容的RS加如，分别重启
vrrp_instance test {
!state MASTER
state BACKUP
interface bond0
lvs_sync_daemon_interface bond0
virtual_router_id 51
priority 99
advert_int 1
nopreempt
garp_master_delay 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.100.8/22 dev bond0 label bond0:1
}
}
virtual_server 192.168.100.8 53 {
delay_loop 30
lb_algo rr
#lb_kind DR
lb_kind TUN
ha_suspend
persistence_timeout 0
protocol TCP
real_server 192.168.100.18 53 {
weight 100
TCP_CHECK {
connect_port 53
connect_timeout 3
nb_get_retry 3
delay_before_retry 10
}
}
real_server 192.168.100.22 53 {
weight 100
TCP_CHECK {
connect_port 53
connect_timeout 3
nb_get_retry 3
delay_before_retry 10
}
}
real_server 192.168.100.38 53 {
weight 100
TCP_CHECK {
connect_port 53
connect_timeout 3
nb_get_retry 3
delay_before_retry 10
}
}
real_server 192.168.128.29 53 {
weight 100
TCP_CHECK {
connect_port 53
connect_timeout 3
nb_get_retry 3
delay_before_retry 10
}
}
}
virtual_server 192.168.100.8 53 {
delay_loop 30
lb_algo rr
lb_kind TUN
#lb_kind DR
ha_suspend
persistence_timeout 0
omega
protocol UDP
real_server 192.168.100.18 53 {
weight 100
TCP_CHECK {
connect_port 53
connect_timeout 3
nb_get_retry 3
delay_before_retry 10
}
}
real_server 192.168.100.22 53 {
weight 100
TCP_CHECK {
connect_port 53
connect_timeout 3
nb_get_retry 3
delay_before_retry 10
}
}
real_server 192.168.100.38 53 {
weight 100
TCP_CHECK {
connect_port 53
connect_timeout 3
nb_get_retry 3
delay_before_retry 10
}
}
real_server 192.168.128.29 53 {
weight 100
TCP_CHECK {
connect_port 53
connect_timeout 3
nb_get_retry 3
delay_before_retry 10
}
}
}
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
UDP 192.168.100.8:53 rr
-> 192.168.128.29:53 Tunnel 100 0 223
-> 192.168.100.38:53 Tunnel 100 0 227
-> 192.168.100.22:53 Tunnel 100 0 224
-> 192.168.100.18:53 Tunnel 100 0 220
TCP 192.168.100.8:53 rr
-> 192.168.128.29:53 Tunnel 100 0 0
-> 192.168.100.38:53 Tunnel 100 0 0
-> 192.168.100.22:53 Tunnel 100 0 0
-> 192.168.100.18:53 Tunnel 100 0 0