GNUers 博客

ixfr-from-differences的功效

常规情况下bind的主备同步是自动增量同步的。但是有些场景下是全量同步，比如自己手动改的zone文件，重新加载进去。一般内部的反解信息是根据所有的zone自动生成的，就会存在PTR记录每次全量同步的量非常大。测试了可以通过打开ixfr-from-differences，在master上自动计算差异，slave就可以做增量同步了。 ixfr-from-differences yes; 上图中可以看到之前没有打开ixfr-from-differences时同步1.9W条记录需要2.6s，开启之后每次增量同步只需要0.02s。开启ixfr-from-differences 时会增加master的CPU、内存开销，所以需要根据实际的情况衡量是否需要打开。

checksum error的原因

今天有同事反馈dig @223.5.5.5的时候看到本地发出去的包是提示“bad udp cksum” xxx > 223.5.5.5.53: [bad udp cksum 0x85e1 -> 0xc2e3!] 8250+ A? www.baidu.com. 实际这个是因为网卡开启了tx checksum，开启之后这个checksum的计算是由网卡硬件自己完成，tcpdump抓包的时候实际还没有去结算checksum，所以一直是bad upd cksum #ethtool -k eth1 Offload parameters for eth1: rx-checksumming: on tx-checksumming: on scatter-gather: on tcp-segmentation-offload: on udp-fragmentation-offload: off generic-segmentation-offload: on generic-receive-offload: on large-receive-offload: off rx-vlan-offload: off tx-vlan-offload: off ntuple-filters: off receive-hashing: off 只能在目标机器进行抓包才能发现是否发出的包checkum是否真的有错误。另外可以选择本地把tx checksum关闭掉 #ethtool -K eth1 tx off 再测试的时候可以看到是OK的了 xxx. > 223.5.5.5.53: [udp sum ok] 44024+ A? www.baidu.com. (31) 实际利用网卡计算checksum显然更好，所以不用太在意这个。

arp_announce引发的1个ARP问题

最近配置服务器时遇到在dummy0上宣告的公网地址不能正常访问公网的问题。网络的基本结构为2个网卡分别上联2个交换机，跑OSPF。在dummy0上会单独宣告1个公网的地址。自己通过在zebra内修改路由表使得访问公网时设置自己宣告的公网地址为源IP。以前这个方案实际线上跑了多次，一直OK。最近有一套服务器安装这样的配置会出现刚启动时是OK的，过段时间就歇菜了。简单地说实际的情况如下：T1 (192.168.1.2 )–> 交换机A的Port X(192.168.1.1)T2 (192.168.2.2 )–> 交换机B的Port X(192.168.2.1)平时默认都走T1，公网的路由表默认学到的网关实际是T2的对端地址。当本地公网地址不能出去的时候，我自己带源地址(架设公网地址是4.4.4.4)ping，发现发送的ARP请求都是这样 Request who-has 192.168.2.1 tell 4.4.4.4 此时交换机上发现4.4.4.4实际不是和自己一个网段的地址，不会进行回复。就使得这本地这个公网地址出不去。解决这个问题就是加内核参数 net.ipv4.conf.all.arp_announce=1 下面是fix后的情况，可以看到使用的是接口上的IP为源IP发的ARP请求。附上参数说明 rp_announce - INTEGER Define different restriction levels for announcing the local source IP address from IP packets in ARP requests sent on interface: 0 - (default) Use any local address, configured on any interface 1 - Try to avoid local addresses that are not in the target's subnet for this interface. This mode is useful when target ...

bind 9.10的Pre-fetch测试

bind 9.10里有个prefetch的特性。文档描述如下 When a query is received for cached data which is to expire shortly, named can refresh the data from the authoritative server immediately, ensuring that the cache always has an answer available. 当DNS递归服务器接受到一个域名的查询请求时，如果这个域名本来已经有cache记录且马上要过期了。named会主动刷新这条记录。实际的逻辑是假如在0s的时候请求了qq.com，qq.com的TTL是600s。如果设置了 prefetch 5 10; 那么在595-599的这段时间内，再次收到qq.com的查询请求，named会自己主动去递归一次。prefetch接受2个参数，第一个是触发namde主动递归的剩余TTL上限，第二个是只有TTL大于10的才做主动递归更新。这个可以提高热域名的cache命中率，但是对请求量很小的冷域名的解析提升无用。参考：1. https://kb.isc.org/article/AA-01122

bind启动时提示953端口被使用

部署DNS的时候遇到个奇葩的问题，总是提示 couldn’t add command channel 0.0.0.0#953: address in use 实际上系统上并没有进程使用953端口。查询了一下找了原因。就是portreserve的问题，关闭后OK。原因根本的原因是自己打包了一个bind的RPM，会引入/etc/portreserve/named,这样当portreserve启动的时候就会占着53端口。单独使用/etc/init.d/named启动的时候会执行portrelease 来释放53端口。 portrelease named

配置OpenVPN只注入特定路由表

通过情况下大家配置VPN时，都会直接把默认网关指向服务端。但是这会影响访问公司内网资源，造成一些不便。如果只是希望走VPN访问部分外网资源，可以不让VPN客户端改默认的网关，通过加一些路由表使得到特定地址走VPN。比如我在公司的时候只是需要访问一下google，可以在VPN服务端新增配置,其中把push redirect-gateway def1 bypass-dhcp给直接注释掉，新push了很多路由表到客户端。另外，在客户端也需要把redirect-gateway给注释掉。这样启动后可以看到只是注入特定的路由表，没有改默认路由。附上服务端配置： port 600xxxproto udpdev tunca key/ca.crtcert key/server.crtkey key/server.key # This file should be kept secretdh key/dh1024.pemserver 10.99.1.0 255.255.255.0ifconfig-pool-persist ipp.txtpush "dhcp-option DNS 208.67.220.220"push "dhcp-option DNS 208.67.222.222"#push redirect-gateway def1 bypass-dhcpkeepalive 10 30comp-lzomax-clients 60user nobodygroup nogrouppersist-keypersist-tunstatus openvpn-google.logverb 3mute 20duplicate-cn add google route rulepush “route 207.223.160.0 255.255.240.0 “push “route 66.249.85.0 255.255.255.0 “push “route 66.249.83.0 255.255.255.0 “push “route 74.125.130.0 255.255.255.0 “push “route 192.178.0.0 255.254.0.0 “push “route 64.233.160.0 255.255.255.0 “push “route 74.125.17.0 255.255.255.0 “push “route 66.249.72.0 255.255.255.0 “push “route 173.194.112.0 255.255.255.0 “push “route 173.194.98.0 255.255.255.0 “push “route 173.194.140.0 255.255.255.0 “push “route 74.125.196.0 255.255.255.0 “push “route 173.194.78.0 255.255.255.0 “push “route 209.85.238.0 255.255.255.0 “push “route 72.14.208.0 255.255.254.0 “push “route 64.233.164.0 255.255.255.0 “push “route 8.15.202.0 255.255.255.0 “push “route 74.125.142.0 255.255.255.0 “push “route 108.177.0.0 255.255.128.0 “push “route 74.125.203.0 255.255.255.0 “push “route 74.125.58.0 255.255.255.0 “push “route 173.194.141.0 255.255.255.0 “push “route 72.14.244.0 255.255.254.0 “push “route 173.194.73.0 255.255.255.0 “push “route 72.14.225.0 255.255.255.0 “push “route 74.125.193.0 255.255.255.0 “push “route 74.125.239.0 255.255.255.0 “push “route 173.255.112.0 255.255.240.0 “push “route 173.194.119.0 255.255.255.0 “push “route 66.249.64.0 255.255.224.0 “push “route 66.249.70.0 255.255.255.0 “push “route 74.125.190.0 255.255.255.0 “push “route 74.125.70.0 255.255.255.0 “push “route 74.125.206.0 255.255.255.0 “push “route 74.125.198.0 255.255.255.0 “push “route 173.194.75.0 255.255.255.0 “push “route 8.34.208.0 255.255.248.0 “push “route 74.125.19.0 255.255.255.0 “push “route 74.125.131.0 255.255.255.0 “push “route 66.102.4.0 255.255.255.0 “push “route 173.194.76.0 255.255.255.0 “push “route 8.34.216.0 255.255.248.0 “push “route 66.249.79.0 255.255.255.0 “push “route 66.249.90.0 255.255.255.0 “push “route 162.216.148.0 255.255.252.0 “push “route 173.194.32.0 255.255.255.0 “push “route 173.194.142.0 255.255.255.0 “push “route 74.125.238.0 255.255.255.0 “push “route 74.125.18.0 255.255.255.0 “push “route 74.125.234.0 255.255.255.0 “push “route 173.194.79.0 255.255.255.0 “push “route 173.194.40.0 255.255.255.0 “push “route 66.249.64.0 255.255.255.0 “push “route 74.125.68.0 255.255.255.0 “push “route 74.125.43.0 255.255.255.0 “push “route 192.158.28.0 255.255.252.0 “push “route 8.35.192.0 255.255.248.0 “push “route 74.125.0.0 255.255.0.0 “push “route 209.85.128.0 255.255.128.0 “push “route 66.249.67.0 255.255.255.0 “push “route 66.249.84.0 255.255.255.0 “push “route 1.2.3.0 255.255.255.0 “push “route 74.125.232.0 255.255.255.0 “push “route 173.194.96.0 255.255.255.0 “push “route 74.125.118.0 255.255.255.0 “push “route 74.125.28.0 255.255.255.0 “push “route 173.194.121.0 255.255.255.0 “push “route 70.32.144.0 255.255.255.0 “push “route 74.125.186.0 255.255.255.0 “push “route 74.125.31.0 255.255.255.0 “push “route 64.233.166.0 255.255.255.0 “push “route 74.125.207.0 255.255.255.0 “push “route 8.8.8.0 255.255.255.0 “push “route 173.194.65.0 255.255.255.0 “push “route 74.125.138.0 255.255.255.0 “push “route 173.194.34.0 255.255.255.0 “push “route 74.125.192.0 255.255.255.0 “push “route 66.249.91.0 255.255.255.0 “push “route 74.125.229.0 255.255.255.0 “push “route 74.125.88.0 255.255.254.0 “push “route 74.125.37.0 255.255.255.0 “push “route 74.125.40.0 255.255.255.0 “push “route 74.125.176.0 255.255.255.0 “push “route 64.233.171.0 255.255.255.0 “push “route 173.194.70.0 255.255.255.0 “push “route 193.142.125.0 255.255.255.0 “push “route 74.125.187.0 255.255.255.0 “push “route 74.125.29.0 255.255.255.0 “push “route 74.125.16.0 255.255.255.0 “push “route 66.249.65.0 255.255.255.0 “push “route 173.194.66.0 255.255.255.0 “push “route 74.125.202.0 255.255.255.0 “push “route 173.194.68.0 255.255.255.0 “push “route 173.194.120.0 255.255.255.0 “push “route 173.194.113.0 255.255.255.0 “push “route 216.239.38.0 255.255.255.0 “push “route 146.148.0.0 255.255.128.0 “push “route 64.233.160.0 255.255.224.0 “push “route 66.102.2.0 255.255.255.0 “push “route 66.249.88.0 255.255.255.0 “push “route 72.14.192.0 255.255.192.0 “push “route 66.249.78.0 255.255.255.0 “push “route 173.194.45.0 255.255.255.0 “push “route 74.125.183.0 255.255.255.0 “push “route 74.125.230.0 255.255.255.0 “push “route 74.125.129.0 255.255.255.0 “push “route 70.32.148.0 255.255.254.0 “push “route 172.253.0.0 255.255.0.0 “push “route 74.125.116.0 255.255.255.0 “push “route 173.194.97.0 255.255.255.0 “push “route 64.233.186.0 255.255.255.0 “push “route 23.236.48.0 255.255.240.0 “push “route 74.125.76.0 255.255.255.0 “push “route 74.125.26.0 255.255.255.0 “push “route 74.125.36.0 255.255.255.0 “push “route 74.125.63.0 255.255.255.0 “push “route 66.249.80.0 255.255.255.0 “push “route 142.250.0.0 255.254.0.0 “push “route 173.194.124.0 255.255.255.0 “push “route 64.233.172.0 255.255.255.0 “push “route 173.194.43.0 255.255.255.0 “push “route 23.251.128.0 255.255.224.0 “push “route 74.125.235.0 255.255.255.0 “push “route 162.222.176.0 255.255.248.0 “push “route 74.125.137.0 255.255.255.0 “push “route 173.194.72.0 255.255.255.0 “push “route 173.194.136.0 255.255.255.0 “push “route 74.125.188.0 255.255.255.0 “push “route 74.125.185.0 255.255.255.0 “push “route 173.194.42.0 255.255.255.0 “push “route 74.125.226.0 255.255.255.0 “push “route 74.125.227.0 255.255.255.0 “push “route 216.239.35.0 255.255.255.0 “push “route 1.0.0.0 255.255.255.0 “push “route 173.194.46.0 255.255.255.0 “push “route 74.125.205.0 255.255.255.0 “push “route 216.239.34.0 255.255.255.0 “push “route 74.125.117.0 255.255.255.0 “push “route 173.194.44.0 255.255.255.0 “push “route 74.125.182.0 255.255.255.0 “push “route 74.125.178.0 255.255.255.0 “push “route 74.125.30.0 255.255.255.0 “push “route 216.239.39.0 255.255.255.0 “push “route 74.125.231.0 255.255.255.0 “push “route 66.249.92.0 255.255.255.0 “push “route 66.102.0.0 255.255.240.0 “push “route 216.239.44.0 255.255.254.0 “push “route 74.125.74.0 255.255.255.0 “push “route 173.194.33.0 255.255.255.0 “push “route 216.58.192.0 255.255.224.0 “push “route 173.194.117.0 255.255.255.0 “push “route 74.125.191.0 255.255.255.0 “push “route 74.125.22.0 255.255.255.0 “push “route 173.194.35.0 255.255.255.0 “push “route 74.125.201.0 255.255.255.0 “push “route 216.239.32.0 255.255.224.0 “push “route 74.125.121.0 255.255.255.0 “push “route 66.249.89.0 255.255.255.0 “push “route 108.59.80.0 255.255.240.0 “push “route 74.125.224.0 255.255.255.0 “push “route 172.217.0.0 255.255.0.0 “push “route 74.125.119.0 255.255.255.0 “push “route 113.197.106.0 255.255.255.0 “push “route 64.233.173.0 255.255.255.0 “push “route 66.102.3.0 255.255.255.0 “push “route 74.125.177.0 255.255.255.0 “push “route 74.125.41.0 255.255.255.0 “push “route 74.125.189.0 255.255.255.0 “push “route 74.125.24.0 255.255.255.0 “push “route 74.125.236.0 255.255.255.0 “push “route 74.125.143.0 255.255.255.0 “push “route 8.35.200.0 255.255.248.0 “push “route 173.194.67.0 255.255.255.0 “push “route 72.14.228.0 255.255.255.0 “push “route 173.194.36.0 255.255.255.0 “push “route 74.125.184.0 255.255.255.0 “push “route 64.233.168.0 255.255.255.0 “push “route 173.194.41.0 255.255.255.0 “push “route 74.125.90.0 255.255.254.0 “push “route 173.194.118.0 255.255.255.0 “push “route 173.194.37.0 255.255.255.0 “push “route 107.178.192.0 255.255.192.0 “push “route 173.194.0.0 255.255.0.0 “push “route 74.125.180.0 255.255.255.0 “push “route 74.125.200.0 255.255.255.0 “push “route 74.125.233.0 255.255.255.0 “push “route 74.125.122.0 255.255.255.0 “push “route 70.32.128.0 255.255.224.0 “push “route 130.211.0.0 255.255.0.0 “push “route 74.125.237.0 255.255.255.0 “push “route 74.125.42.0 255.255.255.0 “push “route 173.194.64.0 255.255.255.0 “push “route 74.125.20.0 255.255.255.0 “push “route 173.194.127.0 255.255.255.0 “push “route 74.125.128.0 255.255.255.0 “push “route 173.194.39.0 255.255.255.0 “push “route 74.125.194.0 255.255.255.0 “push “route 66.249.77.0 255.255.255.0 “push “route 173.194.69.0 255.255.255.0 “push “route 74.125.136.0 255.255.255.0 “push “route 74.125.54.0 255.255.254.0 “push “route 173.194.99.0 255.255.255.0 “push “route 66.249.74.0 255.255.255.0 “push “route 66.249.93.0 255.255.255.0 “push “route 66.249.69.0 255.255.255.0 “push “route 74.125.25.0 255.255.255.0 “push “route 74.125.228.0 255.255.255.0 “push “route 64.233.165.0 255.255.255.0 “push “route 216.239.36.0 255.255.255.0 “push “route 173.194.91.0 255.255.255.0 “push “route 74.125.21.0 255.255.255.0 “push “route 74.125.73.0 255.255.255.0 “push “route 216.239.32.0 255.255.255.0 “push “route 108.170.192.0 255.255.192.0 “push “route 199.223.232.0 255.255.248.0 “push “route 74.125.225.0 255.255.255.0 “push “route 199.192.112.0 255.255.252.0 “push “route 173.194.77.0 255.255.255.0 “push “route 66.249.73.0 255.255.255.0 “push “route 66.249.81.0 255.255.255.0 “push “route 64.233.167.0 255.255.255.0 “push “route 8.8.4.0 255.255.255.0 “push “route 74.125.72.0 255.255.255.0 “push “route 74.125.135.0 255.255.255.0 “push “route 74.125.45.0 255.255.255.0 “push “route 216.239.33.0 255.255.255.0 “push “route 107.167.160.0 255.255.224.0 “push “route 173.194.38.0 255.255.255.0 “push “route 1.1.1.0 255.255.255.0 “push “route 74.125.195.0 255.255.255.0 “push “route 74.125.204.0 255.255.255.0 “push “route 173.194.126.0 255.255.255.0 “push “route 74.125.23.0 255.255.255.0 “push “route 66.249.66.0 255.255.255.0 “push “route 66.249.76.0 255.255.255.0 “push “route 173.194.71.0 255.255.255.0 “push “route 74.125.181.0 255.255.255.0 “# add opendns route rulepush “route 208.67.0.0 255.255.0.0 " 客户端配置：```bash clientdev tunproto udpremote xx.xx.xx.xx 60xxxresolv-retry infinitenobindpersist-keypersist-tunca openvzca.crtcert openvz.crtkey openvz.keyns-cert-type servercomp-lzoverb 3mute 20keepalive 20 60;redirect-gatewaymax-routes 10000

nginx做cache时，缓存时间的指定顺序

其实做cache的server，绝大部分都是优先根据站源header里的Expires或者Cache-control里的max-age时间来设置本地cache文件的有效期。只有当站源没有这2个参数或者使用了其他的参数忽略了站源header的时候，才是使用nginx内的 proxy_cache_valid设置缓存时间。基本的顺序如下1. 后端有设置有效期的一些headernginx-proxy根据 “X-Accel-Expires”，“Expires”，“Cache-Control”设置响应缓存的参数。这些header指定的有效期优先级高于proxy_cache_valid。当然，也可以使用proxy_ignore_headers忽略部分后端设置的header，达到忽略后端服务器设置的目的，一般不建议这样做。后端没有设置Cache-control之类的根据proxy_cache_valid设置的缓存时间cache. 3.根据 proxy_cache_path的inactive进行cache的过期.

nginx的合并回源

最近使用nginx做缓存，发现当大量客户端访问一个没有cache的文件时，回源的流量非常大，在站源上查看日志也看到确实有并发的请求。这个就是需要改成合并回源，当cache内没有数据的时候，只允许1个请求去站源请求数据，写到本地cache。nginx从1.1.12开始原生支持合并回源了。主要有2个配置项(proxy_cache_lock和proxy_cache_lock_timeout控制). [proxy_cache_lock和proxy_cache_lock_timeout控制](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_lock) proxy_cache_lock on; #一次只允许 cache数据被1个请求更新 proxy_cache_lock_timeout 15s; # 设置cache lock的时间 ```bash proxy_cache_use_stale error timeout invalid_header http_500 http_502 http_503 http_504 http_404; # 设置几种异常情况下任然使用过期的cache数据需要注意的是对于缓存数据，是根据自己设置的proxy_cache_key进行识别的，别自己把一些非必要的变量设置在了 proxy_cache_key内，导致对应同一个URL都有不同的cache。附上完整配置文件 user app root; worker_processes auto; worker_cpu_affinity auto; worker_rlimit_nofile 100000; error_log “pipe:/opt//install/cronolog/sbin/cronolog /home/app/webserver/logs/cronolog/%Y/%m/%Y-%m-%d-error_log” warn; pid /home/app/webserver/logs/nginx.pid; events { use epoll; worker_connections 20480; } http { include mime.types; ```bash default_type application/octet-stream; root /home/app/webserver/htdocs; sendfile on; tcp_nopush on; server_tokens off; keepalive_timeout 0; client_header_timeout 1m; send_timeout 1m; client_max_body_size 3m; ...

一个因VLAN tag和MTU引发的线上故障

前段时间帮其他部门的同事排查了一个故障和MTU有点关系。同事找到我说线上的服务器访问有点问题，并直接给出了关键点:使用ping的时候 -s 1468能通，但是-s 1469就不通了。简单看了一下是个虚拟机，再综合宿主机的情况看了下。就判断出是在虚拟机上的网卡加vlan tag引发的问题。因为我们习惯性的都是在宿主机上打vlan tag，单独建个网桥，并把虚拟机的网卡桥接到这个网桥上。让同事把对应的VM业务停掉后，按照我自己的方式重新修改了宿主机和VM的配置文件，启动后一切正常。从原理上分析，vlan tag占用4个字节，以太网的MTU最大可以设置1500.当在VM内的包如果直接发了1469,那么就会造成1469+4(vlan tag)+20(ip头)+8(icmp头)>1500，就会被分片，可能在分配重组的时候丢掉了vlan 信息，造成通信异常。解决方案无非2种。1. 完全重新修改宿主机的配置，重新生成VM。vlan tag在宿主机上打.2. 修改VM网卡的MTU设置，设置为1496.这样通过MSS协商，TCP的通信会协商MSS为1496-40=1456.可以保证TCP通信正常。

使用route-map修改收到的路由

在quagga内，可以配置zebra使得对外部OSPF/BGP学习到的路由进行一些处理。比如学到了1段到192.168.0.0/16的路由，如果想自己单独指定到这段地址时使用特定IP做源IP，那么可以直接在配置 ip prefix-list INNET1 seq 5 permit 192.168.0.0/16 le 32 route-map Server2INNET1 permit 10 match ip address prefix-list INNET1 set src 10.10.7.6 这样当收到102.168.0.0/16这段路由时，实际注入kernel的时候会加上src,变成类似 192.168.0.0/16 dev eth0 proto kernel scope link src 10.10.7.6 metric 11 此时如果服务器去访问192.168.0.0/16的地址时，便自动使用10.10.7.6这个IP做源IP。也可以直接默认的源IP改掉，这样默认都使用这个IP做源IP出去。 ip prefix-list DEFAULT seq 5 permit 0.0.0.0/0 ! route-map Server2default permit 10 match ip address prefix-list DEFAULT set src 10.10.7.6 也可以在zebra内添加静态的路由，修改到特定目标网段的时使用的源IP ip route 10.0.0.0/8 10.10.6.5 ip route 10.0.0.0/8 10.10.7.5 ip route 172.16.0.0/12 10.10.6.5 ip route 172.16.0.0/12 10.10.7.5 ip prefix-list static seq 10 permit 172.16.0.0/12 ip prefix-list default seq 5 permit 0.0.0.0/0 ...