IPV6 ready

1. 政策趋势
从去年年底开始国家开始力推IPV6。目前看到的相关文件有
1. “推进互联网协议第六版(IPv6)规模部署行动计划”,这个由中共中央办公厅、国务院办公厅印发。
2. “工业和信息化部关于贯彻落实《推进互联网协议第六版(IPv6)规模部署行动计划》的通知”,这是由工信部印发。

2. 基本方案
周末有时间,把blog重新整理了一下,目前支持native IPV6 only环境访问。相关的方案
1. DNS服务器迁移到cloudflare。原本是准备自己再弄个VM,部署IPV6的DNS。后来觉得太费钱,直接用了cloudflare。
2. V6地址是通过He.net的隧道接入。

3. 注意事项
使用隧道接入V6,需要注意防火墙规则的设定,主要是以下几点:
1. 阿里云的网络安全组上放开he的隧道接入IP。
2. 服务器上itables-v4放开he的隧道接入IP。
3. 服务器上的ip6tables打开相关的端口。
4. nginx的上的geoip库都使用IPV6(V6的包括了V4)的

geoip_country /usr/share/GeoIP/GeoIPv6.dat;
geoip_city /usr/share/GeoIP/GeoIPCityv6.dat;
log_format main '$remote_addr - "$geoip_city_country_code:$geoip_city" - $remote_addr:$remote_port - $remote_user [$time_local] $host "$request"'
' $status $body_bytes_sent "$http_referer" '
'"$http_user_agent - $agent - $upstream_cache_status" "$http_x_forwarded_for" "$upstream_addr" "$ssl_protocol $ssl_cipher http=$http2" $request_time $upst
ream_response_time $tcpinfo_rtt';

相关的防火墙规则如下:
v4规则

# Generated by iptables-save v1.4.21 on Mon Jun 11 09:07:48 2018
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [154797:201811368]
-A INPUT -s 216.218.221.6/32 -j ACCEPT
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 443 -j ACCEPT
-A INPUT -p udp -m state --state NEW -m udp --dport 53 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 53 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
COMMIT
# Completed on Mon Jun 11 09:07:48 2018

v6规则

*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [56:3808]
:OUTPUT ACCEPT [7834:1611396]
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p ipv6-icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 443 -j ACCEPT
-A INPUT -p udp -m state --state NEW -m udp --dport 53 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 53 -j ACCEPT
-A INPUT -j DROP
COMMIT
# Completed on Mon Jun 11 09:08:18 2018

4. 参考链接
1. http://www.gov.cn/zhengce/2017-11/26/content_5242389.htm
2. http://www.miit.gov.cn/n1146295/n1652858/n1652930/n3757020/c6154756/content.html

发表在 System | IPV6 ready已关闭评论

配置802.11r提升家用AP漫游体验

问题

网络的拓扑图大致如下

虽然在客厅和主卧都有部署无线路由器,2个AP的SSID一致,实际使用的时候从客厅走入主卧时实际接入端还是在连接之前的AP。另外也经常出现电脑一会连AP1,一会连AP2导致无线中断。
所有没有AC+AP的方案,都会存在以上的情况,为了提升体验就在openwrt上配置好802.11r,结果还是比较满意的。

配置方案

所有的AP需要都是桥接在同一个LAN中,我的2个WNDR3700都是把路由器的WAN口设置一个静态的IP(方便登陆管理),另外无线网络都是桥接到WAN上,所有AP的客户端的IP实际都是走光猫DHCP。

配置802.11r实际比较简单,只需要在openwrt上把wpad安装一下(默认是wpad-mini),涉及的配置如下:
1. NASID: PMK-R0 Key Holder identifier,每个AP不同,可以设置为MAC地址
2. Mobility Domain: 每个AP相同,随便设置4位字符
3. R0 Key Lifetime: 每个AP相同,可以就用默认值10000
4. R1 Key Holder: 每个AP不同,可以设置为MAC地址
5. Reassociation Deadline: 每个AP相同,可以设置为默认的1000
6. r0kh(External R0 Key Holder List),格式Valid format: ,,<128-bit key as hex string> ,每个设备的配置都是相同的,可以按照格式:
– MAC,NASID,32位字符串
7. r1kh (External R1 Key Holder List),格式:,,<128-bit key as hex string> ,每个设备相同,可以按照格式:
– MAC,MAC,32位字符串
我的实际配置如下:

External R0 Key Holder List:
  - A4:2B:8C:0C:D7:B3,A42B8C0CD7B3,8a7fcc966ed0691ff2809e1f38c16999
  - 04:A1:51:9B:0D:25,04A1519B0D25,8a7fcc966ed0691ff2809e1f38c16999
External R1 Key Holder List:
  - A4:2B:8C:0C:D7:B3,A4:2B:8C:0C:D7:B3,8a7fcc966ed0691ff2809e1f38c16999
  - 04:A1:51:9B:0D:25,04:A1:51:9B:0D:25,8a7fcc966ed0691ff2809e1f38c16999
发表在 System | 配置802.11r提升家用AP漫游体验已关闭评论

cobbler配置

流程记录

  1. 软件包安装 并关闭SELinux

– 配置epel源,直接使用yum安装

yum install cobbler cobbler-web dhcp bind pykickstart tftp -y
  • 需要注意dhcp/bind需要单独安装一下,cobbler没依赖dhcp/bind
sed -i 's#SELINUX=enforcing#SELINUX=disabled#g' /etc/selinux/config
setenforce 0
systemctl restart cobblerd
  1. 配置cobbler参数

– 修改/etc/cobbler/settings的参数

allow_dynamic_settings: 1
default_password_crypted: "$1$random-p$yqLCVPP/OBjIT02WekZic1"
next_server: 192.168.117.134
manage_dhcp: 1
manage_dns: 1
server: 192.168.117.134
manage_rsync: 1
pxe_just_once: 1 #防止循环装机
manage_forward_zones: ['gnuers.org']
manage_reverse_zones: ['10.0.0', '192.168', '172.16.123']
  • 修改dhcp池配置 /etc/cobbler/dhcp.template
subnet 192.168.117.0 netmask 255.255.255.0 {
option routers 192.168.117.2;
option domain-name-servers 192.168.117.2;
option subnet-mask 255.255.255.0;
range dynamic-bootp 192.168.117.10 192.168.117.20;
  • 修改/etc/cobbler/named.template配置,named监听所有IP
options {
listen-on port 53 { any; };
directory "/var/named";
dump-file "/var/named/data/cache_dump.db";
statistics-file "/var/named/data/named_stats.txt";
memstatistics-file "/var/named/data/named_mem_stats.txt";
allow-query { localhost; };
recursion yes;
};
  • 获取网络loader(本机有syslinux可以不安装)
cobbler get-loaders
  • 开启cobbler服务
systemctl enable cobblerd && systemctl start cobblerd
systemctl enable httpd && systemctl start httpd
systemctl enable rsyncd.service && systemctl start rsyncd.service
systemctl enable tftp && systemctl start tftp
  • 配置cobbler-web的密码
htdigest /etc/cobbler/users.digest "Cobbler" cobbler

配置后可以打开https://192.168.117.134/cobbler_web 登陆web页面
– 确认cobbler配置

cobbler check
  1. 导入安装镜像

– 挂载ISO到本地并导入cobbler
直接导入会给建立一个默认的profile,使用的ks是 /var/lib/cobbler/kickstarts/sample_end.ks

mount CentOS-7u2.iso /mnt/CentOS7U2
cobbler import --arch=x86_64 --breed=redhat --os-version=rhel7 --path=/mnt/CentOS7U2/ --name=CentOS7U2
  • 查看命令
cobbler profile list
cobbler profile remove --name CentOS7U2-x86_64
cobbler distro list
cobbler distro remove --name CentOS7U2-x86_64
  1. 添加自己的profile

– 先配置自己的ks文件,再做profile添加
新增自己修改过的ks文件到var/lib/cobbler/kickstarts/7u2diy.ks

cobbler profile add --name=Centos7U2-DIY --distro=CentOS7U2-x86_64 --kickstart=/var/lib/cobbler/kickstarts/7u2diy.ks
  • 设置自动运行的profile
    直接修改/var/lib/tftpboot/pxelinux.cfg/default 中ONTIMEOUT为Centos7U2-DIY
  1. 创建system给特定的机器按指定的模板做自动安装
cobbler system add --name=VMDIY --profile=Centos7U2-DIY --interface=eth0 --mac=00:0c:29:e7:a2:e4 --gateway=192.168.117.2 --ip-address=192.168.117.66 --netmask=255.255.255.0 --static=1 --dns-name=test.alipay.com

mac地址为00:0c:29:e7:a2:e4的服务器开启后会按预设的模板进行装机,并分配指定IP
如果想重装已经装过的服务器,需要先设置netboot标志

cobbler system edit --name=VMDIY --netboot-enabled=1
cobbler sync

KS文件

firewall --disabled
auth --enableshadow --passalgo=sha512
url --url=$tree
text
firstboot --enable
ignoredisk --only-use=sda
keyboard --vckeymap=cn --xlayouts='cn'
lang zh_CN.UTF-8
selinux --disabled
network --bootproto=dhcp --device=eth0 --onboot=yes --ipv6=auto --activate
network --hostname=Centos
rootpw --iscrypted $6$BMgqLp5skYPt6XbU$OQJIOjkvBS2l9Cykbudrtbz8Ym/F9Oc6B9IINXmzunY0pxcWSzzBucTYMV.4bnrZL8.cuhVVPaTRREwksk7Fx.
services --enabled="chronyd"
timezone Asia/Shanghai --isUtc
user --name=pm --password=$6$mYf50m5qC0pf4.ta$0k6i9qdr2I67DAN0C4ToOMF0Kh6mAUVf9A8oNP3WU.VJrcQwpMT1JsyArvopUxCH1Sq/NnAMur8RzTfYQUVux1 --iscrypted --gecos="pm"
bootloader --append="net.ifnames=0 biosdevname=0" --location=mbr --boot-drive=sda
clearpart --none --initlabel
part /home --fstype="ext4" --ondisk=sda --size=8192
part swap --fstype="swap" --ondisk=sda --size=1023
part /boot --fstype="ext4" --ondisk=sda --size=2048
part / --fstype="ext4" --ondisk=sda --size=9214
%packages
@^minimal
@core
chrony
kexec-tools
vim
wget
bind-utils
keepalived
quagga
rsync
%end
%addon com_redhat_kdump --enable --reserve-mb='auto'
%end
%anaconda
pwpolicy root --minlen=6 --minquality=1 --notstrict --nochanges --notempty
pwpolicy user --minlen=6 --minquality=1 --notstrict --nochanges --emptyok
pwpolicy luks --minlen=6 --minquality=1 --notstrict --nochanges --notempty
%end
%post --interpreter=/bin/bash
cd /etc/yum.repos.d/ && rm -frv *
/bin/cat </etc/yum.repos.d/CentOS-Base.repo
[base]
name=CentOS-$releasever - Base - mirrors.aliyun.com
failovermethod=priority
baseurl=http://mirrors.aliyun.com/centos/$releasever/os/$basearch/
gpgcheck=1
gpgkey=http://mirrors.aliyun.com/centos/RPM-GPG-KEY-CentOS-7
[updates]
name=CentOS-$releasever - Updates - mirrors.aliyun.com
failovermethod=priority
baseurl=http://mirrors.aliyun.com/centos/$releasever/updates/$basearch/
gpgcheck=1
gpgkey=http://mirrors.aliyun.com/centos/RPM-GPG-KEY-CentOS-7
[extras]
name=CentOS-$releasever - Extras - mirrors.aliyun.com
failovermethod=priority
baseurl=http://mirrors.aliyun.com/centos/$releasever/extras/$basearch/
gpgcheck=1
gpgkey=http://mirrors.aliyun.com/centos/RPM-GPG-KEY-CentOS-7
[centosplus]
name=CentOS-$releasever - Plus - mirrors.aliyun.com
failovermethod=priority
baseurl=http://mirrors.aliyun.com/centos/$releasever/centosplus/$basearch/
gpgcheck=1
enabled=0
gpgkey=http://mirrors.aliyun.com/centos/RPM-GPG-KEY-CentOS-7
[contrib]
name=CentOS-$releasever - Contrib - mirrors.aliyun.com
failovermethod=priority
baseurl=http://mirrors.aliyun.com/centos/$releasever/contrib/$basearch/
gpgcheck=1
enabled=0
gpgkey=http://mirrors.aliyun.com/centos/RPM-GPG-KEY-CentOS-7
)2>&1 >> /root/post-install.log
%end

命令说明

  1. 镜像相关的操作
[root@Centos kickstarts]# cobbler distro ##
usage
=====
cobbler distro add
cobbler distro copy
cobbler distro edit
cobbler distro find
cobbler distro list
cobbler distro remove
cobbler distro rename
cobbler distro report
  1. 安装模板配置
[root@Centos kickstarts]# cobbler profile
usage
=====
cobbler profile add
cobbler profile copy
cobbler profile dumpvars
cobbler profile edit
cobbler profile find
cobbler profile getks
cobbler profile list
cobbler profile remove
cobbler profile rename
cobbler profile report
  1. 添加发型版支持
    编辑 /var/lib/cobbler/distro_signatures.json
[root@Centos kickstarts]# cobbler signature reload
usage
=====
cobbler signature reload
cobbler signature report
cobbler signature update

遇到的问题

  1. 内存不足,VM 只分了1G 内存
mount: wrong fs type, bad option, bad superblock on /dev/loop0
missing codepage or helper program, or other error
in some cases useful info is found in syslog - try
dmesg | tail or so
umount: /run/initramfs/squashfs: not mounted
/sbin/dmsquash-live-root: line 273: printf: write error: no space left on device
  1. 开启gpxe后无法安装
    因为渲染出来的启动菜单的参数有问题,启动不了的时候都可以看看pxe的菜单内的参数是否正常 /var/lib/tftpboot/pxelinux.cfg/default

参考

  1. https://yhuan.online/index.php/automation/cobbler.html
  2. https://wsgzao.github.io/post/cobbler/
  3. http://cobbler.github.io/manuals/quickstart/
  4. http://cobbler.github.io/manuals/2.8.0/
发表在 System | cobbler配置已关闭评论

bind 9.11和9.12简单测试总结

bind 9.11 实际已经发布很久了,之前是简单的做过测试,简单做个总结。
从功能上说9.11的几个特点:
1. 持续完善了9.10 开始有的prefetch 功能
2. 相比9.10 RPZ 的性能得到提升。
3. 支持了dnstap。
4. 支持了Catalog zone,zone的增加删除更加方便。
5. 终于支持了ECS(EDNS Client-Subnet)。

9.12在9.11的基础上主要是新增了
1. stale-answer ,递归失败的时候使用历史记录做响应。
2. 完善了dnstap的文件轮转。

实际做了一些测试,简单的总结如下:
1. 9.11.2 默认开始了cookie,实测9.11.2在递归的时候失败率相比9.9版本明显增加,性能上有少量的提升。
2. dnstap对性能的影响比较小,关闭query log后 dnstap 关闭/开启的性能对比大概是13W/S VS 10W/S。相比传统的querylog的性能影响实在好太多了。但是因为是二进制的文件,查看需要用dnstap-read还是非常不方便的。
3. 9.12的stale-answer还很不完善,最主要的问题是还是会先尝试同步做一次递归,失败了再用历史记录响应,测试中也经常出现找不到历史缓存的情况。目前了解是的akamai收购了nominum,这个patch应该是nominum 提供的。
4. bind 9.10/9.11/9.12都支持了 EDNS Client-Subnet扩展协议,不过实际配置是比较脑残的。估计再等一两个版本会好一点。

5. bind 9.11/9.12目前用于生产环境的风险还是比较高,建议企业用户继续使用9.9。
整体来说这两年bind的发展相比之前还是快了很多,只是dns这个领域目前国内厂商因为国内运营商的各种奇葩要求做的工作还是很多的,无论是从性能,管控的便捷程度来说国内的成熟度都是超过国外的(除了对新扩展协议的跟进支持)。

发表在 dns | bind 9.11和9.12简单测试总结已关闭评论

openssh 7.6p1编译打包

新版本代码获取

从官网下载 https://openbsd.hk/pub/OpenBSD/OpenSSH/portable/openssh-7.6p1.tar.gz

编译RPM包注意事项

需要注意新版的openssh的spec文件里使用的pam配置文件会导致无法登陆

#%PAM-1.0
auth       required     pam_stack.so service=system-auth
account    required     pam_nologin.so
account    required     pam_stack.so service=system-auth
password   required     pam_stack.so service=system-auth
session    required     pam_stack.so service=system-auth

需要修改为

#%PAM-1.0
auth       required     pam_sepermit.so
auth       include      password-auth
account    required     pam_nologin.so
account    include      password-auth
password   include      password-auth
# pam_selinux.so close should be the first session rule
session    required     pam_selinux.so close
session    required     pam_loginuid.so
# pam_selinux.so open should only be followed by sessions to be executed in the user context
session    required     pam_selinux.so open env_params
session    optional     pam_keyinit.so force revoke
session    include      password-auth

spec文件在./contrib/redhat下,可以直接rpmbuild -ba openssh.spec 编译。

发表在 System | 留下评论

各导航系统使用的时钟

概述

美国GPS导航卫星采用了铯原子钟和铷原子钟结合的方式;

欧盟Galileo导航卫星采用了铷原子钟和被动型氢原子钟结合的方式;

俄罗斯Glonass-K三代导航卫星也将采用铷原子钟和被动型氢原子钟结合的方式。

北斗导航卫星正在开展铷原子钟和被动型氢原子钟相结合的授时方式。

铷原子钟:体积小重量轻、功耗低、技术难度相对较低、可靠性高等优势,但长期稳定度和漂移率指标相对较差;

铯原子钟:低漂移特性,不过寿命是致命短板;

被动型氢原子钟:稳定度指标在传统三样中最优,研发难道高。

1. GPS

早期的GPS使用的霍普金斯大学研制的石英振荡器,相对频率稳定度为10^-11/秒。误差为14m。

1974年以后,GPS卫星采用铷原子钟,相对频率稳定度达到 10^-12/秒,误差8m。

1977年,BOKCK II型采用了马斯频率和时间系统公司研制的铯原子钟后,相对稳定频率达到10^-13/秒,误差再降为2.9m。

1981年,休斯公司研制的相对稳定频率为10^-14/秒的氢原子钟使BLOCK IIR型卫星误差降至仅为1m。

2. 北斗

  1. 早期北斗试验系统的卫星原子钟是由瑞士进口
  2. 北斗二号的星载原子钟逐渐开始使用中国航天科工二院203所提供的国产铷原子钟。
  3. 2015年有发射载有星载氢原子钟的北斗卫星。
  4. 203所2017年9月30日成功研发了星载铯钟,预计北斗三号可能使用。
发表在 NTP | 各导航系统使用的时钟已关闭评论

tshark实时抓包获取DNS请求信息

1. tshark的安装

CentOS6 YUM源内的wireshark安装后,不支持GeoIP,需要自己编译一下最新的版本。

yum install  GeoIP GeoIP-devel geoipupdate -y
./configure  --with-geoip=/usr/share/GeoIP/ --enable-tshark=yes
make rpm-package

wireshark自身Makefile带了各发型版的打包功能,所以直接make rpm-package就能make出rpm包

2. 使用范例

  1. 抓取DNS请求
sudo /usr/local/bin/tshark   -i eth0  -o "ip.use_geoip:TRUE" -Y "udp.dstport == 53" -T fields -E separator='|' -e ip.src -e ip.geoip.src_country -e ip.geoip.src_asnum -e dns.flags -e dns.id -e dns.qry.name -e dns.qry.type -e  dns.count.answers  -e dns.count.answers -e dns.flags.rcode  -e  ip.len

抓到的内容如下:

211.138.19.28|China,China|AS24445 Henan Mobile Communications Co.,Ltd,AS24445 Henan Mobile Communications Co.,Ltd|0x00000010|0x0000355f|fxxxx.com|28||0||82
221.204.186.218|China,China|AS4837 CNCGROUP China169 Backbone,AS4837 CNCGROUP China169 Backbone|0x00000000|0x00004111|xxxx.com|1||0||75
123.157.135.3|China,China|AS4837 CNCGROUP China169 Backbone,AS4837 CNCGROUP China169 Backbone|0x00000000|0x00001108|xxxx.com|1||0||85
58.30.131.56|China,China|AS9811 srit corp.,beijing.,AS9811 srit corp.,beijing.|0x00000000|0x00005901|xxxxx.com|28||0||82
101.226.66.17|China,China|AS4812 China Telecom (Group),AS4812 China Telecom (Group)|0x00000010|0x00008d2a|xxxx.com|1||0||92
60.31.184.168|China,China|AS4837 CNCGROUP China169 Backbone,AS4837 CNCGROUP China169 Backbone|0x00000000|0x00005b22|xxxxx.com|1||0||79
60.31.184.168|China,China|AS4837 CNCGROUP China169 Backbone,AS4837 CNCGROUP China169 Backbone|0x00000000|0x0000da7e|lxxxx.com|1||0||74
74.125.176.202|United States,United States|AS15169 Google Inc.,AS15169 Google Inc.|0x00000000|0x0000ca93|xxxx.com|1||0||79
  1. 抓取响应内容
sudo /usr/local/bin/tshark   -i eth0  -o "ip.use_geoip:TRUE" -Y "udp.srcport == 53" -T fields -E separator='|' -e ip.src -e ip.geoip.src_country -e ip.geoip.src_asnum -e dns.flags -e dns.id -e dns.qry.name -e dns.qry.type -e  dns.count.answers  -e dns.count.answers -e dns.flags.rcode  -e  ip.len

3. 参考文档

  1. 支持的DNS字段 https://www.wireshark.org/docs/dfref/d/dns.html
发表在 dns | 留下评论

使用TLS增强docker的安全性

之前部署docker的时候都是没对docker的HTTP/socker接口进行加密。最近在线上部署的时候就得考虑这个。使用证书对docker进行加密主要参考官方的文档:
1. https://docs.docker.com/v1.13/engine/security/https/
2. https://github.com/docker/swarm/issues/341

docker engine配置TLS

主要注意做swarm cluster的时候 需要签发证书的时候subjectAltName里把本机的IP。生成证书的从网上找到的一个脚本做了修改,直接在里面把集群的IP都填进去,这样每个机器可以证书相同:

#!/bin/bash
# This script will help you setup Docker for TLS authentication.
# Run it passing in the arguement for the FQDN of your docker server
#
# For example:
#    ./create-docker-tls.sh myhost.docker.com
#
# The script will also create a profile.d (if it exists) entry
# which configures your docker client to use TLS
#
# We will also overwrite /etc/sysconfig/docker (again, if it exists) to configure the daemon.
# A backup will be created at /etc/sysconfig/docker.unixTimestamp
#
# MIT License applies to this script.  I don't accept any responsibility for
# damage you may cause using it.
#

set -e
STR=2048
if [ "$#" -gt 1 ]; then
  DOCKER_HOST1="$1"
  DOCKER_HOST2="$2"
else
  echo " => ERROR: You must specify the docker FQDN as the first arguement to this scripts! <="
  exit 1
fi

if [ "$USER" == "root" ]; then
  echo " => WARNING: You're running this script as root, therefore root will be configured to talk to docker"
  echo " => If you want to have other users query docker too, you'll need to symlink /root/.docker to /theuser/.docker"
fi

echo " => Using : $DOCKER_HOST1  You MUST connect to docker using this host!"

echo " => Ensuring config directory exists..."
cd ./cert

echo " => Verifying ca.srl"
if [ ! -f "ca.src" ]; then
  echo " => Creating ca.srl"
  echo 01 > ca.srl
fi

echo " => Generating CA key"
openssl genrsa \
  -out ca-key.pem $STR

echo " => Generating CA certificate"
openssl req \
  -new \
  -key ca-key.pem \
  -x509  \
  -sha256  \
  -days 3650 \
  -nodes \
  -subj "/CN=$" \
  -out ca.pem

echo " => Generating server key"
openssl genrsa \
  -out server-key.pem $STR

echo " => Generating server CSR"
openssl req \
  -subj "/CN=$DOCKER_HOST1" \
  -new \
  -sha256  \
  -key server-key.pem \
  -out server.csr

echo " => Signing server CSR with CA"
echo subjectAltName = "DNS:$DOCKER_HOST1,DNS:$DOCKER_HOST2,IP:127.0.0.1,IP:XXXXXX,IP:XXXXXX,IP:XXXXX,IP:XXXXX"  > extfile-server.cnf
openssl x509 \
  -req \
  -days 3650 \
   -sha256  \
  -in server.csr \
  -CA ca.pem \
  -CAkey ca-key.pem \
  -out server-cert.pem \
  -extfile extfile-server.cnf

echo " => Generating client key"
openssl genrsa \
  -out key.pem $STR

echo " => Generating client CSR"
openssl req \
  -subj "/CN=docker.client" \
  -new \
  -key key.pem \
  -out client.csr

echo " => Creating extended key usage"
echo extendedKeyUsage = clientAuth > extfile.cnf

echo " => Signing client CSR with CA"
openssl x509 \
  -req \
  -days 3650 \
  -sha256  \
  -in client.csr \
  -CA ca.pem \
  -CAkey ca-key.pem \
  -out cert.pem \
  -extfile extfile.cnf

if [ -d "/etc/profile.d" ]; then
  echo " => Creating profile.d/docker"
  sudo sh -c "echo '#!/bin/bash
export DOCKER_CERT_PATH=/home/$USER/.docker
export DOCKER_HOST=tcp://$DOCKER_HOST1:2376
export DOCKER_TLS_VERIFY=1' > /etc/profile.d/docker.sh"
  sudo chmod +x /etc/profile.d/docker.sh
  source /etc/profile.d/docker.sh
else
  echo " => WARNING: No /etc/profile.d directoy on your system."
  echo " =>   You will need to set the following environment variables before running the docker client:"
  echo " =>   DOCKER_HOST=tcp://$DOCKER_HOST1:2376"
  echo " =>   DOCKER_TLS_VERIFY=1"
fi

OPTIONS="--tlsverify --tlscacert=$HOME/.docker/ca.pem --tlscert=$HOME/.docker/server-cert.pem --tlskey=$HOME/.docker/server-key.pem -H=0.0.0.0:2376"
if [ -f "/etc/sysconfig/docker" ]; then
  echo " => Configuring /etc/sysconfig/docker"
  BACKUP="/etc/sysconfig/docker.$(date +"%s")"
  sudo mv /etc/sysconfig/docker $BACKUP
  sudo sh -c "echo '# The following line was added by ./create-certs docker TLS configuration script
OPTIONS=\"$OPTIONS\"
# A backup of the old file is at $BACKUP.' >> /etc/sysconfig/docker"
  echo " => Backup file location: $BACKUP"
else
  echo " => WARNING: No /etc/sysconfig/docker file found on your system."
  echo " =>   You will need to configure your docker daemon with the following options:"
  echo " =>   $OPTIONS"
fi

export DOCKER_HOST=tcp://DOCKER_HOST:2376
export DOCKER_TLS_VERIFY=1
echo " => Done! You just need to restart docker for the changes to take effect"

附上docker.service

[Unit]
Description=Docker Application Container Engine
Documentation=http://docs.docker.com
After=network.target
Wants=docker-storage-setup.service

[Service]
Type=notify
EnvironmentFile=-/etc/sysconfig/docker
EnvironmentFile=-/etc/sysconfig/docker-storage
EnvironmentFile=-/etc/sysconfig/docker-network
Environment=GOTRACEBACK=crash
ExecStart=/usr/bin/dockerd  -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock $OPTIONS   \
           --tlsverify --tlscacert=/etc/docker/cert/ca.pem --tlscert=/etc/docker/cert/server-cert.pem --tlskey=/etc/docker/cert/server-key.pem \
           --storage-driver=overlay \
           --cluster-store etcd://xxxxxx:2379/vxlan \
           --cluster-advertise=bond0:2375 \
          $DOCKER_STORAGE_OPTIONS \
          $DOCKER_NETWORK_OPTIONS \
          $ADD_REGISTRY \
          $BLOCK_REGISTRY \
          $INSECURE_REGISTRY
LimitNOFILE=1048576
LimitNPROC=1048576
LimitCORE=infinity
MountFlags=slave
TimeoutStartSec=1min
Restart=on-failure

[Install]
WantedBy=multi-user.target

swarm使用TLS

  1. 创建manage
sudo docker run --restart=always -v /etc/docker/cert/:/cert/ --name swarm-manage -d -p 8888:2375 swarm -l debug  manage  --tlsverify --tlscacert=/cert/ca.pem --tlscert=/cert/server-cert.pem --tlskey=/cert/server-key.pem  etcd://xxxxx:2379/swarm
  1. 启动agent
sudo docker run --restart=always --name swarm-agent -d  swarm join --addr=`hostname -i`:2375  etcd://xxxxx:2379/swarm

使用TLS连接swarm

$export DOCKER_HOST=tcp://xxxxx:8888 DOCKER_TLS_VERIFY=1
$docker version
Client:
 Version:      1.13.1
 API version:  1.24 (downgraded from 1.26)
 Go version:   go1.7.5
 Git commit:   092cba3
 Built:        Wed Feb  8 06:38:28 2017
 OS/Arch:      linux/amd64

Server:
 Version:      swarm/1.2.6
 API version:  1.22 (minimum version )
 Go version:   go1.7.1
 Git commit:   `git rev-parse --short HEAD`
 Built:        `date -u`
 OS/Arch:      linux/amd64
 Experimental: false
发表在 docker | 留下评论

利用BGP community黑洞路由

场景

在被攻击的时候,当入口无法承受巨大的流量时大家采用的方式是切换业务IP,然后把之前的IP做黑洞。
在与ISP对接时,每个ISP都有自己的BGP配置规范。接入方可以参考commuity属性对自己的路由做很多设置,包括MED,Localpref,AS-PATH 添加、路由定向宣告等,另外一个常用的就是黑洞某条路由

模拟拓扑


测试的环境有4个路由器:
– R1:企业路由器
– R2:ISP路由器
– R3:其他ISP的路由器
– R4: 其他ISP的客户

测试的方案

先把R1-R4的BGP调通,然后分别按下属操作:
1. R1上添加prefix-list把5.5.5.6/32这个明细路由直接发送给R2,并设置community属性4134:666(电信的黑洞属性).
2. R2上添加对community 4134:666的匹配操作

ip community-list  standard  cm-blackhole permit 4134:666
route-map out-filter permit 20
    match community cm-blackhole
    set local-preference 10
    set ip next-hop 172.20.20.1
    set community additive no-export
route-map out-filter permit 30
    set local-preference 30
    set metric 30

可以观察在R1-R4上的路由情况:

R1 路由

094846cab3a9# show ip bgp
BGP table version is 0, local router ID is 10.10.0.22
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
              i internal, r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 4.4.4.0/24       0.0.0.0                121          32768 ?
*> 5.5.5.0/24       0.0.0.0                121          32768 ?
*> 5.5.5.6/32       0.0.0.0                121          32768 ?
*> 6.6.6.0/24       0.0.0.0                121          32768 ?
*> 8.8.8.0/24       0.0.0.0                121          32768 ?
*  10.10.0.0/16     10.10.0.23             121              0 65010 ?
*>                  0.0.0.0                121          32768 ?
*> 100.100.100.1/32 0.0.0.0                121          32768 ?
*> 100.100.100.2/32 10.10.0.23             121              0 65010 ?
*> 100.100.100.3/32 10.10.0.23                             0 65010 65002 ?
*> 100.100.100.4/32 10.10.0.23                             0 65010 65002 65003 ?
*  172.18.0.0       10.10.0.23             121              0 65010 ?
*>                  0.0.0.0                121          32768 ?

Displayed  11 out of 13 total prefixes
094846cab3a9# show ip bgp neighbors 10.10.0.23 advertised-routes
BGP table version is 0, local router ID is 10.10.0.22
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
              i internal, r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 4.4.4.0/24       10.10.0.22             121          32768 ?
*> 5.5.5.0/24       10.10.0.22             121          32768 ?
*> 5.5.5.6/32       10.10.0.22             121          32768 ?
*> 8.8.8.0/24       10.10.0.22             121          32768 ?
*> 100.100.100.1/32 10.10.0.22             121          32768 ?

R2路由

05fe39a5b056# show ip bgp neighbors 10.10.0.22 routes
BGP table version is 0, local router ID is 10.10.0.23
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
              i internal, r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 4.4.4.0/24       10.10.0.22             110              0 65001 65001 ?
*> 5.5.5.0/24       10.10.0.22             110              0 65001 65001 ?
*> 5.5.5.6/32       10.10.0.22             110              0 65001 65001 ?
*> 8.8.8.0/24       10.10.0.22             100     250       0 65010 65001 ?
*> 100.100.100.1/32 10.10.0.22             100     250       0 65010 65001 ?

Displayed  5 out of 12 total prefixes
05fe39a5b056# show ip bgp 5.5.5.6/32
BGP routing table entry for 5.5.5.6/32
Paths: (1 available, best #1, table Default-IP-Routing-Table)
  Advertised to non peer-group peers:
  10.10.0.24
  65001 65001
    10.10.0.22 from 10.10.0.22 (10.10.0.22)
      Origin incomplete, metric 110, localpref 100, valid, external, best
      Community: 4134:666
      Last update: Tue Mar 14 07:17:16 2017

可以看到R2收到的 5.5.5.6/32路由具有4134:666这个community属性。
然后再看看R3的

R3 路由

cc6a781cbc3a# show ip bgp neighbors  10.10.0.23 routes
BGP table version is 0, local router ID is 10.10.0.24
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
              i internal, r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 4.4.4.0/24       10.10.0.23              30             0 65010 65001 65001 ?
*> 5.5.5.0/24       10.10.0.23              30             0 65010 65001 65001 ?
*> 5.5.5.6/32       172.20.20.1                            0 65010 65001 65001 ?
*> 8.8.8.0/24       10.10.0.23              30             0 65010 65010 65001 ?
*  10.10.0.0/16     10.10.0.23              30             0 65010 ?
*> 100.100.100.1/32 10.10.0.23              30             0 65010 65010 65001 ?
*> 100.100.100.2/32 10.10.0.23              30             0 65010 ?
*  172.18.0.0       10.10.0.23              30             0 65010 ?

Displayed  8 out of 14 total prefixes
cc6a781cbc3a# show ip bgp 5.5.5.6/32
BGP routing table entry for 5.5.5.6/32
Paths: (1 available, best #1, table Default-IP-Routing-Table, not advertised to EBGP peer)
  Not advertised to any peer
  65010 65001 65001
    172.20.20.1 from 10.10.0.23 (10.10.0.23)
      Origin incomplete, localpref 100, valid, external, best
      Community: 4134:666 no-export
      Last update: Tue Mar 14 07:17:44 2017

可以看到R2把我们想要黑洞的路由5.5.5.6/32转发给R3时,按照需求标记了 no-export属性,并把路由的下一条改到了不存的一个IP 172.20.20.1(quagga上不能直接写127.0.0.1,会导致邻居无法建立)。

R4路由

db71d04826e4# show ip bgp neighbors 10.10.0.24 routes
BGP table version is 0, local router ID is 10.10.0.25
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
              i internal, r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 4.4.4.0/24       10.10.0.24                             0 65002 65010 65001 65001 ?
*> 5.5.5.0/24       10.10.0.24                             0 65002 65010 65001 65001 ?
*> 8.8.8.0/24       10.10.0.24                             0 65002 65010 65010 65001 ?
*  10.10.0.0/16     10.10.0.24             121              0 65002 ?
*> 100.100.100.1/32 10.10.0.24                             0 65002 65010 65010 65001 ?
*> 100.100.100.2/32 10.10.0.24                             0 65002 65010 ?
*> 100.100.100.3/32 10.10.0.24             121              0 65002 ?
*  172.18.0.0       10.10.0.24             121              0 65002 ?

从上面可以看到R4上完全没有5.5.5.6/32的路由,虽然/24的路由指向了R3,但是因为R3本地5.5.5.6/32的路由被指向了一个无效的IP,因此从R4访问被黑洞的IP 5.5.5.6/32的流量就止于R3。在实际的运营商网络中,一般会在路由器上把这个 172.20.20.1 设置为NULL0,并丢弃其所有的流量。

完整配置

附上完整的配置:

R1   
----
log file /var/log/quagga/bgpd.log
password bgp
router bgp 65001
 bgp router-id 10.10.0.22
 redistribute connected metric 121
 neighbor 10.10.0.23 remote-as 65010
 neighbor 10.10.0.23 password DOCKER
 neighbor 10.10.0.23 ebgp-multihop 255
 neighbor 10.10.0.23 next-hop-self
 neighbor 10.10.0.23 route-map out-filter  out
 distance bgp 250 200 150
!
!
ip prefix-list blackhole seq 5 permit 5.5.5.6/32
!ip prefix-list blackhole seq 10 permit 5.5.5.0/24
ip prefix-list r1-out seq 5 permit 4.4.4.0/24
ip prefix-list r1-out seq 6 permit 5.5.5.0/24
ip prefix-list r1-out seq 11 permit 8.8.8.0/24
ip prefix-list r1-out seq 15 permit 100.100.100.0/23 ge 24
ip prefix-list r1-out seq 25 permit 10.0.0.0/8
ip prefix-list r1-out seq 50 deny any
!
route-map out-filter permit 5
 match ip address prefix-list  blackhole
 set community 4134:666

route-map out-filter permit 10
 match ip address prefix-list  r1-out

!

R2
---

log file /var/log/quagga/bgpd.log
password bgp
router bgp 65010
 distance bgp 250  200 150
 bgp router-id 10.10.0.23
 neighbor 10.10.0.22 remote-as  65001
 neighbor 10.10.0.24 remote-as  65002
 neighbor 10.10.0.22 password DOCKER
 neighbor 10.10.0.24 password DOCKER
 neighbor 10.10.0.22 route-map in-filter in
 neighbor 10.10.0.24 route-map out-filter out
 neighbor 10.10.0.22 ebgp-multihop
 neighbor 10.10.0.24 ebgp-multihop
 neighbor 10.10.0.22 next-hop-self
 neighbor 10.10.0.24 next-hop-self
 redistribute connected  metric 121
 access-list all permit any
ip prefix-list from-r1-in seq 5 permit 4.4.4.0/24
ip prefix-list from-r1-in seq 6 permit 5.5.5.0/24 le 32
!ip prefix-list from-r1-in seq 7 permit 8.8.8.0/24
!ip prefix-list from-r1-in seq 15 permit 100.100.100.0/24 le 32
ip prefix-list from-r1-in seq 20 permit 10.0.0.0/8
ip prefix-list from-r1-in seq 50 deny any

ip prefix-list from-r1-in-t1 seq 7 permit 8.8.8.0/24 le 32
ip prefix-list from-r1-in-t1 seq 15 permit 100.100.100.0/24 le 32

ip prefix-list to-r3 seq 5 permit any


ip community-list  standard  cm-blackhole permit 4134:666
!ip community-list  standard  cm-blackhole permit

route-map out-filter permit 20
 match community cm-blackhole
 set local-preference 10
 set ip next-hop 172.20.20.1
 set community additive no-export
route-map out-filter permit 30
 set local-preference 30
 set metric 30

route-map in-filter permit 5
  match ip address prefix-list from-r1-in-t1
   set as-path prepend 65010
   set metric 100
   set local-preference 250
   set community 65002:4134

route-map in-filter permit 10
 match ip address prefix-list from-r1-in
 set as-path prepend 65001
 set metric 110

 R3
 ---
 log file /var/log/quagga/bgpd.log
password bgp
router bgp 65002
 distance bgp 250  200 150
 bgp router-id 10.10.0.24
 neighbor 10.10.0.23 remote-as  65010
 neighbor 10.10.0.23 password DOCKER
 neighbor 10.10.0.23 ebgp-multihop
 neighbor 10.10.0.23 next-hop-self
 neighbor 10.10.0.25 remote-as  65003
 neighbor 10.10.0.25 password DOCKER
 neighbor 10.10.0.25 ebgp-multihop
 neighbor 10.10.0.25 next-hop-self
 redistribute connected  metric 121
 access-list all permit any



 R4
 ---
 log file /var/log/quagga/bgpd.log
password bgp
router bgp 65003
 distance bgp 250  200 150
 bgp router-id  10.10.0.25
 neighbor  10.10.0.24 remote-as  65002
 neighbor  10.10.0.24 password DOCKER
 redistribute connected  metric 121
 access-list all permit any
发表在 System | 留下评论

BGP路由重分发过滤

以下图为例

R1 配置

在R1给R2发送路由时,把6.6.6.0/24去掉。对应的配置为

log file /var/log/quagga/bgpd.log
password bgp
router bgp 65001
 distance bgp 250  200 150
 bgp router-id 10.10.0.22
 neighbor 10.10.0.23 remote-as  65010
 neighbor 10.10.0.23 password DOCKER
 neighbor 10.10.0.23 ebgp-multihop
 neighbor 10.10.0.23 prefix-list r1-out out
 neighbor 10.10.0.23 next-hop-self
 redistribute connected  metric 121
 access-list all permit any
ip prefix-list r1-out seq 5 permit 4.4.4.0/24
ip prefix-list r1-out seq 6 permit 5.5.5.0/24
!ip prefix-list r1-out seq 10 permit 6.6.6.0/24
ip prefix-list r1-out seq 11 permit 8.8.8.0/24
ip prefix-list r1-out seq 15 permit 100.100.100.0/23 ge 24 le 32
ip prefix-list r1-out seq 25 permit 10.0.0.0/8
ip prefix-list r1-out seq 50 deny any

可以看到R1给R2发送的路由中把本地的 6.6.6.6去掉了

 094846cab3a9# show ip bgp neighbors 10.10.0.23 advertised-routes
BGP table version is 0, local router ID is 10.10.0.22
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
              i internal, r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network          Next Hop            Metric LocPrf Weight Path
*> 4.4.4.0/24       10.10.0.22             121          32768 ?
*> 5.5.5.0/24       10.10.0.22             121          32768 ?
*> 8.8.8.0/24       10.10.0.22             121          32768 ?
*> 100.100.100.1/32 10.10.0.22             121          32768 ?
Total number of prefixes 4

R2配置

log file /var/log/quagga/bgpd.log
password bgp
router bgp 65010
 distance bgp 250  200 150
 bgp router-id 10.10.0.23
 neighbor 10.10.0.22 remote-as  65001
 neighbor 10.10.0.24 remote-as  65002
 neighbor 10.10.0.22 password DOCKER
 neighbor 10.10.0.24 password DOCKER
 neighbor 10.10.0.22 prefix-list from-r1-in in
 neighbor 10.10.0.22 ebgp-multihop
 neighbor 10.10.0.24 ebgp-multihop
 neighbor 10.10.0.22 next-hop-self
 neighbor 10.10.0.24 next-hop-self
 redistribute connected  metric 121
 access-list all permit any
ip prefix-list from-r1-in seq 5 permit 4.4.4.0/24
ip prefix-list from-r1-in seq 6 permit 5.5.5.0/24
!ip prefix-list from-r1-in seq 11 permit 8.8.8.0/24
ip prefix-list from-r1-in seq 15 permit 100.100.100.0/24 le 32
ip prefix-list from-r1-in seq 20 permit 10.0.0.0/8
ip prefix-list from-r1-in seq 50 deny any

R2 上查看从R1接受到的路由无8.8.8.8

05fe39a5b056# show ip bgp neighbors 10.10.0.22 routes
BGP table version is 0, local router ID is 10.10.0.23
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
              i internal, r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network          Next Hop            Metric LocPrf Weight Path
*> 4.4.4.0/24       10.10.0.22             121              0 65001 ?
*> 5.5.5.0/24       10.10.0.22             121              0 65001 ?
*> 100.100.100.1/32 10.10.0.22             121              0 65001 ?
Displayed  3 out of 9 total prefixes

R2如果想用route-map做控制,相应的配置如下

log file /var/log/quagga/bgpd.log
password bgp
router bgp 65010
 distance bgp 250  200 150
 bgp router-id 10.10.0.23
 neighbor 10.10.0.22 remote-as  65001
 neighbor 10.10.0.24 remote-as  65002
 neighbor 10.10.0.22 password DOCKER
 neighbor 10.10.0.24 password DOCKER
 neighbor 10.10.0.22 route-map myfilter in
 neighbor 10.10.0.22 ebgp-multihop
 neighbor 10.10.0.24 ebgp-multihop
 neighbor 10.10.0.22 next-hop-self
 neighbor 10.10.0.24 next-hop-self
 redistribute connected  metric 121
 access-list all permit any
ip prefix-list from-r1-in seq 5 permit 4.4.4.0/24
ip prefix-list from-r1-in seq 6 permit 5.5.5.0/24
ip prefix-list from-r1-in seq 15 permit 100.100.100.0/24 le 32
ip prefix-list from-r1-in seq 20 permit 10.0.0.0/8
##因为route-map是默认deny的,未匹配的都被deny了。
route-map myfilter permit 10
 match ip address prefix-list from-r1-in

R3配置

log file /var/log/quagga/bgpd.log
password bgp
router bgp 65002
 distance bgp 250  200 150
 bgp router-id 10.10.0.24
 neighbor 10.10.0.23 remote-as  65010
 neighbor 10.10.0.23 password DOCKER
 neighbor 10.10.0.23 ebgp-multihop
 neighbor 10.10.0.23 next-hop-self
 redistribute connected  metric 121
 access-list all permit any
发表在 net | 标签为 | 留下评论