linux下绑定特定中断到指定CPU

如果大家用的万兆网卡跑linux或者nginx做大规模的负载均衡,那么肯定会遇到网卡中断占耗尽一个CPU的情况,会发现有一个ksoftirqd进程耗CPU非常厉害。这个时候就需要把万兆网卡的多个队列分别绑定到不同的核上。简单的在自己的笔记本上测试一下把单个中断绑定到指定CPU的方式。

1.先查看中断号

root@debian:/home/pm/downloads# cat /proc/interrupts
 CPU0 CPU1 CPU2 CPU3
 0: 55 0 0 0 IO-APIC-edge timer
 1: 9828 0 0 0 IO-APIC-edge i8042
 8: 1 0 0 0 IO-APIC-edge rtc0
 9: 64153 0 0 0 IO-APIC-fasteoi acpi
 12: 12812 0 0 0 IO-APIC-edge i8042
 16: 50287 0 0 0 IO-APIC-fasteoi mmc0, ehci_hcd:usb1
 23: 32 0 0 0 IO-APIC-fasteoi ehci_hcd:usb2
 40: 2053 0 0 0 PCI-MSI-edge eth0
 41: 84430 0 0 0 PCI-MSI-edge ahci
 42: 540 0 0 0 PCI-MSI-edge snd_hda_intel
43: 189429 3306 1140 185873 PCI-MSI-edge iwlwifi
 44: 367098 96565 0 0 PCI-MSI-edge i915
NMI: 389 212 369 216 Non-maskable interrupts
LOC: 686664 448243 658281 387582 Local timer interrupts
SPU: 0 0 0 0 Spurious interrupts
PMI: 389 212 369 216 Performance monitoring interrupts
IWI: 0 0 0 0 IRQ work interrupts
RES: 1618241 979307 1837671 950705 Rescheduling interrupts
CAL: 393 803 859 871 Function call interrupts
TLB: 68943 88659 91389 45605 TLB shootdowns
TRM: 0 0 0 0 Thermal event interrupts
THR: 0 0 0 0 Threshold APIC interrupts
MCE: 0 0 0 0 Machine check exceptions
MCP: 15 15 15 15 Machine check polls

2.分别测试把中断号43绑定到不同的CPU,各个 CPU 用二进制和十六进制表示就是:                 CPU              Binary          oct

CPU 0    00000001         1
CPU 1    00000010         2
CPU 2    00000100         4
CPU 3    00001000         8

所以

echo 8 > /proc/irq/43/smp_affinity就会把网卡中断绑定到CPU3上,同理1,2,8会把中断绑定到CPU0,CPU1,CPU3上。但把中断绑定到制定的核后,我们再cat /proc/interrupts查看的话会发现对于中断号43,只有CPU3对应的数字在增加。

root@debian:/home/pm/downloads# cat /proc/interrupts
 CPU0 CPU1 CPU2 CPU3
 0: 57 0 0 0 IO-APIC-edge timer
 1: 12224 0 0 0 IO-APIC-edge i8042
 8: 1 0 0 0 IO-APIC-edge rtc0
 9: 72086 0 0 0 IO-APIC-fasteoi acpi
 12: 13634 0 0 0 IO-APIC-edge i8042
 16: 61911 0 0 0 IO-APIC-fasteoi mmc0, ehci_hcd:usb1
 23: 32 0 0 0 IO-APIC-fasteoi ehci_hcd:usb2
 40: 2310 0 0 0 PCI-MSI-edge eth0
 41: 89086 0 0 0 PCI-MSI-edge ahci
 42: 540 0 0 0 PCI-MSI-edge snd_hda_intel

43: 189429 3306 1140 279511 PCI-MSI-edge iwlwifi

 44: 367098 146592 0 0 PCI-MSI-edge i915
NMI: 459 262 442 272 Non-maskable interrupts
LOC: 830429 549702 798508 469421 Local timer interrupts
SPU: 0 0 0 0 Spurious interrupts
PMI: 459 262 442 272 Performance monitoring interrupts
IWI: 0 0 0 0 IRQ work interrupts
RES: 1992918 1266807 2245210 1188155 Rescheduling interrupts
CAL: 393 804 860 872 Function call interrupts
TLB: 94270 118496 115458 47381 TLB shootdowns
TRM: 0 0 0 0 Thermal event interrupts
THR: 0 0 0 0 Threshold APIC interrupts
MCE: 0 0 0 0 Machine check exceptions
MCP: 17 17 17 17 Machine check polls
ERR: 0
MIS: 0

另外测试了一下直接把一个中断号绑定在2个核上比如 echo 6 >/proc/irq/43/smp_affinity , 但是结果表明不能同时把这个中断绑定到2个核。linux下有个软中断负载平衡的服务irqbalance。

此条目发表在Admin分类目录。将固定链接加入收藏夹。

发表评论