Comprehensive Firewall Comparison¶
Recently I've reviewed several different hardware Systems like the Zimaboard or the famous dec740. At the Zimaboard due to popular request I benchmarked several different firewall solutions. As those the benchmark results came out quite differently I decided to make a comprehensive comparison of several different firewall solutions, i.e
- Linux based
- OpenWrt 23.03.3 (generic-ext4-combined-efi.img.gz)
- VyOS 1.4 Rolling (vyos-1.4-rolling-202303040745-amd64.iso)
- ClearOS (ClearOS-DVD-x86_64.iso)
- Endian Firewall Community Edition (community_community-x64_3.3.2_recovery_softwarex86-64_20201101220035.iso)
- Linux with nftables (alpine-extended-3.glkLulU6.17.2-x86_64.iso)
- tnsr
- danos
- Sophos UTM (asg-9.714-4.1.iso)
- BSD based
- OpenBSD pf (install72.iso)
- OPNsense 23.1 (OPNsense-23.1-OpenSSL-vga-amd64.img.bz2)
- pfSense 2.6 (pfSense-CE-2.6FwQPrt4.6.0-RELEASE-amd64.iso.gz)
- Unknown
- Arista Untangle (Registration: An unexpected error occurred
- Mikrotik RouterOS x86 (mikrotik-7.8.iso)
Benchmarking setup¶
In order to find out differences I will run several benchmarks with T-Rex 3.02 on an Intel i7 8600k with a Mellanox Connectx-4 (Dual Port) on CentOS 7.9. The device under test (DUT) is a Asrock x470 Taichi Ultimate with a Ryzen 3700x and 32 GB RAM of DDR4. It has a Mellanox Connectx-3 (Dual Port) NIC.
There are several different types of benchmarks I will run on all firewalls. There will be two iterations, first running with default settings, second with optimized settings. Most Firewall solutions come with default values, e.g. for connection limits.
Every firewall will get a WAN connection in order to load updates and for management. The WAN interface is an Intel 1G NIC which will never be benchmarked, only the 10G interfaces will.
pfSense 2.6 CE¶
OpenBSD¶
https://www.openbsd.org/faq/pf/example1.html
Danos¶
Doesn't recognize Connectx-3 as dataplane. The docs say MLX3 may be supported but sadly it seems like it's not.
Sophos¶
Doesn't recognize ANY interface card, neither Acquantia nor Mellanox