31.6. Link Aggregation and Failover

Written by Andrew Thompson.

31.6.1. Introduction

The lagg(4) interface allows aggregation of multiple network interfaces as one virtual interface for the purpose of providing fault-tolerance and high-speed links.

31.6.2. Operating Modes

The following operating modes are supported by lagg(4):

Failover

Sends and receives traffic only through the master port. If the master port becomes unavailable, the next active port is used. The first interface added is the master port and any interfaces added after that are used as failover devices. If failover to a non-master port occurs, the original port will become master when it becomes available again.

Cisco® Fast EtherChannel®

Cisco® Fast EtherChannel® (FEC) is a static setup and does not negotiate aggregation with the peer or exchange frames to monitor the link. If the switch supports LACP, that should be used instead.

FEC balances outgoing traffic across the active ports based on hashed protocol header information and accepts incoming traffic from any active port. The hash includes the Ethernet source and destination address and, if available, the VLAN tag, and the IPv4 or IPv6 source and destination address.

LACP

The IEEE® 802.3ad Link Aggregation Control Protocol (LACP) and the Marker Protocol. LACP will negotiate a set of aggregable links with the peer in to one or more Link Aggregated Groups (LAGs). Each LAG is composed of ports of the same speed, set to full-duplex operation. The traffic will be balanced across the ports in the LAG with the greatest total speed. In most cases, there will only be one LAG which contains all ports. In the event of changes in physical connectivity, LACP will quickly converge to a new configuration.

LACP balances outgoing traffic across the active ports based on hashed protocol header information and accepts incoming traffic from any active port. The hash includes the Ethernet source and destination address and, if available, the VLAN tag, and the IPv4 or IPv6 source and destination address.

Loadbalance

This is an alias of FEC mode.

Round-robin

Distributes outgoing traffic using a round-robin scheduler through all active ports and accepts incoming traffic from any active port. This mode violates Ethernet frame ordering and should be used with caution.

31.6.3. Examples

Example 31.1. LACP Aggregation with a Cisco® Switch

This example connects two interfaces on a FreeBSD machine to the switch as a single load balanced and fault tolerant link. More interfaces can be added to increase throughput and fault tolerance. Frame ordering is mandatory on Ethernet links and any traffic between two stations always flows over the same physical link, limiting the maximum speed to that of one interface. The transmit algorithm attempts to use as much information as it can to distinguish different traffic flows and balance across the available interfaces.

On the Cisco® switch, add the FastEthernet0/1 and FastEthernet0/2 interfaces to channel group 1:

interface FastEthernet0/1 channel-group 1 mode active channel-protocol lacp ! interface FastEthernet0/2 channel-group 1 mode active channel-protocol lacp

Create the lagg(4) interface using fxp0 and fxp1, and bring the interfaces up with the IP address of 10.0.0.3/24:

# ifconfig fxp0 up # ifconfig fxp1 up # ifconfig lagg0 create # ifconfig lagg0 up laggproto lacp laggport fxp0 laggport fxp1 10.0.0.3/24

View the interface status by running:

# ifconfig lagg0

Ports marked as ACTIVE are part of the active aggregation group that has been negotiated with the remote switch. Traffic will be transmitted and received through active ports. Use the verbose output of ifconfig(8) to view the LAG identifiers.

lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8<VLAN_MTU> ether 00:05:5d:71:8d:b8 media: Ethernet autoselect status: active laggproto lacp laggport: fxp1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> laggport: fxp0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>

To see the port status on the Cisco® switch, use show lacp neighbor:

switch# show lacp neighbor Flags: S - Device is requesting Slow LACPDUs F - Device is requesting Fast LACPDUs A - Device is in Active mode P - Device is in Passive mode Channel group 1 neighbors Partner's information: LACP port Oper Port Port Port Flags Priority Dev ID Age Key Number State Fa0/1 SA 32768 0005.5d71.8db8 29s 0x146 0x3 0x3D Fa0/2 SA 32768 0005.5d71.8db8 29s 0x146 0x4 0x3D

For more detail, type show lacp neighbor detail.

To retain this configuration across reboots, the following entries can be added to /etc/rc.conf:

ifconfig_fxp0="up" ifconfig_fxp1="up" cloned_interfaces="lagg0" ifconfig_lagg0="laggproto lacp laggport fxp0 laggport fxp1 10.0.0.3/24"

Example 31.2. Failover Mode

Failover mode can be used to switch over to a secondary interface if the link is lost on the master interface. To configure failover mode, first bring the underlying physical interfaces up. Then, create the lagg(4) interface, using fxp0 as the master interface and fxp1 as the secondary interface, and assign an IP address of 10.0.0.15/24:

# ifconfig fxp0 up # ifconfig fxp1 up # ifconfig lagg0 create # ifconfig lagg0 up laggproto failover laggport fxp0 laggport fxp1 10.0.0.15/24

The interface should now look something like this:

# ifconfig lagg0 lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8<VLAN_MTU> ether 00:05:5d:71:8d:b8 inet 10.0.0.15 netmask 0xffffff00 broadcast 10.0.0.255 media: Ethernet autoselect status: active laggproto failover laggport: fxp1 flags=0<> laggport: fxp0 flags=5<MASTER,ACTIVE>

Traffic will be transmitted and received on fxp0. If the link is lost on fxp0, fxp1 will become the active link. If the link is restored on the master interface, it will once again become the active link.

To retain this configuration across reboots, the following entries can be added to /etc/rc.conf:

ifconfig_fxp0="up" ifconfig_fxp1="up" cloned_interfaces="lagg0" ifconfig_lagg0="laggproto failover laggport fxp0 laggport fxp1 10.0.0.15/24"

Example 31.3. Failover Mode Between Wired and Wireless Interfaces

For laptop users, it is usually desirable to configure the wireless device as a secondary interface, which is used when the wired connection is not available. With lagg(4), it is possible to use one IP address, prefer the wired connection for both performance and security reasons, while maintaining the ability to transfer data over the wireless connection.

In this setup, override the underlying wireless interface's MAC address to match that of the lagg(4), which is inherited from the wired interface.

In this example, the wired interface, bge0, is the master, and the wireless interface, wlan0, is the failover interface. The wlan0 device was created from iwn0, which will be configured with the wired connection's MAC address. The first step is to determine the MAC address of the wired interface:

# ifconfig bge0 bge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4> ether 00:21:70:da:ae:37 inet6 fe80::221:70ff:feda:ae37%bge0 prefixlen 64 scopeid 0x2 nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> media: Ethernet autoselect (1000baseT <full-duplex>) status: active

Replace bge0 to match the system's interface name. The ether line will contain the MAC address of the wired interface. Now, change the MAC address of the underlying wireless interface:

# ifconfig iwn0 ether 00:21:70:da:ae:37

Bring the wireless interface up, but do not set an IP address:

# ifconfig wlan0 create wlandev iwn0 ssid my_router up

Bring the bge0 interface up. Create the lagg(4) interface with bge0 as master, and failover to wlan0:

# ifconfig bge0 up # ifconfig lagg0 create # ifconfig lagg0 up laggproto failover laggport bge0 laggport wlan0

The interface will now look something like this:

# ifconfig lagg0 lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8<VLAN_MTU> ether 00:21:70:da:ae:37 media: Ethernet autoselect status: active laggproto failover laggport: wlan0 flags=0<> laggport: bge0 flags=5<MASTER,ACTIVE>

Then, start the DHCP client to obtain an IP address:

# dhclient lagg0

To retain this configuration across reboots, the following entries can be added to /etc/rc.conf:

ifconfig_bge0="up" ifconfig_iwn0="ether 00:21:70:da:ae:37" wlans_iwn0="wlan0" ifconfig_wlan0="WPA" cloned_interfaces="lagg0" ifconfig_lagg0="laggproto failover laggport bge0 laggport wlan0 DHCP"

All FreeBSD documents are available for download at http://ftp.FreeBSD.org/pub/FreeBSD/doc/

Questions that are not answered by the documentation may be sent to <freebsd-questions@FreeBSD.org>.

Send questions about this document to <freebsd-doc@FreeBSD.org>.