| Front | Info | Lists | Newsfeeds | Study Guide | What is BSD? | |
|
pfSense Load Balancing and Failover with Relayd
Jeremy C. Reed This is a brief extract from the book pfSense Essentials. See http://www.reedmedia.net/books/pfsense/ for details. The pfSense system can be used for managed IP forwarding by providing Layer 3 network redirection to other systems, with manual or automated failover. This is available via the Services → Load Balancer and Status → Load Balancer menu pages. This technology is powered by the OpenBSD relayd daemon which manages the packet-level redirection rules with PF packet filter rules to forward traffic between it and target servers. For high availability and automated failover, it can monitor (aka health-checks) the target hosts before forwarding to them. Note that this feature may be deprecated in later versions of pfSense. An alternate solution is HAProxy, a TCP, HTTP, and HTTPS load balancer, which is available to be installed from the pfSense packages. pfSense also provides network redundancy by sharing the same IP addresses over multiple pfSense systems and a multi-WAN setup for Internet gateway failover. Unmanaged connection redirections to other hosts can also be done via the packet filter using stateful address translation. An alternate method and definition of load balancing by prioritizing traffic can be done via the ALTQ technology. The Services → Load Balancer pages have links to take you to the page views for the Pools for listing or defining destination server(s) for a specific service, the mode of operation (load balancer or failover), and what monitor to use; Virtual Servers which lists or defines the listening connection and what destination(s) it uses; Monitors which lists or defines the techniques for checking if the target server is up; and Settings for a few additional daemon tunables. On the top right of the pages are shortcuts for Related status and Related log entries which are also covered below. A pool defines a single destination or a cluster of systems providing a network service. A virtual server (configured in an upcoming section) enables the pfSense load balancer to monitor that these target systems are up and to redirect traffic to and from them. The pools list and additions are available via the main Services → Load Balancer page. No pools are configured by default, but when defined, this page has a table briefly describing them. Use the Add button to add a load balancer pool. Adding a pool does not start the load balancing or failover service yet, but just adds a definition (known as a table) for the destinations and what monitor is used to check it. The daemon is enabled (after adding this pool) via the Services → Load Balancer → Virtual Servers page as covered in Section 2, “Virtual Servers”. For each entry, the Pools table shows the mode as Load Balance or Manual Failover; the single active failover server or the cluster of load balancing servers; the port the remote server listens on; the check-health monitor (with a link to review or edit the monitor configurations); and an optional description that briefly explains or identifies the purpose of this target pool. These are explained in the upcoming sections. A different view of this same pool information is available at Status → Load Balancer as seen in Section 5, “Load Balancer Pools Status”. The entries have corresponding action icons to edit, copy, or remove the entry. Click the Edit pool pencil icon to make changes as described in the following section. Click the Copy pool clone icon to duplicate the entry so you can create another pool entry based on similar settings. To remove a pool, click its Delete pool trash can icon. It will first check that no virtual servers use the corresponding pool. Adding or editing a pool is accessed via the Services → Load Balancer → Pools page by clicking the Add button, Edit pool pencil icon (to modify an existing entry), or Copy pool clone icon (to start a new entry based on a previously added pool). The form has several fields and the name, port, and an enabled target server address is required. Enter a unique single word in the required Name field. It cannot contain a space or a slash (/). It cannot be longer than 16 characters. It cannot match an existing alias name and cannot be one of pfSense's reserved table names (bogons, bogonsv6, negate_networks, snort2c, sshguard, sshlockout, tonatsubnets, virusprot, vpn_networks, or webConfiguratorlockout.) This name is selected in the upcoming Virtual Servers configuration. The Mode drop-down field is used to select if you want this pool to define a Load Balance cluster of servers or an individual enabled Manual Failover destination server (and optionally a disabled choice too). The connections are somewhat balanced using the round-robin looping of the PF packet filter translation. Choosing Manual Failover from the drop-down puts it into a simple failover mode. The Status → Load Balancer page can be used to see if the daemon detected that the destination server is down and to manually select an alternate destination (with its corresponding radio button). While this option is called Manual, automated failover can be activated using the Virtual Server's Fall-back Pool selection. Note that the Load Balance selection also provides a type of failover support, as the daemon will remove servers from the pool of destination servers if they are not available. You may want to add an explanation about this pool entry in the Description field, especially if you have many pools with vague names. Enter a valid port number or port alias name in the required Port field. This is the listening port on the remote server. Port aliases can be setup via the Firewall → Aliases → Ports page. Note that if using an alias list, it will only use the first port number if the corresponding Virtual Server configuration only listens on a single port. On host checks, the daemon will temporarily mark a destination host as down if the check fails. To make it more tolerant by trying it more times, set a count number in the Retry field. To keep the default of zero, leave the field blank. The maximum is 65535. The health check is defined in the Monitor drop-down field. By default, the choices are ICMP (the default), TCP, HTTP, HTTPS, and SMTP. You may have different selections as seen and setup via the Services → Load Balancer → Monitors page. NoteIf you need a specific type of monitoring, you may want to set it up first. See Section 3, “Load Balancer Monitors” for details. The IP addresses in the load balance pool or the failover pools are entered, one at a time, in the Server IP Address field. This is address of the target server listening on the port defined above. You may enter an IPv4 or IPv6 address or subnet. If using a subnet, use a small range with slash (/) notation like /26 through /32, as it cannot be larger than 64 addresses. Note that host aliases do not work. Click the Add to pool button after each entry. Below the address entry field and button, there are two lists for the pool members. When using the default Load Balance mode, each addition will be added to the right Enabled (Default) list. You may add multiple entries in the Load Balance mode. The Manual Failover mode only has a single address or subnet. The first addition is placed on the right Enabled (Default) side. The second or later entries (when clicking Add to pool an additional time) will go into the left Disabled column. The Disabled list is not known by the relayd daemon and will not be used for redirection. It is only used manually by the pfSense administrator. The Move to disabled list button under the right list may be used to move the selected entry (or multiple entries) to the left column. In Load Balance mode, the Move to enabled list button on the left side may be used to move disabled members to the default (right) list. If you make a mistake or need to remove an entry or entries, select them and click the corresponding Remove button. (It may prompt you to verify this action.) Note that at least one enabled IP address is required. For either mode, the value of using this feature is to have a list of generally available servers in the Enabled list. The Status → Load Balancer → Pools page may be used to manually select hosts including from the disabled to switch what pool to monitor and redirect to. This is covered in Section 5, “Load Balancer Pools Status”. Click the Save button to introduce these pool settings. This will take you back to the table view of all pools. You will need to also click the Apply Changes button to use the new pool's configuration. (But if you don't have a virtual server setup, applying the changes doesn't matter yet — and can be done after adding that.) NoteYou may have firewall rules that restrict the monitoring or access to the pool hosts (at the defined port). See the Firewall → Rules page to review your settings or to allow access for your load balancer. The Services → Load Balancer → Virtual Servers subpage lists the existing listening server addresses and ports. These are virtual as the real service is provided at the destinations defined in the enabled pool. No virtual servers are configured by default. Click the Add button to add one. The table (when virtual servers are defined) lists the protocol “tcp” which in pfSense implies that it will do PF packet filter redirection or “dns” which means it will be a limited Layer 7 DNS relay; the local IP address and port that the pfSense server is redirecting (or relaying) for; the pool (defined earlier) of destination servers; and an automatic fall-back pool (if used for failover). The upcoming section explains these in more detail. Each entry will have action icons to modify, duplicate, or remove the corresponding virtual server configuration. Click the Edit virtual server pencil icon to adjust its settings (as covered in the following section). Use the Copy virtual server clone icon to make a copy of the configuration — it will take you to the entry form with values preset using the original copy. To remove a virtual server, click the Delete virtual server icon which will then prompt you to confirm the removal. After making changes, you may need to use the Apply Changes button for them to be activated. For status details about these same enabled virtual servers, see the Status → Load Balancer → Virtual Servers subpage (see Section 6, “Load Balancer Virtual Servers Status”). The form for adding or editing a virtual server configuration is accessed by clicking the Add button from the Services → Load Balancer → Virtual Servers button or by using the Edit virtual server pencil icon or Copy virtual server clone icon actions. The virtual server needs a name and a local IP address to listen on. Enter a keyword to identify it in the Name field. Each virtual server must have a different name with a maximum of 32 characters. It cannot contain slashes (/) nor any spaces. If you need a more detailed name or longer explanation, enter it in the optional Description field. In the IP Address field enter the IP address or host alias name to listen on. If the load balancer is redirecting internal services for outside use, then the address could be the WAN IP address, for example. You may also use a host alias as previously setup via Firewall → Aliases. The Port field is used to define the port for the incoming connections. If your virtual server's listening port is the same as the port used for the destination server(s) in the pool, you can keep this field empty. NoteJust like the destination addresses and port used by a pool need to be allowed in the packet filter, the traffic into this virtual server also needs to be allowed by the PF packet filter firewall. See and adjust your settings in Firewall → Rules as needed. Note it does not make sense to have the virtual server handling on the same port as something else already on the same IP address. It may not work as you expect, so keep track of your different listening services and network translations. The next selection is the Virtual Server Pool drop-down menu. Select the load balancer pool that was previously defined (and seen at Services → Load Balancer) for your destination(s). By default, it has the first load balancer pool defined. The Layer 3 redirection load balancer can auto failover to use a different pool if the main pool is all down. To enable this, select the previously-defined pool in the Fall-back Pool drop-down menu. (Note this option cannot be used with following Relay Protocol set to “DNS”.) This Load Balancer implementation managed by pfSense is mainly for Layer 3 redirection. This is the default mode “TCP” as set in the Relay Protocol drop-down menu. It has very limited use as a Layer 7 application relay for the historic DNS protocol via the “DNS” selection. This DNS feature tells the relayd daemon to listen only on UDP and relay via UDP. This enables tracking of the stateless UDP by using DNS headers. WarningThis DNS feature is broken for most modern uses of DNS. Recent implementations of DNS use EDNS0 (Extension Mechanisms) support and accept DNSSEC records which breaks this DNS-based state checking. In addition, changing the Relay Protocol for an existing virtual server to “DNS” may have unexpected results. You may need to click the Restart Service shortcut button to restart relayd. Click the Save button to introduce these changes. This will take you back to the table view of your virtual servers. To activate these new configuration changes, click the Apply Changes button. Note it may take several seconds for the service to be active. A monitor is a health-check to confirm that the destination server or specific service is up. The relayd daemon periodically checks the members of the pool and the redirection is only enabled for those that are up. By default, pfSense has some predefined monitors seen at Services → Load Balancer → Monitors. These may be selected when setting up the load balancer pool (see Section 1.1, “Add or Edit Load Balancer Pool”). The default monitor definitions are: ICMP to do an ICMP or ICMPv6 ECHO_REQUEST ping to the host; TCP to do a simple TCP connect to see if the host is up; HTTP to do a simple HTTP HEAD request of the default path (/) and check that it returns a “200 OK” status; HTTPS does the same HTTP check but wrapped in SSL; and SMTP which checks if the returned data from the connection is a “220” mail service ready greeting. (No load balancer using a monitor is configured by default, but when a pool is added, the default monitor choice is ICMP.) More specific HTTP and HTTPS monitors may be setup as shown in the following section. In addition, a generic method for optionally sending some data and checking what it returns is available.
You may choose to copy an existing monitor configuration to speed up creating a monitor. (Creating a new monitor may be a better idea than modifying one of the predefined defaults as that would make your system different than other pfSense standard setups.) Then the basic step would be to give it a new name and whatever change as needed such as entering an HTTP hostname to check for. Introducing a new health-check or modifying an existing one is done via the Services → Load Balancer → Monitors subpage by clicking the Add button, the Copy monitor clone icon action, or the Edit monitor pencil icon action. If cloning an existing monitor, be sure to set a new name. The Description field is not optional (like in most pfSense forms). Enter a brief summary or explanation of the purpose of this monitor there. Then enter the type of health-check if not using the default ICMP in the Type drop-down field. Note that the type may introduce an additional form for type-specific settings. For example, for HTTPS, select the result that should be in the returned header's status line in the HTTPS Code drop-down field. The choices contain many status codes from “100 Continue” to “505 HTTP Version Not Supported.” The default is the successful “200 OK.” When finished, click the Save button. A few tunables for the relayd daemon may be configured via the Services → Load Balancer → Settings subpage. These global settings are for all the load balancer configurations and not specific to a specific load balancer pool or virtual server. It will attempt to check a host's state for up to 1000 milliseconds. This may be changed by entering the number in milliseconds in the Timeout field. It checks each host every 10 seconds. To change this, enter how often in seconds in the Interval field. It will start up three daemon processes, each able to handle all relayed connections. To change the number of processes at startup, set the Prefork field. The allowed range is 1 to 32. Click the Save button to save the configurations. Then it will indicate the changes need to be applied, so click the Apply Changes button to reload the load balancer. To see what pools and virtual servers are currently up see the Status → Load Balancer page and Status → Load Balancer → Virtual Servers subpage. (Status for the virtual servers is covered in the next section.) The main page shows the pools. For each, it shows the configured mode (like Load balancing or Manual failover) and the name of the monitor (like TCP). The most important use is to quickly see if the destination servers in the cluster (or individual failover) are up, down, or disabled and to manually select what servers should be in use. Servers that the health-check monitor determined are up are marked in light green and those that are currently unavailable due to monitoring are in light red. Pool entries that have been in use will have a percentage number indicating its availability. This may be used as an indicator (when compared to others in the same pool) on how often it was available for use. Those at 0.00% have probably never been in use, so you may want to check your IP address or monitor or that remote system to verify it is correct and up. The Servers column lists all of the load balancing or manual failover server addresses (with port numbers). For load balancing it has checkboxes and for manual failover it uses radio selection buttons. Unchecking a load balancing server selection implies that it will be disabled; pfSense will know about the configuration but the relayd daemon will not. For manual failover, just select the other single server to failover to. (Note that the load balancer will not automatically change the manual failover based on monitoring.) While the interface allows you to uncheck all the load balancing servers to disable them, it will always require at least one enabled even if its monitoring indicates it is unavailable (with red). (When the page is reloaded, one will be checked.) Click the Save button to change the server selections (or single manual selection). Then press Apply Changes to activate it. Clicking the Reset button simply reloads the page (after prompting if okay) with the previously-enabled settings displayed. A sortable table view of the virtual server style load balancers is available at the Status → Load Balancer → Virtual Servers subpage. This shows its name, IP address and port number (colon delimited), each of its servers in the pool, status, and description. The status is shown in light green for a working or active entry. It is light red if the virtual service is down. Or if the relayd daemon is not running or the status is unknown it is in light blue. This may also indicate the total sessions count and statistics about the last and average sessions. (These details may be a minute old.) For the logging messages for the relayd load balancer, see the Status → System Logs → Load Balancer page. After completed remote host monitoring checks, it logs about changed states such as going from up to down. A state of unknown may indicate that the host hadn't been checked yet or is disabled. The new state notification may also report round trip times and their availability percentage. The logs may also contain messages about corresponding PF tables updates. To search these logs, use the Log filter shortcut. By default, it will display the latest 50 entries. To modify this or its other log view settings, see the Manage log shortcut. The original chapter has cross-references for various topics. See the book for the supplementing chapters and sections about the Package Manager, Virtual IPs, Multi-WAN Load Balancing with Gateway Group, Network Address Translation, Firewall Aliases, Advanced Log Filter, Manage Log, and more. See http://www.reedmedia.net/books/pfsense/ for details.
DiscussionDiscuss this article below.
|
BSD Links · Advocacy· Drivers · Events · Flavours · FAQs · Guides · Programming · Security · Software · User Groups |