Sandstorm behind HAProxy in pfSense via SSL Passthrough (TLS SNI extension)

This scenario provides step-by-step instructions on running a Sandstorm server behind an HAProxy reverse proxy so we can make use of SNI and host multiple  domains on a single IP. HAProxy here does not do anything more than route SSL connections to the proper server. All SSL encryption and decryption occurs on the destination server.
This works via a TLS extension called SNI (Server Name Identification) which sends the initial SSL handshake request in plain-text, allowing HAProxy to read the requested domain name and forward it to the correct server.  These steps ought to work for other server instances, not exclusively Sandstorm.
pfSense, a popular open source router, allows for GUI-based configuration of many command-line input (CLI) only utilities, such as in our case here. The text-based .cfg file that is generated by pfSense can be used as a general template for the CLI version.
It is advisable to change your HTTPS pfSense management interface port to something other than the default 443, as there is a risk that pfSense will present its internal router login page to the public if HAProxy were to fail for some reason and stop listening on 443. Same for its HTTP port, if you have HTTP management enabled. 

This option can be found in the System Menu, under Advanced, Admin Access. Adjust the TCP port field here to something other than 443.

NEW EDIT 3/11/18: A method to mitigate some service failures in pfSense is to install and configure the “Service_Watchdog” package in the pfSense offerings to restart HAProxy (and any other service of your choice, such as OpenVPN, in my case) in case of detected service failures.

In pfSense, browse to System menu, and select Package Manager.
Under the “Available Packages” section, install the latest stable (non development) package. In this example, the latest available is  v0.48_1
 
We are now going to add the backend server, which corresponds to your Sandstorm server instance, in SSL passthrough mode.
Select the “Backend” tab and click the “Add” button.
Enter a descriptive name for the backend server. While in this example the format “myhost.sandcats.io” is used the Name field, this has no relationship to your actual domain name. You can enter anything you like here.
In the Server List section, enter a descriptive name in the Name field (again, I used myhost.sandcats.io in the screenshot, but it can be anything you like)
Enter your Sandstorm server’s HTTPS port (if you are using the *.sandcats.io free DNS and wildcard SSL certificate, this should be 443).
Make sure to select the “SSL checkbox in the same row. You can leave the “Weight” box empty, as we are not using HAProxy for load balancing.
Leave everything else the same and click “Save”, and then “Apply Changes” on the next prompt.

When finished, the Backend section will look something like this. In this screenshot, there are two unrelated servers. For our purposes, look at the *.sandstorm.io entry.  I’ve changed my “Health Check” setting from the default “HTTP” to Basic, which does a simple socket check on the host. The Sandstorm server returns unrecognizable data via the HTTP check to the Health Check service and it shows up as being down, and I didn’t feel like troubleshooting.

 

Next, we need to create the Frontend entry in HAProxy, which will represent the server that will answer HTTP/HTTPS requests on port 80/443 respectively via your public/WAN IP.
Select the Frontend tab, and click the Add button.
Enter a descriptive name in the Name field.
In the External Address section, select “WAN Address IPV4” . In the example screenshot, there is no specific WAN (Public IP) address because there is only one available via the ISP. If you have more than one, enter the corresponding IP, or leave it blank to listen on all of them.
Enter port 443 in the Port field.
Do not select the “SSL offload” checkbox.
Further down the configuration page, under Type, change to “ssl/https TCP” mode.
In the Access Control List (ACL) section, enter a descriptive name for the Name field, and select “Server Name Indication TLS extension ends with”.
In the Value field, input the your Sandstorm server’s domain name. If you used the built-in free DNS and SSL certificate provider sandcats.io, then enter it in the format “hostname.sandcats.io”. Conversely you can use your own domain name if your server is set up that way. Do not enter any additional characters that are not part of the domain.
It is important to select the “…ends with” option, instead of the “…matches” option here (unlike in the other two entries above it in the example screenshot) because Sandstorm will dynamically generate subdomains whenever starting up different grains (their terminology for container instances). The “…matches” option would break the connection because HAProxy would not pass through the dynamically created subdomains.
In the Actions section, leave “Use backend” as the default choice, and enter the name of the ACL you created above (the value of the Name field).
Click Save, and Apply Changes.
The process we are configuring here is simple:
HAProxy listens for HTTPS requests on your WAN IP:443 address:port (Front end)
Client makes request for yourhost.sandcats.io
HAProxy reads the request, interprets the SNI information asking for a specific domain, then forwards all requests to and from that server, thereby ”proxying” connection between client and server. (Backend)
 
Here is what this example setup looks like, serving up 3 different domains, all running through HAProxy.
Next, we need to allow port 443 through the firewall, as it is blocked by default by the pfSense firewall. (pfSense by default blocks all but explicitly allowed traffic, other than the default “allow all access FROM LAN network” rule that it autoconfigures during setup.)
Click on Firewall on the menu bar, and select Rules.
in the WAN tab, click Add  to create a new firewall rule.
Adjust entries as follows:
Action: Pass
Interface: WAN
Address family: IPV4
Protocol: TCP
Source: Any
Destination: WAN address
Destionation port range: HTTPS 443
 
This is what the firewall rule will look like once created. This example screenshot shows the newly created rule at the top. (Ignore topmost rule with the red x – that’s generated by the system from a different setting, and is not the entry I am referring to above.)
 
Now to test the site:
w00t!
The haproxy.cfg file that is generated by pfSense is also available. This can be used as a template for non-GUI installations. It’s useless for use by pfSense directly because the file is auto-generated every time HAProxy starts up. The section “Listen HAProxyLocalStats” in the cfg file is specific to the example pfSense installation, and can be ignored. I included it for the sake of completion.
NOTE: I’ve modified the IP addresses, domain names and ACL names in the .cfg file to preserve the security of the network used in the example and other identifying information. Thus the information in the .cfg file may not match the screenshots exactly.
2nd NOTE: The file above has a .txt extension due to WordPress security restrictions.

One thought on “Sandstorm behind HAProxy in pfSense via SSL Passthrough (TLS SNI extension)

Leave a Reply

Your email address will not be published. Required fields are marked *