Cellular in OT - Part 2
In this series we’re exploring various architectures we’ve encountered over the last decade involving the deployment of cellular gateways in OT / SCADA environments. By and large these cellular gateways have been deployed to replace previous communication paths such as POTS/Dial-Up or private licensed and unlicensed radio systems, microwave or perhaps integrate locations that were previously “offline”. I say monitored and imply these are used solely for the “DA” portion of SCADA (Supervisory Control and Data Acquisition) but I know first-hand that many are performing the “C” of SCADA over cellular as well. ***I’ll leave the discussions about whether or not anyone should do that to others; the focus in this series is on the various architectures once the decision to connect has already been made.***
By the end of Part 1 we agreed assigning public static IPs to cellular gateways was a bad idea, right? We agreed. No more. Never again. We’re on the same page - cool.
Here in Part 2 we’ll look at Carrier Private Networks and specifically what we’ll be referring to as the Full Tunnel Private Network. This is a pretty loaded topic unto itself but I’ll try and highlight the overall architecture and some of the interesting options. Also it should be noted that for the purposes of this post I’ll be referencing the architecture used by Verizon Wireless simply because I have more direct experience with it. Let’s enumerate some of the key pieces.
1) Private IP Pool - the customer will designate to the carrier what IP subnet(s) they would like to assign to their cellular gateways. Customer must make this selection with a unique unused subnet in their environment as they will route to this subnet eventually over VPN. In this example below, the sample selection was 192.168.210.0 /24 and we can see the cellular gateways WAN IPs are 192.168.210.1, 2, 3, and 4 accordingly.
“OK Josh, great now I have cellular gateways deployed with private IPs. How in the heck do I reach those private IPs from our HQ?”
2) VPN to Carrier - the customer will need to purchase and setup compatible equipment to serve as a VPN termination endpoint to the carrier. Verizon requires that the router/fw explicitly support IPSec, BGP, and GRE as all three are needed to complete the connection. We have mostly done these with either physical Cisco ISR routers or virtual Cisco CSR routers. Also note in the example above there are TWO Verizon termination endpoints which is understandably for redundancy purposes. The customer may also choose to supply redundant routers (that could even be at two different locations) for their side of the connection. Depending on the criticality of this connectivity, we don’t see it as often but we have completed almost every variation under the sun. Ultimately, once this Full Tunnel Private Network turnup is complete the SCADA Server in the above example is able to reach 192.168.210.1, 2, 3, and 4.
“Hey wait, does that even help me? How do I actually reach my assets on the LAN side of those cellular gateways”
Option 1 - IP Passthrough. If there is truly only a single asset and no other network at the remote site you could configure the cellular gateway for IP Passthrough which effectively makes your asset fully and directly accessible on the cellular IP.
Option 2 - Port Forwarding. If there is a small network, perhaps multiple devices/services which need to be accessed and your overall application will support it, you may be able to setup a port forwarding table to map all assets and services you need to reach remotely.
“Get this junk out of my face. I want complete network access to the remote sites. Give me full NET to NET connectivity or get bent. There better be an Option 3”
I’ll do you one better and provide a 3 and 4.
Option 3 - VPN Overlay. If your remote site cellular gateways or perhaps a small edge firewall deployed just behind it share a common VPN capability with the firewall/router at HQ, you could build up a hub-and-spoke VPN between HQ and your remote sites and get access to the full remote site networks (192.168.101, 102, 103, 104.x in above drawing). There is more coordination required here for unique remote site LANs not overlapping with each other or any existing HQ reachable network, etc.
Option 4 - DMNR (Dynamic Mobile Network Routing). If this feature is available in your cellular gateway, for an additional fee to the carrier you may be able to leverage it for similar networking functionality as a VPN overlay. If you add DMNR into your private network build with the carrier, your cellular gateways advertise their LAN networks to the carrier and the carrier exchanges those routes with HQ using BGP. The end result is again full access to remote site networks. Same caveats for LAN IP planning as Option 3 apply here.
“Any other decisions I have to make when building a Private Network? I’m tired.”
A few other options worth a quick mention:
Internet Access - The customer can choose whether these cellular gateways have internet access. If you’re thinking why would you want that, one common reason is to leverage the cellular gateway vendor’s cloud management platform which is close to a must-have in large deployments to keep gateway and radio module firmware up-to-date. If a customer wants this, they would make specific selections on their private network build to route all traffic back and out their HQ where they can further filter/restrict per their firewall policy. **I have previously mentioned a possible option to enable DIA (Direct Internet Access) via the carrier but I believe VZW has currently deprecated this option.**
Mobile to Mobile - The customer may only need individual remote sites accessible to the HQ or perhaps it would benefit their application for the cellular gateways to be able to reach each other directly. This is often a checkbox selection when completing the private network build form with the carrier.
Stay tuned as we add to this series! In Part 3 we explore Zero Tunnel Carrier Private Networks.
P.S. A reader raised the spot-on point that deeming the above architecture as more secure, more private, etc. all assumes you trust your carrier. It’s extremely relevant to acknowledge that in all architectures deploying cellular gateways on public infrastructure through the services of a 3rd party, you’ve entrusted some or all of the controls to them. An errant configuration update performed by this 3rd party to their infrastructure could change the exposure of your assets. For the purposes of this article series, we are assuming that risk has been accepted as we continue to explore alternatives that can be leveraged exclusively, or perhaps in combination to limit the overall risk of exposure.