Forti SD-WAN Hub and Spoke: Part One
UPDATE: I couldn’t get this working without the frig detailed below. However, there’s probably still some value in this post. (Translation: I’m too lazy to do a similar one) but I did get hold of some evaluation licenses and set up topology the detalied in the link below using FortiOS 7.2.5 – and it worked fine.
https://docs.fortinet.com/document/fortimanager/7.2.0/single-datacenter-for-enterprise/503190I
If you’re deploying an SD-WAN topology of any scale in the Fortinet world it is recommended that you use the Fortimanager but I was interested in understanding what was going on with the Fortigate firewalls themselves. (Incidentally, getting Fortimanager to play with Fortigates is quite fiddly in GNS3. This link helped me (https://forum.fortinet.com/tm.aspx?m=173981) )
I used a Fortinet reference design as the basis for my lab so most of the configurations are a cut and paste job and this post will only make sense in conjunction with this link:
https://docs.fortinet.com/document/fortigate/7.0.0/sd-wan-self-healing-with-bgp/559415/overview
This describes how to create a Hub and Spoke ADVPN with BGP routing and an SD-WAN overlay to create a resilient or ‘self-heal’ environment. Like all SD-WAN designs I have seen posted, it does not mention the necessary underlay routing which in this instance threw up an interesting curve ball.
My GNS3 topology is below – the clouds are just for management access. The red chequered rectanges are Fortigate firewalls running FortiOS version 7.0.5.
The above translates to the network topology below with the HUB firewall forming a BGP adjacency with the DC router in ASN 65001 and adjacencies with the branch firewalls B1 and B2 in ASN 65011
When the BGP DC (Datacentre) paths were initially advertised to B1 and B2 by the HUB firewall, the next hop was 192.168.100.1 (the DC router) and was this was an address that B1 and B2 didn’t know how to reach.
This could have been fixed with a route but I configured the HUB firewall to advertise ‘next hop self’ to its B1 and B2 BGP peers through the BGP neighbor group configuration. (I will use the US spelling of ‘neighbour’ to avoid confusion. Apparently, the Normans put the ‘ou’ in British English to Frenchify it.)
Setting this does what exactly what it says in that the next hop changes…
To get the VPNs up, I had configured static routes pointing the subnets for the BGP neighbor ranges down the relevant VPN tunnel interfaces. (This becomes relevant later)
Neigbor ranges..
…and related static routes…
This was enough to get the BGP adjacencies to form and share BGP paths. However, there was still a connectivity issue between the Hub and Branch laptops:
The BGP path to the B1 subnet 192.168.136.0/24 was in place on the HUB Fortigate with the correct next hop of the relevent BGP peer pointing down the correct VPN tunnel interface:
The recursive lookup for the gateway 169.254.16.10 returned the static route for the VPN,169.254.16.0/24, shown earlier.
This route’s gateway was undefined (0.0.0.0) which is to be expected as the connecting peers are dynamically addressed. However, I found the subnet needs a route with a gateway for the routing to work. (Fortigate introduced tunnel-ids in version 7.0.0:
If this was designed to fix this issue it didn’t appear to do so.)
I would pursue this further if this was going into production but for the purposes of the lab I added a host route for each BGP gateway pointing to a static gateway of the same address. (Although the tunnels are dynamically addressed, they start with given values so I know what the first and second tunnel addresses will be
(config vpn ipsect phase1-interface
set ipv4-start-ip 169.254.16.10
set ipv4-end-ip 169.254.16.250)
It’s probably more easily illustrated than explained:
It’s not pretty but it works.
Connectivity from Hub to Branch:
Branch to Hub:
Incidentally, we can see the ADVPN ‘shortcut’ tunnel pop up when doing the latter:
These are the vpn tunnels on B1 before pinging the B2 laptop:
And then after where a new tunnel directly between B1 and B2 has been created:
This has been quite a long post so I will post the configs of the devices here and then show how the SD-WAN elements worked in a separate post. (The configs are lab configs so there may be extraneous elements where I was trying things out but hopefully these will be self-evident. Aslo, I have had to change the suffix for the Fortigate configs from *.conf to *.txt to keep my WordPress editor happy)