Fast Restoration on IP – MPLS Fast ReRoute

Service providers that have a lot of real-time traffic through their network, like mobile network operators (MNOs), are very keen on a fast restoration of service once a failure occurs in the network. In the past a lot of networks were based on SDH/SONET transport networks, which took care of sub-second (50ms) failovers. Nowadays Ethernet is THE standard for any transport within a service provider network. This introduces an issue, as Ethernet is not built for automatic failover when certain things fail.

Now there are many ways to solve this and I want to dig deeper in these technologies in several posts.  I will discuss various protocols that can solve the fast restoration requirement in different ways. Some are used in local situations (so failover to local neighbor, like a twin sibling) and others can be used in inter-site locations or can be an end-to-end protection for certain traffic.

The posts are broken down as follows:

1. MPLS Fast ReRoute (this post)
2. IP Loop Free Alternate
3. BGP PIC Core/Edge
4. Hierarchical Forwarding

Please be aware that these technologies are all related to fast restore the layer 3 forwarding path, therefore restoring the MPLS forwarding path. The MPLS forwarding path may be used for layer 2 forwarding as well. What these posts do not cover is fast restoration on layer 2 level. With the current “cloud” initiatives and next generation datacenter networks we have some extensive options for layer 2 failovers.

I can (and probably will :)) write another blog post series on those kinds of failover mechanisms.

The current blog posts are focused on the Core service provider routing to offer resilient paths through the core layer 3 or MPLS cloud in the service provider network.

MPLS Fast Reroute introduction

When MPLS was invented the first application apart from fast packet switching was creating dedicated ‘circuit-like’ connections through the network. This was done using the RSVP protocol that signals a PATH message through the network and each hop reports a label back, creating an end-to-end label switched path (LSP) according to a pre-defined path through the network.

When this initial (unidirectional) path is set-up through the network, all traffic can be send through it. Now in case of a failure we want to protect this primary path. The path is signaled with either static next-hops or the ingress node can use the IGP database to calculate the path.

Be aware that your IGP needs support for this and it needs to be a link-state protocol (OSPF or IS-IS) as then every router has a full overview of the connections in the network. I will not go in to very much detail on how RSVP works and how it utilizes the IGP database to perform a C-SPF calculation. If you want I can spend another blog about this. Just leave a comment :).

Now we have a path that we can use for our traffic we want some protection. MPLS FastReroute (or FRR) is a technique that ensures this RSVP signaled path is protected. There are a couple ways to do this.

Protection

There are three ways to protect the path:

• Link protection
• Node protection
• LSP protection / end-to-end protection

It very much depends on your network topology and what you want to accomplish as far as path protection. Then there are two ways of ensuring the protection. One is a manual protection where the backup path is manually configured and signaled as an additional tunnel through the network. The second is automatic, where the router figures out which links to use for the protection and automatically signaling those paths through the network.

Why do we need it? Well the technology is introduced to ensure equal failover times as with SDH/SONET transmission networks. When using a LDP network, you need to wait for IGP convergence before the new path is ready for traffic. During tests I found out that this takes around 300-400ms when using core routing platforms (Juniper MX, Cisco ASR9k).  When using MPLS FRR you reduce this to around 50ms as the routers already have a backup path ready that should already be programmed in the relevant ASICs.

Link protection

In smaller networks I usually see link protection used. For node protection you need a larger topology so this is not always possible, or when possible not very useful. Link protection is to ensure all links are secured using a backup path as the following drawing illustrates:

The primary tunnel follows the path R1-R2-R3-R5-R6 using MPLS labels according to the drawing. When the link between R2 and R3 fails a backup tunnel is signaled by R2 to R3, around the protected link. When the link breaks R2 pushes an additional label on top of the label stack and sends it to R4. Then R4 will pop off this label (PHP behavior) and R3 will see the standard label 15 as it usually expects.

Node protection (or link-node-protection)

Node protection is used in larger environments to protect the link and the node in case of failures. As the name already says, this is the same technology as link protection, but then the backup path is signaled completely around the node, instead of just the link. As you can see in the previous example R3 is still used in transit and just it’s link to R2 is protected. In the following drawing you can see that LSR3 is fully protected as the backup path terminates on LSR4.

LSP / end-to-end protection

From what I’ve seen, Juniper is the only one that actually implements this. I’m sure it’s possible on Cisco as well, but when configuring the ‘fast-reroute’ command on a LSP it will signal a backup path through the network fully excluding any node/link that the primary path travels through. This sound pretty rigorous and it is J, but it makes sense in a square based (ladder) design as seen in the drawing below

The orange path from R1 to R3 is the primary tunnel and the red-large-dashed tunnel from R1 through R4, R5 and R6 is the back-up path that Juniper routers automatically signal when fast-reroute is enabled.

In smaller topologies with just a couple PE’s, this is do-able, but when your topology grows you require a backup path for every LSP and that can be hundreds or thousands in the larger deployments, making it very difficult to troubleshoot.

The other protections like link and node protection create a backup path around a specific link or node and all LSP’s that travel through those routers can use the same backup path in case of failures.

So when you have a specific case where you want end-to-end protection of your LSP, this is the way to go, but under normal circumstances I would recommend using link or node protection, which scales much better!

Interoperability

Now vendor interoperability is very important when it comes to Fast Rerouting. In the beginning when this was developed there were several drafts published that all used different objects in RSVP (DETOUR, BYPASS, etc.). Therefore some people might tell you that Cisco and Juniper FRR doesn’t work together.

This is long gone! But you have to configure it correctly. Like I already said, when you configure fast-reroute on a Cisco LSP it means it will use a backup tunnel when it’s available (manually configured). You require additional commands for creating the backups automatically (auto-tunnel), where you also configure whether you want link or node protection.

When you configure fast-reroute under a Juniper LSP it will signal a end-to-end protected path, which might not be what you want. You need to configure link or node-link protection under the Juniper LSP to advertise the desired protection. Then RSVP needs to be configured on each router to support either link and/or node-protection by enabling this under the interfaces configured in RSVP.

When configured correctly they perfectly interoperate!

RFC 4090 (http://tools.ietf.org/html/rfc4090) defines the finalized Fast Reroute standard, which is based on a draft by Avici. All vendors implemented this RFC and like I said, when configured with the correct commands, you can let them interoperate perfectly with each other.

Configuration

Below are some configuration examples. The first is a example of a Cisco IOS router. You see a tunnel configured and auto-tunnel being enabled to signal the backup path automatically for link-protection. Keep in mind that the backup tunnel needs to be configured on every node that you want link protection on. The ‘n-hop’ command configured ensures the link-protection, when ‘nnhop’ would be configured it would mean node-protection.

mpls traffic-eng auto-tunnel backup nhop-only 
interface Tunnel1
 ip unnumbered loopback0
 tunnel destination x.x.x.x
 tunnel mode mpls traffic-eng
 tunnel mpls traffic-eng path-option 1 dynamic
 tunnel mpls traffic-eng fast-reroute

The following example is for Juniper JUNOS routers. You see the same type of protection configured including the automatic protection for links. This is done using the link-protection command under the RSVP protocol. Additionally the same command needs to be configured under the LSP configuration.

[edit protocols]
mpls {
      label-switched-path lsp-name {
            to x.x.x.x;
            link-protection;
      }
}
rsvp {
      interface interface-name {
            link-protection;
      }
}

Summary

I hope I was able to give you a quick and brief overview of the different ways of protection for traffic engineering tunnels in MPLS networks. This was only one way of protecting traffic. Currently this is the most commonly used technology by service providers in the world, but others are rising that don’t require so much configuration, but do require tuning and sometimes specific network designs.

Stay tuned for the next blogpost about IP Loop Free Alternate!

Rick