CEF and Traceroute with ECMP

When you use traceroute you might see multiple paths when you use ECMP and CEF. This depends on CEF Load Balancing Algorithms.

Here is an example.

We have a router that uses ECMP to reach a destination:

R1#show ip route 4.4.4.4
Routing entry for 4.4.4.4/32
  Known via "ospf 1", distance 110, metric 3, type intra area
  Last update from 192.168.123.2 on GigabitEthernet0/0, 00:24:28 ago
  Routing Descriptor Blocks:
  * 192.168.123.3, from 4.4.4.4, 00:26:56 ago, via GigabitEthernet0/0
      Route metric is 3, traffic share count is 1
    192.168.123.2, from 4.4.4.4, 00:24:28 ago, via GigabitEthernet0/0
      Route metric is 3, traffic share count is 1

There are two paths to get to 4.4.4.4.

This also shows up in CEF:

R1#show ip cef 4.4.4.4 detail
4.4.4.4/32, epoch 0, per-destination sharing
  nexthop 192.168.123.2 GigabitEthernet0/0
  nexthop 192.168.123.3 GigabitEthernet0/0

This router uses per-destination. You can also take a detailed look like this:

R1#show ip cef 4.4.4.4 internal 
4.4.4.4/32, epoch 0, RIB[I], refcnt 5, per-destination sharing
  sources: RIB 
  feature space:
    IPRM: 0x00028000
  ifnums:
    GigabitEthernet0/0(2): 192.168.123.2, 192.168.123.3
  path list 10BD5854, 3 locks, per-destination, flags 0x49 [shble, rif, hwcn]
    path 0F3534E0, share 1/1, type attached nexthop, for IPv4
      nexthop 192.168.123.2 GigabitEthernet0/0, IP adj out of GigabitEthernet0/0, addr 192.168.123.2 0F9F82D0
    path 0F35354C, share 0/1, type attached nexthop, for IPv4
      nexthop 192.168.123.3 GigabitEthernet0/0, IP adj out of GigabitEthernet0/0, addr 192.168.123.3 0F9F8400
  output chain:
    loadinfo 0F353DBC, per-session, 2 choices, flags 0083, 5 locks
      flags [Per-session, for-rx-IPv4, 2buckets]
      2 hash buckets
        < 0 > IP adj out of GigabitEthernet0/0, addr 192.168.123.2 0F9F82D0
        < 1 > IP adj out of GigabitEthernet0/0, addr 192.168.123.3 0F9F8400
      Subblocks:
        None

Here is the exact route that CEF uses:

R1#show ip cef exact-route 192.168.123.1 4.4.4.4
192.168.123.1 -> 4.4.4.4 =>IP adj out of GigabitEthernet0/0, addr 192.168.123.2

With a trace you'll see the router uses both paths:

R1#traceroute 4.4.4.4 numeric     
Type escape sequence to abort.
Tracing the route to 4.4.4.4
VRF info: (vrf in name/id, vrf out name/id)
  1 192.168.123.2 2 msec
    192.168.123.3 1 msec
    192.168.123.2 1 msec
  2 192.168.34.4 2 msec
    192.168.24.4 3 msec *

To understand why this happens, take a closer look at the packets of that trace:

Frame 2: 60 bytes on wire (480 bits), 60 bytes captured (480 bits)
Ethernet II, Src: 52:54:00:88:5a:88 (52:54:00:88:5a:88), Dst: 52:54:00:88:65:a8 (52:54:00:88:65:a8)
Internet Protocol Version 4, Src: 192.168.123.1, Dst: 4.4.4.4
User Datagram Protocol, Src Port: 49175, Dst Port: 33434
    Source Port: 49175
    Destination Port: 33434
    Length: 8
    Checksum: 0x797a [unverified]
    [Checksum Status: Unverified]
    [Stream index: 0]

Frame 4: 60 bytes on wire (480 bits), 60 bytes captured (480 bits)
Ethernet II, Src: 52:54:00:88:5a:88 (52:54:00:88:5a:88), Dst: 52:54:00:e6:7d:1a (52:54:00:e6:7d:1a)
Internet Protocol Version 4, Src: 192.168.123.1, Dst: 4.4.4.4
User Datagram Protocol, Src Port: 49176, Dst Port: 33435
    Source Port: 49176
    Destination Port: 33435
    Length: 8
    Checksum: 0x7978 [unverified]
    [Checksum Status: Unverified]
    [Stream index: 1]

The UDP port numbers are different and since we use the universal load-sharing algorithm, CEF takes the L4 destination port into the equation:

R1#show run all | incl load-sharing             
ip cef load-sharing algorithm universal EBE637E1