Most enterprise data-centre fabrics today mix vendors — a Juniper QFX5100 leaf paired with a Nexus 9336C-FX2 spine, an Arista 7280 border leaf, and a Juniper MX edge router. EVPN-VXLAN interop between Junos, Cisco NX-OS and Arista EOS is fully mature.
1. Fabric Topology and Address Plan
Reference fabric for this guide:
- Spines: 2 × Juniper QFX5120-48Y. ASN 65000.
- Leaves: 1 × Cisco Nexus 9336C-FX2, 1 × Arista 7280QR-C32, 1 × Juniper QFX5120.
- Hosts: A device dual-homed in active-active MH and a single-homed Linux server.
- VNIs: 10100 (VLAN 100 "Web") and 10200 (VLAN 200 "DB"). Anycast gateway on each leaf.
1.1 Address Plan
Lo0 overlays (gateways, /32):
Spine-A 10.255.0.1/32 ASN 65000
Spine-B 10.255.0.2/32 ASN 65000
Leaf-JN 10.255.1.3/32 ASN 65003
Leaf-NX 10.255.1.1/32 ASN 65001
Leaf-AE 10.255.1.2/32 ASN 65002
Anycast GW MAC: 00:00:5e:00:01:01 / 00:00:5e:00:01:02
Anycast GW IP: 10.100.0.1/24 and 10.200.0.1/24
P2P underlay: 10.10.X.Y/31 between every spine and every leaf.
2. Underlay: eBGP Spine to Leaf
2.1 Junos — Spine-A
set interfaces lo0 unit 0 family inet address 10.255.0.1/32
set interfaces et-0/49 unit 0 family inet address 10.10.0.0/31
set policy-options policy-statement EXPORT-LO0 term 1 from route-filter 10.255.0.0/24 orlonger
set policy-options policy-statement EXPORT-LO0 term 1 then accept
set protocols bgp group UNDERLAY type external export EXPORT-LO0
set protocols bgp group UNDERLAY neighbor 10.10.0.1 peer-as 65003
set protocols bgp group UNDERLAY neighbor 10.10.0.3 peer-as 65001
set protocols bgp group UNDERLAY neighbor 10.10.0.5 peer-as 65002
set protocols bgp group UNDERLAY bfd-liveness-detection minimum-interval 50
2.2 Cisco NX-OS — Leaf-NX
interface Ethernet1/1
description to-SpineA
no switchport
ip address 10.10.0.3/31
interface loopback0
ip address 10.255.1.1/32
router bgp 65001
bestpath as-path multipath-relax
address-family ipv4 unicast
maximum-paths 64
neighbor 10.10.0.2 remote-as 65000
neighbor 10.10.0.2 bfd
network 10.255.1.1/32
2.3 Arista EOS — Leaf-AE
interface Ethernet1
description to-SpineA
no switchport
ip address 10.10.0.5/31
interface Loopback0
ip address 10.255.1.2/32
router bgp 65002
maximum-paths 64
neighbor 10.10.0.4 remote-as 65000
neighbor 10.10.0.4 bfd
network 10.255.1.2/32
Two underlay best-practices that survive every vendor split: (a) BFD liveness at 50 ms × 3; (b) MED comparison configured consistently on spines.
3. Overlay: BGP EVPN (AFI/SAFI 25/70)
3.1 Junos Leaf-JN
set switching-options vrf-id 1024
set protocols evpn encapsulation vxlan default-bridge-loopback 10.255.1.3
set protocols evpn enhancement bgp
set protocols bgp group OVERLAY type internal local-address 10.255.1.3
set protocols bgp group OVERLAY multicast-hops 2
set protocols bgp group OVERLAY family evpn signaling
set protocols bgp group OVERLAY neighbor 10.255.0.1
set protocols bgp group OVERLAY neighbor 10.255.0.2
3.2 NX-OS Leaf-NX
evpn
vni 10100 l2
rd auto
route-target import auto
route-target export auto
router bgp 65001
address-family l2vpn evpn
neighbor 10.255.0.1 activate
neighbor 10.255.0.2 activate
3.3 Arista EOS Leaf-AE
router bgp 65002
address-family evpn
neighbor 10.255.0.1 activate
neighbor 10.255.0.2 activate
4. Anycast Gateway and IRB/SVI
Junos: set interfaces irb unit 100 family inet address 10.100.0.1/24
set interfaces irb unit 100 mac 00:00:5e:00:01:01
NX-OS: interface Vlan100, ip address 10.100.0.1/24,
fabric forwarding anycast-gateway-mac 0000.5e00.0101
EOS: interface Vlan100, ip address 10.100.0.1/24,
ip virtual-router address 10.100.0.1
5. Multi-Homing: Type-2 Routes and ESI
set interfaces et-0/51 esi 00:11:22:33:44:55:66:77:88:99
set interfaces et-0/51 esi mode all-active
set routing-instances WEB protocols evpn interface et-0/51 esi 00:11:22:33:44:55:66:77:88:99
6. Verification — Five Commands
- Underlay:
ping 10.255.1.1 source lo0.0from every leaf — must be lossless. - EVPN session:
show bgp evpn summary. - MAC table:
show vxlan address-table. - Type-2 count:
show bgp evpn route-type 2 count. - MH election:
show evpn ethernet-segment.
7. Field-Troubleshooting Top Interop Bugs
7.1 MAC table empty after peer-up
Look at route-targets. NX-OS auto-RT vs Junos explicit RT is the most common cause.
7.2 Type-2 routes duplicated but BUM broken
VXLAN multicast group mismatch — all three default to head-end-replication if no multicast is configured. Pick head-end-replication on day one.
7.3 Active/active host sees only one MAC path
DF election stuck. Both leaves must advertise the same ESI string.
8. Closing
Stay explicit. The minute you start relying on auto for RD/RT, you will be dodging cross-vendor behavior for years. For OSPF underlay, see OSPF on MX. For MX CoS atop this fabric, see MX CoS deep dive. For an EX3400/4300 leaf, see EX as VXLAN leaf.
Discussion (1)
We just migrated our NX-OS spines from vPC to BGP EVPN using this guide. Saved us a week.