L2VPN WG Olen Stokes Internet Draft Extreme Networks Intended Status : Standard Vach Kompella Pranjal Kumar Dutta Alcatel-Lucent Giles Heron Tellabs Yetik Serbest AT&T Shane Amante Level 3 Communications Expires: May 2008 November 5, 2007 OAM (Operation, Administration and Maintenance) for Virtual Private LAN Service (VPLS) draft-stokes-vkompella-l2vpn-vpls-oam-00.txt Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. This document may only be posted in an Internet-Draft. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html This Internet-Draft will expire on May 5,2008. Stokes, et. al. Expires May 2008 [Page 1] Internet-Draft OAM for VPLS November 2007 Copyright Notice Copyright (C) The IETF Trust (2007). Abstract This document describes a methodology and a set of mechanisms for Operation, Administration and Maintenance (OAM) of a general VPN (Virtual Private Network) service, that is applied here to a Virtual Private LAN Service [RFC4761][RFC4762]. [L2VPN-CFM] describes the capabilities of CFM [802.1ag] and VCCV [VCCV] for OAM operations among the bridge module components in the VPLS. The OAM functions described in this document builds the additional capabilities into VPLS OAM to detect, verify and isolate connectivity faults among the VPLS forwarder components. The methodology adopted extends LSP Ping concepts described in [RFC4379] and VCCV capabilities described in [VCCV]. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119. 1. Table of Contents 1. Table of Contents.................................. 2 2. Terminology ........................................3 3. Introduction........................................3 4. CFM and VCCV capabilities...........................6 5. VPLS OAM Function Placement.........................8 5.1 Discovery........................................10 5.2 Service Verification.............................10 5.3 Connectivity Fault Management....................10 5.3.1 Connectivity Fault Detection.................10 5.3.2 Connectivity Fault Verification..............11 5.3.3 Connectivity Fault Localization..............11 5.4 Partial Mesh Resolution..........................11 5.5 MAC Populate and MAC Purge.......................12 5.6 Performance Monitoring.......................... 12 6. Architectural Considerations........................13 6.1 MEP and MIP Identifiers..........................13 6.2 FIB Traversal....................................14 6.3 Control plane and Dataplane Consistency..........14 7. Packet Formats, Encapsulations and Extensions.......14 7.1 Hierarchical L2 Circuit FEC Stack Sub-type.......15 7.2 L2 VPN ID Sub-TLV................................16 Stokes, at al. Expires May 5, 2008 [Page 2] Internet-Draft OAM for VPLS November 2007 7.3 L2 specific Sub-TLVs.............................17 7.4 VPLS OAM Encapsulation...........................19 7.4.1 Extensions to VCCV to support VPLS OAM.......19 7.4.2 Signaling VPLS OAM capabilities..............20 7.5 Dataplane processing for VPLS OAM................21 7.6 Egress Node Control Plane Processing.............22 7.7 Error Code TLV...................................23 7.8 Vendor-specific Extensions.......................24 8. VPLS OAM Functions..................................25 8.1 VPLS Topology Discovery..........................25 8.1.1 Ethernet VPLS Node TLV.......................26 8.1.2 VPLS Topology Discovery Procedures...........28 8.1.3 Operation of limited VPLS Nodes..............29 8.2 VPLS Service Verification........................29 8.3 VPLS Connectivity Check..........................30 8.4 VPLS Ping........................................30 8.4.1 VPLS Ping Packet Format......................31 8.4.2 VPLS Ping Procedures.........................31 8.5 VPLS Traceroute..................................33 8.5.1 VPLS Traceroute Packet Format................33 8.5.2 VPLS Traceroute Procedures...................35 9. General VPLS Testing Procedures.....................36 10. Load Sharing Considerations........................38 11. Scalability Considerations.........................40 12. Security Considerations............................40 13. IANA Considerations................................41 14. Acknowledgements...................................41 2. Terminology This document uses the terminology defined in [L2VPN-OAM-REQ] [RFC4379], [RFC4761], [RFC4762], [RFC4664], [802.1ag] and [VCCV]. Throughout this document VPLS means the emulated bridged LAN service offered to a customer. H-VPLS means the hierarchical connectivity or layout of MTU and PE devices offering the VPLS[4762]. The terms spoke node and MTU in H-VPLS are used interchangeably. 3. Introduction Virtual Private LAN Service (VPLS), also known as Transparent LAN Service (TLS) is a useful service provider offering and is described in [RFC4761][RFC4762]. A VPLS is created using a collection of one or more point-to-point pseudowires (PWs) [RFC3965] configured in a flat, full-mesh topology. Each PW provides a logical interconnect between two “VPLS Forwarders” in the VPLS capable Provider Edge(PE) devices. The mesh topology of VPLS forwarders provides a LAN segment or broadcast domain that is fully capable of learning and forwarding on Stokes, at al. Expires May 5, 2008 [Page 3] Internet-Draft OAM for VPLS November 2007 Ethernet MAC addresses at the PE devices. This VPLS core configuration can be augmented with additional non- meshed spoke nodes to provide a Hierarchical VPLS (H-VPLS) service [RFC4762]. For the purposes of this document, the terms "H-VPLS" and "H-VPLS instance" are used to signify a generic VPLS instance that may or may not be hierarchical in topology. An Operation, Administration and Maintenance (OAM) provides the instrumentation of telecommunications networks and equips the network operator with the tools to monitor, detect and verify faults in their network infrastructure. Service providers deploying VPLS should be able to test operations of all its components across the entire hierarchy in an H-VPLS. [L2VPN-CFM] describes the capabilities of CFM[802.1ag] and VCCV[VCCV] for OAM among the "bridge" components in the VPLS capable PE devices. However CFM in combination with VCCV does not provide verification or problem determination for “VPLS forwarders”[L2VPN-CFM]. This document describes a generic methodology for OAM among VPN forwarders and demonstrates its ability for OAM operations among the VPLS forwarder components in an H-VPLS. Application of this methodology for other VPN services is outside the scope of this document. The solutions described in this document build the capabilities required to address the VPLS transport specific requirements of VPLS OAM. +-----+ +-----+ + CE1 +--+ +---| CE2 | +-----+ | ................... | +-----+ VPLS A | +----+ +----+ | VPLS A | |VPLS| |VPLS| | +--| PE |--Routed---| PE |--+ +----+ Backbone +----+ / . | . \ _ /\_ +-----+ / . | . \ / \ / \ +-----+ + CE +--+ . | . +--\ Access \----| CE | +-----+ . +----+ . | Network | +-----+ VPLS B .....|VPLS|........ \ / VPLS B | PE | ^ ------- +----+ | | | | | +-----+ | | CE3 | +-- Emulated LAN +-----+ VPLS A Figure 1 Stokes, at al. Expires May 5, 2008 [Page 4] Internet-Draft OAM for VPLS November 2007 It is required to outline the generic architecture of VPLS and its components in order to understand the purpose and context of the VPLS OAM functions described in this document. The diagrams in Figure 1. and Figure 2. demonstrate the VPLS reference model as explained in [RFC4664]. PE devices that are VPLS-capable provide a logical interconnect such that Customer Edge (CE) devices belonging to a specific VPLS appear to be on a single bridged Ethernet. |-----Routed Backbone-----| | (P Routers) |PSN Tunnels, Emulated LAN | |Pseudowires ....................................................................... . | | . . |---------------------|----| |--------|-----------------| . . | --------------------|--- | | -------|---------------- | . . | VPLS Forwarder | | VPLS Forwarder | . . | ----------|------------- | | ----------|------------- | . ..|.................................................................|.. | | Emulated LAN | | | Emulated LAN | | | Interface | VPLS-PEs | | Interface | | | | <----> | | | | ----------|------------ | | ----------|------------ | | | Bridge | | | | Bridge | | | -|--------|---------|-- | | ---|-------|---------|- | |--|--------|---------|----| |----|-------|---------|---| | | | | | | | | Access | | | Access | | | Networks| | | Networks| | | | | | | | | | | | | CE devices CE devices Figure 2 From Figure 2, we see that in VPLS, a CE device attaches, possibly through an access network, to a "bridge" module of a VPLS PE. Within the VPLS PE, the bridge module attaches, through an "Emulated LAN Interface”, to an Emulated LAN. For each VPLS, there is an Emulated LAN instance. Figure 2 shows some internal structure to the Emulated LAN: it consists of "VPLS Forwarder" modules connected by PWs, where the PWs may be traveling through Packet Switched Network(PSN) tunnels over a routed backbone. These tunnels can be IP tunnels, such as Generic Routing Encapsulation (GRE), or MPLS tunnels, established by Resource Reservation Protocol - Traffic Engineering (RSVP-TE) or LDP. Stokes, at al. Expires May 5, 2008 [Page 5] Internet-Draft OAM for VPLS November 2007 A "VPLS instance" consists of a set of VPLS Forwarders (no more than one per PE) connected by the PWs in a full-mesh configuration. In H-VPLS configuration, PWs can also be used to connect the spoke nodes to the VPLS core nodes. Service providers may design a scalable H-VPLS topology by interconnecting multiple full-mesh domains with inter-domain spokes. [RFC4762] describes about multi-domain VPLS services. Note that the context of "inter-domain" here is NOT inter- provider or inter-AS, rather multiple inter-connected full-mesh domains of a single provider. In such network designs data frames between CE devices may traverse multiple VPLS forwarders spanning across the domains. This document defines a set of the OAM mechanisms among the VPLS forwarders in an H-VPLS instance. The PSN network-level OAM is outside the scope of the document. The methodology adopted in the mechanisms require extensions to the existing LSP Ping concepts described in [RFC4379] and VCCV defined in [VCCV]. The inter-provider or inter-AS VPLS OAM aspects would be addressed in a later revision. Note that in inter-domain VPLS the OAM domains are not nested domains, rather placed at the same level. 4. CFM [802.1ag] and VCCV [VCCV] capabilities Ethernet OAM, as being defined in [802.1ag] and [Y.1731], offers the set of functionalities that have been designed for proactive and on-demand OAM for the ethernet bridge layer. [802.1ag] provides the tools for the Connectivity Fault Management (CFM) and [Y.1731] adds capabilities for the performance monitoring, measurement aspects into Ethernet OAM. In its current form, CFM has addressed the "bridging" aspects of the ethernet service layer in VPLS. The OAM tools provided by CFM in combination with PW VCCV methods can be deployed in H-VPLS for troubleshooting and monitoring the operation of bridge modules in a VPLS service[L2VPN-CFM]. [L2VPN-CFM] describes about the ability of the combination of CFM and VCCV to meet OAM requirements of a L2VPN. While the combination of CFM and VCCV provides significant information for a network operator, the combination does not provide a comprehensive OAM solution for an H-VPLS environment. When applied to the [RFC4664] reference model of a VPLS PE node, CFM does not provide verification or problem determination of the emulated LAN, which is the network of VPLS forwarders. VCCV provides OAM coverage for the PWs alone in the emulated LAN, but does not address the association or mapping of PWs with the VPLS forwarders. OAM functions for H-VPLS are required to address the VPLS transport specific issues with adequate granularity to provide a comprehensive set of monitoring, troubleshooting Stokes, at al. Expires May 5, 2008 [Page 6] Internet-Draft OAM for VPLS November 2007 facility to the VPLS service provider. Some of the issues associated with VPLS transport are as below: - In H-VPLS, ethernet service infrastructure is provided by a set of ethernet PWs setup by control plane protocols such as LDP, BGP or L2TP. [RFC4761] describes VPLS with BGP as PW setup and control protocol. [RFC4762] describes VPLS with LDP as PW setup and control protocol. [RFC4667] defines the L2TP extensions for PW signaling in L2VPNs. Verification of control plane and data plane consistency is a significant and integral function of VPLS OAM. Those issues can occur if there are bugs in the encapsulation/decapsulation procedures at the PEs, or bugs in the forwarding procedures at intermediate nodes(especially where dataplane and control plane are decoupled). CFM is agnostic of the PW control plane and the PW transport that provides the emulated ethernet bridge service. In order to build control and data plane validation capabilities into [802.1ag], CFM protocol would be required to carry at least VPLS transport layer (PW) control information. - For troubleshooting and validation, it is necessary to know the PW behind which a specific MAC address is located. As CFM entities are not present in intermediate VPLS forwarders in an H- VPLS[L2VPN-CFM], it is not possible to obtain information with such granularity from CFM operations. - To determine the exact cause of the failure, it is desirable to have the ability in VPLS OAM entities to send replies to the originator of an OAM message via out of band channels. CFM messages (LBM, LTM etc) cannot report back problems to the sender in case of PW dataplane failure in the direction in which reply is to be sent. - When PSN is MPLS based, some implementations may provide for load sharing a PW across multiple MPLS PSN Tunnels. The OAM functions should be able convey the multipath information and test the operations over each of the tunnels. Such capabilities are not present in [802.1ag] as the architecture is agnostic to MPLS transport and out of its scope. A comprehensive VPLS OAM solution should also address such issues. - A VPLS provider should be able to perform offline verification of MAC learning and MAC forwarding capabilities among the VPLS forwarders without the customer data traffic. It is desirable to have tools that enable the operator to populate and purge MAC addresses in a VPLS instance. Such capabilities are not present in CFM tools. Stokes, at al. Expires May 5, 2008 [Page 7] Internet-Draft OAM for VPLS November 2007 Combination of CFM and VCCV leaves a significant portion of the emulated LAN completely opaque from a standardized OAM perspective [L2VPN-CFM]. In summary, a comprehensive VPLS OAM solution is required to address both bridging and transport aspects of the VPLS. The solutions described in [L2VPN-CFM] and the solutions proposed in this document may co-exist in a VPLS service provider network. Moreover, it is preferable to retain a layered approach to VPLS OAM, with CFM to be used for Ethernet Service OAM and solutions in this document to address VPLS forwarder and transport specific issues. 5. VPLS OAM Function Placement Figure 3. shows the placement of VPLS OAM functions described in this document inline with the positioning of CFM and PW VCCV components described in [L2VPN-CFM]. A VPLS OAM domain comprises of the set of VPLS forwarders located in the service aware nodes in an H-VPLS instance. A Maintenance Association Endpoint (MEP) is created for the VPLS forwarder in any H-VPLS node with a customer attachment circuit (AC). A MEP or a Maintenance Domain Intermediate Point(MIP) is created for a VPLS forwarder in any H-VPLS node without a local AC. Since MEPs are located at the edge of the H-VPLS, they are responsible for filtering outbound OAM frames from leaving the VPLS OAM domain. The term "VPLS OAM Entity" is used throughout the document to refer to a VPLS MEP or MIP corresponding to a VPLS forwarder. CFM packets generated by bridge MEPs or forwarded by bridge MIPs are not processed by VPLS OAM Entities; those are forwarded by the VPLS forwarder as data packets based on their destination MAC addresses. VPLS OAM Packets are originated and exchanged only among the VPLS OAM entities, and are never forwarded to the bridge modules. Bridge MEPs at VPLS edge devices may participate as MIPs at customer MD level 5 and would respond to LTM[802.1ag] messages received from customer domain. The VPLS OAM functions should expose the VPLS transport issues in adequate depth but should not expose such details to the customer OAM. It is also important to have such layered approach for security reasons. Stokes, at al. Expires May 5, 2008 [Page 8] Internet-Draft OAM for VPLS November 2007 |--- PW-1 ---| |- PW-13 -| |--- PW-3 ---| Emulated | | | | | | LAN | | | | | | ..................................................................... . | | | | | | . . |-------|-----| |---|-----|---| |---|-----|---| |-----|-------| . . | ----------- | | ----------- | | ----------- | | ----------- | . . | VPLS | | VPLS | | VPLS | | VPLS | . . | Forwarder | | Forwarder | | Forwarder | | Forwarder | . . | VPLS OAM MEP| | VPLS OAM MIP| | VPLS OAM MIP| | VPLS OAM MEP| . . | -----|----- | | -----|----- | | ----------- | | -----|----- | . ..|.................................................|................ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | -----|----- | | -----|----- | | -----|----- | | -----|----- | | |MEP X MD | | | |MEP X MD | | | |MEP X MD | | | |MEP X MD | | | | | LVL| | | | | Lvl| | | | | Lvl| | | | | LVL| | | | | 3 | | | | | 3 | | | | | 3 | | | | | 3 | | | | | | | | | | | | | | | | | | | |MIP X MD | | | |MIP X MD | | | |MIP X MD | | | |MIP X MD | | | | | LVL| | | | | LVL| | | | | LVL| | | | | LVL| | | | | 4 | | | | | 4 | | | | | 4 | | | | | 4 | | | | Bridge | | | | Bridge | | | | Bridge | | | | Bridge | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |MEP X MD | | | | | | | | | | | |MEP X MD | | | | | LVL| | | | | | | | | | | | | LVL| | | | | 4 | | | | | | | | | | | | | 4 | | | | | | | | |---------| | | |---------| | | | | | | | | | | | |-------------| |-------------| | | | | | | |MIP X MD | | | |MIP X MD | | | | | LVL| | PE1-rs PE3-rs | | | LVL| | | | | 5 | | | | | 5 | | | | | | | | | | | | | -----|----- | | -----|----- | |------|------| |------|------| | | | | CE-1 CE-4 MTU1-s MTU3-s Figure 3: HVPLS network with CFM,VCCV and VPLS OAM Stokes, at al. Expires May 5, 2008 [Page 9] Internet-Draft OAM for VPLS November 2007 A VPLS capable PE device may be connected with customer AC as well as spoke. In such cases the corresponding VPLS OAM entity performs both MEP and MIP functions. A VPLS OAM entity is required to support the following set of functionalities. 5.1 Discovery Discovery function enables a VPLS OAM Entity to learn sufficient information (e.g IP addresses, MAC addreses etc.) about the other VPLS OAM entities(current layout of PEs and MTUs participating in the H-VPLS) in the same VPLS service. Such function is also necessary for the utilities defined in this document to exchange the OAM packets among the VPLS OAM entities. 5.2 Service Verification Service verification has to do with whether the service aware nodes have been consistently configured for the service or not. VPLS services may be provisioned manually or may be auto discovered with BGP[L2VPN-FRMK]. It is desirable to do service verification and report misconfiguration to the operator on a proactive basis. 5.3 Connectivity Fault Management Connectivity Fault Management (CFM) is an inherent aspect of OAM and signifies the functions to validate that customer (service) frames can be exchanged between any two VPLS service forwarders in VPLS. There are three aspects of fault management in a provider network. It is to note that the term "CFM" used in this context is generic and not the CFM framework in [802.1ag]. 5.3.1 Connectivity Fault Detection The primary goal of Connectivity Fault Detection is proactive connectivity fault monitoring among the VPLS Forwarders in a VPLS. A secondary goal of such permanent continuity check is topology discovery and service verification in the H-VPLS. This document describes how the existing standardized mechanisms can be used for fault detection. Stokes, at al. Expires May 5, 2008 [Page 10] Internet-Draft OAM for VPLS November 2007 5.3.2 Connectivity Fault Verification There are five logical steps to verify the operation of an H-VPLS: - Verify that all H-VPLS nodes are operational - Verify that all H-VPLS peer sessions are operational - Verify that all H-VPLS Tunnels are transporting data packets correctly. - Verify that VPLS service frames can be exchanged between any two VPLS forwarders in the H-VPLS. - Verify that actual customer devices can communicate with devices at any site. Existing mechanisms can verify the first three steps. This document describes how to address the final two issues. This requires exchange of VPLS OAM messages among the VPLS OAM entities and the intermediate VPLS forwarders to forward those messages using Forwarding Information Base (FIB) lookups just as they would be for conventional customer (service) frame. This document describes the VPLS Ping function that addresses this requirement. 5.3.3 Connectivity Fault Localization Connectivity Fault Localization is to isolate and diagnose failures in case a VPLS forwarder or customer MAC address is not reachable from the Connectivity Fault Verification phase. A secondary goal of fault isolation is to provide information of VPLS forwarders in the H-VPLS that is traversed by service frames to reach the destination MAC address. This document describes the VPLS Traceroute function for isolation of such faults. Both the VPLS Ping and VPLS Traceroute functions have the ability to send replies to the sender through the control plane (out of band channel), so as to allow for error reporting when all else fails. 5.4 Partial Mesh Detection and Resolution The full mesh of PWs in the H-VPLS core with split horizon rules [RFC4761][RFC4762] emulates a LAN segment over MPLS/IP PSN network. If one or more of these PWs is absent, so that there is not really a full mesh, various higher layer protocols that expect a LAN-like service may fail to work as expected [ROSEN_MESH]. This includes protocols from routing to bridge control. Therefore it is desirable to have procedures that enables the VPLS OAM entities (in the PE devices participating in the full-mesh) to determine automatically whether there is really a full-mesh or not. It is also desirable to Stokes, at al. Expires May 5, 2008 [Page 11] Internet-Draft OAM for VPLS November 2007 have procedures that cause the H-VPLS components to adapt to PW failures[L2VPN-OAM-REQ]. There are following ways partial mesh may occur in VPLS[ROSEN-MESH]. A. Configuration errors. B. Failure of the auto-discovery process. C. Failure of the control plane to properly establish all the necessary PWs. This in turn may be due to bugs, or to resource shortages at the PEs. D. Failure of the data plane to carry traffic correctly on all the established PWs comprising the full-mesh. This can occur if there are bugs in the encapsulation/decapsulation procedures at the PEs, or bugs in the forwarding procedures at intermediate nodes (especially where dataplane and control plane are decoupled). To monitor and detect mesh PW failures, existing PW VCCV mechanisms can be used at PW level. At VPLS forwarder level keeplive messages can be sent between the corresponding VPLS OAM entities using the methodology described in this document. The PW control protocols can be extended to notify mesh failures in a scalable and reliable manner. The critical component of the solution is the ability to react to a partial mesh after such condition is detected. This requires efficient coordination and computation among the participating VPLS OAM entities such that a consistent set of PWs are "removed" from the mesh topology, thus resulting in a "subset" full-mesh. A solution for partial mesh resolution is currently under progress and details will be provided in a future revision. 5.5 MAC Populate and MAC Purge It is useful to test the MAC learning and forwarding capabilities in the VPLS instance in the absence of customer's data traffic. During provisioning a VPLS provider may use MAC Populate functionality to install MAC addresses across the VPLS forwarders and perform VPLS Ping, VPLS Traceroute functions on those MAC addresses. MAC Purge can used to remove the installed MAC addresses from the VPLS forwarders. A detailed methodology for MAC Populate and Purge will be provided in a future revision. 5.6 Performance Monitoring Performance measurement of a VPLS service is required to verify Service Level Agreement (SLA)s with the customer to whom a VPLS Stokes, at al. Expires May 5, 2008 [Page 12] Internet-Draft OAM for VPLS November 2007 service is being offered. Specific performance measurement or diagnostic functions are Frame Loss, Frame Delay, Frame Delay variation etc [L2VPN-OAM-REQ]. Flooded frames in H-VPLS may be unknown unicast, multicast, or broadcast frames that require frame replication along a point-to- multipoint (P2MP) downstream path. As a consequence, the performance of flooded and non-flooded frames can be significantly different. Multicast applications e.g multiparty conference calls, can be sensitive to frame delay and frame delay variation and P2MP diagnostics is required for such applications in H-VPLS. Existing performance measurement functionalities in Ethernet OAM [Y.1731] can be used to test the performance of an H-VPLS service. Moreover, service providers are already using external measurement devices to perform some of these functions[L2VPN-CFM]. This version of the document does not address any performance measurement issues and may be described in a future revision if requirement of such solution arises. 6. Architectural Considerations This section describes the structural model or the methodology used in the VPLS OAM mechanisms described in this document. The encapsulation of the VPLS OAM packets and the traversal methodology across the VPLS forwarders is described. It is to note that the methodology is designed for OAM in a general L2 VPN Service. OAM solutions for L2 VPN services other than VPLS are out of scope of this version of the document. 6.1 MEP and MIP Identifiers The methodology used in this document uses a MAC address to identify the VPLS OAM entities, which is unique in the H-VPLS. The MAC address SHOULD be a MAC address of the service node such that an un-encapsulated frame received from an H-VPLS service node with this destination MAC address will be delivered to the receiving node's control plane. We also use loopback or "always on" IP addresses of the VPLS service aware node that allows the MEPs and MIPs to address each other, which is actually important for error reporting when everything else fails. Apart from the IP and MAC addresses the MEP indentifier is qualified with the VPLS instance identifier. It is to note that for VPLS OAM it is not required to explicitly configure MEP or MIP corresponding to a VPLS forwarder. When a VPLS service is provisioned in a PE device, the corresponding VPLS OAM entity is automatically configured. Stokes, at al. Expires May 5, 2008 [Page 13] Internet-Draft OAM for VPLS November 2007 6.2 FIB Traversal In order to determine state and accurate performance measurement of a VPLS service, VPLS OAM packets should receive the same data path forwarding behavior as that of VPLS service frames/packets [L2VPN- OAM-REQ]. The VPLS OAM packets traverse the VPLS Forwarders using the same forwarding lookup tables in the dataplane as the Customer frames. 6.3 Control plane and Data plane consistency It is desirable to have the capability in VPLS OAM that enables a Service Provider to troubleshoot and isolate issues associated with VPLS control plane and transport. Such tools detect misconfiguration or inconsistency between control plane and dataplane in an H-VPLS, esp. when both are decoupled from each other. VPLS service frames may be black holed or may be leaked to other service instances due to misconfiguration of the PWs in data plane. Such inconsistency may also occur temporarily when control plane is in transient state, thus causing service disruption or traffic leaks between VPLS instances. During verification and localization of faults it is imperative to detect such issues in the VPLS forwarders with adequate level of accuracy. In the mechanisms described in this document, typically connectivity is first checked against the dataplane. If a VPLS OAM request packet makes it to the destination OAM entity, the reply packet is sent along the data plane. Otherwise, if a reply is not received within the desired interval, the sender sends another request packet along the data plane, requesting a reply back on the control plane. If this also fails, a final attempt may be made, with request sent along the control plane, and the reply back along the control plane. If this fails, then the network is probably partitioned. Such multi-step probing facilitates determination of control plane and dataplane inconsistencies with adequate accuracy. Moreover it is important to send reply to the originator via out of band channels when the dataplane in the reverse direction has failed. 7. Packet Formats, Encapsulations and Extensions VPLS OAM uses the packet format and encapsulations of MPLS Echo Request/Reply (LSP Ping) described in [RFC4379]. This section describes the extensions proposed to LSP Ping and VCCV required for all VPLS OAM operations. The extensions specific to a VPLS OAM function are defined when such functions are described in the later sections of the document. It is suggested that first-time readers Stokes, at al. Expires May 5, 2008 [Page 14] Internet-Draft OAM for VPLS November 2007 skip this section and read subsequent sections; the document is structured in this way to avoid forward references. The common header format of MPLS Echo Request/Reply packets is described in [RFC4379]. An MPLS Echo Request/Reply is sent as an IPV4 or IPV6 packet encapsulated in UDP. The V flag (Validate FEC Stack) in the header SHOULD be set in all VPLS OAM messages to perform control plane and dataplane consistency validation. The header encapsulates one or more TLVs. 7.1 Hierarchical L2 Circuit FEC Stack Sub-type MPLS Echo Request/Reply message carries Target FEC Stack TLV [RFC4379] that contains the control plane information to be validated against the data plane at the receiver. For L2 VPN OAM purposes a new Target FEC Stack Sub-type is defined as "Hierarchical L2 Circuit" that carries the unique identifier of the L2 VPN for which an OAM message is exchanged. Sub-Type Length Value Field -------- ------ ----------- 17 variable Hierarchical L2 Circuit The format of Hierarchical L2 Circuit FEC Stack Sub-type is shown below: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | L2VPN ID Sub-TLV | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Encapsulation Type | Must Be Zero | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | L2 specific Sub-TLV (Optional) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 5. Hierarchical L2 Circuit Sub-type L2VPN ID Sub-TLV = This Sub-TLV carries the identification of the L2VPN. Details of this Sub-TLV are provided in Section 7.2. Stokes, at al. Expires May 5, 2008 [Page 15] Internet-Draft OAM for VPLS November 2007 Encapsulation Type = This is the same type field described in [RFC4447],[RFC4446]. L2 specific Sub-TLV = This Sub-TLV carries additional information specific to a L2 VPN service. See Section 7.3 for details. Note that L2VPN ID Sub-TLV and L2 specific Sub-TLV follows the identical Sub-TLV format prescribed in [RFC4379]. The interpretation of the specific Sub-TLV depends on the position of its occurrence. 7.2 L2VPN ID Sub-TLV The L2VPN ID Sub-TLV follows the Sub-TLV format prescribed in [RFC4379] as below: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Value | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 6. L2VPN ID Sub-TLV The type of the L2VPN ID is based on the L2 VPN identifier associated with the specific signaling protocol used for setting up the PWs in the L2 VPN. Following types are defined in this version: L2VPN ID Type Description ============= ====================================== 0x0001 Value field carries VC-ID if PW-id FEC Element (FEC128) is used by LDP to signal the PW [RFC4447]. 0x0002 Value field carries Attachment Group Identifier (AGI) from Generalized PW-id FEC Element (FEC129) if LDP is used to signal the PW[RFC4447]. If L2TP is used as the signaling protocol to setup the PW then AGI is taken from the AGI AVP[RFC4667]. 0x0003 Value field carries Route Distinguisher (RD) if the BGP procedures defined in [RFC4761] is used to setup the PWs in a L2VPN. Stokes, at al. Expires May 5, 2008 [Page 16] Internet-Draft OAM for VPLS November 2007 7.3 L2 specific Sub-TLVs The "L2 specific Sub-TLVs" use the same Type field as defined in [RFC4446]. These are: VC type Description ======= ================================== 0x0001 Frame Relay DLCI (Martini Mode ) 0x0002 ATM AAL5 SDU VCC transport 0x0003 ATM transparent cell transport 0x0004 Ethernet Tagged Mode 0x0005 Ethernet 0x0006 HDLC 0x0007 PPP 0x0008 SONET/SDH Circuit Emulation Service Over MPLS 0x0009 ATM n-to-one VCC cell transport 0x000A ATM n-to-one VPC cell transport 0x000B IP Layer2 Transport 0x000C ATM one-to-one VCC Cell Mode 0x000D ATM one-to-one VPC Cell Mode 0x000E ATM AAL5 PDU VCC transport 0x000F Frame-Relay Port mode 0x0010 SONET/SDH Circuit Emulation over Packet 0x0011 Structure-agnostic E1 over Packet 0x0012 Structure-agnostic T1 (DS1) over Packet 0x0013 Structure-agnostic E3 over Packet 0x0014 Structure-agnostic T3 (DS3) over Packet 0x0015 CESoPSN basic mode 0x0016 TDMoIP AAL1 Mode 0x0017 CESoPSN TDM with CAS 0x0018 TDMoIP AAL2 Mode 0x0019 Frame Relay DLCI [RFC4762] specifies that Ethernet or Ethernet Tagged Mode VC type to used for VPLS. For VPLS, the L2 specific Sub-TLV SHOULD contain sufficient information to identify the target remote service node and to allow the remote node to be able to respond. The L2-specific Sub-TLV is included in VPLS OAM messages to support implementations where processing is distributed, in case the MAC header is not passed along with the IP packet to control plane. So in such cases the sender and target MAC addresses are identified from the L2-specific Sub-TLV. Stokes, at al. Expires May 5, 2008 [Page 17] Internet-Draft OAM for VPLS November 2007 The following Sub-TLV is proposed for VC type Ethernet: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type (0x0005) | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Target Ethernet MAC address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | 802.1Q Tag | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sender Ethernet MAC address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | 802.1Q Tag | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 7. L2-specific Sub-TLV Type The field MUST be set to a value of 0x0005 for Ethernet VC. Length This field specifies the total length of the Value fields in octets. Target Ethernet MAC address This field specifies the target Ethernet MAC address of the remote spoke or customer device. It is included here to assist implementations when processing VPLS OAM message reply. Sender Ethernet MAC address This field specifies the sender's Ethernet MAC that can be used in a reply. 802.1Q Tag These fields specify the 802.1Q Tag associated with the Target and Sender Ethernet MACs described above. If the Ethernet address is associated with a VLAN, the 802.1Q Tag field MUST be set to the VLAN tag. If the MAC address is not associated with a VLAN (untagged VLAN), it MUST be set to zero. Since an 802.1Q tag is 12-bits, the high 4 bits of the fields MUST be set to zero. Stokes, at al. Expires May 5, 2008 [Page 18] Internet-Draft OAM for VPLS November 2007 7.4 VPLS OAM Encapsulation A VPLS OAM Packet should look like a customer data frame that allows lookup methods and encapsulations used in the dataplane to be preserved. The VPLS OAM packet receives the same forwarding treatment as the customer frames. It is desirable for both operational and security reasons to be able to easily recognize in the data plane that a received packet is a VPLS OAM packet. VPLS OAM PDUs are sent as LSP Ping (MPLS echo request/reply) packets[RFC4379]. The LSP Ping header MUST be used in accordance with [RFC4379] and MUST contain the target FEC Stack containing the L2VPN ID Sub-TLV. The Sub-TLV indicates the VPLS instance to be verified. The VPLS OAM PDU is encapsulated in Ethernet header identical to the conventional customer frames and can be forwarded using FIB lookup. Throughout the document we would use the term “VPLS IP UDP LSP Ping packet” to specify the format used for VPLS OAM packet. The PW Associated Channel Header (PW-ACH)[RFC4385] is used to transport and identify the VPLS OAM packets between the VPLS forwarders. 7.4.1 Extensions to VCCV to Support VPLS OAM VCCV [VCCV] provides various control channel(CC) types between a PW's ingress and egress points over which connectivity verification messages can be sent for PW OAM. A PW provisioned in a VPLS may be a Single Segment PW (SS-PW) or a Multi Segment PW (MS-PW). MH-VCCV control channel type is proposed in [SEG-PW] for diagnostics across the MS-PW. This document defines a new control channel type for transporting VPLS OAM packets across a PW. The PW end-points should be able to continue operating the PW diagnostics packets over VCCV and to discriminate those from the VPLS OAM Packets that uses the VCCV control channel. The VPLS OAM packets are indicated using a new L2VPN-OAM Control Channel. The format of the corresponding L2VPN-VCCV Control Word (CW) that encapsulates the VPLS OAM Packet is described as below: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 1| 0x00 | Reserved = 0 | Channel Type = L2VPN OAM| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | L2VPN TTL | TBD | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 8. L2VPN-VCCV Control Word Stokes, at al. Expires May 5, 2008 [Page 19] Internet-Draft OAM for VPLS November 2007 The L2VPN-VCCV CW is meaningful only to the egress point of a PW used in the L2VPN where it is intercepted and passed on to the L2VPN forwarder that processes the CW. When the PW is used in VPLS, it is the VPLS forwarder that processes the L2VPN-VCCV CW. In MS-PW case, the PW switching points (S-PE) SHOULD NOT intercept or process the CW and SHOULD transparently pass it along until it reaches the L2VPN forwarder at the PW egress. A new CC Type is defined for L2VPN OAM. The use of L2VPN TTL is explained in detail in section 7.6. 7.4.2 Signaling VPLS OAM capabilities Like in VCCV for PW diagnostics, VPLS OAM capabilities are signaled using the VCCV parameter included in the interface parameter field of the PW ID FEC TLV or the Sub-TLV interface parameter of the Generalized PW ID FEC TLV as described in [VCCV]. The L2VPN-VCCV CW is indicated using a new CC type in the VCCV capability parameter field. Signaling of such capabilities when BGP is used as PW setup in VPLS[RFC4761] will be addressed in a later revision. The VCCV parameter ID is defined as follows in [RFC4446]: Parameter ID Length Description 0x0c 4 VCCV The format of the VCCV parameter field is defined as follows in [VCCV]: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | 0x0c | 0x04 | CC Types | CV Types | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 9. VCCV Parameter Field 0x01 Type 1: PWE3 control word with 0001b as first nibble as defined in [RFC4385]. 0x02 Type 2: MPLS Router Alert Label. 0x04 Type 3: MPLS PW De-multiplexor Label TTL = 1. 0x08 Type 4: MH-VCCV Control Word 0x10 Type 5: L2VPN-VCCV Control Word Stokes, at al. Expires May 5, 2008 [Page 20] Internet-Draft OAM for VPLS November 2007 7.5 Dataplane processing for VPLS OAM For VPLS OAM functions such as VPLS Traceroute to work properly, the basic operation of H-VPLS needs further definition in the VPLS forwarder. In a typical two tier hierarchy H-VPLS, a data packet normally goes from an H-VPLS spoke node (MTU/u-PE) forwarder to an H- VPLS core node (PE) forwarder. In the core node forwarder, the PW Label received from the spoke node is used to determine the VPLS instance with which the encapsulated frame is associated. The frame is un-encapsulated and a check is made for the frame's destination MAC address. If the destination MAC address is known, the frame is forwarded to the downstream core node forwarder from whom the MAC address is learned. As the frame is forwarded, the frame is again encapsulated using the PW label received from the downstream core node forwarder. A similar process occurs at the downstream core node forwarder. The PW label is checked, the frame is un-encapsulated, the destination MAC address is checked, and the frame is again encapsulated and sent to the remote spoke node forwarder. The packet containing the newly encapsulated frame uses the PW label received from the destination spoke node forwarder. Each time this process occurs, if a VPLS OAM packet is received as the PW payload then VPLS forwarder SHOULD process the associated L2VPN-VCCV CW. As the VPLS OAM Packet exits the PW at egress, the VPLS forwarder identifies the packet belonging to a specific VPLS instance from the PW Label. If the L2VPN TTL in the received packet is 1, then the packet MUST be passed to the node's control plane (VPLS OAM Entity). If it cannot be passed to the control plane, the packet SHOULD be discarded. The packet MUST NOT be forwarded to the customer network. Note that the rate at which packets are passed to the control plane SHOULD be regulated to prevent overloading or DoS attacks. If the received TTL value is not 1 then the forwarder checks if the destination MAC address in the ethernet frame is a MAC identifying this forwarder-cum-VPLS OAM entity. Note that it also checks its forwarding database associated with this VPLS to see if the target MAC is associated with(learned from) a locally attached customer network. If the MAC is found to be local, then this node is the egress node for the VPLS OAM packet and MUST be passed to the node's control plane. If the destination MAC address is not local and learned from a downstream VPLS forwarder, then the VPLS OAM packet along with the CW SHOULD be sent to the downstream node with the PW label received from Stokes, at al. Expires May 5, 2008 [Page 21] Internet-Draft OAM for VPLS November 2007 it. If the destination MAC address is not known by this forwarder, then it SHOULD flood the VPLS OAM packet to all downstream VPLS forwarders. Each time the VPLS OAM packet is forwarded, the L2VPN TTL SHOULD be decremented and the result SHOULD be placed in the L2VPN TTL field in the outgoing CW. It is to note that L2VPN TTL is available only to the VPLS forwarder as it receives the frames from an emulated LAN interface(PW). If the destination MAC address in the VPLS OAM packet is not known then the packet is flooded to multiple VPLS forwarders. In that case the decremented L2VPN TTL described above is placed in each outgoing CW. In normal operation, the L2VPN TTL value SHOULD be set to a value larger than the number of "H-VPLS Hops" or VPLS forwarders through which the data packet will traverse, e.g. 255. 7.6 Egress Node Control Plane Processing This section describes the processing of a VPLS OAM packet at the egress node control plane. The egress node control plane receives the VPLS OAM packet as IP UDP LSP Ping packet. The Hierarchical L2 Circuit FEC Stack Sub-type identifies it as VPLS OAM packet and the packet is delivered to the VPLS OAM entity for processing. The VPLS OAM entity checks the Target Ethernet MAC address field in L2- specific Sub-TLV to determine if it is a MAC address that identifies the local VPLS forwarder. If it is, a successful reply is returned. If the MAC address does not belong to the local VPLS forwarder, an additional check is made of the forwarding database associated with the VPLS indicated by the L2VPN ID Field in Hierarchical L2 Circuit FEC Stack Sub-type. If the MAC address is present and is associated with a locally attached customer network for this H-VPLS, a successful reply is returned. If not, the L2VPN TTL is checked to determine if this is a VPLS Traceroute Packet and reply is sent accordingly. VPLS OAM replies require new LSP Ping Return Code information. To avoid defining codes that are specific to VPLS, an encoding of the Error Code TLV is proposed. This encoding is shown in Section 7.8. Note that this method of operation allows VPLS OAM functions to check for both remote node MACs and the remote CE device MACs. When the target MAC address in a VPLS OAM packet is unknown in a core node, the packet is flooded. As such, it potentially reaches other spoke nodes in addition to the spoke node where the destination is actually present. Stokes, at al. Expires May 5, 2008 [Page 22] Internet-Draft OAM for VPLS November 2007 7.7 Error Code TLV The proposed Error Code TLV for use with [RFC4379] is shown below. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type (0x0004) | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Error Code Sub-TLV | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 10. Error Code TLV Type The field MUST be set to a value of 0x0004 for Error Code TLV and is subjected to IANA approval. Length This field specifies the total length of the Value fields in octets. In this case the Value field is an Error Code Sub-TLV. Error Code Sub-TLV The proposed encoding for these Sub-TLVs is shown below. The Error Code Sub-TLV uses the Target FEC Sub-Type values defined in [RFC4379] as the type field value. The proposed encoding for a VPLS FEC Stack Sub-Type is shown below. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type (0x000A) | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Error Code | Must Be Zero | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 11. Error Code Sub-TLV Type The field MUST be set to a value of 0x000A for Hierarchical L2 Circuit. Length This field specifies the total length of the Value field Stokes, at al. Expires May 5, 2008 [Page 23] Internet-Draft OAM for VPLS November 2007 in octets. In this case the Value field is an Error Code Sub-TLV. Error Code The following codes are defined: Value Meaning ----- ------- 1 Replying router is the egress for the given MAC address 2 Replying router is not a member of the indicated HVPLS 3 Replying router is a member of the indicated HVPLS, but is not one of the "Downstream Routers" 4 Replying router is one of the "Downstream Routers" and it has the given MAC address in its forwarding database (but it is not the egress) 5 Replying router is one of the "Downstream Routers" but the label mapping is incorrect 6 Replying router is one of the "Downstream Routers" but it does not have the given MAC address in its forwarding database 7 Replying router is one of the "Downstream Routers" and is a member of the indicated HVPLS, but it does not have the given MAC address and is unable to flood the request 8 Replying Router is one of the “Downstream Routers” and is a member of the indicated VPLS, and has the given MAC address, but the MAC is not learned from a downstream node and is unable to forward the request. 7.8 Vendor-specific Extensions A vendor TLV is defined for vendor-specific extensions. One of the issues with defining TLVs for service definitions is that there is no standard for service definitions. It may be possible to exchange high-level information such as operational status, but finer details are sometimes vendor-specific. Therefore a new vendor-specific TLV type is proposed. The resulting list of TLV types is shown below. Type # Value Field ------ ----------- 1 Target FEC Stack 2 Downstream Mapping 3 Pad Stokes, at al. Expires May 5, 2008 [Page 24] Internet-Draft OAM for VPLS November 2007 4 Not Assigned 5 Vendor Enterprise Number 6 Not Assigned 7 Interface and Label Stack 8 Not Assigned 9 Errored TLVs 10 Reply TOS Byte 11 Vendor-specific The following format is proposed for the Vendor-specific TLV: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type = (0x0005) | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Vendor OUI | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | ~ Value ~ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 12. Vendor Specific TLV A vendor MAY encode vendor-specific information in these vendor TLVs. In general, if possible, the TLVs SHOULD contain values that are strings, so that all vendors are able to display the values. A request or response of a VPLS Ping or VPLS Traceroute MAY contain one or more vendor TLVs. A VPLS OAM entity that does not understand a vendor TLV MAY ignore but SHOULD forward it. 8. VPLS OAM Functions This version of the document defines the following VPLS OAM functions using the methodology and extensions described in Section 7. 8.1 VPLS Topology Discovery A spoke node (MTU) normally knows the IP address of the immediate core node (PE) with which the H-VPLS spoke is connected to. It typically does not know the IP addresses or MAC addresses of remote spoke nodes and remote core nodes. In order to perform VPLS OAM functions such as VPLS Ping, VPLS Traceroute etc. to check connectivity between the H-VPLS nodes, a VPLS OAM entity must know Stokes, at al. Expires May 5, 2008 [Page 25] Internet-Draft OAM for VPLS November 2007 the MAC address of a remote VPLS OAM entity. Furthermore, it is desirable to identify a VPLS OAM entity by its IP address for out of band error reporting when everything else fails. The bridge MEPs [L2VPN-CFM] located in the VPLS PE devices may contain the CCM database that has MAC address information of all other bridge MEPs in that VPLS instance. Note that the bridge MEP and the VPLS MEP in a service aware node may use the same MAC address. [L2VPN-CFM] further describes the method of distributing IP addresses of the bridge MEPs by using the optional Sender TLV. However the CCM database does not contain information of the VPLS OAM MIPs. Both IP and MAC address of a VPLS OAM entity could be distributed through the control plane protocols used to setup the PWs in the H-VPLS. It is also desirable to have such mechanism in control plane in order to do VPLS service verification. See section 8.2 for details. [RFC4762] defined a MAC TLV that is used in a LDP Address Withdraw Message for MAC address withdrawal in topology change. In a similar manner, a LDP Address Message is used to distribute the required information about VPLS OAM entities across the OAM domain. It is possible to extend such mechanisms into other signaling protocols, in BGP auto discovery[RFC4761] and in L2TP[RFC4667]. 8.1.1 Ethernet VPLS node TLV To distribute H-VPLS node information, a new TLV is proposed for use with a LDP Address Message. This new Ethernet VPLS node TLV is shown below. The type field MUST be 0x406 and is subjected to IANA approval. Stokes, at al. Expires May 5, 2008 [Page 26] Internet-Draft OAM for VPLS November 2007 For IPv4: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |U|F| Type (0x406) | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Reserved | Address Family | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IPv4 Address of VPLS node #1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MAC address of VPLS node #1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | CoS |K| 802.1Q Tag #1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IPv4 address of VPLS node #2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MAC address of VPLS node #2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | CoS |K| 802.1Q Tag #2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 13. Ethernet VPLS Node TLV. The Ethernet VPLS Node TLV carries node information (MAC Address and IP Address) for a list of nodes. Note that the node's IP address is included to allow remote nodes to correlate the node's MAC address with its IP address. In this manner an operator can request operations such VPLS Ping or VPLS Traceroute by specifying the IP address of the remote service node. The CoS bits MAY be used to specify class of service provisioned for the VPLS instance. Details of its use is a subject of further study. The K-bit indicates if the node has enabled transmission of periodic keepalives for proactive connectivity fault detection with this node. The receivers of the node information MAY detect connectivity faults with this node by monitoring its the keepalives. The scope of the Ethernet VPLS node TLV is the VPLS instance specified in the FEC TLV in the LDP Address Message. If any parameter in the VPLS Node TLV is modified by a node then it MUST withdraw the node information for the VPLS advertised earlier and SHOULD re-advertise with the new parameters. Stokes, at al. Expires May 5, 2008 [Page 27] Internet-Draft OAM for VPLS November 2007 8.1.2 VPLS Topology Discovery Procedures This section describes the VPLS topology discovery using the control plane mechanism. A method is described when LDP is used as the signaling protocol for setting up the PWs in H-VPLS[RFC4762]. When a spoke node and a core node establish a new peer relationship in LDP for an H-VPLS, they exchange one or more Address Messages with the VPLS Node TLV. The spoke node sends information about a single H- VPLS node (itself). The IP address SHOULD be the address by which the node is known to its peers. The MAC address SHOULD be a MAC address of the service node such that an un-encapsulated frame received from an H-VPLS peer with this destination MAC address will be delivered to the node's control plane. When two core nodes establish a mesh relationship, they also exchange one or more Address Messages with the above information via the targeted LDP session(s) between them. In this case each core node sends the MAC address information for all of the spokes to which it is connected, as well as for itself. Returning to the case of a core node establishing a peer connection with a spoke node, the core node sends to the spoke the entire MAC addresses that it has learned from all of its core peers as well as for itself. In this manner, each spoke node will know the MAC addresses of all of the other spokes in the H-VPLS. Note that in large networks multiple Address Messages may be required to ensure that media MTU sizes limitations are not exceeded. When a core node loses the PW connection to a spoke, it instructs all of its core peers to remove the corresponding spoke MAC address by sending the above information in an Address Withdrawal Message [RFC3036]. The core nodes in turn pass that information along in LDP Address Withdrawal Messages to their spokes nodes. When a core node loses the PW connection to another core node, it instructs all of its spoke nodes to remove all of the corresponding MAC addresses (received from the remote core node earlier) by sending the above information in an LDP Address Withdrawal Message. At the end of these procedures each service aware node will have complete information of all other service aware nodes in that VPLS instance. Throughout the document we will term the database built by LDP Address Messages as VPLS Node Database. From the VPLS Node Database a service aware node can lookup any remote service node information and can exchange VPLS OAM packets to test the data plane operation between itself and the remote node. Stokes, at al. Expires May 5, 2008 [Page 28] Internet-Draft OAM for VPLS November 2007 When a spoke node is dual-homed to two core PE nodes, the spoke node sends its node information to both the core PE nodes. Both the core PE nodes further propagate the spoke node information to all remote nodes. As a result of this, a core node in the H-VPLS may receive the spoke node information from at least two different peers. It is to note that the VPLS Node database MUST NOT be used for forwarding of customer or VPLS OAM frames. Apart from VPLS OAM there may be other applications that may make use of the VPLS node database and is outside the scope of this document. For example the VPLS Node database in a VPLS MEP can be used as input to the CCM database in the bridge MEP. 8.1.3 Operation of limited H-VPLS nodes There may be some implementations that do not want to maintain a list of IP and MAC addresses of all VPLS aware remote service points. In such an environment, an operator can still invoke a VPLS ping or VPLS traceroute if the MAC address of the remote spoke or customer device is provided. There may be some cases where an operator knows the IP address of a remote spoke, but not the MAC address. A method to obtain the MAC address when only the IP address is known may be described in a future version. If proactive topology discovery mechanism is not used then VPLS Traceroute can be used for the same purpose on an adhoc basis. For example, Traceroute sent for an unknown destination MAC address is flooded across the VPLS forwarders in the H-VPLS and each VPLS OAM entity in the H-VPLS would reply. However VPLS Trace route operations reveals only the data plane view of the topology. See section 8.5 for details of VPLS Traceroute operations. 8.2 VPLS Service Verification The secondary goal of the methodology used in topology discovery is VPLS service verification. Section 8.1 describes the method for distributing a VPLS node or OAM entity information using the control plane protocol. Such control plane view of the VPLS topology may be used as reference by data plane verification techniques. The nodes that have enabled periodic keeplives can be checked for connectivity faults by a receiver of the node information. The definitions and procedures for such keepalives are outside the scope of this document. For example, CCM [L2VPN-CFM] between the bridge MEPs MAY be used as the keepalives for monitoring the connectivity Stokes, at al. Expires May 5, 2008 [Page 29] Internet-Draft OAM for VPLS November 2007 between the VPLS MEPs. Typically a VPLS MEP may set the K-Bit in its node information if the corresponding bridge MEP has enabled CCM. A receiving VPLS MEP MAY check for CCMs from the sender MEP. Further, when a CCM message is received from a remote bridge MEP the VPLS Node Database is referred to check whether the remote bridge MEP is a member in the H-VPLS. A VPLS OAM entity that sends a VPLS OAM message is be cross verified with the VPLS Node Database to detect service misconfiguration, wrong cross connects etc. Service verification may also be performed on an on-demand basis using VPLS Traceroute. When a VPLS traceroute is sent for an unknown destination MAC address, each VPLS OAM entity replies. Sender of each reply can be verified against the VPLS Node database whether it is member of that service or not. 8.3 VPLS Connectivity Check In an H-VPLS connectivity fault is detected when two CE devices fails to communicate with each other across the H-VPLS domain. Such fault may occur due to connectivity failure between two or more VPLS forwarders located at the edge devices (or spoke nodes) in the HVPLS. This version of the document does not define any mechanism for proactive connectivity fault detection. CCM[802.1ag] can be used for connectivity checks in an H-VPLS on a proactive basis. CCM frames are exchanged periodically among the bridge MEPs in the VPLS edge devices[L2VPN-CFM]. If a bridge MEP does not receive CCM from a remote bridge MEP in the expected interval then a connectivity fault is detected between with the remote bridge MEP and is reported to the network operator. The bridge MEP may also generate fault notification to the VPLS MEP to trigger fault verification and isolation among the corresponding VPLS MEPs. 8.4 VPLS Ping VPLS Ping function is used to verify that data packets can be exchanged between any two VPLS forwarders in the H-VPLS. A VPLS Ping Message is issued by a source VPLS OAM entity to a target VPLS OAM entity for fault verification when a fault is detected between the corresponding VPLS forwarders. The appropriate target forwarder in front of fault will respond with a VPLS Ping Reply. The target forwarder behind the fault will not respond. VPLS Ping is a reactive or on-demand utility that may to be used by the network operator when fault is reported by the connectivity fault detection mechanisms. A VPLS Ping Message is also used to verify connectivity of customer MAC addresses learned in the H-VPLS. The VPLS Ping Request traverses Stokes, at al. Expires May 5, 2008 [Page 30] Internet-Draft OAM for VPLS November 2007 all the downstream VPLS forwarders from which the customer MAC address is learned. Note that a data frame with destination MAC as the customer MAC address follows the same set of VPLS forwarders. The VPLS OAM Entity in the egress node for the customer MAC address will respond with a VPLS Ping Reply. If the target customer MAC address is not known by an intermediate VPLS forwarder that receives the VPLS Ping Message, then the message is flooded to all downstream forwarders. In that case the VPLS Ping message would reach each leaf/spoke node (VPLS OAM MEP) in the H-VPLS. Each spoke node (MEP) will reply to the VPLS Ping Message. 8.4.1 VPLS Ping Message Format VPLS Ping is sent as the MPLS Echo Request message format [RFC4379]. It encodes the Hierarchical L2 Circuit FEC Stack Sub-type identifying the VPLS instance for which the Ping function is initiated. The Message is sent with Reply Mode = 4 (Reply via application level control channel) in the MPLS Echo Request header. The VPLS OAM entity initiating the VPLS Ping encapsulates the VPLS IP UDP MPLS Echo Request with the L2VPN-VCCV CW and with the same label stack as normal customer data, and then sends the packet to downstream forwarder. The message is sent as unicast with source MAC address of the sender VPLS OAM entity and destination MAC address as that of target VPLS OAM entity or a target customer MAC address. The L2- specific TLV SHOULD be filled as per the source MAC address and destination MAC Address. VPLS Ping Message MAY include a PAD TLV [RFC4379]. PAD TLV contains variable (>=1) number of octets. The first octet takes value 1 (Drop PAD TLV from reply) or 2 (Copy PAD TLV to reply) from the table as defined in [RFC4379]; all other octets (if any) are ignored. The receiver should verify that PAD TLV is received in its entirety, but otherwise ignores the content of this TLV apart from the first octet. PAD TLV with variable sizes may be used to test VPLS path MTU between the test nodes. It may also be used for various test data patterns for diagnosis. The initiator may send a test pattern in the PAD TLV and checks it for sanity/bit errors with the pattern received in echo reply. Further switching delay at every VPLS forwarder may vary based of the data frame size. So PAD TLV may be also used for one way and two way delay measurements for various frame sizes. 8.4.2 VPLS Ping Procedures VPLS Ping mechanism requires the sender VPLS OAM entity to know the MAC address of the destination VPLS OAM entity. To initiate a VPLS Ping, a packet is created with the format as described in Section 8.4.1. The destination Ethernet MAC address is of the remote VPLS OAM Stokes, at al. Expires May 5, 2008 [Page 31] Internet-Draft OAM for VPLS November 2007 entity or of a customer device. The source Ethernet MAC address is of the MAC belonging to the sender. The L2VPN TTL in L2VPN-VCCV CW SHOULD be set to 255. A VPLS Ping packet is processed by a VPLS forwarder as described in section 7.5. As a VPLS Ping packet exits a PW at the next VPLS service node, the corresponding VPLS forwarder checks if L2VPN TTL is 1 and if found to be true then the packet is passed to control plane. If L2VPN TTL > 1, the forwarder checks if the destination MAC address is a MAC belonging to this node; it also checks its forwarding database associated with this VPLS to see if the destination MAC is associated with a locally attached customer network. If the MAC is found to be local, then this node is the egress node for this VPLS Ping. The VPLS Ping packet is passed to the node's control plane. If the destination MAC address is not local then the VPLS Ping packet is further encapsulated with PW label received from downstream service node from where the MAC is learned and sends to the downstream node. If the destination MAC address is neither local nor is learned from a downstream forwarder then the VPLS Ping packet is flooded to all downstream VPLS forwarders. If the VPLS Ping packet is sent to control plane, a VPLS Ping reply is created in the same way MPLS Echo reply is created [RFC4379]. The reply is sent based on the value indicated in the Reply mode field in VPLS Ping Message. For security purposes, before replying to a VPLS Ping, a receiver VPLS OAM entity SHOULD check the sender information to ensure that the sender is known to be in the specified H-VPLS. Such information MAY be obtained from the VPLS Node Database. The replying node SHOULD check the PW Label over which the VPLS Ping packet is received against the label mapping corresponding to the L2VPN ID in the Hierarchical L2 Circuit. This check is necessary for detecting potential misconfiguration or inconsistency between the control plane and data plane. If Label mapping is incorrect then VPLS Ping Reply SHOULD be sent with appropriate Error Code described in Section 7.7. Typically VPLS Ping request is triggered to first check connectivity against the dataplane. If the request packet makes it to the destination VPLS OAM entity, the reply packet is sent along the data plane. Otherwise, if a reply is not received within the desired interval, the sender sends another request packet along the data plane, requesting a reply back on the control plane. If this also fails, a final attempt may be made, with request sent along the control plane, and the reply back along the control plane. If this fails, then the network is probably partitioned. Stokes, at al. Expires May 5, 2008 [Page 32] Internet-Draft OAM for VPLS November 2007 8.5 VPLS Traceroute When a VPLS Ping operation is unsuccessful, additional capabilities are required to isolate and identify the problem location to a specific VPLS forwarder. The problem may be at the intermediate VPLS forwarders or may be at the VPLS forwarders in the edge devices. VPLS Traceroute is used to localize and isolate such connectivity faults. In normal condition (no fault) VPLS Traceroute can be used for tracing the VPLS forwarders to reach a remote VPLS forwarder or a customer MAC address. Each VPLS OAM entity in the forwarding path responds with a reply. 8.5.1 VPLS Traceroute Message Format VPLS Traceroute uses the same MPLS echo request packet format as used for VPLS Ping. The VPLS Traceroute Message SHOULD contain one or more Downstream Mapping TLVs described in [RFC4379]. In the Downstream Mapping TLV, the Downstream Router ID field contains the IPv4 address of the next H-VPLS node that would be used to reach the Target Ethernet MAC address in the VPLS indicated in the Hierarchical L2 Circuit FEC Stack Sub-Type. The Downstream Label field contains the label stack used to reach the downstream peer. This includes the label(s) for the underlying tunnel LSP (if PSN Tunnels are RSVP-TE Tunnels) and the PW label from the targeted LDP session with the downstream peer. The Protocol field for the PW label indicates a value of 3 for (targeted) LDP. When the indicated Target Ethernet MAC is not known and a packet with this destination MAC would be flooded, the information for all HVPLS peers to which the packet would be flooded is added. For the case where the packet cannot be flooded (such as a limit on MAC addresses that has been exceeded for this HVPLS), a new LSP ping Return code is defined. This new Value 7 is shown in Section 8.1.6. For the case where the destination MAC address is known, but the packet would not be forwarded, no Downstream Mapping TLV is included. For example, the packet may have been received at one core node from another core node, but the MAC address was previously learned from a different core node. The Downstream Mapping TLV includes a Multipath Type to address ECMP considerations. The Multipath types defined deal with IP addresses and MPLS labels. In an H-VPLS, it is possible that a hash may be performed on the source and/or destination MAC addresses of the Stokes, at al. Expires May 5, 2008 [Page 33] Internet-Draft OAM for VPLS November 2007 encapsulated L2 frame. See Section 10 for a discussion of load sharing or ECMP with a MAC-based hash. For the case where a hash is performed on MAC address (es) including the destination MAC address, new MAC-based Multipath types are proposed. The resulting list of Multipath types is shown below. Multipath Type IP Address or Next Label -------------------- ------------------------ 0 no multipath (nothing; M = 0) 2 IP address IP addresses 4 IP address range low/high address pairs 8 Bit-masked IP IP address prefix and bit mask address set 9 Bit-masked label set Label prefix and bit mask 10 MAC address MAC addresses 11 MAC address range low/high MAC pairs This results in the Downstream Mapping format shown below. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MTU | Address Type | DS Flags | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Downstream IP Address (4 or 16 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Downstream Interface Address (4 or 16 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Multipath Type| Depth Limit | Multipath Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IP Address or Next label or MAC Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ . . (more IP Addresses or Next labels or MAC Addresses) . +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Downstream Label | Protocol | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ . . . +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Downstream Label | Protocol | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 14. Downstream Mapping TLV Value Stokes, at al. Expires May 5, 2008 [Page 34] Internet-Draft OAM for VPLS November 2007 In the case of a MAC address, the following format is used for the "IP Address or Next label or MAC Address" field(s): +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Destination Ethernet MAC address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | 802.1Q Tag | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 15. The semantics of the new Multipath Types and the associated MAC Address(es) are shown below. Value Meaning ----- ------- 10 a list of single MAC addresses is provided, any one of which will cause the hash to match this MP path. 11 a list of MAC address ranges is provided, any one of which will cause the hash to match this MP path. See Section 10 for additional ECMP considerations. 8.5.2 VPLS Traceroute Procedures To initiate VPLS Traceroute, an MPLS echo request is created using the destination Ethernet MAC address of the remote service node or customer device. The L2VPN TTL in the L2VPN-VCCV CW is set successively to 1, 2, 3 ...etc. As with MPLS traceroute [RFC4379], the echo request contains one or more Downstream Mapping TLVs. For L2VPN TTL=1, all the peer nodes (and corresponding PW labels) for the sender with respect to the Target MAC address being traced SHOULD be sent in the echo request. As the echo request may be flooded to multiple nodes, the sending node may receive replies from multiple remote nodes. Thus, for n>1, the Downstream Mapping TLVs from all of the received echo replies for L2VPN TTL=(n-1) are copied to the echo request with L2VPN TTL=n. Note that this allows an operator to determine which remote nodes would receive a flood of an actual customer data packet destined to the target MAC address. The VPLS Traceroute packet is processed by a VPLS forwarder as described in section 7.5. As a VPLS OAM packet exits a PW at the next VPLS service node, the corresponding VPLS forwarder first checks if Stokes, at al. Expires May 5, 2008 [Page 35] Internet-Draft OAM for VPLS November 2007 the L2VPN TTL is 1 and if it is true then the packet is passed to the control plane. If the L2VPN TTL > 1, the forwarder checks if the destination MAC address is a MAC belonging to this node. It also checks its forwarding database associated with this VPLS to see if the target MAC is associated with a locally attached customer network. If the MAC is found to be local, then this node is the egress node for this VPLS traceroute. The packet is be passed to the node's control plane. In the control plane, an echo reply is created as described in [RFC4379]. This includes completing the Downstream Mapping TLV. The reply is sent based on the value indicated in the Reply Mode field of the echo request. For security purposes, before replying to a VPLS Traceroute, a node SHOULD check the sender information to ensure that the sender is known to be in the specified H-VPLS. See Section 12 for further discussion. If the destination MAC address is not local, and the value of the L2VPN TTL is greater than 1, the L2VPN TTL is decremented and the result is placed in the CW for the resulting packet. This packet is encapsulated in the same manner as a customer data packet and then passed to the downstream node that would normally be used to reach the indicated Ethernet MAC. If the packet is flooded to multiple peer nodes, this same L2VPN TTL value is placed for each of the outgoing packets. 9. General VPLS Testing Procedures Testing H-VPLS nodes and peer sessions requires minimal network impact since this is done on a per machine and a per VPLS service node basis. Also, monitoring for these failures can be done with SNMP traps. It is important to note that the VPLS OAM functions are expected to catch a significant percentage of the most likely failure cases. If the PSN is MPLS based, then tunnel LSPs are maintained by their associated MPLS signaling protocols, RSVP-TE [RFC3209] or LDP [RFC3036]. Locally detected failures such as link outages or node failures invoke MPLS recovery actions. These actions can include local recovery as in the case of RSVP-FRR [RFC4090]. In the case of a successful local recovery of a tunnel LSP, the H-VPLS nodes may be notified. FRR recovery is one of the ways that an existing service can become corrupted [L2VPN-CFM]. It would be desirable to do a service verification following such a tunnel recovery. This could be done using a VPLS Traceroute to a non-existent MAC address. This VPLS Stokes, at al. Expires May 5, 2008 [Page 36] Internet-Draft OAM for VPLS November 2007 Traceroute could be initiated by an operator or automatically at the ingress of the FRR LSP based on some indication that a recovery action has occurred. If a local recovery is not possible, then RSVP- TE or LDP notifies the H-VPLS node using the tunnel LSP of the failure. This in turn can generate a SNMP trap or an operator error message. In addition to failures and changes detected outside of MPLS, both MPLS signaling protocols have control plane failure detection mechanisms. Both have Hello protocols that can detect link failures as well as MPLS control plane failures. LDP also has a Keep Alive protocol. These Hello and Keep Alive protocols run on the time frame of multiple seconds. They also provide failure notification to the H- VPLS node using the tunnel. As above, this can generate a SNMP trap or an operator error message. In current [RFC4671] or [RFC4672] environments, loss of a targeted LDP session to a peer is normally a key operator notification. While a failed tunnel LSP can generate a notification as described above, these failures can be temporary in nature due to routing transients. LSP ping[RFC4379] is designed to catch failures where the control plane believes that a tunnel LSP is operational but it is in fact not functioning correctly. This corrupted LSP is much less likely to occur than a LSP going down "properly." When used as a background check, it should be used in addition to the above tunnel LSP failure detection methods and not as a replacement. When any of these methods detects a tunnel LSP failure, the H-VPLS node can switch to another LSP if one is available. When the failure is detected by MPLS ping, MPLS traceroute can be used to assist in failure isolation. VPLS OAM is designed to detect problems in the transport aspects of the VPLS operation. It detects flooding and/or MAC learning problems in the network which are not checked in the above tests. Note that the number of possible spoke-to-spoke tests to check an entire H-VPLS can be significant. Therefore, care should be taken when executing VPLS OAM functions as a background test to avoid overloading the network or the H-VPLS nodes. Note also that an individual core node's operation is checked by multiple spoke-to-spoke checks. All of the above tests check the operation of an H-VPLS among the MEPs. It is also possible to use VPLS Ping and Traceroute to check for customer device MAC addresses. While not specified by H-VPLS, there is normally additional information available to an operator to Stokes, at al. Expires May 5, 2008 [Page 37] Internet-Draft OAM for VPLS November 2007 check for problems between the edge of the HVPLS and the customer. These include: - the state of the local customer VLAN or port - this is the simplest test and will normally catch the most likely failures - the L2 MAC entries for the local customer VLAN or port - the H-VPLS transmit/receive statistics As described above, VPLS ping and VPLS traceroute work with previously defined MPLS tests to provide an end-to-end test capability for an H-VPLS. 10. Load sharing considerations Some implementations provide for load sharing a pseudowire across multiple MPLS PSN Tunnels. Note that load sharing is not applicable to other tunneling mechanisms such as GRE etc. Such load shared based implementations have HVPLS test implications and are an important aspect of the methodology used in this document. When customer data entering a VPLS at an ingress node is transmitted to another node over multiple (load sharing) tunnel LSPs, each of these LSPs SHOULD be tested. VPLS Pings and VPLS Traceroutes SHOULD be sent over each of these LSPs. There may also be multiple load sharing tunnel LSPs between a core node which is not the traffic ingress and a downstream node (which may or may not be the traffic egress). At such a core node, a decision is made as to which load sharing tunnel LSP to use to forward a VPLS packet. This decision is often based on a hash of some "random" field. There are at least three options. One option is to hash on the IP addresses of an encapsulated IP packet. This option would potentially need to be combined with another option to handle non-IP frames. A second option is to hash on the label stack of the received VPLS packet. This forces all packets received on a tunnel LSP for the same VPLS to use the same load sharing tunnel LSP to the next core node. This method distributes traffic among the load sharing tunnel LSPs on a VPLS basis. A third option is to hash on fields of the VPLS packet after it has been un-encapsulated. Such a hash could use the destination and source MAC addresses of the un-encapsulated packet. Thus, traffic received on a tunnel LSP for the same VPLS may use any of the load Stokes, at al. Expires May 5, 2008 [Page 38] Internet-Draft OAM for VPLS November 2007 sharing tunnel LSPs to the next core node. This method distributes traffic among the load sharing tunnel LSPs on a MAC address pair basis. The first and third options normally produce a more optimal distribution of packets since IP addresses and MAC addresses should be more random than VPLS labels. This advantage may be somewhat reduced for the first option if customers' data contains a significant amount of non-IP traffic. It may also be somewhat reduced for the third option if customers use a single router at each site to connect to the HVPLS. The second option has an advantage from an VPLS testing perspective. Since the label stack for a VPLS Ping or VPLS Traceroute is the same as for customer traffic, the second option ensures that VPLS pings and traceroutes are testing all of the downstream LSPs used by customer data. The first option has the disadvantage that a hash on the IP address of an encapsulated ping/traceroute packet uses an address in the 127/8 range and not a true customer IP address. The third option has the disadvantage that a hash on the MAC address of a spoke node may differ from the hash on a true customer MAC address. However, remember that actual customer MAC addresses can be used in a VPLS ping/traceroute and these will use the same path as the customer data when using a MAC-based hash. Remember from above that VPLS pings and traceroutes SHOULD be sent using all of the load sharing tunnel LSPs at the ingress node. Load sharing designs and hash algorithms remain implementation options. There are trade-offs between optimal load sharing and testability. Of course, testing using IP ping and traceroute has similar exposures from the effects of equal cost multipath. The methodology described thus far provides a means to verify that all remote nodes can be reached. It also provides an operator with a means to verify operation for particular customer MAC addresses. It does not provide a means to verify all load sharing paths in a HVPLS from a single node. The Multipath information contained in the Downstream Mapping TLV in [RFC4379] provides additional capabilities for verifying all load sharing paths. Use of this information in a VPLS traceroute environment, to test all load sharing paths in an HVPLS, will be discussed in a future version. Stokes, at al. Expires May 5, 2008 [Page 39] Internet-Draft OAM for VPLS November 2007 11. Scalability Considerations Mechanisms developed for VPLS OAM need to be such that per-service OAM can be supported even though the OAM may only be used for limited VPLS service instances, e.g. premium VPLS service instances, and may not be used for best-effort VPLS services. VPLS OAM should be scalable such that a service aware device can support OAM for each VPLS service that is supported by the device. 12. Security Considerations For security purposes, the edge of a provider's HVPLS network is defined to be a spoke node or a PE that has directly attached customers. Some customers and providers may desire that the provider edge node participates in the customer network. This could be the case when the customer is using the provider's node as a default gateway. In such a configuration, the provider edge node's IP address and Ethernet MAC address are known in the customer network. This would be the case as shown in figure 3 where the bridge module in a VPLS capable device is provisioned for MIP at customer MD level. However, no other provider network information should be exposed to the customer network, esp. VPLS transport related information. When the provider is not furnishing a default gateway function, no provider network information SHOULD be exposed to the customer network. The VPLS OAM messages are transported inside the H-VPLS in the same manner as customer data. This is required to properly test the HVPLS. However, care must be taken to prevent provider network information contained in these test packets from being exposed to the customer network. A test packet that is forwarded to the customer network exposes provider network information to the customer network. Therefore, spoke nodes SHOULD always check for such test packets. Any detected test packet SHOULD NOT be forwarded to the customer network. Another security concern is the receipt of a VPLS ping or traceroute from a node that is not a member of the HVPLS. Should a HVPLS node respond to a test request from a non-HVPLS member, the response would improperly expose provider network information. To prevent this from happening, the HVPLS node MAY check to ensure that the return Ethernet MAC address is one of the MAC addresses that it has learned using the Ethernet VPLS node TLV described in Section 8.1. Note that this requires maintaining the MAC information of VPLS OAM entities during the entire operation of the H-VPLS. Stokes, at al. Expires May 5, 2008 [Page 40] Internet-Draft OAM for VPLS November 2007 13. IANA Considerations This document requires allocation of a new Target FEC Stack Sub-TLV type[RFC4379] that is managed by IANA. Hierarchical L2 Circuit FEC stack is defined in this document and the requested type is 17. A new TLV is defined for [RFC4379] as Error Code TLV to convey results of a VPLS OAM operation. The requested type is 4. Two new Multipath Types are defined to be used with Downstream Mapping TLV[RFC4379] for MAC based hash keys. The types requested are as follows: 10 a list of single MAC addresses is provided, any one of which will cause the hash to match this MP path. 11 a list of MAC address ranges is provided, any one of which will cause the hash to match this MP path A LDP TLV type is defined in this document as VPLS Node TLV and the requested type is 0x0405. A control channel type is defined as L2VPN-VCCV to be used with [VCCV] procedures. The bitmask of requested control channel type is 0x10. 14. Acknowledgments The authors would like to thank the following people who have provided valuable comments and feedback on the topics discussed in this document: Mustapha Aissaoui Florin Balus Matthew Bocci Som Barua Andrew Dolganow Tiberiu Grigoriu Kireeti Kompella Marc Lasserre Ananth Nagarajan Ray Qiu This document was prepared using 2-Word-v2.0.template.dot. Stokes, at al. Expires May 5, 2008 [Page 41] Internet-Draft OAM for VPLS November 2007 References Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC4761] Kompella, K., Rekhter, Y.," Virtual Private LAN Service (VPLS) Using BGP for Auto-Discovery and Signaling", RFC 4761, January 2007. [RFC4762] Lasserre, M., Kompella, V.,"Virtual Private LAN Service (VPLS) Using Label Distribution Protocol(LDP) Signaling", RFC 4762, January 2007. [RFC4667] Luo, W.,"Layer 2 Virtual Private Network (L2VPN) Extensions for Layer 2 Tunneling Protocol(L2TP)", RFC 4667, [RFC4379] Kompella, K. et al.,"Detecting Multi-Protocol Label Switched (MPLS) Data Plane Failures", RFC 4379, February 2006. [VCCV] Nadeau, T. et al., "Pseudo Wire Virtual Circuit Connectivity Verification(VCCV)", draft-ietf-pwe3-vccv-13.txt (Work in progress). [RFC4447] Martini, L. et al.,"Pseudowire Setup and Maintenance Using the Label Distribution Protocol (LDP)", RFC 4447, April 2006. [RFC4448] Martini, L.,"Encapsulation Methods for Transport of Ethernet Frames Over IP/MPLS Networks", RFC 4448, April 2006. [RFC4446] Martini, L.,"IANA Allocations for pseudo Wire Edge to Edge Emulation(PWE3)", RFC 4446, April 2006. [RFC3036] Andersson, L. et al., "LDP Specification", RFC 3036, January 2001 Stokes, at al. Expires May 5, 2008 [Page 42] Internet-Draft OAM for VPLS November 2007 Informative References [L2VPN-CFM] Stokes, O. et al., "OAM for L2 VPN Networks Using CFM and VCCV", draft-ietf-stokes-l2vpn-cfm-vccv-oam-00-0.txt (Work in progress). [L2VPN-OAM-REQ] Mohan, D. et al., "L2VPN OAM Requirements and Framework", draft-ietf-l2vpn-oam-req-frmk-08.txt (Work in progress). [MEF-OAM-REQ] MEF Service OAM Requirements & Framework, Technical Specification, Draft version 1.1. [802.1ag] Connectivity Fault Management, IEEE 802.1ag draft 6.1, work in progress. [ROSEN-MESH] Rosen, E.,"Detecting and Reacting to Failures of the Full Mesh in IPLS and VPLS", draft-rosen-l2vpn-mesh-failure-02.txt [Y.1731] ITU-T Recommendation Y.1731, OAM Functions and Mechanisms for Ethernet based Networks, May 2006. [RFC3209] Awduche, D. et al., "RSVP-TE: Extensions to RSVP for LSP Tunnels", RFC3209, December 2001. [RFC4090] Pan, P. et al.,"Fast Reroute Extensions to RSVP-TE for LSP Tunnels", RFC4090, May 2005. [SEG-PW] Martini, L. et. al.,"Segmented Pseudo Wire", draft-ietf-pwe3-segmented-pw-05.txt, work in progress. Stokes, at al. Expires May 5, 2008 [Page 43] Internet-Draft OAM for VPLS November 2007 Author's Addresses Olen Stokes Extreme Networks PO Box 14129 RTP, NC 27709 USA Email: ostokes@extremenetworks.com Vach Kompella Alcatel-Lucent 701 E Middlefield Road, Mountain View, CA 94043 USA E-mail : vach.kompella@alcatel-lucent.com Pranjal Kumar Dutta Alcatel-Lucent 701 E Middlefield Road, Mountain View, CA 94043 USA E-mail : pdutta@alcatel-lucent.com Giles Heron Tellabs 24-28 Easton Street High Wycombe Bucks HP11 INT UK E-mail : giles.heron@tellabs.com Yetik Serbest AT&T Labs 9505 Arboretum Blvd. Austin, TX 78759 USA E-mail : yetik_serbest@labs.att.com Shane Amante Level 3 Communications, LLC 1025 Eldorado Blvd Broomfield, CO 80021 USA Email: shane@level3.net Stokes, at al. Expires May 5, 2008 [Page 44] Internet-Draft OAM for VPLS November 2007 Intellectual Property Statement The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Disclaimer of Validity This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Copyright Statement Copyright (C) The IETF Trust (2007). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. Acknowledgment Funding for the RFC Editor function is currently provided by the Internet Society. Stokes, at al. Expires May 5, 2008 [Page 45]