L2VPN WG Olen Stokes Internet Draft Extreme Networks Intended Status : Standard Vach Kompella Pranjal Kumar Dutta Alcatel-Lucent Giles Heron Tellabs Yetik Serbest AT&T Shane Amante Level 3 Communications Expires: May 2008 November 18, 2007 OAM (Operation, Administration and Maintenance) for Virtual Private LAN Service (VPLS) draft-stokes-vkompella-l2vpn-vpls-oam-01.txt Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. This document may only be posted in an Internet-Draft. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html This Internet-Draft will expire on May 18, . Stokes, et. al. Expires May 2008 [Page 1] Internet-Draft OAM for VPLS November 2007 Copyright Notice Copyright (C) The IETF Trust (2007). Abstract This document describes a methodology and a set of mechanisms for Operation, Administration and Maintenance (OAM) of a general VPN (Virtual Private Network) service, that is applied here to a Virtual Private LAN Service [RFC4761][RFC4762]. [L2VPN-CFM] describes the capabilities of CFM [802.1ag] and VCCV [VCCV] for OAM operations among the bridge module components in the VPLS. The OAM functions described in this document builds the additional capabilities into VPLS OAM to detect, verify and isolate connectivity faults among the VPLS forwarder components. The methodology adopted extends LSP Ping concepts described in [RFC4379] and VCCV capabilities described in [VCCV]. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119. 1. Table of Contents 1. Table of Contents.................................. 2 2. Terminology ........................................3 3. Introduction........................................3 4. CFM and VCCV capabilities...........................6 5. VPLS OAM Function Placement.........................8 5.1 Discovery........................................10 5.2 Service Verification.............................10 5.3 Connectivity Fault Management....................10 5.3.1 Connectivity Fault Detection.................10 5.3.2 Connectivity Fault Verification..............11 5.3.3 Connectivity Fault Localization..............11 5.4 Partial Mesh Resolution..........................11 5.5 MAC Populate and MAC Purge.......................12 5.6 Performance Monitoring.......................... 12 6. Architectural Considerations........................13 6.1 MEP and MIP Identifiers..........................13 6.2 FIB Traversal....................................14 6.3 Control plane and Dataplane Consistency..........14 7. Packet Formats, Encapsulations and Extensions.......14 7.1 Hierarchical L2 Circuit FEC Stack Sub-type.......15 7.2 L2 VPN ID Sub-TLV................................16 Stokes, at al. Expires May 18, 2008 [Page 2] Internet-Draft OAM for VPLS November 2007 7.3 L2 specific Sub-TLVs.............................17 7.4 VPLS OAM Encapsulation...........................19 7.4.1 PW Associated Channel Type for VPLS OAM......19 7.4.2 VCCV CV Type for VPLS OAM....................21 7.5 Dataplane processing for VPLS OAM................21 7.6 Egress Node Control Plane Processing.............23 7.7 Error Code TLV...................................23 7.8 Vendor-specific Extensions.......................25 8. VPLS OAM Functions..................................26 8.1 VPLS Topology Discovery..........................26 8.1.1 Ethernet VPLS Node TLV.......................27 8.1.2 VPLS Topology Discovery Procedures...........28 8.1.3 Operation of limited VPLS Nodes..............29 8.2 VPLS Service Verification........................30 8.3 VPLS Connectivity Check..........................30 8.4 VPLS Ping........................................31 8.4.1 VPLS Ping Packet Format......................31 8.4.2 VPLS Ping Procedures.........................32 8.5 VPLS Traceroute..................................33 8.5.1 VPLS Traceroute Packet Format................33 8.5.2 VPLS Traceroute Procedures...................36 9. General VPLS Testing Procedures.....................37 10. Load Sharing Considerations........................38 11. Scalability Considerations.........................40 12. Security Considerations............................40 13. IANA Considerations................................41 14. Acknowledgements...................................42 2. Terminology This document uses the terminology defined in [L2VPN-OAM-REQ] [RFC4379], [RFC4761], [RFC4762], [RFC4664], [802.1ag] and [VCCV]. Throughout this document VPLS means the emulated bridged LAN service offered to a customer. H-VPLS means the hierarchical connectivity or layout of MTU and PE devices offering the VPLS[4762]. The terms spoke node and MTU in H-VPLS are used interchangeably. 3. Introduction Virtual Private LAN Service (VPLS), also known as Transparent LAN Service (TLS) is a useful service provider offering and is described in [RFC4761][RFC4762]. A VPLS is created using a collection of one or more point-to-point pseudowires (PWs) [RFC3965] configured in a flat, full-mesh topology. Each PW provides a logical interconnect between two ?VPLS Forwarders? in the VPLS capable Provider Edge(PE) devices. The mesh topology of VPLS forwarders provides a LAN segment or broadcast domain that is fully capable of learning and forwarding on Stokes, at al. Expires May 18, 2008 [Page 3] Internet-Draft OAM for VPLS November 2007 Ethernet MAC addresses at the PE devices. This VPLS core configuration can be augmented with additional non- meshed spoke nodes to provide a Hierarchical VPLS (H-VPLS) service [RFC4762]. For the purposes of this document, the terms "H-VPLS" and "H-VPLS instance" are used to signify a generic VPLS instance that may or may not be hierarchical in topology. An Operation, Administration and Maintenance (OAM) provides the instrumentation of telecommunications networks and equips the network operator with the tools to monitor, detect and verify faults in their network infrastructure. Service providers deploying VPLS should be able to test operations of all its components across the entire hierarchy in an H-VPLS. [L2VPN-CFM] describes the capabilities of CFM[802.1ag] and VCCV[VCCV] for OAM among the "bridge" components in the VPLS capable PE devices. However CFM in combination with VCCV does not provide verification or problem determination for ?VPLS forwarders?[L2VPN-CFM]. This document describes a generic methodology for OAM among VPN forwarders and demonstrates its ability for OAM operations among the VPLS forwarder components in an H-VPLS. Application of this methodology for other VPN services is outside the scope of this document. The solutions described in this document build the capabilities required to address the VPLS transport specific requirements of VPLS OAM. +-----+ +-----+ + CE1 +--+ +---| CE2 | +-----+ | ................... | +-----+ VPLS A | +----+ +----+ | VPLS A | |VPLS| |VPLS| | +--| PE |--Routed---| PE |--+ +----+ Backbone +----+ / . | . \ _ /\_ +-----+ / . | . \ / \ / \ +-----+ + CE +--+ . | . +--\ Access \----| CE | +-----+ . +----+ . | Network | +-----+ VPLS B .....|VPLS|........ \ / VPLS B | PE | ^ ------- +----+ | | | | | +-----+ | | CE3 | +-- Emulated LAN +-----+ VPLS A Figure 1 Stokes, at al. Expires May 18, 2008 [Page 4] Internet-Draft OAM for VPLS November 2007 It is required to outline the generic architecture of VPLS and its components in order to understand the purpose and context of the VPLS OAM functions described in this document. The diagrams in Figure 1. and Figure 2. demonstrate the VPLS reference model as explained in [RFC4664]. PE devices that are VPLS-capable provide a logical interconnect such that Customer Edge (CE) devices belonging to a specific VPLS appear to be on a single bridged Ethernet. |-----Routed Backbone-----| | (P Routers) |PSN Tunnels, Emulated LAN | |Pseudowires ....................................................................... . | | . . |---------------------|----| |--------|-----------------| . . | --------------------|--- | | -------|---------------- | . . | VPLS Forwarder | | VPLS Forwarder | . . | ----------|------------- | | ----------|------------- | . ..|.................................................................|.. | | Emulated LAN | | | Emulated LAN | | | Interface | VPLS-PEs | | Interface | | | | <----> | | | | ----------|------------ | | ----------|------------ | | | Bridge | | | | Bridge | | | -|--------|---------|-- | | ---|-------|---------|- | |--|--------|---------|----| |----|-------|---------|---| | | | | | | | | Access | | | Access | | | Networks| | | Networks| | | | | | | | | | | | | CE devices CE devices Figure 2 From Figure 2, we see that in VPLS, a CE device attaches, possibly through an access network, to a "bridge" module of a VPLS PE. Within the VPLS PE, the bridge module attaches, through an "Emulated LAN Interface?, to an Emulated LAN. For each VPLS, there is an Emulated LAN instance. Figure 2 shows some internal structure to the Emulated LAN: it consists of "VPLS Forwarder" modules connected by PWs, where the PWs may be traveling through Packet Switched Network(PSN) tunnels over a routed backbone. These tunnels can be IP tunnels, such as Generic Routing Encapsulation (GRE), or MPLS tunnels, established by Resource Reservation Protocol - Traffic Engineering (RSVP-TE) or LDP. Stokes, at al. Expires May 18, 2008 [Page 5] Internet-Draft OAM for VPLS November 2007 A "VPLS instance" consists of a set of VPLS Forwarders (no more than one per PE) connected by the PWs in a full-mesh configuration. In H-VPLS configuration, PWs can also be used to connect the spoke nodes to the VPLS core nodes. Service providers may design a scalable H-VPLS topology by interconnecting multiple full-mesh domains with inter-domain spokes. [RFC4762] describes about multi-domain VPLS services. Note that the context of "inter-domain" here is NOT inter- provider or inter-AS, rather multiple inter-connected full-mesh domains of a single provider. In such network designs data frames between CE devices may traverse multiple VPLS forwarders spanning across the domains. This document defines a set of the OAM mechanisms among the VPLS forwarders in an H-VPLS instance. The PSN network-level OAM is outside the scope of the document. The methodology adopted in the mechanisms require extensions to the existing LSP Ping concepts described in [RFC4379] and VCCV defined in [VCCV]. The inter-provider or inter-AS VPLS OAM aspects would be addressed in a later revision. Note that in inter-domain VPLS the OAM domains are not nested domains, rather placed at the same level. 4. CFM [802.1ag] and VCCV [VCCV] capabilities Ethernet OAM, as being defined in [802.1ag] and [Y.1731], offers the set of functionalities that have been designed for proactive and on-demand OAM for the ethernet bridge layer. [802.1ag] provides the tools for the Connectivity Fault Management (CFM) and [Y.1731] adds capabilities for the performance monitoring, measurement aspects into Ethernet OAM. In its current form, CFM has addressed the "bridging" aspects of the ethernet service layer in VPLS. The OAM tools provided by CFM in combination with PW VCCV methods can be deployed in H-VPLS for troubleshooting and monitoring the operation of bridge modules in a VPLS service[L2VPN-CFM]. [L2VPN-CFM] describes about the ability of the combination of CFM and VCCV to meet OAM requirements of a L2VPN. While the combination of CFM and VCCV provides significant information for a network operator, the combination does not provide a comprehensive OAM solution for an H-VPLS environment. When applied to the [RFC4664] reference model of a VPLS PE node, CFM does not provide verification or problem determination of the emulated LAN, which is the network of VPLS forwarders. VCCV provides OAM coverage for the PWs alone in the emulated LAN, but does not address the association or mapping of PWs with the VPLS forwarders. OAM functions for H-VPLS are required to address the VPLS transport specific issues with adequate granularity to provide a comprehensive set of monitoring, troubleshooting Stokes, at al. Expires May 18, 2008 [Page 6] Internet-Draft OAM for VPLS November 2007 facility to the VPLS service provider. Some of the issues associated with VPLS transport are as below: - In H-VPLS, ethernet service infrastructure is provided by a set of ethernet PWs setup by control plane protocols such as LDP, BGP or L2TP. [RFC4761] describes VPLS with BGP as PW setup and control protocol. [RFC4762] describes VPLS with LDP as PW setup and control protocol. [RFC4667] defines the L2TP extensions for PW signaling in L2VPNs. Verification of control plane and data plane consistency is a significant and integral function of VPLS OAM. Those issues can occur if there are bugs in the encapsulation/decapsulation procedures at the PEs, or bugs in the forwarding procedures at intermediate nodes(especially where dataplane and control plane are decoupled). CFM is agnostic of the PW control plane and the PW transport that provides the emulated ethernet bridge service. In order to build control and data plane validation capabilities into [802.1ag], CFM protocol would be required to carry at least VPLS transport layer (PW) control information. - For troubleshooting and validation, it is necessary to know the PW behind which a specific MAC address is located. As CFM entities are not present in intermediate VPLS forwarders in an H- VPLS[L2VPN-CFM], it is not possible to obtain information with such granularity from CFM operations. - To determine the exact cause of the failure, it is desirable to have the ability in VPLS OAM entities to send replies to the originator of an OAM message via out of band channels. CFM messages (LBM, LTM etc) cannot report back problems to the sender in case of PW dataplane failure in the direction in which reply is to be sent. - When PSN is MPLS based, some implementations may provide for load sharing a PW across multiple MPLS PSN Tunnels. The OAM functions should be able convey the multipath information and test the operations over each of the tunnels. Such capabilities are not present in [802.1ag] as the architecture is agnostic to MPLS transport and out of its scope. A comprehensive VPLS OAM solution should also address such issues. - A VPLS provider should be able to perform offline verification of MAC learning and MAC forwarding capabilities among the VPLS forwarders without the customer data traffic. It is desirable to have tools that enable the operator to populate and purge MAC addresses in a VPLS instance. Such capabilities are not present in CFM tools. Stokes, at al. Expires May 18, 2008 [Page 7] Internet-Draft OAM for VPLS November 2007 Combination of CFM and VCCV leaves a significant portion of the emulated LAN completely opaque from a standardized OAM perspective [L2VPN-CFM]. In summary, a comprehensive VPLS OAM solution is required to address both bridging and transport aspects of the VPLS. The solutions described in [L2VPN-CFM] and the solutions proposed in this document may co-exist in a VPLS service provider network. Moreover, it is preferable to retain a layered approach to VPLS OAM, with CFM to be used for Ethernet Service OAM and solutions in this document to address VPLS forwarder and transport specific issues. 5. VPLS OAM Function Placement Figure 3. shows the placement of VPLS OAM functions described in this document inline with the positioning of CFM and PW VCCV components described in [L2VPN-CFM]. A VPLS OAM domain comprises of the set of VPLS forwarders located in the service aware nodes in an H-VPLS instance. A Maintenance Association Endpoint (MEP) is created for the VPLS forwarder in any H-VPLS node with a customer attachment circuit (AC). A MEP or a Maintenance Domain Intermediate Point(MIP) is created for a VPLS forwarder in any H-VPLS node without a local AC. Since MEPs are located at the edge of the H-VPLS, they are responsible for filtering outbound OAM frames from leaving the VPLS OAM domain. The term "VPLS OAM Entity" is used throughout the document to refer to a VPLS MEP or MIP corresponding to a VPLS forwarder. CFM packets generated by bridge MEPs or forwarded by bridge MIPs are not processed by VPLS OAM Entities; those are forwarded by the VPLS forwarder as data packets based on their destination MAC addresses. VPLS OAM Packets are originated and exchanged only among the VPLS OAM entities, and are never forwarded to the bridge modules. Bridge MEPs at VPLS edge devices may participate as MIPs at customer MD level 5 and would respond to LTM[802.1ag] messages received from customer domain. The VPLS OAM functions should expose the VPLS transport issues in adequate depth but should not expose such details to the customer OAM. It is also important to have such layered approach for security reasons. Stokes, at al. Expires May 18, 2008 [Page 8] Internet-Draft OAM for VPLS November 2007 |--- PW-1 ---| |- PW-13 -| |--- PW-3 ---| Emulated | | | | | | LAN | | | | | | ..................................................................... . | | | | | | . . |-------|-----| |---|-----|---| |---|-----|---| |-----|-------| . . | ----------- | | ----------- | | ----------- | | ----------- | . . | VPLS | | VPLS | | VPLS | | VPLS | . . | Forwarder | | Forwarder | | Forwarder | | Forwarder | . . | VPLS OAM MEP| | VPLS OAM MIP| | VPLS OAM MIP| | VPLS OAM MEP| . . | -----|----- | | -----|----- | | ----------- | | -----|----- | . ..|.................................................|................ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | -----|----- | | -----|----- | | -----|----- | | -----|----- | | |MEP X MD | | | |MEP X MD | | | |MEP X MD | | | |MEP X MD | | | | | LVL| | | | | Lvl| | | | | Lvl| | | | | LVL| | | | | 3 | | | | | 3 | | | | | 3 | | | | | 3 | | | | | | | | | | | | | | | | | | | |MIP X MD | | | |MIP X MD | | | |MIP X MD | | | |MIP X MD | | | | | LVL| | | | | LVL| | | | | LVL| | | | | LVL| | | | | 4 | | | | | 4 | | | | | 4 | | | | | 4 | | | | Bridge | | | | Bridge | | | | Bridge | | | | Bridge | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |MEP X MD | | | | | | | | | | | |MEP X MD | | | | | LVL| | | | | | | | | | | | | LVL| | | | | 4 | | | | | | | | | | | | | 4 | | | | | | | | |---------| | | |---------| | | | | | | | | | | | |-------------| |-------------| | | | | | | |MIP X MD | | | |MIP X MD | | | | | LVL| | PE1-rs PE3-rs | | | LVL| | | | | 5 | | | | | 5 | | | | | | | | | | | | | -----|----- | | -----|----- | |------|------| |------|------| | | | | CE-1 CE-4 MTU1-s MTU3-s Figure 3: HVPLS network with CFM,VCCV and VPLS OAM Stokes, at al. Expires May 18, 2008 [Page 9] Internet-Draft OAM for VPLS November 2007 A VPLS capable PE device may be connected with customer AC as well as spoke. In such cases the corresponding VPLS OAM entity performs both MEP and MIP functions. A VPLS OAM entity is required to support the following set of functionalities. 5.1 Discovery Discovery function enables a VPLS OAM Entity to learn sufficient information (e.g IP addresses, MAC addreses etc.) about the other VPLS OAM entities(current layout of PEs and MTUs participating in the H-VPLS) in the same VPLS service. Such function is also necessary for the utilities defined in this document to exchange the OAM packets among the VPLS OAM entities. 5.2 Service Verification Service verification has to do with whether the service aware nodes have been consistently configured for the service or not. VPLS services may be provisioned manually or may be auto discovered with BGP[L2VPN-FRMK]. It is desirable to do service verification and report misconfiguration to the operator on a proactive basis. 5.3 Connectivity Fault Management Connectivity Fault Management (CFM) is an inherent aspect of OAM and signifies the functions to validate that customer (service) frames can be exchanged between any two VPLS service forwarders in VPLS. There are three aspects of fault management in a provider network. It is to note that the term "CFM" used in this context is generic and not the CFM framework in [802.1ag]. 5.3.1 Connectivity Fault Detection The primary goal of Connectivity Fault Detection is proactive connectivity fault monitoring among the VPLS Forwarders in a VPLS. A secondary goal of such permanent continuity check is topology discovery and service verification in the H-VPLS. This document describes how the existing standardized mechanisms can be used for fault detection. 5.3.2 Connectivity Fault Verification There are five logical steps to verify the operation of an H-VPLS: Stokes, at al. Expires May 18, 2008 [Page 10] Internet-Draft OAM for VPLS November 2007 - Verify that all H-VPLS nodes are operational - Verify that all H-VPLS peer sessions are operational - Verify that all H-VPLS Tunnels are transporting data packets correctly. - Verify that VPLS service frames can be exchanged between any two VPLS forwarders in the H-VPLS. - Verify that actual customer devices can communicate with devices at any site. Existing mechanisms can verify the first three steps. This document describes how to address the final two issues. This requires exchange of VPLS OAM messages among the VPLS OAM entities and the intermediate VPLS forwarders to forward those messages using Forwarding Information Base (FIB) lookups just as they would be for conventional customer (service) frame. This document describes the VPLS Ping function that addresses this requirement. 5.3.3 Connectivity Fault Localization Connectivity Fault Localization is to isolate and diagnose failures in case a VPLS forwarder or customer MAC address is not reachable from the Connectivity Fault Verification phase. A secondary goal of fault isolation is to provide information of VPLS forwarders in the H-VPLS that is traversed by service frames to reach the destination MAC address. This document describes the VPLS Traceroute function for isolation of such faults. Both the VPLS Ping and VPLS Traceroute functions have the ability to send replies to the sender through the control plane (out of band channel), so as to allow for error reporting when all else fails. 5.4 Partial Mesh Detection and Resolution The full mesh of PWs in the H-VPLS core with split horizon rules [RFC4761][RFC4762] emulates a LAN segment over MPLS/IP PSN network. If one or more of these PWs is absent, so that there is not really a full mesh, various higher layer protocols that expect a LAN-like service may fail to work as expected [ROSEN_MESH]. This includes protocols from routing to bridge control. Therefore it is desirable to have procedures that enables the VPLS OAM entities (in the PE devices participating in the full-mesh) to determine automatically whether there is really a full-mesh or not. It is also desirable to have procedures that cause the H-VPLS components to adapt to PW failures[L2VPN-OAM-REQ]. There are following ways partial mesh may occur in VPLS[ROSEN-MESH]. Stokes, at al. Expires May 18, 2008 [Page 11] Internet-Draft OAM for VPLS November 2007 A. Configuration errors. B. Failure of the auto-discovery process. C. Failure of the control plane to properly establish all the necessary PWs. This in turn may be due to bugs, or to resource shortages at the PEs. D. Failure of the data plane to carry traffic correctly on all the established PWs comprising the full-mesh. This can occur if there are bugs in the encapsulation/decapsulation procedures at the PEs, or bugs in the forwarding procedures at intermediate nodes (especially where dataplane and control plane are decoupled). To monitor and detect mesh PW failures, existing PW VCCV mechanisms can be used at PW level. At VPLS forwarder level keeplive messages can be sent between the corresponding VPLS OAM entities using the methodology described in this document. The PW control protocols can be extended to notify mesh failures in a scalable and reliable manner. The critical component of the solution is the ability to react to a partial mesh after such condition is detected. This requires efficient coordination and computation among the participating VPLS OAM entities such that a consistent set of PWs are "removed" from the mesh topology, thus resulting in a "subset" full-mesh. A solution for partial mesh resolution is currently under progress and details will be provided in a future revision. 5.5 MAC Populate and MAC Purge It is useful to test the MAC learning and forwarding capabilities in the VPLS instance in the absence of customer's data traffic. During provisioning a VPLS provider may use MAC Populate functionality to install MAC addresses across the VPLS forwarders and perform VPLS Ping, VPLS Traceroute functions on those MAC addresses. MAC Purge can used to remove the installed MAC addresses from the VPLS forwarders. A detailed methodology for MAC Populate and Purge will be provided in a future revision. 5.6 Performance Monitoring Performance measurement of a VPLS service is required to verify Service Level Agreement (SLA)s with the customer to whom a VPLS service is being offered. Specific performance measurement or diagnostic functions are Frame Loss, Frame Delay, Frame Delay variation etc [L2VPN-OAM-REQ]. Stokes, at al. Expires May 18, 2008 [Page 12] Internet-Draft OAM for VPLS November 2007 Flooded frames in H-VPLS may be unknown unicast, multicast, or broadcast frames that require frame replication along a point-to- multipoint (P2MP) downstream path. As a consequence, the performance of flooded and non-flooded frames can be significantly different. Multicast applications e.g multiparty conference calls, can be sensitive to frame delay and frame delay variation and P2MP diagnostics is required for such applications in H-VPLS. Existing performance measurement functionalities in Ethernet OAM [Y.1731] can be used to test the performance of an H-VPLS service. Moreover, service providers are already using external measurement devices to perform some of these functions[L2VPN-CFM]. This version of the document does not address any performance measurement issues and may be described in a future revision if requirement of such solution arises. 6. Architectural Considerations This section describes the structural model or the methodology used in the VPLS OAM mechanisms described in this document. The encapsulation of the VPLS OAM packets and the traversal methodology across the VPLS forwarders is described. It is to note that the methodology is designed for OAM in a general L2 VPN Service. OAM solutions for L2 VPN services other than VPLS are out of scope of this version of the document. 6.1 MEP and MIP Identifiers The methodology used in this document uses a MAC address to identify the VPLS OAM entities, which is unique in the H-VPLS. The MAC address SHOULD be a MAC address of the service node such that an un-encapsulated frame received from an H-VPLS service node with this destination MAC address will be delivered to the receiving node's control plane. We also use loopback or "always on" IP addresses of the VPLS service aware node that allows the MEPs and MIPs to address each other, which is actually important for error reporting when everything else fails. Apart from the IP and MAC addresses the MEP indentifier is qualified with the VPLS instance identifier. It is to note that for VPLS OAM it is not required to explicitly configure MEP or MIP corresponding to a VPLS forwarder. When a VPLS service is provisioned in a PE device, the corresponding VPLS OAM entity is automatically configured. 6.2 FIB Traversal In order to determine state and accurate performance measurement of a VPLS service, VPLS OAM packets should receive the same data path Stokes, at al. Expires May 18, 2008 [Page 13] Internet-Draft OAM for VPLS November 2007 forwarding behavior as that of VPLS service frames/packets [L2VPN- OAM-REQ]. The VPLS OAM packets traverse the VPLS Forwarders using the same forwarding lookup tables in the dataplane as the Customer frames. 6.3 Control plane and Data plane consistency It is desirable to have the capability in VPLS OAM that enables a Service Provider to troubleshoot and isolate issues associated with VPLS control plane and transport. Such tools detect misconfiguration or inconsistency between control plane and dataplane in an H-VPLS, esp. when both are decoupled from each other. VPLS service frames may be black holed or may be leaked to other service instances due to misconfiguration of the PWs in data plane. Such inconsistency may also occur temporarily when control plane is in transient state, thus causing service disruption or traffic leaks between VPLS instances. During verification and localization of faults it is imperative to detect such issues in the VPLS forwarders with adequate level of accuracy. In the mechanisms described in this document, typically connectivity is first checked against the dataplane. If a VPLS OAM request packet makes it to the destination OAM entity, the reply packet is sent along the data plane. Otherwise, if a reply is not received within the desired interval, the sender sends another request packet along the data plane, requesting a reply back on the control plane. If this also fails, a final attempt may be made, with request sent along the control plane, and the reply back along the control plane. If this fails, then the network is probably partitioned. Such multi-step probing facilitates determination of control plane and dataplane inconsistencies with adequate accuracy. Moreover it is important to send reply to the originator via out of band channels when the dataplane in the reverse direction has failed. 7. Packet Formats, Encapsulations and Extensions VPLS OAM uses the packet format and encapsulations of MPLS Echo Request/Reply (LSP Ping) described in [RFC4379]. This section describes the extensions proposed to LSP Ping and VCCV required for all VPLS OAM operations. The extensions specific to a VPLS OAM function are defined when such functions are described in the later sections of the document. It is suggested that first-time readers skip this section and read subsequent sections; the document is structured in this way to avoid forward references. Stokes, at al. Expires May 18, 2008 [Page 14] Internet-Draft OAM for VPLS November 2007 The common header format of MPLS Echo Request/Reply packets is described in [RFC4379]. An MPLS Echo Request/Reply is sent as an IPV4 or IPV6 packet encapsulated in UDP. The V flag (Validate FEC Stack) in the header SHOULD be set in all VPLS OAM messages to perform control plane and dataplane consistency validation. The header encapsulates one or more TLVs. 7.1 Hierarchical L2 Circuit FEC Stack Sub-type MPLS Echo Request/Reply message carries Target FEC Stack TLV [RFC4379] that contains the control plane information to be validated against the data plane at the receiver. For L2 VPN OAM purposes a new Target FEC Stack Sub-type is defined as "Hierarchical L2 Circuit" that carries the unique identifier of the L2 VPN for which an OAM message is exchanged. Sub-Type Length Value Field -------- ------ ----------- 17 variable Hierarchical L2 Circuit The format of Hierarchical L2 Circuit FEC Stack Sub-type is shown below: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | L2VPN ID Sub-TLV | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Encapsulation Type | Must Be Zero | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | L2 specific Sub-TLV (Optional) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 5. Hierarchical L2 Circuit Sub-type L2VPN ID Sub-TLV = This Sub-TLV carries the identification of the L2VPN. Details of this Sub-TLV are provided in Section 7.2. Encapsulation Type = This is the same type field described in [RFC4447],[RFC4446]. Stokes, at al. Expires May 18, 2008 [Page 15] Internet-Draft OAM for VPLS November 2007 L2 specific Sub-TLV = This Sub-TLV carries additional information specific to a L2 VPN service. See Section 7.3 for details. Note that L2VPN ID Sub-TLV and L2 specific Sub-TLV follows the identical Sub-TLV format prescribed in [RFC4379]. The interpretation of the specific Sub-TLV depends on the position of its occurrence. 7.2 L2VPN ID Sub-TLV The L2VPN ID Sub-TLV follows the Sub-TLV format prescribed in [RFC4379] as below: +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Value | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 6. L2VPN ID Sub-TLV The type of the L2VPN ID is based on the L2 VPN identifier associated with the specific signaling protocol used for setting up the PWs in the L2 VPN. Following types are defined in this version: L2VPN ID Type Description ============= ====================================== 0x0001 Value field carries VC-ID if PW-id FEC Element (FEC128) is used by LDP to signal the PW [RFC4447]. 0x0002 Value field carries Attachment Group Identifier (AGI) from Generalized PW-id FEC Element (FEC129) if LDP is used to signal the PW[RFC4447]. If L2TP is used as the signaling protocol to setup the PW then AGI is taken from the AGI AVP[RFC4667]. 0x0003 Value field carries Route Distinguisher (RD) if the BGP procedures defined in [RFC4761] is used to setup the PWs in a L2VPN. Stokes, at al. Expires May 18, 2008 [Page 16] Internet-Draft OAM for VPLS November 2007 7.3 L2 specific Sub-TLVs The "L2 specific Sub-TLVs" use the same Type field as defined in [RFC4446]. These are: VC type Description ======= ================================== 0x0001 Frame Relay DLCI (Martini Mode ) 0x0002 ATM AAL5 SDU VCC transport 0x0003 ATM transparent cell transport 0x0004 Ethernet Tagged Mode 0x0005 Ethernet 0x0006 HDLC 0x0007 PPP 0x0008 SONET/SDH Circuit Emulation Service Over MPLS 0x0009 ATM n-to-one VCC cell transport 0x000A ATM n-to-one VPC cell transport 0x000B IP Layer2 Transport 0x000C ATM one-to-one VCC Cell Mode 0x000D ATM one-to-one VPC Cell Mode 0x000E ATM AAL5 PDU VCC transport 0x000F Frame-Relay Port mode 0x0010 SONET/SDH Circuit Emulation over Packet 0x0011 Structure-agnostic E1 over Packet 0x0012 Structure-agnostic T1 (DS1) over Packet 0x0013 Structure-agnostic E3 over Packet 0x0014 Structure-agnostic T3 (DS3) over Packet 0x0015 CESoPSN basic mode 0x0016 TDMoIP AAL1 Mode 0x0017 CESoPSN TDM with CAS 0x0018 TDMoIP AAL2 Mode 0x0019 Frame Relay DLCI [RFC4762] specifies that Ethernet or Ethernet Tagged Mode VC type to used for VPLS. For VPLS, the L2 specific Sub-TLV SHOULD contain sufficient information to identify the target remote service node and to allow the remote node to be able to respond. The L2-specific Sub-TLV is included in VPLS OAM messages to support implementations where processing is distributed, in case the MAC header is not passed along with the IP packet to control plane. So in such cases the sender and target MAC addresses are identified from the L2-specific Sub-TLV. Stokes, at al. Expires May 18, 2008 [Page 17] Internet-Draft OAM for VPLS November 2007 The following Sub-TLV is proposed for VC type Ethernet: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type (0x0005) | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Target Ethernet MAC address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | 802.1Q Tag | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sender Ethernet MAC address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | 802.1Q Tag | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 7. L2-specific Sub-TLV Type The field MUST be set to a value of 0x0005 for Ethernet VC. Length This field specifies the total length of the Value fields in octets. Target Ethernet MAC address This field specifies the target Ethernet MAC address of the remote spoke or customer device. It is included here to assist implementations when processing VPLS OAM message reply. Sender Ethernet MAC address This field specifies the sender's Ethernet MAC that can be used in a reply. 802.1Q Tag These fields specify the 802.1Q Tag associated with the Target and Sender Ethernet MACs described above. If the Ethernet address is associated with a VLAN, the 802.1Q Tag field MUST be set to the VLAN tag. If the MAC address is not associated with a VLAN (untagged VLAN), it MUST be set to zero. Since an 802.1Q tag is 12-bits, the high 4 bits of the fields MUST be set to zero. Stokes, at al. Expires May 18, 2008 [Page 18] Internet-Draft OAM for VPLS November 2007 7.4 VPLS OAM Encapsulation A VPLS OAM Packet should look like a customer data frame that allows lookup methods and encapsulations used in the dataplane to be preserved. The VPLS OAM packet receives the same forwarding treatment as the customer frames. It is desirable for both operational and security reasons to be able to easily recognize in the data plane that a received packet is a VPLS OAM packet. VPLS OAM PDUs are sent as LSP Ping (MPLS echo request/reply) packets [RFC4379]. The LSP Ping header MUST be used in accordance with [RFC4379] and MUST contain the target FEC Stack containing the L2VPN ID Sub-TLV. The Sub-TLV indicates the VPLS instance to be verified. The VPLS OAM PDU (IPv4/IPV6 UDP LSP Ping packet) is encapsulated in Ethernet header identical to the conventional customer frames and is forwarded by VPLS forwarders using FIB lookup. Throughout the document we would use the term ?VPLS IP UDP LSP Ping packet? to specify the format used for VPLS OAM packet. The PW Associated Channel Header (PW-ACH)[RFC4385] is used to transport and identify the VPLS OAM packets across VPLS forwarders. VCCV [VCCV] provides the mechanism for transporting PW connectivity verification messages across a SS-PW using the PW-ACH control word. [SEG-PW] describes two methods of PW diagnostics across the MS-PW. One that re-uses the SS-PW control with PW Label TTL expiry and another that uses the MH-VCCV control word. This document describes the method of transporting VPLS OAM packets over these existing VCCV control channels that uses the PW-ACH. The PW end-points should be able to continue operating the PW diagnostics over the VCCV channel and to discriminate those from the VPLS OAM Packets that uses the same PW-ACH. 7.4.1 Pseudowire Associated Channel Type for VPLS OAM The PW Associated Channel Types used by VCCV rely on previously allocated numbers from the Pseudowire Associated Channel Types Registry [RFC4385]. In particular, 0x021 (Internet Protocol Version 4) is used whenever an IPv4 payload follows the PW-ACH, 0r 0x57 is used when an IPv6 payload follows the PW-ACH. In cases where an Ethernet payload is carried by the PW-ACH, a new Pseudowire Associated Channel Types Registry [RFC4385] entry of 0x08 is used. This new channel type is defined as Ethernet Channel type. VPLS OAM packets MUST be transported across the PW over the respective VCCV control words with channel type as the ethernet channel. Stokes, at al. Expires May 18, 2008 [Page 19] Internet-Draft OAM for VPLS November 2007 The payload of the VCCV control words with Ethernet channel type contains a 4 byte Ethernet Control Header (ECH) shim followed by the Ethernet Payload. The ECH qualifies the Ethernet payload and its format is shown as below: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |ECH-TTL | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 8. Ethernet Channel Header. ECH-TTL = TTL for the Ethernet Control Channel. The use of ECH-TTL when VPLS OAM packets are transported is explained in detail in section 7.6. The other applications of ECH-TTL are outside the scope of this document. The figure below shows the format when MH-VCCV CW is negotiated to be used over a MS-PW PW and ethernet channel Type is used. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0 0 0 1| 0x00 | Reserved = 0 | Channel Type = Ethernet | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MH-TTL | MH-VCCV Sub-TLV | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ECH-TTL | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Ethernet Header + Data | / / | | +---------------------------------------------------------------+ Figure 9. L2VPN-VCCV Control Word The ECH is meaningful to the egress point of a PW where it is intercepted and passed on to the L2VPN forwarder that processes the VCCV control word. When the PW is used in VPLS, it is the VPLS forwarder that processes the ECH. When VPLS OAM packets are transported using this method, the ethernet header following the ECH SHOULD be identical to the customer frames in the corresponding VPLS instance. The ethernet header is followed by the IPv4/IPv6 UDP LSP Ping packet. Stokes, at al. Expires May 18, 2008 [Page 20] Internet-Draft OAM for VPLS November 2007 When PW Label TTL expiry method with SS-PW control word is used as control channel over MS-PW [SEG-PW] then the PW TTL for VPLS OAM packets SHOULD be large enough (e.g 255) so that the packet reaches the terminating PE (T-PE) of the MS-PW. Similarly when MH-VCCV control word [SEG-PW] is used as control channel over the MS-PW then MH-TTL SHOULD be set to a large value(e.g 255) for VPLS OAM packets. 7.4.2 VCCV CV Type for VPLS OAM VCCV can support several Connectivity Verification types (CV types) or protocols. This section defines a new CV type to be used for VPLS OAM. The new CV type applies to both MPLS and L2TPv3 Pseuodowire demultiplexors. The CV Type indicator field is defined as a bitmask used to indicate the specific CV type or types of VCCV packets that may be sent on the VCCV control channel. The value shown below augment those already defined in [VCCV]. They represent the numerical value corresponding to the actual bit being set in the CV Type bitfield. The CV type for VPLS OAM is defined as follows: Bit(Value) Description -------------- -------------- Bit 6(0x40) Ethernet IPv4/IPV6 UDP LSP Ping. 7.5 Dataplane processing for VPLS OAM For VPLS OAM functions such as VPLS Traceroute to work properly, the basic operation of H-VPLS needs further definition in the VPLS forwarder. In a typical two tier hierarchy H-VPLS, a data packet normally goes from an H-VPLS spoke node (MTU/u-PE) forwarder to an H- VPLS core node (PE) forwarder. In the core node forwarder, the PW Label received from the spoke node is used to determine the VPLS instance with which the encapsulated frame is associated. The frame is un-encapsulated and a check is made for the frame's destination MAC address. If the destination MAC address is known, the frame is forwarded to the downstream core node forwarder from whom the MAC address is learned. As the frame is forwarded, the frame is again encapsulated using the PW label received from the downstream core node forwarder. A similar process occurs at the downstream core node forwarder. The PW label is checked, the frame is un-encapsulated, the destination MAC address is checked, and the frame is again encapsulated and sent Stokes, at al. Expires May 18, 2008 [Page 21] Internet-Draft OAM for VPLS November 2007 to the remote spoke node forwarder. The packet containing the newly encapsulated frame uses the PW label received from the destination spoke node forwarder. Each time this process occurs, if a VPLS OAM packet is received as the PW payload then VPLS forwarder processes the respective VCCV CW. As the VPLS OAM Packet exits the PW at egress, the VPLS forwarder identifies the packet belonging to a specific VPLS instance from the PW Label. If the ECH-TTL in the received packet is 1, then the packet MUST be passed to the node's control plane (VPLS OAM Entity). If it cannot be passed to the control plane, the packet SHOULD be discarded. The packet MUST NOT be forwarded to the customer network. Note that the rate at which packets are passed to the control plane SHOULD be regulated to prevent overloading or DoS attacks. If the received ECH-TTL value is not 1 then the forwarder checks if the destination MAC address in the ethernet frame is a MAC identifying this forwarder-cum-VPLS OAM entity. Note that it also checks its forwarding database associated with this VPLS to see if the target MAC is associated with(learned from) a locally attached customer network. If the MAC is found to be local, then this node is the egress node for the VPLS OAM packet and MUST be passed to the node's control plane. If the destination MAC address is not local and learned from a downstream VPLS forwarder, then the VPLS OAM packet along with the CW SHOULD be sent to the downstream node with the PW label received from it. If the destination MAC address is not known by this forwarder, then it SHOULD flood the VPLS OAM packet to all downstream VPLS forwarders. Each time the VPLS OAM packet is forwarded, the ECH-TTL SHOULD be decremented and the result SHOULD be placed in the ECH-TTL field in the outgoing CW. It is to note that ECH-TTL is available only to the VPLS forwarder as it receives the frames from an emulated LAN interface(PW). If the destination MAC address in the VPLS OAM packet is not known then the packet is flooded to multiple VPLS forwarders. In that case the decremented ECH-TTL described above is placed in each outgoing CW. In normal operation, the ECH-TTL value SHOULD be set to a value larger than the number of "H-VPLS Hops" or VPLS forwarders through which the data packet will traverse, e.g. 255. Stokes, at al. Expires May 18, 2008 [Page 22] Internet-Draft OAM for VPLS November 2007 7.6 Egress Node Control Plane Processing This section describes the processing of a VPLS OAM packet at the egress node control plane. The egress node control plane receives the VPLS OAM packet as IP UDP LSP Ping packet. The Hierarchical L2 Circuit FEC Stack Sub-type identifies it as VPLS OAM packet and the packet is delivered to the VPLS OAM entity for processing. The VPLS OAM entity checks the Target Ethernet MAC address field in L2- specific Sub-TLV to determine if it is a MAC address that identifies the local VPLS forwarder. If it is, a successful reply is returned. If the MAC address does not belong to the local VPLS forwarder, an additional check is made of the forwarding database associated with the VPLS indicated by the L2VPN ID Field in Hierarchical L2 Circuit FEC Stack Sub-type. If the MAC address is present and is associated with a locally attached customer network for this H-VPLS, a successful reply is returned. If not, the ECH-TTL is checked to determine if this is a VPLS Traceroute Packet and reply is sent accordingly. VPLS OAM replies require new LSP Ping Return Code information. To avoid defining codes that are specific to VPLS, an encoding of the Error Code TLV is proposed. This encoding is shown in Section 7.8. Note that this method of operation allows VPLS OAM functions to check for both remote node MACs and the remote CE device MACs. When the target MAC address in a VPLS OAM packet is unknown in a core node, the packet is flooded. As such, it potentially reaches other spoke nodes in addition to the spoke node where the destination is actually present. 7.7 Error Code TLV The proposed Error Code TLV for use with [RFC4379] is shown below. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type (0x0004) | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Error Code Sub-TLV | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 10. Error Code TLV Type The field MUST be set to a value of 0x0004 for Error Code Stokes, at al. Expires May 18, 2008 [Page 23] Internet-Draft OAM for VPLS November 2007 TLV and is subjected to IANA approval. Length This field specifies the total length of the Value fields in octets. In this case the Value field is an Error Code Sub-TLV. Error Code Sub-TLV The proposed encoding for these Sub-TLVs is shown below. The Error Code Sub-TLV uses the Target FEC Sub-Type values defined in [RFC4379] as the type field value. The proposed encoding for a VPLS FEC Stack Sub-Type is shown below. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type (0x000A) | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Error Code | Must Be Zero | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 11. Error Code Sub-TLV Type The field MUST be set to a value of 0x000A for Hierarchical L2 Circuit. Length This field specifies the total length of the Value field in octets. In this case the Value field is an Error Code Sub-TLV. Error Code The following codes are defined: Value Meaning ----- ------- 1 Replying router is the egress for the given MAC address 2 Replying router is not a member of the indicated HVPLS 3 Replying router is a member of the indicated HVPLS, but is not one of the "Downstream Routers" 4 Replying router is one of the "Downstream Routers" and it has the given MAC address in its forwarding Stokes, at al. Expires May 18, 2008 [Page 24] Internet-Draft OAM for VPLS November 2007 database (but it is not the egress) 5 Replying router is one of the "Downstream Routers" but the label mapping is incorrect 6 Replying router is one of the "Downstream Routers" but it does not have the given MAC address in its forwarding database 7 Replying router is one of the "Downstream Routers" and is a member of the indicated HVPLS, but it does not have the given MAC address and is unable to flood the request 8 Replying Router is one of the ?Downstream Routers? and is a member of the indicated VPLS, and has the given MAC address, but the MAC is not learned from a downstream node and is unable to forward the request. 7.8 Vendor-specific Extensions A vendor TLV is defined for vendor-specific extensions. One of the issues with defining TLVs for service definitions is that there is no standard for service definitions. It may be possible to exchange high-level information such as operational status, but finer details are sometimes vendor-specific. Therefore a new vendor-specific TLV type is proposed. The resulting list of TLV types is shown below. Type # Value Field ------ ----------- 1 Target FEC Stack 2 Downstream Mapping 3 Pad 4 Not Assigned 5 Vendor Enterprise Number 6 Not Assigned 7 Interface and Label Stack 8 Not Assigned 9 Errored TLVs 10 Reply TOS Byte 11 Vendor-specific The following format is proposed for the Vendor-specific TLV: Stokes, at al. Expires May 18, 2008 [Page 25] Internet-Draft OAM for VPLS November 2007 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type = (0x0005) | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Vendor OUI | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | ~ Value ~ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 12. Vendor Specific TLV A vendor MAY encode vendor-specific information in these vendor TLVs. In general, if possible, the TLVs SHOULD contain values that are strings, so that all vendors are able to display the values. A request or response of a VPLS Ping or VPLS Traceroute MAY contain one or more vendor TLVs. A VPLS OAM entity that does not understand a vendor TLV MAY ignore but SHOULD forward it. 8. VPLS OAM Functions This version of the document defines the following VPLS OAM functions using the methodology and extensions described in Section 7. 8.1 VPLS Topology Discovery A spoke node (MTU) normally knows the IP address of the immediate core node (PE) with which the H-VPLS spoke is connected to. It typically does not know the IP addresses or MAC addresses of remote spoke nodes and remote core nodes. In order to perform VPLS OAM functions such as VPLS Ping, VPLS Traceroute etc. to check connectivity between the H-VPLS nodes, a VPLS OAM entity must know the MAC address of a remote VPLS OAM entity. Furthermore, it is desirable to identify a VPLS OAM entity by its IP address for out of band error reporting when everything else fails. The bridge MEPs [L2VPN-CFM] located in the VPLS PE devices may contain the CCM database that has MAC address information of all other bridge MEPs in that VPLS instance. Note that the bridge MEP and the VPLS MEP in a service aware node may use the same MAC address. [L2VPN-CFM] further describes the method of distributing IP addresses of the bridge MEPs by using the optional Sender TLV. However the CCM database does not contain information of the VPLS OAM MIPs. Both IP and MAC address of a VPLS OAM entity could be distributed through the Stokes, at al. Expires May 18, 2008 [Page 26] Internet-Draft OAM for VPLS November 2007 control plane protocols used to setup the PWs in the H-VPLS. It is also desirable to have such mechanism in control plane in order to do VPLS service verification. See section 8.2 for details. [RFC4762] defined a MAC TLV that is used in a LDP Address Withdraw Message for MAC address withdrawal in topology change. In a similar manner, a LDP Address Message is used to distribute the required information about VPLS OAM entities across the OAM domain. It is possible to extend such mechanisms into other signaling protocols, in BGP auto discovery[RFC4761] and in L2TP[RFC4667]. 8.1.1 Ethernet VPLS node TLV To distribute H-VPLS node information, a new TLV is proposed for use with a LDP Address Message. This new Ethernet VPLS node TLV is shown below. The type field MUST be 0x406 and is subjected to IANA approval. For IPv4: 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |U|F| Type (0x406) | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Reserved | Address Family | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IPv4 Address of VPLS node #1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MAC address of VPLS node #1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | CoS |K| 802.1Q Tag #1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IPv4 address of VPLS node #2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MAC address of VPLS node #2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | CoS |K| 802.1Q Tag #2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 13. Ethernet VPLS Node TLV. The Ethernet VPLS Node TLV carries node information (MAC Address and IP Address) for a list of nodes. Note that the node's IP address is included to allow remote nodes to correlate the node's MAC address with its IP address. In this manner Stokes, at al. Expires May 18, 2008 [Page 27] Internet-Draft OAM for VPLS November 2007 an operator can request operations such VPLS Ping or VPLS Traceroute by specifying the IP address of the remote service node. The CoS bits MAY be used to specify class of service provisioned for the VPLS instance. Details of its use is a subject of further study. The K-bit indicates if the node has enabled transmission of periodic keepalives for proactive connectivity fault detection with this node. The receivers of the node information MAY detect connectivity faults with this node by monitoring its the keepalives. The scope of the Ethernet VPLS node TLV is the VPLS instance specified in the FEC TLV in the LDP Address Message. If any parameter in the VPLS Node TLV is modified by a node then it MUST withdraw the node information for the VPLS advertised earlier and SHOULD re-advertise with the new parameters. 8.1.2 VPLS Topology Discovery Procedures This section describes the VPLS topology discovery using the control plane mechanism. A method is described when LDP is used as the signaling protocol for setting up the PWs in H-VPLS[RFC4762]. When a spoke node and a core node establish a new peer relationship in LDP for an H-VPLS, they exchange one or more Address Messages with the VPLS Node TLV. The spoke node sends information about a single H- VPLS node (itself). The IP address SHOULD be the address by which the node is known to its peers. The MAC address SHOULD be a MAC address of the service node such that an un-encapsulated frame received from an H-VPLS peer with this destination MAC address will be delivered to the node's control plane. When two core nodes establish a mesh relationship, they also exchange one or more Address Messages with the above information via the targeted LDP session(s) between them. In this case each core node sends the MAC address information for all of the spokes to which it is connected, as well as for itself. Returning to the case of a core node establishing a peer connection with a spoke node, the core node sends to the spoke the entire MAC addresses that it has learned from all of its core peers as well as for itself. In this manner, each spoke node will know the MAC addresses of all of the other spokes in the H-VPLS. Note that in large networks multiple Address Messages may be required to ensure that media MTU sizes limitations are not exceeded. Stokes, at al. Expires May 18, 2008 [Page 28] Internet-Draft OAM for VPLS November 2007 When a core node loses the PW connection to a spoke, it instructs all of its core peers to remove the corresponding spoke MAC address by sending the above information in an Address Withdrawal Message [RFC3036]. The core nodes in turn pass that information along in LDP Address Withdrawal Messages to their spokes nodes. When a core node loses the PW connection to another core node, it instructs all of its spoke nodes to remove all of the corresponding MAC addresses (received from the remote core node earlier) by sending the above information in an LDP Address Withdrawal Message. At the end of these procedures each service aware node will have complete information of all other service aware nodes in that VPLS instance. Throughout the document we will term the database built by LDP Address Messages as VPLS Node Database. From the VPLS Node Database a service aware node can lookup any remote service node information and can exchange VPLS OAM packets to test the data plane operation between itself and the remote node. When a spoke node is dual-homed to two core PE nodes, the spoke node sends its node information to both the core PE nodes. Both the core PE nodes further propagate the spoke node information to all remote nodes. As a result of this, a core node in the H-VPLS may receive the spoke node information from at least two different peers. It is to note that the VPLS Node database MUST NOT be used for forwarding of customer or VPLS OAM frames. Apart from VPLS OAM there may be other applications that may make use of the VPLS node database and is outside the scope of this document. For example the VPLS Node database in a VPLS MEP can be used as input to the CCM database in the bridge MEP. 8.1.3 Operation of limited H-VPLS nodes There may be some implementations that do not want to maintain a list of IP and MAC addresses of all VPLS aware remote service points. In such an environment, an operator can still invoke a VPLS ping or VPLS traceroute if the MAC address of the remote spoke or customer device is provided. There may be some cases where an operator knows the IP address of a remote spoke, but not the MAC address. A method to obtain the MAC address when only the IP address is known may be described in a future version. If proactive topology discovery mechanism is not used then VPLS Traceroute can be used for the same purpose on an adhoc basis. For Stokes, at al. Expires May 18, 2008 [Page 29] Internet-Draft OAM for VPLS November 2007 example, Traceroute sent for an unknown destination MAC address is flooded across the VPLS forwarders in the H-VPLS and each VPLS OAM entity in the H-VPLS would reply. However VPLS Trace route operations reveals only the data plane view of the topology. See section 8.5 for details of VPLS Traceroute operations. 8.2 VPLS Service Verification The secondary goal of the methodology used in topology discovery is VPLS service verification. Section 8.1 describes the method for distributing a VPLS node or OAM entity information using the control plane protocol. Such control plane view of the VPLS topology may be used as reference by data plane verification techniques. The nodes that have enabled periodic keeplives can be checked for connectivity faults by a receiver of the node information. The definitions and procedures for such keepalives are outside the scope of this document. For example, CCM [L2VPN-CFM] between the bridge MEPs MAY be used as the keepalives for monitoring the connectivity between the VPLS MEPs. Typically a VPLS MEP may set the K-Bit in its node information if the corresponding bridge MEP has enabled CCM. A receiving VPLS MEP MAY check for CCMs from the sender MEP. Further, when a CCM message is received from a remote bridge MEP the VPLS Node Database is referred to check whether the remote bridge MEP is a member in the H-VPLS. A VPLS OAM entity that sends a VPLS OAM message is be cross verified with the VPLS Node Database to detect service misconfiguration, wrong cross connects etc. Service verification may also be performed on an on-demand basis using VPLS Traceroute. When a VPLS traceroute is sent for an unknown destination MAC address, each VPLS OAM entity replies. Sender of each reply can be verified against the VPLS Node database whether it is member of that service or not. 8.3 VPLS Connectivity Check In an H-VPLS connectivity fault is detected when two CE devices fails to communicate with each other across the H-VPLS domain. Such fault may occur due to connectivity failure between two or more VPLS forwarders located at the edge devices (or spoke nodes) in the HVPLS. This version of the document does not define any mechanism for proactive connectivity fault detection. CCM[802.1ag] can be used for connectivity checks in an H-VPLS on a proactive basis. CCM frames are exchanged periodically among the bridge MEPs in the VPLS edge devices[L2VPN-CFM]. If a bridge MEP does not receive CCM from a remote bridge MEP in the expected interval then a connectivity fault Stokes, at al. Expires May 18, 2008 [Page 30] Internet-Draft OAM for VPLS November 2007 is detected between with the remote bridge MEP and is reported to the network operator. The bridge MEP may also generate fault notification to the VPLS MEP to trigger fault verification and isolation among the corresponding VPLS MEPs. 8.4 VPLS Ping VPLS Ping function is used to verify that data packets can be exchanged between any two VPLS forwarders in the H-VPLS. A VPLS Ping Message is issued by a source VPLS OAM entity to a target VPLS OAM entity for fault verification when a fault is detected between the corresponding VPLS forwarders. The appropriate target forwarder in front of fault will respond with a VPLS Ping Reply. The target forwarder behind the fault will not respond. VPLS Ping is a reactive or on-demand utility that may to be used by the network operator when fault is reported by the connectivity fault detection mechanisms. A VPLS Ping Message is also used to verify connectivity of customer MAC addresses learned in the H-VPLS. The VPLS Ping Request traverses all the downstream VPLS forwarders from which the customer MAC address is learned. Note that a data frame with destination MAC as the customer MAC address follows the same set of VPLS forwarders. The VPLS OAM Entity in the egress node for the customer MAC address will respond with a VPLS Ping Reply. If the target customer MAC address is not known by an intermediate VPLS forwarder that receives the VPLS Ping Message, then the message is flooded to all downstream forwarders. In that case the VPLS Ping message would reach each leaf/spoke node (VPLS OAM MEP) in the H-VPLS. Each spoke node (MEP) will reply to the VPLS Ping Message. 8.4.1 VPLS Ping Message Format VPLS Ping is sent as the MPLS Echo Request message format [RFC4379]. It encodes the Hierarchical L2 Circuit FEC Stack Sub-type identifying the VPLS instance for which the Ping function is initiated. The Message is sent with Reply Mode = 4 (Reply via application level control channel) in the MPLS Echo Request header. The VPLS OAM entity initiating the VPLS Ping encapsulates the VPLS IP UDP MPLS Echo Request with the L2VPN-VCCV CW and with the same label stack as normal customer data, and then sends the packet to downstream forwarder. The message is sent as unicast with source MAC address of the sender VPLS OAM entity and destination MAC address as that of target VPLS OAM entity or a target customer MAC address. The L2- specific TLV SHOULD be filled as per the source MAC address and destination MAC Address. Stokes, at al. Expires May 18, 2008 [Page 31] Internet-Draft OAM for VPLS November 2007 VPLS Ping Message MAY include a PAD TLV [RFC4379]. PAD TLV contains variable (>=1) number of octets. The first octet takes value 1 (Drop PAD TLV from reply) or 2 (Copy PAD TLV to reply) from the table as defined in [RFC4379]; all other octets (if any) are ignored. The receiver should verify that PAD TLV is received in its entirety, but otherwise ignores the content of this TLV apart from the first octet. PAD TLV with variable sizes may be used to test VPLS path MTU between the test nodes. It may also be used for various test data patterns for diagnosis. The initiator may send a test pattern in the PAD TLV and checks it for sanity/bit errors with the pattern received in echo reply. Further switching delay at every VPLS forwarder may vary based of the data frame size. So PAD TLV may be also used for one way and two way delay measurements for various frame sizes. 8.4.2 VPLS Ping Procedures VPLS Ping mechanism requires the sender VPLS OAM entity to know the MAC address of the destination VPLS OAM entity. To initiate a VPLS Ping, a packet is created with the format as described in Section 8.4.1. The destination Ethernet MAC address is of the remote VPLS OAM entity or of a customer device. The source Ethernet MAC address is of the MAC belonging to the sender. The ECH-TTL in L2VPN-VCCV CW SHOULD be set to 255. A VPLS Ping packet is processed by a VPLS forwarder as described in section 7.5. As a VPLS Ping packet exits a PW at the next VPLS service node, the corresponding VPLS forwarder checks if ECH-TTL is 1 and if found to be true then the packet is passed to control plane. If ECH-TTL > 1, the forwarder checks if the destination MAC address is a MAC belonging to this node; it also checks its forwarding database associated with this VPLS to see if the destination MAC is associated with a locally attached customer network. If the MAC is found to be local, then this node is the egress node for this VPLS Ping. The VPLS Ping packet is passed to the node's control plane. If the destination MAC address is not local then the VPLS Ping packet is further encapsulated with PW label received from downstream service node from where the MAC is learned and sends to the downstream node. If the destination MAC address is neither local nor is learned from a downstream forwarder then the VPLS Ping packet is flooded to all downstream VPLS forwarders. If the VPLS Ping packet is sent to control plane, a VPLS Ping reply is created in the same way MPLS Echo reply is created [RFC4379]. The reply is sent based on the value indicated in the Reply mode field in VPLS Ping Message. Stokes, at al. Expires May 18, 2008 [Page 32] Internet-Draft OAM for VPLS November 2007 For security purposes, before replying to a VPLS Ping, a receiver VPLS OAM entity SHOULD check the sender information to ensure that the sender is known to be in the specified H-VPLS. Such information MAY be obtained from the VPLS Node Database. The replying node SHOULD check the PW Label over which the VPLS Ping packet is received against the label mapping corresponding to the L2VPN ID in the Hierarchical L2 Circuit. This check is necessary for detecting potential misconfiguration or inconsistency between the control plane and data plane. If Label mapping is incorrect then VPLS Ping Reply SHOULD be sent with appropriate Error Code described in Section 7.7. Typically VPLS Ping request is triggered to first check connectivity against the dataplane. If the request packet makes it to the destination VPLS OAM entity, the reply packet is sent along the data plane. Otherwise, if a reply is not received within the desired interval, the sender sends another request packet along the data plane, requesting a reply back on the control plane. If this also fails, a final attempt may be made, with request sent along the control plane, and the reply back along the control plane. If this fails, then the network is probably partitioned. 8.5 VPLS Traceroute When a VPLS Ping operation is unsuccessful, additional capabilities are required to isolate and identify the problem location to a specific VPLS forwarder. The problem may be at the intermediate VPLS forwarders or may be at the VPLS forwarders in the edge devices. VPLS Traceroute is used to localize and isolate such connectivity faults. In normal condition (no fault) VPLS Traceroute can be used for tracing the VPLS forwarders to reach a remote VPLS forwarder or a customer MAC address. Each VPLS OAM entity in the forwarding path responds with a reply. 8.5.1 VPLS Traceroute Message Format VPLS Traceroute uses the same MPLS echo request packet format as used for VPLS Ping. The VPLS Traceroute Message SHOULD contain one or more Downstream Mapping TLVs described in [RFC4379]. In the Downstream Mapping TLV, the Downstream Router ID field contains the IPv4 address of the next H-VPLS node that would be used to reach the Target Ethernet MAC address in the VPLS indicated in the Hierarchical L2 Circuit FEC Stack Sub-Type. The Downstream Label field contains the label stack used to reach the downstream peer. This includes the label(s) for the underlying tunnel LSP (if PSN Tunnels are RSVP-TE Tunnels) and the PW label from the targeted LDP Stokes, at al. Expires May 18, 2008 [Page 33] Internet-Draft OAM for VPLS November 2007 session with the downstream peer. The Protocol field for the PW label indicates a value of 3 for (targeted) LDP. When the indicated Target Ethernet MAC is not known and a packet with this destination MAC would be flooded, the information for all HVPLS peers to which the packet would be flooded is added. For the case where the packet cannot be flooded (such as a limit on MAC addresses that has been exceeded for this HVPLS), a new LSP ping Return code is defined. This new Value 7 is shown in Section 8.1.6. For the case where the destination MAC address is known, but the packet would not be forwarded, no Downstream Mapping TLV is included. For example, the packet may have been received at one core node from another core node, but the MAC address was previously learned from a different core node. The Downstream Mapping TLV includes a Multipath Type to address ECMP considerations. The Multipath types defined deal with IP addresses and MPLS labels. In an H-VPLS, it is possible that a hash may be performed on the source and/or destination MAC addresses of the encapsulated L2 frame. See Section 10 for a discussion of load sharing or ECMP with a MAC-based hash. For the case where a hash is performed on MAC address (es) including the destination MAC address, new MAC-based Multipath types are proposed. The resulting list of Multipath types is shown below. Multipath Type IP Address or Next Label -------------------- ------------------------ 0 no multipath (nothing; M = 0) 2 IP address IP addresses 4 IP address range low/high address pairs 8 Bit-masked IP IP address prefix and bit mask address set 9 Bit-masked label set Label prefix and bit mask 10 MAC address MAC addresses 11 MAC address range low/high MAC pairs This results in the Downstream Mapping format shown below. Stokes, at al. Expires May 18, 2008 [Page 34] Internet-Draft OAM for VPLS November 2007 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MTU | Address Type | DS Flags | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Downstream IP Address (4 or 16 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Downstream Interface Address (4 or 16 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Multipath Type| Depth Limit | Multipath Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IP Address or Next label or MAC Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ . . (more IP Addresses or Next labels or MAC Addresses) . +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Downstream Label | Protocol | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ . . . +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Downstream Label | Protocol | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 14. Downstream Mapping TLV Value In the case of a MAC address, the following format is used for the "IP Address or Next label or MAC Address" field(s): +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Destination Ethernet MAC address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | 802.1Q Tag | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 15. The semantics of the new Multipath Types and the associated MAC Address(es) are shown below. Value Meaning ----- ------- 10 a list of single MAC addresses is provided, any one of which will cause the hash to match this MP path. 11 a list of MAC address ranges is provided, any one Stokes, at al. Expires May 18, 2008 [Page 35] Internet-Draft OAM for VPLS November 2007 of which will cause the hash to match this MP path. See Section 10 for additional ECMP considerations. 8.5.2 VPLS Traceroute Procedures To initiate VPLS Traceroute, an MPLS echo request is created using the destination Ethernet MAC address of the remote service node or customer device. The ECH-TTL in the L2VPN-VCCV CW is set successively to 1, 2, 3 ...etc. As with MPLS traceroute [RFC4379], the echo request contains one or more Downstream Mapping TLVs. For ECH-TTL=1, all the peer nodes (and corresponding PW labels) for the sender with respect to the Target MAC address being traced SHOULD be sent in the echo request. As the echo request may be flooded to multiple nodes, the sending node may receive replies from multiple remote nodes. Thus, for n>1, the Downstream Mapping TLVs from all of the received echo replies for ECH-TTL=(n-1) are copied to the echo request with ECH-TTL=n. Note that this allows an operator to determine which remote nodes would receive a flood of an actual customer data packet destined to the target MAC address. The VPLS Traceroute packet is processed by a VPLS forwarder as described in section 7.5. As a VPLS OAM packet exits a PW at the next VPLS service node, the corresponding VPLS forwarder first checks if the ECH-TTL is 1 and if it is true then the packet is passed to the control plane. If the ECH-TTL > 1, the forwarder checks if the destination MAC address is a MAC belonging to this node. It also checks its forwarding database associated with this VPLS to see if the target MAC is associated with a locally attached customer network. If the MAC is found to be local, then this node is the egress node for this VPLS traceroute. The packet is be passed to the node's control plane. In the control plane, an echo reply is created as described in [RFC4379]. This includes completing the Downstream Mapping TLV. The reply is sent based on the value indicated in the Reply Mode field of the echo request. For security purposes, before replying to a VPLS Traceroute, a node SHOULD check the sender information to ensure that the sender is known to be in the specified H-VPLS. See Section 12 for further discussion. Stokes, at al. Expires May 18, 2008 [Page 36] Internet-Draft OAM for VPLS November 2007 If the destination MAC address is not local, and the value of the ECH-TTL is greater than 1, the ECH-TTL is decremented and the result is placed in the CW for the resulting packet. This packet is encapsulated in the same manner as a customer data packet and then passed to the downstream node that would normally be used to reach the indicated Ethernet MAC. If the packet is flooded to multiple peer nodes, this same ECH-TTL value is placed for each of the outgoing packets. 9. General VPLS Testing Procedures Testing H-VPLS nodes and peer sessions requires minimal network impact since this is done on a per machine and a per VPLS service node basis. Also, monitoring for these failures can be done with SNMP traps. It is important to note that the VPLS OAM functions are expected to catch a significant percentage of the most likely failure cases. If the PSN is MPLS based, then tunnel LSPs are maintained by their associated MPLS signaling protocols, RSVP-TE [RFC3209] or LDP [RFC3036]. Locally detected failures such as link outages or node failures invoke MPLS recovery actions. These actions can include local recovery as in the case of RSVP-FRR [RFC4090]. In the case of a successful local recovery of a tunnel LSP, the H-VPLS nodes may be notified. FRR recovery is one of the ways that an existing service can become corrupted [L2VPN-CFM]. It would be desirable to do a service verification following such a tunnel recovery. This could be done using a VPLS Traceroute to a non-existent MAC address. This VPLS Traceroute could be initiated by an operator or automatically at the ingress of the FRR LSP based on some indication that a recovery action has occurred. If a local recovery is not possible, then RSVP- TE or LDP notifies the H-VPLS node using the tunnel LSP of the failure. This in turn can generate a SNMP trap or an operator error message. In addition to failures and changes detected outside of MPLS, both MPLS signaling protocols have control plane failure detection mechanisms. Both have Hello protocols that can detect link failures as well as MPLS control plane failures. LDP also has a Keep Alive protocol. These Hello and Keep Alive protocols run on the time frame of multiple seconds. They also provide failure notification to the H- VPLS node using the tunnel. As above, this can generate a SNMP trap or an operator error message. In current [RFC4671] or [RFC4672] environments, loss of a targeted LDP session to a peer is normally a key operator notification. While Stokes, at al. Expires May 18, 2008 [Page 37] Internet-Draft OAM for VPLS November 2007 a failed tunnel LSP can generate a notification as described above, these failures can be temporary in nature due to routing transients. LSP ping[RFC4379] is designed to catch failures where the control plane believes that a tunnel LSP is operational but it is in fact not functioning correctly. This corrupted LSP is much less likely to occur than a LSP going down "properly." When used as a background check, it should be used in addition to the above tunnel LSP failure detection methods and not as a replacement. When any of these methods detects a tunnel LSP failure, the H-VPLS node can switch to another LSP if one is available. When the failure is detected by MPLS ping, MPLS traceroute can be used to assist in failure isolation. VPLS OAM is designed to detect problems in the transport aspects of the VPLS operation. It detects flooding and/or MAC learning problems in the network which are not checked in the above tests. Note that the number of possible spoke-to-spoke tests to check an entire H-VPLS can be significant. Therefore, care should be taken when executing VPLS OAM functions as a background test to avoid overloading the network or the H-VPLS nodes. Note also that an individual core node's operation is checked by multiple spoke-to-spoke checks. All of the above tests check the operation of an H-VPLS among the MEPs. It is also possible to use VPLS Ping and Traceroute to check for customer device MAC addresses. While not specified by H-VPLS, there is normally additional information available to an operator to check for problems between the edge of the HVPLS and the customer. These include: - the state of the local customer VLAN or port - this is the simplest test and will normally catch the most likely failures - the L2 MAC entries for the local customer VLAN or port - the H-VPLS transmit/receive statistics As described above, VPLS ping and VPLS traceroute work with previously defined MPLS tests to provide an end-to-end test capability for an H-VPLS. 10. Load sharing considerations Some implementations provide for load sharing a pseudowire across multiple MPLS PSN Tunnels. Note that load sharing is not applicable to other tunneling mechanisms such as GRE etc. Such load shared based Stokes, at al. Expires May 18, 2008 [Page 38] Internet-Draft OAM for VPLS November 2007 implementations have HVPLS test implications and are an important aspect of the methodology used in this document. When customer data entering a VPLS at an ingress node is transmitted to another node over multiple (load sharing) tunnel LSPs, each of these LSPs SHOULD be tested. VPLS Pings and VPLS Traceroutes SHOULD be sent over each of these LSPs. There may also be multiple load sharing tunnel LSPs between a core node which is not the traffic ingress and a downstream node (which may or may not be the traffic egress). At such a core node, a decision is made as to which load sharing tunnel LSP to use to forward a VPLS packet. This decision is often based on a hash of some "random" field. There are at least three options. One option is to hash on the IP addresses of an encapsulated IP packet. This option would potentially need to be combined with another option to handle non-IP frames. A second option is to hash on the label stack of the received VPLS packet. This forces all packets received on a tunnel LSP for the same VPLS to use the same load sharing tunnel LSP to the next core node. This method distributes traffic among the load sharing tunnel LSPs on a VPLS basis. A third option is to hash on fields of the VPLS packet after it has been un-encapsulated. Such a hash could use the destination and source MAC addresses of the un-encapsulated packet. Thus, traffic received on a tunnel LSP for the same VPLS may use any of the load sharing tunnel LSPs to the next core node. This method distributes traffic among the load sharing tunnel LSPs on a MAC address pair basis. The first and third options normally produce a more optimal distribution of packets since IP addresses and MAC addresses should be more random than VPLS labels. This advantage may be somewhat reduced for the first option if customers' data contains a significant amount of non-IP traffic. It may also be somewhat reduced for the third option if customers use a single router at each site to connect to the HVPLS. The second option has an advantage from an VPLS testing perspective. Since the label stack for a VPLS Ping or VPLS Traceroute is the same as for customer traffic, the second option ensures that VPLS pings and traceroutes are testing all of the downstream LSPs used by customer data. The first option has the disadvantage that a hash on the IP address Stokes, at al. Expires May 18, 2008 [Page 39] Internet-Draft OAM for VPLS November 2007 of an encapsulated ping/traceroute packet uses an address in the 127/8 range and not a true customer IP address. The third option has the disadvantage that a hash on the MAC address of a spoke node may differ from the hash on a true customer MAC address. However, remember that actual customer MAC addresses can be used in a VPLS ping/traceroute and these will use the same path as the customer data when using a MAC-based hash. Remember from above that VPLS pings and traceroutes SHOULD be sent using all of the load sharing tunnel LSPs at the ingress node. Load sharing designs and hash algorithms remain implementation options. There are trade-offs between optimal load sharing and testability. Of course, testing using IP ping and traceroute has similar exposures from the effects of equal cost multipath. The methodology described thus far provides a means to verify that all remote nodes can be reached. It also provides an operator with a means to verify operation for particular customer MAC addresses. It does not provide a means to verify all load sharing paths in a HVPLS from a single node. The Multipath information contained in the Downstream Mapping TLV in [RFC4379] provides additional capabilities for verifying all load sharing paths. Use of this information in a VPLS traceroute environment, to test all load sharing paths in an HVPLS, will be discussed in a future version. 11. Scalability Considerations Mechanisms developed for VPLS OAM need to be such that per-service OAM can be supported even though the OAM may only be used for limited VPLS service instances, e.g. premium VPLS service instances, and may not be used for best-effort VPLS services. VPLS OAM should be scalable such that a service aware device can support OAM for each VPLS service that is supported by the device. 12. Security Considerations For security purposes, the edge of a provider's HVPLS network is defined to be a spoke node or a PE that has directly attached customers. Some customers and providers may desire that the provider edge node participates in the customer network. This could be the case when the customer is using the provider's node as a default gateway. In such a configuration, the provider edge node's IP address and Ethernet MAC address are known in the customer network. This would be the case as shown in figure 3 where the bridge module Stokes, at al. Expires May 18, 2008 [Page 40] Internet-Draft OAM for VPLS November 2007 in a VPLS capable device is provisioned for MIP at customer MD level. However, no other provider network information should be exposed to the customer network, esp. VPLS transport related information. When the provider is not furnishing a default gateway function, no provider network information SHOULD be exposed to the customer network. The VPLS OAM messages are transported inside the H-VPLS in the same manner as customer data. This is required to properly test the HVPLS. However, care must be taken to prevent provider network information contained in these test packets from being exposed to the customer network. A test packet that is forwarded to the customer network exposes provider network information to the customer network. Therefore, spoke nodes SHOULD always check for such test packets. Any detected test packet SHOULD NOT be forwarded to the customer network. Another security concern is the receipt of a VPLS ping or traceroute from a node that is not a member of the HVPLS. Should a HVPLS node respond to a test request from a non-HVPLS member, the response would improperly expose provider network information. To prevent this from happening, the HVPLS node MAY check to ensure that the return Ethernet MAC address is one of the MAC addresses that it has learned using the Ethernet VPLS node TLV described in Section 8.1. Note that this requires maintaining the MAC information of VPLS OAM entities during the entire operation of the H-VPLS. 13. IANA Considerations This document requires allocation of a new Target FEC Stack Sub-TLV type[RFC4379] that is managed by IANA. Hierarchical L2 Circuit FEC stack is defined in this document and the requested type is 17. A new TLV is defined for [RFC4379] as Error Code TLV to convey results of a VPLS OAM operation. The requested type is 4. Two new Multipath Types are defined to be used with Downstream Mapping TLV[RFC4379] for MAC based hash keys. The types requested are as follows: 10 a list of single MAC addresses is provided, any one of which will cause the hash to match this MP path. 11 a list of MAC address ranges is provided, any one of which will cause the hash to match this MP path Stokes, at al. Expires May 18, 2008 [Page 41] Internet-Draft OAM for VPLS November 2007 A LDP TLV type is defined in this document as VPLS Node TLV and the requested type is 0x0405. A control channel type is defined as L2VPN-VCCV to be used with [VCCV] procedures. The bitmask of requested control channel type is 0x10. 14. Acknowledgments The authors would like to thank the following people who have provided valuable comments and feedback on the topics discussed in this document: Mustapha Aissaoui Florin Balus Matthew Bocci Som Barua Andrew Dolganow Tiberiu Grigoriu Kireeti Kompella Marc Lasserre Ananth Nagarajan Ray Qiu This document was prepared using 2-Word-v2.0.template.dot. References Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC4761] Kompella, K., Rekhter, Y.," Virtual Private LAN Service (VPLS) Using BGP for Auto-Discovery and Signaling", RFC 4761, January 2007. [RFC4762] Lasserre, M., Kompella, V.,"Virtual Private LAN Service (VPLS) Using Label Distribution Protocol(LDP) Signaling", RFC 4762, January 2007. [RFC4667] Luo, W.,"Layer 2 Virtual Private Network (L2VPN) Extensions for Layer 2 Tunneling Protocol(L2TP)", RFC 4667, [RFC4379] Kompella, K. et al.,"Detecting Multi-Protocol Label Switched (MPLS) Data Plane Failures", RFC 4379, February 2006. Stokes, at al. Expires May 18, 2008 [Page 42] Internet-Draft OAM for VPLS November 2007 [VCCV] Nadeau, T. et al., "Pseudo Wire Virtual Circuit Connectivity Verification(VCCV)", draft-ietf-pwe3-vccv-13.txt (Work in progress). [RFC4447] Martini, L. et al.,"Pseudowire Setup and Maintenance Using the Label Distribution Protocol (LDP)", RFC 4447, April 2006. [RFC4448] Martini, L.,"Encapsulation Methods for Transport of Ethernet Frames Over IP/MPLS Networks", RFC 4448, April 2006. [RFC4446] Martini, L.,"IANA Allocations for pseudo Wire Edge to Edge Emulation(PWE3)", RFC 4446, April 2006. [RFC3036] Andersson, L. et al., "LDP Specification", RFC 3036, January 2001 Informative References [L2VPN-CFM] Stokes, O. et al., "OAM for L2 VPN Networks Using CFM and VCCV", draft-ietf-stokes-l2vpn-cfm-vccv-oam-00-0.txt (Work in progress). [L2VPN-OAM-REQ] Mohan, D. et al., "L2VPN OAM Requirements and Framework", draft-ietf-l2vpn-oam-req-frmk-08.txt (Work in progress). [MEF-OAM-REQ] MEF Service OAM Requirements & Framework, Technical Specification, Draft version 1.1. [802.1ag] Connectivity Fault Management, IEEE 802.1ag draft 6.1, work in progress. [ROSEN-MESH] Rosen, E.,"Detecting and Reacting to Failures of the Full Mesh in IPLS and VPLS", draft-rosen-l2vpn-mesh-failure-02.txt [Y.1731] ITU-T Recommendation Y.1731, OAM Functions and Mechanisms for Ethernet based Networks, May 2006. [RFC3209] Awduche, D. et al., "RSVP-TE: Extensions to RSVP for LSP Tunnels", RFC3209, December 2001. Stokes, at al. Expires May 18, 2008 [Page 43] Internet-Draft OAM for VPLS November 2007 [RFC4090] Pan, P. et al.,"Fast Reroute Extensions to RSVP-TE for LSP Tunnels", RFC4090, May 2005. [SEG-PW] Martini, L. et. al.,"Segmented Pseudo Wire", draft-ietf-pwe3-segmented-pw-05.txt, work in progress. Stokes, at al. Expires May 18, 2008 [Page 44] Internet-Draft OAM for VPLS November 2007 Author's Addresses Olen Stokes Extreme Networks PO Box 14129 RTP, NC 27709 USA Email: ostokes@extremenetworks.com Vach Kompella Alcatel-Lucent 701 E Middlefield Road, Mountain View, CA 94043 USA E-mail : vach.kompella@alcatel-lucent.com Pranjal Kumar Dutta Alcatel-Lucent 701 E Middlefield Road, Mountain View, CA 94043 USA E-mail : pdutta@alcatel-lucent.com Giles Heron Tellabs 24-28 Easton Street High Wycombe Bucks HP11 INT UK E-mail : giles.heron@tellabs.com Yetik Serbest AT&T Labs 9505 Arboretum Blvd. Austin, TX 78759 USA E-mail : yetik_serbest@labs.att.com Shane Amante Level 3 Communications, LLC 1025 Eldorado Blvd Broomfield, CO 80021 USA Email: shane@level3.net Stokes, at al. Expires May 18, 2008 [Page 45] Internet-Draft OAM for VPLS November 2007 Intellectual Property Statement The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Disclaimer of Validity This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Copyright Statement Copyright (C) The IETF Trust (2007). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. Acknowledgment Funding for the RFC Editor function is currently provided by the Internet Society. Stokes, at al. Expires May 18, 2008 [Page 46]