L2 VPN Working Group O. Stokes Internet Draft Extreme Networks Intended status: Informational S. Amamte Level 3 Communications P. Dutta Alcatel-Lucent Y. Serbest AT&T Expires: May 2008 November 5, 2007 OAM for L2 VPN Networks Using CFM and VCCV draft-stokes-l2vpn-cfm-vccv-oam-00.txt Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. This document may not be modified, and derivative works of it may not be created, except to publish it as an RFC and to translate it into languages other than English. This document may only be posted in an Internet-Draft. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html This Internet-Draft will expire on March 5, 2008. Stokes, et al. Expires May 5, 2008 [Page 1] Internet-Draft OAM for L2 VPN Networks Using CFM and VCCV November 2007 Copyright Notice Copyright (C) The IETF Trust (2007). Abstract The IEEE has defined P802.1ag Connectivity Fault Management (CFM) for managing bridged LAN networks. The IETF has defined L2 Virtual Private Networks (VPNs) that provide an emulated LAN service for such bridged networks. These L2 VPNs are created using a collection of one or more point-to-point pseudo wires (PWs). The IETF has also defined Virtual Circuit Connectivity Verification (VCCV) for managing these PWs. This memo discusses the ability of a combination of CFM and VCCV to meet the OAM requirements of a L2 VPN. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [RFC2119]. For the purposes of this document, the terms "HVPLS" and "HVPLS instance" are used to signify a generic VPLS instance that may or not be hierarchical in topology. See Section 1 for additional information. VCCV provides for the use of LSP-Ping [RFC4379] or ICMP Ping Connectivity Verification types (CV types). The applicable mode depends on the underlying PSN [VCCV]. For the purposes of this document, VCCV is described as using a generic "Ping" CV type. The type actually used will depend on the underlying PSN. See Section 2.3 for additional information. Table of Contents 1 . Introduction..........................................................3 1.1 . OAM Functionality................................................4 1.1.1 . Topology discovery..........................................4 1.1.2 . Service verification........................................4 1.1.3 . Fault detection.............................................5 1.1.4 . Connectivity verification...................................5 1.1.5 . Performance Monitoring......................................5 2 . CFM and VCCV applicability............................................5 2.1 . CFM function placement...........................................6 2.2 . VCCV function placement.........................................12 2.3 . VCCV Connectivity Verification types............................12 3 . CFM and VCCV Capabilities............................................12 3.1 . Topology discovery..............................................12 3.2 . Service verification............................................12 Stokes, et al. Expires May 5, 2008 [Page 2] Internet-Draft OAM for L2 VPN Networks Using CFM and VCCV November 2007 3.3 . Fault detection.................................................13 3.4 . Connectivity verification.......................................14 3.4.1 . Operational nodes..........................................14 3.4.2 . Operational peer sessions..................................15 3.4.3 . Operational tunnels........................................15 3.4.4 . Exchange data between HVPLS nodes..........................15 3.4.5 . Data exchange between customer devices.....................16 3.5 CFM and VCCV Summary..............................................17 4 . Additional CCM and VCCV considerations...............................18 5 . Network operation based on CFM and VCCV..............................19 5.1 . Combined CFM and VCCV solution..................................19 5.2 . CFM-only solution...............................................20 5.3 . CFM and VCCV combined with other protocols......................20 6 . Concerns.............................................................21 7 . Suggested practices..................................................22 8 . Security Considerations..............................................23 9 . IANA Considerations..................................................23 10 . Acknowledgments.....................................................23 11 . References..........................................................24 11.1 . Normative References...........................................24 11.2 . Informative References.........................................25 Author's Addresses.......................................................25 Intellectual Property Statement..........................................25 Disclaimer of Validity...................................................26 1. Introduction The IEEE has defined P802.1ag Connectivity Fault Management (CFM) for managing bridged LAN networks [P802.1ag]. Both service providers and their customers can use CFM to manage a bridged network. The IETF has defined L2 Virtual Private Networks (VPNs) that provide an emulated LAN service for such bridged networks. These L2 VPNs are created using a collection of one or more point-to-point pseudo wires (PWs) [RFC3985]. These PWs can be configured in a flat, full-mesh topology to provide a Virtual Private LAN Service (VPLS) [RFC4761][RFC4762]. This VPLS core configuration can be augmented with additional non-meshed spoke nodes to provide a Hierarchical VPLS (HVPLS) service [RFC4762]. PWs can also be used to connect these spoke nodes to the VPLS core nodes. Any Operation, Administration, and Maintenance (OAM) standard for L2 VPNs must support both flat VPLS and hierarchical HVPLS configurations. In addition, any L2 VPN OAM solution must support all standardized methods by which the L2 VPN can be signaled. For example, L2 VPNs signaled using BGP [RFC4761] and L2 VPNs signaled using LDP [RFC4762] must both be supported. Also, all standardized tunnel protocols must be supported. Stokes, et al. Expires May 5, 2008 [Page 3] Internet-Draft OAM for L2 VPN Networks Using CFM and VCCV November 2007 For the purposes of this document, the terms "HVPLS" and "HVPLS instance" are used to signify a generic VPLS instance that may or not be hierarchical in topology. The IETF has defined Virtual Circuit Connectivity Verification (VCCV) for monitoring individual PWs [VCCV]. VCCV does not monitor the interactions among PWs. This memo discusses the ability of a combination of CFM and VCCV to meet the OAM requirements of a L2 VPN. This memo does not specifically address the OAM requirements of the non-L2 VPN portions of a provider's network. 1.1. OAM Functionality Providing OAM services for a HVPLS network requires at least the following functions: 1. Topology discovery 2. Service verification 3. Fault detection 4. Connectivity verification 5. Performance monitoring With the exception of fault detection, all of the above functions were discussed in [TestHVPLS]. 1.1.1. Topology discovery Topology discovery requires determining the current members of a HVPLS network. In [TestHVPLS], topology discovery concentrated on the ability to create a database containing both the IP and MAC addresses of all of the nodes participating in the HVPLS. This database is often required as one protocol may identify a node by its IP address while another protocol may identify the node by its MAC address. For example, BGP knows a PE [RFC4761] node, and LDP knows a PE-rs [RFC4762] node, by its IP address. CFM knows each of these nodes by its MAC address. This database allows network operators to cross-reference between protocol addresses. 1.1.2. Service verification Service verification determines if the service aware nodes have been consistently configured and any associated hardware has been successfully updated [TestHVPLS]. This verification must be such that feedback is available in a timely manner following network configuration. Stokes, et al. Expires May 5, 2008 [Page 4] Internet-Draft OAM for L2 VPN Networks Using CFM and VCCV November 2007 1.1.3. Fault detection Although a service may have been configured and verified, there remains a possibility that the service will be interrupted at a later time. Network operators need notification of such interruptions as quickly as possible to minimize the impact to their customers. While lower layer protocols may identify some problems, it is impossible to completely guarantee the continued operation of a HVPLS network without some sort of detection operating at the HVPLS level. Fault detection for a HVPLS network must identify data plane problems that cannot be detected by the HVPLS control plane or will not be detected by the HVPLS control plane in a timely manner. As such, it is expected that some type of periodic health check message will be required. 1.1.4. Connectivity verification There are five logical steps to verifying the connectivity of a HVPLS network [TestHVPLS]: 1. All nodes are operational 2. All peer sessions are operational 3. All tunnels are transporting data packets correctly 4. Data packets can be exchanged between any two nodes 5. Actual customer devices can communicate with devices at any site The above steps are representative of the steps that would be taken by a network operator while trying to identify the cause of a HVPLS network problem. 1.1.5. Performance Monitoring Performance monitoring, such as determining round-trip times, jitter, average packet throughput, etc. [TestHVPLS] may be described in a future version. In the meantime, service providers are already using external measurement devices to perform some of this function. 2. CFM and VCCV applicability The L2VPN work group has taken the direction that testing of the "bridge" functionality in a HVPLS node should be done using bridge OAM standards. Therefore, CFM should be used to test the bridge module component of the VPLS reference model described in [RFC4664]. In addition, VCCV is to be used to test the individual PWs shown in the [RFC4664] reference model. Stokes, et al. Expires May 5, 2008 [Page 5] Internet-Draft OAM for L2 VPN Networks Using CFM and VCCV November 2007 2.1. CFM function placement While a HVPLS node may participate in a CFM Maintenance Domain (MD) at the customer level, complete testing of the HVPLS portion of the network requires that a CFM Maintenance Area (MA) be created at the provider level with the HVPLS nodes as Maintenance Points (MPs). A Maintenance association End Point (MEP) is created for any HVPLS node with a customer attachment circuit (AC). A MEP or a Maintenance domain Intermediate Point (MIP) can be created for any HVPLS node without a local AC. An example of such a configuration using the reference model in [RFC4664] is shown in Figure 1. |-----Routed Backbone-----| | (P Routers) |PSN Tunnels, Emulated LAN | |Pseudowires ....................................................................... . | | . . |---------------------|----| |--------|-----------------| . . | --------------------|--- | | -------|---------------- | . . | VPLS Forwarder | | VPLS Forwarder | . . | ----------|------------- | | ----------|------------- | . ..|.................................................................... | | Emulated LAN | | | Emulated LAN | | | Interface | VPLS-PEs | | Interface | | | | <----> | | | | | | | | | | ----------|------------ | | ----------|------------ | | | MEP X MDLevel 3 | | | | MEP X MDLevel 3 | | | | | | | | | | | | | | MIP X MDLevel 4 | | | | MIP X MDLevel 4 | | | | | | | | | | | | | | | | | | | | | | Bridge | | | | Bridge | | | | | | | | | | | | | | | | | | | | | | | | | | X MEPs X MDLvl 4 X | | | | X MEPs X MDLvl 4 X | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | X MIPs X MDLvl 5 X | | | | X MIPs X MDLvl 5 X | | | | | | | | | | | | | | | | | --|-------|---------|-- | | --|-------|---------|-- | |---|-------|---------|----| |---|-------|---------|----| | | | | | | | | Access | | | Access | | | Networks| | | Networks| | | | | | | | | | | | | CE devices CE devices Figure 1: RFC-4664 reference model with CFM Stokes, et al. Expires May 5, 2008 [Page 6] Internet-Draft OAM for L2 VPN Networks Using CFM and VCCV November 2007 Note that the bridge module component has a single emulated LAN interface. Section 2.2 from [RFC4664] states: 'This framework specifies that each "bridge module" have a single "Emulated LAN interface".' CFM packets are not processed by the emulated LAN components. For example, VPLS forwarders do not recognize CFM packets. They forward CFM packets based on their destination MAC addresses. The MD Levels chosen in Figure 1 are based on Section L.3, "MD Level allocation alternative," in [P802.1ag]. MD Level 5 is a customer MD Level. MD Levels 4 and 3 are service provider MD Levels. The MIPs shown at MD Level 5 are present if the service provider participates in the customer's MA. The MEPs at MD Level 4 are Up MEPs whose MD includes the bridge module components. The MIP at MD Level 5 and the MEP at MD Level 4 are normally at the point where the customer attaches to the provider's network. The MEPs at MD Level 3 are Down MEPs whose MD is the Emulated LAN portion of the reference model. To apply CFM to a HVPLS network, the example network from Figure 3 in [RFC4762] is used. A modified version of that example is shown Figure 2. Stokes, et al. Expires May 5, 2008 [Page 7] Internet-Draft OAM for L2 VPN Networks Using CFM and VCCV November 2007 PE2-rs +--------+ | | | -- | | / \ | CE-1 | \S / | \ | -- | \ +--------+ \ MTU1-s PE1-rs / | +--------+ +--------+ / | | | | | / | | -- | PW-1 | -- |---/PW-12 | | / \--|- - - - | / \ | | PW-23 | \S / | | \S / | | | -- | | -- |---\PW-13 | +--------+ +--------+ \ | \ | MTU3-s +--------+ +--------+ | | | | | -- | PW-3 | -- | | / \ |- - - - | / \--| | \S / | | \S / | | -- | | -- | +--------+ +--------+ PE3-rs \ \ \ CE-3 Figure 2: Example HVPLS network In Figure 2, the three PE-rs nodes have no local attachments. Therefore, these nodes do not require a bridge module component for HVPLS operation. However, as is discussed in Section 3.4, there are potential benefits to having a CFM presence in these nodes. Figure 3 shows CFM applied to the example network of Figure 2. Note that bridge module components are included in the PE-rs nodes even though they are not strictly required. Stokes, et al. Expires May 5, 2008 [Page 8] Internet-Draft OAM for L2 VPN Networks Using CFM and VCCV November 2007 |--- PW-1 ---| |- PW-13 -| |--- PW-3 ---| Emulated | | | | | | LAN | | | | | | ..................................................................... . | | | | | | . . |-------|-----| |---|-----|---| |---|-----|---| |-----|-------| . . | ----------- | | ----------- | | ----------- | | ----------- | . . | VPLS | | VPLS | | VPLS | | VPLS | . . | Forwarder | | Forwarder | | Forwarder | | Forwarder | . . | -----|----- | | -----|----- | | ----------- | | -----|----- | . ..|.................................................|................ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | -----|----- | | -----|----- | | -----|----- | | -----|----- | | |MEP X MD | | | |MEP X MD | | | |MEP X MD | | | |MEP X MD | | | | | LVL| | | | | Lvl| | | | | Lvl| | | | | LVL| | | | | 3 | | | | | 3 | | | | | 3 | | | | | 3 | | | | | | | | | | | | | | | | | | | | | | | |MIP X MD | | | | | | | | | | | |MIP X MD | | | | | LVL| | | | | | | | | | | | | LVL| | | | | 4 | | | | | | | | | | | | | 4 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bridge | | | | Bridge | | | | Bridge | | | | Bridge | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |MEP X MD | | | | | | | | | | | |MEP X MD | | | | | LVL| | | | | | | | | | | | | LVL| | | | | 4 | | | | | | | | | | | | | 4 | | | | | | | | |---------| | | |---------| | | | | | | | | | | | |-------------| |-------------| | | | | | | |MIP X MD | | | |MIP X MD | | | | | LVL| | PE1-rs PE3-rs | | | LVL| | | | | 5 | | | | | 5 | | | | | | | | | | | | | -----|----- | | -----|----- | |------|------| |------|------| | | | | CE-1 CE-4 MTU1-s MTU3-s Figure 3: HVPLS network with CFM In the network shown in Figure 3, it is important to note that packets transmitted from MTU1-s to MTU2-s do not pass through the bridge module components of PE1-rs or PE3-rs. The network shown in Figure 2, and the associated function placement shown in Figure 3, assume that the customer attaches to the provider's network at the edge of the L2 VPN. However, a provider may use Ethernet access networks such that the Stokes, et al. Expires May 5, 2008 [Page 9] Internet-Draft OAM for L2 VPN Networks Using CFM and VCCV November 2007 customer attachment point to the provider network is remote from the L2 VPN. For example, a provider may attach a customer as shown in Figure 4. Provider Bridge B1 MTU-s/PE-rs +--------+ +--------+ | | Ethernet | | | -- | Access Network | -- | PW(s) | / \--|- - - - - - - - - -| / \ | - - - - - | \S / | | \S / | | -- | | -- | +--------+ +--------+ / / / CE-1 Figure 4: Remote customer attachment In this case, the provider would normally utilize CFM to the edge of the provider network. This would lead to the CFM function placement shown below in Figure 5. The customer-level MIP and the provider-level MEP are created on provider bridge B1 in order to maximize CFM coverage. Stokes, et al. Expires May 5, 2008 [Page 10] Internet-Draft OAM for L2 VPN Networks Using CFM and VCCV November 2007 |-- PW(s) --- Emulated | LAN | ........................ . | . |-------|-----| . | ----------- | . | VPLS | . | Forwarder | . | -----|----- | ..|..................... | | | | -----|----- | | |MEP X MD | | | | | LVL| | | | | 3 | | | | | | | | |MIP X MD | | | | | LVL| | |-----------------| | | | 4 | | | --------------- | | | | | | | | | | | | | | | | Bridge | | | | Bridge | | | | | | | | | | | | | | | | | | | | | | |MEP X | | | | | | | | | |MD | | | | | | | | | | |LVL | | | | | | | | | | | 4 | | | | | | | | | | | | | | | | | | | | | |MIP X | | | | | | | | | |MD | | | | | | | | | | |LVL | | | | | | | | | | | 5 | | | | | | | | | | -----|---|----- | | -----|----- | |------|---|------| |------|------| | | | | |----------------| CE-1 B1 MTU/PE-rs Figure 5: CFM function placement with Ethernet access networks Stokes, et al. Expires May 5, 2008 [Page 11] Internet-Draft OAM for L2 VPN Networks Using CFM and VCCV November 2007 2.2. VCCV function placement VCCV is used to verify the individual PWs that are used in the emulated LAN of the [RFC4664] reference model. As such, VCCV can be used to verify the connections between the VPLS forwarders that comprise the emulated LAN. 2.3. VCCV Connectivity Verification types VCCV provides for the use of LSP-Ping [RFC4379] or ICMP Ping Connectivity Verification types (CV types). The applicable mode depends on the underlying PSN [VCCV]. For the purposes of this document, VCCV is described as using a generic "Ping" CV type. The type actually used will depend on the underlying PSN. Note that VCCV also provides for the use of Bidirectional Forwarding Detection (BFD) [BFD] as a CV type. 3. CFM and VCCV Capabilities The OAM discussions in the following sections are based on the function placements described in Section 2. 3.1. Topology discovery Each MEP in the network sends Continuity Check Messages (CCMs) on a periodic basis. By including the optional Sender ID TLV in the messages, each MEP and MIP can learn the IP addresses of the MEPs in the network to which it has connectivity. If desired, this database can be compared to a configured database of expected network nodes or to a database obtained via an autodiscovery method such as BGP [RFC4761]. In the network in Figure 3, nodes PE1-rs and PE3-rs are shown with a MEP and an associated bridge module component even though they are not required for HVPLS operation. Without these MEPs and bridge modules, these nodes would not appear in the CCM topology database of MTU1-s or MTU3-s. MIPs do not send CCM messages. Therefore, a database built using CCM messages would not include MIPs. Should any HVPLS node have only a MIP entity, it would not appear in a topology database created using CCMs. VCCV is not applicable to topology discovery. 3.2. Service verification As discussed in Section 3.1, if a new node with a MEP is added to a HVPLS network, other nodes will be able to verify connectivity to the new MEP by receipt of a CCM from the new node. Depending on the timings between the establishment of all of Stokes, et al. Expires May 5, 2008 [Page 12] Internet-Draft OAM for L2 VPN Networks Using CFM and VCCV November 2007 the new PWs and the sending of the new node's initial CCM, the other nodes may recognize the new MEP quickly following its configuration. However, in a stable environment, the new node might require a maximum of 35 minutes to ensure that it received a CCM message from all of its MEP peers. In sizeable networks, the CCM transmission interval may be set to a large value to minimize system loading. For example, the CFM defined intervals include 10 seconds, 1 minute, and 10 minutes [P802.1ag]. In addition, a node should wait multiple transmission intervals before assuming that it does not have connectivity to another MEP peer. Thus, using CCM receipt to ensure that a new node with a MEP is correctly included in the network may require waiting up to 35 minutes. As MIPs do not send CCM messages, only the nodes that have a MEP at a certain level within a given MA will participate in CCM-based service verification. In addition to confirming the receipt of CCMs from every expected MEP peer, MEPs can also verify that that they are not receiving CCMs from unexpected MEPs or CCMs for unexpected MAs. MEPs can also check for loops back to themselves by ensuring that they do not receive their own CCMs. A new HVPLS node that runs VCCV can verify quickly that the PWs to its direct peers are operational. For example, when node MTU1-s is added to the network in Figure 2, it can quickly verify that the PW to PE1-rs is operational. It will have to depend on the receipt of CCMs from other nodes to verify the operation of the entire emulated LAN. CCM messages do not support the CFM Data TLV. As such, using only CCM does not verify that the MTU settings of all of the network's components are consistent. VCCV using Ping can check MTU settings on an individual PW basis. VCCV using BFD does not have this capability. 3.3. Fault detection Once a new HVPLS node has been verified as operational, network administrators need the continued operation of the network to be monitored. This requires that periodic verification be performed by maintenance entities. CCM messages are transmitted by MEPs on a periodic basis. A fault is declared when multiple consecutive messages are not received. The period for CCM message transmission can be from 3.3 milliseconds to 10 minutes [P802.1ag]. There are obvious trade-offs involved between improved failure detection time and increased system utilization. CCM-based failure detection monitors the entire HVPLS network including the emulated LAN. Each node maintains a CCM database of all of the MEPs in each HVPLS instance to track received CCM messages. Each MEP knows when it has lost Stokes, et al. Expires May 5, 2008 [Page 13] Internet-Draft OAM for L2 VPN Networks Using CFM and VCCV November 2007 connectivity to any other MEP in the HVPLS. In addition, CCMs have a Remote Defect Indicator (RDI) flag that indicates whether or not the sending MEP is receiving CCMs from all of its expected MEP peers. The receipt of a CCM with the RDI flag set indicates that there is a missing connection somewhere in the network. For example, a partial mesh in the HVPLS core would result in RDI flags being set in CCMs. VCCV provides failure detection for individual PWs. With VCCV using BFD, connectivity messages are transmitted on a periodic basis on each VCCV session (on each PW). As BFD is designed to be a "low-overhead" protocol [BFD], the per- message processing overhead for VCCV with BFD should be less than for CCMs. It is therefore possible that the failure detection times for VCCV using BFD may be less than for CFM using CCMs. Also, since the VCCV database tracks only local PWs, it should be smaller than a CCM database. However, VCCV does not detect failures in the VPLS forwarder part of the emulated LAN. VCCV using Ping can also be used to provide failure detection by periodically sending ping packets. While not specified in either [P082.1ag] or [VCCV], it is possible that an implementation could exchange information between the two protocols. Such interactions may be discussed in a future version of this document. 3.4. Connectivity verification Connectivity verification in this context deals mainly with problem determination. It provides tools that a network operator can use to determine the cause of a problem. 3.4.1. Operational nodes MEPs and MIPs can create a database with information concerning the receipt of CCMs from every known MEP. This database provides operators with a readily available confirmation of the operational state of network nodes. As discussed in Section 3.1 and Section 3.2, CCMs are transmitted on a periodic basis. Therefore, use of a CCM database as an indication that a remote node is operational has an amount of uncertainty that is proportional to the CCM transmit interval. If VCCV is being used, VCCV provides verification that all peer nodes are operational. This information is readily available to network operators. As with CCMs, there is an amount of uncertainty that is proportional to the interval at which BFD or Ping messages are being sent. Stokes, et al. Expires May 5, 2008 [Page 14] Internet-Draft OAM for L2 VPN Networks Using CFM and VCCV November 2007 3.4.2. Operational peer sessions As the flooding of CCMs is subjected to the split-horizon operation described in [RFC4762], receipt of CCMs from all known MEPs can only occur when all peer sessions are operational. VCCV by definition tests peer sessions. If all VCCV sessions are operational, then all peer sessions are operational. Both CCM and VCCV verification of operational peer sessions are subject to the same timing windows discussed in Section 3.4.1. 3.4.3. Operational tunnels As both CCM and VCCV messages are encapsulated using the same tunnels that are used for normal data transmission, the node and peer session verifications discussed above provide confirmation about operational tunnels. Note that many tunneling protocols have their own OAM capabilities that operators can invoke based on the results of HVPLS and/or PW tests. For example, MPLS LSPs can be verified using LSP-Ping. In addition, BFD can be used to monitor operation between any two forwarding engines. 3.4.4. Exchange data between HVPLS nodes CCMs are sent to a group MAC address. CCMs are processed and forwarded in the MEP/MIP control plane. As such, CCMs check flood lists, but verification is based on the control plane's ability to check hardware entries. A network operator can initiate Loopback Messages (LBMs) and Linktrace Messages (LTMs) [P802.1ag] to any MEP or MIP in the HVPLS. LBMs check unicast forwarding. LBMs use a unicast destination MAC address and are processed by the normal data plane. MIPs flood LBMs for unknown unicast MAC addresses. This requires that the data planes have the filters necessary to insure that LBMs are not forwarded outside of the MD. LTMs check the path to a MEP/MIP. LTM frames use a group destination MAC address and verify the path by the control plane's checking of known hardware entries. LTMs are not flooded by MEPs/MIPs and therefore do not check the flooding of unknown unicast packets. The destination of a LTM must be the known MAC address of a MEP/MIP. Every MEP/MIP involved must have already learned that MAC address. Note that periodically transmitting CCMs insures that the MAC addresses of MEPs are known. As is shown in Figure 3, packets forwarded by a VPLS forwarder are not processed by the bridge module component. For example, consider a LTM packet sent from Stokes, et al. Expires May 5, 2008 [Page 15] Internet-Draft OAM for L2 VPN Networks Using CFM and VCCV November 2007 MTU1-s with a destination of MTU3-s. The destination MAC address of this packet will be a group address as defined in Table 8-10, "Linktrace Message Group Destination MAC Addresses," of [P802.1ag]. This will be an unknown MAC address to the VPLS forwarders of PE1-rs and PE3-rs. As such, the VPLS forwarders will flood the packet. The bridge modules in PE1-rs and PE3-rs will not process the packet prior to it being flooded to MTU3-s. In addition, the bridge module components of PE1-rs and PE3-rs will only know the destination as reachable over the same emulated LAN interface from which the LTM was received. As a result, MTU1-s will receive a LTR only from MTU3-s. An operator initiating a LTM at MTU1-s would not see PE1-rs or PE3-rs in the path to MTU3-s. Note that while PE1-rs and PE3-rs do not respond to LTMs destined for MTU3-s in the above discussion, they will respond to LBMs and LTMs addressed directly to them. VCCV using Ping can be used to provide an on-demand verification of data exchange between peers. Note that VCCV requires a "control channel" [VCCV] which includes the normal encapsulation plus either: o Use of a PW control word o Addition of a router alert label o Setting of the PW Label TTL = 1 LBMs and VCCV Pings can include variable size data fields to verify operation of MTU sized packets. LTMs do not support the CFM Data TLV. BFD packets used for VCCV do not support variable packet sizes. 3.4.5. Data exchange between customer devices A network operator can initiate LTMs to any MAC address, including customer MAC addresses. This provides verification of the path taken within the HVPLS network towards the customer MAC. It does not provide verification of the actual forwarding of the packet at the egress from the HVPLS network. LTM frames use a group destination MAC address and verify the path by the control plane's checking of known hardware entries. LTMs do not support the CFM Data TLV so this verification cannot be performed using MTU-sized packets. Note that LTMs require that every MEP/MIP involved has already learned the customer MAC address. This can sometimes place additional burden on network operators to insure that the customer MAC has been learned at each MEP/MIP. VCCV is not applicable to verification using customer MAC addresses. Stokes, et al. Expires May 5, 2008 [Page 16] Internet-Draft OAM for L2 VPN Networks Using CFM and VCCV November 2007 3.5 CFM and VCCV Summary A brief summary of the capabilities of CFM and VCCV as discussed above are shown below in Table 2 and Table 2. +-----------------------+----------------------------+-------------------------+ | | CFM | VCCV | +-----------------------+----------------------------+-------------------------+ | Topology Discovery | Discover MEPs using receipt| Not applicable. | | | of CCMs. If Sender ID TLV | | | | included, can build | | | | database of IP and MAC | | | | addresses. | | +-----------------------+----------------------------+-------------------------+ | Service Verification | Verify MEPS using receipt | Verify peers. | | | of CCMs. Time required to | | | | verify proportional to CCM | | | | transmit interval. | | +-----------------------+----------------------------+-------------------------+ | Fault Detection | Monitor entire HVPLS using | Monitor individual PWs. | | | CCMs. Detection time | Detection time | | | proportional to CCM | proportional to | | | transmit interval. | transmission interval. | | | | VCCV using BFD allows | | | | use of "low overhead" | | | | protocol. | +-----------------------+----------------------------+-------------------------+ Table 1: Summary of CFM and VCCV Capabilities - Part 1 Stokes, et al. Expires May 5, 2008 [Page 17] Internet-Draft OAM for L2 VPN Networks Using CFM and VCCV November 2007 +-----------------------+----------------------------+-------------------------+ | | CFM | VCCV | +-----------------------+----------------------------+-------------------------+ | Connectivity | | | | Verification | | | +-----------------------+----------------------------+-------------------------+ | Operational Nodes | Maintain database based on | If all VCCV sessions are| | | receipt of CCMs from known | operational, then all | | | MEPs. Uncertainty | nodes are operational. | | | proportional to CCM | | | | transmit interval. | | +-----------------------+----------------------------+-------------------------+ | Operational Peer | Same as above. | Same as above. | | Sessions | | | +-----------------------+----------------------------+-------------------------+ | Operational Tunnels| Same as above. | Same as above. | +-----------------------+----------------------------+-------------------------+ | Exchange Data | Operator can initiate LBM | Operator could force | | Between HVPLS Nodes| and LTM messages to any | on-demand verification | | | other MEP/MIP. | of an individual PW if | | | | VCCV is using Ping. | +-----------------------+----------------------------+-------------------------+ | Exchange Data | Operator can initiate LTM | Not applicable. | | Between Customer | to any MAC address. | | | Devices | Requires every node has | | | | already learned the | | | | customer MAC address. | | +-----------------------+----------------------------+-------------------------+ Table 2: Summary of CFM and VCCV capabilities - Part 2 4. Additional CCM and VCCV considerations When deployed throughout an entire HVPLS network, the scalability of any OAM solution must be considered. The target MAC address for a LBM is used as the destination MAC in the Ethernet header. The target MAC address for a LTM is contained in a destination MAC field that is processed in the control plane. CFM is designed with the expectation that target MAC addresses are already learned, as flooding of LTMs is not permitted. Therefore, it is expected that the forwarding tables for each HVPLS instance contain an entry for every MEP in the MA. These MAC addresses may not be required for normal HVPLS operation. As this is on a per-instance basis, this may result in a significant increase in table sizes. When a MP at the provider level is defined on a HVPLS node, hardware filters are required to prevent provider level CFM packets from being forwarded outside of the provider network. This is on a per-instance basis. It is reasonable to expect Stokes, et al. Expires May 5, 2008 [Page 18] Internet-Draft OAM for L2 VPN Networks Using CFM and VCCV November 2007 that similar filters would be required with any other OAM solution. In addition, the provider nodes may also be participating in customer MAs. If so, then the node implementation will already be capable of installing CFM type filters. VCCV also requires filters to insure that OAM packets are not forwarded beyond the VCCV session. These filters must trap the control channel chosen from the list in Section 3.4.4. If both CFM and VCCV are used, then both sets of hardware filters will be required. CFM system load is driven by the transmit interval and the number of MEPs in the MA. If CFM is not intended for rapid fault detection, the transmit interval can be set to as much as 10 minutes, which should result in minimal system load. If CFM is intended for rapid fault detection, the transmit interval can be set to as little as 3.3 milliseconds, potentially resulting in an extremely large system load. VCCV system load is driven by the negotiated transmission interval and the number of VCCV sessions (number of PWs). Note that it is possible that different VCCV sessions may negotiate to different transmission times. Setting low transmission intervals on a large number of PWs will potentially result in an extremely large system load. 5. Network operation based on CFM and VCCV Given the capabilities discussed in Section 3, there are multiple approaches that can be used to deploy OAM in a HVPLS network. 5.1. Combined CFM and VCCV solution In this environment, CFM is used for topology discovery, service verification, and connectivity verification. VCCV is used for fault detection. When a HVPLS instance is created or updated, CCM messages with Sender ID TLVs provide the initial verification. LBM messages with Data TLVs are used to verify that MTU- sized packets can be sent to any peer in the event of troubleshooting. LTM messages can also be used for additional troubleshooting. VCCV sessions are established for every PW for fault detection. This network configuration has several drawbacks. There is no "on-demand" system- wide service verification. To improve system-wide service verification time when using CCM messages, the transmit intervals need to be set small enough to force CCM messages in the timeframe expected by an operator. Assuming that interval is on the order of seconds, this would require sending CCM messages at a far greater rate than is required to maintain the MAC information needed for problem determination. The system burden of VCCV sessions on every PW would be driven by the expected fault detection interval. Stokes, et al. Expires May 5, 2008 [Page 19] Internet-Draft OAM for L2 VPN Networks Using CFM and VCCV November 2007 The LTM exposures discussed in Section 3.4.4 apply to this solution. In a hierarchical HVPLS environment, PE-rs nodes in the data path will not respond to LTMs that are not addressed to them. 5.2. CFM-only solution This environment is the same as described above in Section 5.1 except that CCM is used for fault detection instead of VCCV. The system burden of CCMs at an aggressive rate is potentially greater than the system burden of VCCV at a similar rate. This exposure is due to the fact that CCMs are received from every MEP in the HVPLS network instead of just from peers. This distinction is obviously more important in a hierarchical HVPLS environment than in a flat VPLS environment. Also, when VCCV is using BFD, the basic protocol is designed to be a "low overhead" protocol. This environment has the advantage that it eliminates one periodic hello-type message and one set of hardware filters. This environment also potentially reduces the concerns about system-wide service verification time as discussed in Section 5.1 by reducing the time required for a new node to receive CCM messages from every other node. The same LTM exposures discussed in Section 3.4.4 apply to this solution. In a hierarchical HVPLS environment, PE-rs nodes in the data path will not respond to LTMs that are not addressed to them. 5.3. CFM and VCCV combined with other protocols There are certain to be scaling limits to how rapidly fault detection can be run on every HVPLS instance or on every PW. In addition, it may not be prudent to attempt to detect HVPLS or PW failures at a rate faster than the underlying tunneling protocol can detect and recover from failures. One approach is that rapid fault detection is best handled at the tunneling protocol level and that fault detection at the PW and/or HVPLS level is on a more extended timeframe. For example, Section 5.2.2 of [OAM_MSG] states: "For PWE3 over an MPLS PSN and an MPLS-IP PSN, it is effective to run a defect detection mechanism over a PSN Tunnel frequently and run one over every individual PW within that PSN Tunnel less frequently. However in case the PSN traffic is distributed over Equal Cost Multi Paths (ECMP), it may be difficult to guarantee that PSN OAM messages follow the same path as a specific PW. A Service Provider might therefore decide to focus on defect detection over PWs." In an environment where ECMP is not a problem, rapid fault detection could be achieved by closely monitoring the underlying tunnels. HVPLS instances, and their associated PWs, could be verified when they are created or updated. Fault detection at the HVPLS and/or PW level would then become a matter of checking for Stokes, et al. Expires May 5, 2008 [Page 20] Internet-Draft OAM for L2 VPN Networks Using CFM and VCCV November 2007 corruption. Note that one possible catalyst for corruption is the recovery action taken when a tunnel failure is detected. Therefore, it would be desirable to have some sort of failure detection or verification that could be run on the affected PWs and/or HVPLS instances soon after a tunnel recovery action. With VCCV using Ping, it is possible to check the affected PWs immediately following the recovery action. Also, it is possible to check only the PWs affected by a tunnel recovery action. Therefore, fault detection could be done using a combination of protocols. Rapid fault detection could be done on tunnels using tunnel OAM protocols (for example, BFD on a LSP). This would significantly reduce the system overhead compared to performing rapid fault detection on every HVPLS instance or on every PW. Background, or less frequent, fault detection could be done on a complete HVPLS instance using CCM. In this case, CCM could potentially use a large transmit interval. Service verification could be done on a per-PW basis by VCCV using Ping. This could be automatically done immediately following configuration changes or tunnel recovery actions. Topology discovery and system-wide service verification could be done using CCM. The CCM transmit interval chosen would determine how rapidly this information is available. This trade-off between improved verification response and increased system load is a drawback of this combination of protocols. Connectivity verification could be done using CFM LBMs and LTMs. The same LTM exposures discussed in Section 3.4.4 still apply to this solution. In a hierarchical HVPLS environment, PE-rs nodes in the data path will not respond to LTMs that are not addressed to them. 6. Concerns While the combination of CFM and VCCV provides significant information for a network operator, the combination does not provide a complete OAM solution for a L2 VPN environment. When applied to the [RFC4664] reference model of a L2 VPN node, CFM does not recognize the components of the emulated LAN. As such, CFM does not provide verification or problem determination for VPLS forwarders or PWs. VCCV provides coverage for the PWs in the emulated LAN, but does not address the VPLS forwarders. This leaves a significant portion of the emulated LAN completely opaque from a standardized OAM perspective. Some of the resulting exposures include: Stokes, et al. Expires May 5, 2008 [Page 21] Internet-Draft OAM for L2 VPN Networks Using CFM and VCCV November 2007 o There is no OAM protocol to address any problems in the VPLS forwarders. For example, there is no OAM protocol to troubleshoot incorrect VPLS forwarder flooding. o There is no OAM protocol to address the interactions of VPLS forwarders with their associated PWs. For example, there is no OAM protocol to troubleshoot incorrect mappings of MAC addresses to PWs. o Receipt of a CCM message with the Remote Defection Indication (RDI) bit may be the result of a missing PW. However, there is no information about the location of the missing PW(s). Note that simply trying to apply CFM inside the emulated LAN will not result in a complete OAM solution for a L2 VPN emulated LAN. It does not appear that CFM was designed to cover the internal mechanisms of a generic emulated LAN. The CFM standard does not provide guidance for applying it inside of an emulated LAN. The formats used by CFM are specifically designed to carry LAN/bridge information. For example, the Reply Egress TLV in a Linktrace Reply (LTR) message contains fields with MAC Address and Port ID information [P802.1ag]. In a L2 VPN emulated LAN environment, the information required would include the PW label, the PW peer IP address, and the tunnel ID. Between CFM and VCCV, only CFM CCMs provide verification on a complete HVPLS instance basis. CCMs are sent based only on a periodic transmit timer. There is no ability to request an on-demand CCM transmission for verification purposes. Therefore, verifying the complete network following a configuration change or a network event using CCMs requires waiting a complete CCM transmit interval. When CCMs are being used for rapid fault detection, this transmit interval is potentially small. However, when CCMs are not being used for rapid fault detection, this transmit interval would normally be increased to reduce system loading. In this case, the interval could easily become longer than the expected time for system verification to complete. The fact that LTM messages are only propagated in the direction of known MAC addresses may cause confusion on the part of network operators when the destination is a customer MAC. It may also be difficult for an operator to force a customer MAC to be re-learned in order to continue a troubleshooting effort. 7. Suggested practices CFM and VCCV should be used in a L2 VPN environment to perform the functions that they do well in that environment. MIPs should be installed at the customer attachment points as shown in Figure 1 and Figure 3 if a provider decides to participate in a customer's MA. Stokes, et al. Expires May 5, 2008 [Page 22] Internet-Draft OAM for L2 VPN Networks Using CFM and VCCV November 2007 Installing Up MEPs at the customer attachment points as shown in Figure 1 and Figure 3 provides the most complete coverage of a HVPLS network. The associated MD includes all of the bridge module and emulated LAN components. In this MD, CCMs should be used to provide continuity checking on the entire L2 VPN. The CCM transmit interval should be set as desired based on the existence of other monitoring protocols as well as system processing capacity. LBMs and LTMs can be used to troubleshoot a domain that includes all of the L2 VPN components. Given the exposures discussed in Section 6, another OAM protocol specifically designed for the emulated LAN function shown in Figure 1 and Figure 3 may be required. The need to install Down MEPs at the emulated LAN interface will be based on the capabilities and expectations of that emulated LAN OAM protocol. It is anticipated that any OAM protocol that addresses the emulated LAN components will use make use of VCCV. The role of VCCV in monitoring PWs (and thus their underlying tunnels) will be based on the existence of other monitoring protocols as well as system processing capacity. For CFM and VCCV to be used successfully on a widespread basis in L2 VPN environments, ease of use will be an important requirement. The creation of a MD at the customer attachment points should require minimal configuration. There should be a default MD Level (for example, 4) and a default algorithm to create any required IDs. The use of LBMs and LTMs should be as close as possible to the well-known use of IP ping and traceroute. 8. Security Considerations As no new protocols are defined, any security considerations are based on the security considerations of the protocols that are referenced in this document. It is important to note that any L2 VPN OAM solution should insure that information about a service provider's network is not leaked to the customer network. 9. IANA Considerations This document has no actions for IANA. 10. Acknowledgments The authors would like to thank the following people who have provided comments and feedback on the topics discussed in this document: Stokes, et al. Expires May 5, 2008 [Page 23] Internet-Draft OAM for L2 VPN Networks Using CFM and VCCV November 2007 Florin Balus Som Barua Matthew Bocci Andrew Dolganow Tiberiu Grigoriu Giles Heron Vach Kompella Kireeti Kompella Marc Lasserre Ananth Nagarajan Ray Qiu This document was prepared using 2-Word-v2.0.template.dot. 11. References 11.1. Normative References [BFD] Katz, D. and Ward, D., "Bidirectional Forwarding Detection", draft-ietf- bfd-base-06.txt, March 2007. [P802.1ag] IEEE Draft P802.1ag/D8 "Virtual Bridge Local Area Networks - Amendment 5: Connectivity Fault Management", Work in Progress, February 2007. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC3985] Bryant, S. and Pate, P. (Editors), "Pseudo Wire Emulation Edge-to-Edge (PWE3) Architecture", RFC 3985, March 2005. [RFC4379] Kompella, K., and Swallow, G., "Detecting Multi-Protocol Label Switched (MPLS) Data Plane Failures", RFC 4379, February 2006. [RFC4664] Andersson, L. and Rosen, E. (Editors), "Framework for Layer 2 Virtual Private Networks (L2VPNs)", RFC 4664, September 2006. [RFC4761] Kompella, K. and Rekhter, Y. (Editors), "Virtual Private LAN Service (VPLS) Using BGP for Auto-Discovery and Signaling", RFC 4761, January 2007. [RFC4762] Lasserre, M. and Kompella, V. (Editors), "Virtual Private LAN Service (VPLS) Using Label Distribution Protocol (LDP) Signaling", RFC 4762, January 2007. [VCCV] Nadeau, T., Pignataro, C. and Aggarwal, R. (Editors), "Pseudo Wire Virtual Circuit Connectivity Verification (VCCV)", draft-ietf-pwe3- vccv-13.txt, March 2007. Stokes, et al. Expires May 5, 2008 [Page 24] Internet-Draft OAM for L2 VPN Networks Using CFM and VCCV November 2007 11.2. Informative References [OAM_MSG] Nadeau, T., Morrow, M., Busschbach, P., Aissaoui, M. and Allan, D., "Pseudo Wire (PW) OAM Message Mapping", draft-ietf-pwe3-oam-msg-map- 05.txt, March 2007. [TestHVPLS] Stokes, O., Kompella, V., Heron, G. and Serbest, Y., "Testing Hierarchical Virtual Private LAN Services", draft-stokes-vkompella- ppvpn-hvpls-oam-01, December 2002. Author's Addresses Olen Stokes Extreme Networks PO Box 14129 RTP, NC 27709, USA Email: ostokes@extremenetworks.com Shane Amante Level 3 Communications, LLC 1025 Eldorado Blvd Broomfield, CO 80021, USA Email: shane@level3.net Pranjal Kumar Dutta Alcatel-Lucent 701 E Middlefield Road, Mountain View, CA 94043, USA E-mail: pdutta@alcatel-lucent.com Yetik Serbest AT&T Labs 9505 Arboretum Blvd. Austin, TX 78759, USA E-mail: yetik_serbest@labs.att.com Intellectual Property Statement The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Stokes, et al. Expires May 5, 2008 [Page 25] Internet-Draft OAM for L2 VPN Networks Using CFM and VCCV November 2007 Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Disclaimer of Validity This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Copyright Statement Copyright (C) The IETF Trust (2007). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. Acknowledgment Funding for the RFC Editor function is currently provided by the Internet Society. Stokes, et al. Expires May 5, 2008 [Page 26]