Internet Engineering Task Force (IETF)                M. Kuehlewind, Ed.
Request for Comments: 7560                                    ETH Zurich
Category: Informational                                 R. Scheffenegger
ISSN: 2070-1721                                             NetApp, Inc.
                                                              B. Briscoe
                                                             August 2015
Internet Engineering Task Force (IETF)                M. Kuehlewind, Ed.
Request for Comments: 7560                                    ETH Zurich
Category: Informational                                 R. Scheffenegger
ISSN: 2070-1721                                             NetApp, Inc.
                                                              B. Briscoe
                                                             August 2015

Problem Statement and Requirements for Increased Accuracy in Explicit Congestion Notification (ECN) Feedback




Explicit Congestion Notification (ECN) is a mechanism where network nodes can mark IP packets, instead of dropping them, to indicate congestion to the endpoints. An ECN-capable receiver will feed this information back to the sender. ECN is specified for TCP in such a way that it can only feed back one congestion signal per Round-Trip Time (RTT). In contrast, ECN for other transport protocols, such as RTP/UDP and SCTP, is specified with more accurate ECN feedback. Recent new TCP mechanisms (like Congestion Exposure (ConEx) or Data Center TCP (DCTCP)) need more accurate ECN feedback in the case where more than one marking is received in one RTT. This document specifies requirements for an update to the TCP protocol to provide more accurate ECN feedback.


Status of This Memo


This document is not an Internet Standards Track specification; it is published for informational purposes.


This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Not all documents approved by the IESG are a candidate for any level of Internet Standard; see Section 2 of RFC 5741.

本文件是互联网工程任务组(IETF)的产品。它代表了IETF社区的共识。它已经接受了公众审查,并已被互联网工程指导小组(IESG)批准出版。并非IESG批准的所有文件都适用于任何级别的互联网标准;见RFC 5741第2节。

Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at


Copyright Notice


Copyright (c) 2015 IETF Trust and the persons identified as the document authors. All rights reserved.

版权所有(c)2015 IETF信托基金和确定为文件作者的人员。版权所有。

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents ( in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.

本文件受BCP 78和IETF信托有关IETF文件的法律规定的约束(自本文件出版之日起生效。请仔细阅读这些文件,因为它们描述了您对本文件的权利和限制。从本文件中提取的代码组件必须包括信托法律条款第4.e节中所述的简化BSD许可证文本,并提供简化BSD许可证中所述的无担保。

Table of Contents


   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
     1.1.  Terminology . . . . . . . . . . . . . . . . . . . . . . .   4
   2.  Recap of Classic ECN and ECN Nonce in IP/TCP  . . . . . . . .   5
   3.  Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . .   6
   4.  Requirements  . . . . . . . . . . . . . . . . . . . . . . . .   8
   5.  Design Approaches . . . . . . . . . . . . . . . . . . . . . .  11
     5.1.  Redefinition of ECN/NS Header Bits  . . . . . . . . . . .  11
     5.2.  Using Other Header Bits . . . . . . . . . . . . . . . . .  13
     5.3.  Using a TCP Option  . . . . . . . . . . . . . . . . . . .  13
   6.  Security Considerations . . . . . . . . . . . . . . . . . . .  14
   7.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  14
     7.1.  Normative References  . . . . . . . . . . . . . . . . . .  14
     7.2.  Informative References  . . . . . . . . . . . . . . . . .  14
   Appendix A.  Ambiguity of the More Accurate ECN Feedback in DCTCP  16
   Acknowledgements  . . . . . . . . . . . . . . . . . . . . . . . .  17
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  17
   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
     1.1.  Terminology . . . . . . . . . . . . . . . . . . . . . . .   4
   2.  Recap of Classic ECN and ECN Nonce in IP/TCP  . . . . . . . .   5
   3.  Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . .   6
   4.  Requirements  . . . . . . . . . . . . . . . . . . . . . . . .   8
   5.  Design Approaches . . . . . . . . . . . . . . . . . . . . . .  11
     5.1.  Redefinition of ECN/NS Header Bits  . . . . . . . . . . .  11
     5.2.  Using Other Header Bits . . . . . . . . . . . . . . . . .  13
     5.3.  Using a TCP Option  . . . . . . . . . . . . . . . . . . .  13
   6.  Security Considerations . . . . . . . . . . . . . . . . . . .  14
   7.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  14
     7.1.  Normative References  . . . . . . . . . . . . . . . . . .  14
     7.2.  Informative References  . . . . . . . . . . . . . . . . .  14
   Appendix A.  Ambiguity of the More Accurate ECN Feedback in DCTCP  16
   Acknowledgements  . . . . . . . . . . . . . . . . . . . . . . . .  17
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  17
1. Introduction
1. 介绍

Explicit Congestion Notification (ECN) [RFC3168] is a mechanism where network nodes can mark IP packets instead of dropping them to indicate congestion to the endpoints. An ECN-capable receiver will feed this information back to the sender. ECN is specified for TCP in such a way that only one feedback signal can be transmitted per Round-Trip Time (RTT). This is sufficient for preexisting TCP congestion control mechanisms that perform only one reduction in sending rate per RTT, independent of the number of ECN congestion marks. But recently proposed or deployed mechanisms like Congestion Exposure (ConEx) [RFC6789] or Data Center TCP (DCTCP) [DCTCP] need more accurate ECN feedback than 'classic ECN' [RFC3168] to work correctly in the case where more than one marking is received in any one RTT.


For an in-depth discussion of the application benefits of using ECN (including with sufficiently granular feedback), see [ECN-BENEFITS].


ECN is also defined for transport protocols beside TCP. ECN feedback as defined for RTP/UDP [RFC6679] provides a very detailed level of information, delivering individual counters for all four ECN codepoints as well as lost and duplicate segments, but at the cost of high signalling overhead. ECN feedback for SCTP has been proposed in [SCTP-ECN]. This delivers a counter for the number of ECN-capable packets that were marked due to congestion (since the last sender-side window reduction), but it comes at the cost of increased overhead.


Today, implementations of DCTCP already exist that alter TCP's ECN feedback protocol in proprietary ways (DCTCP was released in Microsoft Windows 8, and implementations exist for Linux and FreeBSD). However, the changes DCTCP makes to TCP omit capability negotiation, relying instead on uniform configuration across all hosts and network devices with ECN capability. A primary motivation for this document is to intervene before each proprietary implementation invents its own non-interoperable handshake, which could lead to _de facto_ consumption of the few flags or codepoints that remain available for standardizing capability negotiation.

今天,DCTCP的实现已经存在,以专有方式改变TCP的ECN反馈协议(DCTCP在Microsoft Windows 8中发布,并且存在针对Linux和FreeBSD的实现)。但是,DCTCP对TCP所做的更改忽略了能力协商,而是依赖于具有ECN能力的所有主机和网络设备的统一配置。本文档的一个主要动机是在每个专有实现发明自己的不可互操作握手之前进行干预,这可能会导致“事实上”消耗仍可用于标准化能力协商的少数标志或代码点。

This document lists requirements for a robust and interoperable TCP/ ECN feedback protocol that is more accurate than classic ECN [RFC3168] and that all implementations of new TCP extensions, like ConEx and/or DCTCP, can use. While a new feedback scheme should still deliver as much information as classic ECN, this document also clarifies what has to be taken into consideration in addition. Thus, the listed requirements should be addressed in the specification of a more accurate ECN feedback scheme. A few solutions have already been


proposed. Section 5 demonstrates how to use the requirements to compare them, by briefly sketching their high-level design choices and discussing the benefits and drawbacks of each.


The scope of these requirements is not limited to any specific environment and is intended for general deployment over public and private IP networks. Candidate solutions should try to adhere to all these requirements, but, where this is not possible, they should justify the deviation. The ordering of the requirements listed in this document is not to be taken as an order of importance, because each requirement might have different weight in different deployment scenarios.


These requirements are only concerned with the type and quality of the ECN feedback signal. The requirements do not stipulate how a TCP sender might react to the improved ECN signal. The requirements also do not imply that any modifications to TCP senders or receivers are obligatory.


1.1. Terminology
1.1. 术语

We use the following terminology from [RFC3168] and [RFC3540]:


The ECN field in the IP header:


Not-ECT: the not ECN-Capable Transport codepoint,


CE: the Congestion Experienced codepoint,


ECT(0): the first ECN-Capable Transport codepoint, and


ECT(1): the second ECN-Capable Transport codepoint.


The ECN flags in the TCP header:


CWR: the Congestion Window Reduced flag,


ECE: the ECN-Echo flag, and


NS: ECN Nonce Sum.


In this document, the ECN feedback scheme as specified in [RFC3168] is called 'classic ECN' and any new proposal is called a 'more accurate ECN feedback' scheme. A 'congestion mark' is defined as an IP packet where the CE codepoint is set. A 'congestion episode' refers to one or more congestion marks that belong to the same overload situation in the network (usually during one RTT). A TCP segment with the acknowledgement flag set is simply called an ACK.


2. Recap of Classic ECN and ECN Nonce in IP/TCP
2. IP/TCP中的经典ECN和ECN协议综述

ECN requires two bits in the IP header. The ECN capability of a packet is indicated when either one of the two bits is set. A network node can set both bits simultaneously when it experiences congestion. This leads to the four codepoints (Not-ECT, ECT(0), ECT(1), and CE) as listed above.


In the TCP header, the first two bits in byte 14 are defined as ECN feedback for each half-connection. A TCP receiver signals the reception of a congestion mark using the ECN-Echo (ECE) flag in the TCP header. For reliability, the receiver continues to set the ECE flag on every ACK. To enable the TCP receiver to determine when to stop setting the ECE flag, the sender sets the CWR flag upon reception of an ECE feedback signal. This always leads to a full RTT of ACKs with ECE set. Thus, the receiver cannot signal back any additional CE markings arriving within the same RTT.

在TCP报头中,字节14中的前两位被定义为每个半连接的ECN反馈。TCP接收器使用TCP报头中的ECN Echo(ECE)标志发出拥塞标记接收信号。为了可靠性,接收器继续在每个ACK上设置ECE标志。为了使TCP接收器能够确定何时停止设置ECE标志,发送方在收到ECE反馈信号时设置CWR标志。这总是导致ECE设置的完整RTT ACK。因此,接收器不能将到达同一RTT内的任何附加CE标记发回信号。

The ECN Nonce [RFC3540] is an experimental addition to ECN that the TCP sender can use to protect itself against accidental or malicious concealment of CE-marked or dropped packets. This addition defines the last bit of byte 13 in the TCP header as the Nonce Sum (NS) flag. The receiver maintains a nonce sum that counts the occurrence of ECT(1) packets and signals the least significant bit of this sum on the NS flag. There are no known deployments of a TCP stack that makes use of the ECN Nonce extension.

ECN Nonce[RFC3540]是对ECN的一个实验性补充,TCP发送方可以使用它来保护自己免受CE标记或丢弃数据包的意外或恶意隐藏。此加法将TCP头中字节13的最后一位定义为Nonce Sum(NS)标志。接收机保持一个统计ECT(1)数据包出现次数的非同步和,并在NS标志上发送该和的最低有效位。没有已知的使用ECN Nonce扩展的TCP堆栈部署。

       0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15
     |               |           | N | C | E | U | A | P | R | S | F |
     | Header Length | Reserved  | S | W | C | R | C | S | S | Y | I |
     |               |           |   | R | E | G | K | H | T | N | N |
       0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15
     |               |           | N | C | E | U | A | P | R | S | F |
     | Header Length | Reserved  | S | W | C | R | C | S | S | Y | I |
     |               |           |   | R | E | G | K | H | T | N | N |

Figure 1: The (Post-ECN Nonce) Definition of the TCP Header Flags

图1:TCP头标志的(Post ECN Nonce)定义

An alternative for a sender to assure feedback integrity has been proposed where the sender itself occasionally inserts a CE mark or reorders packets, and checks that the receiver feeds these back faithfully [TEST-RCV]. This alternative consumes no header bits or codepoints, and it releases the ECT(1) codepoint in the IP header and the NS flag in the TCP header for other uses.


3. Use Cases
3. 用例

The following two examples serve to show where existing mechanisms would already benefit from more accurate ECN feedback information. However, as it is hard to predict the future, once a more accurate ECN feedback mechanism that adheres to the requirements stated in this document is widely deployed, it's very likely that additional uses will be found. The examples listed below are in no particular order.


ConEx is an experimental approach that allows a sender to relay congestion feedback provided by the receiver into the network along the forward data path. ConEx information can be used for traffic management to limit traffic proportionate to the actual congestion being caused, rather than limiting traffic based on rate or volume [RFC6789]. A ConEx sender uses selective acknowledgements (SACK) [RFC2018] for accurate feedback of loss signals, but until now TCP has offered no equivalent accurate feedback for ECN.


DCTCP offers very low and predictable queuing delay. DCTCP changes the reaction to congestion of a TCP sender and additionally requires switches/routers to have ECN enabled and configured with a low step threshold and no signal smoothing, so it is currently only used in private networks, e.g., internal to data centers. DCTCP was released in Microsoft Windows 8, and implementations exist for Linux and FreeBSD. To retrieve sufficient congestion information, the different DCTCP implementations use a proprietary ECN feedback protocol, but they omit capability negotiation. Moreover, the feedback protocol proposed in [DCTCP] only works if there are no losses at all, and otherwise it gets very confused (see Appendix A). Therefore, if a generic, more accurate ECN feedback scheme were available, it would solve two problems for DCTCP: i) the need for a consistent variant of DCTCP to be deployed network-wide and ii) the inability to cope with ACK loss.


Classic ECN-TCP would not benefit from more accurate ECN feedback, but it would not suffer either. The same signal that is currently conveyed with ECN following the specification given in [RFC3168] would be available.


The following scenarios should briefly show where accurate ECN feedback is needed or adds value:


A sender with standardized TCP congestion control that supports ConEx: In this case, the ConEx mechanism uses the extra information per RTT to re-echo the precise congestion information, but the congestion control algorithm still ignores multiple marks per RTT [RFC5681].


A sender using DCTCP congestion control without ConEx: The congestion control algorithm uses the extra info per RTT to perform its decrease depending on the number of congestion marks.


A sender using DCTCP congestion control and supporting ConEx: Both the congestion control algorithm and ConEx use the more accurate ECN feedback mechanism.


As-yet-unspecified sender mechanisms: The above are two examples of more general interest in sender mechanisms that respond to the extent of congestion feedback, not just its existence. It will greatly simplify incremental deployment if the sender can unilaterally deploy new behaviours and rely on the presence of generic receivers that have already implemented more accurate feedback.


A TCP sender using congestion control as specified in RFC 5681 without ConEx: No accurate feedback is necessary here. The congestion control algorithm still reacts to only one signal per RTT. But, it is best to feed back all the information the receiver gets, whether or not the sender uses it -- at least as long as overhead is low or zero.

使用RFC 5681中指定的拥塞控制的TCP发送方,无需ConEx:此处无需准确反馈。拥塞控制算法仍然只对每个RTT的一个信号作出反应。但是,最好是反馈接收方获得的所有信息,不管发送方是否使用这些信息——至少只要开销低或为零。

Using CE for checking integrity: If a more accurate ECN feedback scheme feeds all occurrences of CE marks back, a sender could perform integrity checking by occasionally injecting CE marks itself. Specifically, a sender can send packets that it randomly marks with CE (at low frequency), then check if feedback is received for these packets. The congestion notification feedback for these self-injected markings would not require a congestion control reaction [TEST-RCV].


4. Requirements
4. 要求

The requirements of the accurate ECN feedback protocol are to have fairly accurate (not necessarily perfect), timely, and protected signalling. This leads to the following requirements, which should be discussed for any proposed more accurate ECN feedback scheme:


Resilience The ECN feedback signal is carried within the ACK. Pure TCP ACKs can get lost without recovery (not just due to congestion but also due to deliberate ACK thinning). Moreover, delayed ACKs are commonly used with TCP. Typically, an ACK is triggered after two data segments (or more, e.g., due to receive segment coalescing, ACK compression, ACK congestion control [RFC5690], or other phenomena; see [RFC3449]). In a high-congestion situation where most of the packets are marked with CE, an accurate feedback mechanism should still be able to signal sufficient congestion information. Thus, the accurate ECN feedback extension has to take delayed ACKs and ACK loss into account. Also, a more accurate feedback protocol should still provide more accurate feedback than classic ECN when delayed ACKs cover more than two segments, or when a thin stream disables Nagle's algorithm [RFC896]. Finally, the feedback mechanism should not be impacted by reordering of ACKs, even when the ACKed sequence number does not increase.

ECN反馈信号在ACK中传输。纯TCP ACK可能在没有恢复的情况下丢失(不仅是由于拥塞,还由于故意的ACK细化)。此外,延迟确认通常与TCP一起使用。通常,ACK在两个数据段之后触发(或更多,例如,由于接收段合并、ACK压缩、ACK拥塞控制[RFC5690]或其他现象;请参阅[RFC3449])。在大多数数据包都带有CE标记的高拥塞情况下,准确的反馈机制应该仍然能够发出足够拥塞信息的信号。因此,准确的ECN反馈扩展必须考虑延迟ACK和ACK丢失。此外,当延迟的ack覆盖两个以上的段时,或者当一个瘦流禁用了Nagle算法时,一个更精确的反馈协议仍然应该比经典的ECN提供更精确的反馈[RFC896]。最后,即使确认的序列号没有增加,反馈机制也不应受到确认重新排序的影响。

Timeliness A CE mark can be induced by the sending host, or more commonly a network node on the transmission path, and is then echoed by the receiver in the TCP ACK. Thus, when this information arrives at the sender, it is naturally already about one RTT old. With a sufficient ACK rate, a further delay of a small number of packets can be tolerated. However, this information will become stale with large delays, given the dynamic nature of networks. TCP congestion control (which itself partly introduces these dynamics) operates on a time scale of one RTT. Thus, to be timely, congestion feedback information should be delivered within about one RTT.

CE标记可以由发送主机,或者更常见的是由传输路径上的网络节点引起,然后由接收机在TCP ACK中回响。因此,当该信息到达发送方时,它自然已经大约有一个RTT了。在具有足够的ACK速率的情况下,可以容忍少量数据包的进一步延迟。然而,考虑到网络的动态特性,这些信息将变得陈旧且延迟时间较长。TCP拥塞控制(其本身部分引入了这些动态)在一个RTT的时间尺度上运行。因此,为了及时,拥塞反馈信息应该在大约一个RTT内传递。

Integrity The integrity of the feedback in a more accurate ECN feedback scheme should be assured, at least as well as the ECN Nonce. Alternatively, it should at least be possible to give strong incentives for the receiver and network nodes to cooperate honestly.


Given there are known problems with ECN Nonce deployment, this document only requires that the integrity of the more accurate ECN feedback can be assured; it does not require that the ECN Nonce mechanism is employed to achieve this. Indeed, if integrity could be provided in another manner, a more accurate ECN feedback protocol might repurpose the nonce sum (NS) flag in the TCP header.

鉴于ECN Nonce部署存在已知问题,本文件仅要求确保更准确的ECN反馈的完整性;它不需要使用ECN Nonce机制来实现这一点。事实上,如果可以以另一种方式提供完整性,则更准确的ECN反馈协议可能会重新利用TCP报头中的nonce sum(NS)标志。

If the more accurate ECN feedback scheme provides sufficient information, the integrity check could be performed by, e.g., deterministically setting the CE in the sender and monitoring the respective feedback (similar to ECT(1) and the ECN Nonce sum). Whether a sender should enforce when it detects wrong feedback information, and what kind of enforcement it should apply, are policy issues that need not be specified as part of the more accurate ECN feedback signal scheme itself, but rather when specifying an update to core TCP mechanisms like congestion control that make use of the more accurate ECN signal.

如果更准确的ECN反馈方案提供了足够的信息,则完整性检查可以通过(例如)确定地在发送方中设置CE并监视相应的反馈(类似于ECT(1)和ECN Nonce sum)来执行。发送方在检测到错误的反馈信息时是否应强制执行,以及应采用何种强制执行方式,这些都是政策问题,不需要作为更准确的ECN反馈信号方案本身的一部分加以规定,而是在指定对核心TCP机制(如拥塞控制)的更新时使用更精确的ECN信号。

Accuracy Classic ECN feeds back one congestion notification per RTT; this is sufficient for classic TCP congestion control, which reduces the sending rate at most once per RTT. Thus, the more accurate ECN feedback scheme should ensure that, if a congestion episode occurs, at least one congestion notification is echoed and received per RTT as classic ECN would do. Of course, the goal of a more accurate ECN extension is to reconstruct the number of CE markings more accurately. In the best case, the new scheme should even allow reconstruction of the exact number of payload bytes that a CE-marked packet was carrying. However, it is accepted that it may be too complex for a sender to get the exact number of congestion markings or marked bytes in all situations. Ideally, the feedback scheme should preserve the order in which any (of the four) ECN signals were received. And, ideally, it would even be possible for the sender to determine which of the packets covered by one delayed ACK were congestion marked, e.g., if the flow consists of packets of different sizes, or to allow for future protocols where the order of the markings may be important.

Accuracy Classic ECN每个RTT反馈一个拥塞通知;这对于传统的TCP拥塞控制来说已经足够了,它可以减少每个RTT最多一次的发送速率。因此,更准确的ECN反馈方案应该确保,如果发生拥塞事件,每个RTT至少会像经典ECN那样回显和接收一个拥塞通知。当然,更准确的ECN扩展的目标是更准确地重建CE标记的数量。在最好的情况下,新方案甚至应该允许重建CE标记的数据包所承载的有效负载字节的确切数量。然而,对于发送方来说,在所有情况下获取拥塞标记或标记字节的确切数量可能过于复杂,这一点是可以接受的。理想情况下,反馈方案应保持接收任何(四个)ECN信号的顺序。并且,理想情况下,发送方甚至可以确定由一个延迟ACK覆盖的分组中的哪一个被拥塞标记,例如,如果流由不同大小的分组组成,或者考虑到标记的顺序可能很重要的未来协议。

In the best case, a sender that sees more accurate ECN feedback information would be able to reconstruct the occurrence of any of the four codepoints (Not-ECT, CE, ECT(0), ECT(1)). However, assuming the sender marks all data packets as ECN-capable and uses a default setting of ECT(0) (as with [RFC3168]), solely feeding back the occurrence of CE and ECT(1) might be sufficient. Because the sender can keep account of the transmitted segments with any of the three ECN codepoints, conveying any two of these back to the sender is sufficient for it to reconstruct the third as


observed by the receiver. Thus, a more accurate ECN feedback scheme should at least provide information on two of these signals, e.g., CE and ECT(1).


If a more accurate ECN scheme can reliably deliver feedback in most but not all circumstances, ideally the scheme should at least not introduce bias. In other words, undetected loss of some ACKs should be as likely to increase as decrease the sender's estimate of the probability of ECN marking.


Complexity Implementation should be as simple as possible, and only a minimum of additional state information should be needed. This will enable more accurate ECN feedback to be used as the default feedback mechanism, even if only one ECN feedback signal per RTT is needed.


Overhead A more accurate ECN feedback signal should limit the additional network load, because ECN feedback is ultimately not critical information (in the worst case, loss will still be available as a congestion signal of last resort). As feedback information has to be provided frequently and in a timely fashion, potentially all or a large fraction of TCP acknowledgements might carry this information. Ideally, no additional segments should be exchanged compared to a TCP session as specified in RFC 3168, and the overhead in each segment should be minimized.

开销更准确的ECN反馈信号应限制额外的网络负载,因为ECN反馈最终不是关键信息(在最坏的情况下,丢失仍将作为最后手段的拥塞信号)。由于必须频繁及时地提供反馈信息,所有或大部分TCP确认都可能包含此信息。理想情况下,与RFC 3168中指定的TCP会话相比,不应交换额外的段,并且每个段中的开销应最小化。

Backward and forward compatibility Given more accurate ECN feedback will involve a change to the TCP protocol, it should be negotiated between the two TCP endpoints. If either end does not support the more accurate feedback, they should both be able to fall back to classic ECN feedback.


A more accurate ECN feedback extension should aim to traverse most middleboxes, including firewalls and Network Address Translators (NATs). Further, a feedback mechanism should provide a method to fall back to classic ECN signalling if the new signal is suppressed by certain middleboxes.


In order to avoid a fork in the TCP protocol specifications, if experiments with the new ECN feedback protocol are successful, the intention is to eventually update RFC 3168 for any TCP/ECN sender, not just for ConEx or DCTCP senders. Then, future senders will be able to unilaterally deploy new behaviours that exploit the existence of more accurate ECN feedback in receivers (forward

为了避免TCP协议规范中出现分叉,如果新ECN反馈协议的实验成功,目的是最终为任何TCP/ECN发送方更新RFC 3168,而不仅仅是为ConEx或DCTCP发送方。然后,未来的发送者将能够单方面部署新的行为,利用接收者中存在的更准确的ECN反馈(转发)

compatibility). Conversely, even if another sender only needs one ECN feedback signal per RTT, it should be able to use more accurate ECN feedback and simply ignore the excess information.


Furthermore, the receiver should not make assumptions about the mechanism that was used to set the markings nor about any interpretation or reaction to the congestion signal. The receiver only needs to faithfully reflect congestion information back to the sender.


5. Design Approaches
5. 设计方法

This section introduces some possible design approaches for TCP ECN feedback. The purpose of this section is to give examples of how trade-offs might be needed between the requirements, as input to future IETF work to specify a protocol. The order is not significant, and there is no intention to endorse any particular approach.

本节介绍一些可能的TCP ECN反馈设计方法。本节的目的是举例说明如何在需求之间进行权衡,作为未来IETF工作的输入,以指定协议。该命令并不重要,也无意认可任何特定方法。

All approaches presented below (and proposed so far) are able to provide accurate ECN feedback information as long as no ACK loss occurs and the congestion rate is reasonable. In the case of a high ACK loss rate or very high congestion (CE-marking) rate, the proposed schemes have different resilience characteristics depending on the number of bits used for the encoding. While classic ECN provides reliable (but inaccurate) feedback of a maximum of one congestion signal per RTT, the proposed schemes do not implement an explicit acknowledgement mechanism for the feedback (as, e.g., the ECE/CWR exchange of [RFC3168]).


5.1. Redefinition of ECN/NS Header Bits
5.1. ECN/NS头比特的重新定义

Schemes in this category can additionally use the NS bit for capability negotiation during the TCP handshake exchange. Thus a more accurate ECN could be negotiated without changing the classic ECN negotiation and thus being backwards compatible.


Schemes in this category can simply redefine the ECN header flags, ECE and CWR, to encode the occurrence of a CE marking at the receiver. This approach provides very limited resilience against loss of ACK, particularly pure ACKs (no payload and therefore delivered unreliably).


A couple of schemes have been proposed so far:


o A naive 1-bit scheme that sends one ECE for each CE received could use CWR to increase robustness against ACK loss by introducing redundant information on the next ACK, but this is still vulnerable to ACK loss.

o 对于每个接收到的CE发送一个ECE的简单的1位方案,可以使用CWR通过在下一个ACK上引入冗余信息来提高针对ACK丢失的鲁棒性,但是这仍然容易受到ACK丢失的影响。

o The scheme defined for DCTCP [DCTCP], which toggles the ECE feedback on an immediate ACK whenever the CE marking changes, and otherwise feeds back delayed ACKs with the ECE value unchanged. Appendix A demonstrates that this scheme is still ambiguous to the sender if the ACKs are pure ACKs, and if some may have been lost.

o 为DCTCP[DCTCP]定义的方案,每当CE标记改变时,将ECE反馈切换到立即ACK上,否则在ECE值不变的情况下反馈延迟ACK。附录A表明,如果ACK是纯ACK,并且某些ACK可能已丢失,则此方案对发送方仍然不明确。

Alternatively, the receiver uses the three ECN/NS header flags, ECE, CWR, and NS, to represent a counter that signals the accumulated number of CE markings it has received. Resilience against loss is better than the flag-based schemes but may not suffice in the presence of extended ACK loss that otherwise would not affect the TCP sender's performance.


A number of coding schemes have been proposed so far in this category:


o A 3-bit counter scheme continuously feeds back the three least significant bits of a CE counter;

o 3位计数器方案连续地反馈CE计数器的三个最低有效位;

o A scheme that defines a standardized lookup table to map the eight codepoints onto either a CE counter or an ECT(1) counter.

o 一种定义标准化查找表的方案,用于将八个代码点映射到CE计数器或ECT(1)计数器上。

These proposed schemes provide accumulated information on CE marking feedback, similar to the number of acknowledged bytes in the TCP header. Due to the limited number of bits, the ECN feedback information will wrap much more often than the acknowledgement field. Thus, feedback information could be lost due to a relatively small sequence of pure-ACK losses. Resilience could be increased by introducing redundancy, e.g., send each counter increase two or more times. Of course, any of these additional mechanisms will increase the complexity. If the congestion rate is greater than the ACK rate (multiplied by the number of congestion marks that can be signaled per ACK), the congestion information cannot correctly be fed back. Covering the worst case (where every packet is CE marked) can potentially be realized by dynamically adapting the ACK rate and redundancy. This again increases complexity and perhaps the signalling overhead as well. Schemes that do not repurpose the ECN NS bit could still support the ECN Nonce.

这些建议的方案提供了CE标记反馈的累积信息,类似于TCP报头中的已确认字节数。由于比特数有限,ECN反馈信息将比确认字段更频繁地包装。因此,反馈信息可能由于相对较小的纯ACK丢失序列而丢失。弹性可以通过引入冗余来提高,例如,将每个计数器增加两倍或两倍以上。当然,任何这些额外的机制都会增加复杂性。如果拥塞率大于ACK率(乘以每个ACK可发出信号的拥塞标记数),则无法正确反馈拥塞信息。覆盖最坏情况(每个数据包都有CE标记)可能通过动态调整ACK速率和冗余来实现。这又增加了复杂性,可能还会增加信令开销。不重新调整ECN NS位用途的方案仍然可以支持ECN Nonce。

5.2. Using Other Header Bits
5.2. 使用其他头比特

As seen in Figure 1, there are currently three unused flags in the TCP header. The proposed 3-bit counter or codepoint schemes could be extended by one or more bits to add higher resilience against ACK loss. The relative gain would be exponentially higher resilience against ACK loss, while the respective drawbacks would remain identical.


Alternatively, a new method could standardize the use of the bits in the Urgent Pointer field (see [RFC6093]) to signal more bits of its congestion signal counter, but only whenever the Urgent Flag is not set. As this is often the case, resilience could be increased without additional header overhead.


Any proposal to use such bits would need to check the likelihood that some middleboxes might discard or 'normalize' the currently unused flag bits or a non-zero Urgent Pointer when the Urgent Flag is cleared. If during experimentation certain bits have been proven to be usable, the assignment of any of these bits would then require an IETF standards action.


5.3. Using a TCP Option
5.3. 使用TCP选项

Alternatively, a new TCP option could be introduced, to help maintain the accuracy and integrity of ECN feedback between receiver and sender. Such an option could provide higher resilience and even more information, e.g., as much as is provided by a proposal for SCTP that counts the number of CE marked packet [SCTP-ECN] since the last CWR was observed, or by ECN for RTP/UDP [RFC6679]. The latter explicitly provides the total number of packets during a connection where the IP ECN field is set to ECT(0), ECT(1), CE, or Not-ECT, as well as the number of lost packets. However, deploying new TCP options has its own challenges. Moreover, to actually achieve high resilience, this option would need to be carried by most or all ACKs as the receiver cannot know if and when ACKs may be dropped. Thus, this approach would introduce considerable signalling overhead even though ECN feedback is not extremely critical information (in the worst case, loss will still be available to provide a strong congestion feedback signal). Nevertheless, such a TCP option could be used in addition to a more accurate ECN feedback scheme in the TCP header or in addition to classic ECN, only when needed and when space is available.

或者,可以引入新的TCP选项,以帮助保持接收方和发送方之间ECN反馈的准确性和完整性。这样的选项可以提供更高的弹性和更多的信息,例如,与统计自上次观察到CWR以来CE标记数据包[SCTP-ECN]数量的SCTP提案或RTP/UDP[RFC6679]的ECN提案提供的信息一样多。后者明确地提供在IP ECN字段设置为ECT(0)、ECT(1)、CE或Not ECT的连接期间的分组总数,以及丢失分组的数目。然而,部署新的TCP选项有其自身的挑战。此外,为了实际实现高弹性,大多数或所有ack都需要携带此选项,因为接收机无法知道是否以及何时可能丢弃ack。因此,即使ECN反馈不是非常关键的信息,这种方法也会带来相当大的信令开销(在最坏的情况下,丢失仍然可以提供强拥塞反馈信号)。然而,只有在需要和空间可用时,才可以在TCP报头中更精确的ECN反馈方案之外或在经典ECN之外使用这种TCP选项。

6. Security Considerations
6. 安全考虑

ECN feedback information must only be used if the other information contained in a received TCP segment indicates that the congestion was genuinely part of the flow and not spoofed. That is, the normal TCP acceptance techniques have to be used to verify that the segment is part of the flow before returning any contained ECN information, and, similarly, ECN feedback is only accepted on valid ACKs.


Given ECN feedback is used as input for congestion control, the respective algorithm would not react appropriately if ECN feedback were lost and the resilience mechanism to recover it was inadequate. This resilience requirement is articulated in Section 4. However, it should be noted that ECN feedback is not the last resort against congestion collapse, because if there is insufficient response to ECN, loss will ensue, and TCP will still react appropriately to loss.


A receiver could suppress ECN feedback information leading to its connections consuming excess sender or network resources. This problem is similar to that seen with the classic ECN feedback scheme and should be addressed by integrity checking as required in Section 4.


7. References
7. 工具书类
7.1. Normative References
7.1. 规范性引用文件

[RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition of Explicit Congestion Notification (ECN) to IP", RFC 3168, DOI 10.17487/RFC3168, September 2001, <>.

[RFC3168]Ramakrishnan,K.,Floyd,S.,和D.Black,“向IP添加显式拥塞通知(ECN)”,RFC 3168,DOI 10.17487/RFC3168,2001年9月<>.

[RFC3540] Spring, N., Wetherall, D., and D. Ely, "Robust Explicit Congestion Notification (ECN) Signaling with Nonces", RFC 3540, DOI 10.17487/RFC3540, June 2003, <>.

[RFC3540]Spring,N.,Weterral,D.,和D.Ely,“具有非同步信号的鲁棒显式拥塞通知(ECN)信令”,RFC 3540,DOI 10.17487/RFC3540,2003年6月<>.

7.2. Informative References
7.2. 资料性引用

[DCTCP] Bensley, S., Eggert, L., and D. Thaler, "Microsoft's Datacenter TCP (DCTCP): TCP Congestion Control for Datacenters", Work in Progress, draft-bensley-tcpm-dctcp-05, July 2015.


[ECN-BENEFITS] Fairhurst, G. and M. Welzl, "The Benefits of using Explicit Congestion Notification (ECN)", Work in Progress draft-ietf-aqm-ecn-benefits-06, July 2015.


[RFC896] Nagle, J., "Congestion Control in IP/TCP Internetworks", RFC 896, DOI 10.17487/RFC0896, January 1984, <>.

[RFC896]Nagle,J.,“IP/TCP互联网中的拥塞控制”,RFC 896,DOI 10.17487/RFC0896,1984年1月<>.

[RFC2018] Mathis, M., Mahdavi, J., Floyd, S., and A. Romanow, "TCP Selective Acknowledgment Options", RFC 2018, DOI 10.17487/RFC2018, October 1996, <>.

[RFC2018]Mathis,M.,Mahdavi,J.,Floyd,S.,和A.Romanow,“TCP选择性确认选项”,RFC 2018,DOI 10.17487/RFC2018,1996年10月<>.

[RFC3449] Balakrishnan, H., Padmanabhan, V., Fairhurst, G., and M. Sooriyabandara, "TCP Performance Implications of Network Path Asymmetry", BCP 69, RFC 3449, DOI 10.17487/RFC3449, December 2002, <>.

[RFC3449]Balakrishnan,H.,Padmanabhan,V.,Fairhurst,G.,和M.Sooriyabandara,“网络路径不对称的TCP性能影响”,BCP 69,RFC 3449,DOI 10.17487/RFC3449,2002年12月<>.

[RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion Control", RFC 5681, DOI 10.17487/RFC5681, September 2009, <>.

[RFC5681]Allman,M.,Paxson,V.和E.Blanton,“TCP拥塞控制”,RFC 5681,DOI 10.17487/RFC56812009年9月<>.

[RFC5690] Floyd, S., Arcia, A., Ros, D., and J. Iyengar, "Adding Acknowledgement Congestion Control to TCP", RFC 5690, DOI 10.17487/RFC5690, February 2010, <>.

[RFC5690]Floyd,S.,Arcia,A.,Ros,D.,和J.Iyengar,“将确认拥塞控制添加到TCP”,RFC 5690,DOI 10.17487/RFC5690,2010年2月<>.

[RFC6093] Gont, F. and A. Yourtchenko, "On the Implementation of the TCP Urgent Mechanism", RFC 6093, DOI 10.17487/RFC6093, January 2011, <>.

[RFC6093]Gont,F.和A.Yourtchenko,“关于TCP紧急机制的实施”,RFC 6093,DOI 10.17487/RFC6093,2011年1月<>.

[RFC6679] Westerlund, M., Johansson, I., Perkins, C., O'Hanlon, P., and K. Carlberg, "Explicit Congestion Notification (ECN) for RTP over UDP", RFC 6679, DOI 10.17487/RFC6679, August 2012, <>.

[RFC6679]Westerlund,M.,Johansson,I.,Perkins,C.,O'Hanlon,P.,和K.Carlberg,“UDP上RTP的显式拥塞通知(ECN)”,RFC 6679,DOI 10.17487/RFC66792012年8月<>.

[RFC6789] Briscoe, B., Ed., Woundy, R., Ed., and A. Cooper, Ed., "Congestion Exposure (ConEx) Concepts and Use Cases", RFC 6789, DOI 10.17487/RFC6789, December 2012, <>.

[RFC6789]Briscoe,B.,Ed.,Woundy,R.,Ed.,和A.Cooper,Ed.,“拥塞暴露(ConEx)概念和用例”,RFC 6789,DOI 10.17487/RFC6789,2012年12月<>.

[SCTP-ECN] Stewart, R., Tuexen, M., and X. Dong, "ECN for Stream Control Transmission Protocol (SCTP)", Work in Progress, draft-stewart-tsvwg-sctpecn-05, January 2014.


[TEST-RCV] Moncaster, T., Briscoe, B., and A. Jacquet, "A TCP Test to Allow Senders to Identify Receiver Non-Compliance", Work in Progress, draft-moncaster-tcpm-rcv-cheat-03, July 2014.


Appendix A. Ambiguity of the More Accurate ECN Feedback in DCTCP

As defined in [DCTCP], a DCTCP receiver feeds back ECE=0 on delayed ACKs as long as CE remains 0, and also immediately sends an ACK with ECE=0 when CE transitions to 1. Similarly, it continually feeds back ECE=1 on delayed ACKs while CE remains 1 and immediately feeds back ECE=1 when CE transitions to 0. A sender can unambiguously decode this scheme if there is never any ACK loss, and the sender assumes there will never be any ACK loss.


The following two examples show that the feedback sequence becomes highly ambiguous to the sender if either of these conditions is broken. Below, '0' represents ECE=0, '1' represents ECE=1, and '.' represents a gap of one segment between delayed ACKs. Now imagine that the sender receives the following sequence of feedback on three pure ACKs:




When the receiver sent this sequence, it could have been any of the following four sequences:


a. 0.0.0 (0 x CE)

a. 0.0.0(0 x CE)

b. 010.0 (1 x CE)

b. 010.0(1 x CE)

c. 0.010 (1 x CE)

c. 0.010(1 x CE)

d. 01010 (2 x CE)

d. 01010(2 x CE)

where any of the 1s represent a possible pure ACK carrying ECE feedback that could have been lost. If the sender guesses (a), it might be correct, or it might miss 1 or 2 congestion marks over 5 packets. Therefore, when confronted with this simple sequence (that is not contrived), a sender can guess that congestion might have been 0%, 20%, or 40%, but it doesn't know which.

其中任何1表示可能丢失的纯ACK ECE反馈。如果发送方猜测(a),则可能是正确的,或者可能在5个数据包中错过1或2个拥塞标记。因此,当面对这个简单的序列时(这不是人为的),发送方可以猜测拥塞可能是0%、20%或40%,但它不知道是哪个。

Sequences with a longer gap (e.g., 0...0.0) become far more ambiguous. It helps a little if the sender knows the distance the receiver uses between delayed ACKs, and it helps a lot if the distance is 1, i.e., no delayed ACKs. However, even without delayed ACKs there will still be ambiguity whenever there are pure ACK losses.




Thanks to Gorry Fairhurst for his review and for ideas on CE-based integrity checking and to Mohammad Alizadeh for suggesting the need to avoid bias.

感谢戈里·费尔赫斯特(Gorry Fairhurst)的评论和基于CE的诚信检查的想法,感谢穆罕默德·阿利扎德(Mohammad Alizadeh)建议避免偏见的必要性。

Bob Briscoe was partly funded by the European Community under its Seventh Framework Programme through the Reducing Internet Transport Latency (RITE) project (ICT-317700) and through the Trilogy 2 project (ICT-317756). The views expressed here are solely those of the authors, in the context of the mentioned funding projects.

Bob Briscoe通过减少互联网传输延迟(RITE)项目(ICT-317700)和Trilogy 2项目(ICT-317756),由欧洲共同体在其第七个框架方案下提供部分资金。这里所表达的观点仅是作者在上述资助项目背景下的观点。

Authors' Addresses


Mirja Kuehlewind (editor) ETH Zurich Gloriastrasse 35 Zurich 8092 Switzerland

Mirja Kuehlewind(编辑)ETH苏黎世Gloriastrasse 35苏黎世8092瑞士


Richard Scheffenegger NetApp, Inc. Am Euro Platz 2 Vienna 1120 Austria

Richard Scheffenegger NetApp,Inc.位于奥地利维也纳欧洲广场2号,邮编:1120

   Phone: +43 1 3676811 3146
   Phone: +43 1 3676811 3146

Bob Briscoe BT B54/77, Adastral Park Martlesham Heath Ipswich IP5 3RE United Kingdom

Bob Briscoe BT B54/77,英国阿达斯特拉尔公园马特勒沙姆希思伊普斯维奇IP5

   Phone: +44 1473 645196
   Phone: +44 1473 645196