Internet Engineering Task Force (IETF) H. Shah
Request for Comments: 7306 Broadcom Corporation
Category: Standards Track F. Marti
ISSN: 2070-1721 W. Noureddine
A. Eiriksson
Chelsio Communications, Inc.
R. Sharp
Intel Corporation
June 2014
Internet Engineering Task Force (IETF) H. Shah
Request for Comments: 7306 Broadcom Corporation
Category: Standards Track F. Marti
ISSN: 2070-1721 W. Noureddine
A. Eiriksson
Chelsio Communications, Inc.
R. Sharp
Intel Corporation
June 2014
Remote Direct Memory Access (RDMA) Protocol Extensions
远程直接内存访问（RDMA）协议扩展
Abstract
摘要
This document specifies extensions to the IETF Remote Direct Memory Access Protocol (RDMAP) as specified in RFC 5040. RDMAP provides read and write services directly to applications and enables data to be transferred directly into Upper-Layer Protocol (ULP) Buffers without intermediate data copies. The extensions specified in this document provide the following capabilities and/or improvements: Atomic Operations and Immediate Data.
本文件规定了RFC 5040中规定的IETF远程直接内存访问协议（RDMAP）的扩展。RDMAP直接向应用程序提供读写服务，使数据能够直接传输到上层协议（ULP）缓冲区，而无需中间数据拷贝。本文档中指定的扩展提供了以下功能和/或改进：原子操作和即时数据。
Status of This Memo
关于下段备忘
This is an Internet Standards Track document.
这是一份互联网标准跟踪文件。
This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 5741.
本文件是互联网工程任务组（IETF）的产品。它代表了IETF社区的共识。它已经接受了公众审查，并已被互联网工程指导小组（IESG）批准出版。有关互联网标准的更多信息，请参见RFC 5741第2节。
Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at http://www.rfc-editor.org/info/rfc7306.
有关本文件当前状态、任何勘误表以及如何提供反馈的信息，请访问http://www.rfc-editor.org/info/rfc7306.
Copyright Notice
版权公告
Copyright (c) 2014 IETF Trust and the persons identified as the document authors. All rights reserved.
版权所有（c）2014 IETF信托基金和确定为文件作者的人员。版权所有。
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
本文件受BCP 78和IETF信托有关IETF文件的法律规定的约束(http://trustee.ietf.org/license-info)自本文件出版之日起生效。请仔细阅读这些文件，因为它们描述了您对本文件的权利和限制。从本文件中提取的代码组件必须包括信托法律条款第4.e节中所述的简化BSD许可证文本，并提供简化BSD许可证中所述的无担保。
Table of Contents
目录
1. Introduction ....................................................4
1.1. Discovery of RDMAP Extensions ..............................5
2. Requirements Language ...........................................5
3. Glossary ........................................................6
4. Header Format Extensions ........................................7
4.1. RDMAP Control and Invalidate STag Fields ...................7
4.2. RDMA Message Definitions ...................................9
5. Atomic Operations ...............................................9
5.1. Atomic Operation Details ..................................10
5.1.1. FetchAdd ...........................................10
5.1.2. CmpSwap ............................................12
5.2. Atomic Operations .........................................13
5.2.1. Atomic Operation Request Message ...................14
5.2.2. Atomic Operation Response Message ..................17
5.3. Atomicity Guarantees ......................................18
5.4. Atomic Operations Ordering and Completion Rules ...........18
6. Immediate Data .................................................20
6.1. RDMAP Interactions with ULP for Immediate Data ............20
6.2. Immediate Data Header Format ..............................21
6.3. Immediate Data or Immediate Data with SE Message ..........21
6.4. Ordering and Completions ..................................22
7. Ordering and Completions Table .................................22
8. Error Processing ...............................................25
8.1. Errors Detected at the Local Peer .........................25
8.2. Errors Detected at the Remote Peer ........................26
9. Security Considerations ........................................26
10. IANA Considerations ...........................................27
10.1. RDMAP Message Atomic Operation Subcodes ..................27
10.2. RDMAP Queue Numbers ......................................28
11. References ....................................................29
11.1. Normative References .....................................29
11.2. Informative References ...................................29
12. Acknowledgments ...............................................30
Appendix A. DDP Segment Formats for RDMA Messages .................31
A.1. DDP Segment for Atomic Operation Request ..................32
A.2. DDP Segment for Atomic Response ...........................33
A.3. DDP Segment for Immediate Data and Immediate Data with SE .33
1. Introduction ....................................................4
1.1. Discovery of RDMAP Extensions ..............................5
2. Requirements Language ...........................................5
3. Glossary ........................................................6
4. Header Format Extensions ........................................7
4.1. RDMAP Control and Invalidate STag Fields ...................7
4.2. RDMA Message Definitions ...................................9
5. Atomic Operations ...............................................9
5.1. Atomic Operation Details ..................................10
5.1.1. FetchAdd ...........................................10
5.1.2. CmpSwap ............................................12
5.2. Atomic Operations .........................................13
5.2.1. Atomic Operation Request Message ...................14
5.2.2. Atomic Operation Response Message ..................17
5.3. Atomicity Guarantees ......................................18
5.4. Atomic Operations Ordering and Completion Rules ...........18
6. Immediate Data .................................................20
6.1. RDMAP Interactions with ULP for Immediate Data ............20
6.2. Immediate Data Header Format ..............................21
6.3. Immediate Data or Immediate Data with SE Message ..........21
6.4. Ordering and Completions ..................................22
7. Ordering and Completions Table .................................22
8. Error Processing ...............................................25
8.1. Errors Detected at the Local Peer .........................25
8.2. Errors Detected at the Remote Peer ........................26
9. Security Considerations ........................................26
10. IANA Considerations ...........................................27
10.1. RDMAP Message Atomic Operation Subcodes ..................27
10.2. RDMAP Queue Numbers ......................................28
11. References ....................................................29
11.1. Normative References .....................................29
11.2. Informative References ...................................29
12. Acknowledgments ...............................................30
Appendix A. DDP Segment Formats for RDMA Messages .................31
A.1. DDP Segment for Atomic Operation Request ..................32
A.2. DDP Segment for Atomic Response ...........................33
A.3. DDP Segment for Immediate Data and Immediate Data with SE .33
The RDMA Protocol [RFC5040] provides capabilities for zero-copy data communications that preserve memory protection semantics, enabling more efficient network protocol implementations. The RDMA Protocol is part of the iWARP family of specifications which also include RFC 5041 [RFC5041], RFC 5044 [RFC5044], and RFC 6581 [RFC6581]. This document specifies the following extensions to the RDMA Protocol (RDMAP):
RDMA协议[RFC5040]为零拷贝数据通信提供了保留内存保护语义的功能，从而实现了更高效的网络协议实现。RDMA协议是iWARP系列规范的一部分，该系列规范还包括RFC 5041[RFC5041]、RFC 5044[RFC5044]和RFC 6581[RFC6581]。本文档指定了RDMA协议（RDMAP）的以下扩展：
o Atomic Operations can be performed on remote memory locations. Support for Atomic Operations enhances the usability of RDMAP in distributed shared-memory environments.
o 原子操作可以在远程内存位置上执行。对原子操作的支持增强了RDMAP在分布式共享内存环境中的可用性。
o Immediate Data messages allow the ULP at the sender to provide a small amount of data. When an Immediate Data message is sent following an RDMA Write Message, the combination of the two messages is an implementation of RDMA Write with Immediate message that is found in other RDMA transport protocols.
o 即时数据消息允许发送方的ULP提供少量数据。在RDMA写入消息之后发送即时数据消息时，这两条消息的组合是RDMA写入与其他RDMA传输协议中的即时消息的实现。
Other RDMA transport protocols define the functionality added by these extensions leading to differences in RDMA applications and/or Upper-Layer Protocols. Removing these differences in the transport protocols simplifies these applications and ULPs, and that is the main motivation for the extensions specified in this document.
其他RDMA传输协议定义了这些扩展添加的功能，从而导致RDMA应用程序和/或上层协议的差异。消除传输协议中的这些差异简化了这些应用程序和ULP，这是本文档中指定的扩展的主要动机。
RSockets [RSOCKETS] is an example of RDMA-enabled middleware that provides a socket interface as the upper-edge interface and utilizes RDMA to provide more efficient networking for socket-based applications. RSockets is aware of Immediate Data support in InfiniBand [IB]. RSockets cannot utilize the RDMA Write with Immediate Data operation from InfiniBand. The addition of the Immediate Data operation specified in this document will alleviate this difference in RSockets when running on InfiniBand and iWARP.
RSockets[RSockets]是支持RDMA的中间件的一个示例，它提供一个套接字接口作为上边缘接口，并利用RDMA为基于套接字的应用程序提供更高效的网络。RSockets了解InfiniBand[IB]中的即时数据支持。RSockets无法利用InfiniBand的RDMA写即时数据操作。当在InfiniBand和iWARP上运行时，添加本文档中指定的即时数据操作将缓解RSockets中的这种差异。
Structured high-performance computing applications based on the Message-Passing Interface [MPI] may use Atomic Operations defined in this specification. DAT Atomics [DAT_ATOMICS] is an example of RDMA-enabled middleware that provides a portable RDMA programming interface for various RDMA transport protocols. DAT Atomics includes a primitive for InfiniBand that is not supported by iWARP RDMA-enabled Network Interface Controllers or RNICs. The addition of Atomic Operations as specified in this document will allow Atomic Operations in DAT Atomics to work for both InfiniBand and RNICs interchangeably.
基于消息传递接口[MPI]的结构化高性能计算应用程序可以使用本规范中定义的原子操作。DAT Atomics[DAT_Atomics]是支持RDMA的中间件的一个示例，它为各种RDMA传输协议提供了一个可移植的RDMA编程接口。DAT Atomics包含InfiniBand原语，该原语不受支持iWARP RDMA的网络接口控制器或RNIC的支持。本文件中规定的原子操作的添加将允许DAT Atomics中的原子操作在InfiniBand和RNIC中交替工作。
For more background on RDMA Protocol applicability, see "Applicability of Remote Direct Memory Access Protocol (RDMA) and Direct Data Placement Protocol (DDP)" [RFC5045].
有关RDMA协议适用性的更多背景信息，请参阅“远程直接内存访问协议（RDMA）和直接数据放置协议（DDP）的适用性”[RFC5045]。
Today there are RDMA applications and/or ULPs that are aware of the existence of Atomic and Immediate Data operations for RDMA transports such as InfiniBand and application programming interfaces such as Open Fabrics Verbs [OFAVERBS]. Today, these applications need to be aware that RDMAP does not support certain of these operations. Typically, the availability of these capabilities is exposed to the applications through adapter query interfaces in software. Applications then have to decide to use or not use Immediate Data or Atomic Operations based on the results of the query interfaces. Such query interfaces typically return the scope of atomicity guarantees, not the individual Atomic Operations supported. Therefore, this specification requires all Atomic Operations defined within to be supported if an RNIC supports any Atomic Operations.
如今，有一些RDMA应用程序和/或ULP知道RDMA传输（如InfiniBand）存在原子和即时数据操作，也有一些应用程序编程接口（如Open Fabrics Verbs）[OFAVERBS]。今天，这些应用程序需要知道RDMAP不支持这些操作中的某些操作。通常，这些功能的可用性通过软件中的适配器查询接口向应用程序公开。然后，应用程序必须根据查询接口的结果决定是否使用即时数据或原子操作。这种查询接口通常返回原子性保证的范围，而不是支持的单个原子操作。因此，如果RNIC支持任何原子操作，则本规范要求支持在中定义的所有原子操作。
In cases where heterogeneous hardware, with differing support for Atomic Operations and Immediate Data Operations, is deployed for use by RDMA applications and/or ULPs, applications are either statically configured to use or not use optional features or use application-specific negotiation mechanisms. For the extensions covered by this document, it is RECOMMENDED that RDMA applications and/or ULPs negotiate at the application or ULP level the usage of these extensions. The definition of such application-specific mechanisms is outside the scope of this specification. For backward compatibility, existing applications and/or ULPs should not assume that these extensions are supported.
在部署异构硬件以供RDMA应用程序和/或ULP使用（对原子操作和即时数据操作的支持不同）的情况下，应用程序可以静态配置为使用或不使用可选功能，也可以使用特定于应用程序的协商机制。对于本文档涵盖的扩展，建议RDMA应用程序和/或ULP在应用程序或ULP级别协商这些扩展的使用。此类特定于应用程序的机制的定义不在本规范的范围内。为了向后兼容，现有应用程序和/或ULP不应假定支持这些扩展。
In the absence of application-specific negotiation of the features defined within this specification, the new operations can be attempted, and reported errors can be used to determine a remote peer's capabilities. In the case of Atomics, a FetchAdd operation with "Add Data" set to 0 can safely be used to determine the existence of Atomic Operations without modifying the content of a remote peer's memory. A Remote Operation Error or Unexpected OpCode error will be reported by the remote peer if there is an Immediate Data or Atomic Operation that is not supported by the remote peer.
在本规范中定义的功能没有特定于应用程序的协商的情况下，可以尝试新的操作，并且可以使用报告的错误来确定远程对等方的功能。就原子而言，“添加数据”设置为0的FetchAdd操作可以安全地用于确定原子操作的存在，而无需修改远程对等内存的内容。如果远程对等方不支持即时数据或原子操作，则远程对等方将报告远程操作错误或意外操作码错误。
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119].
本文件中的关键词“必须”、“不得”、“要求”、“应”、“不应”、“应”、“不应”、“建议”、“可”和“可选”应按照RFC 2119[RFC2119]中所述进行解释。
This document is an extension of RFC 5040, and key words are defined in the glossary of that document.
本文档是RFC 5040的扩展，该文档的词汇表中定义了关键词。
Atomic Operation - an operation that results in an execution of a memory operation at a specific ULP Buffer address on a remote node using the Tagged Buffer data transfer model. The consumer can use Atomic Operations to read, modify, and write memory at the destination ULP Buffer address, while at the same time guaranteeing that no other Atomic Operation read or write accesses to the ULP Buffer address targeted by the Atomic Operation will occur across any other RDMAP Streams on an RNIC at the Responder.
原子操作-使用标记缓冲区数据传输模型在远程节点上的特定ULP缓冲区地址执行内存操作的操作。使用者可以使用原子操作来读取、修改和写入目标ULP缓冲区地址处的内存，同时确保响应器上RNIC上的任何其他RDMAP流不会发生对原子操作目标ULP缓冲区地址的其他原子操作读取或写入访问。
Atomic Operation Request - an RDMA Message used by the Data Source to perform an Atomic Operation at the Responder.
原子操作请求—数据源用于在响应程序上执行原子操作的RDMA消息。
Atomic Operation Response - an RDMA Message used by the Responder to describe the completion of an Atomic Operation at the Responder.
原子操作响应-响应程序使用的RDMA消息，用于描述响应程序原子操作的完成情况。
CmpSwap - an Atomic Operation that is used to compare and swap a value at a specific address on a remote node.
CmpSwap—一种原子操作，用于比较和交换远程节点上特定地址的值。
FetchAdd - an Atomic Operation that is used to atomically increment a value at a specific ULP Buffer address on a remote node.
FetchAdd—一种原子操作，用于在远程节点上的特定ULP缓冲区地址处以原子方式递增值。
Immediate Data - a small fixed-size portion of data sent from the Data Source to a Data Sink.
即时数据-从数据源发送到数据接收器的固定大小的小数据部分。
Immediate Data Message - an RDMA Message used by the Data Source to send Immediate Data to the Data Sink.
即时数据消息—数据源用于向数据接收器发送即时数据的RDMA消息。
Immediate Data with Solicited Event (SE) Message - an RDMA Message used by the Data Source to send Immediate Data with Solicited Event to the Data Sink.
即时数据请求事件（SE）消息-数据源使用RDMA消息将即时数据请求事件发送到数据接收器。
iWARP - a suite of wire protocols comprised of RFC 5040, RFC 5041, RFC 5044, and RFC 6581.
iWARP—一套有线协议，由RFC 5040、RFC 5041、RFC 5044和RFC 6581组成。
Requester - the sender of an RDMA Atomic Operation request.
Requester—RDMA原子操作请求的发送方。
Responder - the receiver of an RDMA Atomic Operation request.
响应者-RDMA原子操作请求的接收者。
RNIC - RDMA-enabled Network Interface Controller. In this context, this would be a network I/O adapter or embedded controller with iWARP functionality.
RNIC-支持RDMA的网络接口控制器。在这种情况下，这将是一个具有iWARP功能的网络I/O适配器或嵌入式控制器。
ULP - Upper-Layer Protocol. The protocol layer above the one currently being referenced. The ULP for RFC 5040 / RFC 5041 is expected to be an OS, Application, adaptation layer, or proprietary device. The RFC 5040 / RFC 5041 documents do not specify a ULP -- they provide a set of semantics that allow a ULP to be designed to utilize RFC 5040 / RFC 5041.
ULP-上层协议。当前引用的协议层之上的协议层。RFC 5040/RFC 5041的ULP应为操作系统、应用程序、适配层或专有设备。RFC 5040/RFC 5041文档没有指定ULP——它们提供了一组语义，允许ULP设计为使用RFC 5040/RFC 5041。
The control information of RDMA Messages is included in header fields defined in RFC 5041, the Direct Data Placement (DDP) protocol. RFC 5040 defines the RDMAP header formats layered on the DDP header definition. This specification extends RFC 5040 with the following new formats:
RDMA消息的控制信息包含在RFC 5041（直接数据放置（DDP）协议）中定义的头字段中。RFC 5040定义了分层在DDP头定义上的RDMAP头格式。本规范使用以下新格式扩展RFC 5040：
o Four new RDMA Messages carry additional RDMAP headers. The Immediate Data operation and Immediate Data with Solicited Event operation each include 8 bytes of data following the RDMAP header. Atomic Operations include Atomic Request or Atomic Response headers following the RDMAP header. The RDMAP header for Atomic Request messages is 52 bytes long as specified in Figure 4. The RDMAP header for Atomic Response Messages is 32 bytes long as specified in Figure 5.
o 四条新的RDMA消息带有额外的RDMAP头。即时数据操作和即时数据请求事件操作都包括RDMAP头后面的8字节数据。原子操作包括RDMAP头之后的原子请求或原子响应头。原子请求消息的RDMAP头的长度为52字节，如图4所示。原子响应消息的RDMAP头的长度为32字节，如图5所示。
o Introduction of a new queue for untagged Buffers (QN=3) used for Atomic Response tracking.
o 引入用于原子响应跟踪的未标记缓冲区（QN=3）的新队列。
For reference, Figure 1 depicts the format of the DDP Control and RDMAP Control Fields, in the style and convention of RFC 5040:
为了便于参考，图1以RFC 5040的样式和惯例描述了DDP控件和RDMAP控件字段的格式：
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|T|L| Resrv | DV| RV|Rsv| Opcode|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Invalidate STag |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|T|L| Resrv | DV| RV|Rsv| Opcode|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Invalidate STag |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 1: DDP Control and RDMAP Control Fields
图1:DDP控制和RDMAP控制字段
The DDP Control Field consists of the T (Tagged), L (Last), Resrv, and DV (DDP protocol Version) fields [RFC5041]. The RDMAP Control Field consists of the RV (RDMA Version), Rsv, and Opcode fields [RFC5040].
DDP控制字段由T（标记）、L（最后）、Resrv和DV（DDP协议版本）字段组成[RFC5041]。RDMAP控制字段包括RV（RDMA版本）、Rsv和操作码字段[RFC5040]。
This specification adds values for the RDMA Opcode field to those specified in RFC 5040. Figure 2 defines the new values of the RDMA Opcode field that are used for the RDMA Messages defined in this specification.
本规范将RDMA操作码字段的值添加到RFC 5040中指定的值。图2定义了用于本规范中定义的RDMA消息的RDMA操作码字段的新值。
As shown in Figure 2, STag (Steering Tag) and Tagged Offset are not applicable for the RDMA Messages defined in this specification. Figure 2 also shows the appropriate Queue Number for each Opcode.
如图2所示，STag（转向标签）和Tagged Offset不适用于本规范中定义的RDMA消息。图2还显示了每个操作码的相应队列号。
All RDMA Messages defined in this specification MUST have:
本规范中定义的所有RDMA消息必须具有：
The RDMA Version (RV) field: 01b.
RDMA版本（RV）字段：01b。
Opcode field: Set to one of the values in Figure 2.
操作码字段：设置为图2中的一个值。
Invalidate STag: Set to zero by the sender, ignored by the receiver.
无效STag：发送方设置为零，接收方忽略。
-------+-----------+-------+------+-------+---------+-------------
RDMA | Message | Tagged| STag | Queue | In- | Message
Opcode | Type | Flag | and | Number| validate| Length
| | | TO | | STag | Communicated
| | | | | | between DDP
| | | | | | and RDMAP
-------+-----------+-------+------+-------+---------+-------------
1000b | Immediate | 0 | N/A | 0 | N/A | Yes
| Data | | | | |
-------+-----------+----------------------------------------------
1001b | Immediate | 0 | N/A | 0 | N/A | Yes
| Data with | | | | |
| SE | | | | |
-------+-----------+----------------------------------------------
1010b | Atomic | 0 | N/A | 1 | N/A | Yes
| Request | | | | |
-------+-----------+----------------------------------------------
1011b | Atomic | 0 | N/A | 3 | N/A | Yes
| Response | | | | |
-------+-----------+----------------------------------------------
-------+-----------+-------+------+-------+---------+-------------
RDMA | Message | Tagged| STag | Queue | In- | Message
Opcode | Type | Flag | and | Number| validate| Length
| | | TO | | STag | Communicated
| | | | | | between DDP
| | | | | | and RDMAP
-------+-----------+-------+------+-------+---------+-------------
1000b | Immediate | 0 | N/A | 0 | N/A | Yes
| Data | | | | |
-------+-----------+----------------------------------------------
1001b | Immediate | 0 | N/A | 0 | N/A | Yes
| Data with | | | | |
| SE | | | | |
-------+-----------+----------------------------------------------
1010b | Atomic | 0 | N/A | 1 | N/A | Yes
| Request | | | | |
-------+-----------+----------------------------------------------
1011b | Atomic | 0 | N/A | 3 | N/A | Yes
| Response | | | | |
-------+-----------+----------------------------------------------
Figure 2: Additional RDMA Usage of DDP Fields
图2:DDP字段的额外RDMA使用
Note: N/A means Not Applicable.
注：不适用表示不适用。
This extension defines RDMAP use of Queue Number 3 for Untagged Buffers for Atomic Responses. This queue is used for tracking outstanding Atomic Requests.
此扩展定义了RDMAP使用队列号3作为原子响应的未标记缓冲区。此队列用于跟踪未完成的原子请求。
All other DDP and RDMAP Control Fields are set as described in RFC 5040.
所有其他DDP和RDMAP控制字段的设置如RFC 5040所述。
The following figure defines which RDMA Headers are used on each new RDMA Message and which new RDMA Messages are allowed to carry ULP payload.
下图定义了在每个新RDMA消息上使用的RDMA头，以及允许哪些新RDMA消息携带ULP有效负载。
-------+-----------+-------------------+-------------------------
RDMA | Message | RDMA Header Used | ULP Message allowed in
Message| Type | | the RDMA Message
OpCode | | |
| | |
-------+-----------+-------------------+-------------------------
1000b | Immediate | Immediate Data | No
| Data | Header |
-------+-----------+-------------------+-------------------------
1001b | Immediate | Immediate Data | No
| Data with | Header |
| SE | |
-------+-----------+-------------------+-------------------------
1010b | Atomic | Atomic Request | No
| Request | Header |
-------+-----------+-------------------+-------------------------
1011b | Atomic | Atomic Response | No
| Response | Header |
-------+-----------+-------------------+-------------------------
-------+-----------+-------------------+-------------------------
RDMA | Message | RDMA Header Used | ULP Message allowed in
Message| Type | | the RDMA Message
OpCode | | |
| | |
-------+-----------+-------------------+-------------------------
1000b | Immediate | Immediate Data | No
| Data | Header |
-------+-----------+-------------------+-------------------------
1001b | Immediate | Immediate Data | No
| Data with | Header |
| SE | |
-------+-----------+-------------------+-------------------------
1010b | Atomic | Atomic Request | No
| Request | Header |
-------+-----------+-------------------+-------------------------
1011b | Atomic | Atomic Response | No
| Response | Header |
-------+-----------+-------------------+-------------------------
Figure 3: RDMA Message Definitions
图3:RDMA消息定义
The RDMA Protocol Specification in RFC 5040 does not include support for Atomic Operations, which are an important building block for implementing distributed shared memory.
RFC 5040中的RDMA协议规范不包括对原子操作的支持，原子操作是实现分布式共享内存的重要构建块。
This document extends the RDMA Protocol specification with a set of basic Atomic Operations and specifies their resource and ordering rules. The Atomic Operations specified in this document provide equivalent functionality to the InfiniBand RDMA transport as well as extended Atomic Operations defined in Open Fabrics Verbs, to allow applications that use these primitives to work interchangeably over iWARP. Other operations are left for future consideration.
本文档使用一组基本的原子操作扩展了RDMA协议规范，并指定了它们的资源和排序规则。本文档中指定的原子操作提供了与InfiniBand RDMA传输等效的功能，以及开放结构谓词中定义的扩展原子操作，以允许使用这些原语的应用程序在iWARP上互换工作。其他操作留待将来考虑。
Atomic Operations as specified in this document execute a 64-bit memory operation at a specified destination ULP Buffer address on a Responder node using the Tagged Buffer data transfer model. The operations atomically read, modify, and write back the contents of the destination ULP Buffer address and guarantee that Atomic Operations on this ULP Buffer address by other RDMAP Streams on the
本文档中指定的原子操作使用标记的缓冲区数据传输模型在响应节点上的指定目标ULP缓冲区地址执行64位内存操作。这些操作以原子方式读取、修改和写回目标ULP缓冲区地址的内容，并保证此ULP缓冲区地址上的原子操作由
same RNIC do not occur between the read and the write caused by the Atomic Operation. Therefore, the Responder RNIC MUST implement mechanisms to prevent Atomic Operations to a memory registered for Atomic Operations while an Atomic Operation targeting the memory is in progress. The Requester of an Atomic Operation cannot rely on Atomic Operation behavior at the Responder across multiple RNICs or with respect to other applications/ULPs running at the Responder that can access the ULP Buffer. It is OPTIONAL for an RNIC to provide such behavior when implementing the Atomic Operations specified in this document. An RNIC that supports Atomic Operations as specified in this document MUST implement both the FetchAdd operation as specified in Section 5.1.1 and the CmpSwap operation as specified in Section 5.1.2. The advertisement of Tagged Buffer information for Atomic Operations is outside the scope of this specification and is handled by the ULPs.
原子操作导致的读写之间不会出现相同的RNIC。因此，响应程序RNIC必须实现机制，以防止在针对内存的原子操作正在进行时，对为原子操作注册的内存执行原子操作。原子操作的请求者不能跨多个RNIC依赖于响应者处的原子操作行为，也不能依赖于响应者处运行的可访问ULP缓冲区的其他应用程序/ULP。RNIC在实现本文档中指定的原子操作时提供此类行为是可选的。支持本文件规定的原子操作的RNIC必须实现第5.1.1节规定的FetchAdd操作和第5.1.2节规定的CmpSwap操作。原子操作标记缓冲区信息的公布不在本规范的范围内，由ULPs处理。
Implementation note: It is RECOMMENDED that the applications do not use the ULP Buffer addresses used for Atomic Operations for other RDMA operations due to the lack of atomicity guarantees between operations other than Atomic Operations.
实施说明：建议应用程序不要将用于原子操作的ULP缓冲区地址用于其他RDMA操作，因为原子操作以外的操作之间缺乏原子性保证。
Implementation note: Errors related to the alignment in the following sections cover Atomic Operations targeted at a ULP Buffer address that is not aligned to a 64-bit boundary.
实施说明：以下各节中与对齐相关的错误包括针对未与64位边界对齐的ULP缓冲区地址的原子操作。
Atomic Operation Request Messages use the same remote addressing mechanism as RDMA Reads and Writes. The ULP Buffer address specified in the request is in the address space of the Remote Peer to which the Atomic Operation is targeted.
原子操作请求消息使用与RDMA读写相同的远程寻址机制。请求中指定的ULP缓冲区地址位于原子操作所针对的远程对等方的地址空间中。
Atomic Operation Response Messages MUST use the Untagged Buffer model with QN=3. Queue number 3 will be used to track outstanding Atomic Operation Request messages at the Requester. When the Atomic Operation Response message is received, the Message Sequence Number (MSN) will be used to locate the corresponding Atomic Operation request in order to complete the Atomic Operation request.
原子操作响应消息必须使用QN=3的未标记缓冲区模型。队列号3将用于跟踪请求者处未完成的原子操作请求消息。当收到原子操作响应消息时，将使用消息序列号（MSN）定位相应的原子操作请求，以完成原子操作请求。
The following subsections describe the Atomic Operations in more detail.
以下小节将更详细地描述原子操作。
The FetchAdd Atomic Operation requests the Responder to read a 64-bit Original Remote Data Value at a 64-bit aligned ULP Buffer address in the Responder's memory, perform the FetchAdd operation on multiple fields of selectable length specified by 64-bit "Add Mask", and write
FetchAdd原子操作请求响应程序读取响应程序内存中64位对齐ULP缓冲区地址处的64位原始远程数据值，对64位“添加掩码”指定的多个可选长度字段执行FetchAdd操作，然后写入
the result back to the same ULP Buffer address. The Atomic addition is performed independently on each one of these fields. A bit set in the Add Mask field specifies the field boundary; for each field, a bit is set at the most significant bit position for each field, causing any carry out of that bit position to be discarded when the addition is performed.
将结果返回到同一ULP缓冲区地址。原子加法是在这些场中的每一个场上独立执行的。添加掩码字段中设置的位指定字段边界；对于每个字段，在每个字段的最高有效位位置设置一个位，从而在执行加法时丢弃该位位置的任何进位。
FetchAdd Atomic Operations MUST target ULP Buffer addresses that are 64-bit aligned. FetchAdd Atomic Operations that target ULP Buffer addresses that are not 64-bit aligned MUST be surfaced as errors, and the Responder's memory MUST NOT be modified in such cases. Additionally, an error MUST be surfaced and a terminate message MUST be generated. The setting of the Add Mask field to 0x0000000000000000 results in Atomic Add of 64-bit Original Remote Data Value and 64-bit "Add Data".
FetchAdd原子操作必须以64位对齐的ULP缓冲区地址为目标。针对未64位对齐的ULP缓冲区地址的FetchAdd原子操作必须显示为错误，并且在这种情况下不得修改响应程序的内存。此外，必须出现错误并生成终止消息。将“添加掩码”字段设置为0x0000000000000000将导致64位原始远程数据值和64位“添加数据”的原子添加。
The pseudocode below describes a masked FetchAdd Atomic Operation.
下面的伪代码描述了掩蔽的FetchAdd原子操作。
bit_location = 1
bit_location = 1
carry = 0
carry = 0
Remote Data Value = 0
远程数据值=0
for bit = 0 to 63
对于位=0到63
{
{
if (bit != 0 ) bit_location = bit_location << 1
if (bit != 0 ) bit_location = bit_location << 1
val1 = (Original Remote Data Value & bit_location) >> bit
val1 = (Original Remote Data Value & bit_location) >> bit
val2 = (Add Data & bit_location) >> bit
sum = carry + val1 + val2
val2 = (Add Data & bit_location) >> bit
sum = carry + val1 + val2
carry = (sum & 2) >> 1
carry = (sum & 2) >> 1
sum = sum & 1
sum=sum&1
if (sum)
若有（总和）
Remote Data Value |= bit_location
远程数据值|=位_位置
carry = ((carry) && (!(Add Mask & bit_location)))
carry = ((carry) && (!(Add Mask & bit_location)))
}
}
The FetchAdd operation is performed in the endian format of the target memory. The "Original Remote Data Value" is converted from the endian format of the target memory for return and returned to the Requester. The fields are in big-endian format on the wire.
FetchAdd操作以目标内存的endian格式执行。“原始远程数据值”从目标内存的endian格式转换为return，并返回给请求者。导线上的字段采用大端格式。
The Requester specifies:
请求者指定：
o Remote STag
o 远牡鹿
o Remote Tagged Offset
o 远程标记偏移量
o Add Data
o 添加数据
o Add Mask
o 添加遮罩
The Responder returns:
响应者返回：
o Original Remote Data
o 原始远程数据
The CmpSwap Atomic Operation requires the Responder to read a 64-bit value at a ULP Buffer address that is 64-bit aligned in the Responder's memory, to perform an AND logical operation using the 64-bit Compare Mask field in the Atomic Operation Request header, then to compare it with the result of a logical AND operation of the Compare Mask and the Compare Data fields in the header. If the two values are equal, the Responder is required to swap masked bits in the same ULP Buffer address with the masked Swap Data. If the two masked compare values are not equal, the contents of the Responder's memory are not changed. In either case, the original value read from the ULP Buffer address is converted from the endian format of the target memory for return and returned to the Requester. The fields are in big-endian format on the wire.
CmpSwap原子操作要求响应程序读取响应程序内存中64位对齐的ULP缓冲区地址处的64位值，以使用原子操作请求标头中的64位比较掩码字段执行AND逻辑操作，然后将其与比较掩码和标头中比较数据字段的逻辑“与”操作的结果进行比较。如果两个值相等，则响应程序需要将同一ULP缓冲区地址中的屏蔽位与屏蔽交换数据交换。如果两个屏蔽比较值不相等，则响应程序内存的内容不会更改。在任何一种情况下，从ULP缓冲区地址读取的原始值都会从目标内存的endian格式转换为return并返回给请求者。导线上的字段采用大端格式。
The Requester specifies:
请求者指定：
o Remote STag
o 远牡鹿
o Remote Tagged Offset
o 远程标记偏移量
o Swap Data
o 交换数据
o Swap Mask
o 交换掩码
o Compare Data
o 比较数据
o Compare Mask
o 比较遮罩
The Responder returns:
响应者返回：
o Original Remote Data Value
o 原始远程数据值
The following pseudocode describes the masked CmpSwap operation result.
以下伪代码描述屏蔽的CmpSwap操作结果。
if (!((Compare Data ^ Original Remote Data Value) &
if (!((Compare Data ^ Original Remote Data Value) &
Compare Mask))
比较掩码）
then
然后
Remote Data Value =
远程数据值=
(Original Remote Data Value & ~(Swap Mask))
（原始远程数据值&~（交换掩码））
| (Swap Data & Swap Mask)
|（交换数据和交换掩码）
else
其他的
Remote Data Value = Original Remote Data Value
远程数据值=原始远程数据值
After the operation, the remote data Buffer MUST contain the "Original Remote Data Value" (if comparison did not match) or the masked "Swap Data" (if the comparison did match). CmpSwap Atomic Operations MUST target ULP Buffer addresses that are 64-bit aligned.
操作后，远程数据缓冲区必须包含“原始远程数据值”（如果比较不匹配）或屏蔽的“交换数据”（如果比较不匹配）。CmpSwap原子操作必须以64位对齐的ULP缓冲区地址为目标。
If a CmpSwap Atomic Operation is attempted on a target ULP Buffer address that is not 64-bit aligned:
如果尝试在未64位对齐的目标ULP缓冲区地址上执行CmpSwap原子操作：
o The operation MUST NOT be performed,
o 不得执行该操作，
o The Responder's memory MUST NOT be modified,
o 不得修改响应者的内存，
o The result MUST be surfaced as an error, and
o 结果必须显示为错误，并且
o A terminate message MUST be generated. (See Section 8.2 for the contents of the terminate message.)
o 必须生成终止消息。（有关终止消息的内容，请参见第8.2节。）
The Atomic Operation Request and Response are RDMA Messages. An Atomic Operation makes use of the DDP Untagged Buffer Model. Atomic Operation Request messages MUST use the same Queue Number as RDMA Read Requests (QN=1). Reusing the same Queue Number for Atomic Request messages allows the Atomic Operations to reuse the same infrastructure (e.g., Outbound and Inbound RDMA Read Queue Depth
原子操作请求和响应是RDMA消息。原子操作使用DDP未标记的缓冲区模型。原子操作请求消息必须使用与RDMA读取请求相同的队列号（QN=1）。对原子请求消息重用相同的队列号允许原子操作重用相同的基础结构（例如，出站和入站RDMA读取队列深度）
(ORD/IRD) flow control) as defined for RDMA Read Requests. Atomic Operation Response messages MUST set Queue Number (QN) to 3 in the DDP header.
（ORD/IRD）流控制），如RDMA读取请求所定义。原子操作响应消息必须在DDP头中将队列号（QN）设置为3。
The RDMA Message OpCode for an Atomic Request Message is 1010b. The RDMA Message OpCode for an Atomic Response Message is 1011b.
原子请求消息的RDMA消息操作码为1010b。原子响应消息的RDMA消息操作码为1011b。
The Atomic Operation Request Message carries an Atomic Operation Header that describes the ULP Buffer address in the Responder's memory. The Atomic Operation Request header immediately follows the DDP header. The RDMAP layer passes to the DDP layer a RDMAP Control Field. The following figure depicts the Atomic Operation Request Header that is used for all Atomic Operation Request Messages:
原子操作请求消息携带一个原子操作报头，该报头描述响应程序内存中的ULP缓冲区地址。原子操作请求头紧跟在DDP头之后。RDMAP层将RDMAP控制字段传递给DDP层。下图描述了用于所有原子操作请求消息的原子操作请求标头：
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reserved (Not Used) |AOpCode|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Request Identifier |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Remote STag |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Remote Tagged Offset |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Add or Swap Data |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Add or Swap Mask |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Compare Data |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Compare Mask |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reserved (Not Used) |AOpCode|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Request Identifier |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Remote STag |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Remote Tagged Offset |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Add or Swap Data |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Add or Swap Mask |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Compare Data |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Compare Mask |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 4: Atomic Operation Request Header
图4：原子操作请求头
Reserved (Not Used): 28 bits
保留（未使用）：28位
This field is set to zero on transmit, ignored on receive.
此字段在发送时设置为零，在接收时忽略。
Atomic Operation Code (AOpCode): 4 bits.
原子操作码（AOpCode）：4位。
See Figure 5. All Atomic Operation Codes from Figure 5 MUST be implemented by an RNIC that supports Atomic Operations.
参见图5。图5中的所有原子操作代码必须由支持原子操作的RNIC实现。
Request Identifier: 32 bits.
请求标识符：32位。
The Request Identifier specifies a number that is used to identify the Atomic Operation Request Message. The value used in this field is selected by the RNIC that sends the message, and it is reflected back to the Local Peer in the Atomic Operation Response message.
请求标识符指定用于标识原子操作请求消息的数字。此字段中使用的值由发送消息的RNIC选择，并在原子操作响应消息中反映回本地对等方。
Remote STag: 32 bits.
远程STag:32位。
The Remote STag identifies the Remote Peer's Tagged Buffer targeted by the Atomic Operation. The Remote STag is associated with the RDMAP Stream through a mechanism that is outside the scope of the RDMAP specification.
远程STag标识原子操作所针对的远程对等方的标记缓冲区。远程STag通过RDMAP规范范围之外的机制与RDMAP流相关联。
Remote Tagged Offset: 64 bits.
远程标记偏移量：64位。
The Remote Tagged Offset specifies the starting offset, in octets, from the base of the Remote Peer's Tagged Buffer targeted by the Atomic Operation. The Remote Tagged Offset MAY start at an arbitrary offset but MUST represent a ULP Buffer address that is 64-bit aligned.
远程标记的偏移量指定从原子操作所针对的远程对等方标记缓冲区的底部开始的偏移量（以八位字节为单位）。远程标记的偏移量可以从任意偏移量开始，但必须表示64位对齐的ULP缓冲区地址。
Add or Swap Data: 64 bits.
添加或交换数据：64位。
The Add or Swap Data field specifies the 64-bit "Add Data" value in an Atomic FetchAdd Operation or the 64-bit "Swap Data" value in an Atomic Swap or CmpSwap Operation.
添加或交换数据字段指定原子FetchAdd操作中的64位“添加数据”值，或原子交换或CmpSwap操作中的64位“交换数据”值。
Add or Swap Mask: 64 bits
添加或交换掩码：64位
This field is used in masked Atomic Operations (FetchAdd and CmpSwap) to perform a bitwise logical AND operation as specified in the definition of these operations. For non-masked Atomic Operations (Swap), this field is set to ffffffffffffffffh on transmit and ignored by the receiver.
此字段用于屏蔽原子操作（FetchAdd和CmpSwap），以执行这些操作定义中指定的按位逻辑“与”操作。对于非屏蔽原子操作（Swap），此字段在发送时设置为FFFFFFFFFFFFFFH，并被接收器忽略。
Compare Data: 64 bits.
比较数据：64位。
The Compare Data field specifies the 64-bit "Compare Data" value in an Atomic CmpSwap Operation. For Atomic Operations FetchAdd and Atomic Swap, the Compare Data field is set to zero on transmit and ignored by the receiver.
比较数据字段指定原子CmpSwap操作中的64位“比较数据”值。对于原子操作FetchAdd和原子交换，比较数据字段在传输时设置为零，并被接收器忽略。
Compare Mask: 64 bits
比较掩码：64位
This field is used in masked Atomic Operation CmpSwap to perform a bitwise logical AND operation as specified in the definition of these operations. For Atomic Operations FetchAdd and Swap, this field is set to ffffffffffffffffh on transmit and ignored by the receiver.
此字段用于屏蔽原子操作CmpSwap，以执行这些操作定义中指定的位逻辑“与”操作。对于原子操作FetchAdd和Swap，此字段在发送时设置为ffffffffffffffffh，并被接收器忽略。
---------+-----------+----------+----------+---------+---------
Atomic | Atomic | Add or | Add or | Compare | Compare
Operation| Operation | Swap | Swap | Data | Mask
Code | | Data | Mask | |
---------+-----------+----------+----------+---------+---------
0000b | FetchAdd | Add Data | Add Mask | N/A | N/A
---------+-----------+----------+----------+---------+---------
0010b | CmpSwap | Swap Data| Swap Mask| Valid | Valid
---------+-----------+-----------------------------------------
---------+-----------+----------+----------+---------+---------
Atomic | Atomic | Add or | Add or | Compare | Compare
Operation| Operation | Swap | Swap | Data | Mask
Code | | Data | Mask | |
---------+-----------+----------+----------+---------+---------
0000b | FetchAdd | Add Data | Add Mask | N/A | N/A
---------+-----------+----------+----------+---------+---------
0010b | CmpSwap | Swap Data| Swap Mask| Valid | Valid
---------+-----------+-----------------------------------------
Figure 5: Atomic Operation Message Definitions
图5：原子操作消息定义
The Atomic Operation Request Message has the following semantics:
原子操作请求消息具有以下语义：
1. An Atomic Operation Request Message MUST reference an Untagged Buffer. That is, the Local Peer's RDMAP layer MUST request that the DDP mark the Message as Untagged.
1. 原子操作请求消息必须引用未标记的缓冲区。也就是说，本地对等方的RDMAP层必须请求DDP将消息标记为未标记。
2. One Atomic Operation Request Message MUST consume one Untagged Buffer.
2. 一个原子操作请求消息必须使用一个未标记的缓冲区。
3. The Responder's RDMAP layer MUST process an Atomic Operation Request Message. A valid Atomic Operation Request Message MUST NOT be delivered to the Responder's ULP (i.e., it is processed by the RDMAP layer).
3. 响应者的RDMAP层必须处理原子操作请求消息。有效的原子操作请求消息不得传递到响应者的ULP（即，它由RDMAP层处理）。
4. At the Responder, an error MUST be surfaced in response to delivery to the Remote Peer's RDMAP layer of an Atomic Operation Request Message with an Atomic Operation Code that the RNIC does not support.
4. 在响应者处，必须出现错误，以响应使用RNIC不支持的原子操作代码向远程对等方的RDMAP层传递原子操作请求消息。
5. An Atomic Operation Request Message MUST reference the RDMA Read Request Queue. That is, the Requester's RDMAP layer MUST request that the DDP layer set the Queue Number field to one.
5. 原子操作请求消息必须引用RDMA读取请求队列。也就是说，请求者的RDMAP层必须请求DDP层将队列号字段设置为1。
6. The Requester MUST pass to the DDP layer Atomic Operation Request Messages in the order they were submitted by the ULP.
6. 请求者必须按照ULP提交的顺序将原子操作请求消息传递给DDP层。
7. The Responder MUST process the Atomic Operation Request Messages in the order they were sent.
7. 响应者必须按照发送顺序处理原子操作请求消息。
8. If the Responder receives a valid Atomic Operation Request Message, it MUST respond with a valid Atomic Operation Response Message.
8. 如果响应程序收到有效的原子操作请求消息，则必须使用有效的原子操作响应消息进行响应。
The Atomic Operation Response Message carries an Atomic Operation Response Header that contains the "Original Request Identifier" and "Original Remote Data Value". The Atomic Operation Response Header immediately follows the DDP header. The RDMAP layer passes to the DDP layer a RDMAP Control Field. The following figure depicts the Atomic Operation Response header that is used for all Atomic Operation Response Messages:
原子操作响应消息携带一个包含“原始请求标识符”和“原始远程数据值”的原子操作响应头。原子操作响应头紧跟在DDP头之后。RDMAP层将RDMAP控制字段传递给DDP层。下图描述了用于所有原子操作响应消息的原子操作响应标头：
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Original Request Identifier |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Original Remote Data Value |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Original Request Identifier |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Original Remote Data Value |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 6: Atomic Operation Response Header
图6：原子操作响应头
Original Request Identifier: 32 bits.
原始请求标识符：32位。
The Original Request Identifier is set to the value specified in the Request Identifier field that was originally provided in the corresponding Atomic Operation Request Message.
原始请求标识符设置为请求标识符字段中指定的值，该字段最初在相应的原子操作请求消息中提供。
Original Remote Data Value: 64 bits.
原始远程数据值：64位。
The Original Remote Value specifies the original 64-bit value stored at the ULP Buffer address targeted by the Atomic Operation.
原始远程值指定存储在原子操作目标ULP缓冲区地址的原始64位值。
The Atomic Operation Response Message has the following semantics:
原子操作响应消息具有以下语义：
1. The Atomic Operation Response Message for the associated Atomic Operation Request Message travels in the opposite direction.
1. 关联的原子操作请求消息的原子操作响应消息以相反方向移动。
2. An Atomic Operation Response Message MUST consume an Untagged Buffer. That is, the Responder RDMAP layer MUST request that the DDP mark the Message as Untagged.
2. 原子操作响应消息必须使用未标记的缓冲区。也就是说，响应程序RDMAP层必须请求DDP将消息标记为未标记。
3. An Atomic Operation Response Message MUST reference the Queue Number 3. That is, the Responder's RDMAP layer MUST request that the DDP layer set the Queue Number field to 3.
3. 原子操作响应消息必须引用队列号3。也就是说，响应者的RDMAP层必须请求DDP层将队列号字段设置为3。
4. The Responder MUST ensure that a sufficient number of Untagged Buffers are available on the RDMA Read Request Queue (Queue with DDP Queue Number 1) to support the maximum number of Atomic Operation Requests negotiated by the ULP in addition to the maximum number of RDMA Read Requests negotiated by the ULP.
4. 响应者必须确保RDMA读取请求队列（DDP队列号为1的队列）上有足够数量的未标记缓冲区，以支持ULP协商的最大原子操作请求数以及ULP协商的最大RDMA读取请求数。
5. The Requester MUST ensure that a sufficient number of Untagged Buffers are available on the RDMA Atomic Response Queue (Queue with DDP Queue Number 3) to support the maximum number of Atomic Operation Requests negotiated by the ULP.
5. 请求者必须确保RDMA原子响应队列（DDP队列号为3的队列）上有足够数量的未标记缓冲区，以支持ULP协商的最大原子操作请求数。
6. The RDMAP layer MUST Deliver the Atomic Operation Response Message to the ULP.
6. RDMAP层必须将原子操作响应消息传递给ULP。
7. At the Requester, when an invalid Atomic Operation Response Message is delivered to the Remote Peer's RDMAP layer, an error is surfaced.
7. 在请求方，当一个无效的原子操作响应消息被传递到远程对等方的RDMAP层时，就会出现一个错误。
8. When the Responder receives Atomic Operation Request messages, the Responder RDMAP layer MUST pass Atomic Operation Response Messages to the DDP layer, in the order that the Atomic Operation Request Messages were received by the RDMAP layer, at the Responder.
8. 当响应者接收原子操作请求消息时，响应者RDMAP层必须按照RDMAP层在响应者处接收原子操作请求消息的顺序，将原子操作响应消息传递给DDP层。
Atomicity of the Read-Modify-Write (RMW) on the Responder's node by the Atomic Operation MUST be assured in the context of concurrent atomic accesses by other RDMAP Streams on the same RNIC.
在同一RNIC上的其他RDMAP流并发原子访问的上下文中，必须确保原子操作在响应程序节点上的读修改写（RMW）的原子性。
In addition to the ordering and completion rules described in RFC 5040, the following rules apply to implementations of the Atomic Operations.
除了RFC 5040中描述的排序和完成规则外，以下规则适用于原子操作的实现。
1. For an Atomic Operation, the Requester MUST NOT consider the contents of the Tagged Buffer at the Responder to be modified by that specific Atomic Operation until the Atomic Operation Response Message has been Delivered to RDMAP at the Requester.
1. 对于原子操作，Requester必须不考虑响应者的标记缓冲器的内容，通过特定的原子操作来修改，直到原子操作响应消息已被递送到Requester的RDMAP。
2. Atomicity guarantees MUST be provided within the scope of a single RNIC.
2. 原子性保证必须在单个RNIC的范围内提供。
Implementation Note: This requirement for atomicity among operations is limited to the scope of a single RNIC. Atomicity guarantees are OPTIONAL with respect to access to the Tagged Buffer by any other method than an Atomic Operation via the same RNIC. Examples of such accesses that may not be atomic with respect to an Atomic Operation include accesses via other RNICs and local processor memory access to the Tagged Buffer.
实施说明：操作之间的原子性要求仅限于单个RNIC的范围。原子性保证对于通过同一RNIC的原子操作以外的任何其他方法访问标记缓冲区是可选的。对于原子操作而言可能不是原子的此类访问的示例包括通过其他RNIC的访问和对标记缓冲区的本地处理器内存访问。
3. Atomic Operation Request Messages MUST NOT start processing at the Responder until they have been Delivered to RDMAP by DDP.
3. 在DDP将原子操作请求消息发送到RDMAP之前，不得在响应程序上开始处理这些消息。
4. Atomic Operation Response Messages MAY be generated at the Responder after subsequent RDMA Write Messages or Send Messages have been Placed or Delivered.
4. 原子操作响应消息可在后续RDMA写入消息或发送消息放置或交付后在响应者处生成。
5. Atomic Operation Response Message processing at the Responder MUST be started only after the Atomic Operation Request Message has been Delivered by the DDP layer (thus, all previous RDMA Messages on that DDP Stream have been Delivered).
5. 只有在DDP层传递了原子操作请求消息（因此，该DDP流上以前的所有RDMA消息都已传递）之后，才能在响应者处启动原子操作响应消息处理。
6. Send Messages MAY be Completed at the Responder before prior incoming Atomic Operation Request Messages have completed their response processing.
6. 在先前传入的原子操作请求消息完成其响应处理之前，可以在响应者处完成发送消息。
7. An Atomic Operation MUST NOT be Completed at the Requester until the DDP layer Delivers the associated incoming Atomic Operation Response Message.
7. 在DDP层交付相关的传入原子操作响应消息之前，不得在请求者处完成原子操作。
8. If more than one outstanding Atomic Request Message is supported by both peers, the Atomic Operation Request Messages MUST be processed in the order they were delivered by the DDP layer on the Responder. Atomic Operation Response Messages MUST be submitted to the DDP layer on the Responder in the order the Atomic Operation Request Messages were Delivered by DDP.
8. 如果两个对等方都支持多个未完成的原子请求消息，则必须按照DDP层在响应方上传递的顺序处理原子操作请求消息。原子操作响应消息必须按照DDP传递原子操作请求消息的顺序提交到响应程序上的DDP层。
The Immediate Data operation is typically used in conjunction with an RDMA Write Operation to improve ULP processing efficiency. The efficiency is gained by causing an RDMA Completion to be generated immediately following the RDMA Write operation. This RDMA Completion delivers 8 bytes of Immediate Data at the Remote Peer. The combination of an RDMA Write Message followed by an Immediate Data Operation has the same behavior as the RDMA Write with Immediate Data operation found in InfiniBand. An Immediate Data operation that is not preceded by an RDMA Write operation causes an RDMA Completion.
即时数据操作通常与RDMA写入操作结合使用，以提高ULP处理效率。效率是通过在RDMA写入操作之后立即生成RDMA完成来实现的。此RDMA完成在远程对等端提供8字节的即时数据。RDMA写入消息和立即数据操作的组合与InfiniBand中的RDMA写入立即数据操作具有相同的行为。未在RDMA写入操作之前执行的立即数据操作会导致RDMA完成。
For Immediate Data operations, the following are the interactions between the RDMAP Layer and the ULP:
对于即时数据操作，以下是RDMAP层和ULP之间的交互：
o At the Data Source:
o 在数据源：
- The ULP passes to the RDMAP Layer the following:
- ULP通过以下方式传递到RDMAP层：
* 8 bytes of ULP Immediate Data
* 8字节的ULP即时数据
- When the Immediate Data operation Completes, an indication of the Completion results.
- 即时数据操作完成时，显示完成结果。
o At the Data Sink:
o 在数据接收器处：
- If the Immediate Data operation is Completed successfully, the RDMAP Layer passes the following information to the ULP Layer:
- 如果立即数据操作成功完成，RDMAP层会将以下信息传递给ULP层：
* 8 bytes of Immediate Data
* 8字节的即时数据
* An Event, if the Data Sink is configured to generate an Event.
* 如果数据接收器配置为生成事件，则为事件。
- If the Immediate Data operation is Completed in error, the Data Sink RDMAP Layer will pass up the corresponding error information to the Data Sink ULP and send a Terminate Message to the Data Source RDMAP Layer. The Data Source RDMAP Layer will then pass up the Terminate Message to the ULP.
- 如果立即数据操作错误完成，数据接收器RDMAP层将向数据接收器ULP传递相应的错误信息，并向数据源RDMAP层发送终止消息。然后，数据源RDMAP层将终止消息传递给ULP。
The Immediate Data and Immediate Data with SE Messages carry Immediate Data as shown in Figure 7. The RDMAP layer passes to the DDP layer an RDMAP Control Field and 8 bytes of Immediate Data. The first 8 bytes of the data following the DDP header contains the Immediate Data. See Appendix A.3 for the DDP segment format of an Immediate Data or Immediate Data with SE Message.
即时数据和带有SE消息的即时数据携带即时数据，如图7所示。RDMAP层向DDP层传递一个RDMAP控制字段和8字节的即时数据。DDP头后面的数据的前8个字节包含即时数据。即时数据或带有SE报文的即时数据的DDP段格式见附录A.3。
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Immediate Data |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Immediate Data |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 7: Immediate Data or Immediate Data with SE Message Header
图7：即时数据或带有SE消息头的即时数据
Immediate Data: 64 bits. 8 bytes of data transferred from the Data Source to an untagged Buffer at the Data Sink.
即时数据：64位。8字节的数据从数据源传输到数据接收器的未标记缓冲区。
The Immediate Data or Immediate Data with SE Message uses the DDP Untagged Buffer Model to transfer Immediate Data from the Data Source to the Data Sink.
即时数据或带有SE消息的即时数据使用DDP未标记缓冲区模型将即时数据从数据源传输到数据接收器。
o An Immediate Data or Immediate Data with SE Message MUST reference an Untagged Buffer. That is, the Local Peer's RDMAP Layer MUST request that the DDP layer mark the Message as Untagged.
o 即时数据或带有SE消息的即时数据必须引用未标记的缓冲区。也就是说，本地对等方的RDMAP层必须请求DDP层将消息标记为未标记。
o One Immediate Data or Immediate Data with SE Message MUST consume one Untagged Buffer.
o 一个即时数据或带有SE消息的即时数据必须使用一个未标记的缓冲区。
o At the Remote Peer, the Immediate Data and Immediate Data with SE Messages MUST be Delivered to the Remote Peer's ULP in the order they were sent.
o 在远程对等方，即时数据和带有SE消息的即时数据必须按照发送顺序发送到远程对等方的ULP。
o For an Immediate Data or Immediate Data with SE Message, the Local Peer's RDMAP Layer MUST request that the DDP layer set the Queue Number field to zero.
o 对于即时数据或带有SE消息的即时数据，本地对等方的RDMAP层必须请求DDP层将队列号字段设置为零。
o For an Immediate Data or Immediate Data with SE Message, the Local Peer's RDMAP Layer MUST request that the DDP layer transmit 8 bytes of data.
o 对于即时数据或带有SE消息的即时数据，本地对等方的RDMAP层必须请求DDP层传输8字节的数据。
o The Local Peer MUST issue Immediate Data and Immediate Data with SE Messages in the order they were submitted by the ULP.
o 本地对等方必须按照ULP提交的顺序发布即时数据和带有SE消息的即时数据。
o The Remote Peer MUST check that Immediate Data and Immediate Data with SE Messages include exactly 8 bytes of data from the DDP layer. The DDP header carries the length field that is reported by the DDP layer.
o 远程对等方必须检查即时数据和带有SE消息的即时数据是否包含来自DDP层的8字节数据。DDP头包含DDP层报告的长度字段。
Ordering and completion rules for Immediate Data are the same as those for a Send operation as described in Section 5.5 of RFC 5040.
即时数据的排序和完成规则与RFC 5040第5.5节中描述的发送操作的规则相同。
The following table summarizes the ordering relationships for Atomic and Immediate Data operations from the standpoint of the Local Peer issuing the Operations. Note that in the table that follows, Send includes Send, Send with Invalidate, Send with Solicited Event, and Send with Solicited Event and Invalidate. Also note that in the table below, Immediate Data includes Immediate Data and Immediate Data with Solicited Event.
下表从发出操作的本地对等方的角度总结了原子和即时数据操作的顺序关系。请注意，在下表中，Send包括Send、Send with Invalidate、Send with required Event以及Send with required Event and Invalidate。还要注意，在下表中，即时数据包括即时数据和请求事件的即时数据。
---------+----------+-------------+-------------+------------------
First | Second | Placement | Placement | Ordering
Operation| Operation| Guarantee at| Guarantee at| Guarantee at
| | Remote Peer | Local Peer | Remote Peer
---------+----------+-------------+-------------+------------------
Immediate| Send | No Placement| Not | Completed in
Data | | Guarantee | Applicable | Order
| | between Send| |
| | Payload and | |
| | Immediate | |
| | Data | |
---------+----------+-------------+-------------+------------------
Immediate| RDMA | No Placement| Not | Not
Data | Write | Guarantee | Applicable | Applicable
| | between RDMA| |
| | Write | |
| | Payload and | |
| | Immediate | |
| | Data | |
---------+----------+-------------+-------------+------------------
First | Second | Placement | Placement | Ordering
Operation| Operation| Guarantee at| Guarantee at| Guarantee at
| | Remote Peer | Local Peer | Remote Peer
---------+----------+-------------+-------------+------------------
Immediate| Send | No Placement| Not | Completed in
Data | | Guarantee | Applicable | Order
| | between Send| |
| | Payload and | |
| | Immediate | |
| | Data | |
---------+----------+-------------+-------------+------------------
Immediate| RDMA | No Placement| Not | Not
Data | Write | Guarantee | Applicable | Applicable
| | between RDMA| |
| | Write | |
| | Payload and | |
| | Immediate | |
| | Data | |
---------+----------+-------------+-------------+------------------
Immediate| RDMA | No Placement| RDMA Read | RDMA Read
Data | Read | Guarantee | Response | Response
| | between | will not be | Message will
| | Immediate | Placed until| not be
| | Data and | Immediate | generated
| | RDMA Read | Data is | until
| | Request | Placed at | Immediate Data
| | | Remote Peer | has been
| | | | Completed
---------+----------+-------------+-------------+------------------
Immediate| Atomic | No Placement| Atomic | Atomic
Data | | Guarantee | Response | Response
| | between | will not be | Message will
| | Immediate | Placed until| not be
| | Data and | Immediate | generated
| | Atomic | Data is | until
| | Request | Placed at | Immediate Data
| | | Remote Peer | has been
| | | | Completed
---------+----------+-------------+-------------+------------------
Immediate| Immediate| No Placement| Not | Completed in
Data or | Data | Guarantee | Applicable | Order
Send | | | |
---------+----------+-------------+-------------+------------------
RDMA | Immediate| No Placement| Not | Immediate Data
Write | Data | Guarantee | Applicable | is Completed
| | | | after RDMA
| | | | Write is Placed
| | | | and Delivered
---------+----------+-------------+-------------+------------------
RDMA Read| Immediate| No Placement| Immediate | Not Applicable
| Data | Guarantee | Data MAY be |
| | between | Placed |
| | Immediate | before |
| | Data and | RDMA Read |
| | RDMA Read | Response is |
| | Request | generated |
---------+----------+-------------+-------------+------------------
Atomic | Immediate| No Placement| Immediate | Not Applicable
| Data | Guarantee | Data MAY be |
| | between | Placed |
| | Immediate | before |
| | Data and | Atomic |
| | Atomic | Response is |
| | Request | generated |
---------+----------+-------------+-------------+------------------
Immediate| RDMA | No Placement| RDMA Read | RDMA Read
Data | Read | Guarantee | Response | Response
| | between | will not be | Message will
| | Immediate | Placed until| not be
| | Data and | Immediate | generated
| | RDMA Read | Data is | until
| | Request | Placed at | Immediate Data
| | | Remote Peer | has been
| | | | Completed
---------+----------+-------------+-------------+------------------
Immediate| Atomic | No Placement| Atomic | Atomic
Data | | Guarantee | Response | Response
| | between | will not be | Message will
| | Immediate | Placed until| not be
| | Data and | Immediate | generated
| | Atomic | Data is | until
| | Request | Placed at | Immediate Data
| | | Remote Peer | has been
| | | | Completed
---------+----------+-------------+-------------+------------------
Immediate| Immediate| No Placement| Not | Completed in
Data or | Data | Guarantee | Applicable | Order
Send | | | |
---------+----------+-------------+-------------+------------------
RDMA | Immediate| No Placement| Not | Immediate Data
Write | Data | Guarantee | Applicable | is Completed
| | | | after RDMA
| | | | Write is Placed
| | | | and Delivered
---------+----------+-------------+-------------+------------------
RDMA Read| Immediate| No Placement| Immediate | Not Applicable
| Data | Guarantee | Data MAY be |
| | between | Placed |
| | Immediate | before |
| | Data and | RDMA Read |
| | RDMA Read | Response is |
| | Request | generated |
---------+----------+-------------+-------------+------------------
Atomic | Immediate| No Placement| Immediate | Not Applicable
| Data | Guarantee | Data MAY be |
| | between | Placed |
| | Immediate | before |
| | Data and | Atomic |
| | Atomic | Response is |
| | Request | generated |
---------+----------+-------------+-------------+------------------
Atomic | Send | No Placement| Send Payload| Not Applicable
| | Guarantee | MAY be |
| | between Send| Placed |
| | Payload and | before |
| | Atomic | Atomic |
| | Request | Response is |
| | | generated |
---------+----------+-------------+-------------+------------------
Atomic | RDMA | No Placement| RDMA Write | Not
| Write | Guarantee | Payload MAY | Applicable
| | between RDMA| be Placed |
| | Write | before |
| | Payload and | Atomic |
| | Atomic | Response is |
| | Request | generated |
---------+----------+-------------+-------------+------------------
Atomic | RDMA | No Placement| No Placement| RDMA Read
| Read | Guarantee | Guarantee | Response
| | between | between | Message will
| | Atomic | Atomic | not be
| | Request and | Response | generated
| | RDMA Read | and RDMA | until Atomic
| | Request | Read | Response Message
| | | Response | has been
| | | | generated
---------+----------+-------------+-------------+------------------
Atomic | Atomic | Placed in | No Placement| Second Atomic
| | order | Guarantee | Request
| | | between two | Message will
| | | Atomic | not be
| | | Responses | processed
| | | | until first
| | | | Atomic Response
| | | | has been
| | | | generated
---------+----------+-------------+-------------+------------------
Send | Atomic | No Placement| Atomic | Atomic Response
| | Guarantee | Response | Message will not
| | between Send| will not be | be generated
| | Payload and | Placed at | until Send has
| | Atomic | the Local | been Completed
| | Request | Peer until |
| | | Send Payload|
| | | is Placed |
| | | at the |
| | | Remote Peer |
---------+----------+-------------+-------------+------------------
Atomic | Send | No Placement| Send Payload| Not Applicable
| | Guarantee | MAY be |
| | between Send| Placed |
| | Payload and | before |
| | Atomic | Atomic |
| | Request | Response is |
| | | generated |
---------+----------+-------------+-------------+------------------
Atomic | RDMA | No Placement| RDMA Write | Not
| Write | Guarantee | Payload MAY | Applicable
| | between RDMA| be Placed |
| | Write | before |
| | Payload and | Atomic |
| | Atomic | Response is |
| | Request | generated |
---------+----------+-------------+-------------+------------------
Atomic | RDMA | No Placement| No Placement| RDMA Read
| Read | Guarantee | Guarantee | Response
| | between | between | Message will
| | Atomic | Atomic | not be
| | Request and | Response | generated
| | RDMA Read | and RDMA | until Atomic
| | Request | Read | Response Message
| | | Response | has been
| | | | generated
---------+----------+-------------+-------------+------------------
Atomic | Atomic | Placed in | No Placement| Second Atomic
| | order | Guarantee | Request
| | | between two | Message will
| | | Atomic | not be
| | | Responses | processed
| | | | until first
| | | | Atomic Response
| | | | has been
| | | | generated
---------+----------+-------------+-------------+------------------
Send | Atomic | No Placement| Atomic | Atomic Response
| | Guarantee | Response | Message will not
| | between Send| will not be | be generated
| | Payload and | Placed at | until Send has
| | Atomic | the Local | been Completed
| | Request | Peer until |
| | | Send Payload|
| | | is Placed |
| | | at the |
| | | Remote Peer |
---------+----------+-------------+-------------+------------------
RDMA | Atomic | No Placement| Atomic | Not
Write | | Guarantee | Response | Applicable
| | between RDMA| will not be |
| | Write | Placed at |
| | Payload and | the Local |
| | Atomic | Peer until |
| | Request | RDMA Write |
| | | Payload |
| | | is Placed |
| | | at the |
| | | Remote Peer |
---------+----------+-------------+-------------+------------------
RDMA | Atomic | No Placement| No Placement| Atomic Response
Read | | Guarantee | Guarantee | Message will
| | between | between | not be generated
| | Atomic | Atomic | until RDMA
| | Request and | Response | Read Response
| | RDMA Read | and RDMA | has been
| | Request | Read | generated
| | | Response |
---------+----------+-------------+-------------+------------------
---------+----------+-------------+-------------+------------------
RDMA | Atomic | No Placement| Atomic | Not
Write | | Guarantee | Response | Applicable
| | between RDMA| will not be |
| | Write | Placed at |
| | Payload and | the Local |
| | Atomic | Peer until |
| | Request | RDMA Write |
| | | Payload |
| | | is Placed |
| | | at the |
| | | Remote Peer |
---------+----------+-------------+-------------+------------------
RDMA | Atomic | No Placement| No Placement| Atomic Response
Read | | Guarantee | Guarantee | Message will
| | between | between | not be generated
| | Atomic | Atomic | until RDMA
| | Request and | Response | Read Response
| | RDMA Read | and RDMA | has been
| | Request | Read | generated
| | | Response |
---------+----------+-------------+-------------+------------------
In addition to the error processing described in Section 7 of RFC 5040, the following rules apply for the new RDMA Messages defined in this specification.
除了RFC 5040第7节中描述的错误处理外，以下规则适用于本规范中定义的新RDMA消息。
The Local Peer MUST send a Terminate Message for each of the following cases:
对于以下情况，本地对等方必须发送终止消息：
1. For errors detected while creating an Atomic Request, Atomic Response, Immediate Data, or Immediate Data with SE Message, or other reasons not directly associated with an incoming Message, the Terminate Message and Error code are sent instead of the Message. In this case, the Error Type and Error Code fields are included in the Terminate Message, but the Terminated DDP Header and Terminated RDMA Header fields are set to zero.
1. 对于在创建原子请求、原子响应、即时数据或带有SE消息的即时数据时检测到的错误，或与传入消息不直接相关的其他原因，将发送终止消息和错误代码，而不是消息。在这种情况下，终止消息中包括错误类型和错误代码字段，但终止的DDP头和终止的RDMA头字段设置为零。
2. For errors detected on an incoming Atomic Request, Atomic Response, Immediate Data, or Immediate Data with SE (after the Message has been Delivered by DDP), the Terminate Message is sent at the earliest possible opportunity, preferably in the next
2. 对于在传入的原子请求、原子响应、即时数据或SE的即时数据上检测到的错误（在DDP交付消息之后），在尽可能早的时间（最好是在下一个时间）发送终止消息
outgoing RDMA Message. In this case, the Error Type, Error Code, and Terminated DDP Header fields are included in the Terminate Message, but the Terminated RDMA Header field is set to zero.
传出RDMA消息。在这种情况下，终止消息中包括错误类型、错误代码和终止的DDP头字段，但终止的RDMA头字段设置为零。
On incoming Atomic Requests, Atomic Responses, Immediate Data, and Immediate Data with Solicited Event, the following MUST be validated:
对于传入的原子请求、原子响应、即时数据和带有请求事件的即时数据，必须验证以下内容：
o The DDP layer MUST validate all DDP Segment fields.
o DDP层必须验证所有DDP段字段。
o The RDMA OpCode MUST be valid.
o RDMA操作码必须有效。
o The RDMA Version MUST be valid.
o RDMA版本必须有效。
On incoming Atomic requests the following additional validation MUST be performed:
对于传入的原子请求，必须执行以下附加验证：
o The RDMAP layer MUST validate that the Remote Peer's Tagged ULP Buffer address references a ULP Buffer address that is 64-bit aligned. In the case of an error, the RDMAP layer MUST generate a Terminate Message indicating RDMA Layer Remote Operation Error with Error Code Name "Catastrophic error, localized to RDMAP Stream" as described in Section 4.8 of RFC 5040. Implementation Note: A ULP implementation can avoid this error by having the target ULP Buffer of an Atomic Operation 64-bit aligned.
o RDMAP层必须验证远程对等方的标记ULP缓冲区地址是否引用64位对齐的ULP缓冲区地址。如果发生错误，RDMAP层必须生成一条终止消息，指示RDMA层远程操作错误，错误代码名为“灾难性错误，本地化为RDMAP流”，如RFC 5040第4.8节所述。实现说明：ULP实现可以通过使原子操作的目标ULP缓冲区64位对齐来避免此错误。
This document specifies extensions to the RDMA Protocol specification in RFC 5040, and as such the Security Considerations discussed in Section 8 of RFC 5040 apply. In particular, Atomic Operations use ULP Buffer addresses for the Remote Peer Buffer addressing used in RFC 5040 as required by the security model described in RFC 5042 [RFC5042].
本文件规定了RFC 5040中RDMA协议规范的扩展，因此RFC 5040第8节中讨论的安全注意事项适用。具体而言，原子操作根据RFC 5042[RFC5042]中描述的安全模型的要求，将ULP缓冲区地址用于RFC 5040中使用的远程对等缓冲区寻址。
RDMAP and related protocols may be used by applications that exhibit distinctive traffic characteristics such as message timing, source, destination, and size patterns. Examples include structured high-performance computing applications based on the MPI interface. For such applications, analysis of encrypted traffic could reveal sensitive information, e.g., the nature of the application, size of data set being used, and information about the application's rate of progress. Such information can be hidden from passive observation via use of Encapsulating Security Payload version 3 (ESPv3) Traffic Flow Confidentiality [RFC4303] to obfuscate the encrypted traffic's characteristics. ESPv3 implementation requirements for RDMAP are specified in [RFC7146].
RDMAP和相关协议可由表现出独特流量特征（如消息定时、源、目的地和大小模式）的应用程序使用。示例包括基于MPI接口的结构化高性能计算应用程序。对于此类应用程序，对加密流量的分析可能会揭示敏感信息，例如，应用程序的性质、使用的数据集的大小以及有关应用程序进度的信息。通过使用封装安全有效负载版本3（ESPv3）流量机密性[RFC4303]来混淆加密流量的特征，可以隐藏此类信息，避免被动观察。[RFC7146]中规定了RDMAP的ESPv3实施要求。
IANA has added the following entries to the "RDMAP Message Operation Codes" registry of "Remote Direct Data Placement (RDDP)" registry:
IANA已将以下条目添加到“远程直接数据放置（RDDP）”注册表的“RDMAP消息操作代码”注册表中：
0x8, Immediate Data, this specification
0x8，即时数据，此规范
0x9, Immediate Data with Solicited Event, this specification
0x9，请求事件的即时数据，此规范
0xA, Atomic Request, this specification
0xA，原子请求，此规范
0xB, Atomic Response, this specification
0xB，原子响应，此规范
In addition, the following registry has been added to the "Remote Direct Data Placement (RDDP)" registry. The following section specifies the registry, its initial contents, and the administration policy in more detail.
此外，以下注册表已添加到“远程直接数据放置（RDDP）”注册表中。以下部分详细说明了注册表、其初始内容和管理策略。
Name of the registry: "RDMAP Message Atomic Operation Subcodes"
注册表名称：“RDMAP消息原子操作子代码”
Namespace details: RDMAP Message Atomic Operation Subcodes are 4-bit values.
命名空间详细信息：RDMAP消息原子操作子代码是4位值。
Information that must be provided to assign a new value: An IESG-approved Standards Track specification defining the semantics and interoperability requirements of the proposed new value and the fields to be recorded in the registry.
分配新值必须提供的信息：IESG批准的标准跟踪规范，定义了拟议新值的语义和互操作性要求以及要记录在注册表中的字段。
Fields to record in the registry: RDMAP Message Atomic Operation Subcode, Atomic Operation, RFC Reference.
要在注册表中记录的字段：RDMAP消息原子操作子代码、原子操作、RFC引用。
Initial registry contents:
初始注册表内容：
0x0, FetchAdd, this specification
0x0，FetchAdd，此规范
0x1, Reserved, this specification
0x1，保留，此规范
0x2, CmpSwap, this specification
0x2，CmpSwap，本规范
Note: An experimental RDMAP Message Operation Code has already been allocated; hence, there is no need for an experimental RDMAP Message Atomic Operation Subcode.
注：已分配实验性RDMAP消息操作代码；因此，不需要实验性的RDMAP消息原子操作子代码。
All other values are Unassigned and available to IANA for assignment. New RDMAP Message Atomic Operation Subcodes should be assigned sequentially in order to better support implementations that process RDMAP Message Atomic Operations in hardware.
所有其他值均未分配，IANA可用于分配。新的RDMAP消息原子操作子代码应该按顺序分配，以便更好地支持在硬件中处理RDMAP消息原子操作的实现。
Allocation Policy: Standards Action [RFC5226]
分配政策：标准行动[RFC5226]
Name of the registry: "RDMAP DDP Untagged Queue Numbers"
注册表名称：“RDMAP DDP未标记队列号”
Namespace details: RDMAP DDP Untagged Queue numbers are 32-bit values.
命名空间详细信息：RDMAP DDP未标记的队列号是32位值。
Information that must be provided to assign a new value: An IESG-approved Standards Track specification defining the semantics and interoperability requirements of the proposed new value and the fields to be recorded in the registry.
分配新值必须提供的信息：IESG批准的标准跟踪规范，定义了拟议新值的语义和互操作性要求以及要记录在注册表中的字段。
Fields to record in the registry: RDMAP DDP Untagged Queue Numbers, Queue Usage Description, RFC Reference.
要在注册表中记录的字段：RDMAP DDP未标记的队列号、队列使用说明、RFC参考。
Initial registry contents:
初始注册表内容：
0x00000000, Queue 0 (Send operation Variants), [RFC5040]
0x00000000，队列0（发送操作变体），[RFC5040]
0x00000001, Queue 1 (RDMA Read Request operations), [RFC5040]
0x00000001，队列1（RDMA读取请求操作），[RFC5040]
0x00000002, Queue 2 (Terminate operations), [RFC5040]
0x00000002，队列2（终止操作），[RFC5040]
0x00000003, Queue 3 (Atomic Response operations), this specification
0x00000003，队列3（原子响应操作），此规范
Note: An experimental RDMAP Message Operation Code has already been allocated; hence, there is no need for an experimental RDMAP DDP Untagged Queue Number.
注：已分配实验性RDMAP消息操作代码；因此，不需要实验性的RDMAP DDP未标记队列号。
All other values are Unassigned and available to IANA for assignment. New RDMAP queue numbers should be assigned sequentially in order to better support implementations that perform RDMAP queue selection in hardware.
所有其他值均未分配，IANA可用于分配。应按顺序分配新的RDMAP队列号，以便更好地支持在硬件中执行RDMAP队列选择的实现。
Allocation Policy: Standards Action [RFC5226]
分配政策：标准行动[RFC5226]
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC2119]Bradner，S.，“RFC中用于表示需求水平的关键词”，BCP 14，RFC 2119，1997年3月。
[RFC4303] Kent, S., "IP Encapsulating Security Payload (ESP)", RFC 4303, December 2005.
[RFC4303]Kent，S.，“IP封装安全有效载荷（ESP）”，RFC 4303，2005年12月。
[RFC5040] Recio, R., Metzler, B., Culley, P., Hilland, J., and D. Garcia, "A Remote Direct Memory Access Protocol Specification", RFC 5040, October 2007.
[RFC5040]Recio，R.，Metzler，B.，Culley，P.，Hilland，J.，和D.Garcia，“远程直接内存访问协议规范”，RFC 50402007年10月。
[RFC5041] Shah, H., Pinkerton, J., Recio, R., and P. Culley, "Direct Data Placement over Reliable Transports", RFC 5041, October 2007.
[RFC5041]Shah，H.，Pinkerton，J.，Recio，R.，和P.Culley，“可靠传输上的直接数据放置”，RFC 50412007年10月。
[RFC5042] Pinkerton, J. and E. Deleganes, "Direct Data Placement Protocol (DDP) / Remote Direct Memory Access Protocol (RDMAP) Security", RFC 5042, October 2007.
[RFC5042]Pinkerton，J.和E.Deleganes，“直接数据放置协议（DDP）/远程直接内存访问协议（RDMAP）安全”，RFC 50422007年10月。
[RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 5226, May 2008.
[RFC5226]Narten，T.和H.Alvestrand，“在RFCs中编写IANA注意事项部分的指南”，BCP 26，RFC 5226，2008年5月。
[RFC7146] Black, D. and P. Koning, "Securing Block Storage Protocols over IP: RFC 3723 Requirements Update for IPsec v3", RFC 7146, April 2014.
[RFC7146]Black，D.和P.Koning，“通过IP保护块存储协议：IPsec v3的RFC 3723要求更新”，RFC 7146，2014年4月。
[DAT_ATOMICS] DAT Collaborative, "IB Transport Specific Extensions for DAT 2.0", User Direct Access Programming Library, <http://www.datcollaborative.org/DAT_IB_Extensions.pdf>.
[DAT_ATOMICS]DAT Collaborative，“针对DAT 2.0的IB传输特定扩展”，用户直接访问编程库<http://www.datcollaborative.org/DAT_IB_Extensions.pdf>.
[IB] InfiniBand Trade Association, "InfiniBand Architecture Specification Volumes 1 and 2", Release 1.1, November 2002, <http://www.infinibandta.org/specs>.
[IB]InfiniBand贸易协会，“InfiniBand体系结构规范第1卷和第2卷”，1.11902年11月发行<http://www.infinibandta.org/specs>.
[MPI] Message Passing Interface Forum, "MPI: A Message-Passing Interface Standard, Version 3.0", September 2012, <http://www.mpi-forum.org/docs/mpi-3.0/mpi30-report.pdf>.
[MPI]消息传递接口论坛，“MPI:消息传递接口标准，版本3.0”，2012年9月<http://www.mpi-forum.org/docs/mpi-3.0/mpi30-report.pdf>.
[OFAVERBS] Rosenstock, H., "Subject: Re: [PATCH 0/2] Add support for enhanced atomic operations", message to the linux-rdma mailing list, <http://www.spinics.net/lists/linux-rdma/msg02405.html>.
[OFAVERBS]Rosenstock，H.，“主题：Re:[PATCH 0/2]添加对增强原子操作的支持”，发给linux rdma邮件列表的消息<http://www.spinics.net/lists/linux-rdma/msg02405.html>.
[RFC5044] Culley, P., Elzur, U., Recio, R., Bailey, S., and J. Carrier, "Marker PDU Aligned Framing for TCP Specification", RFC 5044, October 2007.
[RFC5044]Culley，P.，Elzur，U.，Recio，R.，Bailey，S.，和J.Carrier，“TCP规范的标记PDU对齐帧”，RFC 5044，2007年10月。
[RFC5045] Bestler, C., Ed., and L. Coene, "Applicability of Remote Direct Memory Access Protocol (RDMA) and Direct Data Placement (DDP)", RFC 5045, October 2007.
[RFC5045]Bestler，C.，Ed.，和L.Coene，“远程直接内存访问协议（RDMA）和直接数据放置（DDP）的适用性”，RFC 5045，2007年10月。
[RFC6581] Kanevsky, A., Ed., Bestler, C., Ed., Sharp, R., and S. Wise, "Enhanced Remote Direct Memory Access (RDMA) Connection Establishment", RFC 6581, April 2012.
[RFC6581]Kanevsky，A.，Ed.，Bestler，C.，Ed.，Sharp，R.，和S.Wise，“增强型远程直接内存访问（RDMA）连接建立”，RFC 65812012年4月。
[RSOCKETS] Hefty, S., "RDMA CM - RDMA enabled Sockets library for Open Fabrics", <http://git.openfabrics.org/?p=~shefty/ librdmacm.git;a=summary>.
[RSOCKETS]Hefty，S.，“开放结构的RDMA CM-支持RDMA的套接字库”<http://git.openfabrics.org/?p=~shefty/librdmacm.git；a=摘要>。
The authors would like to acknowledge the following individuals who provided valuable comments and suggestions.
作者要感谢以下提供了宝贵意见和建议的个人。
o David Black
o 大卫·布莱克
o Arkady Kanevsky
o 阿卡迪·卡内夫斯基
o Bernard Metzler
o 伯纳德·梅茨勒
o Jim Pinkerton
o 吉姆
o Tom Talpey
o 汤姆塔尔佩
o Steve Wise
o 史蒂夫·怀斯
o Don Wood
o 唐伍德
This appendix is for information only and is NOT part of the standard. It simply depicts the DDP Segment format for the various RDMA Messages.
本附录仅供参考，不属于本标准的一部分。它简单地描述了各种RDMA消息的DDP段格式。
The following figure depicts an Atomic Operation Request, DDP Segment:
下图描述了原子操作请求DDP段：
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DDP Control | RDMA Control |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reserved (Not Used) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DDP (Atomic Operation Request) Queue Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DDP (Atomic Operation Request) Message Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DDP (Atomic Operation Request) Message Offset |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reserved (Not Used) |AOpCode|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Request Identifier |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Remote STag |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Remote Tagged Offset |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Add or Swap Data |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Add or Swap Mask |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Compare Data |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Compare Mask |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DDP Control | RDMA Control |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reserved (Not Used) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DDP (Atomic Operation Request) Queue Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DDP (Atomic Operation Request) Message Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DDP (Atomic Operation Request) Message Offset |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reserved (Not Used) |AOpCode|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Request Identifier |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Remote STag |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Remote Tagged Offset |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Add or Swap Data |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Add or Swap Mask |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Compare Data |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Compare Mask |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
The following figure depicts an Atomic Operation Response, DDP Segment:
下图描述了原子操作响应DDP段：
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DDP Control | RDMA Control |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reserved (Not Used) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DDP (Atomic Operation Request) Queue Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DDP (Atomic Operation Request) Message Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DDP (Atomic Operation Request) Message Offset |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Original Request Identifier |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Original Remote Value |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DDP Control | RDMA Control |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reserved (Not Used) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DDP (Atomic Operation Request) Queue Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DDP (Atomic Operation Request) Message Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DDP (Atomic Operation Request) Message Offset |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Original Request Identifier |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Original Remote Value |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
The following figure depicts an Immediate Data or Immediate Data with SE, DDP Segment:
下图描述了即时数据或带有SE、DDP段的即时数据：
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DDP Control | RDMA Control |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reserved (Not Used) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DDP (Send) Queue Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DDP (Send) Message Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DDP Message Offset |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Immediate Data |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DDP Control | RDMA Control |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reserved (Not Used) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DDP (Send) Queue Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DDP (Send) Message Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DDP Message Offset |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Immediate Data |
+ +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Authors' Addresses
作者地址
Hemal Shah Broadcom Corporation 5300 California Avenue Irvine, CA 92617 US Phone: 1-949-926-6941 EMail: hemal@broadcom.com
Hemal Shah Broadcom Corporation美国加利福尼亚州欧文市加利福尼亚大道5300号92617电话：1-949-926-6941电子邮件：hemal@broadcom.com
Felix Marti Chelsio Communications, Inc. 370 San Aleso Ave. Sunnyvale, CA 94085 US Phone: 1-408-962-3600 EMail: felix@chelsio.com
Felix Marti Chelsio Communications，Inc.加利福尼亚州桑尼维尔圣阿莱索大道370号，邮编94085美国电话：1-408-962-3600电子邮件：felix@chelsio.com
Asgeir Eiriksson Chelsio Communications, Inc. 370 San Aleso Ave. Sunnyvale, CA 94085 US Phone: 1-408-962-3600 EMail: asgeir@chelsio.com
Asgeir Eiriksson Chelsio Communications，Inc.加利福尼亚州桑尼维尔圣阿莱索大道370号，邮编94085美国电话：1-408-962-3600电子邮件：asgeir@chelsio.com
Wael Noureddine Chelsio Communications, Inc. 370 San Aleso Ave. Sunnyvale, CA 94085 US Phone: 1-408-962-3600 EMail: wael@chelsio.com
Wael Noureddine Chelsio Communications，Inc.加利福尼亚州桑尼维尔圣阿莱索大道370号，邮编94085美国电话：1-408-962-3600电子邮件：wael@chelsio.com
Robert Sharp Intel Corporation 1300 South Mopac Expy, Mailstop: AN4-4B Austin, TX 78746 US Phone: 1-512-362-1407 EMail: robert.o.sharp@intel.com
Robert Sharp Intel Corporation 1300 South Mopac Expy，邮箱：AN4-4B德克萨斯州奥斯汀78746美国电话：1-512-362-1407电子邮件：Robert.o。sharp@intel.com