Network Working Group                                           R. Recio
Request for Comments: 5040                                    B. Metzler
Category: Standards Track                                IBM Corporation
                                                               P. Culley
                                                              J. Hilland
                                                 Hewlett-Packard Company
                                                               D. Garcia
                                                            October 2007
        
Network Working Group                                           R. Recio
Request for Comments: 5040                                    B. Metzler
Category: Standards Track                                IBM Corporation
                                                               P. Culley
                                                              J. Hilland
                                                 Hewlett-Packard Company
                                                               D. Garcia
                                                            October 2007
        

A Remote Direct Memory Access Protocol Specification

一种远程直接存储器访问协议规范

Status of This Memo

关于下段备忘

This document specifies an Internet standards track protocol for the Internet community, and requests discussion and suggestions for improvements. Please refer to the current edition of the "Internet Official Protocol Standards" (STD 1) for the standardization state and status of this protocol. Distribution of this memo is unlimited.

本文件规定了互联网社区的互联网标准跟踪协议,并要求进行讨论和提出改进建议。有关本协议的标准化状态和状态,请参考当前版本的“互联网官方协议标准”(STD 1)。本备忘录的分发不受限制。

Abstract

摘要

This document defines a Remote Direct Memory Access Protocol (RDMAP) that operates over the Direct Data Placement Protocol (DDP protocol). RDMAP provides read and write services directly to applications and enables data to be transferred directly into Upper Layer Protocol (ULP) Buffers without intermediate data copies. It also enables a kernel bypass implementation.

本文档定义了通过直接数据放置协议(DDP协议)运行的远程直接内存访问协议(RDMAP)。RDMAP直接向应用程序提供读写服务,使数据能够直接传输到上层协议(ULP)缓冲区,而无需中间数据拷贝。它还支持内核旁路实现。

Table of Contents

目录

   1. Introduction ....................................................4
      1.1. Architectural Goals ........................................4
      1.2. Protocol Overview ..........................................5
      1.3. RDMAP Layering .............................................7
   2. Glossary ........................................................8
      2.1. General ....................................................8
      2.2. LLP .......................................................10
      2.3. Direct Data Placement (DDP) ...............................11
      2.4. Remote Direct Memory Access (RDMA) ........................13
   3. ULP and Transport Attributes ...................................15
      3.1. Transport Requirements and Assumptions ....................15
      3.2. RDMAP Interactions with the ULP ...........................16
   4. Header Format ..................................................19
      4.1. RDMAP Control and Invalidate STag Field ...................20
      4.2. RDMA Message Definitions ..................................23
      4.3. RDMA Write Header .........................................24
      4.4. RDMA Read Request Header ..................................24
      4.5. RDMA Read Response Header .................................26
      4.6. Send Header and Send with Solicited Event Header ..........26
      4.7. Send with Invalidate Header and Send with SE and
           Invalidate Header .........................................26
      4.8. Terminate Header ..........................................26
   5. Data Transfer ..................................................32
      5.1. RDMA Write Message ........................................32
      5.2. RDMA Read Operation .......................................33
           5.2.1. RDMA Read Request Message ..........................33
           5.2.2. RDMA Read Response Message .........................35
      5.3. Send Message Type .........................................36
      5.4. Terminate Message .........................................37
      5.5. Ordering and Completions ..................................38
   6. RDMAP Stream Management ........................................41
      6.1. Stream Initialization .....................................41
      6.2. Stream Teardown ...........................................42
           6.2.1. RDMAP Abortive Termination .........................43
   7. RDMAP Error Management .........................................43
      7.1. RDMAP Error Surfacing .....................................44
      7.2. Errors Detected at the Remote Peer on Incoming
           RDMA Messages .............................................45
   8. Security Considerations ........................................46
      8.1. Summary of RDMAP-Specific Security Requirements ...........46
           8.1.1. RDMAP (RNIC) Requirements ..........................47
           8.1.2. Privileged Resource Manager Requirements ...........48
      8.2. Security Services for RDMAP ...............................49
           8.2.1. Available Security Services ........................49
           8.2.2. Requirements for IPsec Services for RDMAP ..........50
   9. IANA Considerations ............................................51
        
   1. Introduction ....................................................4
      1.1. Architectural Goals ........................................4
      1.2. Protocol Overview ..........................................5
      1.3. RDMAP Layering .............................................7
   2. Glossary ........................................................8
      2.1. General ....................................................8
      2.2. LLP .......................................................10
      2.3. Direct Data Placement (DDP) ...............................11
      2.4. Remote Direct Memory Access (RDMA) ........................13
   3. ULP and Transport Attributes ...................................15
      3.1. Transport Requirements and Assumptions ....................15
      3.2. RDMAP Interactions with the ULP ...........................16
   4. Header Format ..................................................19
      4.1. RDMAP Control and Invalidate STag Field ...................20
      4.2. RDMA Message Definitions ..................................23
      4.3. RDMA Write Header .........................................24
      4.4. RDMA Read Request Header ..................................24
      4.5. RDMA Read Response Header .................................26
      4.6. Send Header and Send with Solicited Event Header ..........26
      4.7. Send with Invalidate Header and Send with SE and
           Invalidate Header .........................................26
      4.8. Terminate Header ..........................................26
   5. Data Transfer ..................................................32
      5.1. RDMA Write Message ........................................32
      5.2. RDMA Read Operation .......................................33
           5.2.1. RDMA Read Request Message ..........................33
           5.2.2. RDMA Read Response Message .........................35
      5.3. Send Message Type .........................................36
      5.4. Terminate Message .........................................37
      5.5. Ordering and Completions ..................................38
   6. RDMAP Stream Management ........................................41
      6.1. Stream Initialization .....................................41
      6.2. Stream Teardown ...........................................42
           6.2.1. RDMAP Abortive Termination .........................43
   7. RDMAP Error Management .........................................43
      7.1. RDMAP Error Surfacing .....................................44
      7.2. Errors Detected at the Remote Peer on Incoming
           RDMA Messages .............................................45
   8. Security Considerations ........................................46
      8.1. Summary of RDMAP-Specific Security Requirements ...........46
           8.1.1. RDMAP (RNIC) Requirements ..........................47
           8.1.2. Privileged Resource Manager Requirements ...........48
      8.2. Security Services for RDMAP ...............................49
           8.2.1. Available Security Services ........................49
           8.2.2. Requirements for IPsec Services for RDMAP ..........50
   9. IANA Considerations ............................................51
        
   10. References ....................................................52
      10.1. Normative References .....................................52
      10.2. Informative References ...................................53
   Appendix A. DDP Segment Formats for RDMA Messages .................54
      A.1. DDP Segment for RDMA Write ................................54
      A.2. DDP Segment for RDMA Read Request .........................55
      A.3. DDP Segment for RDMA Read Response ........................56
      A.4. DDP Segment for Send and Send with Solicited Event ........56
      A.5. DDP Segment for Send with Invalidate and Send with SE and
           Invalidate ................................................57
      A.6. DDP Segment for Terminate .................................58
   Appendix B. Ordering and Completion Table .........................59
   Appendix C. Contributors ..........................................61
        
   10. References ....................................................52
      10.1. Normative References .....................................52
      10.2. Informative References ...................................53
   Appendix A. DDP Segment Formats for RDMA Messages .................54
      A.1. DDP Segment for RDMA Write ................................54
      A.2. DDP Segment for RDMA Read Request .........................55
      A.3. DDP Segment for RDMA Read Response ........................56
      A.4. DDP Segment for Send and Send with Solicited Event ........56
      A.5. DDP Segment for Send with Invalidate and Send with SE and
           Invalidate ................................................57
      A.6. DDP Segment for Terminate .................................58
   Appendix B. Ordering and Completion Table .........................59
   Appendix C. Contributors ..........................................61
        

Table of Figures

图表

   Figure 1: RDMAP Layering ...........................................7
   Figure 2: Example of MPA, DDP, and RDMAP Header Alignment over TCP .8
   Figure 3: DDP Control, RDMAP Control, and Invalidate STag Fields ..20
   Figure 4: RDMA Usage of DDP Fields ................................22
   Figure 5: RDMA Message Definitions ................................23
   Figure 6: RDMA Read Request Header Format .........................24
   Figure 7: Terminate Header Format .................................27
   Figure 8: Terminate Control Field .................................27
   Figure 9: Terminate Control Field Values ..........................29
   Figure 10: Error Type to RDMA Message Mapping .....................32
   Figure 11: RDMA Write, DDP Segment Format .........................54
   Figure 12: RDMA Read Request, DDP Segment Format ..................55
   Figure 13: RDMA Read Response, DDP Segment Format .................56
   Figure 14: Send and Send with Solicited Event, DDP Segment Format .56
   Figure 15: Send with Invalidate and Send with SE and Invalidate,
              DDP Segment Format .....................................57
   Figure 16: Terminate, DDP Segment Format ..........................58
   Figure 17: Operation Ordering .....................................59
        
   Figure 1: RDMAP Layering ...........................................7
   Figure 2: Example of MPA, DDP, and RDMAP Header Alignment over TCP .8
   Figure 3: DDP Control, RDMAP Control, and Invalidate STag Fields ..20
   Figure 4: RDMA Usage of DDP Fields ................................22
   Figure 5: RDMA Message Definitions ................................23
   Figure 6: RDMA Read Request Header Format .........................24
   Figure 7: Terminate Header Format .................................27
   Figure 8: Terminate Control Field .................................27
   Figure 9: Terminate Control Field Values ..........................29
   Figure 10: Error Type to RDMA Message Mapping .....................32
   Figure 11: RDMA Write, DDP Segment Format .........................54
   Figure 12: RDMA Read Request, DDP Segment Format ..................55
   Figure 13: RDMA Read Response, DDP Segment Format .................56
   Figure 14: Send and Send with Solicited Event, DDP Segment Format .56
   Figure 15: Send with Invalidate and Send with SE and Invalidate,
              DDP Segment Format .....................................57
   Figure 16: Terminate, DDP Segment Format ..........................58
   Figure 17: Operation Ordering .....................................59
        
1. Introduction
1. 介绍

Today, communications over TCP/IP typically require copy operations, which add latency and consume significant CPU and memory resources. The Remote Direct Memory Access Protocol (RDMAP) enables removal of data copy operations and enables reduction in latencies by allowing a local application to read or write data on a remote computer's memory with minimal demands on memory bus bandwidth and CPU processing overhead, while preserving memory protection semantics.

如今,TCP/IP上的通信通常需要拷贝操作,这会增加延迟并消耗大量CPU和内存资源。远程直接内存访问协议(RDMAP)允许本地应用程序读取或写入远程计算机内存上的数据,而对内存总线带宽和CPU处理开销的要求最小,同时保留内存保护语义,从而消除数据复制操作并降低延迟。

RDMAP is layered on top of Direct Data Placement (DDP) and uses the two buffer models available from DDP. DDP-related terminology is discussed in Section 2.3. As RDMAP builds on DDP, the reader is advised to become familiar with [DDP].

RDMAP是在直接数据放置(DDP)之上分层的,并使用DDP提供的两种缓冲区模型。DDP相关术语见第2.3节。由于RDMAP建立在DDP之上,建议读者熟悉[DDP]。

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119].

本文件中的关键词“必须”、“不得”、“要求”、“应”、“不应”、“应”、“不应”、“建议”、“可”和“可选”应按照RFC 2119[RFC2119]中所述进行解释。

1.1. Architectural Goals
1.1. 建筑目标

RDMAP has been designed with the following high-level architectural goals:

RDMAP的设计具有以下高层体系结构目标:

* Provide a data transfer operation that allows a Local Peer to transfer up to 2^32 - 1 octets directly into a previously Advertised Buffer (i.e., Tagged Buffer) located at a Remote Peer without requiring a copy operation. This is referred to as the RDMA Write data transfer operation.

* 提供一种数据传输操作,允许本地对等机将多达2^32-1个八位字节直接传输到位于远程对等机的先前公布的缓冲区(即标记的缓冲区)中,而无需复制操作。这称为RDMA写数据传输操作。

* Provide a data transfer operation that allows a Local Peer to retrieve up to 2^32 - 1 octets directly from a previously Advertised Buffer (i.e., Tagged Buffer) located at a Remote Peer without requiring a copy operation. This is referred to as the RDMA Read data transfer operation.

* 提供一种数据传输操作,允许本地对等机直接从位于远程对等机的先前公布的缓冲区(即标记的缓冲区)检索多达2^32-1个八位字节,而无需复制操作。这称为RDMA读取数据传输操作。

* Provide a data transfer operation that allows a Local Peer to send up to 2^32 - 1 octets directly into a buffer located at a Remote Peer that has not been explicitly Advertised. This is referred to as the Send (Send with Invalidate, Send with Solicited Event, and Send with Solicited Event and Invalidate) data transfer operation.

* 提供一种数据传输操作,允许本地对等机将多达2^32-1个八位字节直接发送到位于远程对等机的缓冲区中,该缓冲区未显式公布。这称为发送(发送时失效、发送时请求的事件以及发送时请求的事件和失效)数据传输操作。

* Enable the local ULP to use the Send Operation Type (includes Send, Send with Invalidate, Send with Solicited Event, and Send with Solicited Event and Invalidate) to signal to the remote ULP the Completion of all previous Messages initiated by the local ULP.

* 使本地ULP能够使用发送操作类型(包括发送、发送时失效、发送时请求的事件以及发送时请求的事件和失效)向远程ULP发送完成本地ULP发起的所有先前消息的信号。

* Provide for all operations on a single RDMAP Stream to be reliably transmitted in the order that they were submitted.

* 提供单个RDMAP流上的所有操作,以便按照提交顺序可靠地传输。

* Provide RDMAP capabilities independently for each Stream when the LLP supports multiple data Streams within an LLP connection.

* 当LLP在一个LLP连接中支持多个数据流时,为每个流独立提供RDMAP功能。

1.2. Protocol Overview
1.2. 协议概述

RDMAP provides seven data transfer operations. Except for the RDMA Read operation, each operation generates exactly one RDMA Message. Following is a brief overview of the RDMA Operations and RDMA Messages:

RDMAP提供了七种数据传输操作。除了RDMA读取操作外,每个操作只生成一条RDMA消息。以下是RDMA操作和RDMA消息的简要概述:

1. Send - A Send operation uses a Send Message to transfer data from the Data Source into a buffer that has not been explicitly Advertised by the Data Sink. The Send Message uses the DDP Untagged Buffer Model to transfer the ULP Message into the Data Sink's Untagged Buffer.

1. 发送-发送操作使用发送消息将数据从数据源传输到数据接收器未显式播发的缓冲区。发送消息使用DDP未标记缓冲区模型将ULP消息传输到数据接收器的未标记缓冲区。

2. Send with Invalidate - A Send with Invalidate operation uses a Send with Invalidate Message to transfer data from the Data Source into a buffer that has not been explicitly Advertised by the Data Sink. The Send with Invalidate Message includes all functionality of the Send Message, with one addition: an STag field is included in the Send with Invalidate Message. After the message has been Placed and Delivered at the Data Sink, the Remote Peer's buffer identified by the STag can no longer be accessed remotely until the Remote Peer's ULP re-enables access and Advertises the buffer.

2. 无效发送-无效发送操作使用无效发送消息将数据从数据源传输到数据接收器未显式播发的缓冲区。“发送时失效”消息包括发送消息的所有功能,还有一个附加功能:在“发送时失效”消息中包含一个STag字段。消息在数据接收器处放置和传递后,在远程对等方的ULP重新启用访问并播发缓冲区之前,STag标识的远程对等方的缓冲区不能再被远程访问。

3. Send with Solicited Event (Send with SE) - A Send with Solicited Event operation uses a Send with Solicited Event Message to transfer data from the Data Source into an Untagged Buffer at the Data Sink. The Send with Solicited Event Message is similar to the Send Message, with one addition: when the Send with Solicited Event Message has been Placed and Delivered, an Event may be generated at the recipient, if the recipient is configured to generate such an Event.

3. 发送请求事件(发送请求事件)-发送请求事件操作使用发送请求事件消息将数据从数据源传输到数据接收器的未标记缓冲区。“请求发送事件”消息与“发送消息”类似,只是增加了一点:当“请求发送事件”消息已放置和传递时,如果收件人配置为生成此类事件,则可能会在收件人处生成事件。

4. Send with Solicited Event and Invalidate (Send with SE and Invalidate) - A Send with Solicited Event and Invalidate operation uses a Send with Solicited Event and Invalidate Message to transfer data from the Data Source into a buffer that has not been explicitly Advertised by the Data Sink. The Send with Solicited Event and Invalidate Message is similar to the Send with Invalidate Message, with one addition: when the Send with

4. 使用请求事件发送并使其无效(使用SE发送并使其无效)—使用请求事件发送并使其无效操作使用请求事件发送并使其无效消息将数据从数据源传输到数据接收器未显式播发的缓冲区。使用请求事件和无效消息发送与使用无效消息发送类似,只是增加了一点:当使用

Solicited Event and Invalidate Message has been Placed and Delivered, an Event may be generated at the recipient, if the recipient is configured to generate such an Event.

已放置并传递请求事件和无效消息,如果收件人配置为生成此类事件,则可能会在收件人处生成事件。

5. Remote Direct Memory Access Write - An RDMA Write operation uses an RDMA Write Message to transfer data from the Data Source to a previously Advertised Buffer at the Data Sink.

5. 远程直接内存访问写入-RDMA写入操作使用RDMA写入消息将数据从数据源传输到数据接收器上先前公布的缓冲区。

The ULP at the Remote Peer, which in this case is the Data Sink, enables the Data Sink Tagged Buffer for access and Advertises the buffer's size (length), location (Tagged Offset), and Steering Tag (STag) to the Data Source through a ULP-specific mechanism. The ULP at the Local Peer, which in this case is the Data Source, initiates the RDMA Write operation. The RDMA Write Message uses the DDP Tagged Buffer Model to transfer the ULP Message into the Data Sink's Tagged Buffer. Note: the STag associated with the Tagged Buffer remains valid until the ULP at the Remote Peer invalidates it or the ULP at the Local Peer invalidates it through a Send with Invalidate or Send with Solicited Event and Invalidate.

远程对等方的ULP(在本例中为数据接收器)启用数据接收器标记的缓冲区进行访问,并通过ULP特定机制向数据源播发缓冲区的大小(长度)、位置(标记的偏移量)和转向标记(STag)。本地对等端的ULP(在本例中为数据源)启动RDMA写入操作。RDMA写入消息使用DDP标记缓冲区模型将ULP消息传输到数据接收器的标记缓冲区。注意:与标记缓冲区相关联的STag保持有效,直到远程对等方的ULP使其无效,或本地对等方的ULP通过发送带无效或发送带请求的事件使其无效。

6. Remote Direct Memory Access Read - The RDMA Read operation transfers data to a Tagged Buffer at the Local Peer, which in this case is the Data Sink, from a Tagged Buffer at the Remote Peer, which in this case is the Data Source. The ULP at the Data Source enables the Data Source Tagged Buffer for access and Advertises the buffer's size (length), location (Tagged Offset), and Steering Tag (STag) to the Data Sink through a ULP-specific mechanism. The ULP at the Data Sink enables the Data Sink Tagged Buffer for access and initiates the RDMA Read operation. The RDMA Read operation consists of a single RDMA Read Request Message and a single RDMA Read Response Message, and the latter may be segmented into multiple DDP Segments.

6. 远程直接内存访问读取-RDMA读取操作将数据从远程对等(本例中为数据源)的标记缓冲区传输到本地对等(本例中为数据接收器)的标记缓冲区。数据源处的ULP启用数据源标记缓冲区进行访问,并通过ULP特定机制向数据接收器播发缓冲区的大小(长度)、位置(标记偏移量)和转向标记(STag)。数据接收器处的ULP启用数据接收器标记的缓冲区进行访问,并启动RDMA读取操作。RDMA读取操作由单个RDMA读取请求消息和单个RDMA读取响应消息组成,后者可分为多个DDP段。

The RDMA Read Request Message uses the DDP Untagged Buffer Model to Deliver the STag, starting Tagged Offset, and length for both the Data Source and Data Sink Tagged Buffers to the Remote Peer's RDMA Read Request Queue.

RDMA读取请求消息使用DDP未标记缓冲区模型将STag、起始标记偏移量以及数据源和数据接收器标记缓冲区的长度传递给远程对等方的RDMA读取请求队列。

The RDMA Read Response Message uses the DDP Tagged Buffer Model to Deliver the Data Source's Tagged Buffer to the Data Sink, without any involvement from the ULP at the Data Source.

RDMA读取响应消息使用DDP标记缓冲区模型将数据源的标记缓冲区传送到数据接收器,而无需ULP参与数据源。

Note: the Data Source STag associated with the Tagged Buffer remains valid until the ULP at the Data Source invalidates it or the ULP at the Data Sink invalidates it through a Send with

注意:与标记的缓冲区关联的数据源STag保持有效,直到数据源的ULP使其无效或数据接收器的ULP通过发送使其无效

Invalidate or Send with Solicited Event and Invalidate. The Data Sink STag associated with the Tagged Buffer remains valid until the ULP at the Data Sink invalidates it.

失效或发送请求的事件并失效。与标记缓冲区关联的数据接收器STag保持有效,直到数据接收器处的ULP使其无效。

7. Terminate - A Terminate operation uses a Terminate Message to transfer to the Remote Peer information associated with an error that occurred at the Local Peer. The Terminate Message uses the DDP Untagged Buffer Model to transfer the Message into the Data Sink's Untagged Buffer.

7. Terminate-Terminate操作使用Terminate消息将与本地对等机上发生的错误相关联的信息传输到远程对等机。Terminate消息使用DDP Untaged Buffer模型将消息传输到数据接收器的Untaged Buffer中。

1.3. RDMAP Layering
1.3. RDMAP分层

RDMAP is dependent on DDP, subject to the requirements defined in Section 3.1, "Transport Requirements and Assumptions". Figure 1, "RDMAP Layering", depicts the relationship between Upper Layer Protocols (ULPs), RDMAP, DDP protocol, the framing layer, and the transport. For LLP protocol definitions of each LLP, see [MPA], [TCP], and [SCTP].

RDMAP取决于DDP,符合第3.1节“运输要求和假设”中定义的要求。图1“RDMAP分层”描述了上层协议(ULP)、RDMAP、DDP协议、帧层和传输之间的关系。有关每个LLP的LLP协议定义,请参见[MPA]、[TCP]和[SCTP]。

                 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                 |                                     |
                 |     Upper Layer Protocol (ULP)      |
                 |                                     |
                 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                 |                                     |
                 |              RDMAP                  |
                 |                                     |
                 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                 |                                     |
                 |           DDP protocol              |
                 |                                     |
                 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                 |                 |                   |
                 |       MPA       |                   |
                 |                 |                   |
                 +-+-+-+-+-+-+-+-+-+       SCTP        |
                 |                 |                   |
                 |       TCP       |                   |
                 |                 |                   |
                 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        
                 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                 |                                     |
                 |     Upper Layer Protocol (ULP)      |
                 |                                     |
                 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                 |                                     |
                 |              RDMAP                  |
                 |                                     |
                 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                 |                                     |
                 |           DDP protocol              |
                 |                                     |
                 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                 |                 |                   |
                 |       MPA       |                   |
                 |                 |                   |
                 +-+-+-+-+-+-+-+-+-+       SCTP        |
                 |                 |                   |
                 |       TCP       |                   |
                 |                 |                   |
                 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        

Figure 1: RDMAP Layering

图1:RDMAP分层

If RDMAP is layered over DDP/MPA/TCP, then the respective headers and ULP Payload are arranged as follows (Note: For clarity, MPA header and CRC fields are included but MPA markers are not shown):

如果RDMAP是在DDP/MPA/TCP上分层的,则相应的报头和ULP有效载荷的排列如下(注意:为清楚起见,包括了MPA报头和CRC字段,但未显示MPA标记):

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                                                               |
    //                           TCP Header                        //
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |         MPA Header            |                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
    |                                                               |
    //                        DDP Header                           //
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                                                               |
    //                        RDMA Header                          //
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                                                               |
    //                        ULP Payload                          //
    //                 (shown with no pad bytes)                   //
    //                                                             //
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                           MPA CRC                             |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        
     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                                                               |
    //                           TCP Header                        //
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |         MPA Header            |                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
    |                                                               |
    //                        DDP Header                           //
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                                                               |
    //                        RDMA Header                          //
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                                                               |
    //                        ULP Payload                          //
    //                 (shown with no pad bytes)                   //
    //                                                             //
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                           MPA CRC                             |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        

Figure 2: Example of MPA, DDP, and RDMAP Header Alignment over TCP

图2:TCP上MPA、DDP和RDMAP头对齐的示例

2. Glossary
2. 术语汇编
2.1. General
2.1. 全体的

Advertisement (Advertised, Advertise, Advertisements, Advertises) - the act of informing a Remote Peer that a local RDMA Buffer is available to it. A Node makes available an RDMA Buffer for incoming RDMA Read or RDMA Write access by informing its RDMA/DDP peer of the Tagged Buffer identifiers (STag, base address, and buffer length). This Advertisement of Tagged Buffer information is not defined by RDMA/DDP and is left to the ULP. A typical method would be for the Local Peer to embed the Tagged Buffer's Steering Tag, base address, and length in a Send Message destined for the Remote Peer.

广告(广告、广告、广告、广告)-通知远程对等方本地RDMA缓冲区可用的行为。节点通过通知其RDMA/DDP对等方已标记的缓冲区标识符(STag、基址和缓冲区长度),为传入的RDMA读或RDMA写访问提供RDMA缓冲区。RDMA/DDP没有定义标记缓冲区信息的播发,而是留给ULP。一种典型的方法是,本地对等方将标记缓冲区的引导标记、基址和长度嵌入到发送给远程对等方的发送消息中。

Completion - Refer to "RDMA Completion" in Section 2.4.

完工-参考第2.4节中的“RDMA完工”。

Completed - See "RDMA Completion" in Section 2.4.

已完成-参见第2.4节中的“RDMA完成”。

Complete - See "RDMA Completion" in Section 2.4.

完成-参见第2.4节中的“RDMA完成”。

Completes - See "RDMA Completion" in Section 2.4.

完成-参见第2.4节中的“RDMA完成”。

Data Sink - The peer receiving a data payload. Note that the Data Sink can be required to both send and receive RDMA/DDP Messages to transfer a data payload.

数据接收器-接收数据有效负载的对等方。请注意,可以要求数据接收器发送和接收RDMA/DDP消息以传输数据有效负载。

Data Source - The peer sending a data payload. Note that the Data Source can be required to both send and receive RDMA/DDP Messages to transfer a data payload.

数据源-发送数据有效负载的对等方。请注意,可以要求数据源发送和接收RDMA/DDP消息以传输数据有效负载。

Data Delivery (Delivery, Delivered, Delivers) - Delivery is defined as the process of informing the ULP or consumer that a particular Message is available for use. This is specifically different from "Placement", which may generally occur in any order, while the order of "Delivery" is strictly defined. See "Data Placement" in Section 2.3.

数据交付(Delivery,Delivered,Delivers)-交付被定义为通知ULP或消费者特定消息可供使用的过程。这与通常在任何订单中出现的“放置”特别不同,而“交付”的订单是严格定义的。参见第2.3节中的“数据放置”。

Delivery - See Data Delivery in Section 2.1.

交付-见第2.1节中的数据交付。

Delivered - See Data Delivery in Section 2.1.

已交付-见第2.1节中的数据交付。

Delivers - See Data Delivery in Section 2.1.

交付-参见第2.1节中的数据交付。

Fabric - The collection of links, switches, and routers that connect a set of Nodes with RDMA/DDP protocol implementations.

结构-链接、交换机和路由器的集合,它们通过RDMA/DDP协议实现连接一组节点。

Fence (Fenced, Fences) - To block the current RDMA Operation from executing until prior RDMA Operations have Completed.

围栏(Fenced,Fences)-阻止当前RDMA操作执行,直到之前的RDMA操作完成。

iWARP - A suite of wire protocols comprised of RDMAP, DDP, and MPA. The iWARP protocol suite may be layered above TCP, SCTP, or other transport protocols.

iWARP—由RDMAP、DDP和MPA组成的一套有线协议。iWARP协议套件可以分层在TCP、SCTP或其他传输协议之上。

Local Peer - The RDMA/DDP protocol implementation on the local end of the connection. Used to refer to the local entity when describing a protocol exchange or other interaction between two Nodes.

本地对等-连接本地端上的RDMA/DDP协议实现。在描述两个节点之间的协议交换或其他交互时,用于引用本地实体。

Node - A computing device attached to one or more links of a Fabric (network). A Node in this context does not refer to a specific application or protocol instantiation running on the computer. A Node may consist of one or more RNICs installed in a host computer.

节点-连接到结构(网络)的一个或多个链路的计算设备。此上下文中的节点不引用计算机上运行的特定应用程序或协议实例化。节点可以由安装在主机中的一个或多个RNIC组成。

Placement - See "Data Placement" in Section 2.3.

放置-参见第2.3节中的“数据放置”。

Placed - See "Data Placement" in Section 2.3.

放置-参见第2.3节中的“数据放置”。

Places - See "Data Placement" in Section 2.3.

位置-参见第2.3节中的“数据放置”。

Remote Peer - The RDMA/DDP protocol implementation on the opposite end of the connection. Used to refer to the remote entity when describing protocol exchanges or other interactions between two Nodes.

远程对等-连接另一端的RDMA/DDP协议实现。在描述两个节点之间的协议交换或其他交互时,用于指代远程实体。

RNIC - RDMA Network Interface Controller. In this context, this would be a network I/O adapter or embedded controller with iWARP and Verbs functionality.

RNIC-RDMA网络接口控制器。在这种情况下,这将是一个具有iWARP和Verbs功能的网络I/O适配器或嵌入式控制器。

RNIC Interface (RI) - The presentation of the RNIC to the Verbs Consumer as implemented through the combination of the RNIC and the RNIC driver.

RNIC接口(RI)-通过RNIC和RNIC驱动程序的组合,向用户呈现RNIC。

Termination - See "RDMAP Abortive Termination" in Section 2.4.

终止-参见第2.4节中的“RDMAP中止终止”。

Terminated - See "RDMAP Abortive Termination" in Section 2.4.

终止-参见第2.4节中的“RDMAP中止终止”。

Terminate - See "RDMAP Abortive Termination" in Section 2.4.

终止-参见第2.4节中的“RDMAP中止终止”。

Terminates - See "RDMAP Abortive Termination" in Section 2.4.

终止-参见第2.4节中的“RDMAP中止终止”。

ULP - Upper Layer Protocol. The protocol layer above the one currently being referenced. The ULP for RDMA/DDP is expected to be an OS, Application, adaptation layer, or proprietary device. The RDMA/DDP documents do not specify a ULP -- they provide a set of semantics that allow a ULP to be designed to utilize RDMA/DDP.

ULP-上层协议。当前引用的协议层之上的协议层。RDMA/DDP的ULP应为操作系统、应用程序、适配层或专有设备。RDMA/DDP文档没有指定ULP——它们提供了一组语义,允许ULP设计为使用RDMA/DDP。

ULP Payload - The ULP data that is contained within a single protocol segment or packet (e.g., a DDP Segment).

ULP有效负载-包含在单个协议段或数据包(例如DDP段)内的ULP数据。

Verbs - An abstract description of the functionality of an RNIC Interface. The OS may expose some or all of this functionality via one or more APIs to applications. The OS will also use some of the functionality to manage the RNIC Interface.

动词-RNIC接口功能的抽象描述。操作系统可以通过一个或多个API向应用程序公开部分或全部功能。操作系统还将使用一些功能来管理RNIC接口。

2.2. LLP
2.2. 律师事务所

LLP - Lower Layer Protocol. The protocol layer beneath the protocol layer currently being referenced. For example, for DDP, the LLP is SCTP, MPA, or other transport protocols. For RDMA, the LLP is DDP.

LLP-低层协议。当前引用的协议层下的协议层。例如,对于DDP,LLP是SCTP、MPA或其他传输协议。对于RDMA,LLP为DDP。

LLP Connection - Corresponds to an LLP transport-level connection between the peer LLP layers on two Nodes.

LLP连接-对应于两个节点上对等LLP层之间的LLP传输级别连接。

LLP Stream - Corresponds to a single LLP transport-level Stream between the peer LLP layers on two Nodes. One or more LLP Streams may map to a single transport-level LLP connection. For transport protocols that support multiple Streams per connection (e.g., SCTP), an LLP Stream corresponds to one transport-level Stream.

LLP流-对应于两个节点上对等LLP层之间的单个LLP传输级流。一个或多个LLP流可以映射到单个传输级LLP连接。对于每个连接支持多个流的传输协议(例如,SCTP),LLP流对应于一个传输级流。

MULPDU - Maximum ULPDU. The current maximum size of the record that is acceptable for DDP to pass to the LLP for transmission.

MULPDU-最大ULPDU。DDP可以传递给LLP进行传输的记录的当前最大大小。

ULPDU - Upper Layer Protocol Data Unit. The data record defined by the layer above MPA.

ULPDU-上层协议数据单元。MPA以上层定义的数据记录。

2.3. Direct Data Placement (DDP)
2.3. 直接数据放置(DDP)

Data Placement (Placement, Placed, Places) - For DDP, this term is specifically used to indicate the process of writing to a data buffer by a DDP implementation. DDP Segments carry Placement information, which may be used by the receiving DDP implementation to perform Data Placement of the DDP Segment ULP Payload. See "Data Delivery".

数据放置(Placement,Placed,Places)-对于DDP,该术语专门用于表示DDP实现写入数据缓冲区的过程。DDP段携带放置信息,接收DDP实现可使用该信息来执行DDP段ULP有效载荷的数据放置。见“数据交付”。

DDP Abortive Teardown - The act of closing a DDP Stream without attempting to Complete in-progress and pending DDP Messages.

DDP中止拆卸-关闭DDP流而不尝试完成正在进行和挂起的DDP消息的行为。

DDP Graceful Teardown - The act of closing a DDP Stream such that all in-progress and pending DDP Messages are allowed to Complete successfully.

DDP优雅拆卸-关闭DDP流的行为,以便允许所有正在进行和挂起的DDP消息成功完成。

DDP Control Field - A fixed 16-bit field in the DDP Header. The DDP Control Field contains an 8-bit field whose contents are reserved for use by the ULP.

DDP控制字段-DDP标头中的固定16位字段。DDP控制字段包含一个8位字段,其内容保留供ULP使用。

DDP Header - The header present in all DDP segments. The DDP Header contains control and Placement fields that are used to define the final Placement location for the ULP Payload carried in a DDP Segment.

DDP标头-所有DDP段中存在的标头。DDP标头包含控制和放置字段,用于定义DDP段中携带的ULP有效负载的最终放置位置。

DDP Message - A ULP-defined unit of data interchange, which is subdivided into one or more DDP segments. This segmentation may occur for a variety of reasons, including segmentation to respect the maximum segment size of the underlying transport protocol.

DDP报文—ULP定义的数据交换单元,细分为一个或多个DDP段。这种分段可能由于各种原因而发生,包括为了尊重底层传输协议的最大分段大小而进行的分段。

DDP Segment - The smallest unit of data transfer for the DDP protocol. It includes a DDP Header and ULP Payload (if present). A DDP Segment should be sized to fit within the underlying transport protocol MULPDU.

DDP段-DDP协议的最小数据传输单元。它包括DDP标头和ULP有效负载(如果存在)。DDP段的大小应适合基础传输协议MULPDU。

DDP Stream - A sequence of DDP Messages whose ordering is defined by the LLP. For SCTP, a DDP Stream maps directly to an SCTP Stream. For MPA, a DDP Stream maps directly to a TCP connection, and a single DDP Stream is supported. Note that DDP has no ordering guarantees between DDP Streams.

DDP流—DDP消息序列,其顺序由LLP定义。对于SCTP,DDP流直接映射到SCTP流。对于MPA,DDP流直接映射到TCP连接,并且支持单个DDP流。请注意,DDP在DDP流之间没有订购保证。

Direct Data Placement - A mechanism whereby ULP data contained within DDP Segments may be Placed directly into its final destination in memory without processing of the ULP. This may occur even when the DDP Segments arrive out of order. Out-of-order Placement support may require the Data Sink to implement the LLP and DDP as one functional block.

直接数据放置-DDP段中包含的ULP数据可直接放置到内存中的最终目的地,而无需处理ULP的一种机制。即使DDP段到达时出现故障,也可能发生这种情况。无序放置支持可能需要数据接收器将LLP和DDP实现为一个功能块。

Direct Data Placement Protocol (DDP) - Also, a wire protocol that supports Direct Data Placement by associating explicit memory buffer placement information with the LLP payload units.

直接数据放置协议(DDP)-也是一种有线协议,通过将显式内存缓冲区放置信息与LLP有效负载单元关联,支持直接数据放置。

Message Offset (MO) - For the DDP Untagged Buffer Model, specifies the offset, in bytes, from the start of a DDP Message.

消息偏移量(MO)-对于DDP未标记缓冲区模型,指定从DDP消息开始的偏移量(以字节为单位)。

Message Sequence Number (MSN) - For the DDP Untagged Buffer Model, specifies a sequence number that is increasing with each DDP Message.

消息序列号(MSN)-对于DDP未标记缓冲区模型,指定随每条DDP消息增加的序列号。

Queue Number (QN) - For the DDP Untagged Buffer Model, identifies a destination Data Sink queue for a DDP Segment.

队列号(QN)-对于DDP未标记缓冲区模型,标识DDP段的目标数据接收器队列。

Steering Tag - An identifier of a Tagged Buffer on a Node, valid as defined within a protocol specification.

转向标记-节点上标记缓冲区的标识符,在协议规范中定义有效。

STag - Steering Tag

STag-转向标签

Tagged Buffer - A buffer that is explicitly Advertised to the Remote Peer through exchange of an STag, Tagged Offset, and length.

标记的缓冲区-通过交换STag、标记的偏移量和长度显式地通告给远程对等方的缓冲区。

Tagged Buffer Model - A DDP data transfer model used to transfer Tagged Buffers from the Local Peer to the Remote Peer.

标记缓冲区模型-DDP数据传输模型,用于将标记缓冲区从本地对等点传输到远程对等点。

Tagged DDP Message - A DDP Message that targets a Tagged Buffer.

标记的DDP消息-以标记的缓冲区为目标的DDP消息。

Tagged Offset (TO) - The offset within a Tagged Buffer on a Node.

标记的偏移量(到)-节点上标记的缓冲区内的偏移量。

Untagged Buffer - A buffer that is not explicitly Advertised to the Remote Peer. Untagged Buffers support one of the two available data transfer mechanisms called the Untagged Buffer Model. An Untagged Buffer is used to send asynchronous control messages to the Remote Peer for RDMA Read, Send, and Terminate requests. Untagged Buffers handle Untagged DDP Messages.

未标记缓冲区-未显式播发给远程对等方的缓冲区。未标记缓冲区支持两种可用的数据传输机制之一,称为未标记缓冲区模型。未标记的缓冲区用于向远程对等方发送异步控制消息,以进行RDMA读取、发送和终止请求。未标记的缓冲区处理未标记的DDP消息。

Untagged Buffer Model - A DDP data transfer model used to transfer Untagged Buffers from the Local Peer to the Remote Peer.

未标记缓冲区模型-DDP数据传输模型,用于将未标记缓冲区从本地对等点传输到远程对等点。

Untagged DDP Message - A DDP Message that targets an Untagged Buffer.

未标记DDP消息-针对未标记缓冲区的DDP消息。

2.4. Remote Direct Memory Access (RDMA)
2.4. 远程直接内存访问(RDMA)

Completion Queues (CQs) - Logical components of the RNIC Interface that conceptually represent how an RNIC notifies the ULP about the completion of the transmission of data, or the completion of the reception of data; see [RDMASEC].

完成队列(CQ)-RNIC接口的逻辑组件,概念上表示RNIC如何通知ULP数据传输完成或数据接收完成;见[RDMASEC]。

Event - An indication provided by the RDMAP layer to the ULP to indicate a Completion or other condition requiring immediate attention.

事件-RDMAP层向ULP提供的指示,指示完成或其他需要立即注意的情况。

Invalidate STag - A mechanism used to prevent the Remote Peer from reusing a previous explicitly Advertised STag, until the Local Peer makes it available through a subsequent explicit Advertisement. The STag cannot be accessed remotely until it is explicitly Advertised again.

Invalidate STag-一种机制,用于防止远程对等方重用先前显式公布的STag,直到本地对等方通过后续显式公布使其可用。在再次显式播发之前,无法远程访问STag。

RDMA Completion (Completion, Completed, Complete, Completes) - For RDMA, Completion is defined as the process of informing the ULP that a particular RDMA Operation has performed all functions specified for the RDMA Operations, including Placement and Delivery. The Completion semantic of each RDMA Operation is distinctly defined.

RDMA完成(Completion,Completed,Complete,Completed)-对于RDMA,完成定义为通知ULP特定RDMA操作已执行RDMA操作指定的所有功能的过程,包括放置和交付。每个RDMA操作的完成语义都有明确的定义。

RDMA Message - A data transfer mechanism used to fulfill an RDMA Operation.

RDMA消息-用于完成RDMA操作的数据传输机制。

RDMA Operation - A sequence of RDMA Messages, including control Messages, to transfer data from a Data Source to a Data Sink. The following RDMA Operations are defined: RDMA Writes, RDMA Read, Send, Send with Invalidate, Send with Solicited Event, Send with Solicited Event and Invalidate, and Terminate.

RDMA操作—一系列RDMA消息,包括控制消息,用于将数据从数据源传输到数据接收器。定义了以下RDMA操作:RDMA写入、RDMA读取、发送、无效发送、请求事件发送、请求事件发送、无效发送和终止。

RDMA Protocol (RDMAP) - A wire protocol that supports RDMA Operations to transfer ULP data between a Local Peer and the Remote Peer.

RDMA协议(RDMAP)-一种支持RDMA操作的有线协议,用于在本地对等方和远程对等方之间传输ULP数据。

RDMAP Abortive Termination (Termination, Terminated, Terminate, Terminates) - The act of closing an RDMAP Stream without attempting to Complete in-progress and pending RDMA Operations.

RDMAP中止终止(Termination,Terminated,Terminate,Terminates)-关闭RDMAP流而不尝试完成正在进行和挂起的RDMA操作的行为。

RDMAP Graceful Termination - The act of closing an RDMAP Stream such that all in-progress and pending RDMA Operations are allowed to Complete successfully.

RDMAP优雅终止-关闭RDMAP流的行为,以便允许所有正在进行和挂起的RDMA操作成功完成。

RDMA Read - An RDMA Operation used by the Data Sink to transfer the contents of a source RDMA buffer from the Remote Peer to the Local Peer. An RDMA Read operation consists of a single RDMA Read Request Message and a single RDMA Read Response Message.

RDMA Read—数据接收器用于将源RDMA缓冲区的内容从远程对等传输到本地对等的RDMA操作。RDMA读取操作由单个RDMA读取请求消息和单个RDMA读取响应消息组成。

RDMA Read Request - An RDMA Message used by the Data Sink to request the Data Source to transfer the contents of an RDMA buffer. The RDMA Read Request Message describes both the Data Source and Data Sink RDMA buffers.

RDMA Read Request—数据接收器用于请求数据源传输RDMA缓冲区内容的RDMA消息。RDMA读取请求消息描述数据源和数据接收器RDMA缓冲区。

RDMA Read Request Queue - The queue used for processing RDMA Read Requests. The RDMA Read Request Queue has a DDP Queue Number of 1.

RDMA读取请求队列—用于处理RDMA读取请求的队列。RDMA读取请求队列的DDP队列号为1。

RDMA Read Response - An RDMA Message used by the Data Source to transfer the contents of an RDMA buffer to the Data Sink, in response to an RDMA Read Request. The RDMA Read Response Message only describes the data sink RDMA buffer.

RDMA Read Response—数据源用于响应RDMA读取请求,将RDMA缓冲区的内容传输到数据接收器的RDMA消息。RDMA读取响应消息仅描述数据接收器RDMA缓冲区。

RDMAP Stream - An association between a pair of RDMAP implementations, possibly on different Nodes, which transfer ULP data using RDMA Operations. There may be multiple RDMAP Streams on a single Node. An RDMAP Stream maps directly to a single DDP Stream.

RDMAP流-一对RDMAP实现之间的关联,可能在不同的节点上,使用RDMA操作传输ULP数据。单个节点上可能有多个RDMAP流。RDMAP流直接映射到单个DDP流。

RDMA Write - An RDMA Operation that transfers the contents of a source RDMA Buffer from the Local Peer to a destination RDMA Buffer at the Remote Peer using RDMA. The RDMA Write Message only describes the Data Sink RDMA buffer.

RDMA写入—一种RDMA操作,使用RDMA将源RDMA缓冲区的内容从本地对等点传输到远程对等点的目标RDMA缓冲区。RDMA写入消息仅描述数据接收器RDMA缓冲区。

Remote Direct Memory Access (RDMA) - A method of accessing memory on a remote system in which the local system specifies the remote location of the data to be transferred. Employing an RNIC in the remote system allows the access to take place without interrupting the processing of the CPU(s) on the system.

远程直接内存访问(RDMA)—一种访问远程系统内存的方法,其中本地系统指定要传输的数据的远程位置。在远程系统中使用RNIC允许在不中断系统上CPU处理的情况下进行访问。

Send - An RDMA Operation that transfers the contents of a ULP Buffer from the Local Peer to an Untagged Buffer at the Remote Peer.

发送-一种RDMA操作,将ULP缓冲区的内容从本地对等方传输到远程对等方的未标记缓冲区。

Send Message Type - A Send Message, Send with Invalidate Message, Send with Solicited Event Message, or Send with Solicited Event and Invalidate Message.

发送消息类型-发送消息、使用无效消息发送、使用请求的事件消息发送或使用请求的事件和无效消息发送。

Send Operation Type - A Send Operation, Send with Invalidate Operation, Send with Solicited Event Operation, or Send with Solicited Event and Invalidate Operation.

发送操作类型-发送操作、发送时使用无效操作、发送时使用请求的事件操作或发送时使用请求的事件和无效操作。

Solicited Event (SE) - A facility by which an RDMA Operation sender may cause an Event to be generated at the recipient, if the recipient is configured to generate such an Event, when a Send with Solicited Event Message or Send with Solicited Event and Invalidate Message is received. Note: The Local Peer's ULP can use the Solicited Event mechanism to ensure that Messages designated as important to the ULP are handled in an expeditious manner by the Remote Peer's ULP. The ULP at the Local Peer can indicate a given Send Message Type is important by using the Send with Solicited Event Message or Send with Solicited Event and Invalidate Message. The ULP at the Remote Peer can choose to only be notified when valid Send with Solicited Event Messages and/or Send with Solicited Event and Invalidate Messages arrive and handle other valid incoming Send Messages or Send with Invalidate Messages at its leisure.

请求事件(SE)-当接收到请求事件发送消息或请求事件发送并失效消息时,如果收件人被配置为生成此类事件,RDMA操作发送方可通过此功能在收件人处生成事件。注意:本地对等方的ULP可以使用请求事件机制来确保远程对等方的ULP以快速的方式处理指定为对ULP重要的消息。本地对等方的ULP可以通过使用请求事件发送消息或请求事件发送和失效消息来指示给定的发送消息类型是重要的。远程对等方的ULP可以选择仅在有效发送请求的事件消息和/或发送请求的事件和无效消息时收到通知,并在空闲时处理其他有效传入发送消息或发送无效消息。

Terminate - An RDMA Message used by a Node to pass an error indication to the peer Node on an RDMAP Stream. This operation is for RDMAP use only.

Terminate—节点用于在RDMAP流上将错误指示传递给对等节点的RDMA消息。此操作仅用于RDMAP。

ULP Buffer - A buffer owned above the RDMAP layer and Advertised to the RDMAP layer either as a Tagged Buffer or an Untagged ULP Buffer.

ULP缓冲区—RDMAP层之上拥有的缓冲区,作为标记缓冲区或未标记ULP缓冲区播发给RDMAP层。

ULP Message - The ULP data that is handed to a specific protocol layer for transmission. Data boundaries are preserved as they are transmitted through iWARP.

ULP消息-传递给特定协议层进行传输的ULP数据。数据边界在通过iWARP传输时被保留。

3. ULP and Transport Attributes
3. ULP和传输属性
3.1. Transport Requirements and Assumptions
3.1. 运输要求和假设

RDMAP MUST be layered on top of the Direct Data Placement Protocol [DDP].

RDMAP必须分层在直接数据放置协议[DDP]之上。

RDMAP requires the following DDP support:

RDMAP需要以下DDP支持:

* RDMAP uses three queues for Untagged Buffers:

* RDMAP对未标记的缓冲区使用三个队列:

* Queue Number 0 (used by RDMAP for Send, Send with Invalidate, Send with Solicited Event, and Send with Solicited Event and Invalidate operations).

* 队列号0(RDMAP用于发送、发送时失效、发送时请求的事件以及发送时请求的事件和失效操作)。

* Queue Number 1 (used by RDMAP for RDMA Read operations).

* 队列号1(由RDMAP用于RDMA读取操作)。

* Queue Number 2 (used by RDMAP for Terminate operations).

* 队列号2(RDMAP用于终止操作)。

* DDP maps a single RDMA Message to a single DDP Message.

* DDP将单个RDMA消息映射到单个DDP消息。

* DDP uses the STag and Tagged Offset provided by the RDMAP for Tagged Buffer Messages (i.e., RDMA Write and RDMA Read Response).

* DDP将RDMAP提供的STag和Tagged偏移量用于标记缓冲区消息(即RDMA写入和RDMA读取响应)。

* When the DDP layer Delivers an Untagged DDP Message to the RDMAP layer, DDP provides the length of the DDP Message. This ensures that RDMAP does not have to carry a length field in its header.

* 当DDP层将未标记的DDP消息传递给RDMAP层时,DDP提供DDP消息的长度。这确保RDMAP不必在其头中携带长度字段。

* When the RDMAP layer provides an RDMA Message to the DDP layer, DDP must insert the RsvdULP field value provided by the RDMAP layer into the associated DDP Message.

* 当RDMAP层向DDP层提供RDMA消息时,DDP必须将RDMAP层提供的RsvdULP字段值插入关联的DDP消息中。

* When the DDP layer Delivers a DDP Message to the RDMAP layer, DDP provides the RsvdULP field.

* 当DDP层向RDMAP层传递DDP消息时,DDP提供RsvdULP字段。

* The RsvdULP field must be 1 octet for DDP Tagged Messages and 5 octets for DDP Untagged Messages.

* 对于DDP标记的消息,RsvdULP字段必须为1个八位字节;对于DDP未标记的消息,RsvdULP字段必须为5个八位字节。

* DDP propagates to RDMAP all operation or protection errors (used by RDMAP Terminate) and, when appropriate, the DDP Header fields of the DDP Segment that encountered the error.

* DDP将所有操作或保护错误(由RDMAP Terminate使用)以及遇到错误的DDP段的DDP头字段(如果适用)传播到RDMAP。

* If an RDMA Operation is aborted by DDP or a lower layer, the contents of the Data Sink buffers associated with the operation are considered indeterminate.

* 如果RDMA操作被DDP或较低层中止,则与该操作相关联的数据接收器缓冲区的内容被认为是不确定的。

* DDP, in conjunction with the lower layers, provides reliable, in-order Delivery.

* DDP与下层结合,提供可靠、有序的交付。

3.2. RDMAP Interactions with the ULP
3.2. RDMAP与ULP的交互

RDMAP provides the ULP with access to the following RDMA Operations as defined in this specification:

RDMAP为ULP提供了访问本规范中定义的以下RDMA操作的权限:

* Send

* 邮寄

* Send with Solicited Event

* 发送请求的事件

* Send with Invalidate

* 无效发送

* Send with Solicited Event and Invalidate

* 发送请求的事件并使其无效

* RDMA Write

* RDMA写入

* RDMA Read

* RDMA读取

For Send Operation Types, the following are the interactions between the RDMAP layer and the ULP:

对于发送操作类型,以下是RDMAP层和ULP之间的交互:

* At the Data Source:

* 在数据源:

* The ULP passes to the RDMAP layer the following:

* ULP通过以下方式传递到RDMAP层:

* ULP Message Length

* ULP消息长度

* ULP Message

* ULP消息

* An indication of the Send Operation Type, where the valid types are: Send, Send with Solicited Event, Send with Invalidate, or Send with Solicited Event and Invalidate.

* 发送操作类型的指示,其中有效类型为:发送、请求事件发送、无效发送或请求事件发送和无效。

* An Invalidate STag, if the Send Operation Type was Send with Invalidate or Send with Solicited Event and Invalidate.

* 无效STag,如果发送操作类型为发送时使用无效或发送时使用请求的事件和无效。

* When the Send Operation Type Completes, an indication of the Completion results.

* 发送操作类型完成时,显示完成结果。

* At the Data Sink:

* 在数据接收器处:

* If the Send Operation Type Completed successfully, the RDMAP layer passes the following information to the ULP Layer:

* 如果发送操作类型成功完成,RDMAP层将以下信息传递给ULP层:

* ULP Message Length

* ULP消息长度

* ULP Message

* ULP消息

* An Event, if the Data Sink is configured to generate an Event.

* 如果数据接收器配置为生成事件,则为事件。

* An Invalidated STag, if the Send Operation Type was Send with Invalidate or Send with Solicited Event and Invalidate.

* 无效STag,如果发送操作类型为“发送时失效”或“发送时请求的事件并失效”。

* If the Send Operation Type Completed in error, the Data Sink RDMAP layer will pass up the corresponding error information to the Data Sink ULP and send a Terminate Message to the Data Source RDMAP layer. The Data Source RDMAP layer will then pass up the Terminate Message to the ULP.

* 如果发送操作类型错误完成,数据接收器RDMAP层将向数据接收器ULP传递相应的错误信息,并向数据源RDMAP层发送终止消息。然后,数据源RDMAP层将终止消息传递给ULP。

For RDMA Write operations, the following are the interactions between the RDMAP layer and the ULP:

对于RDMA写入操作,以下是RDMAP层和ULP之间的交互:

* At the Data Source:

* 在数据源:

* The ULP passes to the RDMAP layer the following:

* ULP通过以下方式传递到RDMAP层:

* ULP Message Length

* ULP消息长度

* ULP Message

* ULP消息

* Data Sink STag

* 数据接收STag

* Data Sink Tagged Offset

* 数据接收器标记偏移量

* When the RDMA Write operation Completes, an indication of the Completion results.

* RDMA写入操作完成时,会显示完成结果的指示。

* At the Data Sink:

* 在数据接收器处:

* If the RDMA Write completed successfully, the RDMAP layer does not Deliver the RDMA Write to the ULP. It does Place the ULP Message transferred through the RDMA Write Message into the ULP Buffer.

* 如果RDMA写入成功完成,RDMAP层不会将RDMA写入发送到ULP。它将通过RDMA写入消息传输的ULP消息放入ULP缓冲区。

* If the RDMA Write completed in error, the Data Sink RDMAP layer will pass up the corresponding error information to the Data Sink ULP and send a Terminate Message to the Data Source RDMAP layer. The Data Source RDMAP layer will then pass up the Terminate Message to the ULP.

* 如果RDMA写入错误完成,数据接收器RDMAP层将向数据接收器ULP传递相应的错误信息,并向数据源RDMAP层发送终止消息。然后,数据源RDMAP层将终止消息传递给ULP。

For RDMA Read operations, the following are the interactions between the RDMAP layer and the ULP:

对于RDMA读取操作,以下是RDMAP层和ULP之间的交互:

* At the Data Sink:

* 在数据接收器处:

* The ULP passes to the RDMAP layer the following:

* ULP通过以下方式传递到RDMAP层:

* ULP Message Length

* ULP消息长度

* Data Source STag

* 数据源STag

* Data Sink STag

* 数据接收STag

* Data Source Tagged Offset

* 数据源标记偏移量

* Data Sink Tagged Offset

* 数据接收器标记偏移量

* When the RDMA Read operation Completes, an indication of the Completion results.

* RDMA读取操作完成时,会显示完成结果的指示。

* At the Data Source:

* 在数据源:

* If no error occurred while processing the RDMA Read Request, the Data Source will not pass up any information to the ULP.

* 如果在处理RDMA读取请求时未发生错误,则数据源不会向ULP传递任何信息。

* If an error occurred while processing the RDMA Read Request, the Data Source RDMAP layer will pass up the corresponding error information to the Data Source ULP and send a Terminate Message to the Data Sink RDMAP layer. The Data Sink RDMAP layer will then pass up the Terminate Message to the ULP.

* 如果在处理RDMA读取请求时发生错误,数据源RDMAP层将向数据源ULP传递相应的错误信息,并向数据接收器RDMAP层发送终止消息。然后,数据接收器RDMAP层将终止消息传递给ULP。

For STags made available to the RDMAP layer, following are the interactions between the RDMAP layer and the ULP:

对于RDMAP层可用的STAG,以下是RDMAP层和ULP之间的交互:

* If the ULP enables an STag, the ULP passes the following to the RDMAP layer:

* 如果ULP启用STag,ULP会将以下内容传递给RDMAP层:

* STag;

* 雄鹿

* range of Tagged Offsets that are associated with a given STag;

* 与给定STag关联的标记偏移范围;

* remote access rights (read, write, or read and write) associated with a given, valid STag; and

* 与给定有效STag关联的远程访问权限(读、写或读写);和

* association between a given STag and a given RDMAP Stream.

* 给定STag和给定RDMAP流之间的关联。

* If the ULP disables an STag, the ULP passes to the RDMAP layer the STag.

* 如果ULP禁用STag,ULP将STag传递到RDMAP层。

If an error occurs at the RDMAP layer, the RDMAP layer may pass back error information (e.g., the content of a Terminate Message) to the ULP.

如果在RDMAP层发生错误,RDMAP层可能会将错误信息(例如,终止消息的内容)传回ULP。

4. Header Format
4. 标题格式

The control information of RDMA Messages is included in DDP protocol-defined header fields, with the following exceptions:

RDMA消息的控制信息包含在DDP协议定义的头字段中,但以下例外情况除外:

* The first octet reserved for ULP usage on all DDP Messages in the DDP Protocol (i.e., the RsvdULP Field) is used by RDMAP to carry the RDMA Message Opcode and the RDMAP version. This octet is known as the RDMAP Control Field in this specification. For Send with Invalidate and Send with Solicited Event and Invalidate, RDMAP uses the second through fifth octets, provided by DDP on Untagged DDP Messages, to carry the STag that will be Invalidated.

* RDMAP使用DDP协议中所有DDP消息(即RsvdULP字段)上为ULP使用保留的第一个八位组来携带RDMA消息操作码和RDMAP版本。此八位字节在本规范中称为RDMAP控制字段。对于使用Invalidate发送和使用请求的事件发送以及Invalidate,RDMAP使用DDP在未标记的DDP消息上提供的第二到第五个八位字节来携带将失效的STag。

* The RDMA Message length is passed by the RDMAP layer to the DDP layer on all outbound transfers.

* RDMA消息长度由RDMAP层在所有出站传输中传递给DDP层。

* For RDMA Read Request Messages, the RDMA Read Message Size is included in the RDMA Read Request Header.

* 对于RDMA读取请求消息,RDMA读取消息大小包含在RDMA读取请求标头中。

* The RDMA Message length is passed to the RDMAP layer by the DDP layer on inbound Untagged Buffer transfers.

* RDMA消息长度在入站未标记缓冲区传输时由DDP层传递给RDMAP层。

* Two RDMA Messages carry additional RDMAP headers. The RDMA Read Request carries the Data Sink and Data Source buffer descriptions, including buffer length. The Terminate carries additional information associated with the error that caused the Terminate.

* 两条RDMA消息携带额外的RDMAP头。RDMA读取请求包含数据接收器和数据源缓冲区描述,包括缓冲区长度。终止包含与导致终止的错误相关的附加信息。

4.1. RDMAP Control and Invalidate STag Field
4.1. RDMAP控制和失效STag字段

The version of RDMAP defined by this specification uses all 8 bits of the RDMAP Control Field. The first octet reserved for ULP use in the DDP Protocol MUST be used by the RDMAP to carry the RDMAP Control Field. The ordering of the bits in the first octet MUST be as defined in Figure 3, "DDP Control, RDMAP Control, and Invalidate STag Fields". For Send with Invalidate and Send with Solicited Event and Invalidate, the second through fifth octets of the DDP RsvdULP field MUST be used by RDMAP to carry the Invalidate STag. Figure 3 depicts the format of the DDP Control and RDMAP Control fields. (Note: In Figure 3, the DDP Header is offset by 16 bits to accommodate the MPA header defined in [MPA]. The MPA header is only present if DDP is layered on top of MPA.)

本规范定义的RDMAP版本使用RDMAP控制字段的所有8位。RDMAP必须使用DDP协议中为ULP使用保留的第一个八位组来携带RDMAP控制字段。第一个八位组中位的顺序必须如图3“DDP控制、RDMAP控制和失效STag字段”中所定义。对于使用Invalidate发送和使用请求事件发送以及Invalidate,RDMAP必须使用DDP RsvdULP字段的第二到第五个八位字节来携带Invalidate STag。图3描述了DDP控件和RDMAP控件字段的格式。(注意:在图3中,DDP头被偏移16位以容纳[MPA]中定义的MPA头。仅当DDP分层在MPA顶部时,才会出现MPA头。)

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
                                   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                                   |T|L| Resrv | DV| RV|Rsv| Opcode|
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                     Invalidate STag                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        
    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
                                   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                                   |T|L| Resrv | DV| RV|Rsv| Opcode|
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                     Invalidate STag                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        

Figure 3: DDP Control, RDMAP Control, and Invalidate STag Fields

图3:DDP控件、RDMAP控件和Invalidate STag字段

All RDMA Messages handed by the RDMAP layer to the DDP layer MUST define the value of the Tagged flag in the DDP Header. Figure 4, "RDMA Usage of DDP Fields", MUST be used to define the value of the Tagged flag that is handed to the DDP layer for each RDMA Message.

RDMAP层传递给DDP层的所有RDMA消息必须在DDP头中定义标记标志的值。图4“RDMA使用DDP字段”必须用于定义每个RDMA消息传递给DDP层的标记标志的值。

Figure 4 defines the value of the RDMA Opcode field that MUST be used for each RDMA Message.

图4定义了每个RDMA消息必须使用的RDMA操作码字段的值。

Figure 4 defines when the STag, Queue Number, and Tagged Offset fields MUST be provided for each RDMA Message.

图4定义了何时必须为每个RDMA消息提供STag、Queue Number和taged Offset字段。

For this version of the RDMAP, all RDMA Messages MUST have:

对于此版本的RDMAP,所有RDMA消息必须具有:

* Bits 24-25; RDMA Version field: 01b for an RNIC that complies with this RDMA protocol specification. 00b for an RNIC that complies with the RDMA Consortium's RDMA protocol specification. Both version numbers are valid. Interoperability is dependent on MPA protocol version negotiation (e.g., MPA marker and MPA CRC).

* 位24-25;RDMA版本字段:01b,用于符合此RDMA协议规范的RNIC。00b适用于符合RDMA联盟RDMA协议规范的RNIC。两个版本号都有效。互操作性取决于MPA协议版本协商(例如,MPA标记和MPA CRC)。

* Bits 26-27; Reserved. MUST be set to zero by sender, ignored by the receiver.

* 位26-27;保留的。发送方必须将其设置为零,接收方忽略。

* Bits 28-31; OpCode field: see Figure 4.

* 位28-31;操作码字段:参见图4。

* Bits 32-63; Invalidate STag. However, this field is only valid for Send with Invalidate and Send with Solicited Event and Invalidate Messages (see Figure 4).

* 位32-63;使雄鹿无效。但是,此字段仅对使用Invalidate发送和使用请求事件发送以及使用Invalidate消息发送有效(参见图4)。

For Send, Send with Solicited Event, RDMA Read Request, and Terminate, the Invalidate STag field MUST be set to zero on transmit and ignored by the receiver.

对于Send、Send with requested Event、RDMA Read Request和Terminate,必须在发送时将Invalidate STag字段设置为零,并由接收方忽略。

   -------+-----------+-------+------+-------+-----------+--------------
   RDMA   | Message   | Tagged| STag | Queue | Invalidate| Message
   Message| Type      | Flag  | and  | Number| STag      | Length
   OpCode |           |       | TO   |       |           | Communicated
          |           |       |      |       |           | between DDP
          |           |       |      |       |           | and RDMAP
   -------+-----------+-------+------+-------+-----------+--------------
   0000b  | RDMA Write| 1     | Valid| N/A   | N/A       | Yes
          |           |       |      |       |           |
   -------+-----------+-------+------+-------+-----------+--------------
   0001b  | RDMA Read | 0     | N/A  | 1     | N/A       | Yes
          | Request   |       |      |       |           |
   -------+-----------+-------+------+-------+-----------+--------------
   0010b  | RDMA Read | 1     | Valid| N/A   | N/A       | Yes
          | Response  |       |      |       |           |
   -------+-----------+-------+------+-------+-----------+--------------
   0011b  | Send      | 0     | N/A  | 0     | N/A       | Yes
          |           |       |      |       |           |
   -------+-----------+-------+------+-------+-----------+--------------
   0100b  | Send with | 0     | N/A  | 0     | Valid     | Yes
          | Invalidate|       |      |       |           |
   -------+-----------+-------+------+-------+-----------+--------------
   0101b  | Send with | 0     | N/A  | 0     | N/A       | Yes
          | SE        |       |      |       |           |
   -------+-----------+-------+------+-------+-----------+--------------
   0110b  | Send with | 0     | N/A  | 0     | Valid     | Yes
          | SE and    |       |      |       |           |
          | Invalidate|       |      |       |           |
   -------+-----------+-------+------+-------+-----------+--------------
   0111b  | Terminate | 0     | N/A  | 2     | N/A       | Yes
          |           |       |      |       |           |
   -------+-----------+-------+------+-------+-----------+--------------
   1000b  |           |
   to     | Reserved  |               Not Specified
   1111b  |           |
   -------+-----------+-------------------------------------------------
        
   -------+-----------+-------+------+-------+-----------+--------------
   RDMA   | Message   | Tagged| STag | Queue | Invalidate| Message
   Message| Type      | Flag  | and  | Number| STag      | Length
   OpCode |           |       | TO   |       |           | Communicated
          |           |       |      |       |           | between DDP
          |           |       |      |       |           | and RDMAP
   -------+-----------+-------+------+-------+-----------+--------------
   0000b  | RDMA Write| 1     | Valid| N/A   | N/A       | Yes
          |           |       |      |       |           |
   -------+-----------+-------+------+-------+-----------+--------------
   0001b  | RDMA Read | 0     | N/A  | 1     | N/A       | Yes
          | Request   |       |      |       |           |
   -------+-----------+-------+------+-------+-----------+--------------
   0010b  | RDMA Read | 1     | Valid| N/A   | N/A       | Yes
          | Response  |       |      |       |           |
   -------+-----------+-------+------+-------+-----------+--------------
   0011b  | Send      | 0     | N/A  | 0     | N/A       | Yes
          |           |       |      |       |           |
   -------+-----------+-------+------+-------+-----------+--------------
   0100b  | Send with | 0     | N/A  | 0     | Valid     | Yes
          | Invalidate|       |      |       |           |
   -------+-----------+-------+------+-------+-----------+--------------
   0101b  | Send with | 0     | N/A  | 0     | N/A       | Yes
          | SE        |       |      |       |           |
   -------+-----------+-------+------+-------+-----------+--------------
   0110b  | Send with | 0     | N/A  | 0     | Valid     | Yes
          | SE and    |       |      |       |           |
          | Invalidate|       |      |       |           |
   -------+-----------+-------+------+-------+-----------+--------------
   0111b  | Terminate | 0     | N/A  | 2     | N/A       | Yes
          |           |       |      |       |           |
   -------+-----------+-------+------+-------+-----------+--------------
   1000b  |           |
   to     | Reserved  |               Not Specified
   1111b  |           |
   -------+-----------+-------------------------------------------------
        

Figure 4: RDMA Usage of DDP Fields

图4:DDP字段的RDMA使用

Note: N/A means Not Applicable.

注:不适用表示不适用。

4.2. RDMA Message Definitions
4.2. RDMA消息定义

The following figure defines which RDMA Headers MUST be used on each RDMA Message and which RDMA Messages are allowed to carry ULP Payload:

下图定义了每个RDMA消息必须使用哪些RDMA头,以及允许哪些RDMA消息携带ULP有效负载:

   -------+-----------+-------------------+-------------------------
   RDMA   | Message   | RDMA Header Used  | ULP Message allowed in
   Message| Type      |                   | the RDMA Message
   OpCode |           |                   |
          |           |                   |
   -------+-----------+-------------------+-------------------------
   0000b  | RDMA Write| None              | Yes
          |           |                   |
   -------+-----------+-------------------+-------------------------
   0001b  | RDMA Read | RDMA Read Request | No
          | Request   | Header            |
   -------+-----------+-------------------+-------------------------
   0010b  | RDMA Read | None              | Yes
          | Response  |                   |
   -------+-----------+-------------------+-------------------------
   0011b  | Send      | None              | Yes
          |           |                   |
   -------+-----------+-------------------+-------------------------
   0100b  | Send with | None              | Yes
          | Invalidate|                   |
   -------+-----------+-------------------+-------------------------
   0101b  | Send with | None              | Yes
          | SE        |                   |
   -------+-----------+-------------------+-------------------------
   0110b  | Send with | None              | Yes
          | SE and    |                   |
          | Invalidate|                   |
   -------+-----------+-------------------+-------------------------
   0111b  | Terminate | Terminate Header  | No
          |           |                   |
   -------+-----------+-------------------+-------------------------
   1000b  |           |
   to     | Reserved  |            Not Specified
   1111b  |           |
   -------+-----------+-------------------+-------------------------
        
   -------+-----------+-------------------+-------------------------
   RDMA   | Message   | RDMA Header Used  | ULP Message allowed in
   Message| Type      |                   | the RDMA Message
   OpCode |           |                   |
          |           |                   |
   -------+-----------+-------------------+-------------------------
   0000b  | RDMA Write| None              | Yes
          |           |                   |
   -------+-----------+-------------------+-------------------------
   0001b  | RDMA Read | RDMA Read Request | No
          | Request   | Header            |
   -------+-----------+-------------------+-------------------------
   0010b  | RDMA Read | None              | Yes
          | Response  |                   |
   -------+-----------+-------------------+-------------------------
   0011b  | Send      | None              | Yes
          |           |                   |
   -------+-----------+-------------------+-------------------------
   0100b  | Send with | None              | Yes
          | Invalidate|                   |
   -------+-----------+-------------------+-------------------------
   0101b  | Send with | None              | Yes
          | SE        |                   |
   -------+-----------+-------------------+-------------------------
   0110b  | Send with | None              | Yes
          | SE and    |                   |
          | Invalidate|                   |
   -------+-----------+-------------------+-------------------------
   0111b  | Terminate | Terminate Header  | No
          |           |                   |
   -------+-----------+-------------------+-------------------------
   1000b  |           |
   to     | Reserved  |            Not Specified
   1111b  |           |
   -------+-----------+-------------------+-------------------------
        

Figure 5: RDMA Message Definitions

图5:RDMA消息定义

4.3. RDMA Write Header
4.3. RDMA写入头

The RDMA Write Message does not include an RDMAP header. The RDMAP layer passes to the DDP layer an RDMAP Control Field. The RDMA Write Message is fully described by the DDP Headers of the DDP Segments associated with the Message.

RDMA写入消息不包括RDMAP头。RDMAP层向DDP层传递一个RDMAP控制字段。RDMA写入消息由与消息关联的DDP段的DDP头完全描述。

See Appendix A for a description of the DDP Segment format associated with RDMA Write Messages.

有关与RDMA写入消息相关的DDP段格式的说明,请参见附录A。

4.4. RDMA Read Request Header
4.4. RDMA读取请求头

The RDMA Read Request Message carries an RDMA Read Request Header that describes the Data Sink and Data Source Buffers used by the RDMA Read operation. The RDMA Read Request Header immediately follows the DDP header. The RDMAP layer passes to the DDP layer an RDMAP Control Field. The following figure depicts the RDMA Read Request Header that MUST be used for all RDMA Read Request Messages:

RDMA Read Request消息包含一个RDMA Read Request头,该头描述RDMA Read操作使用的数据接收器和数据源缓冲区。RDMA读取请求头紧跟在DDP头之后。RDMAP层向DDP层传递一个RDMAP控制字段。下图描述了必须用于所有RDMA读取请求消息的RDMA读取请求标头:

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                     Data Sink STag (SinkSTag)                 |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                                                               |
    +                  Data Sink Tagged Offset (SinkTO)             +
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                  RDMA Read Message Size (RDMARDSZ)            |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                     Data Source STag (SrcSTag)                |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                                                               |
    +                 Data Source Tagged Offset (SrcTO)             +
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        
     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                     Data Sink STag (SinkSTag)                 |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                                                               |
    +                  Data Sink Tagged Offset (SinkTO)             +
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                  RDMA Read Message Size (RDMARDSZ)            |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                     Data Source STag (SrcSTag)                |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                                                               |
    +                 Data Source Tagged Offset (SrcTO)             +
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        

Figure 6: RDMA Read Request Header Format

图6:RDMA读取请求头格式

Data Sink Steering Tag: 32 bits.

数据接收器转向标签:32位。

The Data Sink Steering Tag identifies the Data Sink's Tagged Buffer. This field MUST be copied, without interpretation, from the RDMA Read Request into the corresponding RDMA Read Response; this field allows the Data Sink to place the returning data. The STag is associated with the RDMAP Stream through a mechanism that is outside the scope of the RDMAP specification.

数据接收器导向标记标识数据接收器的标记缓冲区。必须将此字段从RDMA读取请求复制到相应的RDMA读取响应中,无需解释;此字段允许数据接收器放置返回的数据。STag通过RDMAP规范范围之外的机制与RDMAP流相关联。

Data Sink Tagged Offset: 64 bits.

数据接收器标记的偏移量:64位。

The Data Sink Tagged Offset specifies the starting offset, in octets, from the base of the Data Sink's Tagged Buffer, where the data is to be written by the Data Source. This field is copied from the RDMA Read Request into the corresponding RDMA Read Response and allows the Data Sink to place the returning data. The Data Sink Tagged Offset MAY start at an arbitrary offset.

数据接收器标记的偏移量指定从数据接收器标记的缓冲区底部开始的偏移量(以八位字节为单位),数据源将在该缓冲区中写入数据。此字段从RDMA读取请求复制到相应的RDMA读取响应中,并允许数据接收器放置返回的数据。数据接收器标记的偏移量可以从任意偏移量开始。

The Data Sink STag and Data Sink Tagged Offset fields describe the buffer to which the RDMA Read data is written.

Data Sink STag和Data Sink Tagged Offset字段描述RDMA读取数据写入的缓冲区。

Note: the DDP layer protects against a wrap of the Data Sink Tagged Offset.

注意:DDP层可防止数据接收器标记偏移的包裹。

RDMA Read Message Size: 32 bits.

RDMA读取消息大小:32位。

The RDMA Read Message Size is the amount of data, in octets, read from the Data Source. A single RDMA Read Request Message can retrieve from 0 to 2^32-1 data octets from the Data Source.

RDMA读取消息大小是从数据源读取的数据量(以八位字节为单位)。单个RDMA读取请求消息可以从数据源检索0到2^32-1个数据八位字节。

Data Source Steering Tag: 32 bits.

数据源转向标记:32位。

The Data Source Steering Tag identifies the Data Source's Tagged Buffer. The STag is associated with the RDMAP Stream through a mechanism that is outside the scope of the RDMAP specification.

数据源转向标记标识数据源的标记缓冲区。STag通过RDMAP规范范围之外的机制与RDMAP流相关联。

Data Source Tagged Offset: 64 bits.

数据源标记的偏移量:64位。

The Tagged Offset specifies the starting offset, in octets, that is to be read from the Data Source's Tagged Buffer. The Data Source Tagged Offset MAY start at an arbitrary offset.

标记的偏移量指定从数据源的标记缓冲区读取的起始偏移量(以八位字节为单位)。标记为偏移量的数据源可以从任意偏移量开始。

The Data Source STag and Data Source Tagged Offset fields describe the buffer from which the RDMA Read data is read.

数据源STag和数据源标记偏移字段描述了从中读取RDMA读取数据的缓冲区。

See Section 7.2, "Errors Detected at the Remote Peer on Incoming RDMA Messages", for a description of error checking required upon processing of an RDMA Read Request at the Data Source.

有关在数据源处理RDMA读取请求时所需的错误检查说明,请参见第7.2节“在远程对等机上检测到的传入RDMA消息错误”。

4.5. RDMA Read Response Header
4.5. RDMA读取响应头

The RDMA Read Response Message does not include an RDMAP header. The RDMAP layer passes to the DDP layer an RDMAP Control Field. The RDMA Read Response Message is fully described by the DDP Headers of the DDP Segments associated with the Message.

RDMA读取响应消息不包括RDMAP头。RDMAP层向DDP层传递一个RDMAP控制字段。RDMA读取响应消息由与消息关联的DDP段的DDP头完全描述。

See Appendix A for a description of the DDP Segment format associated with RDMA Read Response Messages.

有关与RDMA读取响应消息相关的DDP段格式的说明,请参见附录A。

4.6. Send Header and Send with Solicited Event Header
4.6. 发送头和发送请求的事件头

The Send and Send with Solicited Event Messages do not include an RDMAP header. The RDMAP layer passes to the DDP layer an RDMAP Control Field. The Send and Send with Solicited Event Messages are fully described by the DDP Headers of the DDP Segments associated with the Messages.

“发送”和“随请求发送”事件消息不包括RDMAP头。RDMAP层向DDP层传递一个RDMAP控制字段。Send和Send with required事件消息由与消息关联的DDP段的DDP头进行完整描述。

See Appendix A for a description of the DDP Segment format associated with Send and Send with Solicited Event Messages.

有关发送和请求事件消息发送相关的DDP段格式的说明,请参见附录A。

4.7. Send with Invalidate Header and Send with SE and Invalidate Header
4.7. 使用无效标头发送,使用SE和无效标头发送

The Send with Invalidate and Send with Solicited Event and Invalidate Messages do not include an RDMAP header. The RDMAP layer passes to the DDP layer an RDMAP Control Field and the Invalidate STag field (see section 4.1 RDMAP Control and Invalidate STag Field). The Send with Invalidate and Send with Solicited Event and Invalidate Messages are fully described by the DDP Headers of the DDP Segments associated with the Messages.

“使用无效发送”和“使用请求的事件发送”以及“使用无效发送”消息不包含RDMAP头。RDMAP层将RDMAP控制字段和失效STag字段传递给DDP层(参见第4.1节RDMAP控制和失效STag字段)。与消息关联的DDP段的DDP头完全描述了“发送时失效”和“发送时请求的事件”以及“无效”消息。

See Appendix A for a description of the DDP Segment format associated with Send and Send with Solicited Event Messages.

有关发送和请求事件消息发送相关的DDP段格式的说明,请参见附录A。

4.8. Terminate Header
4.8. 端头

The Terminate Message carries a Terminate Header that contains additional information associated with the cause of the Terminate. The Terminate Header immediately follows the DDP header. The RDMAP layer passes to the DDP layer an RDMAP Control Field. The following figure depicts a Terminate Header that MUST be used for the Terminate Message:

Terminate消息包含一个Terminate标头,其中包含与终止原因相关的附加信息。终止标头紧跟在DDP标头之后。RDMAP层向DDP层传递一个RDMAP控制字段。下图描述了必须用于终止消息的终止标头:

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |       Terminate Control             |      Reserved           |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |  DDP Segment Length  (if any) |                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
    |                                                               |
    //                                                             //
    |                  Terminated DDP Header (if any)               |
    +                                                               +
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                                                               |
    //                                                             //
    |                 Terminated RDMA Header (if any)               |
    +                                                               +
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        
     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |       Terminate Control             |      Reserved           |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |  DDP Segment Length  (if any) |                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
    |                                                               |
    //                                                             //
    |                  Terminated DDP Header (if any)               |
    +                                                               +
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                                                               |
    //                                                             //
    |                 Terminated RDMA Header (if any)               |
    +                                                               +
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        

Figure 7: Terminate Header Format

图7:终止标头格式

Terminate Control: 19 bits.

终止控制:19位。

The Terminate Control field MUST have the format defined in Figure 8 below.

Terminate控件字段必须具有下图8中定义的格式。

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    | Layer | EType |   Error Code  |HdrCt|
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        
     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    | Layer | EType |   Error Code  |HdrCt|
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        

Figure 8: Terminate Control Field

图8:终止控制字段

* Figure 9, "Terminate Control Field Values", defines the valid values that MUST be used for this field.

* 图9“终止控制字段值”定义了必须用于此字段的有效值。

* Layer: 4 bits.

* 层:4位。

Identifies the layer that encountered the error.

标识遇到错误的图层。

* EType (RDMA Error Type): 4 bits.

* EType(RDMA错误类型):4位。

Identifies the type of error that caused the Terminate. When the error is detected at the RDMAP layer, the RDMAP layer inserts the Error Type into this field. When the error is detected at an LLP layer, an LLP layer creates the Error Type

标识导致终止的错误类型。在RDMAP层检测到错误时,RDMAP层会将错误类型插入此字段。在LLP层检测到错误时,LLP层将创建错误类型

and the DDP layer passes it up to the RDMAP layer, and the RDMAP layer inserts it into this field.

DDP层将其传递给RDMAP层,RDMAP层将其插入该字段。

* Error Code: 8 bits.

* 错误代码:8位。

This field identifies the specific error that caused the Terminate. When the error is detected at the RDMAP layer, the RDMAP layer creates the Error Code. When the error is detected at an LLP layer, the LLP layer creates the Error Code, the DDP layer passes it up to the RDMAP layer, and the RDMAP layer inserts it into this field.

此字段标识导致终止的特定错误。当在RDMAP层检测到错误时,RDMAP层将创建错误代码。当在LLP层检测到错误时,LLP层创建错误代码,DDP层将其传递给RDMAP层,RDMAP层将其插入该字段。

* HdrCt: 3 bits.

* HdrCt:3位。

Header control bits:

报头控制位:

* M: bit 16. DDP Segment Length valid. See Figure 10 for when this bit SHOULD be set.

* M:第16位。DDP段长度有效。请参见图10了解何时应设置此位。

* D: bit 17. DDP Header Included. See Figure 10 for when this bit SHOULD be set.

* D:第17位。包括DDP标题。请参见图10了解何时应设置此位。

* R: bit 18. RDMAP Header Included. See Figure 10 for when this bit SHOULD be set.

* R:第18位。包括RDMAP头。请参见图10了解何时应设置此位。

   -------+-----------+-------+-------------+------+--------------------
   Layer  | Layer     | Error | Error Type  | Error| Error Code Name
          | Name      | Type  | Name        | Code |
   -------+-----------+-------+-------------+------+--------------------
          |           | 0000b | Local       | None | None - This error
          |           |       | Catastrophic|      | type does not have
          |           |       | Error       |      | an error code. Any
          |           |       |             |      | value in this field
          |           |       |             |      | is acceptable.
          |           +-------+-------------+------+--------------------
          |           |       |             | 00X  | Invalid STag
          |           |       |             +------+--------------------
          |           |       |             | 01X  | Base or bounds
          |           |       |             |      | violation
          |           |       | Remote      +------+--------------------
          |           | 0001b | Protection  | 02X  | Access rights
          |           |       | Error       |      | violation
          |           |       |             +------+--------------------
   0000b  | RDMA      |       |             | 03X  | STag not associated
          |           |       |             |      | with RDMAP Stream
          |           |       |             +------+--------------------
          |           |       |             | 04X  | TO wrap
          |           |       |             +------+--------------------
          |           |       |             | 09X  | STag cannot be
          |           |       |             |      | Invalidated
          |           |       |             +------+--------------------
          |           |       |             | FFX  | Unspecified Error
          |           +-------+-------------+------+--------------------
          |           |       |             | 05X  | Invalid RDMAP
          |           |       |             |      | version
          |           |       |             +------+--------------------
          |           |       |             | 06X  | Unexpected OpCode
          |           |       | Remote      +------+--------------------
          |           | 0010b | Operation   | 07X  | Catastrophic error,
          |           |       | Error       |      | localized to RDMAP
          |           |       |             |      | Stream
          |           |       |             +------+--------------------
          |           |       |             | 08X  | Catastrophic error,
          |           |       |             |      | global
          |           |       |             +------+--------------------
          |           |       |             | 09X  | STag cannot be
          |           |       |             |      | Invalidated
          |           |       |             +------+--------------------
          |           |       |             | FFX  | Unspecified Error
        
   -------+-----------+-------+-------------+------+--------------------
   Layer  | Layer     | Error | Error Type  | Error| Error Code Name
          | Name      | Type  | Name        | Code |
   -------+-----------+-------+-------------+------+--------------------
          |           | 0000b | Local       | None | None - This error
          |           |       | Catastrophic|      | type does not have
          |           |       | Error       |      | an error code. Any
          |           |       |             |      | value in this field
          |           |       |             |      | is acceptable.
          |           +-------+-------------+------+--------------------
          |           |       |             | 00X  | Invalid STag
          |           |       |             +------+--------------------
          |           |       |             | 01X  | Base or bounds
          |           |       |             |      | violation
          |           |       | Remote      +------+--------------------
          |           | 0001b | Protection  | 02X  | Access rights
          |           |       | Error       |      | violation
          |           |       |             +------+--------------------
   0000b  | RDMA      |       |             | 03X  | STag not associated
          |           |       |             |      | with RDMAP Stream
          |           |       |             +------+--------------------
          |           |       |             | 04X  | TO wrap
          |           |       |             +------+--------------------
          |           |       |             | 09X  | STag cannot be
          |           |       |             |      | Invalidated
          |           |       |             +------+--------------------
          |           |       |             | FFX  | Unspecified Error
          |           +-------+-------------+------+--------------------
          |           |       |             | 05X  | Invalid RDMAP
          |           |       |             |      | version
          |           |       |             +------+--------------------
          |           |       |             | 06X  | Unexpected OpCode
          |           |       | Remote      +------+--------------------
          |           | 0010b | Operation   | 07X  | Catastrophic error,
          |           |       | Error       |      | localized to RDMAP
          |           |       |             |      | Stream
          |           |       |             +------+--------------------
          |           |       |             | 08X  | Catastrophic error,
          |           |       |             |      | global
          |           |       |             +------+--------------------
          |           |       |             | 09X  | STag cannot be
          |           |       |             |      | Invalidated
          |           |       |             +------+--------------------
          |           |       |             | FFX  | Unspecified Error
        
   -------+-----------+-------+-------------+------+--------------------
   0001b  | DDP       | See DDP Specification [DDP] for a description of
          |           | the values and names.
   -------+-----------+-------+-----------------------------------------
   0010b  | LLP       | For MPA, see MPA Specification [MPA] for a
          |(e.g., MPA)| description of the values and names.
   -------+-----------+-------+-----------------------------------------
        
   -------+-----------+-------+-------------+------+--------------------
   0001b  | DDP       | See DDP Specification [DDP] for a description of
          |           | the values and names.
   -------+-----------+-------+-----------------------------------------
   0010b  | LLP       | For MPA, see MPA Specification [MPA] for a
          |(e.g., MPA)| description of the values and names.
   -------+-----------+-------+-----------------------------------------
        

Figure 9: Terminate Control Field Values

图9:终止控制字段值

Reserved: 13 bits. This field MUST be set to zero on transmit, ignored on receive.

保留:13位。此字段在发送时必须设置为零,在接收时忽略。

DDP Segment Length: 16 bits

DDP段长度:16位

The length handed up by the DDP layer when the error was detected. It MUST be valid if the M bit is set. It MUST be present when the D bit is set.

检测到错误时DDP层提供的长度。如果设置了M位,则它必须有效。当设置D位时,它必须存在。

Terminated DDP Header: 112 bits for Tagged Messages and 144 bits for Untagged Messages.

终止的DDP报头:112位用于标记消息,144位用于未标记消息。

The DDP Header of the incoming Message that is associated with the Terminate. The DDP Header is not present if the Terminate Error Type is a Local Catastrophic Error. It MUST be present if the D bit is set.

与Terminate关联的传入消息的DDP标头。如果终止错误类型为本地灾难性错误,则DDP标头不存在。如果设置了D位,它必须存在。

Terminated RDMA Header: 224 bits.

终止的RDMA头:224位。

The Terminated RDMA Header is only sent back if the terminate is associated with an RDMA Read Request Message. It MUST be present if the R bit is set.

只有当终止与RDMA读取请求消息关联时,才会发回终止的RDMA头。如果设置了R位,则必须存在。

If the terminate occurs before the first RDMA Read Request byte is processed, the original RDMA Read Request Header is sent back.

如果终止发生在处理第一个RDMA读取请求字节之前,则返回原始RDMA读取请求头。

If the terminate occurs after the first RDMA Read Request byte is processed, the RDMA Read Request Header is updated to reflect the current location of the RDMA Read operation that is in process:

如果终止发生在处理第一个RDMA读取请求字节之后,RDMA读取请求标头将更新,以反映正在处理的RDMA读取操作的当前位置:

* Data Sink STag = Data Sink STag originally sent in the RDMA Read Request.

* Data Sink STag=最初在RDMA读取请求中发送的数据接收器STag。

* Data Sink Tagged Offset = Current offset into the Data Sink Tagged Buffer. For example, if the RDMA Read Request was terminated after 2048 octets were sent, then the Data Sink Tagged Offset = the original Data Sink Tagged Offset + 2048.

* Data Sink Tagged Offset=数据接收器标记缓冲区中的当前偏移量。例如,如果RDMA读取请求在发送2048个八位字节后终止,则数据接收器标记的偏移量=原始数据接收器标记的偏移量+2048。

* Data Message size = Number of bytes left to transfer.

* 数据消息大小=要传输的剩余字节数。

* Data Source STag = Data Source STag in the RDMA Read Request.

* Data Source STag=RDMA读取请求中的数据源STag。

* Data Source Tagged Offset = Current offset into the Data Source Tagged Buffer. For example, if the RDMA Read Request was terminated after 2048 octets were sent, then the Data Source Tagged Offset = the original Data Source Tagged Offset + 2048.

* 数据源标记的偏移量=数据源标记的缓冲区中的当前偏移量。例如,如果RDMA读取请求在发送2048个八位字节后终止,则数据源标记的偏移量=原始数据源标记的偏移量+2048。

Note: if a given LLP does not define any termination codes for the RDMAP Termination message to use, then none would be used for that LLP.

注意:如果给定的LLP没有为RDMAP终止消息定义任何要使用的终止代码,那么该LLP将不使用任何终止代码。

Figure 10, "Error Type to RDMA Message Mapping", maps layer name and error types to each RDMA Message type:

图10,“错误类型到RDMA消息映射”,将层名称和错误类型映射到每个RDMA消息类型:

   ---------+-------------+------------+------------+-----------------
   Layer    | Error Type  | Terminate  | Terminate  | What type of
   Name     | Name        | Includes   | Includes   | RDMA Message can
            |             | DDP Header | RDMA Header| cause the error
            |             | and DDP    |            |
            |             | Segment    |            |
            |             | Length     |            |
   ---------+-------------+------------+------------+-----------------
            | Local       | No         | No         | Any
            | Catastrophic|            |            |
            | Error       |            |            |
            +-------------+------------+------------+-----------------
            | Remote      | Yes, if    | Yes        | Only RDMA Read
   RDMA     | Protection  | possible   |            | Request, Send
            | Error       |            |            | with Invalidate,
            |             |            |            | and Send with SE
            |             |            |            | and Invalidate
            +-------------+------------+------------+-----------------
            | Remote      | Yes, if    | No         | Any
            | Operation   | possible   |            |
            | Error       |            |            |
   ---------+-------------+------------+------------+-----------------
   DDP      | See DDP Spec| Yes        | No         | Any
            | [DDP]       |            |            |
   ---------+-------------+------------+------------+-----------------
   LLP      | See LLP Spec| No         | No         | Any
            | (e.g., MPA) |            |            |
        
   ---------+-------------+------------+------------+-----------------
   Layer    | Error Type  | Terminate  | Terminate  | What type of
   Name     | Name        | Includes   | Includes   | RDMA Message can
            |             | DDP Header | RDMA Header| cause the error
            |             | and DDP    |            |
            |             | Segment    |            |
            |             | Length     |            |
   ---------+-------------+------------+------------+-----------------
            | Local       | No         | No         | Any
            | Catastrophic|            |            |
            | Error       |            |            |
            +-------------+------------+------------+-----------------
            | Remote      | Yes, if    | Yes        | Only RDMA Read
   RDMA     | Protection  | possible   |            | Request, Send
            | Error       |            |            | with Invalidate,
            |             |            |            | and Send with SE
            |             |            |            | and Invalidate
            +-------------+------------+------------+-----------------
            | Remote      | Yes, if    | No         | Any
            | Operation   | possible   |            |
            | Error       |            |            |
   ---------+-------------+------------+------------+-----------------
   DDP      | See DDP Spec| Yes        | No         | Any
            | [DDP]       |            |            |
   ---------+-------------+------------+------------+-----------------
   LLP      | See LLP Spec| No         | No         | Any
            | (e.g., MPA) |            |            |
        

Figure 10: Error Type to RDMA Message Mapping

图10:RDMA消息映射的错误类型

5. Data Transfer
5. 数据传输
5.1. RDMA Write Message
5.1. RDMA写消息

An RDMA Write is used by the Data Source to transfer data to a previously Advertised Tagged Buffer at the Data Sink. The RDMA Write Message has the following semantics:

数据源使用RDMA写操作将数据传输到数据接收器上先前公布的标记缓冲区。RDMA写入消息具有以下语义:

* An RDMA Write Message MUST reference a Tagged Buffer. That is, the Data Source RDMAP layer MUST request that the DDP layer mark the Message as Tagged.

* RDMA写入消息必须引用标记的缓冲区。也就是说,数据源RDMAP层必须请求DDP层将消息标记为已标记。

* A valid RDMA Write Message MUST NOT be delivered to the Data Sink's ULP (i.e., it is placed by the DDP layer).

* 有效的RDMA写入消息不得传递到数据接收器的ULP(即,它由DDP层放置)。

* At the Remote Peer, when an invalid RDMA Write Message is delivered to the Remote Peer's RDMAP layer, an error is surfaced (see Section 7.1, "RDMAP Error Surfacing").

* 在远程对等方,当向远程对等方的RDMAP层发送无效RDMA写入消息时,会出现错误(请参阅第7.1节“RDMAP错误呈现”)。

* The Tagged Offset of a Tagged Buffer MAY start at a non-zero value.

* 标记缓冲区的标记偏移量可以从非零值开始。

* An RDMA Write Message MAY target all or part of a previously Advertised Buffer.

* RDMA写消息可能以先前公布的缓冲区的全部或部分为目标。

* The RDMAP does not define how the buffer(s) are used by an outbound RDMA Write or how they are addressed. For example, an implementation of RDMA may choose to allow a gather-list of non-contiguous data blocks to be the source of an RDMA Write. In this case, the data blocks would be combined by the Data Source and sent as a single RDMA Write Message to the Data Sink.

* RDMAP没有定义出站RDMA写入如何使用缓冲区或如何寻址缓冲区。例如,RDMA的实现可以选择允许非连续数据块的聚集列表作为RDMA写入的源。在这种情况下,数据块将由数据源组合,并作为单个RDMA写入消息发送到数据接收器。

* The Data Source RDMAP layer MUST issue RDMA Write Messages to the DDP layer in the order they were submitted by the ULP.

* 数据源RDMAP层必须按照ULP提交的顺序向DDP层发出RDMA写消息。

* At the Data Source, a subsequent Send (Send with Invalidate, Send with Solicited Event, or Send with Solicited Event and Invalidate) Message MAY be used to signal Delivery of previous RDMA Write Messages to the Data Sink, if the ULP chooses to signal Delivery in this fashion.

* 在数据源处,如果ULP选择以这种方式发送信号,则后续发送(使用无效发送、使用请求事件发送或使用请求事件和无效发送)消息可用于向数据接收器发送先前RDMA写入消息的发送信号。

* If the Local Peer wishes to write to multiple Tagged Buffers on the Remote Peer, the Local Peer MUST use multiple RDMA Write Messages. That is, a single RDMA Write Message can only write to one remote Tagged Buffer.

* 如果本地对等方希望写入远程对等方上的多个标记缓冲区,则本地对等方必须使用多个RDMA写入消息。也就是说,单个RDMA写入消息只能写入一个远程标记缓冲区。

* The Data Source MAY issue a zero-length RDMA Write Message.

* 数据源可能发出长度为零的RDMA写入消息。

5.2. RDMA Read Operation
5.2. RDMA读取操作

The RDMA Read operation MUST consist of a single RDMA Read Request Message and a single RDMA Read Response Message.

RDMA读取操作必须由单个RDMA读取请求消息和单个RDMA读取响应消息组成。

5.2.1. RDMA Read Request Message
5.2.1. RDMA读取请求消息

An RDMA Read Request is used by the Data Sink to transfer data from a previously Advertised Tagged Buffer at the Data Source to a Tagged Buffer at the Data Sink. The RDMA Read Request Message has the following semantics:

数据接收器使用RDMA读取请求将数据从数据源上先前公布的标记缓冲区传输到数据接收器上的标记缓冲区。RDMA读取请求消息具有以下语义:

* An RDMA Read Request Message MUST reference an Untagged Buffer. That is, the Local Peer's RDMAP layer MUST request that the DDP mark the Message as Untagged.

* RDMA读取请求消息必须引用未标记的缓冲区。也就是说,本地对等方的RDMAP层必须请求DDP将消息标记为未标记。

* One RDMA Read Request Message MUST consume one Untagged Buffer.

* 一条RDMA读取请求消息必须使用一个未标记的缓冲区。

* The Remote Peer's RDMAP layer MUST process an RDMA Read Request Message. A valid RDMA Read Request Message MUST NOT be delivered to the Data Sink's ULP (i.e., it is processed by the RDMAP layer).

* 远程对等方的RDMAP层必须处理RDMA读取请求消息。有效的RDMA读取请求消息不得传递到数据接收器的ULP(即,它由RDMAP层处理)。

* At the Remote Peer, when an invalid RDMA Read Request Message is delivered to the Remote Peer's RDMAP layer, an error is surfaced (see Section 7.1, "RDMAP Error Surfacing").

* 在远程对等方,当向远程对等方的RDMAP层发送无效RDMA读取请求消息时,会出现错误(请参阅第7.1节“RDMAP错误呈现”)。

* An RDMA Read Request Message MUST reference the RDMA Read Request Queue. That is, the Local Peer's RDMAP layer MUST request that the DDP layer set the Queue Number field to one.

* RDMA读取请求消息必须引用RDMA读取请求队列。也就是说,本地对等方的RDMAP层必须请求DDP层将队列号字段设置为1。

* The Local Peer MUST pass to the DDP layer RDMA Read Request Messages in the order they were submitted by the ULP.

* 本地对等方必须按照ULP提交的顺序将RDMA读取请求消息传递给DDP层。

* The Remote Peer MUST process the RDMA Read Request Messages in the order they were sent.

* 远程对等方必须按照发送顺序处理RDMA读取请求消息。

* If the Local Peer wishes to read from multiple Tagged Buffers on the Remote Peer, the Local Peer MUST use multiple RDMA Read Request Messages. That is, a single RDMA Read Request Message MUST only read from one remote Tagged Buffer.

* 如果本地对等方希望从远程对等方上的多个标记缓冲区读取,则本地对等方必须使用多个RDMA读取请求消息。也就是说,单个RDMA读取请求消息只能从一个远程标记缓冲区读取。

* AN RDMA Read Request Message MAY target all or part of a previously Advertised Buffer.

* RDMA读取请求消息可能以先前公布的缓冲区的全部或部分为目标。

* If the Data Source receives a valid RDMA Read Request Message, it MUST respond with a valid RDMA Read Response Message.

* 如果数据源收到有效的RDMA读取请求消息,则必须使用有效的RDMA读取响应消息进行响应。

* The Data Sink MAY issue a zero-length RDMA Read Request Message by setting the RDMA Read Message Size field to zero in the RDMA Read Request Header.

* 通过在RDMA读取请求报头中将RDMA读取消息大小字段设置为零,数据接收器可以发出长度为零的RDMA读取请求消息。

* If the Data Source receives a non-zero-length RDMA Read Message Size, the Data Source RDMAP MUST validate the Data Source STag and Data Source Tagged Offset contained in the RDMA Read Request Header.

* 如果数据源接收到非零长度的RDMA读取消息大小,则数据源RDMAP必须验证RDMA读取请求标头中包含的数据源STag和数据源标记偏移量。

* If the Data Source receives an RDMA Read Request Header with the RDMA Read Message Size set to zero, the Data Source RDMAP:

* 如果数据源接收到RDMA读取请求标头,且RDMA读取消息大小设置为零,则数据源RDMAP:

* MUST NOT validate the Data Source STag and Data Source Tagged Offset contained in the RDMA Read Request Header, and

* 不得验证RDMA读取请求标头中包含的数据源STag和数据源标记偏移量,以及

* MUST respond with a zero-length RDMA Read Response Message.

* 必须使用零长度RDMA读取响应消息进行响应。

5.2.2. RDMA Read Response Message
5.2.2. RDMA读取响应消息

The RDMA Read Response Message uses the DDP Tagged Buffer Model to Deliver the contents of a previously requested Data Source Tagged Buffer to the Data Sink, without any involvement from the ULP at the Remote Peer. The RDMA Read Response Message has the following semantics:

RDMA读取响应消息使用DDP标记缓冲区模型将先前请求的数据源标记缓冲区的内容传送到数据接收器,而无需远程对等方的ULP参与。RDMA读取响应消息具有以下语义:

* The RDMA Read Response Message for the associated RDMA Read Request Message travels in the opposite direction.

* 关联RDMA读取请求消息的RDMA读取响应消息以相反方向移动。

* An RDMA Read Response Message MUST reference a Tagged Buffer. That is, the Data Source RDMAP layer MUST request that the DDP mark the Message as Tagged.

* RDMA读取响应消息必须引用标记的缓冲区。也就是说,数据源RDMAP层必须请求DDP将消息标记为已标记。

* The Data Source MUST ensure that a sufficient number of Untagged Buffers are available on the RDMA Read Request Queue (Queue with DDP Queue Number 1) to support the maximum number of RDMA Read Requests negotiated by the ULP.

* 数据源必须确保RDMA读取请求队列(DDP队列号为1的队列)上有足够数量的未标记缓冲区,以支持ULP协商的最大RDMA读取请求数。

* The RDMAP layer MUST Deliver the RDMA Read Response Message to the ULP.

* RDMAP层必须将RDMA读取响应消息传递给ULP。

* At the Remote Peer, when an invalid RDMA Read Response Message is delivered to the Remote Peer's RDMAP layer, an error is surfaced (see Section 7.1, "RDMAP Error Surfacing").

* 在远程对等方,当向远程对等方的RDMAP层发送无效RDMA读取响应消息时,会出现错误(请参阅第7.1节“RDMAP错误呈现”)。

* The Tagged Offset of a Tagged Buffer MAY start at a non-zero value.

* 标记缓冲区的标记偏移量可以从非零值开始。

* The Data Source RDMAP layer MUST pass RDMA Read Response Messages to the DDP layer, in the order that the RDMA Read Request Messages were received by the RDMAP layer, at the Data Source.

* 数据源RDMAP层必须按照RDMAP层在数据源接收RDMA读取请求消息的顺序,将RDMA读取响应消息传递给DDP层。

* The Data Sink MAY validate that the STag, Tagged Offset, and length of the RDMA Read Response Message are the same as the STag, Tagged Offset, and length included in the corresponding RDMA Read Request Message.

* 数据接收器可以验证RDMA读取响应消息的STag、标记的偏移量和长度是否与对应RDMA读取请求消息中包括的STag、标记的偏移量和长度相同。

* A single RDMA Read Response Message MUST write to one remote Tagged Buffer. If the Data Sink wishes to read multiple Tagged Buffers, the Data Sink can use multiple RDMA Read Request Messages.

* 单个RDMA读取响应消息必须写入一个远程标记缓冲区。如果数据接收器希望读取多个带标签的缓冲区,则数据接收器可以使用多个RDMA读取请求消息。

5.3. Send Message Type
5.3. 发送消息类型

The Send Message Type uses the DDP Untagged Buffer Model to transfer data from the Data Source into an Untagged Buffer at the Data Sink.

发送消息类型使用DDP未标记缓冲区模型将数据从数据源传输到数据接收器的未标记缓冲区。

* A Send Message Type MUST reference an Untagged Buffer. That is, the Local Peer's RDMAP layer MUST request that the DDP layer mark the Message as Untagged.

* 发送消息类型必须引用未标记的缓冲区。也就是说,本地对等方的RDMAP层必须请求DDP层将消息标记为未标记。

* One Send Message Type MUST consume one Untagged Buffer.

* 一个发送消息类型必须使用一个未标记的缓冲区。

* The ULP Message sent using a Send Message Type MAY be less than or equal to the size of the consumed Untagged Buffer. The RDMAP layer communicates to the ULP the size of the data written into the Untagged Buffer.

* 使用发送消息类型发送的ULP消息可能小于或等于已使用的未标记缓冲区的大小。RDMAP层将写入未标记缓冲区的数据大小告知ULP。

* If the ULP Message sent via Send Message Type is larger than the Data Sink's Untagged Buffer, it is an error (see Section 9.1, "RDMAP Error Surfacing").

* 如果通过发送消息类型发送的ULP消息大于数据接收器的未标记缓冲区,则为错误(请参阅第9.1节“RDMAP错误显示”)。

* At the Remote Peer, the Send Message Type MUST be Delivered to the Remote Peer's ULP in the order they were sent.

* 在远程对等方,发送消息类型必须按照发送顺序发送到远程对等方的ULP。

* After the Send with Solicited Event or Send with Solicited Event and Invalidate Message is Delivered to the ULP, the RDMAP MAY generate an Event, if the Data Sink is configured to generate such an Event.

* 在请求发送事件或请求发送事件和失效消息被传递到ULP之后,如果数据接收器被配置为生成这样的事件,RDMAP可以生成事件。

* At the Remote Peer, when an invalid Send Message Type is Delivered to the Remote Peer's RDMAP layer, an error is surfaced (see Section 7.1, "RDMAP Error Surfacing").

* 在远程对等方,当向远程对等方的RDMAP层发送无效的发送消息类型时,会出现错误(请参阅第7.1节“RDMAP错误呈现”)。

* The RDMAP does not specify the structure of the buffer(s) used by an outbound RDMA Write nor does it specify how the buffer(s) are addressed. For example, an implementation of RDMA may choose to allow a gather-list of non-contiguous data blocks to be the source of a Send Message Type. In this case, the data blocks would be combined by the Data Source and sent as a single Send Message Type to the Data Sink.

* RDMAP没有指定出站RDMA写入所使用的缓冲区的结构,也没有指定缓冲区的寻址方式。例如,RDMA的实现可以选择允许非连续数据块的聚集列表作为发送消息类型的源。在这种情况下,数据块将由数据源组合,并作为单个发送消息类型发送到数据接收器。

* For a Send Message Type, the Local Peer's RDMAP layer MUST request that the DDP layer set the Queue Number field to zero.

* 对于发送消息类型,本地对等方的RDMAP层必须请求DDP层将队列号字段设置为零。

* The Local Peer MUST issue Send Message Type Messages in the order they were submitted by the ULP.

* 本地对等方必须按照ULP提交的顺序发出发送消息类型的消息。

* The Data Source MAY pass a zero-length Send Message Type. A zero-length Send Message Type MUST consume an Untagged Buffer at the Data Sink. A Send with Invalidate or Send with Solicited Event and Invalidate Message MUST reference an STag. That is, the Local Peer's RDMAP layer MUST pass the RDMA control field and the STag that will be Invalidated to the DDP layer.

* 数据源可能传递长度为零的发送消息类型。长度为零的发送消息类型必须使用数据接收器上的未标记缓冲区。带有Invalidate的发送或带有请求事件和Invalidate消息的发送必须引用STag。也就是说,本地对等方的RDMAP层必须将RDMA控制字段和将失效的STag传递给DDP层。

* When the Send with Invalidate and Send with Solicited Event and Invalidate Message are Delivered to the Remote Peer's RDMAP layer, the RDMAP layer MUST:

* 当发送失效和发送请求事件以及失效消息传递到远程对等方的RDMAP层时,RDMAP层必须:

* Verify the STag that is associated with the RDMAP Stream; and

* 验证与RDMAP流关联的STag;和

* Invalidate the STag if it is associated with the RDMAP Stream; or issue a Terminate Message with the STag Cannot be Invalidated Terminate Error Code, if the STag is not associated with the RDMAP Stream.

* 如果STag与RDMAP流关联,则使其无效;或者,如果STag未与RDMAP流关联,则使用STag不能无效终止错误代码发出终止消息。

5.4. Terminate Message
5.4. 终止消息

The Terminate Message uses the DDP Untagged Buffer Model to transfer-error-related information from the Data Source into an Untagged Buffer at the Data Sink and then ceases all further communications on the underlying DDP Stream. The Terminate Message has the following semantics:

Terminate消息使用DDP Untaged Buffer模型将错误相关信息从数据源传输到数据接收器的Untaged Buffer中,然后停止底层DDP流上的所有进一步通信。终止消息具有以下语义:

* A Terminate Message MUST reference an Untagged Buffer. That is, the Local Peer's RDMAP layer MUST request that the DDP layer mark the Message as Untagged.

* 终止消息必须引用未标记的缓冲区。也就是说,本地对等方的RDMAP层必须请求DDP层将消息标记为未标记。

* A Terminate Message references the Terminate Queue. That is, the Local Peer's RDMAP layer MUST request that the DDP layer set the Queue Number field to two.

* 终止消息引用终止队列。也就是说,本地对等方的RDMAP层必须请求DDP层将队列号字段设置为2。

* One Terminate Message MUST consume one Untagged Buffer.

* 一条终止消息必须使用一个未标记的缓冲区。

* On a single RDMAP Stream, the RDMAP layer MUST guarantee placement of a single Terminate Message.

* 在单个RDMAP流上,RDMAP层必须保证放置单个终止消息。

* A Terminate Message MUST be Delivered to the Remote Peer's RDMAP layer. The RDMAP layer MUST Deliver the Terminate Message to the ULP.

* 必须将终止消息传递到远程对等方的RDMAP层。RDMAP层必须将终止消息传递给ULP。

* At the Remote Peer, when an invalid Terminate Message is delivered to the Remote Peer's RDMAP layer, an error is surfaced (see Section 7.1 "RDMAP Error Surfacing").

* 在远程对等方,当向远程对等方的RDMAP层发送无效终止消息时,会出现错误(请参阅第7.1节“RDMAP错误呈现”)。

* The RDMAP layer Completes in error all ULP operations that have not been provided to the DDP layer.

* RDMAP层错误地完成所有未提供给DDP层的ULP操作。

* After sending a Terminate Message on an RDMAP Stream, the Local Peer MUST NOT send any more Messages on that specific RDMAP Stream.

* 在RDMAP流上发送终止消息后,本地对等方不得在该特定RDMAP流上再发送任何消息。

* After receiving a Terminate Message on an RDMAP Stream, the Remote Peer MAY stop sending Messages on that specific RDMAP Stream.

* 在接收到RDMAP流上的终止消息后,远程对等方可以停止在该特定RDMAP流上发送消息。

5.5. Ordering and Completions
5.5. 排序与完成

It is important to understand the difference between Placement and Delivery ordering since RDMAP provides quite different semantics for the two.

理解放置顺序和交付顺序之间的区别很重要,因为RDMAP为两者提供了完全不同的语义。

Note that many current protocols, both as used in the Internet and elsewhere, assume that data is both Placed and Delivered in order. Taking advantage of this fact allowed applications to take a variety of shortcuts. For RDMAP, many of these shortcuts are no longer safe to use, and could cause application failure.

请注意,在Internet和其他地方使用的许多当前协议都假定数据是按顺序放置和传递的。利用这一事实,应用程序可以采用各种快捷方式。对于RDMAP,这些快捷方式中的许多不再安全,可能会导致应用程序失败。

The following rules apply to implementations of the RDMAP protocol. Note that in these rules, Send includes Send, Send with Invalidate, Send with Solicited Event, and Send with Solicited Event and Invalidate:

以下规则适用于RDMAP协议的实现。请注意,在这些规则中,发送包括发送、发送时失效、发送时请求的事件以及发送时请求的事件和失效:

1. RDMAP does not provide ordering among Messages on different RDMAP Streams.

1. RDMAP不提供不同RDMAP流上消息之间的排序。

2. RDMAP does not provide ordering between operations that are generated from the two ends of an RDMAP Stream.

2. RDMAP不提供从RDMAP流的两端生成的操作之间的排序。

3. RDMA Messages that use Tagged and Untagged Buffers MAY be Placed in any order. If an application uses overlapping buffers (points different Messages or portions of a single Message at the same buffer), then it is possible that the last incoming write to the Data Sink buffer will not be the last outgoing data sent from the Data Source.

3. 使用标记缓冲区和未标记缓冲区的RDMA消息可以按任意顺序放置。如果应用程序使用重叠缓冲区(将不同的消息或单个消息的一部分指向同一缓冲区),则对数据接收器缓冲区的最后一次传入写入可能不是从数据源发送的最后一次传出数据。

4. For a Send operation, the contents of an Untagged Buffer at the Data Sink MAY be indeterminate until the Send is Delivered to the ULP at the Data Sink.

4. 对于发送操作,数据接收器处未标记缓冲区的内容可能是不确定的,直到发送被发送到数据接收器处的ULP。

5. For an RDMA Write operation, the contents of the Tagged Buffer at the Data Sink MAY be indeterminate until a subsequent Send is Delivered to the ULP at the Data Sink.

5. 对于RDMA写入操作,在后续发送发送到数据接收器的ULP之前,数据接收器处标记缓冲区的内容可能是不确定的。

6. For an RDMA Read operation, the contents of the Tagged Buffer at the Data Sink MAY be indeterminate until the RDMA Read Response Message has been Delivered at the Local Peer.

6. 对于RDMA读取操作,在本地对等端传递RDMA读取响应消息之前,数据接收器处标记缓冲区的内容可能是不确定的。

Statements 4, 5, and 6 imply "no peeking" at the data to see if it is done. It is possible for some data to arrive before logically earlier data does, and peeking may cause unpredictable application failure.

语句4、5和6暗示“不要偷看”数据,看它是否完成了。某些数据可能在逻辑上较早的数据到达之前到达,偷看可能会导致不可预测的应用程序失败。

7. If the ULP or Application modifies the contents of Tagged or Untagged Buffers, which are being modified by an RDMA Operation while the RDMAP is processing the RDMA Operation, the state of the Buffers is indeterminate.

7. 如果ULP或应用程序修改已标记或未标记缓冲区的内容,这些内容在RDMAP处理RDMA操作时由RDMA操作修改,则缓冲区的状态不确定。

8. If the ULP or Application modifies the contents of Tagged or Untagged Buffers, which are read by an RDMA Operation while the RDMAP is processing the RDMA Operation, the results of the read are indeterminate.

8. 如果ULP或应用程序修改已标记或未标记缓冲区的内容(RDMA操作在RDMAP处理RDMA操作时读取),则读取结果不确定。

9. The Completion of an RDMA Write or Send Operation at the Local Peer does not guarantee that the ULP Message has yet reached the Remote Peer ULP Buffer or been examined by the Remote ULP.

9. 在本地对等端完成RDMA写入或发送操作并不保证ULP消息已到达远程对等端ULP缓冲区或已被远程ULP检查。

10. Send Messages MUST be Delivered to the ULP at the Remote Peer after they are Delivered to RDMAP by DDP and in the order that they were Delivered to RDMAP.

10. 在DDP将发送消息发送到RDMAP之后,必须按照发送到RDMAP的顺序将发送消息发送到远程对等方的ULP。

Note that DDP ordering rules ensure that this will be the same order that they were submitted at the Local Peer and that any prior RDMA Writes have been submitted for ordered Placement at the Remote Peer. This means that when the ULP sees the Delivery of the Send, the memory buffers targeted by any preceding RDMA Writes and Sends are available to be accessed locally or remotely as authorized. If the ULP overlaps its buffers for different operations, the data from the RDMA Write or Send may be overwritten by subsequent RDMA Operations before the ULP receives and processes the Delivery.

请注意,DDP排序规则确保该顺序与在本地对等机上提交的顺序相同,并且之前的任何RDMA写入都已提交,以便在远程对等机上有序放置。这意味着,当ULP看到发送的交付时,任何之前的RDMA写入和发送所针对的内存缓冲区都可以根据授权在本地或远程访问。如果ULP为不同的操作重叠其缓冲区,则在ULP接收和处理传递之前,来自RDMA写入或发送的数据可能会被后续RDMA操作覆盖。

11. RDMA Read Response Messages MUST be Delivered to the ULP at the Remote Peer after they are Delivered to RDMAP by DDP and in the order that the they were Delivered to RDMAP.

11. RDMA读取响应消息在DDP发送到RDMAP后,必须按照发送到RDMAP的顺序发送到远程对等方的ULP。

DDP ordering rules ensure that this will be the same order that they were submitted at the Local Peer. This means that when the ULP sees the Delivery of the RDMA Read Response, the memory buffers targeted by the RDMA Read Response are available to be accessed locally or remotely as authorized. If the ULP overlaps

DDP订购规则确保该订单与在本地同行提交的订单相同。这意味着,当ULP看到RDMA读取响应的交付时,RDMA读取响应所针对的内存缓冲区可在授权的情况下本地或远程访问。如果ULP重叠

its buffers for different operations, the data from the RDMA Read Response may be overwritten by subsequent RDMA Operations before the ULP receives and processes the Delivery.

对于不同的操作,在ULP接收和处理传递之前,来自RDMA读取响应的数据可能会被后续RDMA操作覆盖。

12. RDMA Read Request Messages, including zero-length RDMA Read Requests, MUST NOT start processing at the Remote Peer until they have been Delivered to RDMAP by DDP.

12. RDMA读取请求消息,包括长度为零的RDMA读取请求,在DDP将其发送到RDMAP之前,不得在远程对等机上开始处理。

Note: the ULP is assured that data written can be read back. For example, if

注意:ULP确保写入的数据可以读回。例如,如果

a) an RDMA Read Request is issued by the local peer, b) the Request targets the same ULP Buffer as a preceding Send or RDMA Write (in the same direction as the RDMA Read Request), and c) there are no other sources of update for the ULP Buffer,

a) RDMA读取请求由本地对等方发出,b)该请求针对与先前发送或RDMA写入相同的ULP缓冲区(与RDMA读取请求的方向相同),以及c)ULP缓冲区没有其他更新源,

then the Remote Peer will send back the data written by the Send or RDMA Write. That is, for this example, the ULP Buffer is Advertised for use on a series of RDMA Messages, is only valid on the RDMAP Stream for which it is Advertised, and is not locally updated while the series of RDMAP Messages are performed. For this example, order rule (12) assures that subsequent local or remote accesses to the ULP Buffer contain the data written by the Send or RDMA Write.

然后,远程对等方将发回由send或RDMA写操作写入的数据。也就是说,对于该示例,ULP缓冲区被通告用于一系列RDMA消息,仅在其被通告的RDMAP流上有效,并且在执行该系列RDMAP消息时不在本地更新。对于该示例,顺序规则(12)确保对ULP缓冲区的后续本地或远程访问包含由发送或RDMA写入写入的数据。

RDMA Read Response Messages MAY be generated at the Remote Peer after subsequent RDMA Write Messages or Send Messages have been Placed or Delivered. Therefore, when an application does an RDMA Read Request followed by an RDMA Write (or Send) to the same buffer, it may get the data from the later RDMA Write (or Send) in the RDMA Read Response Message, even though the operations completed in order at the Local Peer. If this behavior is not desired, the Local Peer ULP must Fence the later RDMA write (or Send) by withholding the RDMA Write Message until all outstanding RDMA Read Responses have been Delivered.

在放置或传递后续RDMA写消息或发送消息之后,可以在远程对等方生成RDMA读响应消息。因此,当应用程序执行RDMA读取请求,然后对同一缓冲区进行RDMA写入(或发送)时,它可能会从RDMA读取响应消息中稍后的RDMA写入(或发送)中获取数据,即使操作在本地对等端按顺序完成。如果不需要此行为,则本地对等ULP必须通过保留RDMA写入消息来限制稍后的RDMA写入(或发送),直到所有未完成的RDMA读取响应都已交付。

13. The RDMAP layer MUST submit RDMA Messages to the DDP layer in the order the RDMA Operations are submitted to the RDMAP layer by the ULP.

13. RDMAP层必须按照ULP将RDMA操作提交到RDMAP层的顺序将RDMA消息提交到DDP层。

14. A Send or RDMA Write Message MUST NOT be considered Complete at the Local Peer (Data Source) until it has been successfully completed at the DDP layer.

14. 在DDP层成功完成发送或RDMA写入消息之前,不得将其视为在本地对等方(数据源)完成。

15. RDMA Operations MUST be Completed at the Local Peer in the order that they were submitted by the ULP.

15. RDMA操作必须按照ULP提交的顺序在本地对等机上完成。

16. At the Data Sink, an incoming Send Message MUST be Delivered to the ULP only after the DDP Message has been Delivered to the RDMAP layer by the DDP layer.

16. 在数据接收器,只有在DDP层将DDP消息传递到RDMAP层之后,传入的发送消息才能传递到ULP。

17. RDMA Read Response Message processing at the Remote Peer (reading the specified Tagged Buffer) MUST be started only after the RDMA Read Request Message has been Delivered by the DDP layer (thus, all previous RDMA Messages have been properly submitted for ordered Placement).

17. 只有在DDP层交付RDMA读取请求消息之后,才能在远程对等机上启动RDMA读取响应消息处理(读取指定的标记缓冲区)(因此,所有以前的RDMA消息都已正确提交以便有序放置)。

18. Send Messages MAY be Completed at the Remote Peer (Data Sink) before prior incoming RDMA Read Request Messages have completed their response processing.

18. 在先前传入的RDMA读取请求消息完成响应处理之前,可以在远程对等方(数据接收器)完成发送消息。

19. An RDMA Read operation MUST NOT be Completed at the Local Peer until the DDP layer Delivers the associated incoming RDMA Read Response Message.

19. 在DDP层传递相关的传入RDMA读取响应消息之前,必须在本地对等方完成RDMA读取操作。

20. If more than one outstanding RDMA Read Request Messages are supported by both peers, the RDMA Read Response Messages MUST be submitted to the DDP layer on the Remote Peer in the order the RDMA Read Request Messages were Delivered by DDP, but the actual read of the buffer contents MAY take place in any order at the Remote Peer.

20. 如果两个对等方都支持多个未完成的RDMA读取请求消息,则必须按照RDMA读取请求消息由DDP交付的顺序将RDMA读取响应消息提交到远程对等方的DDP层,但缓冲区内容的实际读取可以在远程对等方以任何顺序进行。

This simplifies Local Peer Completion processing for RDMA Reads in that a Delivered RDMA Read Response MUST be sufficient to Complete the RDMA Read operation.

这简化了RDMA读取的本地对等完成处理,因为交付的RDMA读取响应必须足以完成RDMA读取操作。

6. RDMAP Stream Management
6. RDMAP流管理

RDMAP Stream management consists of RDMAP Stream Initialization and RDMAP Stream Termination.

RDMAP流管理包括RDMAP流初始化和RDMAP流终止。

6.1. Stream Initialization
6.1. 流初始化

RDMAP Stream initialization occurs after the LLP Stream has been created (e.g., for DDP/MPA over TCP, the first TCP Segment after the SYN, SYN/ACK exchange). The ULP is responsible for transitioning the LLP Stream into RDMA-enabled mode. The switch to RDMA mode typically occurs sometime after LLP Stream setup. Once in RDMA enabled mode, an implementation MUST send only RDMA Messages across the transport Stream until the RDMAP Stream is torn down.

RDMAP流初始化在创建LLP流之后发生(例如,对于TCP上的DDP/MPA,在SYN、SYN/ACK交换之后的第一个TCP段)。ULP负责将LLP流转换为RDMA启用模式。切换到RDMA模式通常在LLP流设置后的某个时间发生。一旦进入RDMA启用模式,实现必须仅在传输流上发送RDMA消息,直到RDMAP流被拆除。

For each direction of an RDMAP Stream:

对于RDMAP流的每个方向:

* For a given RDMAP Stream, the number of outstanding RDMA Read Requests is limited per RDMAP Stream direction.

* 对于给定的RDMAP流,每个RDMAP流方向的未完成RDMA读取请求数是有限的。

* It is the ULP's responsibility to set the maximum number of outstanding, inbound RDMA Read Requests per RDMAP Stream direction.

* ULP负责设置每个RDMAP流方向上未完成的入站RDMA读取请求的最大数量。

* The RDMAP layer MUST provide the maximum number of outstanding, inbound RDMA Read Requests per RDMAP Stream direction that were negotiated between the ULP and the Local Peer's RDMAP layer. The negotiation mechanism is outside the scope of this specification.

* RDMAP层必须提供ULP和本地对等RDMAP层之间协商的每个RDMAP流方向的最大未完成入站RDMA读取请求数。协商机制不在本规范的范围内。

* It is the ULP's responsibility to set the maximum number of outstanding, outbound RDMA Read Requests per RDMAP Stream direction.

* ULP负责设置每个RDMAP流方向上未完成的出站RDMA读取请求的最大数量。

* The RDMAP layer MUST provide the maximum number of outstanding, outbound RDMA Read Requests for the RDMAP Stream direction that were negotiated between the ULP and the Local Peer's RDMAP layer. The negotiation mechanism is outside the scope of this specification.

* RDMAP层必须为在ULP和本地对等方的RDMAP层之间协商的RDMAP流方向提供最大未完成的出站RDMA读取请求数。协商机制不在本规范的范围内。

* The Local Peer's ULP is responsible for negotiating with the Remote Peer's ULP the maximum number of outstanding RDMA Read Requests for the RDMAP Stream direction. It is recommended that the ULP set the maximum number of outstanding, inbound RDMA Read Requests equal to the maximum number of outstanding, outbound RDMA Read Requests for a given RDMAP Stream direction.

* 本地对等方的ULP负责与远程对等方的ULP协商RDMAP流方向的最大未完成RDMA读取请求数。建议ULP将未完成的入站RDMA读取请求的最大数量设置为给定RDMAP流方向的未完成的出站RDMA读取请求的最大数量。

* For outbound RDMA Read Requests, the RDMAP layer MUST NOT exceed the maximum number of outstanding, outbound RDMA Read Requests that were negotiated between the ULP and the Local Peer's RDMAP layer.

* 对于出站RDMA读取请求,RDMAP层不得超过ULP和本地对等方的RDMAP层之间协商的未完成出站RDMA读取请求的最大数量。

* For inbound RDMA Read Requests, the RDMAP layer MUST NOT exceed the maximum number of outstanding, inbound RDMA Read Requests that were negotiated between the ULP and the Local Peer's RDMAP layer.

* 对于入站RDMA读取请求,RDMAP层不得超过ULP和本地对等方的RDMAP层之间协商的未完成的入站RDMA读取请求的最大数量。

6.2. Stream Teardown
6.2. 流撕裂

There are three methods for terminating an RDMAP Stream: ULP Graceful Termination, RDMAP Abortive Termination, and LLP Abortive Termination.

终止RDMAP流有三种方法:ULP优雅终止、RDMAP中止终止和LLP中止终止。

The ULP is responsible for performing ULP Graceful Termination. After a ULP Graceful Termination, either side of the Stream can initiate LLP Graceful Termination, using the graceful termination mechanism provided by the LLP.

ULP负责执行ULP正常终止。在ULP优雅终止之后,流的任何一侧都可以使用LLP提供的优雅终止机制启动LLP优雅终止。

RDMAP Abortive Termination allows the RDMAP to issue a Terminate Message describing the reason the RDMAP Stream was terminated. The next section (6.2.1, "RDMAP Abortive Termination") describes the RDMAP Abortive Termination in detail.

RDMAP中止终止允许RDMAP发出终止消息,描述RDMAP流终止的原因。下一节(6.2.1,“RDMAP中止终止”)详细描述了RDMAP中止终止。

LLP Abortive Termination results due to an LLP error and causes the RDMAP Stream to be torn down midstream, without an RDMAP Terminate Message. While this last method is highly undesirable, it is possible, and the ULP should take this into consideration.

由于LLP错误导致LLP中止终止,并导致RDMAP流在中途中断,而没有RDMAP Terminate消息。虽然最后一种方法非常不受欢迎,但这是可能的,ULP应该考虑到这一点。

6.2.1. RDMAP Abortive Termination
6.2.1. RDMAP中止终止

RDMAP defines a Terminate operation that SHOULD be invoked when either an RDMAP error is encountered or an LLP error is surfaced to the RDMAP layer by the LLP.

RDMAP定义了一个终止操作,当遇到RDMAP错误或LLP向RDMAP层显示LLP错误时,应调用该操作。

It is not always possible to send the Terminate Message. For example, certain LLP errors may occur that cause the LLP Stream to be torn down a) before RDMAP is aware of the error, b) before RDMAP is able to send the Terminate Message, or c) after RDMAP has posted the Terminate Message to the LLP, but it has not yet been transmitted by the LLP.

并非总是可以发送终止消息。例如,某些LLP错误可能会发生,导致LLP流被中断a)在RDMAP意识到错误之前,b)在RDMAP能够发送终止消息之前,或c)在RDMAP将终止消息发布到LLP之后,但LLP尚未发送该消息。

Note that an RDMAP Abortive Termination may entail loss of data. In general, when a Terminate Message is received, it is impossible to tell for sure what unacknowledged RDMA Messages were Completed successfully at the Remote Peer. Thus, the state of all outstanding RDMA Messages is indeterminate, and the Messages SHOULD be considered Completed in error.

请注意,RDMAP中止终止可能会导致数据丢失。通常,当接收到终止消息时,无法确定哪些未确认的RDMA消息在远程对等端成功完成。因此,所有未完成的RDMA消息的状态都是不确定的,应该认为这些消息是错误完成的。

When a peer sends or receives a Terminate Message, it MAY immediately tear down the LLP Stream. The peer SHOULD perform a graceful LLP teardown to ensure the Terminate Message is successfully Delivered.

当对等方发送或接收终止消息时,它可以立即中断LLP流。对等方应执行优雅的LLP拆卸,以确保Terminate消息成功传递。

See Section 4.8, "Terminate Header", for a description of the Terminate Message and its contents. See Section 5.4, "Terminate Message", for a description of the Terminate Message semantics.

有关终止消息及其内容的说明,请参见第4.8节“终止标头”。有关终止消息语义的描述,请参见第5.4节“终止消息”。

7. RDMAP Error Management
7. RDMAP错误管理

The RDMAP protocol does not have RDMAP- or DDP-layer error recovery operations built in. If everything is working, the LLP guarantees will ensure that the Messages are arriving at the destination.

RDMAP协议没有内置RDMAP或DDP层错误恢复操作。如果一切正常,LLP保证将确保消息到达目的地。

If errors are detected at the RDMAP or DDP layer, then the RDMAP, DDP, and LLP Streams are Abortively Terminated (see Section 4.8, "Terminate Header").

如果在RDMAP或DDP层检测到错误,则RDMAP、DDP和LLP流将终止(请参阅第4.8节“终止标头”)。

In general, poor implementations or improper ULP programming cause the errors detected at the RDMAP and DDP layers. In these cases, returning a diagnostic termination error Message and closing the RDMAP Stream is far simpler than attempting to maintain the RDMAP Stream, particularly when the cause of the error is not known.

通常,糟糕的实现或不正确的ULP编程会导致在RDMAP和DDP层检测到错误。在这些情况下,返回诊断终止错误消息并关闭RDMAP流比尝试维护RDMAP流要简单得多,特别是在错误原因未知的情况下。

If an LLP does not support teardown of a Stream independent of other Streams, and an RDMAP error results in the Termination of a specific Stream, then the LLP MUST label the Stream as an erroneous Stream and MUST NOT allow any further data transfer on that Stream after RDMAP requests the Stream to be torn down.

如果LLP不支持独立于其他流的流的拆卸,并且RDMAP错误导致特定流的终止,则LLP必须将该流标记为错误流,并且在RDMAP请求拆卸该流后,不得允许在该流上进行任何进一步的数据传输。

For a specific LLP connection, when all Streams are either gracefully torn down or are labeled as erroneous Streams, the LLP connection MUST be torn down.

对于特定的LLP连接,当所有流都被正常地删除或标记为错误流时,必须删除LLP连接。

Since errors are detected at the Remote Peer (possibly long) after RDMA Messages are passed to the DDP and the LLP at the Local Peer and after the RDMA Operations conveyed by the Messages are Completed, the sender cannot easily determine which of its Messages have been received. (RDMA Reads are an exception to this rule.)

由于在RDMA消息被传递到本地对等端的DDP和LLP之后,以及在消息传递的RDMA操作完成之后,远程对等端(可能很长时间)检测到错误,因此发送方无法轻松确定接收到了哪些消息。(RDMA读取是此规则的一个例外。)

For a list of errors returned to the Remote Peer as a result of an Abortive Termination, see Section 4.8, "Terminate Header".

有关由于中止终止而返回远程对等方的错误列表,请参阅第4.8节“终止标头”。

7.1. RDMAP Error Surfacing
7.1. RDMAP错误表面处理

If an error occurs at the Local Peer, the RDMAP layer MUST attempt to inform the local ULP that the error has occurred.

如果本地对等端发生错误,RDMAP层必须尝试通知本地ULP发生了错误。

The Local Peer MUST send a Terminate Message for each of the following cases:

对于以下情况,本地对等方必须发送终止消息:

1. For errors detected while creating RDMA Write, Send, Send with Invalidate, Send with Solicited Event, Send with Solicited Event and Invalidate, or RDMA Read Requests, or other reasons not directly associated with an incoming Message, the Terminate Message and Error code are sent instead of the request. In this case, the Error Type and Error Code fields are included in the Terminate Message, but the Terminated DDP Header and Terminated RDMA Header fields are set to zero.

1. 对于在创建RDMA写入、发送、无效发送、请求事件发送、请求事件发送和无效发送或RDMA读取请求时检测到的错误,或与传入消息不直接相关的其他原因,将发送终止消息和错误代码,而不是请求。在这种情况下,终止消息中包括错误类型和错误代码字段,但终止的DDP头和终止的RDMA头字段设置为零。

2. For errors detected on an incoming RDMA Write, Send, Send with Invalidate, Send with Solicited Event, Send with Solicited Event and Invalidate, or Read Response Message (after the Message has been Delivered by DDP), the Terminate Message is sent at the earliest possible opportunity, preferably in the next outgoing RDMA Message. In this case, the Error Type, Error Code, ULP PDU

2. 对于在传入RDMA写入、发送、无效发送、请求事件发送、请求事件发送和无效发送或读取响应消息(在DDP传递消息之后)上检测到的错误,终止消息在尽可能早的时间发送,最好在下一个传出RDMA消息中发送。在本例中,错误类型、错误代码、ULP PDU

Length, and Terminated DDP Header fields are included in the Terminate Message, but the Terminated RDMA Header field is set to zero.

终止消息中包括长度和终止的DDP头字段,但终止的RDMA头字段设置为零。

3. For errors detected on an incoming RDMA Read Request Message (after the Message has been Delivered by DDP), the Terminate Message is sent at the earliest possible opportunity, preferably in the next outgoing RDMA Message. In this case, the Error Type, Error Code, ULP PDU Length, Terminated DDP Header, and Terminated RDMA Header fields are included in the Terminate Message.

3. 对于在传入RDMA读取请求消息上检测到的错误(在DDP交付消息之后),终止消息在尽可能早的时间发送,最好在下一个传出RDMA消息中发送。在这种情况下,终止消息中包括错误类型、错误代码、ULP PDU长度、终止DDP头和终止RDMA头字段。

4. If more than one error is detected on incoming RDMA Messages, before the Terminate Message can be sent, then the first RDMA Message (and its associated DDP Segment) that experienced an error MUST be captured by the Terminate Message, in accordance with rules 2 and 3 above.

4. 如果在传入RDMA消息上检测到多个错误,则在发送终止消息之前,必须根据上述规则2和3,由终止消息捕获发生错误的第一条RDMA消息(及其相关DDP段)。

7.2. Errors Detected at the Remote Peer on Incoming RDMA Messages
7.2. 在传入RDMA消息的远程对等端检测到错误

On incoming RDMA Writes, RDMA Read Response, Sends, Send with Invalidate, Send with Solicited Event, Send with Solicited Event and Invalidate, and Terminate Messages, the following must be validated:

在传入RDMA写入、RDMA读取响应、发送、发送时失效、发送时请求事件、发送时请求事件、发送时请求事件和失效以及终止消息时,必须验证以下内容:

1. The DDP layer MUST validate all DDP Segment fields.

1. DDP层必须验证所有DDP段字段。

2. The RDMA OpCode MUST be valid.

2. RDMA操作码必须有效。

3. The RDMA Version MUST be valid.

3. RDMA版本必须有效。

Additionally, on incoming Send with Invalidate and Send with Solicited Event and Invalidate Messages, the following must also be validated:

此外,在使用Invalidate进行传入发送以及使用请求事件和Invalidate消息进行发送时,还必须验证以下内容:

4. The Invalidate STag MUST be valid.

4. 无效STag必须是有效的。

5. The STag MUST be associated to this RDMAP Stream.

5. STag必须与此RDMAP流相关联。

On incoming RDMA Request Messages, the following must be validated:

对于传入的RDMA请求消息,必须验证以下内容:

1. The DDP layer MUST validate all Untagged DDP Segment fields.

1. DDP层必须验证所有未标记的DDP段字段。

2. The RDMA OpCode MUST be valid.

2. RDMA操作码必须有效。

3. The RDMA Version MUST be valid.

3. RDMA版本必须有效。

4. For non-zero length RDMA Read Request Messages:

4. 对于非零长度RDMA读取请求消息:

a. The Data Source STag MUST be valid.

a. 数据源STag必须有效。

b. The Data Source STag MUST be associated to this RDMAP Stream.

b. 数据源STag必须与此RDMAP流相关联。

c. The Data Source Tagged Offset MUST fall in the range of legal offsets associated with the Data Source STag.

c. 标记为偏移量的数据源必须在与数据源STag关联的合法偏移量范围内。

d. The sum of the Data Source Tagged Offset and the RDMA Read Message Size MUST fall in the range of legal offsets associated with the Data Source STag.

d. 数据源标记的偏移量和RDMA读取消息大小之和必须在与数据源STag关联的合法偏移量范围内。

e. The sum of the Data Source Tagged Offset and the RDMA Read Message Size MUST NOT cause the Data Source Tagged Offset to wrap.

e. 数据源标记的偏移量和RDMA读取消息大小之和不得导致数据源标记的偏移量换行。

8. Security Considerations
8. 安全考虑

This section references the resources that discuss protocol- specific security considerations and implications of using RDMAP with existing security services. A detailed analysis of the security issues around implementation and use of the RDMAP can be found in [RDMASEC].

本节参考了讨论特定于协议的安全注意事项以及在现有安全服务中使用RDMAP的含义的资源。有关RDMAP实现和使用的安全问题的详细分析,请参见[RDMASEC]。

[RDMASEC] introduces the RDMA reference model and discusses how the resources of this model are vulnerable to attacks and the types of attack these vulnerabilities are subject to. It also details the levels of Trust available in this peer-to-peer model and how this defines the nature of resource sharing.

[RDMASEC]介绍了RDMA参考模型,并讨论了该模型的资源如何容易受到攻击以及这些漏洞受到的攻击类型。它还详细说明了这种对等模型中可用的信任级别,以及如何定义资源共享的性质。

The IPsec requirements for RDDP are based on the version of IPsec specified in RFC 2401 [RFC2401] and related RFCs, as profiled by RFC 3723 [RFC3723], despite the existence of a newer version of IPsec specified in RFC 4301 [RFC4301] and related RFCs [RFC4303], [RFC4306], [RFC4835]. One of the important early applications of the RDDP protocols is their use with iSCSI [iSER]; RDDP's IPsec requirements follow those of IPsec in order to facilitate that usage by allowing a common profile of IPsec to be used with iSCSI and the RDDP protocols. In the future, RFC 3723 may be updated to the newer version of IPsec, and the IPsec security requirements of any such update should apply uniformly to iSCSI and the RDDP protocols.

RDDP的IPsec要求基于RFC 2401[RFC2401]和相关RFC中规定的IPsec版本,如RFC 3723[RFC3723]所述,尽管存在RFC 4301[RFC4301]和相关RFC[RFC4303]、[RFC4306]、[RFC4835]中规定的较新版本的IPsec。RDDP协议的一个重要早期应用是与iSCSI[iSER]一起使用;RDDP的IPsec要求遵循IPsec的要求,以便通过允许将IPsec的公共配置文件与iSCSI和RDDP协议一起使用来促进这种使用。将来,RFC 3723可能会更新到较新版本的IPsec,任何此类更新的IPsec安全要求都应统一应用于iSCSI和RDDP协议。

8.1. Summary of RDMAP-Specific Security Requirements
8.1. RDMAP特定安全需求摘要

[RDMASEC] defines the security requirements for the implementation of the components of the RDMA reference model, namely the RDMA enabled NIC (RNIC) and the Privileged Resource Manager. An RDMAP implementation conforming to this specification MUST conform to these requirements.

[RDMASEC]定义了实现RDMA参考模型组件的安全要求,即支持RDMA的NIC(RNIC)和特权资源管理器。符合本规范的RDMAP实现必须符合这些要求。

8.1.1. RDMAP (RNIC) Requirements
8.1.1. RDMAP(RNIC)要求

RDMAP provides several countermeasures for all types of attacks as introduced in [RDMASEC]. In the following, this specification lists all security requirements that MUST be implemented by the RNIC. A more detailed discussion of RNIC security requirements can be found in Section 5 of [RDMASEC].

RDMAP为[RDMASEC]中介绍的所有类型的攻击提供了几种对策。在下文中,本规范列出了RNIC必须实施的所有安全要求。有关RNIC安全要求的更详细讨论,请参见[RDMASEC]第5节。

1. An RNIC MUST ensure that a specific Stream in a specific Protection Domain cannot access an STag in a different Protection Domain.

1. RNIC必须确保特定保护域中的特定流不能访问不同保护域中的STag。

2. An RNIC MUST ensure that if an STag is limited in scope to a single Stream, no other Stream can use the STag.

2. RNIC必须确保,如果STag的作用域限于单个流,则其他流不能使用该STag。

3. An RNIC MUST ensure that a Remote Peer is not able to access memory outside of the buffer specified when the STag was enabled for remote access.

3. RNIC必须确保远程对等方无法访问STag启用远程访问时指定的缓冲区之外的内存。

4. An RNIC MUST provide a mechanism for the ULP to establish and revoke the association of a ULP Buffer to an STag and TO range.

4. RNIC必须为ULP提供一种机制,以建立和撤销ULP缓冲区与STag和范围的关联。

5. An RNIC MUST provide a mechanism for the ULP to establish and revoke read, write, or read and write access to the ULP Buffer referenced by an STag.

5. RNIC必须为ULP提供一种机制,以建立和撤销对STag引用的ULP缓冲区的读、写或读写访问。

6. An RNIC MUST ensure that the network interface can no longer modify an Advertised Buffer after the ULP revokes remote access rights for an STag.

6. RNIC必须确保ULP撤销STag的远程访问权限后,网络接口不能再修改播发缓冲区。

7. An RNIC MUST ensure that a Remote Peer is not able to invalidate an STag enabled for remote access, if the STag is shared on multiple streams.

7. 如果STag在多个流上共享,RNIC必须确保远程对等方不能使为远程访问启用的STag无效。

8. An RNIC MUST choose the value of STags in a way difficult to predict. It is RECOMMENDED to sparsely populate them over the full available range.

8. RNIC必须以难以预测的方式选择STAG的价值。建议在整个可用范围内稀疏填充它们。

9. An RNIC MUST NOT enable sharing a Completion Queue (CQ) across ULPs that do not share partial mutual trust.

9. RNIC不得允许在不共享部分互信的ULP之间共享完成队列(CQ)。

10. An RNIC MUST ensure that if a CQ overflows, any Streams that do not use the CQ MUST remain unaffected.

10. RNIC必须确保如果CQ溢出,任何不使用CQ的流必须保持不受影响。

11. An RNIC implementation SHOULD provide a mechanism to cap the number of outstanding RDMA Read Requests.

11. RNIC实现应该提供一种机制来限制未完成的RDMA读取请求的数量。

12. An RNIC MUST NOT enable firmware to be loaded on the RNIC directly from an untrusted Local Peer or Remote Peer, unless the Peer is properly authenticated*, and the update is done via a secure protocol, such as IPsec.

12. RNIC不得允许固件直接从不受信任的本地对等方或远程对等方加载到RNIC上,除非对等方经过正确的身份验证*,并且更新是通过安全协议(如IPsec)完成的。

* by a mechanism outside the scope of this specification. The mechanism presumably entails authenticating that the remote ULP has the right to perform the update.

* 通过本规范范围之外的机制。该机制可能需要验证远程ULP是否有权执行更新。

8.1.2. Privileged Resource Manager Requirements
8.1.2. 特权资源管理器要求

With RDMAP, all reservations of local resources are initiated from local ULPs. To protect from local attacks including unfair resource distribution and gaining unauthorized access to RNIC resources, a Privileged Resource Manager (PRM) must be implemented, which manages all local resource allocation. Note that the PRM must not be provided as an independent component, and its functionality can also be implemented as part of the privileged ULP or as part of the RNIC itself.

使用RDMAP,本地资源的所有保留都从本地ULP启动。为了防止本地攻击,包括不公平的资源分配和未经授权访问RNIC资源,必须实施特权资源管理器(PRM),该管理器管理所有本地资源分配。请注意,PRM不得作为独立组件提供,其功能也可作为特权ULP的一部分或RNIC本身的一部分实现。

A PRM implementation must meet the following security requirements (a more detailed discussion of PRM security requirements can be found in Section 5 of [RDMASEC]):

PRM实施必须满足以下安全要求(关于PRM安全要求的更详细讨论见[RDMASEC]第5节):

1. All Non-Privileged ULP interactions with the RNIC Engine that could affect other ULPs MUST be done using the Resource Manager as a proxy.

1. 所有可能影响其他ULP的与RNIC引擎的非特权ULP交互必须使用资源管理器作为代理来完成。

2. All ULP resource allocation requests for scarce resources MUST also be done using a Privileged Resource Manager.

2. 稀缺资源的所有ULP资源分配请求也必须使用特权资源管理器完成。

3. The Privileged Resource Manager MUST NOT assume that different ULPs share Partial Mutual Trust unless there is a mechanism to ensure that the ULPs do indeed share partial mutual trust.

3. 特权资源管理器不得假设不同的ULP共享部分互信,除非存在确保ULP确实共享部分互信的机制。

4. If Non-Privileged ULPs are supported, the Privileged Resource Manager MUST verify that the Non-Privileged ULP has the right to access a specific Data Buffer before allowing an STag for which the ULP has access rights to be associated with a specific Data Buffer.

4. 如果支持非特权ULP,特权资源管理器必须验证非特权ULP有权访问特定数据缓冲区,然后才能允许ULP有权访问的STag与特定数据缓冲区关联。

5. The Privileged Resource Manager MUST control the allocation of CQ entries.

5. 特权资源管理器必须控制CQ条目的分配。

6. The Privileged Resource Manager SHOULD prevent a Local Peer from allocating more than its fair share of resources.

6. 特权资源管理器应防止本地对等方分配超过其公平份额的资源。

7. RDMA Read Request Queue resource consumption MUST be controlled by the Privileged Resource Manager such that RDMAP/DDP Streams that do not share Partial Mutual Trust do not share RDMA Read Request Queue resources.

7. RDMA读取请求队列资源消耗必须由特权资源管理器控制,以便不共享部分互信的RDMAP/DDP流不共享RDMA读取请求队列资源。

8. If an RNIC provides the ability to share receive buffers across multiple Streams, the combination of the RNIC and the Privileged Resource Manager MUST be able to detect if the Remote Peer is attempting to consume more than its fair share of resources so that the Local Peer can apply countermeasures to detect and prevent the attack.

8. 如果RNIC提供跨多个流共享接收缓冲区的能力,RNIC和特权资源管理器的组合必须能够检测远程对等方是否试图消耗超过其公平共享的资源,以便本地对等方可以应用对策来检测和防止攻击。

8.2. Security Services for RDMAP
8.2. RDMAP的安全服务

RDMAP is using IP-based network services to control, read, and write data buffers over the network. Therefore, all exchanged control and data packets are vulnerable to spoofing, tampering, and information disclosure attacks.

RDMAP使用基于IP的网络服务来控制、读取和写入网络上的数据缓冲区。因此,所有交换的控制和数据包都容易受到欺骗、篡改和信息泄露攻击。

RDMAP Streams that are subject to impersonation attacks or Stream hijacking attacks can be authenticated, have their integrity protected, and be protected from replay attacks. Furthermore, confidentiality protection can be used to protect from eavesdropping.

可以对受到模拟攻击或流劫持攻击的RDMAP流进行身份验证,保护其完整性,并防止重播攻击。此外,保密保护可用于防止窃听。

8.2.1. Available Security Services
8.2.1. 可用的安全服务

The IPsec protocol suite [RFC2401] defines strong countermeasures to protect an IP stream from those attacks. Several levels of protection can guarantee session confidentiality, per-packet source authentication, per-packet integrity, and correct packet sequencing.

IPsec协议套件[RFC2401]定义了强大的对策,以保护IP流免受这些攻击。多个级别的保护可以保证会话机密性、每个数据包源的身份验证、每个数据包的完整性和正确的数据包顺序。

RDMAP security may also profit from SSL or TLS security services provided for TCP-based ULPs [RFC4346]. Used underneath RDMAP, these security services also provide for stream authentication, data integrity, and confidentiality. As discussed in [RDMASEC], limitations on the maximum packet length to be carried over the network and potentially inefficient out-of-order packet processing at the data sink make SSL and TLS less appropriate for RDMAP than IPsec.

RDMAP安全还可以从为基于TCP的ULP提供的SSL或TLS安全服务中获益[RFC4346]。在RDMAP下使用,这些安全服务还提供流身份验证、数据完整性和机密性。正如[RDMASEC]中所讨论的,对网络上要承载的最大数据包长度的限制以及数据接收器处可能效率低下的无序数据包处理,使得SSL和TLS比IPsec更不适合RDMAP。

If SSL is layered on top of RDMAP, SSL does not protect the RDMAP headers. Thus, a man-in-the-middle attack can still occur by modifying the RDMAP header to incorrectly place the data into the wrong buffer, thus effectively corrupting the data stream.

如果SSL分层在RDMAP之上,则SSL不会保护RDMAP头。因此,中间人攻击仍然可以通过修改RDMAP头将数据错误地放入错误的缓冲区而发生,从而有效地破坏数据流。

By remaining independent of ULP and LLP security protocols, RDMAP will benefit from continuing improvements at those layers. Users are provided flexibility to adapt to their specific security requirements and the ability to adapt to future security challenges. Given this,

通过保持独立于ULP和LLP安全协议,RDMAP将从这些层的持续改进中受益。为用户提供了适应其特定安全需求的灵活性,以及适应未来安全挑战的能力。有鉴于此,,

the vulnerabilities of RDMAP to active third-party interference are no greater than any other protocol running over an LLP such as TCP or SCTP.

RDMAP对主动第三方干扰的脆弱性不超过在LLP上运行的任何其他协议,如TCP或SCTP。

8.2.2. Requirements for IPsec Services for RDMAP
8.2.2. RDMAP的IPsec服务要求

Because IPsec is designed to secure arbitrary IP packet streams, including streams where packets are lost, RDMAP can run on top of IPsec without any change. IPsec packets are processed (e.g., integrity checked and possibly decrypted) in the order they are received, and an RDMAP Data Sink will process the decrypted RDMA Messages contained in these packets in the same manner as RDMA Messages contained in unsecured IP packets.

由于IPsec设计用于保护任意IP数据包流,包括数据包丢失的数据流,因此RDMAP可以在IPsec上运行而无需任何更改。IPsec数据包按照接收顺序进行处理(例如,完整性检查并可能解密),RDMAP数据接收器将以与不安全IP数据包中包含的RDMA消息相同的方式处理这些数据包中包含的解密RDMA消息。

The IP Storage working group has defined the normative IPsec requirements for IP Storage [RFC3723]. Portions of this specification are applicable to the RDMAP. In particular, a compliant implementation of IPsec services for RDMAP MUST meet the requirements as outlined in Section 2.3 of [RFC3723]. Without replicating the detailed discussion in [RFC3723], this includes the following requirements:

IP存储工作组定义了IP存储的标准IPsec要求[RFC3723]。本规范的部分内容适用于RDMAP。特别是,RDMAP的IPsec服务的兼容实现必须满足[RFC3723]第2.3节中概述的要求。在不复制[RFC3723]中的详细讨论的情况下,这包括以下要求:

1. The implementation MUST support IPsec ESP [RFC2406], as well as the replay protection mechanisms of IPsec. When ESP is utilized, per-packet data origin authentication, integrity, and replay protection MUST be used.

1. 实现必须支持IPsec ESP[RFC2406],以及IPsec的重播保护机制。使用ESP时,必须使用每包数据源身份验证、完整性和重播保护。

2. It MUST support ESP in tunnel mode and MAY implement ESP in transport mode.

2. 它必须在隧道模式下支持ESP,并且可以在传输模式下实施ESP。

3. It MUST support IKE [RFC2409] for peer authentication, negotiation of security associations, and key management, using the IPsec DOI [RFC2407].

3. 它必须支持IKE[RFC2409],以便使用IPsec DOI[RFC2407]进行对等身份验证、安全关联协商和密钥管理。

4. It MUST NOT interpret the receipt of a IKE Phase 2 delete message as a reason for tearing down the RDMAP stream. Since IPsec acceleration hardware may only be able to handle a limited number of active IKE Phase 2 SAs, idle SAs may be dynamically brought down, and a new SA be brought up again, if activity resumes.

4. 它不能将收到IKE第2阶段删除消息解释为中断RDMAP流的原因。由于IPsec加速硬件可能只能处理有限数量的活动IKE阶段2 SA,因此如果活动恢复,空闲SA可能会动态关闭,并再次启动新SA。

5. It MUST support peer authentication using a pre-shared key, and MAY support certificate-based peer authentication using digital signatures. Peer authentication using the public key encryption methods [RFC2409] SHOULD NOT be used.

5. 它必须支持使用预共享密钥的对等身份验证,并且可能支持使用数字签名的基于证书的对等身份验证。不应使用使用公钥加密方法[RFC2409]的对等身份验证。

6. It MUST support IKE Main Mode and SHOULD support Aggressive Mode. IKE Main Mode with pre-shared key authentication SHOULD NOT be used when either of the peers uses a dynamically assigned IP address.

6. 它必须支持IKE主模式,并且应该支持攻击模式。当任一对等方使用动态分配的IP地址时,不应使用带有预共享密钥身份验证的IKE主模式。

7. When digital signatures are used to achieve authentication, either IKE Main Mode or IKE Aggressive Mode MAY be used. In these cases, an IKE negotiator SHOULD use IKE Certificate Request Payload(s) to specify the certificate authority (or authorities) that are trusted in accordance with its local policy. IKE negotiators SHOULD check the pertinent Certificate Revocation List (CRL) before accepting a PKI certificate for use in IKE's authentication procedures.

7. 当使用数字签名实现身份验证时,可以使用IKE主模式或IKE攻击模式。在这些情况下,IKE谈判者应该使用IKE证书请求有效负载来指定根据其本地策略受信任的证书颁发机构。IKE谈判者在接受用于IKE认证过程的PKI证书之前,应检查相关的证书撤销列表(CRL)。

8. Access to locally stored secret information (pre-shared or private key for digital signing) must be suitably restricted, since compromise of the secret information nullifies the security properties of the IKE/IPsec protocols.

8. 必须适当限制对本地存储的秘密信息(用于数字签名的预共享或私钥)的访问,因为泄露秘密信息会使IKE/IPsec协议的安全属性无效。

9. It MUST follow the guidelines of Section 2.3.4 of [RFC3723] on the setting of IKE parameters to achieve a high level of interoperability without requiring extensive configuration.

9. 它必须遵循[RFC3723]第2.3.4节关于IKE参数设置的指南,以实现高水平的互操作性,而无需大量配置。

Furthermore, implementation and deployment of the IPsec services for RDDP should follow the Security Considerations outlined in Section 5 of [RFC3723].

此外,RDDP的IPsec服务的实现和部署应遵循[RFC3723]第5节中概述的安全注意事项。

9. IANA Considerations
9. IANA考虑

This document requests no direct action from IANA. The following consideration is listed here as commentary.

本文件不要求IANA采取直接行动。以下考虑事项列为评注。

If RDMAP was enabled a priori for a ULP by connecting to a well-known port, this well-known port would be registered for the RDMAP with IANA. The registration of the well-known port will be the responsibility of the ULP specification.

如果通过连接到一个已知端口为ULP预先启用了RDMAP,那么该已知端口将通过IANA为RDMAP注册。著名端口的注册将由ULP规范负责。

10. References
10. 工具书类
10.1. Normative References
10.1. 规范性引用文件

[DDP] Shah, H., Pinkerton, J., Recio, R., and P. Culley, "Direct Data Placement over Reliable Transports", RFC 5041, October 2007.

[DDP]Shah,H.,Pinkerton,J.,Recio,R.,和P.Culley,“可靠传输上的直接数据放置”,RFC 50412007年10月。

[iSER] Ko, M., Chadalapaka, M., Hufferd, J., Elzur, U., Shah, H., and P. Thaler, "Internet Small Computer System Interface (iSCSI) Extensions for Remote Direct Memory Access (RDMA)" RFC 5046, October 2007.

[iSER]Ko,M.,Chadalapaka,M.,Hufferd,J.,Elzur,U.,Shah,H.,和P.Thaler,“用于远程直接内存访问(RDMA)的互联网小型计算机系统接口(iSCSI)扩展”,RFC 5046,2007年10月。

[MPA] Culley, P., Elzur, U., Recio, R., Bailey, S., and J. Carrier, "Marker PDU Aligned Framing for TCP Specification", RFC 5044, October 2007.

[MPA]Culley,P.,Elzur,U.,Recio,R.,Bailey,S.,和J.Carrier,“TCP规范的标记PDU对齐框架”,RFC 5044,2007年10月。

[RDMASEC] Pinkerton, J. and E. Deleganes, "Direct Data Placement Protocol (DDP) / Remote Direct Memory Access Protocol (RDMAP) Security", RFC 5042, October 2007.

[RDMASEC]Pinkerton,J.和E.Deleganes,“直接数据放置协议(DDP)/远程直接内存访问协议(RDMAP)安全”,RFC 50422007年10月。

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.

[RFC2119]Bradner,S.,“RFC中用于表示需求水平的关键词”,BCP 14,RFC 2119,1997年3月。

[RFC2406] Kent, S. and R. Atkinson, "IP Encapsulating Security Payload (ESP)", RFC 2406, November 1998.

[RFC2406]Kent,S.和R.Atkinson,“IP封装安全有效载荷(ESP)”,RFC 2406,1998年11月。

[RFC2407] Piper, D., "The Internet IP Security Domain of Interpretation of ISAKMP", RFC 2407, November 1998.

[RFC2407]Piper,D.,“ISAKMP解释的互联网IP安全域”,RFC 2407,1998年11月。

[RFC2409] Harkins, D. and D. Carrel, "The Internet Key Exchange (IKE)", RFC 2409, November 1998.

[RFC2409]Harkins,D.和D.Carrel,“互联网密钥交换(IKE)”,RFC 2409,1998年11月。

[RFC3723] Aboba, B., Tseng, J., Walker, J., Rangan, V., and F. Travostino, "Securing Block Storage Protocols over IP", RFC 3723, April 2004.

[RFC3723]Aboba,B.,Tseng,J.,Walker,J.,Rangan,V.,和F.Travostino,“通过IP保护块存储协议”,RFC 37232004年4月。

[RFC2401] Kent, S. and R. Atkinson, "Security Architecture for the Internet Protocol", RFC 2401, November 1998.

[RFC2401]Kent,S.和R.Atkinson,“互联网协议的安全架构”,RFC 2401,1998年11月。

[SCTP] Stewart, R., Ed., "Stream Control Transmission Protocol", RFC 4960, September 2007.

[SCTP]Stewart,R.,Ed.“流控制传输协议”,RFC 49602007年9月。

[TCP] Postel, J., "Transmission Control Protocol", STD 7, RFC 793, September 1981.

[TCP]Postel,J.,“传输控制协议”,STD 7,RFC 793,1981年9月。

10.2. Informative References
10.2. 资料性引用

[RFC4301] Kent, S. and K. Seo, "Security Architecture for the Internet Protocol", RFC 4301, December 2005.

[RFC4301]Kent,S.和K.Seo,“互联网协议的安全架构”,RFC 43012005年12月。

[RFC4303] Kent, S., "IP Encapsulating Security Payload (ESP)", RFC 4303, December 2005.

[RFC4303]Kent,S.,“IP封装安全有效载荷(ESP)”,RFC 4303,2005年12月。

[RFC4306] Kaufman, C., "Internet Key Exchange (IKEv2) Protocol", RFC 4306, December 2005.

[RFC4306]Kaufman,C.,“互联网密钥交换(IKEv2)协议”,RFC43062005年12月。

[RFC4346] Dierks, T. and E. Rescorla, "The TLS Protocol Version 1.1", RFC 4346, April 2006.

[RFC4346]Dierks,T.和E.Rescorla,“TLS协议版本1.1”,RFC 4346,2006年4月。

[RFC4835] Manral, V., "Cryptographic Algorithm Implementation Requirements for Encapsulating Security Payload (ESP) and Authentication Header (AH)", RFC 4835, April 2007.

[RFC4835]Manral,V.“封装安全有效载荷(ESP)和身份验证头(AH)的密码算法实现要求”,RFC 4835,2007年4月。

Appendix A. DDP Segment Formats for RDMA Messages
附录A.RDMA报文的DDP段格式

This appendix is for information only and is NOT part of the standard. It simply depicts the DDP Segment format for the various RDMA Messages.

本附录仅供参考,不属于本标准的一部分。它简单地描述了各种RDMA消息的DDP段格式。

A.1. DDP Segment for RDMA Write
A.1. 用于RDMA写入的DDP段

The following figure depicts an RDMA Write, DDP Segment:

下图描述了RDMA写入DDP段:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
                                   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                                   |   DDP Control | RDMA Control  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       Data Sink STag                          |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                   Data Sink Tagged Offset                     |
   +                                                               +
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                   RDMA Write ULP Payload                      |
   //                                                             //
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        
    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
                                   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                                   |   DDP Control | RDMA Control  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       Data Sink STag                          |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                   Data Sink Tagged Offset                     |
   +                                                               +
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                   RDMA Write ULP Payload                      |
   //                                                             //
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        

Figure 11: RDMA Write, DDP Segment Format

图11:RDMA写入,DDP段格式

A.2. DDP Segment for RDMA Read Request
A.2. RDMA读取请求的DDP段

The following figure depicts an RDMA Read Request, DDP Segment:

下图描述了RDMA读取请求,DDP段:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
                                   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                                   |  DDP Control  | RDMA Control  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      Reserved (Not Used)                      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |              DDP (RDMA Read Request) Queue Number             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        DDP (RDMA Read Request) Message Sequence Number        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |             DDP (RDMA Read Request) Message Offset            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                     Data Sink STag (SinkSTag)                 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   +                  Data Sink Tagged Offset (SinkTO)             +
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                  RDMA Read Message Size (RDMARDSZ)            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                     Data Source STag (SrcSTag)                |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   +                 Data Source Tagged Offset (SrcTO)             +
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        
    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
                                   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                                   |  DDP Control  | RDMA Control  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      Reserved (Not Used)                      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |              DDP (RDMA Read Request) Queue Number             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        DDP (RDMA Read Request) Message Sequence Number        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |             DDP (RDMA Read Request) Message Offset            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                     Data Sink STag (SinkSTag)                 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   +                  Data Sink Tagged Offset (SinkTO)             +
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                  RDMA Read Message Size (RDMARDSZ)            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                     Data Source STag (SrcSTag)                |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   +                 Data Source Tagged Offset (SrcTO)             +
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        

Figure 12: RDMA Read Request, DDP Segment format

图12:RDMA读取请求,DDP段格式

A.3. DDP Segment for RDMA Read Response
A.3. RDMA读取响应的DDP段

The following figure depicts an RDMA Read Response, DDP Segment:

下图描述了RDMA读取响应DDP段:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
                                   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                                   |  DDP Control  | RDMA Control  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       Data Sink STag                          |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                   Data Sink Tagged Offset                     |
   +                                                               +
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                RDMA Read Response ULP Payload                 |
   //                                                             //
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        
    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
                                   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                                   |  DDP Control  | RDMA Control  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       Data Sink STag                          |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                   Data Sink Tagged Offset                     |
   +                                                               +
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                RDMA Read Response ULP Payload                 |
   //                                                             //
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        

Figure 13: RDMA Read Response, DDP Segment Format

图13:RDMA读取响应,DDP段格式

A.4. DDP Segment for Send and Send with Solicited Event
A.4. 发送和发送请求事件的DDP段

The following figure depicts a Send and Send with Solicited Request, DDP Segment:

下图描述了发送和请求请求发送,DDP段:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
                                   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                                   |  DDP Control  | RDMA Control  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      Reserved (Not Used)                      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       (Send) Queue Number                     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                 (Send) Message Sequence Number                |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      (Send) Message Offset                    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       Send ULP Payload                        |
   //                                                             //
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        
    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
                                   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                                   |  DDP Control  | RDMA Control  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      Reserved (Not Used)                      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       (Send) Queue Number                     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                 (Send) Message Sequence Number                |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      (Send) Message Offset                    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       Send ULP Payload                        |
   //                                                             //
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        

Figure 14: Send and Send with Solicited Event, DDP Segment Format

图14:发送和发送请求事件,DDP段格式

A.5. DDP Segment for Send with Invalidate and Send with SE and Invalidate

A.5. DDP段,用于发送带无效,发送带SE带无效

The following figure depicts a Send with Invalidate and Send with Solicited and Invalidate Request, DDP Segment:

下图描述了DDP段中的发送和无效以及发送请求和无效请求:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
                                   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                                   |   DDP Control | RDMA Control  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                         Invalidate STag                       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       (Send) Queue Number                     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                 (Send) Message Sequence Number                |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      (Send) Message Offset                    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       Send ULP Payload                        |
   //                                                             //
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        
    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
                                   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                                   |   DDP Control | RDMA Control  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                         Invalidate STag                       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       (Send) Queue Number                     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                 (Send) Message Sequence Number                |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      (Send) Message Offset                    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       Send ULP Payload                        |
   //                                                             //
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        

Figure 15: Send with Invalidate and Send with SE and Invalidate, DDP Segment Format

图15:使用Invalidate发送,使用SE和Invalidate发送,DDP段格式

A.6. DDP Segment for Terminate
A.6. 用于终止的DDP段

The following figure depicts a Terminate, DDP Segment:

下图描述了终止DDP段:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
                                   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                                   |   DDP Control | RDMA Control  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      Reserved (Not Used)                      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                   DDP (Terminate) Queue Number                |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |             DDP (Terminate) Message Sequence Number           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                  DDP (Terminate) Message Offset               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Terminate Control             |      Reserved           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  DDP Segment Length (if any)  |                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
   |                                                               |
   +                                                               +
   |                 Terminated DDP Header (if any)                |
   +                                                               +
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   //                                                             //
   |                 Terminated RDMA Header (if any)               |
   +                                                               +
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        
    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
                                   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                                   |   DDP Control | RDMA Control  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      Reserved (Not Used)                      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                   DDP (Terminate) Queue Number                |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |             DDP (Terminate) Message Sequence Number           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                  DDP (Terminate) Message Offset               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Terminate Control             |      Reserved           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  DDP Segment Length (if any)  |                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
   |                                                               |
   +                                                               +
   |                 Terminated DDP Header (if any)                |
   +                                                               +
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   //                                                             //
   |                 Terminated RDMA Header (if any)               |
   +                                                               +
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
        

Figure 16: Terminate, DDP Segment Format

图16:终止,DDP段格式

Appendix B. Ordering and Completion Table
附录B.订购和完成表

The following table summarizes the ordering relationships that are defined in Section 5.5, "Ordering and Completions", from the standpoint of the local peer issuing the two Operations. Note that in the table that follows, Send includes Send, Send with Invalidate, Send with Solicited Event, and Send with Solicited Event and Invalidate.

下表总结了第5.5节“订购和完成”中定义的订购关系,从发布这两个操作的本地对等方的角度出发。请注意,在下表中,Send包括Send、Send with Invalidate、Send with required Event以及Send with required Event and Invalidate。

   ------+-------+----------------+----------------+----------------
   First | Later | Placement      | Placement      | Ordering
    Op   | Op    | guarantee at   | guarantee at   | guarantee at
         |       | Remote Peer    | Local Peer     | Remote Peer
         |       |                |                |
   ------+-------+----------------+----------------+----------------
   Send  | Send  | No placement   | Not applicable | Completed in
         |       | guarantee. If  |                | order.
         |       | guarantee is   |                |
         |       | necessary, see |                |
         |       | footnote 1.    |                |
   ------+-------+----------------+----------------+----------------
   Send  | RDMA  | No placement   | Not applicable | Not applicable
         | Write | guarantee. If  |                |
         |       | guarantee is   |                |
         |       | necessary, see |                |
         |       | footnote 1.    |                |
   ------+-------+----------------+----------------+----------------
   Send  | RDMA  | No placement   | RDMA Read      | RDMA Read
         | Read  | guarantee      | Response       | Response
         |       | between Send   | Payload will   | Message will
         |       | Payload and    | not be placed  | not be
         |       | RDMA Read      | at the local   | generated until
         |       | Request Header | peer until the | Send has been
         |       |                | Send Payload is| Completed
         |       |                | placed at the  |
         |       |                | Remote Peer    |
   ------+-------+----------------+----------------+----------------
   RDMA  | Send  | No placement   | Not applicable | Not applicable
   Write |       | guarantee. If  |                |
         |       | guarantee is   |                |
         |       | necessary, see |                |
         |       | footnote 1.    |                |
        
   ------+-------+----------------+----------------+----------------
   First | Later | Placement      | Placement      | Ordering
    Op   | Op    | guarantee at   | guarantee at   | guarantee at
         |       | Remote Peer    | Local Peer     | Remote Peer
         |       |                |                |
   ------+-------+----------------+----------------+----------------
   Send  | Send  | No placement   | Not applicable | Completed in
         |       | guarantee. If  |                | order.
         |       | guarantee is   |                |
         |       | necessary, see |                |
         |       | footnote 1.    |                |
   ------+-------+----------------+----------------+----------------
   Send  | RDMA  | No placement   | Not applicable | Not applicable
         | Write | guarantee. If  |                |
         |       | guarantee is   |                |
         |       | necessary, see |                |
         |       | footnote 1.    |                |
   ------+-------+----------------+----------------+----------------
   Send  | RDMA  | No placement   | RDMA Read      | RDMA Read
         | Read  | guarantee      | Response       | Response
         |       | between Send   | Payload will   | Message will
         |       | Payload and    | not be placed  | not be
         |       | RDMA Read      | at the local   | generated until
         |       | Request Header | peer until the | Send has been
         |       |                | Send Payload is| Completed
         |       |                | placed at the  |
         |       |                | Remote Peer    |
   ------+-------+----------------+----------------+----------------
   RDMA  | Send  | No placement   | Not applicable | Not applicable
   Write |       | guarantee. If  |                |
         |       | guarantee is   |                |
         |       | necessary, see |                |
         |       | footnote 1.    |                |
        
   ------+-------+----------------+----------------+----------------
   RDMA  | RDMA  | No placement   | Not applicable | Not applicable
   Write | Write | guarantee. If  |                |
         |       | guarantee is   |                |
         |       | necessary, see |                |
         |       | footnote 1.    |                |
   ------+-------+----------------+----------------+----------------
   RDMA  | RDMA  | No placement   | RDMA Read      | Not applicable
   Write | Read  | guarantee      | Response       |
         |       | between RDMA   | Payload will   |
         |       | Write Payload  | not be placed  |
         |       | and RDMA Read  | at the local   |
         |       | Request Header | peer until the |
         |       |                | RDMA Write     |
         |       |                | Payload is     |
         |       |                | placed at the  |
         |       |                | Remote Peer    |
   ------+-------+----------------+----------------+----------------
   RDMA  | Send  | No placement   | Send Payload   | Not applicable
   Read  |       | guarantee      | may be placed  |
         |       | between RDMA   | at the remote  |
         |       | Read Request   | peer before the|
         |       | Header and Send| RDMA Read      |
         |       | payload        | Response is    |
         |       |                | generated.     |
         |       |                | If guarantee is|
         |       |                | necessary, see |
         |       |                | footnote 2.    |
   ------+-------+----------------+----------------+----------------
   RDMA  | RDMA  | No placement   | RDMA Write     | Not applicable
   Read  | Write | guarantee      | Payload may be |
         |       | between RDMA   | placed at the  |
         |       | Read Request   | Remote Peer    |
         |       | Header and RDMA| before the RDMA|
         |       | Write payload  | Read Response  |
         |       |                | is generated.  |
         |       |                | If guarantee is|
         |       |                | necessary, see |
         |       |                | footnote 2.    |
        
   ------+-------+----------------+----------------+----------------
   RDMA  | RDMA  | No placement   | Not applicable | Not applicable
   Write | Write | guarantee. If  |                |
         |       | guarantee is   |                |
         |       | necessary, see |                |
         |       | footnote 1.    |                |
   ------+-------+----------------+----------------+----------------
   RDMA  | RDMA  | No placement   | RDMA Read      | Not applicable
   Write | Read  | guarantee      | Response       |
         |       | between RDMA   | Payload will   |
         |       | Write Payload  | not be placed  |
         |       | and RDMA Read  | at the local   |
         |       | Request Header | peer until the |
         |       |                | RDMA Write     |
         |       |                | Payload is     |
         |       |                | placed at the  |
         |       |                | Remote Peer    |
   ------+-------+----------------+----------------+----------------
   RDMA  | Send  | No placement   | Send Payload   | Not applicable
   Read  |       | guarantee      | may be placed  |
         |       | between RDMA   | at the remote  |
         |       | Read Request   | peer before the|
         |       | Header and Send| RDMA Read      |
         |       | payload        | Response is    |
         |       |                | generated.     |
         |       |                | If guarantee is|
         |       |                | necessary, see |
         |       |                | footnote 2.    |
   ------+-------+----------------+----------------+----------------
   RDMA  | RDMA  | No placement   | RDMA Write     | Not applicable
   Read  | Write | guarantee      | Payload may be |
         |       | between RDMA   | placed at the  |
         |       | Read Request   | Remote Peer    |
         |       | Header and RDMA| before the RDMA|
         |       | Write payload  | Read Response  |
         |       |                | is generated.  |
         |       |                | If guarantee is|
         |       |                | necessary, see |
         |       |                | footnote 2.    |
        
   ------+-------+----------------+----------------+----------------
   RDMA  | RDMA  | No placement   | No placement   | Second RDMA
   Read  | Read  | guarantee of   | guarantee of   | Read Response
         |       | the two RDMA   | the two RDMA   | will not be
         |       | Read Request   | Read Response  | generated until
         |       | Headers        | Payloads.      | first RDMA Read
         |       | Additionally,  |                | Response is
         |       | there is no    |                | generated.
         |       | guarantee that |                |
         |       | the Tagged     |                |
         |       | Buffers        |                |
         |       | referenced in  |                |
         |       | the RDMA Read  |                |
         |       | will be read in|                |
         |       | order          |                |
        
   ------+-------+----------------+----------------+----------------
   RDMA  | RDMA  | No placement   | No placement   | Second RDMA
   Read  | Read  | guarantee of   | guarantee of   | Read Response
         |       | the two RDMA   | the two RDMA   | will not be
         |       | Read Request   | Read Response  | generated until
         |       | Headers        | Payloads.      | first RDMA Read
         |       | Additionally,  |                | Response is
         |       | there is no    |                | generated.
         |       | guarantee that |                |
         |       | the Tagged     |                |
         |       | Buffers        |                |
         |       | referenced in  |                |
         |       | the RDMA Read  |                |
         |       | will be read in|                |
         |       | order          |                |
        

Figure 17: Operation Ordering

图17:操作顺序

Footnote 1: If the guarantee is necessary, a ULP may insert an RDMA Read operation and wait for it to complete to act as a Fence.

脚注1:如果保证是必要的,ULP可以插入RDMA读取操作,并等待其完成以充当围栏。

Footnote 2: If the guarantee is necessary, a ULP may wait for the RDMA Read operation to complete before performing the Send.

脚注2:如果需要保证,ULP可以在执行发送之前等待RDMA读取操作完成。

Appendix C. Contributors
附录C.贡献者

Dwight Barron Hewlett-Packard Company 20555 SH 249 Houston, TX 77070-2698 USA Phone: 281-514-2769 EMail: dwight.barron@hp.com

德怀特·巴伦·惠普公司20555 SH 249德克萨斯州休斯顿77070-2698美国电话:281-514-2769电子邮件:德怀特。barron@hp.com

Caitlin Bestler Broadcom Corporation 16215 Alton Parkway Irvine, CA 92619-7013 USA Phone: 949-926-6383 EMail: caitlinb@broadcom.com

Caitlin Bestler Broadcom Corporation 16215加利福尼亚州欧文市阿尔顿大道92619-7013美国电话:949-926-6383电子邮件:caitlinb@broadcom.com

John Carrier Cray, Inc. 411 First Avenue S, Suite 600 Seattle, WA 98104-2860 USA Phone: 206-701-2090 EMail: carrier@cray.com

John Carrier Cray,Inc.美国华盛顿州西雅图第一大道S 411号600室98104-2860电话:206-701-2090电子邮件:carrier@cray.com

Ted Compton EMC Corporation Research Triangle Park, NC 27709 USA Phone: 919-248-6075 EMail: compton_ted@emc.com

泰德·康普顿美国北卡罗来纳州三角研究园EMC公司电话:919-248-6075电子邮件:康普顿_ted@emc.com

Uri Elzur Broadcom Corporation 16215 Alton Parkway Irvine, California 92619-7013 USA Phone: +1 (949) 585-6432 EMail: Uri@Broadcom.com

Uri Elzur Broadcom Corporation 16215 Alton Parkway Irvine,California 92619-7013美国电话:+1(949)585-6432电子邮件:Uri@Broadcom.com

Hari Ghadia Gen10 Technology, Inc. 1501 W Shady Grove Road Grand Prairie, TX 75050 Phone: (972) 301 3630 EMail: hghadia@gen10technology.com

Hari Ghadia Gen10 Technology,Inc.德克萨斯州大草原Shady Grove路西1501号75050电话:(972)301 3630电子邮件:hghadia@gen10technology.com

Howard C. Herbert Intel Corporation MS CH7-404 5000 West Chandler Blvd. Chandler, Arizona 85226 Phone: 480-554-3116 EMail: howard.c.herbert@intel.com

霍华德C.赫伯特英特尔公司MS CH7-404 5000西钱德勒大道。亚利桑那州钱德勒85226电话:480-554-3116电子邮件:howard.c。herbert@intel.com

Mike Ko IBM 650 Harry Rd. San Jose, CA 95120 Phone: (408) 927-2085 EMail: mako@us.ibm.com

Mike Ko IBM加利福尼亚州圣何塞哈里路650号95120电话:(408)927-2085电子邮件:mako@us.ibm.com

Mike Krause Hewlett-Packard Company 43LN 19410 Homestead Road Cupertino, CA 95014 USA Phone: 408-447-3191 EMail: krause@cup.hp.com

Mike Krause Hewlett-Packard Company 43LN 19410美国加利福尼亚州库珀蒂诺市家园路95014电话:408-447-3191电子邮件:krause@cup.hp.com

Dave Minturn Intel Corporation MS JF1-210 5200 North East Elam Young Parkway Hillsboro, Oregon 97124 Phone: 503-712-4106 EMail: dave.b.minturn@intel.com

Dave Minturn Intel Corporation MS JF1-210 5200东北俄勒冈州埃拉姆杨公园路希尔斯伯勒97124电话:503-712-4106电子邮件:Dave.b。minturn@intel.com

Mike Penna Broadcom Corporation 16215 Alton Parkway Irvine, California 92619-7013 USA Phone: +1 (949) 926-7149 EMail: MPenna@Broadcom.com

Mike Penna Broadcom Corporation 16215 Alton Parkway Irvine,California 92619-7013美国电话:+1(949)926-7149电子邮件:MPenna@Broadcom.com

Jim Pinkerton Microsoft, Inc. One Microsoft Way Redmond, WA 98052 USA EMail: jpink@microsoft.com

Jim Pinkerton Microsoft,Inc.One Microsoft Way Redmond,WA 98052美国电子邮件:jpink@microsoft.com

Hemal Shah Broadcom Corporation 5300 California Avenue Irvine, CA 92617 USA Phone: +1 (949) 926-6941 EMail: hemal@broadcom.com

Hemal Shah Broadcom Corporation美国加利福尼亚州欧文市加利福尼亚大道5300号92617电话:+1(949)926-6941电子邮件:hemal@broadcom.com

Allyn Romanow Cisco Systems 170 W Tasman Drive San Jose, CA 95134 USA Phone: +1 408 525 8836 EMail: allyn@cisco.com

Allyn Romanow Cisco Systems 170 W美国加利福尼亚州圣何塞塔斯曼大道95134电话:+1 408 525 8836电子邮件:allyn@cisco.com

Tom Talpey Network Appliance 1601 Trapelo Road #16 Waltham, MA 02451 USA Phone: +1 (781) 768-5329 EMail: thomas.talpey@netapp.com

Tom Talpey Network Appliance 1601 Trapelo Road#16 Waltham,MA 02451美国电话:+1(781)768-5329电子邮件:thomas。talpey@netapp.com

Patricia Thaler Broadcom Corporation 16215 Alton Parkway Irvine, CA 92619-7013 USA Phone: +1-916-570-2707 EMail: pthaler@broadcom.com

Patricia Thaler Broadcom Corporation 16215 Alton Parkway Irvine,CA 92619-7013美国电话:+1-916-570-2707电子邮件:pthaler@broadcom.com

Jim Wendt Hewlett-Packard Company 8000 Foothills Boulevard MS 5668 Roseville, CA 95747-5668 USA Phone: +1 916 785 5198 EMail: jim_wendt@hp.com

Jim Wendt Hewlett-Packard Company 8000 Foothills Boulevard MS 5668,加利福尼亚州罗斯维尔95747-5668美国电话:+1 916 785 5198电子邮件:Jim_wendt@hp.com

Madeline Vega IBM 11400 Burnet Rd. Bld.45-2L-007 Austin, TX 78758 USA Phone: 512-838-7739 EMail: mvega1@us.ibm.com

Madeline Vega IBM 11400 Burnet路Bld.45-2L-007德克萨斯州奥斯汀78758美国电话:512-838-7739电子邮件:mvega1@us.ibm.com

Claudia Salzberg IBM 11501 Burnet Rd. Bld.902-5B-014 Austin, TX 78758 USA Phone: 512-838-5156 EMail: salzberg@us.ibm.com

Claudia Salzberg IBM 11501 Burnet路Bld.902-5B-014德克萨斯州奥斯汀78758美国电话:512-838-5156电子邮件:salzberg@us.ibm.com

Authors' Addresses

作者地址

Renato J. Recio IBM Corp. 11501 Burnett Road Austin, TX 78758 USA Phone: 512-838-3685 EMail: recio@us.ibm.com

Renato J.Recio IBM Corp.德克萨斯州奥斯汀伯内特路11501号美国电话:512-838-3685电子邮件:recio@us.ibm.com

Bernard Metzler IBM Research GmbH Zurich Research Laboratory Saeumerstrasse 4 CH-8803 Rueschlikon, Switzerland Phone: +41 44 724 8605 EMail: bmt@zurich.ibm.com

Bernard Metzler IBM Research GmbH苏黎世研究实验室Saeumerstrasse 4 CH-8803 Rueschlikon,瑞士电话:+41 44 724 8605电子邮件:bmt@zurich.ibm.com

Paul R. Culley Hewlett-Packard Company 20555 SH 249 Houston, TX 77070-2698 USA Phone: 281-514-5543 EMail: paul.culley@hp.com

Paul R.Culley Hewlett-Packard Company 20555 SH 249德克萨斯州休斯顿77070-2698美国电话:281-514-5543电子邮件:Paul。culley@hp.com

Jeff Hilland Hewlett-Packard Company 20555 SH 249 Houston, TX 77070-2698 USA Phone: 281-514-9489 EMail: jeff.hilland@hp.com

杰夫·希尔兰·惠普公司20555 SH 249德克萨斯州休斯顿77070-2698美国电话:281-514-9489电子邮件:杰夫。hilland@hp.com

Dave Garcia 24100 Hutchinson Rd. Los Gatos, CA 95033 USA Phone: +1 (831) 247-4464 Email: Dave.Garcia@StanfordAlumni.org

戴夫·加西亚美国加利福尼亚州洛斯加托斯哈钦森路24100号电话:+1(831)247-4464电子邮件:戴夫。Garcia@StanfordAlumni.org

Full Copyright Statement

完整版权声明

Copyright (C) The IETF Trust (2007).

版权所有(C)IETF信托基金(2007年)。

This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights.

本文件受BCP 78中包含的权利、许可和限制的约束,除其中规定外,作者保留其所有权利。

This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

本文件及其包含的信息以“原样”为基础提供,贡献者、他/她所代表或赞助的组织(如有)、互联网协会、IETF信托基金和互联网工程任务组不承担任何明示或暗示的担保,包括但不限于任何保证,即使用本文中的信息不会侵犯任何权利,或对适销性或特定用途适用性的任何默示保证。

Intellectual Property

知识产权

The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79.

IETF对可能声称与本文件所述技术的实施或使用有关的任何知识产权或其他权利的有效性或范围,或此类权利下的任何许可可能或可能不可用的程度,不采取任何立场;它也不表示它已作出任何独立努力来确定任何此类权利。有关RFC文件中权利的程序信息,请参见BCP 78和BCP 79。

Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr.

向IETF秘书处披露的知识产权副本和任何许可证保证,或本规范实施者或用户试图获得使用此类专有权利的一般许可证或许可的结果,可从IETF在线知识产权存储库获取,网址为http://www.ietf.org/ipr.

The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org.

IETF邀请任何相关方提请其注意任何版权、专利或专利申请,或其他可能涵盖实施本标准所需技术的专有权利。请将信息发送至IETF的IETF-ipr@ietf.org.