Internet Engineering Task Force (IETF) V. Bhuvaneswaran
Request for Comments: 8455 A. Basil
Category: Informational Veryx Technologies
ISSN: 2070-1721 M. Tassinari
Hewlett Packard Enterprise
V. Manral
NanoSec
S. Banks
VSS Monitoring
October 2018
Internet Engineering Task Force (IETF) V. Bhuvaneswaran
Request for Comments: 8455 A. Basil
Category: Informational Veryx Technologies
ISSN: 2070-1721 M. Tassinari
Hewlett Packard Enterprise
V. Manral
NanoSec
S. Banks
VSS Monitoring
October 2018
Terminology for Benchmarking Software-Defined Networking (SDN) Controller Performance
软件定义网络（SDN）控制器性能基准测试术语
Abstract
摘要
This document defines terminology for benchmarking a Software-Defined Networking (SDN) controller's control-plane performance. It extends the terminology already defined in RFC 7426 for the purpose of benchmarking SDN Controllers. The terms provided in this document help to benchmark an SDN Controller's performance independently of the controller's supported protocols and/or network services.
本文件定义了软件定义网络（SDN）控制器控制平面性能基准测试术语。它扩展了RFC 7426中已定义的术语，以对SDN控制器进行基准测试。本文档中提供的术语有助于独立于控制器支持的协议和/或网络服务对SDN控制器的性能进行基准测试。
Status of This Memo
关于下段备忘
This document is not an Internet Standards Track specification; it is published for informational purposes.
本文件不是互联网标准跟踪规范；它是为了提供信息而发布的。
This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Not all documents approved by the IESG are a candidate for any level of Internet Standard; see Section 2 of RFC 7841.
本文件是互联网工程任务组（IETF）的产品。它代表了IETF社区的共识。它已经接受了公众审查，并已被互联网工程指导小组（IESG）批准出版。并非IESG批准的所有文件都适用于任何级别的互联网标准；见RFC 7841第2节。
Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at https://www.rfc-editor.org/info/rfc8455.
有关本文件当前状态、任何勘误表以及如何提供反馈的信息，请访问https://www.rfc-editor.org/info/rfc8455.
Copyright Notice
版权公告
Copyright (c) 2018 IETF Trust and the persons identified as the document authors. All rights reserved.
版权所有（c）2018 IETF信托基金和确定为文件作者的人员。版权所有。
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
本文件受BCP 78和IETF信托有关IETF文件的法律规定的约束(https://trustee.ietf.org/license-info)自本文件出版之日起生效。请仔细阅读这些文件，因为它们描述了您对本文件的权利和限制。从本文件中提取的代码组件必须包括信托法律条款第4.e节中所述的简化BSD许可证文本，并提供简化BSD许可证中所述的无担保。
Table of Contents
目录
1. Introduction ....................................................3
1.1. Conventions Used in This Document ..........................3
2. Term Definitions ................................................4
2.1. SDN Terms ..................................................4
2.1.1. Flow ................................................4
2.1.2. Northbound Interface ................................4
2.1.3. Southbound Interface ................................5
2.1.4. Controller Forwarding Table .........................5
2.1.5. Proactive Flow Provisioning Mode ....................5
2.1.6. Reactive Flow Provisioning Mode .....................6
2.1.7. Path ................................................6
2.1.8. Standalone Mode .....................................6
2.1.9. Cluster/Redundancy Mode .............................7
2.1.10. Asynchronous Message ...............................7
2.1.11. Test Traffic Generator .............................7
2.1.12. Leaf-Spine Topology ................................8
2.2. Test Configuration/Setup Terms .............................8
2.2.1. Number of Network Devices ...........................8
2.2.2. Trial Repetition ....................................8
2.2.3. Trial Duration ......................................9
2.2.4. Number of Cluster Nodes .............................9
2.3. Benchmarking Terms .........................................9
2.3.1. Performance .........................................9
2.3.1.1. Network Topology Discovery Time ............9
2.3.1.2. Asynchronous Message Processing Time ......10
2.3.1.3. Asynchronous Message Processing Rate ......10
2.3.1.4. Reactive Path Provisioning Time ...........11
2.3.1.5. Proactive Path Provisioning Time ..........12
2.3.1.6. Reactive Path Provisioning Rate ...........12
2.3.1.7. Proactive Path Provisioning Rate ..........13
2.3.1.8. Network Topology Change Detection Time ....13
1. Introduction ....................................................3
1.1. Conventions Used in This Document ..........................3
2. Term Definitions ................................................4
2.1. SDN Terms ..................................................4
2.1.1. Flow ................................................4
2.1.2. Northbound Interface ................................4
2.1.3. Southbound Interface ................................5
2.1.4. Controller Forwarding Table .........................5
2.1.5. Proactive Flow Provisioning Mode ....................5
2.1.6. Reactive Flow Provisioning Mode .....................6
2.1.7. Path ................................................6
2.1.8. Standalone Mode .....................................6
2.1.9. Cluster/Redundancy Mode .............................7
2.1.10. Asynchronous Message ...............................7
2.1.11. Test Traffic Generator .............................7
2.1.12. Leaf-Spine Topology ................................8
2.2. Test Configuration/Setup Terms .............................8
2.2.1. Number of Network Devices ...........................8
2.2.2. Trial Repetition ....................................8
2.2.3. Trial Duration ......................................9
2.2.4. Number of Cluster Nodes .............................9
2.3. Benchmarking Terms .........................................9
2.3.1. Performance .........................................9
2.3.1.1. Network Topology Discovery Time ............9
2.3.1.2. Asynchronous Message Processing Time ......10
2.3.1.3. Asynchronous Message Processing Rate ......10
2.3.1.4. Reactive Path Provisioning Time ...........11
2.3.1.5. Proactive Path Provisioning Time ..........12
2.3.1.6. Reactive Path Provisioning Rate ...........12
2.3.1.7. Proactive Path Provisioning Rate ..........13
2.3.1.8. Network Topology Change Detection Time ....13
2.3.2. Scalability ........................................14
2.3.2.1. Control Sessions Capacity .................14
2.3.2.2. Network Discovery Size ....................14
2.3.2.3. Forwarding Table Capacity .................15
2.3.3. Security ...........................................15
2.3.3.1. Exception Handling ........................15
2.3.3.2. Handling Denial-of-Service Attacks ........16
2.3.4. Reliability ........................................16
2.3.4.1. Controller Failover Time ..................16
2.3.4.2. Network Re-provisioning Time ..............17
3. Test Setup .....................................................17
3.1. Test Setup - Controller Operating in Standalone Mode ......18
3.2. Test Setup - Controller Operating in Cluster Mode .........19
4. Test Coverage ..................................................20
5. IANA Considerations ............................................21
6. Security Considerations ........................................21
7. Normative References ...........................................21
Acknowledgments ...................................................22
Authors' Addresses ................................................23
2.3.2. Scalability ........................................14
2.3.2.1. Control Sessions Capacity .................14
2.3.2.2. Network Discovery Size ....................14
2.3.2.3. Forwarding Table Capacity .................15
2.3.3. Security ...........................................15
2.3.3.1. Exception Handling ........................15
2.3.3.2. Handling Denial-of-Service Attacks ........16
2.3.4. Reliability ........................................16
2.3.4.1. Controller Failover Time ..................16
2.3.4.2. Network Re-provisioning Time ..............17
3. Test Setup .....................................................17
3.1. Test Setup - Controller Operating in Standalone Mode ......18
3.2. Test Setup - Controller Operating in Cluster Mode .........19
4. Test Coverage ..................................................20
5. IANA Considerations ............................................21
6. Security Considerations ........................................21
7. Normative References ...........................................21
Acknowledgments ...................................................22
Authors' Addresses ................................................23
Software-Defined Networking (SDN) is a networking architecture in which network control is decoupled from the underlying forwarding function and is placed in a centralized location called the SDN Controller. The SDN Controller provides an abstraction of the underlying network and offers a global view of the overall network to applications and business logic. Thus, an SDN Controller provides the flexibility to program, control, and manage network behavior dynamically through northbound and southbound interfaces. Since the network controls are logically centralized, the need to benchmark the SDN Controller's performance becomes significant. This document defines terms to benchmark various controller designs for performance, scalability, reliability, and security, independently of northbound and southbound protocols. A mechanism for benchmarking the performance of SDN Controllers is defined in the companion methodology document [RFC8456]. These two documents provide methods for measuring and evaluating the performance of various controller implementations.
软件定义网络（SDN）是一种网络体系结构，其中网络控制与底层转发功能分离，并置于称为SDN控制器的集中位置。SDN控制器提供底层网络的抽象，并向应用程序和业务逻辑提供整个网络的全局视图。因此，SDN控制器提供了通过北向和南向接口动态编程、控制和管理网络行为的灵活性。由于网络控制在逻辑上是集中的，因此需要对SDN控制器的性能进行基准测试。本文档定义了各种控制器设计的性能、可扩展性、可靠性和安全性基准测试术语，独立于北向和南向协议。SDN控制器性能基准测试机制在配套方法文件[RFC8456]中定义。这两个文档提供了测量和评估各种控制器实现性能的方法。
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.
本文件中的关键词“必须”、“不得”、“必需”、“应”、“不应”、“建议”、“不建议”、“可”和“可选”在所有大写字母出现时（如图所示）应按照BCP 14[RFC2119][RFC8174]所述进行解释。
The terms defined in this section are extensions to the terms defined in [RFC7426] ("Software-Defined Networking (SDN): Layers and Architecture Terminology"). Readers should refer to [RFC7426] before attempting to make use of this document.
本节中定义的术语是对[RFC7426]（“软件定义网络（SDN）：层和架构术语”）中定义的术语的扩展。读者在尝试使用本文档之前，应参考[RFC7426]。
Definition: The definition of "flow" is the same as the definition of "microflows" provided in Section 3.1.5 of [RFC4689].
定义：“流动”的定义与[RFC4689]第3.1.5节中提供的“微流动”的定义相同。
Discussion: A flow can be a set of packets having the same source address, destination address, source port, and destination port, or any combination of these items.
讨论：流可以是具有相同源地址、目标地址、源端口和目标端口的一组数据包，也可以是这些项的任意组合。
Measurement Units: N/A
计量单位：不适用
Definition: The definition of "northbound interface" is the same as the definition of "service interface" provided in [RFC7426].
定义：“北向接口”的定义与[RFC7426]中提供的“服务接口”的定义相同。
Discussion: The northbound interface allows SDN applications and orchestration systems to program and retrieve the network information through the SDN Controller.
讨论：北行接口允许SDN应用程序和编排系统通过SDN控制器编程和检索网络信息。
Measurement Units: N/A
计量单位：不适用
Definition: The southbound interface is the application programming interface provided by the SDN Controller to interact with the SDN nodes.
定义：southbound接口是SDN控制器提供的与SDN节点交互的应用程序编程接口。
Discussion: The southbound interface enables the controller to interact with the SDN nodes in the network for dynamically defining the traffic forwarding behavior.
讨论：southbound接口使控制器能够与网络中的SDN节点交互，以动态定义流量转发行为。
Measurement Units: N/A
计量单位：不适用
Definition: A controller Forwarding Table contains flow entries learned in one of two ways: first, entries can be learned from traffic received through the data plane, or second, these entries can be statically provisioned on the controller and distributed to devices via the southbound interface.
定义：控制器转发表包含通过以下两种方式之一学习的流条目：首先，可以从通过数据平面接收的流量中学习条目，或者第二，可以在控制器上静态设置这些条目，并通过southbound接口分发给设备。
Discussion: The controller Forwarding Table has an aging mechanism that will be applied only for dynamically learned entries.
讨论：控制器转发表具有老化机制，该机制将仅应用于动态学习的条目。
Measurement Units: N/A
计量单位：不适用
Definition: Controller programming flows in Network Devices based on the flow entries provisioned through the controller's northbound interface.
定义：基于通过控制器北行接口提供的流条目，网络设备中的控制器编程流。
Discussion: Network orchestration systems and SDN applications can define the network forwarding behavior by programming the controller, using Proactive Flow Provisioning. The controller can then program the Network Devices with the pre-provisioned entries.
讨论：网络编排系统和SDN应用程序可以通过使用主动流供应对控制器进行编程来定义网络转发行为。然后，控制器可以使用预先设置的条目对网络设备进行编程。
Measurement Units: N/A
计量单位：不适用
Definition: Controller programming flows in Network Devices based on the traffic received from Network Devices through the controller's southbound interface.
定义：基于通过控制器南行接口从网络设备接收的流量，控制器编程在网络设备中流动。
Discussion: The SDN Controller dynamically decides the forwarding behavior based on the incoming traffic from the Network Devices. The controller then programs the Network Devices, using Reactive Flow Provisioning.
讨论：SDN控制器根据来自网络设备的传入流量动态决定转发行为。然后，控制器使用反应流供应对网络设备进行编程。
Measurement Units: N/A
计量单位：不适用
Definition: Refer to Section 5 in [RFC2330].
定义：参考[RFC2330]中的第5节。
Discussion: None
讨论：无
Measurement Units: N/A
计量单位：不适用
Definition: A single controller handles all control-plane functionalities without redundancy, and it is unable to provide high availability and/or automatic failover.
定义：单个控制器在没有冗余的情况下处理所有控制平面功能，并且无法提供高可用性和/或自动故障切换。
Discussion: In standalone mode, one controller manages one or more network domains.
讨论：在独立模式下，一个控制器管理一个或多个网络域。
Measurement Units: N/A
计量单位：不适用
Definition: In this mode, a group of two or more controllers handles all control-plane functionalities.
定义：在此模式下，由两个或多个控制器组成的组处理所有控制平面功能。
Discussion: In cluster mode, multiple controllers are teamed together for the purpose of load sharing and/or high availability. The controllers in the group may operate in active/standby (master/slave) or active/active (equal) mode, depending on the intended purpose.
讨论：在群集模式下，为了负载共享和/或高可用性，多个控制器组合在一起。组中的控制器可在主/备用（主/从）或主/主（同等）模式下运行，具体取决于预期用途。
Measurement Units: N/A
计量单位：不适用
Definition: Any message from the Network Device that is generated for network events.
定义：为网络事件生成的来自网络设备的任何消息。
Discussion: Control messages like flow setup request and response messages are classified as asynchronous messages. The controller has to return a response message. Note that the Network Device will not be in blocking mode and continues to send/receive other control messages.
讨论：流设置请求和响应消息等控制消息被分类为异步消息。控制器必须返回响应消息。请注意，网络设备将不会处于阻塞模式，并继续发送/接收其他控制消息。
Measurement Units: N/A
计量单位：不适用
Definition: The test traffic generator is an entity that generates/receives network traffic.
定义：测试流量生成器是生成/接收网络流量的实体。
Discussion: The test traffic generator typically connects with Network Devices to send/receive real-time network traffic.
讨论：测试流量生成器通常与网络设备连接以发送/接收实时网络流量。
Measurement Units: N/A
计量单位：不适用
Definition: "Leaf-Spine" is a two-layered network topology, where a series of leaf switches that form the access layer are fully meshed to a series of spine switches that form the backbone layer.
定义：“Leaf Spine”是一种两层网络拓扑结构，其中构成接入层的一系列Leaf交换机与构成主干层的一系列Spine交换机完全啮合。
Discussion: In the Leaf-Spine topology, every leaf switch is connected to each of the spine switches in the topology.
讨论：在叶-脊椎拓扑中，每个叶开关都连接到拓扑中的每个脊椎开关。
Measurement Units: N/A
计量单位：不适用
Definition: The number of Network Devices present in the defined test topology.
定义：定义的测试拓扑中存在的网络设备数。
Discussion: The Network Devices defined in the test topology can be deployed using real hardware or can be emulated in hardware platforms.
讨论：测试拓扑中定义的网络设备可以使用真实硬件进行部署，也可以在硬件平台中进行仿真。
Measurement Units: Number of Network Devices.
测量单位：网络设备的数量。
Definition: The number of times the test needs to be repeated.
定义：需要重复测试的次数。
Discussion: The test needs to be repeated for multiple iterations to obtain a reliable metric. It is recommended that this test SHOULD be performed for at least 10 iterations to increase confidence in the measured results.
讨论：需要多次重复测试以获得可靠的度量。建议该测试至少进行10次迭代，以增加测量结果的可信度。
Measurement Units: Number of trials.
测量单位：试验次数。
Definition: Defines the duration of test trials for each iteration.
定义：定义每次迭代的测试持续时间。
Discussion: The Trial Duration forms the basis for "stop" criteria for benchmarking tests. Trials not completed within this time interval are considered incomplete.
讨论：试验持续时间构成基准测试“停止”标准的基础。在此时间间隔内未完成的试验视为不完整。
Measurement Units: Seconds.
测量单位：秒。
Definition: Defines the number of controllers present in the controller cluster.
定义：定义控制器群集中存在的控制器数量。
Discussion: This parameter is relevant when testing the controller's performance in clustering/teaming mode. The number of nodes in the cluster MUST be greater than 1.
讨论：在集群/团队模式下测试控制器性能时，此参数相关。群集中的节点数必须大于1。
Measurement Units: Number of controller nodes.
测量单位：控制器节点数。
This section defines metrics for benchmarking the SDN Controller. The procedure for performing the defined metrics is defined in the companion methodology document [RFC8456].
本节定义了SDN控制器基准测试的指标。相关方法文件[RFC8456]中定义了执行定义指标的程序。
Definition: The time taken by the controller(s) to determine the complete network topology, defined as the interval starting with the first discovery message from the controller(s) at its southbound interface and ending with all features of the static topology determined.
定义：控制器确定完整网络拓扑所用的时间，定义为从控制器在其南行接口发出的第一条发现消息开始到确定静态拓扑的所有特征结束的时间间隔。
Discussion: Network topology discovery is key for the SDN Controller to provision and manage the network, so it is important to measure how quickly the controller discovers the topology to learn the
讨论：网络拓扑发现是SDN控制器提供和管理网络的关键，因此测量控制器发现拓扑以了解网络的速度非常重要
current network state. This benchmark is obtained by presenting a network topology (tree, mesh, or linear) with a specified number of nodes to the controller and waiting for the discovery process to complete. It is expected that the controller supports a network discovery mechanism and uses protocol messages for its discovery process.
当前网络状态。通过向控制器呈现具有指定数量节点的网络拓扑（树、网格或线性），并等待发现过程完成，可以获得该基准。预计控制器支持网络发现机制，并使用协议消息进行发现过程。
Measurement Units: Milliseconds.
测量单位：毫秒。
Definition: The time taken by the controller(s) to process an asynchronous message, defined as the interval starting with an asynchronous message from a Network Device after the discovery of all the devices by the controller(s) and ending with a response message from the controller(s) at its southbound interface.
定义：控制器处理异步消息所用的时间，定义为控制器发现所有设备后，从网络设备发出的异步消息开始，到控制器在其南行接口发出响应消息结束的时间间隔。
Discussion: For SDN to support dynamic network provisioning, it is important to measure how quickly the controller responds to an event triggered from the network. The event can be any notification messages generated by a Network Device upon arrival of a new flow, link down, etc. This benchmark is obtained by sending asynchronous messages from every connected Network Device one at a time for the defined Trial Duration. This test assumes that the controller will respond to the received asynchronous messages.
讨论：要使SDN支持动态网络资源调配，必须测量控制器对网络触发的事件的响应速度。事件可以是网络设备在新流到达、链路断开等时生成的任何通知消息。该基准通过在定义的试用期内从每个连接的网络设备一次发送一条异步消息来获得。此测试假设控制器将响应接收到的异步消息。
Measurement Units: Milliseconds.
测量单位：毫秒。
Definition: The number of responses to asynchronous messages per second (a new flow arrival notification message, link down, etc.) for which the controller(s) performed processing and replied with a valid and productive (non-trivial) response message.
定义：控制器每秒对异步消息（新的流到达通知消息、链路断开等）执行处理并使用有效且有效（非平凡）的响应消息进行响应的响应数。
Discussion: As SDN assures a flexible network and agile provisioning, it is important to measure how many network events (a new flow arrival notification message, link down, etc.) the controller can handle at a time. This benchmark is measured by sending asynchronous messages from every connected Network Device at the rate that the controller processes (without dropping them). This test assumes
讨论：由于SDN确保了灵活的网络和敏捷的资源调配，因此测量控制器一次可以处理多少网络事件（新的流到达通知消息、链路断开等）非常重要。通过以控制器处理（不丢弃）的速率从每个连接的网络设备发送异步消息来测量此基准。本测试假设
that the controller responds to all the received asynchronous messages (the messages can be designed to elicit individual responses).
控制器对所有接收到的异步消息做出响应（这些消息可以设计为引发单独的响应）。
When sending asynchronous messages to the controller(s) at high rates, some messages or responses may be discarded or corrupted and require retransmission to controller(s). Therefore, a useful qualification on the Asynchronous Message Processing Rate is whether the incoming message count equals the response count in each trial. This is called the Loss-Free Asynchronous Message Processing Rate.
当以高速率向控制器发送异步消息时，某些消息或响应可能会被丢弃或损坏，并需要重新传输到控制器。因此，异步消息处理速率的一个有用的限定条件是传入消息计数是否等于每个试验中的响应计数。这称为无丢失异步消息处理速率。
Note that several of the early controller benchmarking tools did not consider lost messages and instead report the maximum response rate. This is called the Maximum Asynchronous Message Processing Rate.
请注意，一些早期的控制器基准工具没有考虑丢失的消息，而是报告最大的响应速率。这称为最大异步消息处理速率。
To characterize both the Loss-Free Asynchronous Message Processing Rate and the Maximum Asynchronous Message Processing Rate, a test can begin the first trial by sending asynchronous messages to the controller(s) at the maximum possible rate and can then record the message reply rate and the message loss rate. The message-sending rate is then decreased by the STEP size. The message reply rate and the message loss rate are recorded. The test ends with a trial where the controller(s) processes all of the asynchronous messages sent without loss. This is the Loss-Free Asynchronous Message Processing Rate.
为了表征无丢失异步消息处理速率和最大异步消息处理速率，测试可以通过以最大可能速率向控制器发送异步消息来开始第一次试验，然后可以记录消息回复速率和消息丢失率。然后，消息发送速率将减小步长。记录消息回复率和消息丢失率。测试以一个试验结束，其中控制器处理发送的所有异步消息，不会丢失。这是无丢失异步消息处理速率。
The trial where the controller(s) produced the maximum response rate is the Maximum Asynchronous Message Processing Rate. Of course, the first trial can begin at a low sending rate with zero lost responses and then increase the rate until the Loss-Free Asynchronous Message Processing Rate and the Maximum Asynchronous Message Processing Rate are discovered.
控制器产生最大响应速率的试验是最大异步消息处理速率。当然，第一次尝试可以以低发送速率开始，零丢失响应，然后增加速率，直到找到无丢失异步消息处理速率和最大异步消息处理速率。
Measurement Units: Messages processed per second.
度量单位：每秒处理的消息。
Definition: The time taken by the controller to set up a path reactively between source and destination nodes, defined as the interval starting with the first flow provisioning request message received by the controller(s) and ending with the last flow provisioning response message sent from the controller(s) at its southbound interface.
定义：控制器在源节点和目标节点之间以反应方式设置路径所用的时间，定义为从控制器接收到的第一条流配置请求消息开始到控制器在其南行接口发送的最后一条流配置响应消息结束的时间间隔。
Discussion: As SDN supports agile provisioning, it is important to measure how fast the controller provisions an end-to-end flow in the data plane. The benchmark is obtained by sending traffic from a source endpoint to the destination endpoint and finding the time difference between the first and last flow provisioning message exchanged between the controller and the Network Devices for the traffic path.
讨论：由于SDN支持敏捷资源调配，因此测量控制器在数据平面中提供端到端流的速度非常重要。基准测试是通过将流量从源端点发送到目标端点，并查找控制器和网络设备之间交换的流量路径的第一个和最后一个流量供应消息之间的时间差来获得的。
Measurement Units: Milliseconds.
测量单位：毫秒。
Definition: The time taken by the controller to proactively set up a path between source and destination nodes, defined as the interval starting with the first proactive flow provisioned in the controller(s) at its northbound interface and ending with the last flow provisioning command message sent from the controller(s) at its southbound interface.
定义：控制器在源节点和目标节点之间主动设置路径所用的时间，定义为从控制器在其北行接口处配置的第一个主动流开始到从控制器发送的最后一个流配置命令消息结束的间隔在其南行界面。
Discussion: For SDN to support pre-provisioning of the traffic path from the application, it is important to measure how fast the controller provisions an end-to-end flow in the data plane. The benchmark is obtained by provisioning a flow on the controller's northbound interface for the traffic to reach from a source to a destination endpoint and finding the time difference between the first and last flow provisioning message exchanged between the controller and the Network Devices for the traffic path.
讨论：对于SDN来说，为了支持来自应用程序的流量路径的预调配，测量控制器在数据平面中调配端到端流的速度非常重要。基准是通过在控制器的北行接口上为流量从源到达目标端点提供流量，并查找控制器和网络设备之间交换的流量路径的第一个和最后一个流量提供消息之间的时间差来获得的。
Measurement Units: Milliseconds.
测量单位：毫秒。
Definition: The maximum number of independent paths a controller can concurrently establish per second between source and destination nodes reactively, defined as the number of paths provisioned per second by the controller(s) at its southbound interface for the flow provisioning requests received for path provisioning at its southbound interface between the start of the trial and the expiry of the given Trial Duration.
定义：控制器每秒可在源节点和目标节点之间以反应方式并发建立的最大独立路径数，定义为控制器每秒提供的路径数在试验开始和给定试验持续时间到期之间，在其南行接口处接收到用于路径供应的流供应请求的南行接口处。
Discussion: For SDN to support agile traffic forwarding, it is important to measure how many end-to-end flows the controller can set up in the data plane. This benchmark is obtained by sending each traffic flow with unique source and destination pairs from the source Network Device and determining the number of frames received at the destination Network Device.
讨论：对于支持敏捷流量转发的SDN，测量控制器在数据平面中可以设置多少端到端流量是很重要的。该基准通过从源网络设备发送具有唯一源和目的地对的每个业务流，并确定在目的地网络设备处接收的帧数来获得。
Measurement Units: Paths provisioned per second.
度量单位：每秒配置的路径。
Definition: The maximum number of independent paths a controller can concurrently establish per second between source and destination nodes proactively, defined as the number of paths provisioned per second by the controller(s) at its southbound interface for the paths provisioned in its northbound interface between the start of the trial and the expiry of the given Trial Duration.
定义：控制器每秒可在源节点和目标节点之间主动并发建立的最大独立路径数，定义为控制器每秒提供的路径数在试验开始和给定试验持续时间到期之间的北行接口中提供的路径的南行接口。
Discussion: For SDN to support pre-provisioning of the traffic path for a larger network from the application, it is important to measure how many end-to-end flows the controller can set up in the data plane. This benchmark is obtained by sending each traffic flow with unique source and destination pairs from the source Network Device. Program the flows on the controller's northbound interface for traffic to reach from each of the unique source and destination pairs, and determine the number of frames received at the destination Network Device.
讨论：为了使SDN支持从应用程序为更大的网络预先提供流量路径，测量控制器可以在数据平面中设置的端到端流量非常重要。该基准通过从源网络设备发送具有唯一源和目的地对的每个业务流来获得。在控制器的北向接口上对流量进行编程，以使流量从每个唯一的源和目的地对到达，并确定在目的地网络设备上接收的帧数。
Measurement Units: Paths provisioned per second.
度量单位：每秒配置的路径。
Definition: The amount of time taken by the controller to detect any changes in the network topology, defined as the interval starting with the notification message received by the controller(s) at its southbound interface and ending with the first topology rediscovery messages sent from the controller(s) at its southbound interface.
定义：控制器检测网络拓扑中任何更改所用的时间量，定义为从控制器在其南行接口接收到的通知消息开始，到控制器在其南行接口发送的第一条拓扑重新发现消息结束的间隔。
Discussion: In order for the controller to support fast network failure recovery, it is critical to measure how fast the controller is able to detect any network-state change events. This benchmark is obtained by triggering a topology change event and measuring the time the controller takes to detect and initiate a topology rediscovery process.
讨论：为了使控制器支持快速网络故障恢复，测量控制器能够检测任何网络状态更改事件的速度至关重要。该基准通过触发拓扑更改事件和测量控制器检测和启动拓扑重新发现过程所需的时间来获得。
Measurement Units: Milliseconds.
测量单位：毫秒。
Definition: The maximum number of control sessions the controller can maintain, defined as the number of sessions that the controller can accept from Network Devices, starting with the first control session and ending with the last control session that the controller(s) accepts at its southbound interface.
定义：控制器可以维护的最大控制会话数，定义为控制器可以从网络设备接受的会话数，从第一个控制会话开始，到控制器在其南行接口接受的最后一个控制会话结束。
Discussion: Measuring the controller's Control Sessions Capacity is important for determining the controller's system and bandwidth resource requirements. This benchmark is obtained by establishing a control session with the controller from each of the Network Devices until the controller fails. The number of sessions that were successfully established will provide the Control Sessions Capacity.
讨论：测量控制器的控制会话容量对于确定控制器的系统和带宽资源需求非常重要。该基准是通过与每个网络设备的控制器建立控制会话来获得的，直到控制器出现故障。成功建立的会话数将提供控制会话容量。
Measurement Units: Maximum number of control sessions.
测量单位：控制会话的最大数量。
Definition: The network size (number of nodes and links) that a controller can discover, defined as the size of a network that the controller(s) can discover, starting with a network topology provided by the user for discovery and ending with the number of nodes and links that the controller(s) can successfully discover.
定义：控制器可以发现的网络大小（节点和链路数），定义为控制器可以发现的网络大小，从用户提供的用于发现的网络拓扑开始，以控制器可以成功发现的节点和链路数结束。
Discussion: Measuring the maximum network size that the controller can discover is key to optimal network planning. This benchmark is obtained by presenting an initial set of Network Devices for discovery to the controller. Based on the initial discovery, the
讨论：测量控制器可以发现的最大网络大小是优化网络规划的关键。该基准是通过向控制器提供用于发现的初始网络设备集而获得的。根据最初的发现
number of Network Devices is increased or decreased to determine the maximum number of nodes and links that the controller can discover.
增加或减少网络设备的数量，以确定控制器可以发现的节点和链路的最大数量。
Measurement Units: Maximum number of network nodes and links.
测量单位：网络节点和链路的最大数量。
Definition: The maximum number of flow entries that a controller can manage in its Forwarding Table.
定义：控制器可在其转发表中管理的最大流条目数。
Discussion: It is important to measure the capacity of a controller's Forwarding Table to determine the number of flows that the controller can forward without flooding or dropping any traffic. This benchmark is obtained by continuously presenting the controller with new flow entries through the Reactive Flow Provisioning mode or the Proactive Flow Provisioning mode until the Forwarding Table becomes full. The maximum number of nodes that the controller can hold in its Forwarding Table will provide the Forwarding Table Capacity.
讨论：测量控制器转发表的容量以确定控制器在不淹没或丢弃任何流量的情况下可以转发的流的数量非常重要。该基准是通过通过反应流供应模式或主动流供应模式持续向控制器提供新的流条目，直到转发表满为止。控制器可在其转发表中容纳的最大节点数将提供转发表容量。
Measurement Units: Maximum number of flow entries managed.
测量单位：管理的最大流量条目数。
Definition: To determine the effect of handling error packets and notifications on performance tests.
定义：确定处理错误数据包和通知对性能测试的影响。
Discussion: This benchmark is to be performed after obtaining the baseline measurement results for the performance tests defined in Section 2.3.1. This benchmark determines the deviation from the baseline performance due to the handling of error or failure messages from the connected Network Devices.
讨论：在获得第2.3.1节中定义的性能测试的基线测量结果后，执行该基准。此基准确定由于处理来自连接网络设备的错误或故障消息而导致的与基准性能的偏差。
Measurement Units: Deviation from baseline metrics while handling Exceptions.
度量单位：处理异常时与基线度量的偏差。
Definition: To determine the effect of handling denial-of-service (DoS) attacks on performance and scalability tests.
定义：确定处理拒绝服务（DoS）攻击对性能和可伸缩性测试的影响。
Discussion: This benchmark is to be performed after obtaining the baseline measurement results for the performance and scalability tests defined in Sections 2.3.1 and 2.3.2. This benchmark determines the deviation from the baseline performance due to the handling of DoS attacks on the controller.
讨论：在获得第2.3.1节和第2.3.2节中定义的性能和可伸缩性测试的基线测量结果后，将执行该基准测试。此基准确定了由于处理控制器上的DoS攻击而导致的与基准性能的偏差。
Measurement Units: Deviation from baseline metrics while handling DoS attacks.
度量单位：处理DoS攻击时与基线度量的偏差。
Definition: The time taken to switch from an active controller to the backup controller when the controllers operate in redundancy mode and the active controller fails, defined as the interval starting when the active controller is brought down and ending with the first rediscovery message received from the new controller at its southbound interface.
定义：当控制器在冗余模式下运行且主动控制器发生故障时，从主动控制器切换到备用控制器所需的时间，定义为从激活控制器关闭时开始，到从新控制器南行接口接收到第一条重新发现消息时结束的间隔。
Discussion: This benchmark determines the impact of provisioning new flows when controllers are teamed together and the active controller fails.
讨论：当控制器组合在一起且活动控制器出现故障时，此基准确定了供应新流的影响。
Measurement Units: Milliseconds.
测量单位：毫秒。
Definition: The time taken by the controller to reroute traffic when there is a failure in existing traffic paths, defined as the interval starting with the first failure notification message received by the controller and ending with the last flow re-provisioning message sent by the controller at its southbound interface.
定义：当现有流量路径出现故障时，控制器重新路由流量所用的时间，定义为从控制器接收到的第一条故障通知消息开始，到控制器在其南行接口发送的最后一条流量重新配置消息结束的间隔。
Discussion: This benchmark determines the controller's re-provisioning ability upon network failures and makes the following assumptions:
讨论：该基准确定控制器在网络故障时的重新配置能力，并做出以下假设：
1. The network topology supports a redundant path between the source and destination endpoints.
1. 网络拓扑支持源端点和目标端点之间的冗余路径。
2. The controller does not pre-provision the redundant path.
2. 控制器不预先设置冗余路径。
Measurement Units: Milliseconds.
测量单位：毫秒。
This section provides common reference topologies that are referred to in individual tests defined in the companion methodology document [RFC8456].
本节提供了在配套方法文件[RFC8456]中定义的单个测试中参考的通用参考拓扑。
+-----------------------------------------------------------+
| Application-Plane Test Emulator |
| |
| +-----------------+ +-------------+ |
| | Application | | Service | |
| +-----------------+ +-------------+ |
| |
+-----------------------------+(I2)-------------------------+
|
| (Northbound Interface)
+-------------------------------+
| +----------------+ |
| | SDN Controller | |
| +----------------+ |
| |
| Device Under Test (DUT) |
+-------------------------------+
| (Southbound Interface)
|
+-----------------------------+(I1)-------------------------+
| |
| +-----------+ +-----------+ |
| | Network | | Network | |
| | Device 2 |--..-| Device n-1| |
| +-----------+ +-----------+ |
| / \ / \ |
| / \ / \ |
| l0 / X \ ln |
| / / \ \ |
| +-----------+ +-----------+ |
| | Network | | Network | |
| | Device 1 |..| Device n | |
| +-----------+ +-----------+ |
| | | |
| +---------------+ +---------------+ |
| | Test Traffic | | Test Traffic | |
| | Generator | | Generator | |
| | (TP1) | | (TP2) | |
| +---------------+ +---------------+ |
| |
| Forwarding-Plane Test Emulator |
+-----------------------------------------------------------+
+-----------------------------------------------------------+
| Application-Plane Test Emulator |
| |
| +-----------------+ +-------------+ |
| | Application | | Service | |
| +-----------------+ +-------------+ |
| |
+-----------------------------+(I2)-------------------------+
|
| (Northbound Interface)
+-------------------------------+
| +----------------+ |
| | SDN Controller | |
| +----------------+ |
| |
| Device Under Test (DUT) |
+-------------------------------+
| (Southbound Interface)
|
+-----------------------------+(I1)-------------------------+
| |
| +-----------+ +-----------+ |
| | Network | | Network | |
| | Device 2 |--..-| Device n-1| |
| +-----------+ +-----------+ |
| / \ / \ |
| / \ / \ |
| l0 / X \ ln |
| / / \ \ |
| +-----------+ +-----------+ |
| | Network | | Network | |
| | Device 1 |..| Device n | |
| +-----------+ +-----------+ |
| | | |
| +---------------+ +---------------+ |
| | Test Traffic | | Test Traffic | |
| | Generator | | Generator | |
| | (TP1) | | (TP2) | |
| +---------------+ +---------------+ |
| |
| Forwarding-Plane Test Emulator |
+-----------------------------------------------------------+
Figure 1
图1
+-----------------------------------------------------------+
| Application-Plane Test Emulator |
| |
| +-----------------+ +-------------+ |
| | Application | | Service | |
| +-----------------+ +-------------+ |
| |
+-----------------------------+(I2)-------------------------+
|
| (Northbound Interface)
+---------------------------------------------------------+
| |
| +------------------+ +------------------+ |
| | SDN Controller 1 | <--E/W--> | SDN Controller n | |
| +------------------+ +------------------+ |
| |
| Device Under Test (DUT) |
+---------------------------------------------------------+
| (Southbound Interface)
|
+-----------------------------+(I1)-------------------------+
| |
| +-----------+ +-----------+ |
| | Network | | Network | |
| | Device 2 |--..-| Device n-1| |
| +-----------+ +-----------+ |
| / \ / \ |
| / \ / \ |
| l0 / X \ ln |
| / / \ \ |
| +-----------+ +-----------+ |
| | Network | | Network | |
| | Device 1 |..| Device n | |
| +-----------+ +-----------+ |
| | | |
| +---------------+ +---------------+ |
| | Test Traffic | | Test Traffic | |
| | Generator | | Generator | |
| | (TP1) | | (TP2) | |
| +---------------+ +---------------+ |
| |
| Forwarding-Plane Test Emulator |
+-----------------------------------------------------------+
+-----------------------------------------------------------+
| Application-Plane Test Emulator |
| |
| +-----------------+ +-------------+ |
| | Application | | Service | |
| +-----------------+ +-------------+ |
| |
+-----------------------------+(I2)-------------------------+
|
| (Northbound Interface)
+---------------------------------------------------------+
| |
| +------------------+ +------------------+ |
| | SDN Controller 1 | <--E/W--> | SDN Controller n | |
| +------------------+ +------------------+ |
| |
| Device Under Test (DUT) |
+---------------------------------------------------------+
| (Southbound Interface)
|
+-----------------------------+(I1)-------------------------+
| |
| +-----------+ +-----------+ |
| | Network | | Network | |
| | Device 2 |--..-| Device n-1| |
| +-----------+ +-----------+ |
| / \ / \ |
| / \ / \ |
| l0 / X \ ln |
| / / \ \ |
| +-----------+ +-----------+ |
| | Network | | Network | |
| | Device 1 |..| Device n | |
| +-----------+ +-----------+ |
| | | |
| +---------------+ +---------------+ |
| | Test Traffic | | Test Traffic | |
| | Generator | | Generator | |
| | (TP1) | | (TP2) | |
| +---------------+ +---------------+ |
| |
| Forwarding-Plane Test Emulator |
+-----------------------------------------------------------+
Figure 2
图2
+-------------------------------------------------------------------+
| Lifecycle | Speed | Scalability | Reliability |
+------------+-------------------+---------------+------------------+
| | 1. Network |1. Network | |
| | Topology | Discovery | |
| | Discovery | Size | |
| | Time | | |
| | | | |
| | 2. Reactive Path | | |
| | Provisioning | | |
| | Time | | |
| | | | |
| | 3. Proactive Path | | |
| Setup | Provisioning | | |
| | Time | | |
| | | | |
| | 4. Reactive Path | | |
| | Provisioning | | |
| | Rate | | |
| | | | |
| | 5. Proactive Path | | |
| | Provisioning | | |
| | Rate | | |
| | | | |
+------------+-------------------+---------------+------------------+
| | 1. Maximum |1. Control |1. Network |
| | Asynchronous | Sessions | Topology |
| | Message | Capacity | Change |
| | Processing Rate| | Detection Time |
| | |2. Forwarding | |
| | 2. Loss-Free | Table |2. Exception |
| | Asynchronous | Capacity | Handling |
| | Message | | |
| Operational| Processing Rate| |3. Handling |
| | | | Denial-of- |
| | 3. Asynchronous | | Service Attacks|
| | Message | | |
| | Processing Time| |4. Network |
| | | | Re-provisioning|
| | | | Time |
| | | | |
+------------+-------------------+---------------+------------------+
| Teardown | | |1. Controller |
| | | | Failover Time |
+------------+-------------------+---------------+------------------+
+-------------------------------------------------------------------+
| Lifecycle | Speed | Scalability | Reliability |
+------------+-------------------+---------------+------------------+
| | 1. Network |1. Network | |
| | Topology | Discovery | |
| | Discovery | Size | |
| | Time | | |
| | | | |
| | 2. Reactive Path | | |
| | Provisioning | | |
| | Time | | |
| | | | |
| | 3. Proactive Path | | |
| Setup | Provisioning | | |
| | Time | | |
| | | | |
| | 4. Reactive Path | | |
| | Provisioning | | |
| | Rate | | |
| | | | |
| | 5. Proactive Path | | |
| | Provisioning | | |
| | Rate | | |
| | | | |
+------------+-------------------+---------------+------------------+
| | 1. Maximum |1. Control |1. Network |
| | Asynchronous | Sessions | Topology |
| | Message | Capacity | Change |
| | Processing Rate| | Detection Time |
| | |2. Forwarding | |
| | 2. Loss-Free | Table |2. Exception |
| | Asynchronous | Capacity | Handling |
| | Message | | |
| Operational| Processing Rate| |3. Handling |
| | | | Denial-of- |
| | 3. Asynchronous | | Service Attacks|
| | Message | | |
| | Processing Time| |4. Network |
| | | | Re-provisioning|
| | | | Time |
| | | | |
+------------+-------------------+---------------+------------------+
| Teardown | | |1. Controller |
| | | | Failover Time |
+------------+-------------------+---------------+------------------+
This document has no IANA actions.
本文档没有IANA操作。
The benchmarking tests described in this document are limited to the performance characterization of controllers in a lab environment with isolated networks.
本文档中描述的基准测试仅限于在具有隔离网络的实验室环境中对控制器的性能进行表征。
The benchmarking network topology will be an independent test setup and MUST NOT be connected to devices that may forward the test traffic into a production network or misroute traffic to the test management network.
基准网络拓扑将是一个独立的测试设置，不得连接到可能将测试流量转发到生产网络或将流量错误路由到测试管理网络的设备。
Further, benchmarking is performed on a "black-box" basis, relying solely on measurements observable external to the controller.
此外，基准测试是在“黑盒”的基础上进行的，完全依赖于控制器外部可观察到的测量。
Special capabilities SHOULD NOT exist in the controller specifically for benchmarking purposes. Any implications for network security arising from the controller SHOULD be identical in the lab and in production networks.
控制器中不应存在专门用于基准测试的特殊功能。在实验室和生产网络中，控制器对网络安全的影响应相同。
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, <https://www.rfc-editor.org/info/rfc2119>.
[RFC2119]Bradner，S.，“RFC中用于表示需求水平的关键词”，BCP 14，RFC 2119，DOI 10.17487/RFC2119，1997年3月<https://www.rfc-editor.org/info/rfc2119>.
[RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, "Framework for IP Performance Metrics", RFC 2330, DOI 10.17487/RFC2330, May 1998, <https://www.rfc-editor.org/info/rfc2330>.
[RFC2330]Paxson，V.，Almes，G.，Mahdavi，J.，和M.Mathis，“IP性能度量框架”，RFC 2330，DOI 10.17487/RFC2330，1998年5月<https://www.rfc-editor.org/info/rfc2330>.
[RFC4689] Poretsky, S., Perser, J., Erramilli, S., and S. Khurana, "Terminology for Benchmarking Network-layer Traffic Control Mechanisms", RFC 4689, DOI 10.17487/RFC4689, October 2006, <https://www.rfc-editor.org/info/rfc4689>.
[RFC4689]Poretsky，S.，Perser，J.，Erramilli，S.，和S.Khurana，“基准网络层流量控制机制的术语”，RFC 4689，DOI 10.17487/RFC4689，2006年10月<https://www.rfc-editor.org/info/rfc4689>.
[RFC7426] Haleplidis, E., Ed., Pentikousis, K., Ed., Denazis, S., Hadi Salim, J., Meyer, D., and O. Koufopavlou, "Software-Defined Networking (SDN): Layers and Architecture Terminology", RFC 7426, DOI 10.17487/RFC7426, January 2015, <https://www.rfc-editor.org/info/rfc7426>.
[RFC7426]Haleplis，E.，Ed.，Pentikousis，K.，Ed.，Denazis，S.，Hadi Salim，J.，Meyer，D.，和O.Koufopavlou，“软件定义网络（SDN）：层和架构术语”，RFC 7426，DOI 10.17487/RFC7426，2015年1月<https://www.rfc-editor.org/info/rfc7426>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[RFC8174]Leiba，B.，“RFC 2119关键词中大写与小写的歧义”，BCP 14，RFC 8174，DOI 10.17487/RFC8174，2017年5月<https://www.rfc-editor.org/info/rfc8174>.
[RFC8456] Bhuvaneswaran, V., Basil, A., Tassinari, M., Manral, V., and S. Banks, "Benchmarking Methodology for Software-Defined Networking (SDN) Controller Performance", RFC 8456, DOI 10.17487/RFC8456, October 2018, <https://www.rfc-editor.org/info/rfc8456>.
[RFC8456]Bhuvaneswaran，V.，Basil，A.，Tassinari，M.，Manral，V.，和S.Banks，“软件定义网络（SDN）控制器性能的基准测试方法”，RFC 8456，DOI 10.17487/RFC8456，2018年10月<https://www.rfc-editor.org/info/rfc8456>.
Acknowledgments
致谢
The authors would like to acknowledge Al Morton (AT&T) for his significant contributions to the earlier draft versions of this document. The authors would like to thank the following individuals for providing their valuable comments to the earlier draft versions of this document: Sandeep Gangadharan (HP), M. Georgescu (NAIST), Andrew McGregor (Google), Scott Bradner, Jay Karthik (Cisco), Ramki Krishnan (VMware), and Boris Khasanov (Huawei).
作者要感谢Al Morton（AT&T）对本文件早期草稿的重要贡献。作者要感谢以下个人对本文件早期草稿提出了宝贵意见：Sandeep Gangadharan（惠普）、M.Georgescu（奈斯特）、Andrew McGregor（谷歌）、Scott Bradner、Jay Karthik（思科）、Ramki Krishnan（VMware）和Boris Khasanov（华为）。
Authors' Addresses
作者地址
Bhuvaneswaran Vengainathan Veryx Technologies Inc. 1 International Plaza, Suite 550 Philadelphia, PA 19113 United States of America
布瓦内斯瓦兰·文盖纳森·维克斯技术公司，美国宾夕法尼亚州费城国际广场1号550室，邮编：19113
Email: bhuvaneswaran.vengainathan@veryxtech.com
Email: bhuvaneswaran.vengainathan@veryxtech.com
Anton Basil Veryx Technologies Inc. 1 International Plaza, Suite 550 Philadelphia, PA 19113 United States of America
Anton Basil Veryx Technologies Inc.美国宾夕法尼亚州费城国际广场1号550室19113
Email: anton.basil@veryxtech.com
Email: anton.basil@veryxtech.com
Mark Tassinari Hewlett Packard Enterprise 8000 Foothills Blvd. Roseville, CA 95747 United States of America
马克·塔西纳里·惠普企业8000山麓大道。美国加利福尼亚州罗斯维尔95747
Email: mark.tassinari@hpe.com
Email: mark.tassinari@hpe.com
Vishwas Manral NanoSec Co 3350 Thomas Rd. Santa Clara, CA 95054 United States of America
美国加利福尼亚州圣克拉拉市托马斯路3350号维斯瓦斯曼拉尔纳米公司，邮编95054
Email: vishwas.manral@gmail.com
Email: vishwas.manral@gmail.com
Sarah Banks VSS Monitoring 930 De Guigne Drive Sunnyvale, CA 94085 United States of America
美国加利福尼亚州桑尼维尔德吉涅大道930号，邮编94085
Email: sbanks@encrypted.net
Email: sbanks@encrypted.net