Internet Engineering Task Force (IETF) A. Morton Request for Comments: 6703 G. Ramachandran Category: Informational G. Maguluri ISSN: 2070-1721 AT&T Labs August 2012
Internet Engineering Task Force (IETF) A. Morton Request for Comments: 6703 G. Ramachandran Category: Informational G. Maguluri ISSN: 2070-1721 AT&T Labs August 2012
Reporting IP Network Performance Metrics: Different Points of View
报告IP网络性能指标:不同的观点
Abstract
摘要
Consumers of IP network performance metrics have many different uses in mind. This memo provides "long-term" reporting considerations (e.g., hours, days, weeks, or months, as opposed to 10 seconds), based on analysis of the points of view of two key audiences. It describes how these audience categories affect the selection of metric parameters and options when seeking information that serves their needs.
IP网络性能指标的使用者有许多不同的用途。本备忘录基于对两个主要受众观点的分析,提供了“长期”报告考虑因素(例如,小时、天、周或月,而不是10秒)。它描述了这些受众类别在寻求满足其需求的信息时如何影响度量参数和选项的选择。
Status of This Memo
关于下段备忘
This document is not an Internet Standards Track specification; it is published for informational purposes.
本文件不是互联网标准跟踪规范;它是为了提供信息而发布的。
This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Not all documents approved by the IESG are a candidate for any level of Internet Standard; see Section 2 of RFC 5741.
本文件是互联网工程任务组(IETF)的产品。它代表了IETF社区的共识。它已经接受了公众审查,并已被互联网工程指导小组(IESG)批准出版。并非IESG批准的所有文件都适用于任何级别的互联网标准;见RFC 5741第2节。
Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at http://www.rfc-editor.org/info/rfc6703.
有关本文件当前状态、任何勘误表以及如何提供反馈的信息,请访问http://www.rfc-editor.org/info/rfc6703.
Copyright Notice
版权公告
Copyright (c) 2012 IETF Trust and the persons identified as the document authors. All rights reserved.
版权所有(c)2012 IETF信托基金和确定为文件作者的人员。版权所有。
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
本文件受BCP 78和IETF信托有关IETF文件的法律规定的约束(http://trustee.ietf.org/license-info)自本文件出版之日起生效。请仔细阅读这些文件,因为它们描述了您对本文件的权利和限制。从本文件中提取的代码组件必须包括信托法律条款第4.e节中所述的简化BSD许可证文本,并提供简化BSD许可证中所述的无担保。
This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The person(s) controlling the copyright in some of this material may not have granted the IETF Trust the right to allow modifications of such material outside the IETF Standards Process. Without obtaining an adequate license from the person(s) controlling the copyright in such materials, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process, except to format it for publication as an RFC or to translate it into languages other than English.
本文件可能包含2008年11月10日之前发布或公开的IETF文件或IETF贡献中的材料。控制某些材料版权的人员可能未授予IETF信托允许在IETF标准流程之外修改此类材料的权利。在未从控制此类材料版权的人员处获得充分许可的情况下,不得在IETF标准流程之外修改本文件,也不得在IETF标准流程之外创建其衍生作品,除了将其格式化以RFC形式发布或将其翻译成英语以外的其他语言。
Table of Contents
目录
1. Introduction ....................................................4 2. Purpose and Scope ...............................................4 3. Reporting Results ...............................................5 3.1. Overview of Metric Statistics ..............................5 3.2. Long-Term Reporting Considerations .........................6 4. Effect of POV on the Loss Metric ................................8 4.1. Loss Threshold .............................................8 4.1.1. Network Characterization ............................8 4.1.2. Application Performance ............................11 4.2. Errored Packet Designation ................................11 4.3. Causes of Lost Packets ....................................11 4.4. Summary for Loss ..........................................12 5. Effect of POV on the Delay Metric ..............................12 5.1. Treatment of Lost Packets .................................12 5.1.1. Application Performance ............................13 5.1.2. Network Characterization ...........................13 5.1.3. Delay Variation ....................................14 5.1.4. Reordering .........................................15 5.2. Preferred Statistics ......................................15 5.3. Summary for Delay .........................................16 6. Reporting Raw Capacity Metrics .................................16 6.1. Type-P Parameter ..........................................17 6.2. A priori Factors ..........................................17 6.3. IP-Layer Capacity .........................................17 6.4. IP-Layer Utilization ......................................18 6.5. IP-Layer Available Capacity ...............................18 6.6. Variability in Utilization and Available Capacity .........19 6.6.1. General Summary of Variability .....................19 7. Reporting Restricted Capacity Metrics ..........................20 7.1. Type-P Parameter and Type-C Parameter .....................21 7.2. A Priori Factors ..........................................21 7.3. Measurement Interval ......................................22 7.4. Bulk Transfer Capacity Reporting ..........................22 7.5. Variability in Bulk Transfer Capacity .....................23 8. Reporting on Test Streams and Sample Size ......................23 8.1. Test Stream Characteristics ...............................23 8.2. Sample Size ...............................................24 9. Security Considerations ........................................25 10. Acknowledgements ..............................................25 11. References ....................................................25 11.1. Normative References .....................................25 11.2. Informative References ...................................26
1. Introduction ....................................................4 2. Purpose and Scope ...............................................4 3. Reporting Results ...............................................5 3.1. Overview of Metric Statistics ..............................5 3.2. Long-Term Reporting Considerations .........................6 4. Effect of POV on the Loss Metric ................................8 4.1. Loss Threshold .............................................8 4.1.1. Network Characterization ............................8 4.1.2. Application Performance ............................11 4.2. Errored Packet Designation ................................11 4.3. Causes of Lost Packets ....................................11 4.4. Summary for Loss ..........................................12 5. Effect of POV on the Delay Metric ..............................12 5.1. Treatment of Lost Packets .................................12 5.1.1. Application Performance ............................13 5.1.2. Network Characterization ...........................13 5.1.3. Delay Variation ....................................14 5.1.4. Reordering .........................................15 5.2. Preferred Statistics ......................................15 5.3. Summary for Delay .........................................16 6. Reporting Raw Capacity Metrics .................................16 6.1. Type-P Parameter ..........................................17 6.2. A priori Factors ..........................................17 6.3. IP-Layer Capacity .........................................17 6.4. IP-Layer Utilization ......................................18 6.5. IP-Layer Available Capacity ...............................18 6.6. Variability in Utilization and Available Capacity .........19 6.6.1. General Summary of Variability .....................19 7. Reporting Restricted Capacity Metrics ..........................20 7.1. Type-P Parameter and Type-C Parameter .....................21 7.2. A Priori Factors ..........................................21 7.3. Measurement Interval ......................................22 7.4. Bulk Transfer Capacity Reporting ..........................22 7.5. Variability in Bulk Transfer Capacity .....................23 8. Reporting on Test Streams and Sample Size ......................23 8.1. Test Stream Characteristics ...............................23 8.2. Sample Size ...............................................24 9. Security Considerations ........................................25 10. Acknowledgements ..............................................25 11. References ....................................................25 11.1. Normative References .....................................25 11.2. Informative References ...................................26
When designing measurements of IP networks and presenting a result, knowledge of the audience is a key consideration. To present a useful and relevant portrait of network conditions, one must answer the following question:
在设计IP网络的测量并给出结果时,受众的知识是一个关键考虑因素。要呈现网络状况的有用且相关的描述,必须回答以下问题:
"How will the results be used?"
“结果将如何使用?”
There are two main audience categories for the report of results:
结果报告有两个主要受众类别:
1. Network Characterization - describes conditions in an IP network for quality assurance, troubleshooting, modeling, Service Level Agreements (SLAs), etc. This point of view (POV) looks inward toward the network where the report consumer intends their actions.
1. 网络特征描述—描述IP网络中的质量保证、故障排除、建模、服务级别协议(SLA)等条件。此观点(POV)着眼于报告使用者打算采取行动的网络。
2. Application Performance Estimation - describes the network conditions in a way that facilitates determining effects on user applications, and ultimately the users themselves. This POV looks outward, toward the user(s), accepting the network as is. This report consumer intends to estimate a network-dependent aspect of performance or design some aspect of an application's accommodation of the network. (These are *not* application metrics; they are defined at the IP layer.)
2. 应用程序性能评估-以便于确定对用户应用程序以及最终对用户自身的影响的方式描述网络状况。此POV向外看,面向用户,接受网络的现状。此报告使用者打算评估与网络相关的性能或设计应用程序适应网络的某些方面。(这些是*不是*应用程序度量;它们是在IP层定义的。)
This memo considers how these different POVs affect both the measurement design (parameters and options of the metrics) and statistics reported when serving the report consumer's needs.
本备忘录考虑了这些不同的POV如何影响测量设计(指标的参数和选项)以及在满足报告消费者需求时报告的统计数据。
The IP Performance Metrics (IPPM) Framework [RFC2330] and other RFCs describing IPPM provide a background for this memo.
IP性能指标(IPPM)框架[RFC2330]和其他描述IPPM的RFC为本备忘录提供了背景。
The purpose of this memo is to clearly delineate two POVs for using measurements and describe their effects on the test design, including the selection of metric parameters and reporting the results.
本备忘录的目的是明确描述两种使用测量的POV,并描述其对试验设计的影响,包括选择度量参数和报告结果。
The scope of this memo primarily covers the test design and reporting of the loss and delay metrics [RFC2680] [RFC2679]. It will also discuss the delay variation [RFC3393] and reordering metrics [RFC4737] where applicable.
本备忘录的范围主要包括损失和延迟度量[RFC2680][RFC2679]的测试设计和报告。在适用的情况下,还将讨论延迟变化[RFC3393]和重新排序度量[RFC4737]。
With capacity metrics growing in relevance to the industry, the memo also covers POV and reporting considerations for metrics resulting from the Bulk Transfer Capacity Framework [RFC3148] and Network Capacity Definitions [RFC5136]. These memos effectively describe two different categories of metrics:
随着容量指标与行业的相关性不断增强,备忘录还涵盖了批量传输容量框架[RFC3148]和网络容量定义[RFC5136]产生的指标的POV和报告注意事项。这些备忘录有效地描述了两类不同的指标:
o Restricted [RFC3148]: includes restrictions of congestion control and the notion of unique data bits delivered, and
o 受限[RFC3148]:包括拥塞控制的限制和传输的唯一数据位的概念,以及
o Raw [RFC5136]: uses a definition of raw capacity without the restrictions of data uniqueness or congestion awareness.
o 原始[RFC5136]:使用原始容量的定义,不受数据唯一性或拥塞感知的限制。
It might seem, at first glance, that each of these metrics has an obvious audience (raw = network characterization, restricted = application performance), but reality is more complex and consistent with the overall topic of capacity measurement and reporting. For example, TCP is usually used in restricted capacity measurement methods, while UDP appears in raw capacity measurement. The raw and restricted capacity metrics will be treated in separate sections, although they share one common reporting issue: representing variability in capacity metric results as part of a long-term report.
乍一看,这些指标似乎都有明显的受众(原始=网络特征,受限=应用程序性能),但现实更为复杂,与容量度量和报告的总体主题一致。例如,TCP通常用于受限容量测量方法,而UDP出现在原始容量测量中。原始容量指标和受限容量指标将在单独的章节中处理,尽管它们有一个共同的报告问题:将容量指标结果的可变性表示为长期报告的一部分。
Sampling, or the design of the active packet stream that is the basis for the measurements, is also discussed.
还讨论了作为测量基础的采样或活动分组流的设计。
This section gives an overview of recommendations, followed by additional considerations for reporting results in the "long term", based on the discussion and conclusions of the major sections that follow.
本节概述了各项建议,随后根据随后各主要章节的讨论和结论,提出了“长期”报告结果的其他考虑因素。
This section gives an overview of reporting recommendations for all the metrics considered in this memo.
本节概述了本备忘录中考虑的所有指标的报告建议。
The minimal report on measurements must include both loss and delay metrics.
测量的最小报告必须包括损失和延迟度量。
For packet loss, the loss ratio defined in [RFC2680] is a sufficient starting point -- especially the existing guidance for setting the loss threshold waiting time. In Section 4.1.1, we have calculated a waiting time -- 51 seconds -- that should be sufficient to differentiate between packets that are truly lost or have long finite delays under general measurement circumstances. Knowledge of
对于数据包丢失,[RFC2680]中定义的丢失率是一个足够的起点——特别是设置丢失阈值等待时间的现有指南。在第4.1.1节中,我们计算了一个等待时间(51秒),它应该足以区分在一般测量情况下真正丢失或具有长有限延迟的数据包。了解
specific conditions can help to reduce this threshold, and a waiting time of approximately 50 seconds is considered to be manageable in practice.
特定条件有助于降低该阈值,在实践中,大约50秒的等待时间被认为是可管理的。
We note that a loss ratio calculated according to [Y.1540] would exclude errored packets from the numerator. In practice, the difference between these two loss metrics is small, if any, depending on whether the last link prior to the Destination contributes errored packets.
我们注意到,根据[Y.1540]计算的丢失率将从分子中排除错误数据包。实际上,这两个丢失度量之间的差异很小(如果有的话),这取决于目的地之前的最后一个链路是否提供了错误数据包。
For packet delay, we recommend providing both the mean delay and the median delay with lost packets designated as undefined (as permitted by [RFC2679]). Both statistics are based on a conditional distribution, and the condition is packet arrival prior to a waiting time dT, where dT has been set to take maximum packet lifetimes into account, as discussed above for loss. Using a long dT helps to ensure that delay distributions are not truncated.
对于数据包延迟,我们建议提供指定为未定义的丢失数据包的平均延迟和中间延迟(如[RFC2679]所允许)。这两种统计数据都基于条件分布,条件是等待时间dT之前的数据包到达,其中dT被设置为考虑最大数据包寿命,如上所述的丢失。使用长dT有助于确保延迟分布不会被截断。
For Packet Delay Variation (PDV), the minimum delay of the conditional distribution should be used as the reference delay for computing PDV according to [Y.1540] or [RFC5481] and [RFC3393]. A useful value to report is a "pseudo" range of delay variation based on calculating the difference between a high percentile of delay and the minimum delay. For example, the 99.9th percentile minus the minimum will give a value that can be compared with objectives in [Y.1541].
对于数据包延迟变化(PDV),应根据[Y.1540]或[RFC5481]和[RFC3393]将条件分布的最小延迟用作计算PDV的参考延迟。报告的一个有用值是基于计算高百分比延迟和最小延迟之间的差值的延迟变化的“伪”范围。例如,99.9%减去最小值将给出一个可与[Y.1541]中的目标进行比较的值。
For both raw capacity and restricted capacity, reporting the variability in a useful way is identified as the main challenge. The min, max, and range statistics are suggested along with a ratio of max to min and moving averages. In the end, a simple plot of the singleton results over time may succeed where summary metrics fail or may serve to confirm that the summaries are valid.
对于原始容量和受限容量,以有用的方式报告可变性被确定为主要挑战。建议使用最小值、最大值和范围统计以及最大值与最小值的比率和移动平均值。最后,如果汇总指标失败,一个简单的单例结果随时间变化的图可能会成功,或者可以用来确认汇总是有效的。
[IPPM-RPT] describes methods to conduct measurements and report the results on a near-immediate time scale (10 seconds, which we consider to be "short-term").
[IPPM-RPT]描述了测量方法,并在近即刻的时间尺度上报告结果(10秒,我们认为是“短期的”)。
Measurement intervals and reporting intervals need not be the same length. Sometimes, the user is only concerned with the performance levels achieved over a relatively long interval of time (e.g., days, weeks, or months, as opposed to 10 seconds). However, there can be risks involved with running a measurement continuously over a long period without recording intermediate results:
测量间隔和报告间隔的长度不必相同。有时,用户只关心在相对较长的时间间隔内(例如,天、周或月,而不是10秒)达到的性能水平。但是,在不记录中间结果的情况下,长时间连续运行测量可能存在风险:
o Temporary power failure may cause loss of all results to date.
o 暂时断电可能导致迄今为止所有结果的丢失。
o Measurement system timing synchronization signals may experience a temporary outage, causing subsets of measurements to be in error or invalid.
o 测量系统定时同步信号可能会经历临时中断,导致测量子集出错或无效。
o Maintenance on the measurement system or on its connectivity to the network under test may be necessary.
o 可能需要对测量系统或其与被测网络的连接进行维护。
For these and other reasons, such as
由于这些和其他原因,例如
o the constraint to collect measurements on intervals similar to user session length,
o 在类似于用户会话长度的间隔上收集度量值的约束,
o the dual use of measurements in monitoring activities where results are needed on a period of a few minutes, or
o 监测活动中测量的双重用途,需要几分钟的结果,或
o the ability to inspect results of a single measurement interval for deeper analysis,
o 能够检查单个测量间隔的结果以进行更深入的分析,
there is value in conducting measurements on intervals that are much shorter than the reporting interval.
对比报告间隔短得多的间隔进行测量是有价值的。
There are several approaches for aggregating a series of measurement results over time in order to make a statement about the longer reporting interval. One approach requires the storage of all metric singletons collected throughout the reporting interval, even though the measurement interval stops and starts many times.
有几种方法可以汇总一系列随时间变化的测量结果,以说明更长的报告间隔。一种方法要求存储在整个报告间隔期间收集的所有公制单件,即使测量间隔多次停止和开始。
Another approach is described in [RFC5835] as "temporal aggregation". This approach would estimate the results for the reporting interval based on combining many individual short-term measurement interval statistics to yield a long-term result. The result would ideally appear in the same form as though a continuous measurement had been conducted. A memo addressing the details of temporal aggregation is yet to be prepared.
[RFC5835]中将另一种方法描述为“时间聚合”。这种方法将基于结合许多单独的短期测量间隔统计数据来估计报告间隔的结果,以产生长期结果。理想情况下,结果将以相同的形式出现,就像进行了连续测量一样。关于时间聚合细节的备忘录尚待编制。
Yet another approach requires a numerical objective for the metric, and the results of each measurement interval are compared with the objective. Every measurement interval where the results meet the objective contribute to the fraction of time with performance as specified. When the reporting interval contains many measurement intervals, it is possible to present the results as "metric A was less than or equal to objective X during Y% of time".
还有一种方法需要度量的数字目标,并且将每个测量间隔的结果与目标进行比较。结果符合目标的每一个测量间隔都有助于达到规定性能的时间分数。当报告间隔包含多个测量间隔时,可以将结果表示为“度量A在Y%的时间内小于或等于目标X”。
NOTE that numerical thresholds of acceptability are not set in IETF performance work and are therefore excluded from the scope of this memo.
请注意,IETF绩效工作中未设定可接受性的数值阈值,因此不在本备忘录的范围内。
In all measurements, it is important to avoid unintended synchronization with network events. This topic is treated in [RFC2330] for Poisson-distributed inter-packet time streams and in [RFC3432] for Periodic streams. Both avoid synchronization by using random start times.
在所有测量中,重要的是避免与网络事件的意外同步。[RFC2330]中针对泊松分布的包间时间流和[RFC3432]中针对周期流处理了此主题。两者都通过使用随机启动时间来避免同步。
There are network conditions where it is simply more useful to report the connectivity status of the Source-Destination path, and to distinguish time intervals where connectivity can be demonstrated from other time intervals (where connectivity does not appear to exist). [RFC2678] specifies a number of one-way and two-way connectivity metrics of increasing complexity. In this memo, we recommend that long-term reporting of loss, delay, and other metrics be limited to time intervals where connectivity can be demonstrated, and that other intervals be summarized as the percent of time where connectivity does not appear to exist. We note that this same approach has been adopted in ITU-T Recommendation [Y.1540] where performance parameters are only valid during periods of service "availability" (evaluated according to a function based on packet loss, and sustained periods of loss ratio greater than a threshold are declared "unavailable").
在某些网络条件下,报告源-目标路径的连接状态更为有用,并且可以区分可显示连接的时间间隔和其他时间间隔(连接似乎不存在)。[RFC2678]指定了许多日益复杂的单向和双向连接度量。在本备忘录中,我们建议将损失、延迟和其他指标的长期报告限制在可证明连通性的时间间隔内,并将其他时间间隔总结为不存在连通性的时间百分比。我们注意到,ITU-T建议[Y.1540]中采用了相同的方法,其中性能参数仅在服务“可用性”期间有效(根据基于数据包丢失的函数进行评估,并且持续的丢失率大于阈值的时段被宣布为“不可用”)。
This section describes the ways in which the loss metric can be tuned to reflect the preferences of the two audience categories, or different POVs. The waiting time before declaring that a packet is lost -- the loss threshold -- is one area where there would appear to be a difference, but the ability to post-process the results may resolve it.
本节介绍如何调整损失指标,以反映两个观众类别或不同POV的偏好。声明数据包丢失之前的等待时间(丢失阈值)是一个可能存在差异的方面,但后处理结果的能力可能会解决这个问题。
RFC 2680 [RFC2680] defines the concept of a waiting time for packets to arrive, beyond which they are declared lost. The text of the RFC declines to recommend a value, instead saying that "good engineering, including an understanding of packet lifetimes, will be needed in practice". Later, in the methodology, they give reasons for waiting "a reasonable period of time" and leave the definition of "reasonable" intentionally vague. Below, we estimate a practical bound on waiting time.
RFC 2680[RFC2680]定义了数据包到达的等待时间的概念,超过该时间,数据包将被声明丢失。RFC的文本拒绝推荐一个值,而是说“在实践中需要良好的工程,包括对数据包寿命的理解”。后来,在方法论中,他们给出了等待“一段合理时间”的理由,故意使“合理”的定义含糊不清。下面,我们估计了等待时间的实际界限。
Practical measurement experience has shown that unusual network circumstances can cause long delays. One such circumstance is when routing loops form during IGP re-convergence following a failure or drastic link cost change. Packets will loop between two routers
实际测量经验表明,异常网络环境可能导致长时间延迟。其中一种情况是,在IGP重新收敛过程中,在发生故障或链路成本急剧变化后形成路由环路。数据包将在两个路由器之间循环
until new routes are installed or until the IPv4 Time-to-Live (TTL) field (or the IPv6 Hop Limit) decrements to zero. Very long delays on the order of several seconds have been measured [Casner] [Cia03].
直到安装新路由或直到IPv4生存时间(TTL)字段(或IPv6跃点限制)减为零。已测量到几秒钟左右的超长延迟[Casner][Cia03]。
Therefore, network characterization activities prefer a long waiting time in order to distinguish these events from other causes of loss (such as packet discard at a full queue, or tail drop). This way, the metric design helps to distinguish more reliably between packets that might yet arrive and those that are no longer traversing the network.
因此,网络表征活动更喜欢长时间的等待,以便将这些事件与其他丢失原因(如满队列中的数据包丢弃或尾部丢弃)区分开来。这样,度量设计有助于更可靠地区分可能到达的数据包和不再穿越网络的数据包。
It is possible to calculate a worst-case waiting time, assuming that a routing loop is the cause. We model the path between Source and Destination as a series of delays in links (t) and queues (q), as these are the dominant contributors to delay (in active measurement, the Source and Destination hosts contribute minimal delay). The normal path delay, D, across n queues (where TTL is decremented at a node with a queue) and n+1 links without encountering a loop, is
假设路由循环是原因,可以计算最坏情况下的等待时间。我们将源和目标之间的路径建模为链路(t)和队列(q)中的一系列延迟,因为它们是延迟的主要贡献者(在主动测量中,源和目标主机贡献最小的延迟)。在n个队列(其中TTL在具有队列的节点处减小)和n+1个链路之间的正常路径延迟D(不遇到循环)为
Path model with n=5 Source --- q1 --- q2 --- q3 --- q4 --- q5 --- Destination t0 t1 t2 t3 t4 t5
Path model with n=5 Source --- q1 --- q2 --- q3 --- q4 --- q5 --- Destination t0 t1 t2 t3 t4 t5
n --- \ D = t + > (t + q) 0 / i i --- i = 1
n --- \ D = t + > (t + q) 0 / i i --- i = 1
Figure 1: Normal Path Delay
图1:正常路径延迟
and the time spent in the loop with L queues is
使用L个队列在循环中花费的时间是
Path model with n=5 and L=3 Time in one loop = (qx+tx + qy+ty + qz+tz)
Path model with n=5 and L=3 Time in one loop = (qx+tx + qy+ty + qz+tz)
qy -- qz | ?/exit? qx--/\ Src --- q1 --- q2 ---/ q3 --- q4 --- q5 --- Dst t0 t1 t2 t3 t4 t5
qy -- qz | ?/exit? qx--/\ Src --- q1 --- q2 ---/ q3 --- q4 --- q5 --- Dst t0 t1 t2 t3 t4 t5
j + L-1 --- \ (TTL - n) R = C > (t + q) where C = --------- / i i max L --- i=j
j + L-1 --- \ (TTL - n) R = C > (t + q) where C = --------- / i i max L --- i=j
Figure 2: Delay Due to Rotations in a Loop
图2:循环中旋转引起的延迟
where n is the total number of queues in the non-loop path (with n+1 links), j is the queue number where the loop begins, C is the number of times a packet circles the loop, and TTL is the packet's initial Time-to-Live value at the Source (or Hop Count in IPv6).
其中,n是非循环路径(具有n+1个链路)中的队列总数,j是循环开始的队列编号,C是数据包在循环中循环的次数,TTL是数据包在源处的初始生存时间值(或IPv6中的跃点计数)。
If we take the delays of all links and queues as 100 ms each, the TTL=255, the number of queues n=5, and the queues in the loop L=4, then using C_max:
如果我们将所有链路和队列的延迟分别设为100 ms,TTL=255,队列数n=5,循环中的队列数L=4,则使用C_max:
D = 1.1 seconds and R ~= 50 seconds, and D + R ~= 51.1 seconds
D = 1.1 seconds and R ~= 50 seconds, and D + R ~= 51.1 seconds
We note that the link delays of 100 ms would span most continents, and a constant queue length of 100 ms is also very generous. When a loop occurs, it is almost certain to be resolved in 10 seconds or less. The value calculated above is an upper limit for almost any real-world circumstance.
我们注意到100毫秒的链路延迟将跨越大多数大陆,100毫秒的恒定队列长度也是非常慷慨的。当循环发生时,几乎可以肯定会在10秒或更短的时间内解决。上面计算的值几乎是任何实际情况下的上限。
A waiting time threshold parameter, dT, set consistent with this calculation, would not truncate the delay distribution (possibly causing a change in its mathematical properties), because the packets that might arrive have been given sufficient time to traverse the network.
与此计算一致的等待时间阈值参数dT不会截断延迟分布(可能导致其数学特性发生变化),因为可能到达的数据包已经有足够的时间穿越网络。
It is worth noting that packets that are stored and deliberately forwarded at a much later time constitute a replay attack on the measurement system and are beyond the scope of normal performance reporting.
值得注意的是,稍后存储并故意转发的数据包构成了对测量系统的重播攻击,超出了正常性能报告的范围。
Fortunately, application performance estimation activities are not adversely affected by the long estimated limit on waiting time, because most applications will use shorter time thresholds. Although the designer's tendency might be to set the loss threshold at a value equivalent to a particular application's threshold, this specific threshold can be applied when post-processing the measurements. A shorter waiting time can be enforced by locating packets with delays longer than the application's threshold and re-designating such packets as lost. Thus, the measurement system can use a single loss waiting time and support both application and network performance POVs simultaneously.
幸运的是,应用程序性能估计活动不会受到等待时间长估计限制的不利影响,因为大多数应用程序将使用更短的时间阈值。尽管设计者倾向于将损失阈值设置为与特定应用程序的阈值相等的值,但在对测量值进行后处理时,可以应用此特定阈值。通过定位延迟大于应用程序阈值的数据包并将这些数据包重新指定为丢失数据包,可以缩短等待时间。因此,测量系统可以使用单个损耗等待时间,同时支持应用程序和网络性能POV。
RFC 2680 designates packets that arrive containing errors as lost packets. Many packets that are corrupted by bit errors are discarded within the network and do not reach their intended destination.
RFC2680将到达的包含错误的数据包指定为丢失的数据包。许多因位错误而损坏的数据包在网络中被丢弃,无法到达其预期目的地。
This is consistent with applications that would check the payload integrity at higher layers and discard the packet. However, some applications prefer to deal with errored payloads on their own, and even a corrupted payload is better than no packet at all.
这与在更高层检查有效负载完整性并丢弃数据包的应用程序是一致的。然而,一些应用程序更喜欢自己处理出错的有效负载,甚至损坏的有效负载也比根本没有数据包要好。
To address this possibility, and to make network characterization more complete, distinguishing between packets that do not arrive (lost) and errored packets that arrive (conditionally lost) is recommended.
为了解决这种可能性,并使网络特征更完整,建议区分未到达(丢失)的数据包和到达(有条件丢失)的错误数据包。
Although many measurement systems use a waiting time to determine whether or not a packet is lost, most of the waiting is in vain. The packets are no longer traversing the network and have not reached their destination.
尽管许多测量系统使用等待时间来确定数据包是否丢失,但大多数等待都是徒劳的。数据包不再穿越网络,也没有到达目的地。
There are many causes of packet loss, including the following:
造成数据包丢失的原因有很多,包括:
1. Queue drop, or discard
1. 队列丢弃或丢弃
2. Corruption of the IP header, or other essential header information
2. IP报头或其他基本报头信息损坏
3. TTL expiration (or use of a TTL value that is too small)
3. TTL过期(或使用太小的TTL值)
4. Link or router failure
4. 链路或路由器故障
5. Layers below the Source-to-Destination IP layer can discard packets that fail error checking, and link-layer checksums often cover the entire packet
5. 源到目标IP层下面的层可以丢弃错误检查失败的数据包,链路层校验和通常覆盖整个数据包
It is reasonable to consider a packet that has not arrived after a large amount of time to be lost (due to one of the causes above) because packets do not "live forever" in the network or have infinite delay.
合理地考虑在丢失大量时间之后(由于以上原因之一)而没有到达的分组,因为分组不在网络中“永远存在”或具有无限延迟。
Given that measurement post-processing is possible (even encouraged in the definitions of IPPM), measurements of loss can easily serve both POVs:
考虑到测量后处理是可能的(甚至在IPPM的定义中鼓励),损失测量可以很容易地为两种POV服务:
o Use a long waiting time to serve network characterization and revise results for specific application delay thresholds as needed.
o 使用较长的等待时间来服务于网络表征,并根据需要修改特定应用程序延迟阈值的结果。
o Distinguish between errored packets and lost packets when possible to aid network characterization, and combine the results for application performance if appropriate.
o 在可能的情况下,区分错误数据包和丢失数据包,以帮助确定网络特征,并在适当的情况下结合应用程序性能的结果。
This section describes the ways in which the delay metric can be tuned to reflect the preferences of the two consumer categories, or different POVs.
本节介绍如何调整延迟度量以反映两个消费者类别或不同POV的偏好。
The delay metric [RFC2679] specifies the treatment of packets that do not successfully traverse the network: their delay is undefined.
延迟度量[RFC2679]指定对未成功穿越网络的数据包的处理:它们的延迟未定义。
>>The *Type-P-One-way-Delay* from Src to Dst at T is undefined (informally, infinite)<< means that Src sent the first bit of a Type-P packet to Dst at wire-time T and that Dst did not receive that packet.
>>从Src到T处Dst的*Type-P-单向延迟*未定义(非正式地说,无限)<<意味着Src在接线时间T向Dst发送了Type-P数据包的第一位,而Dst没有收到该数据包。
It is an accepted but informal practice to assign infinite delay to lost packets. We next look at how these two different treatments align with the needs of measurement consumers who wish to characterize networks or estimate application performance. Also, we look at the way that lost packets have been treated in other metrics: delay variation and reordering.
将无限延迟分配给丢失的数据包是一种公认但非正式的做法。接下来,我们将研究这两种不同的处理方法如何与希望描述网络特征或评估应用程序性能的测量消费者的需求相一致。此外,我们还研究了在其他指标中处理丢失数据包的方式:延迟变化和重新排序。
Applications need to perform different functions, dependent on whether or not each packet arrives within some finite tolerance. In other words, a receiver's packet processing takes only one of two alternative directions (a "fork" in the road):
应用程序需要执行不同的功能,这取决于每个数据包是否在某个有限容差内到达。换句话说,接收方的数据包处理只采取两个可选方向之一(道路上的“岔路口”):
o Packets that arrive within expected tolerance are handled by removing headers, restoring smooth delivery timing (as in a de-jitter buffer), restoring sending order, checking for errors in payloads, and many other operations.
o 到达预期容差范围内的数据包通过移除报头、恢复平滑的传递时间(如在去抖动缓冲区中)、恢复发送顺序、检查有效负载中的错误以及许多其他操作来处理。
o Packets that do not arrive when expected lead to attempted recovery from the apparent loss, such as retransmission requests, loss concealment, or forward error correction to replace the missing packet.
o 未按预期到达的数据包会导致尝试从明显丢失中恢复,例如重传请求、丢失隐藏或前向纠错以替换丢失的数据包。
So, it is important to maintain a distinction between packets that actually arrive and those that do not. Therefore, it is preferable to leave the delay of lost packets undefined and to characterize the delay distribution as a conditional distribution (conditioned on arrival).
因此,在实际到达的数据包和未到达的数据包之间保持区别是很重要的。因此,优选不定义丢失分组的延迟,并且将延迟分布表征为条件分布(在到达时条件化)。
In this discussion, we assume that both loss and delay metrics will be reported for network characterization (at least).
在本讨论中,我们假设损失和延迟度量都将被报告用于网络特性描述(至少)。
Assume that packets that do not arrive are reported as lost, usually as a fraction of all sent packets. If these lost packets are assigned an undefined delay, then the network's inability to deliver them (in a timely way) is relegated only in the loss metric when we report statistics on the delay distribution conditioned on the event of packet arrival (within the loss waiting time threshold). We can say that the delay and loss metrics are orthogonal in that they convey non-overlapping information about the network under test. This is a valuable property whose absence is discussed below.
假设未到达的数据包被报告为丢失,通常是所有已发送数据包的一小部分。如果这些丢失的数据包被分配了一个未定义的延迟,那么当我们报告以数据包到达事件(在丢失等待时间阈值内)为条件的延迟分布统计数据时,网络无法(及时地)交付这些数据包仅在损失度量中降级。我们可以说延迟和损失度量是正交的,因为它们传递的是关于被测网络的非重叠信息。这是一项有价值的财产,其缺失将在下文讨论。
However, if we assign infinite delay to all lost packets, then
然而,如果我们将无限延迟分配给所有丢失的数据包,那么
o The delay metric results are influenced both by packets that arrive and those that do not.
o 延迟度量结果受到达的数据包和未到达的数据包的影响。
o The delay singleton and the loss singleton do not appear to be orthogonal (delay is finite when loss=0; delay is infinite when loss=1).
o 延迟单态和损失单态似乎不是正交的(当损失=0时,延迟是有限的;当损失=1时,延迟是无限的)。
o The network is penalized in both the loss and delay metrics, effectively double-counting the lost packets.
o 网络在丢失和延迟度量方面都受到惩罚,有效地重复计算丢失的数据包。
As further evidence of overlap, consider the Cumulative Distribution Function (CDF) of delay when the value "positive infinity" is assigned to all lost packets. Figure 3 shows a CDF where a small fraction of packets are lost.
作为重叠的进一步证据,考虑当“正无穷大”的值被分配给所有丢失的分组时延迟的累积分布函数(CDF)。图3显示了一个CDF,其中一小部分数据包丢失。
1 | - - - - - - - - - - - - - - - - - -+ | | | _..----'''''''''''''''''''' | ,-'' | ,' | / Mass at | / +infinity | / = fraction || lost |/ 0 |_____________________________________
1 | - - - - - - - - - - - - - - - - - -+ | | | _..----'''''''''''''''''''' | ,-'' | ,' | / Mass at | / +infinity | / = fraction || lost |/ 0 |_____________________________________
0 Delay +o0
0延迟+o0
Figure 3: Cumulative Distribution Function for Delay When Loss = +Infinity
图3:损失=+无穷大时延迟的累积分布函数
We note that a delay CDF that is conditioned on packet arrival would not exhibit this apparent overlap with loss.
我们注意到,以数据包到达为条件的延迟CDF不会表现出与丢失的这种明显重叠。
Although infinity is a familiar mathematical concept, it is somewhat disconcerting to see any time-related metric reported as infinity. Questions are bound to arise and tend to detract from the goal of informing the consumer with a performance report.
尽管无穷大是一个大家熟悉的数学概念,但看到任何与时间相关的度量被报告为无穷大还是有点令人不安。问题不可避免地会出现,并且倾向于偏离向消费者提供绩效报告的目标。
[RFC3393] excludes lost packets from samples, effectively assigning an undefined delay to packets that do not arrive in a reasonable time. Section 4.1 of [RFC3393] describes this specification and its rationale (ipdv = inter-packet delay variation in the quote below).
[RFC3393]从样本中排除丢失的数据包,有效地为未在合理时间内到达的数据包分配未定义的延迟。[RFC3393]第4.1节描述了本规范及其基本原理(ipdv=以下引用中的包间延迟变化)。
The treatment of lost packets as having "infinite" or "undefined" delay complicates the derivation of statistics for ipdv. Specifically, when packets in the measurement sequence are lost, simple statistics such as sample mean cannot be computed. One possible approach to handling this problem is to reduce the event space by conditioning. That is, we consider conditional statistics; namely we estimate the mean ipdv (or other derivative statistic) conditioned on the event that selected packet pairs
将丢失的数据包视为具有“无限”或“未定义”延迟,使ipdv统计数据的推导变得复杂。具体地说,当测量序列中的数据包丢失时,不能计算简单的统计数据,例如样本平均值。处理此问题的一种可能方法是通过调节减少事件空间。也就是说,我们考虑条件统计;也就是说,我们估计平均ipdv(或其他导数统计),条件是所选数据包对发生的事件
arrive at the Destination (within the given timeout). While this itself is not without problems (what happens, for example, when every other packet is lost), it offers a way to make some (valid) statements about ipdv, at the same time avoiding events with undefined outcomes.
到达目的地(在给定的超时时间内)。虽然这本身并非没有问题(例如,当其他数据包丢失时会发生什么情况),但它提供了一种方法,可以对ipdv做出一些(有效)声明,同时避免发生结果未定义的事件。
We note that the argument above applies to all forms of packet delay variation that can be constructed using the "selection function" concept of [RFC3393]. In recent work, the two main forms of delay variation metrics have been compared, and the results are summarized in [RFC5481].
我们注意到,上述参数适用于所有形式的数据包延迟变化,这些变化可以使用[RFC3393]的“选择函数”概念构造。在最近的工作中,比较了延迟变化度量的两种主要形式,结果总结在[RFC5481]中。
[RFC4737] defines metrics that are based on evaluation of packet arrival order and that include a waiting time before declaring that a packet is lost (to exclude the packet from further processing).
[RFC4737]定义了基于数据包到达顺序评估的指标,包括在宣布数据包丢失之前的等待时间(以将数据包排除在进一步处理之外)。
If packets are assigned a delay value, then the reordering metric would declare any packets with infinite delay to be reordered, because their sequence numbers will surely be less than the "Next Expected" threshold when (or if) they arrive. But this practice would fail to maintain orthogonality between the reordering metric and the loss metric. Confusion can be avoided by designating the delay of non-arriving packets as undefined and reserving delay values only for packets that arrive within a sufficiently long waiting time.
如果为数据包分配了一个延迟值,那么重新排序度量将宣布任何具有无限延迟的数据包被重新排序,因为它们到达时(或如果到达)的序列号肯定会小于“下一个预期”阈值。但这种做法将无法保持重新排序度量和损失度量之间的正交性。通过将未到达数据包的延迟指定为未定义,并仅为在足够长的等待时间内到达的数据包保留延迟值,可以避免混淆。
Today in network characterization, the sample mean is one statistic that is almost ubiquitously reported. It is easily computed and understood by virtually everyone in this audience category. Also, the sample is usually filtered on packet arrival, so that the mean is based on a conditional distribution.
今天,在网络表征中,样本平均数是一个几乎被普遍报道的统计数据。它很容易计算和理解几乎每个人在这一类观众。此外,样本通常在数据包到达时过滤,因此平均值基于条件分布。
The median is another statistic that summarizes a distribution, having somewhat different properties from the sample mean. The median is stable in distributions with a few outliers or without them. However, the median's stability prevents it from indicating when a large fraction of the distribution changes value. 50% or more values would need to change for the median to capture the change.
中位数是另一个统计数据,它总结了一个分布,与样本平均数有一些不同的特性。中位数在有少量离群值或没有离群值的分布中是稳定的。但是,中位数的稳定性使其无法指示分布的很大一部分何时改变值。需要更改50%或更多的值,中位数才能捕获更改。
Both the median and sample mean have difficulty with bimodal distributions. The median will reside in only one of the modes, and the mean may not lie in either mode range. For this and other reasons, additional statistics such as the minimum, maximum, and 95th percentile have value when summarizing a distribution.
中位数和样本均值都难以处理双峰分布。中值将仅存在于其中一个模式中,且平均值可能不在任一模式范围内。出于这一原因和其他原因,在汇总分布时,其他统计信息(如最小值、最大值和第95百分位)具有价值。
When both the sample mean and median are available, a comparison will sometimes be informative, because these two statistics are equal only under unusual circumstances, such as when the delay distribution is perfectly symmetrical.
当样本均值和中位数都可用时,比较有时会提供信息,因为这两个统计数据只有在异常情况下才相等,例如延迟分布完全对称时。
Also, these statistics are generally useful from the application performance POV, so there is a common set that should satisfy audiences.
此外,从应用程序性能的角度来看,这些统计数据通常是有用的,因此有一组通用的数据应该能够满足受众的需求。
Plots of the delay distribution may also be useful when single-value statistics indicate that new conditions are present. An empirically derived probability distribution function will usually describe multiple modes more efficiently than any other form of result.
当单值统计数据表明存在新情况时,延迟分布图也可能有用。经验推导的概率分布函数通常比任何其他形式的结果更有效地描述多种模式。
From the perspectives of
从
1. application/receiver analysis, where subsequent processing depends on whether the packet arrives or times out,
1. 应用程序/接收器分析,其中后续处理取决于数据包是到达还是超时,
2. straightforward network characterization without double-counting defects, and
2. 无重复计数缺陷的直接网络表征,以及
3. consistency with delay variation and reordering metric definitions,
3. 与延迟变化和重新排序度量定义的一致性,
the most efficient practice is to distinguish between packets that are truly lost and those that are delayed packets with a sufficiently long waiting time, and to designate the delay of non-arriving packets as undefined.
最有效的做法是区分真正丢失的数据包和等待时间足够长的延迟数据包,并将未到达数据包的延迟指定为未定义。
Raw capacity refers to the metrics defined in [RFC5136], which do not include restrictions such as data uniqueness or flow-control response to congestion.
原始容量是指[RFC5136]中定义的指标,不包括数据唯一性或流量控制对拥塞的响应等限制。
The metrics considered are IP-layer capacity, utilization (or used capacity), and available capacity, for individual links and complete paths. These three metrics form a triad: knowing one metric constrains the other two (within their allowed range), and knowing two determines the third. The link metrics have another key aspect in common: they are single-measurement-point metrics at the egress of a link. The path capacity and available capacity are derived by examining the set of single-point link measurements and taking the minimum value.
考虑的指标是单个链路和完整路径的IP层容量、利用率(或已用容量)和可用容量。这三个指标构成了一个三元组:知道一个指标会约束另外两个指标(在允许的范围内),知道两个指标会决定第三个指标。链路度量有另一个共同的关键方面:它们是链路出口处的单测量点度量。路径容量和可用容量是通过检查单点链路测量集并取最小值得出的。
The concept of "packets of Type-P" is defined in [RFC2330]. The Type-P categorization has critical relevance in all forms of capacity measurement and reporting. The ability to categorize packets based on header fields for assignment to different queues and scheduling mechanisms is now commonplace. When unused resources are shared across queues, the conditions in all packet categories will affect capacity and related measurements. This is one source of variability in the results that all audiences would prefer to see reported in a useful and easily understood way.
[RFC2330]中定义了“P型数据包”的概念。P型分类在所有形式的能力衡量和报告中都具有关键相关性。现在,基于头字段对数据包进行分类以分配给不同队列和调度机制的能力已司空见惯。当跨队列共享未使用的资源时,所有数据包类别中的条件都将影响容量和相关度量。这是所有受众都希望看到以有用且易于理解的方式报告的结果的可变性的来源之一。
Communication of Type-P within the One-Way Active Measurement Protocol (OWAMP) and the Two-Way Active Measurement Protocol (TWAMP) is essentially confined to the Diffserv Code Point (DSCP) [RFC4656]. DSCP is the most common qualifier for Type-P.
单向主动测量协议(OWAMP)和双向主动测量协议(TWAMP)内的P型通信基本上限于区分服务码点(DSCP)[RFC4656]。DSCP是最常见的P型限定符。
Each audience will have a set of Type-P qualifications and value combinations that are of interest. Measurements and reports should have the flexibility to report per-type and aggregate performance.
每位观众都将拥有一套感兴趣的P类资格和价值组合。度量和报告应具有报告每种类型和聚合性能的灵活性。
The audience for network characterization may have detailed information about each link that comprises a complete path (due to ownership, for example), or some of the links in the path but not others, or none of the links.
网络特征化的受众可能具有关于包含完整路径(例如,由于所有权)的每个链路的详细信息,或者路径中的一些链路而不是其他链路,或者没有任何链路。
There are cases where the measurement audience only has information on one of the links (the local access link) and wishes to measure one or more of the raw capacity metrics. This scenario is quite common and has spawned a substantial number of experimental measurement methods (e.g., http://www.caida.org/tools/taxonomy/). Many of these methods respect that their users want a result fairly quickly and in one trial. Thus, the measurement interval is kept short (a few seconds to a minute). For long-term reporting, a sample of short-term results needs to be summarized.
在某些情况下,测量受众仅拥有其中一个链路(本地接入链路)的信息,并希望测量一个或多个原始容量度量。这种情况非常普遍,并产生了大量的实验测量方法(例如。,http://www.caida.org/tools/taxonomy/). 这些方法中的许多都尊重用户希望在一次试验中获得相当快的结果。因此,测量间隔保持较短(几秒到一分钟)。对于长期报告,需要总结短期结果样本。
For links, this metric's theoretical maximum value can be determined from the physical-layer bit rate and the bit rate reduction due to the layers between the physical layer and IP. When measured, this metric takes additional factors into account, such as the ability of the sending device to process and forward traffic under various conditions. For example, the arrival of routing updates may spawn high-priority processes that reduce the sending rate temporarily.
对于链路,该度量的理论最大值可以通过物理层比特率和由于物理层和IP之间的层而导致的比特率降低来确定。在测量时,该指标考虑了其他因素,例如发送设备在各种条件下处理和转发流量的能力。例如,路由更新的到来可能会产生暂时降低发送速率的高优先级进程。
Thus, the measured capacity of a link will be variable, and the maximum capacity observed applies to a specific time, time interval, and other relevant circumstances.
因此,链路的测量容量是可变的,观测到的最大容量适用于特定时间、时间间隔和其他相关情况。
For paths composed of a series of links, it is easy to see how the sources of variability for the results grow with each link in the path. Variability of results will be discussed in more detail below.
对于由一系列链接组成的路径,很容易看到结果的可变性源如何随着路径中的每个链接而增长。下文将更详细地讨论结果的可变性。
The ideal metric definition of link utilization [RFC5136] is based on the actual usage (bits successfully received during a time interval) and the maximum capacity for the same interval.
链路利用率[RFC5136]的理想度量定义基于实际使用率(在一个时间间隔内成功接收的位)和同一时间间隔的最大容量。
In practice, link utilization can be calculated by counting the IP-layer (or other layer) octets received over a time interval and dividing by the theoretical maximum number of octets that could have been delivered in the same interval. A commonly used time interval is 5 minutes, and this interval has been sufficient to support network operations and design for some time. 5 minutes is somewhat long compared with the expected download time for web pages but short with respect to large file transfers and TV program viewing. It is fair to say that considerable variability is concealed by reporting a single (average) utilization value for each 5-minute interval. Some performance management systems have begun to make 1-minute averages available.
实际上,链路利用率可以通过计算在一个时间间隔内接收的IP层(或其他层)八位字节数,并除以在同一时间间隔内可能已发送的理论最大八位字节数来计算。通常使用的时间间隔为5分钟,该时间间隔足以支持网络操作和设计一段时间。与网页的预期下载时间相比,5分钟有点长,但对于大型文件传输和电视节目观看而言,5分钟的时间较短。可以公平地说,通过报告每5分钟间隔的单个(平均)利用率值,隐藏了相当大的可变性。一些绩效管理系统已开始提供1分钟平均值。
There is also a limit on the smallest useful measurement interval. Intervals on the order of the serialization time for a single Maximum Transmission Unit (MTU) packet will observe on/off behavior and report 100% or 0%. The smallest interval needs to be some multiple of MTU serialization time for averaging to be effective.
最小有效测量间隔也有限制。单个最大传输单元(MTU)数据包的序列化时间顺序上的间隔将观察开/关行为,并报告100%或0%。最小间隔需要是MTU序列化时间的几倍,以使平均有效。
The available capacity of a link can be calculated using the capacity and utilization metrics.
可以使用容量和利用率指标计算链路的可用容量。
When available capacity of a link or path is estimated through some measurement technique, the following parameters should be reported:
当通过某种测量技术估计链路或路径的可用容量时,应报告以下参数:
o Name and reference to the exact method of measurement
o 准确测量方法的名称和参考
o IP packet length, octets (including IP header)
o IP数据包长度,八位字节(包括IP报头)
o Maximum capacity that can be assessed in the measurement configuration
o 可在测量配置中评估的最大容量
o Time duration of the measurement
o 测量的持续时间
o All other parameters specific to the measurement method
o 特定于测量方法的所有其他参数
Many methods of available capacity measurement have a maximum capacity that they can measure, and this maximum may be less than the actual available capacity of the link or path. Therefore, it is important to know the capacity value beyond which there will be no measured improvement.
许多可用容量测量方法都有它们可以测量的最大容量,并且该最大容量可能小于链路或路径的实际可用容量。因此,了解容量值非常重要,超过该值将不会有可测量的改进。
The application performance estimation audience may have a desired target capacity value and simply wish to assess whether there is sufficient available capacity. This case simplifies the measurement of link and path capacity to some degree, as long as the measurable maximum exceeds the target capacity.
应用程序性能评估受众可能有一个期望的目标容量值,只是希望评估是否有足够的可用容量。这种情况在一定程度上简化了链路和路径容量的测量,只要可测量的最大容量超过目标容量。
As with most metrics and measurements, assessing the consistency or variability in the results gives the user an intuitive feel for the degree (or confidence) that any one value is representative of other results, or the spread of the underlying distribution of the singleton measurements.
与大多数度量和测量一样,评估结果的一致性或可变性可以让用户直观地感受到任何一个值代表其他结果的程度(或置信度),或者单个测量值的潜在分布范围。
How can utilization be measured and summarized to describe the potential variability in a useful way?
如何测量和总结利用率,以有用的方式描述潜在的可变性?
How can the variability in available capacity estimates be reported, so that the confidence in the results is also conveyed?
如何报告可用容量估算的可变性,从而传达对结果的信心?
We suggest some methods below.
我们建议以下几种方法。
With a set of singleton utilization or available capacity estimates, each representing a time interval needed to ascertain the estimate, we seek to describe the variation over the set of singletons as though reporting summary statistics of a distribution. Three useful summary statistics are
通过一组单例利用率或可用容量估计值(每一个都代表确定估计值所需的时间间隔),我们试图描述单例利用率或可用容量的变化,就像报告分布的汇总统计数据一样。以下是三个有用的汇总统计信息:
o Minimum,
o 最低限度
o Maximum, and
o 最大值,以及
o Range
o 范围
An alternate way to represent the range is as a ratio of maximum to minimum value. This enables an easily understandable statistic to describe the range observed. For example, when maximum = 3*minimum, then the max/min ratio is 3, and users may see variability of this order. On the other hand, capacity estimates with a max/min ratio near 1 are quite consistent and near the central measure or statistic reported.
表示范围的另一种方法是最大值与最小值的比率。这使得一个易于理解的统计数据能够描述观察到的范围。例如,当最大值=3*最小值时,最大值/最小值比率为3,用户可能会看到此顺序的变化。另一方面,最大/最小比率接近1的容量估计值非常一致,接近报告的中心测量值或统计值。
For an ongoing series of singleton estimates, a moving average of n estimates may provide a single value estimate to more easily distinguish substantial changes in performance over time. For example, in a window of n singletons observed in time interval t, a percentage change of x% is declared to be a substantial change and reported as an exception.
对于正在进行的一系列单例估计,n个估计的移动平均值可以提供一个单值估计值,以便更容易区分性能随时间的实质性变化。例如,在时间间隔t内观察到的n个单例窗口中,x%的百分比变化被宣布为实质性变化,并作为例外情况报告。
Often, the most informative summary of the results is a two-axis plot rather than a table of statistics, where time is plotted on the x-axis and the singleton value on the y-axis. The time-series plot can illustrate sudden changes in an otherwise stable range, identify bi-modality easily, and help quickly assess correlation with other time-series. Plots of frequency of the singleton values are likewise useful tools to visualize the variation.
通常情况下,结果的信息最丰富的摘要是双轴图,而不是统计表,其中时间绘制在x轴上,单态值绘制在y轴上。时间序列图可以说明其他稳定范围内的突然变化,轻松识别双模态,并帮助快速评估与其他时间序列的相关性。单态值频率图同样是可视化变化的有用工具。
Restricted capacity refers to the metrics defined in [RFC3148], which include criteria of data uniqueness or flow-control response to congestion.
受限容量是指[RFC3148]中定义的指标,包括数据唯一性标准或对拥塞的流量控制响应标准。
One primary metric considered is Bulk Transfer Capacity (BTC) for complete paths. [RFC3148] defines BTC as
考虑的一个主要指标是完整路径的批量传输容量(BTC)。[RFC3148]将BTC定义为
BTC = data_sent / elapsed_time
BTC = data_sent / elapsed_time
for a connection with congestion-aware flow control, where data_sent is the total number of unique payload bits (no headers).
对于具有拥塞感知流控制的连接,其中发送的数据_是唯一有效负载位(无标头)的总数。
We note that this definition *differs* from the raw capacity definition in Section 2.3.1 of [RFC5136], where IP-layer capacity *includes* all bits in the IP header and payload. This means that restricted capacity BTC is already operating at a disadvantage when compared to the raw capacity at layers below TCP. Further, there are cases where one IP layer is encapsulated in another IP layer or other form of tunneling protocol, designating more and more of the fundamental transport capacity as header bits that are pure overhead to the BTC measurement.
我们注意到,该定义*与[RFC5136]第2.3.1节中的原始容量定义*不同,其中IP层容量*包括*IP报头和有效负载中的所有位。这意味着,与TCP以下各层的原始容量相比,受限容量BTC已经处于劣势。此外,存在这样的情况,其中一个IP层封装在另一IP层或隧道协议的其他形式中,将越来越多的基本传输容量指定为对BTC测量而言是纯开销的报头比特。
We also note that raw and restricted capacity metrics are not orthogonal in the sense defined in Section 5.1.2 above. The information they convey about the network under test is certainly overlapping, but they reveal two different and important aspects of performance.
我们还注意到,在上文第5.1.2节定义的意义上,原始容量指标和受限容量指标不是正交的。它们传递的关于被测网络的信息当然是重叠的,但它们揭示了性能的两个不同且重要的方面。
When thinking about the triad of raw capacity metrics, BTC is most akin to the "IP-Type-P Available Path Capacity", at least in the eyes of a network user who seeks to know what transmission performance a path might support.
当考虑原始容量指标的三元组时,BTC最类似于“IP-Type-P可用路径容量”,至少在寻求了解路径可能支持的传输性能的网络用户眼中是如此。
The concept of "packets of Type-P" is defined in [RFC2330]. The considerations for restricted capacity are identical to the raw capacity section on this topic, with the addition that the various fields and options in the TCP header must be included in the description.
[RFC2330]中定义了“P型数据包”的概念。受限容量的注意事项与本主题的原始容量部分相同,只是说明中必须包括TCP标头中的各种字段和选项。
The vast array of TCP flow-control options are not well captured by Type-P, because they do not exist in the TCP header bits. Therefore, we introduce a new notion here: TCP Configuration of "Type-C". The elements of Type-C describe all of the settings for TCP options and congestion control algorithm variables, including the main form of congestion control in use. Readers should consider the parameters and variables of [RFC3148] and [RFC6349] when constructing Type-C.
Type-P无法很好地捕获大量TCP流控制选项,因为它们不存在于TCP报头位中。因此,我们在这里引入了一个新概念:“C型”TCP配置。C类元素描述TCP选项和拥塞控制算法变量的所有设置,包括使用中的拥塞控制的主要形式。在构造C型时,读者应考虑[RCF3148]和[RCF6399]的参数和变量。
The audience for network characterization may have detailed information about each link that comprises a complete path (due to ownership, for example), or some of the links in the path but not others, or none of the links.
网络特征化的受众可能具有关于包含完整路径(例如,由于所有权)的每个链路的详细信息,或者路径中的一些链路而不是其他链路,或者没有任何链路。
There are cases where the measurement audience only has information on one of the links (the local access link) and wishes to measure one or more BTC metrics. The discussion in Section 6.2 applies here as well.
在某些情况下,测量受众仅拥有其中一条链路(本地接入链路)的信息,并希望测量一个或多个BTC指标。第6.2节中的讨论也适用于此处。
There are limits on a useful measurement interval for BTC. Three factors that influence the interval duration are listed below:
BTC的有效测量间隔存在限制。以下列出了影响间隔时间的三个因素:
1. Measurements may choose to include or exclude the 3-way handshake of TCP connection establishment, which requires at least 1.5 * RTT (round-trip time) and contains both the delay of the path and the host processing time for responses. However, user experience includes the 3-way handshake for all new TCP connections.
1. 测量可以选择包括或排除TCP连接建立的三方握手,这至少需要1.5*RTT(往返时间),并且包含路径延迟和主机响应处理时间。但是,用户体验包括所有新TCP连接的三方握手。
2. Measurements may choose to include or exclude Slow-Start, preferring instead to focus on a portion of the transfer that represents "equilibrium" (which needs to be defined for particular circumstances if used). However, user experience includes the Slow-Start for all new TCP connections.
2. 测量可以选择包括或排除慢启动,而更倾向于关注代表“平衡”的传输部分(如果使用,需要针对特定情况进行定义)。但是,用户体验包括所有新TCP连接的缓慢启动。
3. Measurements may choose to use a fixed block of data to transfer, where the size of the block has a relationship to the file size of the application of interest. This approach yields variable size measurement intervals, where a path with faster BTC is measured for less time than a path with slower BTC, and this has implications when path impairments are time-varying, or transient. Users are likely to turn their immediate attention elsewhere when a very large file must be transferred; thus, they do not directly experience such a long transfer -- they see the result (success or failure) and possibly an objective measurement of the transfer time (which will likely include the 3-way handshake, Slow-Start, and application file management processing time as well as the BTC).
3. 测量可以选择使用固定的数据块进行传输,其中数据块的大小与感兴趣的应用程序的文件大小有关。这种方法产生可变大小的测量间隔,其中BTC速度较快的路径的测量时间少于BTC速度较慢的路径的测量时间,并且当路径损伤是时变的或瞬时的时,这具有影响。当必须传输非常大的文件时,用户可能会立即将注意力转移到其他地方;因此,他们不会直接经历如此长的传输时间——他们看到的是结果(成功或失败),也可能是传输时间的客观度量(可能包括三方握手、慢启动和应用程序文件管理处理时间以及BTC)。
Individual measurement intervals may be short or long, but there is a need to report the results on a long-term basis that captures the BTC variability experienced between each interval. Consistent BTC is a valuable commodity along with the value attained.
单个测量间隔可能短,也可能长,但需要长期报告结果,以捕获每个间隔之间经历的BTC变化。一致的BTC是一种有价值的商品,同时也是获得的价值。
When BTC of a link or path is estimated through some measurement technique, the following parameters should be reported:
当通过某种测量技术估计链路或路径的BTC时,应报告以下参数:
o Name and reference to the exact method of measurement
o 准确测量方法的名称和参考
o Maximum Transmission Unit (MTU)
o 最大传输单位(MTU)
o Maximum BTC that can be assessed in the measurement configuration
o 可在测量配置中评估的最大BTC
o Time and duration of the measurement
o 测量的时间和持续时间
o Number of BTC connections used simultaneously
o 同时使用的BTC连接数
o *All* other parameters specific to the measurement method, especially the congestion control algorithm in use
o *所有*特定于测量方法的其他参数,尤其是使用中的拥塞控制算法
See also [RFC6349].
另见[RFC6349]。
Many methods of BTC measurement have a maximum capacity that they can measure, and this maximum may be less than the available capacity of the link or path. Therefore, it is important to specify the measured BTC value beyond which there will be no measured improvement.
许多BTC测量方法都有它们可以测量的最大容量,并且该最大容量可能小于链路或路径的可用容量。因此,重要的是指定测量的BTC值,超过该值将不会有测量的改善。
The application performance estimation audience may have a desired target capacity value and simply wish to assess whether there is sufficient BTC. This case simplifies the measurement of link and path capacity to some degree, as long as the measurable maximum exceeds the target capacity.
应用程序性能评估受众可能具有期望的目标容量值,并且只希望评估是否有足够的BTC。这种情况在一定程度上简化了链路和路径容量的测量,只要可测量的最大容量超过目标容量。
As with most metrics and measurements, assessing the consistency or variability in the results gives the user an intuitive feel for the degree (or confidence) that any one value is representative of other results, or the underlying distribution from which these singleton measurements have come.
与大多数度量和测量一样,评估结果的一致性或可变性可以让用户直观地感受到任何一个值代表其他结果的程度(或置信度),或者这些单例测量的基本分布。
With two questions looming --
有两个问题迫在眉睫--
1. What ways can BTC be measured and summarized to describe the potential variability in a useful way?
1. 什么样的方法可以测量和总结BTC,以有用的方式描述潜在的可变性?
2. How can the variability in BTC estimates be reported, so that the confidence in the results is also conveyed?
2. 如何报告BTC估计值的可变性,从而传达对结果的信心?
-- we suggest the methods listed in Section 6.6.1 above, and the additional results presentations given in [RFC6349].
--我们建议采用上述第6.6.1节中列出的方法,以及[RFC6349]中给出的其他结果说明。
This section discusses two key aspects of measurement that are sometimes omitted from the report: the description of the test stream on which the measurements are based, and the sample size.
本节讨论了报告中有时忽略的两个测量关键方面:测量所基于的测试流的描述和样本量。
Network characterization has traditionally used Poisson-distributed inter-packet spacing, as this provides an unbiased sample. The average inter-packet spacing may be selected to allow observation of
传统上,网络表征使用泊松分布的数据包间隔,因为这提供了无偏样本。可选择平均分组间间隔以允许观察分组间间隔
specific network phenomena. Other test streams are designed to sample some property of the network, such as the presence of congestion, link bandwidth, or packet reordering.
特定的网络现象。其他测试流被设计用于对网络的某些属性进行采样,例如拥塞、链路带宽或数据包重新排序的存在。
If measuring a network in order to make inferences about applications or receiver performance, then there are usually efficiencies derived from a test stream that has similar characteristics to the sender. In some cases, it is essential to synthesize the sender stream, as with BTC estimates. In other cases, it may be sufficient to sample with a "known bias", e.g., a Periodic stream to estimate real-time application performance.
如果为了推断应用程序或接收器性能而测量网络,那么通常会从与发送方具有类似特征的测试流中获得效率。在某些情况下,合成发送方流是必要的,就像BTC估计一样。在其他情况下,使用“已知偏差”(例如,周期流)进行采样就足以估计实时应用程序性能。
Sample size is directly related to the accuracy of the results and plays a critical role in the report. Even if only the sample size (in terms of number of packets) is given for each value or summary statistic, it imparts a notion of the confidence in the result.
样本量直接关系到结果的准确性,在报告中起着至关重要的作用。即使每个值或汇总统计信息只给出了样本大小(以数据包数量为单位),它也传递了结果可信度的概念。
In practice, the sample size will be selected taking both statistical and practical factors into account. Among these factors are the following:
在实践中,样本量的选择将考虑统计和实际因素。这些因素包括:
1. The estimated variability of the quantity being measured.
1. 被测数量的估计可变性。
2. The desired confidence in the result (although this may be dependent on assumption of the underlying distribution of the measured quantity).
2. 对结果的期望置信度(尽管这可能取决于对测量数量基本分布的假设)。
3. The effects of active measurement traffic on user traffic.
3. 主动测量流量对用户流量的影响。
A sample size may sometimes be referred to as "large". This is a relative and qualitative term. It is preferable to describe what one is attempting to achieve with his sample. For example, stating an implication may be helpful: this sample is large enough that a single outlying value at ten times the "typical" sample mean (the mean without the outlying value) would influence the mean by no more than X.
样本量有时被称为“大”。这是一个相对的、定性的术语。最好描述一个人试图用他的样本实现什么。例如,说明一个含义可能会有帮助:该样本足够大,以至于一个10倍于“典型”样本平均值(不含边缘值的平均值)的单个边缘值对平均值的影响不超过X。
The Appendix of [RFC2330] indicates that a sample size of 128 singletons worked well for goodness-of-fit testing, while a much larger size (8192 singletons) almost always failed.
[RFC2330]的附录表明,128个单件样本的拟合优度测试效果良好,而更大的样本(8192个单件)几乎总是失败。
The security considerations that apply to any active measurement of live networks are relevant here as well. See the Security Considerations section of [RFC4656] for mandatory-to-implement security features that intend to mitigate attacks.
适用于实时网络的任何主动测量的安全注意事项也与此相关。请参阅[RFC4656]中的“安全注意事项”部分,了解实施旨在缓解攻击的安全功能的强制要求。
Measurement systems conducting long-term measurements are more exposed to threats as a by-product of ports open longer to perform their task, and more easily detected measurement activity on those ports. Further, use of long packet waiting times affords an attacker a better opportunity to prepare and launch a replay attack.
进行长期测量的测量系统更容易受到威胁,因为端口打开时间越长,执行任务的时间越长,这些端口上的测量活动越容易被检测到。此外,使用较长的数据包等待时间使攻击者有更好的机会准备和发起重播攻击。
The authors thank Phil Chimento for his suggestion to employ conditional distributions for delay, Steve Konish Jr. for his careful review and suggestions, Dave McDysan and Don McLachlan for useful comments based on their long experience with measurement and reporting, Daniel Genin for his observation of non-orthogonality between raw and restricted capacity metrics (and for noticing our previous omission of this fact), and Matt Zekauskas for suggestions on organizing the memo for easier consumption.
作者感谢Phil Chimento提出的采用延迟条件分布的建议,感谢Steve Konish Jr.提出的仔细审查和建议,感谢Dave McDysan和Don McLachlan根据其长期测量和报告经验提出的有用意见,Daniel Genin对原始容量指标和受限容量指标之间的非正交性进行了观察(并注意到我们之前忽略了这一事实),Matt Zekauskas提出了组织备忘录以便于消费的建议。
[RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, "Framework for IP Performance Metrics", RFC 2330, May 1998.
[RFC2330]Paxson,V.,Almes,G.,Mahdavi,J.,和M.Mathis,“IP性能度量框架”,RFC 2330,1998年5月。
[RFC2678] Mahdavi, J. and V. Paxson, "IPPM Metrics for Measuring Connectivity", RFC 2678, September 1999.
[RFC2678]Mahdavi,J.和V.Paxson,“测量连接性的IPPM度量”,RFC 2678,1999年9月。
[RFC2679] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way Delay Metric for IPPM", RFC 2679, September 1999.
[RFC2679]Almes,G.,Kalidini,S.,和M.Zekauskas,“IPPM的单向延迟度量”,RFC 2679,1999年9月。
[RFC2680] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way Packet Loss Metric for IPPM", RFC 2680, September 1999.
[RFC2680]Almes,G.,Kalidini,S.,和M.Zekauskas,“IPPM的单向数据包丢失度量”,RFC 2680,1999年9月。
[RFC3148] Mathis, M. and M. Allman, "A Framework for Defining Empirical Bulk Transfer Capacity Metrics", RFC 3148, July 2001.
[RFC3148]Mathis,M.和M.Allman,“定义经验批量传输容量指标的框架”,RFC 3148,2001年7月。
[RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation Metric for IP Performance Metrics (IPPM)", RFC 3393, November 2002.
[RFC3393]Demichelis,C.和P.Chimento,“IP性能度量的IP数据包延迟变化度量(IPPM)”,RFC 3393,2002年11月。
[RFC3432] Raisanen, V., Grotefeld, G., and A. Morton, "Network performance measurement with periodic streams", RFC 3432, November 2002.
[RFC3432]Raisanen,V.,Grotefeld,G.,和A.Morton,“周期流的网络性能测量”,RFC 3432,2002年11月。
[RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. Zekauskas, "A One-way Active Measurement Protocol (OWAMP)", RFC 4656, September 2006.
[RFC4656]Shalunov,S.,Teitelbaum,B.,Karp,A.,Boote,J.,和M.Zekauskas,“单向主动测量协议(OWAMP)”,RFC 46562006年9月。
[RFC4737] Morton, A., Ciavattone, L., Ramachandran, G., Shalunov, S., and J. Perser, "Packet Reordering Metrics", RFC 4737, November 2006.
[RFC4737]Morton,A.,Ciavattone,L.,Ramachandran,G.,Shalunov,S.,和J.Perser,“数据包重新排序度量”,RFC 4737,2006年11月。
[RFC5136] Chimento, P. and J. Ishac, "Defining Network Capacity", RFC 5136, February 2008.
[RFC5136]Chimento,P.和J.Ishac,“定义网络容量”,RFC 5136,2008年2月。
[Casner] Casner, S., Alaettinoglu, C., and C. Kuan, "A Fine-Grained View of High-Performance Networking", NANOG 22 Conf., May 20-22 2001, <http://www.nanog.org/presentations/archive/index.php>.
[Casner]Casner,S.,Alaettinoglu,C.,和C.Kuan,“高性能网络的细粒度视图”,NANOG 22 Conf.,2001年5月20-22日<http://www.nanog.org/presentations/archive/index.php>.
[Cia03] Ciavattone, L., Morton, A., and G. Ramachandran, "Standardized Active Measurements on a Tier 1 IP Backbone", IEEE Communications Magazine, Vol. 41 No. 6, pp. 90-97, June 2003.
[Cia03]Ciavattone,L.,Morton,A.,和G.Ramachandran,“第1层IP主干上的标准化主动测量”,《IEEE通信杂志》,第41卷第6期,第90-97页,2003年6月。
[IPPM-RPT] Shalunov, S. and M. Swany, "Reporting IP Performance Metrics to Users", Work in Progress, March 2011.
[IPPM-RPT]Shalunov,S.和M.Swany,“向用户报告IP性能指标”,正在进行的工作,2011年3月。
[RFC5481] Morton, A. and B. Claise, "Packet Delay Variation Applicability Statement", RFC 5481, March 2009.
[RFC5481]Morton,A.和B.Claise,“数据包延迟变化适用性声明”,RFC 54812009年3月。
[RFC5835] Morton, A., Ed., and S. Van den Berghe, Ed., "Framework for Metric Composition", RFC 5835, April 2010.
[RFC5835]Morton,A.,Ed.,和S.Van den Berghe,Ed.,“度量组合框架”,RFC 58352010年4月。
[RFC6349] Constantine, B., Forget, G., Geib, R., and R. Schrage, "Framework for TCP Throughput Testing", RFC 6349, August 2011.
[RFC6349]Constantine,B.,Forget,G.,Geib,R.,和R.Schrage,“TCP吞吐量测试框架”,RFC 6349,2011年8月。
[Y.1540] International Telecommunication Union, "Internet protocol data communication service - IP packet transfer and availability performance parameters", ITU-T Recommendation Y.1540, March 2011.
[Y.1540]国际电信联盟,“互联网协议数据通信服务-IP数据包传输和可用性性能参数”,ITU-T建议Y.1540,2011年3月。
[Y.1541] International Telecommunication Union, "Network performance objectives for IP-based services", ITU-T Recommendation Y.1541, December 2011.
[Y.1541]国际电信联盟,“基于IP的服务的网络性能目标”,ITU-T建议Y.1541,2011年12月。
Authors' Addresses
作者地址
Al Morton AT&T Labs 200 Laurel Avenue South Middletown, NJ 07748 USA
美国新泽西州劳雷尔大道南米德尔顿200号阿尔莫顿AT&T实验室,邮编:07748
Phone: +1 732 420 1571 Fax: +1 732 368 1192 EMail: acmorton@att.com URI: http://home.comcast.net/~acmacm/
Phone: +1 732 420 1571 Fax: +1 732 368 1192 EMail: acmorton@att.com URI: http://home.comcast.net/~acmacm/
Gomathi Ramachandran AT&T Labs 200 Laurel Avenue South Middletown, New Jersey 07748 USA
美国新泽西州南米德尔敦劳雷尔大道200号戈马蒂拉马钱德兰AT&T实验室,邮编:07748
Phone: +1 732 420 2353 EMail: gomathi@att.com
Phone: +1 732 420 2353 EMail: gomathi@att.com
Ganga Maguluri AT&T Labs 200 Laurel Avenue South Middletown, New Jersey 07748 USA
美国新泽西州劳雷尔大道南米德尔顿200号恒河马古鲁里AT&T实验室,邮编:07748
Phone: +1 732 420 2486 EMail: gmaguluri@att.com
Phone: +1 732 420 2486 EMail: gmaguluri@att.com