Internet Engineering Task Force (IETF) W. Hargrave Request for Comments: 8327 LONAP BCP: 214 M. Griswold Category: Best Current Practice 20C ISSN: 2070-1721 J. Snijders NTT N. Hilliard INEX March 2018
Internet Engineering Task Force (IETF) W. Hargrave Request for Comments: 8327 LONAP BCP: 214 M. Griswold Category: Best Current Practice 20C ISSN: 2070-1721 J. Snijders NTT N. Hilliard INEX March 2018
Mitigating the Negative Impact of Maintenance through BGP Session Culling
通过BGP会话筛选减轻维护的负面影响
Abstract
摘要
This document outlines an approach to mitigate the negative impact on networks resulting from maintenance activities. It includes guidance for both IP networks and Internet Exchange Points (IXPs). The approach is to ensure BGP-4 sessions that will be affected by maintenance are forcefully torn down before the actual maintenance activities commence.
本文件概述了一种减轻维护活动对网络造成负面影响的方法。它包括IP网络和Internet交换点(IXP)的指南。该方法旨在确保在实际维修活动开始之前,将受维修影响的BGP-4会话强制拆除。
Status of This Memo
关于下段备忘
This memo documents an Internet Best Current Practice.
本备忘录记录了互联网最佳实践。
This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on BCPs is available in Section 2 of RFC 7841.
本文件是互联网工程任务组(IETF)的产品。它代表了IETF社区的共识。它已经接受了公众审查,并已被互联网工程指导小组(IESG)批准出版。有关BCP的更多信息,请参见RFC 7841第2节。
Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at https://www.rfc-editor.org/info/rfc8327.
有关本文件当前状态、任何勘误表以及如何提供反馈的信息,请访问https://www.rfc-editor.org/info/rfc8327.
Copyright Notice
版权公告
Copyright (c) 2018 IETF Trust and the persons identified as the document authors. All rights reserved.
版权所有(c)2018 IETF信托基金和确定为文件作者的人员。版权所有。
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
本文件受BCP 78和IETF信托有关IETF文件的法律规定的约束(https://trustee.ietf.org/license-info)自本文件出版之日起生效。请仔细阅读这些文件,因为它们描述了您对本文件的权利和限制。从本文件中提取的代码组件必须包括信托法律条款第4.e节中所述的简化BSD许可证文本,并提供简化BSD许可证中所述的无担保。
Table of Contents
目录
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Requirements Language . . . . . . . . . . . . . . . . . . . . 3 3. BGP Session Culling . . . . . . . . . . . . . . . . . . . . . 3 3.1. Voluntary BGP Session Teardown Recommendations . . . . . 4 3.1.1. Maintenance Considerations . . . . . . . . . . . . . 4 3.2. Involuntary BGP Session Teardown Recommendations . . . . 4 3.2.1. Packet-Filter Considerations . . . . . . . . . . . . 5 3.2.2. Hardware Considerations . . . . . . . . . . . . . . . 5 3.3. Procedural Considerations . . . . . . . . . . . . . . . . 6 4. Security Considerations . . . . . . . . . . . . . . . . . . . 6 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 6 6. References . . . . . . . . . . . . . . . . . . . . . . . . . 7 6.1. Normative References . . . . . . . . . . . . . . . . . . 7 6.2. Informative References . . . . . . . . . . . . . . . . . 7 Appendix A. Example Packet Filters . . . . . . . . . . . . . . . 8 A.1. Example Configuration for Cisco IOS, IOS XR, and Arista EOS . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 A.2. Example Configuration for Nokia SR OS . . . . . . . . . . 9 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 10 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 10
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Requirements Language . . . . . . . . . . . . . . . . . . . . 3 3. BGP Session Culling . . . . . . . . . . . . . . . . . . . . . 3 3.1. Voluntary BGP Session Teardown Recommendations . . . . . 4 3.1.1. Maintenance Considerations . . . . . . . . . . . . . 4 3.2. Involuntary BGP Session Teardown Recommendations . . . . 4 3.2.1. Packet-Filter Considerations . . . . . . . . . . . . 5 3.2.2. Hardware Considerations . . . . . . . . . . . . . . . 5 3.3. Procedural Considerations . . . . . . . . . . . . . . . . 6 4. Security Considerations . . . . . . . . . . . . . . . . . . . 6 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 6 6. References . . . . . . . . . . . . . . . . . . . . . . . . . 7 6.1. Normative References . . . . . . . . . . . . . . . . . . 7 6.2. Informative References . . . . . . . . . . . . . . . . . 7 Appendix A. Example Packet Filters . . . . . . . . . . . . . . . 8 A.1. Example Configuration for Cisco IOS, IOS XR, and Arista EOS . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 A.2. Example Configuration for Nokia SR OS . . . . . . . . . . 9 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 10 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 10
BGP Session Culling is the practice of ensuring BGP sessions are forcefully torn down before maintenance activities on a lower-layer network commence -- activities that otherwise would affect the flow of data between the BGP speakers. BGP Session Culling is the practice of ensuring BGP sessions are forcefully torn down before commencing maintenance activities (that otherwise would affect the flow of data between the BGP speakers) on a lower-layer network.
BGP会话剔除是确保在较低层网络上的维护活动开始之前,BGP会话被强制拆除的一种做法,否则会影响BGP扬声器之间的数据流。BGP会话剔除是在较低层网络上开始维护活动(否则会影响BGP扬声器之间的数据流)之前,确保BGP会话被强制拆除的实践。
BGP Session Culling minimizes the amount of disruption that lower-layer network maintenance activities cause, by making BGP speakers preemptively converge onto alternative paths while the lower-layer network's forwarding plane remains fully operational.
BGP会话剔除通过使BGP扬声器在下层网络的转发平面保持完全运行的情况下抢先收敛到备用路径上,将下层网络维护活动造成的中断量降至最低。
The grace period required for a successful application of BGP Session Culling is the sum of the time needed to detect the loss of the BGP session plus the time required for the BGP speaker to converge onto alternative paths. The first value is often governed by the BGP Hold Timer (see Section 6.5 of [RFC4271]), which is commonly between 90 and 180 seconds. The second value is implementation specific, but it could be as much as 15 minutes when a router with a slow control plane is receiving a full set of Internet routes.
成功应用BGP会话剔除所需的宽限期是检测BGP会话丢失所需的时间加上BGP说话人收敛到备用路径所需的时间之和。第一个值通常由BGP保持计时器控制(见[RFC4271]第6.5节),通常在90到180秒之间。第二个值是特定于实现的,但当具有慢速控制平面的路由器接收到全套Internet路由时,它可能长达15分钟。
Throughout this document, the "Caretaker" is defined to be in control of the lower-layer network, while "Operators" directly administrate the BGP speakers. Operators and Caretakers implementing BGP Session Culling are encouraged to avoid using a fixed grace period, and instead to monitor forwarding-plane activity while the culling is taking place and to consider it complete once traffic levels have dropped to a minimum (Section 3.3).
在本文件中,“看门人”被定义为控制下层网络,“运营商”直接管理BGP扬声器。鼓励执行BGP会话剔除的操作者和看守者避免使用固定的宽限期,而是在剔除发生时监视转发平面活动,并且一旦流量水平降至最低,就考虑它完成(第3.3节)。
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.
本文件中的关键词“必须”、“不得”、“必需”、“应”、“不应”、“建议”、“不建议”、“可”和“可选”在所有大写字母出现时(如图所示)应按照BCP 14[RFC2119][RFC8174]所述进行解释。
From the viewpoint of the Operator, there are two types of BGP Session Culling:
从运营商的角度来看,有两种类型的BGP会话剔除:
Voluntary BGP Session Teardown: The Operator initiates the teardown of the potentially affected BGP session by issuing an Administrative Shutdown.
自愿性BGP会话拆除:操作员通过发出管理关闭命令来启动潜在受影响BGP会话的拆除。
Involuntary BGP Session Teardown: The Caretaker of the lower-layer network disrupts (higher-layer) BGP control-plane traffic, causing the BGP Hold Timers of the affected BGP session to expire, subsequently triggering rerouting of end-user traffic.
非自愿BGP会话断开:下层网络的管理员中断(高层)BGP控制平面通信,导致受影响BGP会话的BGP保持计时器过期,随后触发最终用户通信的重新路由。
Before an Operator commences activities that can cause disruption to the flow of data through the lower-layer network, an Operator can reduce loss of traffic by issuing an administrative shutdown to all BGP sessions running across the lower-layer network and wait a few minutes for data-plane traffic to subside.
在运营商开始可能导致通过较低层网络的数据流中断的活动之前,运营商可以通过向在较低层网络上运行的所有BGP会话发出管理关闭命令,并等待几分钟,以使数据平面流量减少,从而减少流量损失。
While architectures exist to facilitate quick network reconvergence (such as BGP Prefix Independent Convergence (PIC) [BGP_PIC]), an Operator cannot assume the remote side has such capabilities. As such, a grace period between the Administrative Shutdown and the impacting maintenance activities is warranted.
虽然存在促进快速网络重新聚合的体系结构(如BGP前缀独立聚合(PIC)[BGP_PIC]),但运营商不能假设远程端具有此类能力。因此,在管理关闭和影响维护活动之间有一段宽限期是有保证的。
After the maintenance activities have concluded, the Operator is expected to restore the BGP sessions to their original Administrative state.
维护活动结束后,操作员应将BGP会话恢复到其原始管理状态。
Initiators of the Administrative Shutdown MAY consider using Graceful Shutdown [RFC8326] to facilitate smooth drainage of traffic prior to session tear down, and the Shutdown Communication [RFC8203] to inform the remote side on the nature and duration of the maintenance activities.
管理关机的发起人可以考虑使用优美关机[RCF8326],以便于在会话崩溃之前畅通流量,以及关闭通信[RCF8203],以向远程侧通知维护活动的性质和持续时间。
In the case where multilateral interconnection between BGP speakers is facilitated through a switched Layer 2 fabric, such as commonly seen at Internet Exchange Points (IXPs), different operational considerations can apply.
在BGP扬声器之间的多边互连通过交换的第2层结构(如在互联网交换点(IXP)常见)来实现的情况下,可以应用不同的操作考虑。
Operational experience shows that many Operators are unable to carry out the Voluntary BGP Session Teardown recommendations, because of the operational cost and risk of coordinating the two configuration changes required. This has an adverse affect on Internet performance.
运营经验表明,许多运营商无法执行自愿BGP会话拆卸建议,因为协调所需的两个配置更改会带来运营成本和风险。这对互联网性能有不利影响。
In the absence of notifications from the lower layer (e.g., Ethernet link down) consistent with the planned maintenance activities in a switched Layer 2 fabric, the Caretaker of the fabric could choose to cull BGP sessions on behalf of the Operators connected to the fabric.
在交换的第2层结构中,如果没有与计划的维护活动一致的来自下层的通知(例如,以太网链路断开),则该结构的管理员可以代表连接到该结构的运营商选择挑选BGP会话。
Such culling of control-plane traffic will preempt the loss of end-user traffic by causing the expiration of BGP Hold Timers ahead of the moment where the expiration would occur without intervention from the fabric's Caretaker.
控制平面流量的这种剔除将通过在没有结构管理员干预的情况下提前导致BGP保持计时器过期来抢占最终用户流量的损失。
In this scenario, BGP Session Culling is accomplished as described in the next subsection, through the application of a combined Layer 3 and Layer 4 (Layer 3/4) packet filter deployed in the Caretaker's switched fabric.
在该场景中,BGP会话剔除如下一小节所述,通过应用部署在看守者的交换结构中的组合的第3层和第4层(第3/4层)分组过滤器来完成。
The peering LAN prefixes used by the IXP form the control plane, and the following considerations apply to the packet-filter design:
IXP使用的对等LAN前缀构成控制平面,以下注意事项适用于包过滤器设计:
o The packet filter MUST only affect BGP traffic specific to the Layer 2 fabric, i.e., traffic forming part of the control plane of the system described, rather than multihop BGP traffic that merely transits.
o 分组过滤器必须仅影响特定于第2层结构的BGP通信量,即构成所述系统控制平面一部分的通信量,而不是仅传输的多跳BGP通信量。
o The packet filter MUST only affect BGP, i.e., TCP port 179.
o 数据包过滤器必须只影响BGP,即TCP端口179。
o The packet filter SHOULD make provision for the bidirectional nature of BGP, i.e., sessions may be established in either direction.
o 分组过滤器应为BGP的双向性质作出规定,即,可以在任一方向上建立会话。
o The packet filter MUST affect all Address Family Identifiers.
o 数据包筛选器必须影响所有地址族标识符。
Appendix A contains examples of correct packet filters for various platforms.
附录A包含各种平台的正确数据包过滤器示例。
Not all hardware is capable of deploying combined Layer 3/4 filters on Layer 2 ports; even on platforms that claim support for such a feature, limitations may exist or hardware resource allocation failures may occur during filter deployment, which may cause unexpected results. These problems may include:
并非所有硬件都能够在第2层端口上部署组合的第3/4层过滤器;即使在声称支持此类功能的平台上,在筛选器部署期间也可能存在限制或硬件资源分配失败,这可能会导致意外结果。这些问题可能包括:
o Platform inability to apply Layer 3/4 filters on ports that already have Layer 2 filters applied.
o 平台无法在已应用第2层筛选器的端口上应用第3/4层筛选器。
o Layer 3/4 filters supported for IPv4 but not for IPv6.
o IPv4支持第3/4层筛选器,但IPv6不支持。
o Layer 3/4 filters supported on physical ports, but not on IEEE 802.1AX Link Aggregate ports [IEEE802.1AX].
o 物理端口支持第3/4层过滤器,但IEEE 802.1AX链路聚合端口[IEEE802.1AX]不支持。
o Failure of the Caretaker to apply filters to all IEEE 802.1AX Link Aggregate ports [IEEE802.1AX].
o 看守人未能将筛选器应用于所有IEEE 802.1AX链路聚合端口[IEEE802.1AX]。
o Limitations in Access Control List (ACL) hardware mechanisms causing filters not to be applied.
o 访问控制列表(ACL)硬件机制中的限制导致无法应用筛选器。
o Fragmentation of ACL lookup memory causing transient ACL application problems that are resolved after ACL removal/ reapplication.
o ACL查找内存碎片导致暂时ACL应用程序问题,这些问题在ACL删除/重新应用后得到解决。
o Temporary service loss during hardware programming.
o 硬件编程期间的临时服务损失。
o Reduction in hardware ACL capacity if the platform enables lossless ACL application.
o 如果平台启用无损ACL应用程序,则会减少硬件ACL容量。
It is advisable for the Caretaker to be aware of the limitations of their hardware and to thoroughly test all complicated configurations in advance to ensure that problems don't occur during production deployments.
建议管理员了解其硬件的局限性,并提前彻底测试所有复杂的配置,以确保在生产部署期间不会出现问题。
The Caretaker of the lower-layer network can monitor data-plane traffic (e.g., interface counters) and carry out the maintenance without impact to traffic once session culling is complete.
一旦会话剔除完成,下层网络的管理员可以监控数据平面流量(例如,接口计数器),并在不影响流量的情况下执行维护。
It is recommended that the packet filters be deployed for the duration of the maintenance only and be removed immediately after the maintenance is completed. To prevent unnecessary troubleshooting, it is RECOMMENDED that Caretakers notify the affected Operators before the maintenance takes place and make it explicit that the Involuntary BGP Session Culling methodology will be applied.
建议仅在维护期间部署数据包过滤器,并在维护完成后立即移除。为避免不必要的故障排除,建议管理员在进行维护之前通知受影响的操作员,并明确说明将采用非自愿BGP会话剔除方法。
There are no security considerations.
没有安全考虑。
This document has no actions for IANA.
本文档没有针对IANA的操作。
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, <https://www.rfc-editor.org/info/rfc2119>.
[RFC2119]Bradner,S.,“RFC中用于表示需求水平的关键词”,BCP 14,RFC 2119,DOI 10.17487/RFC2119,1997年3月<https://www.rfc-editor.org/info/rfc2119>.
[RFC4271] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A Border Gateway Protocol 4 (BGP-4)", RFC 4271, DOI 10.17487/RFC4271, January 2006, <https://www.rfc-editor.org/info/rfc4271>.
[RFC4271]Rekhter,Y.,Ed.,Li,T.,Ed.,和S.Hares,Ed.,“边境网关协议4(BGP-4)”,RFC 4271,DOI 10.17487/RFC4271,2006年1月<https://www.rfc-editor.org/info/rfc4271>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[RFC8174]Leiba,B.,“RFC 2119关键词中大写与小写的歧义”,BCP 14,RFC 8174,DOI 10.17487/RFC8174,2017年5月<https://www.rfc-editor.org/info/rfc8174>.
[BGP_PIC] Bashandy, A., Ed., Filsfils, C., and P. Mohapatra, "BGP Prefix Independent Convergence", Work in Progress, draft-ietf-rtgwg-bgp-pic-06, November 2017.
[BGP_PIC]Bashandy,A.,Ed.,Filsfils,C.,和P.Mohapatra,“BGP前缀独立收敛”,正在进行的工作,草稿-ietf-rtgwg-BGP-PIC-062017年11月。
[IEEE802.1AX] IEEE, "IEEE Standard for Local and metropolitan area networks -- Link Aggregation", IEEE Std 802.1AX-2014, DOI 10.1109/IEEESTD.2014.7055197, December 2014, <http://ieeexplore.ieee.org/servlet/ opac?punumber=6997981>.
[IEEE802.1AX]IEEE,“局域网和城域网的IEEE标准——链路聚合”,IEEE标准802.1AX-2014,DOI 10.1109/IEEESTD.2014.7055197,2014年12月<http://ieeexplore.ieee.org/servlet/ opac?punumber=6997981>。
[RFC8203] Snijders, J., Heitz, J., and J. Scudder, "BGP Administrative Shutdown Communication", RFC 8203, DOI 10.17487/RFC8203, July 2017, <https://www.rfc-editor.org/info/rfc8203>.
[RFC8203]Snijders,J.,Heitz,J.,和J.Scudder,“BGP管理关闭通信”,RFC 8203,DOI 10.17487/RFC8203,2017年7月<https://www.rfc-editor.org/info/rfc8203>.
[RFC8326] Francois, P., Ed., Decraene, B., Ed., Pelsser, C., Patel, K., and C. Filsfils, "Graceful BGP Session Shutdown", RFC 8326, DOI 10.17487/8326, March 2018, <https://www.rfc-editor.org/info/rfc8326>.
[RFC8326]Francois,P.,Ed.,DeClaene,B.,Ed.,Pelsser,C.,Patel,K.,和C.Filsfils,“优雅的BGP会话关闭”,RFC 8326,DOI 10.17487/8326,2018年3月<https://www.rfc-editor.org/info/rfc8326>.
This section includes examples of packet filters performing Involuntary BGP Session Teardown at an IXP using peering LAN prefixes 192.0.2.0/24 and 2001:db8:2::/64 as its control plane.
本节包括使用对等LAN前缀192.0.2.0/24和2001:db8:2::/64作为其控制平面在IXP上执行非自愿BGP会话断开的数据包过滤器示例。
A repository of configuration examples for a number of assorted platforms can be found at <https://github.com/bgp/bgp-session-culling-config-examples>.
许多不同平台的配置示例库可在以下位置找到:<https://github.com/bgp/bgp-session-culling-config-examples>.
ipv6 access-list acl-ipv6-permit-all-except-bgp 10 deny tcp 2001:db8:2::/64 eq bgp 2001:db8:2::/64 20 deny tcp 2001:db8:2::/64 2001:db8:2::/64 eq bgp 30 permit ipv6 any any ! ip access-list acl-ipv4-permit-all-except-bgp 10 deny tcp 192.0.2.0/24 eq bgp 192.0.2.0/24 20 deny tcp 192.0.2.0/24 192.0.2.0/24 eq bgp 30 permit ip any any ! interface Ethernet33 description IXP Participant Affected by Maintenance ip access-group acl-ipv4-permit-all-except-bgp in ipv6 access-group acl-ipv6-permit-all-except-bgp in !
ipv6 access-list acl-ipv6-permit-all-except-bgp 10 deny tcp 2001:db8:2::/64 eq bgp 2001:db8:2::/64 20 deny tcp 2001:db8:2::/64 2001:db8:2::/64 eq bgp 30 permit ipv6 any any ! ip access-list acl-ipv4-permit-all-except-bgp 10 deny tcp 192.0.2.0/24 eq bgp 192.0.2.0/24 20 deny tcp 192.0.2.0/24 192.0.2.0/24 eq bgp 30 permit ip any any ! interface Ethernet33 description IXP Participant Affected by Maintenance ip access-group acl-ipv4-permit-all-except-bgp in ipv6 access-group acl-ipv6-permit-all-except-bgp in !
ip-filter 10 create filter-name "ACL IPv4 Permit All Except BGP" default-action forward entry 10 create match protocol tcp dst-ip 192.0.2.0/24 src-ip 192.0.2.0/24 port eq 179 exit action drop exit exit exit
ip筛选器10创建筛选器名称“ACL IPv4允许除BGP以外的所有”默认操作转发条目10创建匹配协议tcp dst ip 192.0.2.0/24 src ip 192.0.2.0/24端口eq 179退出操作删除退出
ipv6-filter 10 create filter-name "ACL IPv6 Permit All Except BGP" default-action forward entry 10 create match next-header tcp dst-ip 2001:db8:2::/64 src-ip 2001:db8:2::/64 port eq 179 exit action drop exit exit exit
ipv6筛选器10创建筛选器名称“ACL ipv6允许除BGP以外的所有”默认操作转发条目10创建匹配下一个标头tcp dst ip 2001:db8:2::/64 src ip 2001:db8:2::/64端口eq 179退出操作删除退出
interface "port-1/1/1" description "IXP Participant Affected by Maintenance" ingress filter ip 10 filter ipv6 10 exit exit
接口“端口1/1/1”说明“受维护影响的IXP参与者”入口筛选器ip 10筛选器ipv6 10出口
Acknowledgments
致谢
The authors would like to thank the following people for their contributions to this document: Saku Ytti, Greg Hankins, James Bensley, Wolfgang Tremmel, Daniel Roesen, Bruno Decraene, Tore Anderson, John Heasley, Warren Kumari, Stig Venaas, and Brian Carpenter.
作者感谢以下人士对本文件的贡献:Saku Ytti、Greg Hankins、James Bensley、Wolfgang Tremmel、Daniel Roesen、Bruno Decarene、Tore Anderson、John Heasley、Warren Kumari、Stig Venaas和Brian Carpenter。
Authors' Addresses
作者地址
Will Hargrave LONAP Ltd 5 Fleet Place London EC4M 7RD United Kingdom
英国伦敦第七大道5号舰队广场Hargrave LONAP Ltd.会否
Email: will@lonap.net
Email: will@lonap.net
Matt Griswold 20C 1658 Milwaukee Ave # 100-4506 Chicago, IL 60647 United States of America
马特·格里斯沃尔德20C 1658密尔沃基大街100-4506芝加哥,伊利诺伊州,美利坚合众国60647
Email: grizz@20c.com
Email: grizz@20c.com
Job Snijders NTT Communications Theodorus Majofskistraat 100 Amsterdam 1065 SZ The Netherlands
Job Snijders NTT Communications Theodorus Majofskistraat 100阿姆斯特丹1065 SZ荷兰
Email: job@ntt.net
Email: job@ntt.net
Nick Hilliard INEX 4027 Kingswood Road Dublin 24 Ireland
Nick Hilliard INEX 4027 Kingswood路都柏林24号爱尔兰
Email: nick@inex.ie
Email: nick@inex.ie