Internet Engineering Task Force (IETF) J. Dickinson Request for Comments: 8618 J. Hague Category: Standards Track S. Dickinson ISSN: 2070-1721 Sinodun IT T. Manderson ICANN J. Bond Wikimedia Foundation, Inc. September 2019
Internet Engineering Task Force (IETF) J. Dickinson Request for Comments: 8618 J. Hague Category: Standards Track S. Dickinson ISSN: 2070-1721 Sinodun IT T. Manderson ICANN J. Bond Wikimedia Foundation, Inc. September 2019
Compacted-DNS (C-DNS): A Format for DNS Packet Capture
压缩DNS(C-DNS):用于DNS数据包捕获的格式
Abstract
摘要
This document describes a data representation for collections of DNS messages. The format is designed for efficient storage and transmission of large packet captures of DNS traffic; it attempts to minimize the size of such packet capture files but retain the full DNS message contents along with the most useful transport metadata. It is intended to assist with the development of DNS traffic-monitoring applications.
本文档描述DNS消息集合的数据表示。该格式设计用于高效存储和传输DNS流量的大数据包捕获;它试图最小化此类数据包捕获文件的大小,但保留完整的DNS消息内容以及最有用的传输元数据。它旨在帮助开发DNS流量监控应用程序。
Status of This Memo
关于下段备忘
This is an Internet Standards Track document.
这是一份互联网标准跟踪文件。
This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 7841.
本文件是互联网工程任务组(IETF)的产品。它代表了IETF社区的共识。它已经接受了公众审查,并已被互联网工程指导小组(IESG)批准出版。有关互联网标准的更多信息,请参见RFC 7841第2节。
Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at https://www.rfc-editor.org/info/rfc8618.
有关本文件当前状态、任何勘误表以及如何提供反馈的信息,请访问https://www.rfc-editor.org/info/rfc8618.
Copyright Notice
版权公告
Copyright (c) 2019 IETF Trust and the persons identified as the document authors. All rights reserved.
版权(c)2019 IETF信托基金和被确定为文件作者的人员。版权所有。
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
本文件受BCP 78和IETF信托有关IETF文件的法律规定的约束(https://trustee.ietf.org/license-info)自本文件出版之日起生效。请仔细阅读这些文件,因为它们描述了您对本文件的权利和限制。从本文件中提取的代码组件必须包括信托法律条款第4.e节中所述的简化BSD许可证文本,并提供简化BSD许可证中所述的无担保。
Table of Contents
目录
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 3. Data Collection Use Cases . . . . . . . . . . . . . . . . . . 5 4. Design Considerations . . . . . . . . . . . . . . . . . . . . 8 5. Choice of CBOR . . . . . . . . . . . . . . . . . . . . . . . 10 6. C-DNS Format Conceptual Overview . . . . . . . . . . . . . . 10 6.1. Block Parameters . . . . . . . . . . . . . . . . . . . . 14 6.2. Storage Parameters . . . . . . . . . . . . . . . . . . . 14 6.2.1. Optional Data Items . . . . . . . . . . . . . . . . . 15 6.2.2. Optional RRs and OPCODEs . . . . . . . . . . . . . . 16 6.2.3. Storage Flags . . . . . . . . . . . . . . . . . . . . 17 6.2.4. IP Address Storage . . . . . . . . . . . . . . . . . 17 7. C-DNS Format Detailed Description . . . . . . . . . . . . . . 18 7.1. Map Quantities and Indexes . . . . . . . . . . . . . . . 18 7.2. Tabular Representation . . . . . . . . . . . . . . . . . 18 7.3. "File" . . . . . . . . . . . . . . . . . . . . . . . . . 19 7.3.1. "FilePreamble" . . . . . . . . . . . . . . . . . . . 20 7.3.1.1. "BlockParameters" . . . . . . . . . . . . . . . . 20 7.3.1.1.1. "StorageParameters" . . . . . . . . . . . . . 21 7.3.1.1.1.1. "StorageHints" . . . . . . . . . . . . . 22 7.3.1.1.2. "CollectionParameters" . . . . . . . . . . . 24 7.3.2. "Block" . . . . . . . . . . . . . . . . . . . . . . . 25 7.3.2.1. "BlockPreamble" . . . . . . . . . . . . . . . . . 26 7.3.2.2. "BlockStatistics" . . . . . . . . . . . . . . . . 27 7.3.2.3. "BlockTables" . . . . . . . . . . . . . . . . . . 28 7.3.2.3.1. "ClassType" . . . . . . . . . . . . . . . . . 29 7.3.2.3.2. "QueryResponseSignature" . . . . . . . . . . 30 7.3.2.3.3. "Question" . . . . . . . . . . . . . . . . . 33 7.3.2.3.4. "RR" . . . . . . . . . . . . . . . . . . . . 34 7.3.2.3.5. "MalformedMessageData" . . . . . . . . . . . 34
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 3. Data Collection Use Cases . . . . . . . . . . . . . . . . . . 5 4. Design Considerations . . . . . . . . . . . . . . . . . . . . 8 5. Choice of CBOR . . . . . . . . . . . . . . . . . . . . . . . 10 6. C-DNS Format Conceptual Overview . . . . . . . . . . . . . . 10 6.1. Block Parameters . . . . . . . . . . . . . . . . . . . . 14 6.2. Storage Parameters . . . . . . . . . . . . . . . . . . . 14 6.2.1. Optional Data Items . . . . . . . . . . . . . . . . . 15 6.2.2. Optional RRs and OPCODEs . . . . . . . . . . . . . . 16 6.2.3. Storage Flags . . . . . . . . . . . . . . . . . . . . 17 6.2.4. IP Address Storage . . . . . . . . . . . . . . . . . 17 7. C-DNS Format Detailed Description . . . . . . . . . . . . . . 18 7.1. Map Quantities and Indexes . . . . . . . . . . . . . . . 18 7.2. Tabular Representation . . . . . . . . . . . . . . . . . 18 7.3. "File" . . . . . . . . . . . . . . . . . . . . . . . . . 19 7.3.1. "FilePreamble" . . . . . . . . . . . . . . . . . . . 20 7.3.1.1. "BlockParameters" . . . . . . . . . . . . . . . . 20 7.3.1.1.1. "StorageParameters" . . . . . . . . . . . . . 21 7.3.1.1.1.1. "StorageHints" . . . . . . . . . . . . . 22 7.3.1.1.2. "CollectionParameters" . . . . . . . . . . . 24 7.3.2. "Block" . . . . . . . . . . . . . . . . . . . . . . . 25 7.3.2.1. "BlockPreamble" . . . . . . . . . . . . . . . . . 26 7.3.2.2. "BlockStatistics" . . . . . . . . . . . . . . . . 27 7.3.2.3. "BlockTables" . . . . . . . . . . . . . . . . . . 28 7.3.2.3.1. "ClassType" . . . . . . . . . . . . . . . . . 29 7.3.2.3.2. "QueryResponseSignature" . . . . . . . . . . 30 7.3.2.3.3. "Question" . . . . . . . . . . . . . . . . . 33 7.3.2.3.4. "RR" . . . . . . . . . . . . . . . . . . . . 34 7.3.2.3.5. "MalformedMessageData" . . . . . . . . . . . 34
7.3.2.4. "QueryResponse" . . . . . . . . . . . . . . . . . 35 7.3.2.4.1. "ResponseProcessingData" . . . . . . . . . . 36 7.3.2.4.2. "QueryResponseExtended" . . . . . . . . . . . 37 7.3.2.5. "AddressEventCount" . . . . . . . . . . . . . . . 38 7.3.2.6. "MalformedMessage" . . . . . . . . . . . . . . . 39 8. Versioning . . . . . . . . . . . . . . . . . . . . . . . . . 39 9. C-DNS to PCAP . . . . . . . . . . . . . . . . . . . . . . . . 40 9.1. Name Compression . . . . . . . . . . . . . . . . . . . . 42 10. Data Collection . . . . . . . . . . . . . . . . . . . . . . . 42 10.1. Matching Algorithm . . . . . . . . . . . . . . . . . . . 43 10.2. Message Identifiers . . . . . . . . . . . . . . . . . . 45 10.2.1. Primary ID (Required) . . . . . . . . . . . . . . . 45 10.2.2. Secondary ID (Optional) . . . . . . . . . . . . . . 46 10.3. Algorithm Parameters . . . . . . . . . . . . . . . . . . 46 10.4. Algorithm Requirements . . . . . . . . . . . . . . . . . 46 10.5. Algorithm Limitations . . . . . . . . . . . . . . . . . 47 10.6. Workspace . . . . . . . . . . . . . . . . . . . . . . . 47 10.7. Output . . . . . . . . . . . . . . . . . . . . . . . . . 47 10.8. Post-Processing . . . . . . . . . . . . . . . . . . . . 47 11. Implementation Guidance . . . . . . . . . . . . . . . . . . . 47 11.1. Optional Data . . . . . . . . . . . . . . . . . . . . . 48 11.2. Trailing Bytes . . . . . . . . . . . . . . . . . . . . . 48 11.3. Limiting Collection of RDATA . . . . . . . . . . . . . . 49 11.4. Timestamps . . . . . . . . . . . . . . . . . . . . . . . 49 12. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 49 12.1. Transport Types . . . . . . . . . . . . . . . . . . . . 49 12.2. Data Storage Flags . . . . . . . . . . . . . . . . . . . 50 12.3. Response-Processing Flags . . . . . . . . . . . . . . . 51 12.4. AddressEvent Types . . . . . . . . . . . . . . . . . . . 51 13. Security Considerations . . . . . . . . . . . . . . . . . . . 52 14. Privacy Considerations . . . . . . . . . . . . . . . . . . . 52 15. References . . . . . . . . . . . . . . . . . . . . . . . . . 53 15.1. Normative References . . . . . . . . . . . . . . . . . . 53 15.2. Informative References . . . . . . . . . . . . . . . . . 55 Appendix A. CDDL . . . . . . . . . . . . . . . . . . . . . . . . 58 Appendix B. DNS Name Compression Example . . . . . . . . . . . . 69 B.1. NSD Compression Algorithm . . . . . . . . . . . . . . . . 70 B.2. Knot Authoritative Compression Algorithm . . . . . . . . 70 B.3. Observed Differences . . . . . . . . . . . . . . . . . . 71 Appendix C. Comparison of Binary Formats . . . . . . . . . . . . 71 C.1. Comparison with Full PCAP Files . . . . . . . . . . . . . 74 C.2. Simple versus Block Coding . . . . . . . . . . . . . . . 74 C.3. Binary versus Text Formats . . . . . . . . . . . . . . . 75 C.4. Performance . . . . . . . . . . . . . . . . . . . . . . . 75 C.5. Conclusions . . . . . . . . . . . . . . . . . . . . . . . 75 C.6. Block Size Choice . . . . . . . . . . . . . . . . . . . . 76
7.3.2.4. "QueryResponse" . . . . . . . . . . . . . . . . . 35 7.3.2.4.1. "ResponseProcessingData" . . . . . . . . . . 36 7.3.2.4.2. "QueryResponseExtended" . . . . . . . . . . . 37 7.3.2.5. "AddressEventCount" . . . . . . . . . . . . . . . 38 7.3.2.6. "MalformedMessage" . . . . . . . . . . . . . . . 39 8. Versioning . . . . . . . . . . . . . . . . . . . . . . . . . 39 9. C-DNS to PCAP . . . . . . . . . . . . . . . . . . . . . . . . 40 9.1. Name Compression . . . . . . . . . . . . . . . . . . . . 42 10. Data Collection . . . . . . . . . . . . . . . . . . . . . . . 42 10.1. Matching Algorithm . . . . . . . . . . . . . . . . . . . 43 10.2. Message Identifiers . . . . . . . . . . . . . . . . . . 45 10.2.1. Primary ID (Required) . . . . . . . . . . . . . . . 45 10.2.2. Secondary ID (Optional) . . . . . . . . . . . . . . 46 10.3. Algorithm Parameters . . . . . . . . . . . . . . . . . . 46 10.4. Algorithm Requirements . . . . . . . . . . . . . . . . . 46 10.5. Algorithm Limitations . . . . . . . . . . . . . . . . . 47 10.6. Workspace . . . . . . . . . . . . . . . . . . . . . . . 47 10.7. Output . . . . . . . . . . . . . . . . . . . . . . . . . 47 10.8. Post-Processing . . . . . . . . . . . . . . . . . . . . 47 11. Implementation Guidance . . . . . . . . . . . . . . . . . . . 47 11.1. Optional Data . . . . . . . . . . . . . . . . . . . . . 48 11.2. Trailing Bytes . . . . . . . . . . . . . . . . . . . . . 48 11.3. Limiting Collection of RDATA . . . . . . . . . . . . . . 49 11.4. Timestamps . . . . . . . . . . . . . . . . . . . . . . . 49 12. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 49 12.1. Transport Types . . . . . . . . . . . . . . . . . . . . 49 12.2. Data Storage Flags . . . . . . . . . . . . . . . . . . . 50 12.3. Response-Processing Flags . . . . . . . . . . . . . . . 51 12.4. AddressEvent Types . . . . . . . . . . . . . . . . . . . 51 13. Security Considerations . . . . . . . . . . . . . . . . . . . 52 14. Privacy Considerations . . . . . . . . . . . . . . . . . . . 52 15. References . . . . . . . . . . . . . . . . . . . . . . . . . 53 15.1. Normative References . . . . . . . . . . . . . . . . . . 53 15.2. Informative References . . . . . . . . . . . . . . . . . 55 Appendix A. CDDL . . . . . . . . . . . . . . . . . . . . . . . . 58 Appendix B. DNS Name Compression Example . . . . . . . . . . . . 69 B.1. NSD Compression Algorithm . . . . . . . . . . . . . . . . 70 B.2. Knot Authoritative Compression Algorithm . . . . . . . . 70 B.3. Observed Differences . . . . . . . . . . . . . . . . . . 71 Appendix C. Comparison of Binary Formats . . . . . . . . . . . . 71 C.1. Comparison with Full PCAP Files . . . . . . . . . . . . . 74 C.2. Simple versus Block Coding . . . . . . . . . . . . . . . 74 C.3. Binary versus Text Formats . . . . . . . . . . . . . . . 75 C.4. Performance . . . . . . . . . . . . . . . . . . . . . . . 75 C.5. Conclusions . . . . . . . . . . . . . . . . . . . . . . . 75 C.6. Block Size Choice . . . . . . . . . . . . . . . . . . . . 76
Appendix D. Data Fields for Traffic Regeneration . . . . . . . . 77 D.1. Recommended Fields for Traffic Regeneration . . . . . . . 77 D.2. Issues with Small Data Captures . . . . . . . . . . . . . 77 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 78 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 79
Appendix D. Data Fields for Traffic Regeneration . . . . . . . . 77 D.1. Recommended Fields for Traffic Regeneration . . . . . . . 77 D.2. Issues with Small Data Captures . . . . . . . . . . . . . 77 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 78 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 79
There has long been a need for server operators to collect DNS Queries and Responses on authoritative and recursive name servers for monitoring and analysis. This data is used in a number of ways, including traffic monitoring, analyzing network attacks, and "day in the life" (DITL) [ditl] analysis.
长期以来,服务器运营商需要在权威和递归名称服务器上收集DNS查询和响应,以便进行监视和分析。这些数据有多种用途,包括流量监测、分析网络攻击和“日常生活”(DITL)[DITL]分析。
A wide variety of tools already exist that facilitate the collection of DNS traffic data, such as the DNS Statistics Collector (DSC) [dsc], packetq [packetq], dnscap [dnscap], and dnstap [dnstap]. However, there is no standard exchange format for large DNS packet captures. The PCAP ("packet capture") [pcap] format or the PCAP Next Generation (PCAP-NG) [pcapng] format is typically used in practice for packet captures, but these file formats can contain a great deal of additional information that is not directly pertinent to DNS traffic analysis and thus unnecessarily increases the capture file size. Additionally, these tools and formats typically have no filter mechanism to selectively record only certain fields at capture time, requiring post-processing for anonymization or pseudonymization of data to protect user privacy.
已经存在各种各样的工具来促进DNS流量数据的收集,例如DNS统计收集器(DSC)[DSC]、packetq[packetq]、dnscap[dnscap]和dnstap[dnstap]。但是,没有用于大型DNS数据包捕获的标准交换格式。PCAP(“数据包捕获”)[PCAP]格式或PCAP下一代(PCAP-NG)[pcapng]格式通常在实践中用于数据包捕获,但这些文件格式可能包含大量与DNS流量分析不直接相关的附加信息,因此不必要地增加捕获文件的大小。此外,这些工具和格式通常没有过滤机制,在捕获时仅选择性地记录某些字段,需要对数据进行匿名或假名化后处理,以保护用户隐私。
There has also been work on using text-based formats to describe DNS packets (for example, see [dnsxml] and [RFC8427]), but this work is largely aimed at producing convenient representations of single messages.
也有关于使用基于文本的格式来描述DNS数据包的工作(例如,请参见[dnsxml]和[RFC8427]),但这项工作主要是为了方便地表示单个消息。
Many DNS operators may receive hundreds of thousands of Queries per second on a single name server instance, so a mechanism to minimize the storage and transmission size (and therefore upload overhead) of the data collected is highly desirable.
许多DNS运营商每秒可能会在单个名称服务器实例上接收数十万个查询,因此非常需要一种机制来最小化所收集数据的存储和传输大小(从而减少上传开销)。
The format described in this document, C-DNS (Compacted-DNS), focuses on the problem of capturing and storing large packet capture files of DNS traffic with the following goals in mind:
本文档中描述的格式C-DNS(压缩DNS)侧重于捕获和存储DNS流量的大型数据包捕获文件的问题,并牢记以下目标:
o Minimize the file size for storage and transmission.
o 最小化存储和传输的文件大小。
o Minimize the overhead of producing the packet capture file and the cost of any further (general-purpose) compression of the file.
o 将生成数据包捕获文件的开销和进一步(通用)压缩文件的成本降至最低。
This document contains:
本文件包括:
o A discussion of some common use cases in which DNS data is collected; see Section 3.
o 讨论收集DNS数据的一些常见用例;见第3节。
o A discussion of the major design considerations in developing an efficient data representation for collections of DNS messages; see Section 4.
o 讨论为DNS消息集合开发有效数据表示时的主要设计考虑事项;见第4节。
o A description of why the Concise Binary Object Representation (CBOR) [RFC7049] was chosen for this format; see Section 5.
o 说明为何为该格式选择简明二进制对象表示法(CBOR)[RFC7049];见第5节。
o A conceptual overview of the C-DNS format; see Section 6.
o C-DNS格式的概念概述;见第6节。
o The definition of the C-DNS format for the collection of DNS messages; see Section 7.
o 用于收集DNS消息的C-DNS格式的定义;见第7节。
o Notes on converting C-DNS data to PCAP format; see Section 9.
o 关于将C-DNS数据转换为PCAP格式的说明;见第9节。
o Some high-level implementation considerations for applications designed to produce C-DNS; see Section 10.
o 设计用于产生C-DNS的应用程序的一些高级实现注意事项;见第10节。
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.
本文件中的关键词“必须”、“不得”、“必需”、“应”、“不应”、“建议”、“不建议”、“可”和“可选”在所有大写字母出现时(如图所示)应按照BCP 14[RFC2119][RFC8174]所述进行解释。
"Packet" refers to an individual IPv4 or IPv6 packet. Typically, packets are UDP datagrams, but such packets may also be part of a TCP data stream. "Message", unless otherwise qualified, refers to a DNS payload extracted from a UDP datagram or a TCP data stream.
“数据包”指单个IPv4或IPv6数据包。通常,数据包是UDP数据报,但此类数据包也可能是TCP数据流的一部分。除非另有限定,“消息”是指从UDP数据报或TCP数据流提取的DNS有效负载。
The parts of DNS messages are named as they are in [RFC1035]. Specifically, the DNS message has five sections: Header, Question, Answer, Authority, and Additional.
DNS消息的部分按[RFC1035]中的名称命名。具体来说,DNS消息有五个部分:标题、问题、答案、权限和附加信息。
From a purely server operator perspective, collecting full packet captures of all packets going into or out of a name server provides the most comprehensive picture of network activity. However, there are several design choices or other limitations that are common to many DNS installations and operators.
从纯服务器运营商的角度来看,收集进入或离开名称服务器的所有数据包的完整数据包捕获提供了最全面的网络活动图片。但是,许多DNS安装和运营商都有一些设计选择或其他限制。
o DNS servers are hosted in a variety of situations:
o DNS服务器托管在多种情况下:
* Self-hosted servers
* 自托管服务器
* Third-party hosting (including multiple third parties)
* 第三方托管(包括多个第三方)
* Third-party hardware (including multiple third parties)
* 第三方硬件(包括多个第三方)
o Data is collected under different conditions:
o 在不同条件下收集数据:
* On well-provisioned servers running in a steady state
* 在配置良好的服务器上以稳定状态运行
* On heavily loaded servers
* 在负载沉重的服务器上
* On virtualized servers
* 在虚拟化服务器上
* On servers that are under DoS attack
* 在受到DoS攻击的服务器上
* On servers that are unwitting intermediaries in DoS attacks
* 在DoS攻击中作为不知情中介的服务器上
o Traffic can be collected via a variety of mechanisms:
o 可以通过多种机制收集流量:
* Within the name server implementation itself
* 在名称服务器实现本身中
* On the same hardware as the name server itself
* 在与名称服务器本身相同的硬件上
* Using a network tap on an adjacent host to listen to DNS traffic
* 在相邻主机上使用网络点击监听DNS流量
* Using port mirroring to listen from another host
* 使用端口镜像从另一台主机侦听
o The capabilities of data collection (and upload) networks vary:
o 数据收集(和上传)网络的功能各不相同:
* Out-of-band networks with the same capacity as the in-band network
* 与带内网络容量相同的带外网络
* Out-of-band networks with less capacity than the in-band network
* 容量小于带内网络的带外网络
* Everything being on the in-band network
* 一切都在带内网络上
Thus, there is a wide range of use cases, from very limited data collection environments (third-party hardware, servers that are under attack, packet capture on the name server itself and no out-of-band network) to "limitless" environments (self-hosted, well-provisioned servers, using a network tap or port mirroring with out-of-band networks with the same capacity as the in-band network). In the
因此,从非常有限的数据收集环境(第三方硬件、受到攻击的服务器、名称服务器本身上的数据包捕获以及没有带外网络)到“无限”环境,存在着广泛的用例(自托管、资源调配良好的服务器,使用网络分接或端口镜像,带外网络的容量与带内网络的容量相同)
former case, it is infeasible to reliably collect full packet captures, especially if the server is under attack. In the latter case, collection of full packet captures may be reasonable.
在前一种情况下,可靠地收集完整的数据包捕获是不可行的,尤其是当服务器受到攻击时。在后一种情况下,收集完整数据包捕获可能是合理的。
As a result of these restrictions, the C-DNS data format is designed with the most limited use case in mind, such that:
由于这些限制,C-DNS数据格式的设计考虑了最有限的用例,例如:
o Data collection will occur on the same hardware as the name server itself
o 数据收集将在与名称服务器本身相同的硬件上进行
o Collected data will be stored on the same hardware as the name server itself, at least temporarily
o 收集的数据将与名称服务器本身存储在同一硬件上,至少是暂时存储
o Collected data being returned to some central analysis system will use the same network interface as the DNS Queries and Responses
o 返回到某个中央分析系统的收集数据将使用与DNS查询和响应相同的网络接口
o There can be multiple third-party servers involved
o 可能涉及多个第三方服务器
Because of these considerations, a major factor in the design of the format is minimal storage size of the capture files.
由于这些考虑,格式设计中的一个主要因素是捕获文件的最小存储大小。
Another significant consideration for any application that records DNS traffic is that the running of the name server software and the transmission of DNS Queries and Responses are the most important jobs of a name server; capturing data is not. Any data collection system co-located with the name server needs to be intelligent enough to carefully manage its CPU, disk, memory, and network utilization. This leads to designing a format that requires a relatively low overhead to produce and minimizes the requirement for further potentially costly compression.
记录DNS流量的任何应用程序的另一个重要考虑因素是名称服务器软件的运行以及DNS查询和响应的传输是名称服务器最重要的工作;捕获数据并非易事。任何与名称服务器共存的数据采集系统都需要足够智能,以便仔细管理其CPU、磁盘、内存和网络利用率。这将导致设计一种格式,该格式需要相对较低的开销来生成,并最大限度地减少对进一步的潜在昂贵压缩的需求。
However, it is also essential that interoperability with less restricted infrastructure is maintained. In particular, it is highly desirable that the collection format should facilitate the re-creation of common formats (such as PCAP) that are as close to the original as is realistic, given the restrictions above.
然而,保持与限制较少的基础设施的互操作性也是至关重要的。特别是,考虑到上述限制,收集格式应该有助于重新创建尽可能接近原始格式的通用格式(如PCAP)。
This section presents some of the major design considerations used in the development of the C-DNS format.
本节介绍了C-DNS格式开发中使用的一些主要设计注意事项。
1. The basic unit of data is a combined DNS Query and the associated Response (a "Query/Response (Q/R) data item"). The same structure will be used for unmatched Queries and Responses. Queries without Responses will be captured omitting the Response data. Responses without Queries will be captured omitting the Query data (but using the Question section from the Response, if present, as an identifying QNAME).
1. 数据的基本单位是一个组合的DNS查询和相关的响应(“查询/响应(Q/R)数据项”)。相同的结构将用于不匹配的查询和响应。将捕获没有响应的查询,而忽略响应数据。将捕获不带查询的响应,忽略查询数据(但使用响应中的问题部分(如果存在)作为标识QNAME)。
* Rationale: A Query and the associated Response represent the basic level of a client's interaction with the server. Also, combining the Query and Response into one item often reduces storage requirements due to commonality in the data of the two messages.
* 理由:查询和相关响应表示客户端与服务器交互的基本级别。此外,由于两条消息的数据具有共性,因此将查询和响应组合到一个项目中通常会减少存储需求。
In the context of generating a C-DNS file, it is assumed that only those DNS payloads that can be parsed to produce a well-formed DNS message are stored in the structured Query/ Response data items of the C-DNS format and that all other messages will (optionally) be recorded as separate malformed messages. Parsing a well-formed message means, at a minimum, the following:
在生成C-DNS文件的上下文中,假设只有那些可被解析以生成格式良好的DNS消息的DNS有效负载存储在C-DNS格式的结构化查询/响应数据项中,并且所有其他消息(可选)将被记录为单独的格式错误的消息。解析格式良好的消息至少意味着:
* The packet has a well-formed 12-byte DNS Header with a recognized OPCODE.
* 该数据包具有格式良好的12字节DNS报头,带有可识别的操作码。
* The section counts are consistent with the section contents.
* 章节数量与章节内容一致。
* All of the Resource Records (RRs) can be fully parsed.
* 所有资源记录(RRs)都可以完全解析。
2. All top-level fields in each Query/Response data item will be optional.
2. 每个查询/响应数据项中的所有顶级字段都是可选的。
* Rationale: Different operators will have different requirements for data to be available for analysis. Operators with minimal requirements should not have to pay the cost of recording full data, though this will limit the ability to perform certain kinds of data analysis and also to reconstruct packet captures. For example, omitting the RRs from a Response will reduce the C-DNS file size; in principle, Responses can be synthesized if there is enough context. Operators may have different policies for collecting user data and can choose to omit or anonymize certain fields at capture time, e.g., client address.
* 理由:不同的操作员对可用于分析的数据有不同的要求。具有最低要求的操作员不必支付记录完整数据的成本,尽管这将限制执行某些类型的数据分析以及重构数据包捕获的能力。例如,从响应中省略RRs将减小C-DNS文件的大小;原则上,如果有足够的上下文,可以合成响应。运营商可能有不同的收集用户数据的策略,并且可以选择在捕获时省略或匿名某些字段,例如客户端地址。
3. Multiple Query/Response data items will be collected into blocks in the format. Common data in a block will be abstracted and referenced from individual Query/Response data items by indexing. The maximum number of Query/Response data items in a block will be configurable.
3. 多个查询/响应数据项将按以下格式收集到块中。块中的公共数据将通过索引从单个查询/响应数据项中提取和引用。可配置块中查询/响应数据项的最大数量。
* Rationale: This blocking and indexing action provides a significant reduction in the volume of file data generated. Although this introduces complexity, it provides compression of the data that makes use of knowledge of the DNS message structure.
* 理由:这种阻塞和索引操作大大减少了生成的文件数据量。尽管这引入了复杂性,但它利用DNS消息结构的知识提供了数据压缩。
* It is anticipated that the files produced can be subject to further compression using general-purpose compression tools. Measurements show that blocking significantly reduces the CPU required to perform such strong compression. See Appendix C.2.
* 预计生成的文件可以使用通用压缩工具进行进一步压缩。测量结果表明,阻塞显著减少了执行这种强压缩所需的CPU。见附录C.2。
* Examples of commonality between DNS messages are that in most cases the QUESTION RR is the same in the Query and Response and that there is a finite set of Query "signatures" (based on a subset of attributes). For many authoritative servers, there is very likely to be a finite set of Responses that are generated, of which a large number are NXDOMAIN.
* DNS消息之间的共性示例是,在大多数情况下,问题RR在查询和响应中是相同的,并且存在一组有限的查询“签名”(基于属性子集)。对于许多权威服务器,很可能会生成有限的响应集,其中大量响应是在域中生成的。
4. Traffic metadata can optionally be included in each block. Specifically, counts of some types of non-DNS packets (e.g., ICMP, TCP resets) sent to the server may be of interest.
4. 流量元数据可以选择性地包含在每个块中。具体地说,发送到服务器的某些类型的非DNS数据包(例如,ICMP、TCP重置)的计数可能是令人感兴趣的。
5. The wire-format content of malformed DNS messages may optionally be recorded.
5. 可以选择性地记录格式错误的DNS消息的有线格式内容。
* Rationale: Any structured capture format that does not capture the DNS payload byte for byte will be limited to some extent in that it cannot represent malformed DNS messages. Only those messages that can be fully parsed and transformed into the structured format can be fully represented. Note, however, that this can result in rather misleading statistics. For example, a malformed Query that cannot be represented in the C-DNS format will lead to the (well-formed) DNS Response with error code FORMERR appearing as "unmatched". Therefore, it can greatly aid downstream analysis to have the wire format of the malformed DNS messages available directly in the C-DNS file.
* 理由:任何不逐字节捕获DNS有效负载的结构化捕获格式都会受到一定程度的限制,因为它不能表示格式错误的DNS消息。只有那些可以完全解析并转换为结构化格式的消息才能完全表示。然而,请注意,这可能会导致误导性的统计数据。例如,无法以C-DNS格式表示的格式错误的查询将导致(格式正确的)DNS响应,错误代码FORMERR显示为“不匹配”。因此,在C-DNS文件中直接提供格式错误的DNS消息的有线格式可以极大地帮助下游分析。
This document presents a detailed format description for C-DNS. The format uses CBOR [RFC7049].
本文档提供了C-DNS的详细格式说明。该格式使用CBOR[RFC7049]。
The choice of CBOR was made taking a number of factors into account.
CBOR的选择考虑了许多因素。
o CBOR is a binary representation and thus is economical in storage space.
o CBOR是一种二进制表示,因此在存储空间上是经济的。
o Other binary representations were investigated, and whilst all had attractive features, none had a significant advantage over CBOR. See Appendix C for some discussion of this.
o 研究了其他二进制表示法,虽然所有表示法都具有吸引人的特征,但没有一种表示法比CBOR具有显著优势。有关这方面的一些讨论,请参见附录C。
o CBOR is an IETF specification and is familiar to IETF participants. It is based on the now-common ideas of lists and objects and thus requires very little familiarization for those in the wider industry.
o CBOR是IETF规范,IETF参与者熟悉。它是基于现在对列表和对象的常见想法,因此对更广泛行业的人来说几乎不需要熟悉。
o CBOR is a simple format and can easily be implemented from scratch if necessary. Formats that are more complex require library support, which may present problems on unusual platforms.
o CBOR是一种简单的格式,如果需要,可以很容易地从头开始实现。更复杂的格式需要库支持,这在不寻常的平台上可能会出现问题。
o CBOR can also be easily converted to text formats such as JSON [RFC8259] for debugging and other human inspection requirements.
o CBOR还可以轻松转换为JSON[RFC8259]等文本格式,用于调试和其他人工检查需求。
o CBOR data schemas can be described using the Concise Data Definition Language (CDDL) [RFC8610].
o CBOR数据模式可以使用简明数据定义语言(CDDL)[RFC8610]来描述。
The following figures show purely schematic representations of the C-DNS format to convey the high-level structure of the C-DNS format. Section 7 provides a detailed discussion of the CBOR representation and individual elements.
下图显示了C-DNS格式的纯示意图,以传达C-DNS格式的高级结构。第7节详细讨论了CBOR表示和各个要素。
Figure 1 shows the C-DNS format at the top level, including the file header and data blocks. The Query/Response data items, Address/Event Count data items, and Malformed Message data items link to various Block Tables.
图1显示了顶层的C-DNS格式,包括文件头和数据块。查询/响应数据项、地址/事件计数数据项和格式错误的消息数据项链接到各种块表。
+-------+ + C-DNS | +-------+--------------------------+ | File Type Identifier | +----------------------------------+ | File Preamble | | +--------------------------------+ | | Format Version | | +--------------------------------+ | | Block Parameters | +-+--------------------------------+ | Block | | +--------------------------------+ | | Block Preamble | | +--------------------------------+ | | Block Statistics | | +--------------------------------+ | | Block Tables | | +--------------------------------+ | | Query/Response data items | | +--------------------------------+ | | Address/Event Count data items | | +--------------------------------+ | | Malformed Message data items | +-+--------------------------------+ | Block | | +--------------------------------+ | | Block Preamble | | +--------------------------------+ | | Block Statistics | | +--------------------------------+ | | Block Tables | | +--------------------------------+ | | Query/Response data items | | +--------------------------------+ | | Address/Event Count data items | | +--------------------------------+ | | Malformed Message data items | +-+--------------------------------+ | Further Blocks... | +----------------------------------+
+-------+ + C-DNS | +-------+--------------------------+ | File Type Identifier | +----------------------------------+ | File Preamble | | +--------------------------------+ | | Format Version | | +--------------------------------+ | | Block Parameters | +-+--------------------------------+ | Block | | +--------------------------------+ | | Block Preamble | | +--------------------------------+ | | Block Statistics | | +--------------------------------+ | | Block Tables | | +--------------------------------+ | | Query/Response data items | | +--------------------------------+ | | Address/Event Count data items | | +--------------------------------+ | | Malformed Message data items | +-+--------------------------------+ | Block | | +--------------------------------+ | | Block Preamble | | +--------------------------------+ | | Block Statistics | | +--------------------------------+ | | Block Tables | | +--------------------------------+ | | Query/Response data items | | +--------------------------------+ | | Address/Event Count data items | | +--------------------------------+ | | Malformed Message data items | +-+--------------------------------+ | Further Blocks... | +----------------------------------+
Figure 1: The C-DNS Format
图1:C-DNS格式
Figure 2 shows some more-detailed relationships within each Block, specifically those between the Query/Response data item and the relevant Block Tables. Some fields have been omitted for clarity.
图2显示了每个块内的一些更详细的关系,特别是查询/响应数据项和相关块表之间的关系。为了清楚起见,省略了一些字段。
+----------------+ | Query/Response | +-------------------------+ | Time Offset | +-------------------------+ +------------------+ | Client Address |---------+->| IP Address array | +-------------------------+ | +------------------+ | Client Port | | +-------------------------+ | +------------------+ | Transaction ID | +---)->| Name/RDATA array |<--------+ +-------------------------+ | | +------------------+ | | Query Signature |--+ | | | +-------------------------+ | | | +-----------------+ | | Client Hoplimit (q) | +--)---)->| Query Signature | | +-------------------------+ | | +-----------------+-------+ | | Response Delay (r) | | +--| Server Address | | +-------------------------+ | +-------------------------+ | | Query Name |--+--+ | Server Port | | +-------------------------+ | +-------------------------+ | | Query Size (q) | | | Transport Flags | | +-------------------------+ | +-------------------------+ | | Response Size (r) | | | QR Type | | +-------------------------+ | +-------------------------+ | | Response Processing (r) | | | QR Signature Flags | | | +-----------------------+ | +-------------------------+ | | | Bailiwick |--+ | Query OPCODE (q) | | | +-----------------------+ +-------------------------+ | | | Flags | | QR DNS Flags | | +-+-----------------------+ +-------------------------+ | | Extra Query Info (q) | | Query RCODE (q) | | | +-----------------------+ +-------------------------+ | | | Question |--+---+ +--+-Query Class/Type (q) | | | +-----------------------+ | | +-------------------------+ | | | Answer |--+ | | | Query QDCOUNT (q) | | | +-----------------------+ | | | +-------------------------+ | | | Authority |--+ | | | Query ANCOUNT (q) | | | +-----------------------+ | | | +-------------------------+ | | | Additional |--+ | | | Query NSCOUNT (q) | |
+----------------+ | Query/Response | +-------------------------+ | Time Offset | +-------------------------+ +------------------+ | Client Address |---------+->| IP Address array | +-------------------------+ | +------------------+ | Client Port | | +-------------------------+ | +------------------+ | Transaction ID | +---)->| Name/RDATA array |<--------+ +-------------------------+ | | +------------------+ | | Query Signature |--+ | | | +-------------------------+ | | | +-----------------+ | | Client Hoplimit (q) | +--)---)->| Query Signature | | +-------------------------+ | | +-----------------+-------+ | | Response Delay (r) | | +--| Server Address | | +-------------------------+ | +-------------------------+ | | Query Name |--+--+ | Server Port | | +-------------------------+ | +-------------------------+ | | Query Size (q) | | | Transport Flags | | +-------------------------+ | +-------------------------+ | | Response Size (r) | | | QR Type | | +-------------------------+ | +-------------------------+ | | Response Processing (r) | | | QR Signature Flags | | | +-----------------------+ | +-------------------------+ | | | Bailiwick |--+ | Query OPCODE (q) | | | +-----------------------+ +-------------------------+ | | | Flags | | QR DNS Flags | | +-+-----------------------+ +-------------------------+ | | Extra Query Info (q) | | Query RCODE (q) | | | +-----------------------+ +-------------------------+ | | | Question |--+---+ +--+-Query Class/Type (q) | | | +-----------------------+ | | +-------------------------+ | | | Answer |--+ | | | Query QDCOUNT (q) | | | +-----------------------+ | | | +-------------------------+ | | | Authority |--+ | | | Query ANCOUNT (q) | | | +-----------------------+ | | | +-------------------------+ | | | Additional |--+ | | | Query NSCOUNT (q) | |
+-+-----------------------+ | | | +-------------------------+ | | Extra Response Info (r) | |-+ | | | Query ARCOUNT (q) | | | +-----------------------+ | | | | +-------------------------+ | | | Answer |--+ | | | | Query EDNS version (q) | | | +-----------------------+ | | | | +-------------------------+ | | | Authority |--+ | | | | Query EDNS UDP Size (q) | | | +-----------------------+ | | | | +-------------------------+ | | | Additional |--+ | | | | Query OPT RDATA (q) |--+ +-+-----------------------+ | | | +-------------------------+ | | | | | Response RCODE (r) | | | | | +-------------------------+ | + -----------------------------+ | +----------+ | | | | | | + -----------------------------+ | | | | +---------------+ +----------+ | | | +->| Question List |->| Question | | | | | array | | array | | | | +---------------+ +----------+--+ | | | | Name |--+-----)--------------------+ | +-------------+ | | +------------+ | | Class/Type |--)---+-+->| Class/Type | | +-------------+ | | | array | | | | +------------+--+ | | | | CLASS | | +---------------+ +----------+ | | +---------------+ +--->| RR List array |->| RR array | | | | TYPE | +---------+-----+ +----------+--+ | | +---------------+ | Name |--+ | +-------------+ | | Class/Type |------+ +-------------+
+-+-----------------------+ | | | +-------------------------+ | | Extra Response Info (r) | |-+ | | | Query ARCOUNT (q) | | | +-----------------------+ | | | | +-------------------------+ | | | Answer |--+ | | | | Query EDNS version (q) | | | +-----------------------+ | | | | +-------------------------+ | | | Authority |--+ | | | | Query EDNS UDP Size (q) | | | +-----------------------+ | | | | +-------------------------+ | | | Additional |--+ | | | | Query OPT RDATA (q) |--+ +-+-----------------------+ | | | +-------------------------+ | | | | | Response RCODE (r) | | | | | +-------------------------+ | + -----------------------------+ | +----------+ | | | | | | + -----------------------------+ | | | | +---------------+ +----------+ | | | +->| Question List |->| Question | | | | | array | | array | | | | +---------------+ +----------+--+ | | | | Name |--+-----)--------------------+ | +-------------+ | | +------------+ | | Class/Type |--)---+-+->| Class/Type | | +-------------+ | | | array | | | | +------------+--+ | | | | CLASS | | +---------------+ +----------+ | | +---------------+ +--->| RR List array |->| RR array | | | | TYPE | +---------+-----+ +----------+--+ | | +---------------+ | Name |--+ | +-------------+ | | Class/Type |------+ +-------------+
Figure 2: The Query/Response Data Item and Subsidiary Tables
图2:查询/响应数据项和子表
In Figure 2, data items annotated (q) are only present when a Query/Response has a Query, and those annotated (r) are only present when a Query/Response Response is present.
在图2中,注释(q)的数据项仅在查询/响应有查询时出现,注释(r)的数据项仅在查询/响应出现时出现。
A C-DNS file begins with a file header containing a File Type Identifier and a File Preamble. The File Preamble contains information on the file Format Version and an array of Block Parameters items (the contents of which include Collection and Storage Parameters used for one or more Blocks).
C-DNS文件以包含文件类型标识符和文件前导的文件头开始。文件序言包含有关文件格式版本的信息和块参数项数组(其内容包括用于一个或多个块的收集和存储参数)。
The file header is followed by a series of Blocks.
文件头后面跟着一系列块。
A Block consists of a Block Preamble item, some Block Statistics for the traffic stored within the Block, and then various arrays of common data collectively called the Block Tables. This is then followed by an array of the Query/Response data items detailing the Queries and Responses stored within the Block. The array of Query/Response data items is in turn followed by the Address/Event Count data items (an array of per-client counts of particular IP events) and then Malformed Message data items (an array of malformed messages that are stored in the Block).
一个块由一个块前导项、存储在该块中的流量的一些块统计信息,以及各种公共数据数组(统称为块表)组成。然后是查询/响应数据项数组,详细说明块中存储的查询和响应。查询/响应数据项的数组依次是地址/事件计数数据项(特定IP事件的每个客户端计数的数组)和格式错误的消息数据项(存储在块中的格式错误的消息数组)。
The exact nature of the DNS data will affect what Block size is the best fit; however, sample data for a root server indicated that Block sizes up to 10,000 Query/Response data items give good results. See Appendix C.6 for more details.
DNS数据的确切性质将影响最适合的块大小;但是,根服务器的示例数据表明,块大小高达10000个查询/响应数据项可以提供良好的结果。详见附录C.6。
This design exploits data commonality and block-based storage to minimize the C-DNS file size. As a result, C-DNS cannot be streamed below the level of a Block.
此设计利用数据通用性和基于块的存储来最小化C-DNS文件大小。因此,C-DNS不能在块级别以下进行流式传输。
The details of the Block Parameters items are not shown in the diagrams but are discussed here for context.
图中未显示块参数项的详细信息,但此处将根据上下文进行讨论。
An array of Block Parameters items is stored in the File Preamble (with a minimum of one item at index 0); a Block Parameters item consists of a collection of Storage and Collection Parameters that applies to any given Block. An array is used in order to support use cases such as wanting to merge C-DNS files from different sources. The Block Preamble item then contains an optional index for the Block Parameters item that applies for that Block; if not present, the index defaults to 0. Hence, in effect, a global Block Parameters item is defined that can then be overridden per Block.
块参数项的数组存储在文件前导中(索引0处至少有一项);块参数项由应用于任何给定块的存储参数和集合参数的集合组成。阵列用于支持用例,例如希望合并来自不同来源的C-DNS文件。然后,块前导项包含适用于该块的块参数项的可选索引;如果不存在,则索引默认为0。因此,实际上,定义了一个全局块参数项,然后可以在每个块上覆盖该项。
The Block Parameters item includes a Storage Parameters item -- this contains information about the specific data fields stored in the C-DNS file.
块参数项包括存储参数项——它包含有关存储在C-DNS文件中的特定数据字段的信息。
These parameters include:
这些参数包括:
o The sub-second timing resolution used by the data.
o 数据使用的次秒计时分辨率。
o Information (hints) on which optional data are omitted. See Section 6.2.1.
o 省略可选数据的信息(提示)。见第6.2.1节。
o Recorded OPCODES [opcodes] and RR TYPEs [rrtypes]. See Section 6.2.2.
o 记录的操作码[操作码]和RR类型[rrtypes]。见第6.2.2节。
o Flags indicating, for example, whether the data is sampled or anonymized. See Sections 6.2.3 and 14.
o 例如,指示数据是采样还是匿名的标志。见第6.2.3和14节。
o Client and server IPv4 and IPv6 address prefixes. See Section 6.2.4.
o 客户端和服务器IPv4和IPv6地址前缀。见第6.2.4节。
To enable implementations to store data to their precise requirements in as space-efficient a manner as possible, all fields in the following arrays are optional:
为了使实现能够以尽可能节省空间的方式存储符合其精确要求的数据,以下阵列中的所有字段都是可选的:
o Query/Response
o 查询/答复
o Query Signature
o 查询签名
o Malformed Messages
o 格式错误的消息
In other words, an implementation can choose to omit any data item that is not required for its use case (whilst observing the restrictions relating to IP address storage described in Section 6.2.4). In addition, implementations may be configured to not record all RRs or to only record messages with certain OPCODES.
换句话说,实现可以选择省略其用例不需要的任何数据项(同时遵守第6.2.4节中描述的与IP地址存储相关的限制)。此外,可以将实现配置为不记录所有rr或仅记录具有特定操作码的消息。
This does, however, mean that a consumer of a C-DNS file faces two problems:
然而,这确实意味着C-DNS文件的使用者面临两个问题:
1. How can it quickly determine if a file definitely does not contain the data items it requires to complete a particular task (e.g., reconstructing DNS traffic or performing a specific piece of data analysis)?
1. 它如何快速确定文件是否确实不包含完成特定任务所需的数据项(例如,重建DNS流量或执行特定的数据分析)?
2. How can it determine whether a data item is not present because it was (1) explicitly not recorded or (2) not available/present?
2. 它如何确定某个数据项是否不存在,因为它(1)明确未记录或(2)不可用/存在?
For example, capturing C-DNS data from within a name server implementation makes it unlikely that the Client Hoplimit can be recorded. Or, if there is no Query ARCOUNT recorded and no Query OPT RDATA [RFC6891] recorded, is that because no Query contained an OPT RR, or because that data was not stored?
例如,从名称服务器实现中捕获C-DNS数据使得不太可能记录客户端跃点限制。或者,如果没有记录查询ARCOUNT,也没有记录查询OPT RDATA[RFC6891],是因为没有查询包含OPT RR,还是因为没有存储该数据?
The Storage Parameters item therefore also contains a Storage Hints item, which specifies which items the encoder of the file omits from the stored data and will therefore never be present. (This approach is taken because a flag that indicated which items were included for
因此,存储参数项还包含一个存储提示项,该项指定文件的编码器从存储的数据中忽略了哪些项,因此这些项永远不会出现。(之所以采用这种方法,是因为有一个标志指示哪些项目包括在
collection would not guarantee that the item was present -- only that it might be.) An implementation decoding that file can then use these flags to quickly determine whether the input data is not rich enough for its needs.
集合不能保证该项存在——只能保证它可能存在。)解码该文件的实现然后可以使用这些标志快速确定输入数据是否不够丰富,无法满足其需要。
One scenario where this may be particularly important is the case of regenerating traffic. It is possible to collect such a small set of data items that an implementation decoding the file cannot determine if a given Query/Response data item was generated from just a Query, just a Response, or a Query/Response pair. This makes it impossible to reconstruct DNS traffic even if sensible defaults are provided for the missing data items. This is discussed in more detail in Section 9.
这可能特别重要的一个场景是重新生成流量的情况。可以收集如此小的数据项集,以至于解码文件的实现无法确定给定的查询/响应数据项是仅从查询、响应还是查询/响应对生成的。这使得即使为丢失的数据项提供了合理的默认值,也无法重建DNS流量。第9节对此进行了更详细的讨论。
Also included in the Storage Parameters item are explicit arrays listing the RR TYPEs and the OPCODEs to be recorded. These arrays remove any ambiguity over whether, for example, messages containing particular OPCODEs are not present because (1) certain OPCODEs did not occur or (2) the implementation is not configured to record them.
存储参数项中还包括显式数组,其中列出了要记录的RR类型和操作码。例如,由于(1)某些操作码未出现或(2)实现未配置为记录它们,这些数组消除了关于是否不存在包含特定操作码的消息的任何歧义。
In the case of OPCODEs, for a message to be fully parsable, the OPCODE must be known to the collecting implementation. Any message with an OPCODE unknown to the collecting implementation cannot be validated as correctly formed and so must be treated as malformed. Messages with OPCODES known to the recording application but not listed in the Storage Parameters item are discarded by the recording application during C-DNS capture (regardless of whether they are malformed or not).
对于操作码,要使消息完全可解析,收集实现必须知道操作码。任何带有收集实现未知操作码的消息都无法验证为格式正确,因此必须视为格式错误。在C-DNS捕获期间,记录应用程序将丢弃操作码为记录应用程序已知但未在存储参数项中列出的消息(无论其格式是否错误)。
In the case of RRs, each record in a message must be fully parsable, including parsing the record RDATA, as otherwise the message cannot be validated as correctly formed. Any RR with an RR TYPE not known to the collecting implementation cannot be validated as correctly formed and so must be treated as malformed.
对于RRs,消息中的每个记录都必须是完全可解析的,包括解析记录RDATA,否则无法验证消息的格式是否正确。收集实现不知道RR类型的任何RR都无法验证为正确格式,因此必须视为格式不正确。
Once a message is correctly parsed, an implementation is free to record only a subset of the RRs present.
一旦消息被正确解析,实现就可以自由地只记录当前RRs的一个子集。
The Storage Parameters item contains flags that can be used to indicate if:
存储参数项包含可用于指示以下情况的标志:
o the data is anonymized,
o 数据是匿名的,
o the data is produced from sample data, or
o 数据由样本数据生成,或
o names in the data have been normalized (converted to uniform case).
o 数据中的名称已规范化(转换为统一大小写)。
The Storage Parameters item also contains optional fields holding details of the sampling method used and the anonymization method used. It is RECOMMENDED that these fields contain URIs [RFC3986] pointing to resources describing the methods used. See Section 14 for further discussion of anonymization and normalization.
存储参数项还包含可选字段,其中包含所用采样方法和匿名方法的详细信息。建议这些字段包含指向描述所用方法的资源的URI[RFC3986]。有关匿名化和规范化的进一步讨论,请参见第14节。
The format can store either full IP addresses or just IP prefixes; the Storage Parameters item contains fields to indicate if only IP prefixes were stored.
该格式可以存储完整的IP地址,也可以只存储IP前缀;存储参数项包含用于指示是否仅存储IP前缀的字段。
If the IP address prefixes are absent, then full addresses are stored. In this case, the IP version can be directly inferred from the stored address length and the fields "qr-transport-flags" in QueryResponseSignature, "ae-transport-flags" in AddressEventCount, and "mm-transport-flags" in MalformedMessageData (which contain the IP version bit) are optional.
如果没有IP地址前缀,则存储完整地址。在这种情况下,可以从存储的地址长度直接推断IP版本,QueryResponseSignature中的字段“qr transport flags”、“AddressEventCount中的ae transport flags”和MalformedMessageData中的字段“mm transport flags”(包含IP版本位)是可选的。
If IP address prefixes are given, only the prefix bits of addresses are stored. In this case, in order to determine the IP version, the fields "qr-transport-flags" in QueryResponseSignature, "ae-transport-flags" in AddressEventCount, and "mm-transport-flags" in MalformedMessageData MUST be present. See Sections 7.3.2.3.2 and 7.3.2.3.5.
如果给定IP地址前缀,则只存储地址的前缀位。在这种情况下,为了确定IP版本,QueryResponseSignature中的字段“qr传输标志”、AddressEventCount中的“ae传输标志”和MalformedMessageData中的“mm传输标志”必须存在。见第7.3.2.3.2节和第7.3.2.3.5节。
As an example of storing only IP prefixes, if a client IPv6 prefix of 48 is specified, a client address of 2001:db8:85a3::8a2e:370:7334 will be stored as 0x20010db885a3, reducing address storage space requirements. Similarly, if a client IPv4 prefix of 16 is specified, a client address of 192.0.2.1 will be stored as 0xc000 (192.0).
作为仅存储IP前缀的示例,如果指定了客户端IPv6前缀48,则2001:db8:85a3::8a2e:370:7334的客户端地址将存储为0x20010db885a3,从而减少地址存储空间需求。类似地,如果指定了客户端IPv4前缀16,则192.0.2.1的客户端地址将存储为0xc000(192.0)。
The CDDL definition for the C-DNS format is given in Appendix A.
附录A中给出了C-DNS格式的CDDL定义。
All map keys are integers with values specified in the CDDL. String keys would significantly bloat the file size.
所有映射键都是CDDL中指定值的整数。字符串键会显著增大文件大小。
All key values specified are positive integers under 24, so their CBOR representation is a single byte. Positive integer values not currently used as keys in a map are reserved for use in future standard extensions.
所有指定的键值都是24以下的正整数,因此它们的CBOR表示为单字节。当前未在映射中用作键的正整数值保留在将来的标准扩展中使用。
Implementations may choose to add additional implementation-specific entries to any map. Negative integer map keys are reserved for these values. Key values from -1 to -24 also have a single-byte CBOR representation, so such implementation-specific extensions are not at any space efficiency disadvantage.
实现可以选择向任何映射添加其他特定于实现的条目。为这些值保留负整数映射键。从-1到-24的键值也有一个单字节CBOR表示,因此这种特定于实现的扩展不存在任何空间效率的缺点。
An item described as an index is the index of the data item in the referenced array. Indexes are 0-based.
描述为索引的项是引用数组中数据项的索引。索引是基于0的。
The following sections present the C-DNS specification in tabular format with a detailed description of each item.
以下各节以表格形式呈现了C-DNS规范,并对每个项目进行了详细说明。
In all quantities that contain bit flags, bit 0 indicates the least significant bit, i.e., flag "n" in quantity "q" is on if "(q & (1 << n)) != 0".
在包含位标志的所有量中,位0表示最低有效位,即,如果(q&(1<<n))!=0,则数量“q”中的标志“n”处于打开状态。
For the sake of readability, all type and field names defined in the CDDL definition are shown in double quotes. Type names are by convention camel case (e.g., "BlockTables"), and field names are lowercase with hyphens (e.g., "block-tables").
为了可读性,CDDL定义中定义的所有类型和字段名都用双引号表示。按照惯例,类型名是大小写(例如“块表”),字段名是小写加连字符(例如“块表”)。
For the sake of brevity, the following conventions are used in the tables:
为简洁起见,表中使用了以下约定:
o The column M marks whether items in a map are mandatory.
o M列标记地图中的项目是否为必填项。
* X - Mandatory items.
* X-强制性项目。
* C - Conditionally mandatory items. Such items are usually optional but may be mandatory in some configurations.
* C-有条件的强制性项目。这些项目通常是可选的,但在某些配置中可能是强制性的。
* If the column is empty, the item is optional.
* 如果列为空,则该项为可选项。
o The column T gives the CBOR datatype of the item.
o 列T给出了该项的CBOR数据类型。
* U - Unsigned integer.
* U-无符号整数。
* I - Signed integer (i.e., either a CBOR unsigned integer or a CBOR negative integer).
* I-有符号整数(即,CBOR无符号整数或CBOR负整数)。
* B - Boolean.
* B-布尔型。
* S - Byte string.
* S字节字符串。
* T - Text string.
* T-文本字符串。
* M - Map.
* M-地图。
* A - Array.
* A-数组。
In the case of maps and arrays, more information on the type of each value, including the CDDL definition name if applicable, is given in the description.
对于映射和数组,描述中给出了关于每个值类型的更多信息,包括CDDL定义名称(如果适用)。
A C-DNS file has an outer structure "File", an array that contains the following:
C-DNS文件具有外部结构“文件”,即包含以下内容的数组:
+---------------+---+---+-------------------------------------------+ | Field | M | T | Description | +---------------+---+---+-------------------------------------------+ | file-type-id | X | T | String "C-DNS" identifying the file type. | | | | | | | file-preamble | X | M | Version and parameter information for the | | | | | whole file. Map of type "FilePreamble"; | | | | | see Section 7.3.1. | | | | | | | file-blocks | X | A | Array of items of type "Block"; see | | | | | Section 7.3.2. The array may be empty if | | | | | the file contains no data. | +---------------+---+---+-------------------------------------------+
+---------------+---+---+-------------------------------------------+ | Field | M | T | Description | +---------------+---+---+-------------------------------------------+ | file-type-id | X | T | String "C-DNS" identifying the file type. | | | | | | | file-preamble | X | M | Version and parameter information for the | | | | | whole file. Map of type "FilePreamble"; | | | | | see Section 7.3.1. | | | | | | | file-blocks | X | A | Array of items of type "Block"; see | | | | | Section 7.3.2. The array may be empty if | | | | | the file contains no data. | +---------------+---+---+-------------------------------------------+
Information about data in the file. A map containing the following:
有关文件中数据的信息。包含以下内容的地图:
+----------------------+---+---+------------------------------------+ | Field | M | T | Description | +----------------------+---+---+------------------------------------+ | major-format-version | X | U | Unsigned integer "1". The major | | | | | version of the format used in the | | | | | file. See Section 8. | | | | | | | minor-format-version | X | U | Unsigned integer "0". The minor | | | | | version of the format used in the | | | | | file. See Section 8. | | | | | | | private-version | | U | Version indicator available for | | | | | private use by implementations. | | | | | | | block-parameters | X | A | Array of items of type | | | | | "BlockParameters". See Section | | | | | 7.3.1.1. The array must contain | | | | | at least one entry. (The | | | | | "block-parameters-index" item in | | | | | each "BlockPreamble" indicates | | | | | which array entry applies to that | | | | | "Block".) | +----------------------+---+---+------------------------------------+
+----------------------+---+---+------------------------------------+ | Field | M | T | Description | +----------------------+---+---+------------------------------------+ | major-format-version | X | U | Unsigned integer "1". The major | | | | | version of the format used in the | | | | | file. See Section 8. | | | | | | | minor-format-version | X | U | Unsigned integer "0". The minor | | | | | version of the format used in the | | | | | file. See Section 8. | | | | | | | private-version | | U | Version indicator available for | | | | | private use by implementations. | | | | | | | block-parameters | X | A | Array of items of type | | | | | "BlockParameters". See Section | | | | | 7.3.1.1. The array must contain | | | | | at least one entry. (The | | | | | "block-parameters-index" item in | | | | | each "BlockPreamble" indicates | | | | | which array entry applies to that | | | | | "Block".) | +----------------------+---+---+------------------------------------+
Parameters relating to data storage and collection that apply to one or more items of type "Block". A map containing the following:
适用于一个或多个“块”类型项的与数据存储和收集相关的参数。包含以下内容的地图:
+-----------------------+---+---+-----------------------------------+ | Field | M | T | Description | +-----------------------+---+---+-----------------------------------+ | storage-parameters | X | M | Parameters relating to data | | | | | storage in a "Block" item. Map | | | | | of type "StorageParameters"; see | | | | | Section 7.3.1.1.1. | | | | | | | collection-parameters | | M | Parameters relating to collection | | | | | of the data in a "Block" item. | | | | | Map of type | | | | | "CollectionParameters"; see | | | | | Section 7.3.1.1.2. | +-----------------------+---+---+-----------------------------------+
+-----------------------+---+---+-----------------------------------+ | Field | M | T | Description | +-----------------------+---+---+-----------------------------------+ | storage-parameters | X | M | Parameters relating to data | | | | | storage in a "Block" item. Map | | | | | of type "StorageParameters"; see | | | | | Section 7.3.1.1.1. | | | | | | | collection-parameters | | M | Parameters relating to collection | | | | | of the data in a "Block" item. | | | | | Map of type | | | | | "CollectionParameters"; see | | | | | Section 7.3.1.1.2. | +-----------------------+---+---+-----------------------------------+
Parameters relating to how data is stored in the items of type "Block". A map containing the following:
与数据如何存储在“块”类型项中有关的参数。包含以下内容的地图:
+------------------+---+---+----------------------------------------+ | Field | M | T | Description | +------------------+---+---+----------------------------------------+ | ticks-per-second | X | U | Sub-second timing is recorded in | | | | | ticks. This specifies the number of | | | | | ticks in a second. | | | | | | | max-block-items | X | U | The maximum number of items stored in | | | | | any of the arrays in a "Block" item | | | | | (Q/R, Address/Event Count, or | | | | | Malformed Message data items). An | | | | | indication to a decoder of the | | | | | resources needed to process the file. | | | | | | | storage-hints | X | M | Collection of hints as to which fields | | | | | are omitted in the arrays that have | | | | | optional fields. Map of type | | | | | "StorageHints". See Section | | | | | 7.3.1.1.1.1. | | | | | | | opcodes | X | A | Array of OPCODES [opcodes] (unsigned | | | | | integers, each in the range 0 to 15 | | | | | inclusive) recorded by the collecting | | | | | implementation. See Section 6.2.2. | | | | | | | rr-types | X | A | Array of RR TYPEs [rrtypes] (unsigned | | | | | integers, each in the range 0 to 65535 | | | | | inclusive) recorded by the collecting | | | | | implementation. See Section 6.2.2. | | | | | | | storage-flags | | U | Bit flags indicating attributes of | | | | | stored data. | | | | | Bit 0. 1 if the data has been | | | | | anonymized. | | | | | Bit 1. 1 if the data is sampled data. | | | | | Bit 2. 1 if the names have been | | | | | normalized (converted to uniform | | | | | case). | | | | | | | client-address | | U | IPv4 client address prefix length, in | | -prefix-ipv4 | | | the range 1 to 32 inclusive. If | | | | | specified, only the address prefix | | | | | bits are stored. |
+------------------+---+---+----------------------------------------+ | Field | M | T | Description | +------------------+---+---+----------------------------------------+ | ticks-per-second | X | U | Sub-second timing is recorded in | | | | | ticks. This specifies the number of | | | | | ticks in a second. | | | | | | | max-block-items | X | U | The maximum number of items stored in | | | | | any of the arrays in a "Block" item | | | | | (Q/R, Address/Event Count, or | | | | | Malformed Message data items). An | | | | | indication to a decoder of the | | | | | resources needed to process the file. | | | | | | | storage-hints | X | M | Collection of hints as to which fields | | | | | are omitted in the arrays that have | | | | | optional fields. Map of type | | | | | "StorageHints". See Section | | | | | 7.3.1.1.1.1. | | | | | | | opcodes | X | A | Array of OPCODES [opcodes] (unsigned | | | | | integers, each in the range 0 to 15 | | | | | inclusive) recorded by the collecting | | | | | implementation. See Section 6.2.2. | | | | | | | rr-types | X | A | Array of RR TYPEs [rrtypes] (unsigned | | | | | integers, each in the range 0 to 65535 | | | | | inclusive) recorded by the collecting | | | | | implementation. See Section 6.2.2. | | | | | | | storage-flags | | U | Bit flags indicating attributes of | | | | | stored data. | | | | | Bit 0. 1 if the data has been | | | | | anonymized. | | | | | Bit 1. 1 if the data is sampled data. | | | | | Bit 2. 1 if the names have been | | | | | normalized (converted to uniform | | | | | case). | | | | | | | client-address | | U | IPv4 client address prefix length, in | | -prefix-ipv4 | | | the range 1 to 32 inclusive. If | | | | | specified, only the address prefix | | | | | bits are stored. |
| | | | | | client-address | | U | IPv6 client address prefix length, in | | -prefix-ipv6 | | | the range 1 to 128 inclusive. If | | | | | specified, only the address prefix | | | | | bits are stored. | | | | | | | server-address | | U | IPv4 server address prefix length, in | | -prefix-ipv4 | | | the range 1 to 32 inclusive. If | | | | | specified, only the address prefix | | | | | bits are stored. | | | | | | | server-address | | U | IPv6 server address prefix length, in | | -prefix-ipv6 | | | the range 1 to 128 inclusive. If | | | | | specified, only the address prefix | | | | | bits are stored. | | | | | | | sampling-method | | T | Information on the sampling method | | | | | used. See Section 6.2.3. | | | | | | | anonymization | | T | Information on the anonymization | | -method | | | method used. See Section 6.2.3. | +------------------+---+---+----------------------------------------+
| | | | | | client-address | | U | IPv6 client address prefix length, in | | -prefix-ipv6 | | | the range 1 to 128 inclusive. If | | | | | specified, only the address prefix | | | | | bits are stored. | | | | | | | server-address | | U | IPv4 server address prefix length, in | | -prefix-ipv4 | | | the range 1 to 32 inclusive. If | | | | | specified, only the address prefix | | | | | bits are stored. | | | | | | | server-address | | U | IPv6 server address prefix length, in | | -prefix-ipv6 | | | the range 1 to 128 inclusive. If | | | | | specified, only the address prefix | | | | | bits are stored. | | | | | | | sampling-method | | T | Information on the sampling method | | | | | used. See Section 6.2.3. | | | | | | | anonymization | | T | Information on the anonymization | | -method | | | method used. See Section 6.2.3. | +------------------+---+---+----------------------------------------+
An indicator of which fields the collecting implementation omits in the maps with optional fields. Note that hints have a top-down precedence. In other words, where a map contains another map, the hint on the containing map overrides any hints in the contained map and the contained map is omitted. A map containing the following:
在带有可选字段的映射中,收集实现忽略哪些字段的指示符。请注意,提示具有自上而下的优先级。换句话说,当一个映射包含另一个映射时,包含映射上的提示将覆盖包含映射中的任何提示,并且忽略包含映射。包含以下内容的地图:
+------------------+---+---+----------------------------------------+ | Field | M | T | Description | +------------------+---+---+----------------------------------------+ | query-response | X | U | Hints indicating which "QueryResponse" | | -hints | | | fields are omitted; see Section | | | | | 7.3.2.4. If a bit is unset, the field | | | | | is omitted from the capture. | | | | | Bit 0. time-offset | | | | | Bit 1. client-address-index | | | | | Bit 2. client-port | | | | | Bit 3. transaction-id | | | | | Bit 4. qr-signature-index | | | | | Bit 5. client-hoplimit | | | | | Bit 6. response-delay | | | | | Bit 7. query-name-index | | | | | Bit 8. query-size | | | | | Bit 9. response-size |
+------------------+---+---+----------------------------------------+ | Field | M | T | Description | +------------------+---+---+----------------------------------------+ | query-response | X | U | Hints indicating which "QueryResponse" | | -hints | | | fields are omitted; see Section | | | | | 7.3.2.4. If a bit is unset, the field | | | | | is omitted from the capture. | | | | | Bit 0. time-offset | | | | | Bit 1. client-address-index | | | | | Bit 2. client-port | | | | | Bit 3. transaction-id | | | | | Bit 4. qr-signature-index | | | | | Bit 5. client-hoplimit | | | | | Bit 6. response-delay | | | | | Bit 7. query-name-index | | | | | Bit 8. query-size | | | | | Bit 9. response-size |
| | | | Bit 10. response-processing-data | | | | | Bit 11. query-question-sections | | | | | Bit 12. query-answer-sections | | | | | Bit 13. query-authority-sections | | | | | Bit 14. query-additional-sections | | | | | Bit 15. response-answer-sections | | | | | Bit 16. response-authority-sections | | | | | Bit 17. response-additional-sections | | | | | | | query-response | X | U | Hints indicating which | | -signature-hints | | | "QueryResponseSignature" fields are | | | | | omitted; see Section 7.3.2.3.2. If a | | | | | bit is unset, the field is omitted | | | | | from the capture. | | | | | Bit 0. server-address-index | | | | | Bit 1. server-port | | | | | Bit 2. qr-transport-flags | | | | | Bit 3. qr-type | | | | | Bit 4. qr-sig-flags | | | | | Bit 5. query-opcode | | | | | Bit 6. qr-dns-flags | | | | | Bit 7. query-rcode | | | | | Bit 8. query-classtype-index | | | | | Bit 9. query-qdcount | | | | | Bit 10. query-ancount | | | | | Bit 11. query-nscount | | | | | Bit 12. query-arcount | | | | | Bit 13. query-edns-version | | | | | Bit 14. query-udp-size | | | | | Bit 15. query-opt-rdata-index | | | | | Bit 16. response-rcode | | | | | | | rr-hints | X | U | Hints indicating which optional "RR" | | | | | fields are omitted; see Section | | | | | 7.3.2.3.4. If a bit is unset, the | | | | | field is omitted from the capture. | | | | | Bit 0. ttl | | | | | Bit 1. rdata-index | | other-data-hints | X | U | Hints indicating which other datatypes | | | | | are omitted. If a bit is unset, the | | | | | datatype is omitted from the capture. | | | | | Bit 0. malformed-messages | | | | | Bit 1. address-event-counts | +------------------+---+---+----------------------------------------+
| | | | Bit 10. response-processing-data | | | | | Bit 11. query-question-sections | | | | | Bit 12. query-answer-sections | | | | | Bit 13. query-authority-sections | | | | | Bit 14. query-additional-sections | | | | | Bit 15. response-answer-sections | | | | | Bit 16. response-authority-sections | | | | | Bit 17. response-additional-sections | | | | | | | query-response | X | U | Hints indicating which | | -signature-hints | | | "QueryResponseSignature" fields are | | | | | omitted; see Section 7.3.2.3.2. If a | | | | | bit is unset, the field is omitted | | | | | from the capture. | | | | | Bit 0. server-address-index | | | | | Bit 1. server-port | | | | | Bit 2. qr-transport-flags | | | | | Bit 3. qr-type | | | | | Bit 4. qr-sig-flags | | | | | Bit 5. query-opcode | | | | | Bit 6. qr-dns-flags | | | | | Bit 7. query-rcode | | | | | Bit 8. query-classtype-index | | | | | Bit 9. query-qdcount | | | | | Bit 10. query-ancount | | | | | Bit 11. query-nscount | | | | | Bit 12. query-arcount | | | | | Bit 13. query-edns-version | | | | | Bit 14. query-udp-size | | | | | Bit 15. query-opt-rdata-index | | | | | Bit 16. response-rcode | | | | | | | rr-hints | X | U | Hints indicating which optional "RR" | | | | | fields are omitted; see Section | | | | | 7.3.2.3.4. If a bit is unset, the | | | | | field is omitted from the capture. | | | | | Bit 0. ttl | | | | | Bit 1. rdata-index | | other-data-hints | X | U | Hints indicating which other datatypes | | | | | are omitted. If a bit is unset, the | | | | | datatype is omitted from the capture. | | | | | Bit 0. malformed-messages | | | | | Bit 1. address-event-counts | +------------------+---+---+----------------------------------------+
Parameters providing information regarding how data in the file was collected (applicable for some, but not all, collection environments). The values are informational only and serve as metadata to downstream analyzers as to the configuration of a collecting implementation. They can provide context when interpreting what data is present/absent from the capture but cannot necessarily be validated against the data captured.
提供有关如何收集文件中数据的信息的参数(适用于某些收集环境,但不适用于所有收集环境)。这些值仅供参考,并作为下游分析器关于收集实现配置的元数据。它们可以在解释捕获中存在/不存在的数据时提供上下文,但不一定能够根据捕获的数据进行验证。
These parameters have no default. If they do not appear, nothing can be inferred about their value.
这些参数没有默认值。如果它们不出现,则无法推断其值。
A map containing the following items:
包含以下项目的地图:
+------------------+---+---+----------------------------------------+ | Field | M | T | Description | +------------------+---+---+----------------------------------------+ | query-timeout | | U | To be matched with a Query, a Response | | | | | must arrive within this number of | | | | | milliseconds. | | | | | | | skew-timeout | | U | The network stack may report a | | | | | Response before the corresponding | | | | | Query. A Response is not considered | | | | | to be missing a Query until after this | | | | | many microseconds. | | | | | | | snaplen | | U | Collect up to this many bytes per | | | | | packet. | | | | | | | promisc | | B | "true" if promiscuous mode | | | | | [pcap-options] was enabled on the | | | | | interface, "false" otherwise. | | | | | | | interfaces | | A | Array of identifiers (of type text | | | | | string) of the interfaces used for | | | | | collection. | | | | | | | server-addresses | | A | Array of server collection IP | | | | | addresses (of type byte string). | | | | | Metadata for downstream analyzers; | | | | | does not affect collection. | | | | | |
+------------------+---+---+----------------------------------------+ | Field | M | T | Description | +------------------+---+---+----------------------------------------+ | query-timeout | | U | To be matched with a Query, a Response | | | | | must arrive within this number of | | | | | milliseconds. | | | | | | | skew-timeout | | U | The network stack may report a | | | | | Response before the corresponding | | | | | Query. A Response is not considered | | | | | to be missing a Query until after this | | | | | many microseconds. | | | | | | | snaplen | | U | Collect up to this many bytes per | | | | | packet. | | | | | | | promisc | | B | "true" if promiscuous mode | | | | | [pcap-options] was enabled on the | | | | | interface, "false" otherwise. | | | | | | | interfaces | | A | Array of identifiers (of type text | | | | | string) of the interfaces used for | | | | | collection. | | | | | | | server-addresses | | A | Array of server collection IP | | | | | addresses (of type byte string). | | | | | Metadata for downstream analyzers; | | | | | does not affect collection. | | | | | |
| vlan-ids | | A | Array of identifiers (of type unsigned | | | | | integer, each in the range 1 to 4094 | | | | | inclusive) of VLANs [IEEE802.1Q] | | | | | selected for collection. VLAN IDs are | | | | | unique only within an administrative | | | | | domain. | | | | | | | filter | | T | Filter for input, in "tcpdump" | | | | | [pcap-filter] style. | | | | | | | generator-id | | T | Implementation-specific human-readable | | | | | string identifying the collection | | | | | method. | | | | | | | host-id | | T | String identifying the collecting | | | | | host. | +------------------+---+---+----------------------------------------+
| vlan-ids | | A | Array of identifiers (of type unsigned | | | | | integer, each in the range 1 to 4094 | | | | | inclusive) of VLANs [IEEE802.1Q] | | | | | selected for collection. VLAN IDs are | | | | | unique only within an administrative | | | | | domain. | | | | | | | filter | | T | Filter for input, in "tcpdump" | | | | | [pcap-filter] style. | | | | | | | generator-id | | T | Implementation-specific human-readable | | | | | string identifying the collection | | | | | method. | | | | | | | host-id | | T | String identifying the collecting | | | | | host. | +------------------+---+---+----------------------------------------+
Container for data with common collection and storage parameters. A map containing the following:
具有公共收集和存储参数的数据容器。包含以下内容的地图:
+--------------------+---+---+--------------------------------------+ | Field | M | T | Description | +--------------------+---+---+--------------------------------------+ | block-preamble | X | M | Overall information for the "Block" | | | | | item. Map of type "BlockPreamble"; | | | | | see Section 7.3.2.1. | | | | | | | block-statistics | | M | Statistics about the "Block" item. | | | | | Map of type "BlockStatistics"; see | | | | | Section 7.3.2.2. | | | | | | | block-tables | | M | The arrays containing data | | | | | referenced by individual | | | | | "QueryResponse" or | | | | | "MalformedMessage" items. Map of | | | | | type "BlockTables"; see Section | | | | | 7.3.2.3. | | | | | | | query-responses | | A | Details of individual C-DNS Q/R data | | | | | items. Array of items of type | | | | | "QueryResponse"; see Section | | | | | 7.3.2.4. If present, the array must | | | | | not be empty. | | | | | |
+--------------------+---+---+--------------------------------------+ | Field | M | T | Description | +--------------------+---+---+--------------------------------------+ | block-preamble | X | M | Overall information for the "Block" | | | | | item. Map of type "BlockPreamble"; | | | | | see Section 7.3.2.1. | | | | | | | block-statistics | | M | Statistics about the "Block" item. | | | | | Map of type "BlockStatistics"; see | | | | | Section 7.3.2.2. | | | | | | | block-tables | | M | The arrays containing data | | | | | referenced by individual | | | | | "QueryResponse" or | | | | | "MalformedMessage" items. Map of | | | | | type "BlockTables"; see Section | | | | | 7.3.2.3. | | | | | | | query-responses | | A | Details of individual C-DNS Q/R data | | | | | items. Array of items of type | | | | | "QueryResponse"; see Section | | | | | 7.3.2.4. If present, the array must | | | | | not be empty. | | | | | |
| address-event | | A | Per-client counts of ICMP messages | | -counts | | | and TCP resets. Array of items of | | | | | type "AddressEventCount"; see | | | | | Section 7.3.2.5. If present, the | | | | | array must not be empty. | | | | | | | malformed-messages | | A | Details of malformed DNS messages. | | | | | Array of items of type | | | | | "MalformedMessage"; see Section | | | | | 7.3.2.6. If present, the array must | | | | | not be empty. | +--------------------+---+---+--------------------------------------+
| address-event | | A | Per-client counts of ICMP messages | | -counts | | | and TCP resets. Array of items of | | | | | type "AddressEventCount"; see | | | | | Section 7.3.2.5. If present, the | | | | | array must not be empty. | | | | | | | malformed-messages | | A | Details of malformed DNS messages. | | | | | Array of items of type | | | | | "MalformedMessage"; see Section | | | | | 7.3.2.6. If present, the array must | | | | | not be empty. | +--------------------+---+---+--------------------------------------+
Overall information for a "Block" item. A map containing the following:
“块”项目的总体信息。包含以下内容的地图:
+------------------+---+---+----------------------------------------+ | Field | M | T | Description | +------------------+---+---+----------------------------------------+ | earliest-time | C | A | A timestamp (two unsigned integers, of | | | | | type "Timestamp") for the earliest | | | | | record in the "Block" item. The first | | | | | integer is the number of seconds since | | | | | the POSIX epoch [posix-time] | | | | | ("time_t"), excluding leap seconds. | | | | | The second integer is the number of | | | | | ticks (see Section 7.3.1.1.1) since | | | | | the start of the second. This field | | | | | is mandatory unless all block items | | | | | containing a time offset from the | | | | | start of the Block also omit that time | | | | | offset. | | | | | | | block-parameters | | U | The index of the item in the | | -index | | | "block-parameters" array (in the | | | | | "file-preamble" item) applicable to | | | | | this block. If not present, index 0 | | | | | is used. See Section 7.3.1. | +------------------+---+---+----------------------------------------+
+------------------+---+---+----------------------------------------+ | Field | M | T | Description | +------------------+---+---+----------------------------------------+ | earliest-time | C | A | A timestamp (two unsigned integers, of | | | | | type "Timestamp") for the earliest | | | | | record in the "Block" item. The first | | | | | integer is the number of seconds since | | | | | the POSIX epoch [posix-time] | | | | | ("time_t"), excluding leap seconds. | | | | | The second integer is the number of | | | | | ticks (see Section 7.3.1.1.1) since | | | | | the start of the second. This field | | | | | is mandatory unless all block items | | | | | containing a time offset from the | | | | | start of the Block also omit that time | | | | | offset. | | | | | | | block-parameters | | U | The index of the item in the | | -index | | | "block-parameters" array (in the | | | | | "file-preamble" item) applicable to | | | | | this block. If not present, index 0 | | | | | is used. See Section 7.3.1. | +------------------+---+---+----------------------------------------+
Basic statistical information about a "Block" item. A map containing the following:
“块”项目的基本统计信息。包含以下内容的地图:
+---------------------+---+---+-------------------------------------+ | Field | M | T | Description | +---------------------+---+---+-------------------------------------+ | processed-messages | | U | Total number of well-formed DNS | | | | | messages processed from the input | | | | | traffic stream during collection of | | | | | data in this "Block" item. | | | | | | | qr-data-items | | U | Total number of Q/R data items in | | | | | this "Block" item. | | | | | | | unmatched-queries | | U | Number of unmatched Queries in this | | | | | "Block" item. | | | | | | | unmatched-responses | | U | Number of unmatched Responses in | | | | | this "Block" item. | | | | | | | discarded-opcode | | U | Number of DNS messages processed | | | | | from the input traffic stream | | | | | during collection of data in this | | | | | "Block" item but not recorded | | | | | because their OPCODE is not in the | | | | | list to be collected. | | | | | | | malformed-items | | U | Number of malformed messages | | | | | processed from the input traffic | | | | | stream during collection of data in | | | | | this "Block" item. | +---------------------+---+---+-------------------------------------+
+---------------------+---+---+-------------------------------------+ | Field | M | T | Description | +---------------------+---+---+-------------------------------------+ | processed-messages | | U | Total number of well-formed DNS | | | | | messages processed from the input | | | | | traffic stream during collection of | | | | | data in this "Block" item. | | | | | | | qr-data-items | | U | Total number of Q/R data items in | | | | | this "Block" item. | | | | | | | unmatched-queries | | U | Number of unmatched Queries in this | | | | | "Block" item. | | | | | | | unmatched-responses | | U | Number of unmatched Responses in | | | | | this "Block" item. | | | | | | | discarded-opcode | | U | Number of DNS messages processed | | | | | from the input traffic stream | | | | | during collection of data in this | | | | | "Block" item but not recorded | | | | | because their OPCODE is not in the | | | | | list to be collected. | | | | | | | malformed-items | | U | Number of malformed messages | | | | | processed from the input traffic | | | | | stream during collection of data in | | | | | this "Block" item. | +---------------------+---+---+-------------------------------------+
Map of arrays containing data referenced by individual "QueryResponse" or "MalformedMessage" items in this "Block". Each element is an array that, if present, must not be empty.
包含此“块”中单个“QueryResponse”或“MalformedMessage”项引用的数据的数组的映射。每个元素都是一个数组,如果存在,则该数组不能为空。
An item in the "qlist" array contains indexes to values in the "qrr" array. Therefore, if "qlist" is present, "qrr" must also be present. Similarly, if "rrlist" is present, "rr" must also be present.
“qlist”数组中的项包含指向“qrr”数组中的值的索引。因此,如果“qlist”存在,“qrr”也必须存在。同样,如果“rrlist”存在,“rr”也必须存在。
The map contains the following items:
地图包含以下项目:
+-------------------+---+---+---------------------------------------+ | Field | M | T | Description | +-------------------+---+---+---------------------------------------+ | ip-address | | A | Array of IP addresses, in network | | | | | byte order (of type byte string). If | | | | | client or server address prefixes are | | | | | set, only the address prefix bits are | | | | | stored. Each string is therefore up | | | | | to 4 bytes long for an IPv4 address, | | | | | or up to 16 bytes long for an IPv6 | | | | | address. See Section 7.3.1.1.1. | | | | | | | classtype | | A | Array of RR CLASS and TYPE | | | | | information. Type is "ClassType". | | | | | See Section 7.3.2.3.1. | | | | | | | name-rdata | | A | Array where each entry is the | | | | | contents of a single NAME or RDATA in | | | | | wire format (of type byte string). | | | | | Note that NAMEs, and labels within | | | | | RDATA contents, are full domain names | | | | | or labels; no name compression (per | | | | | [RFC1035]) is used on the individual | | | | | names/labels within the format. | | | | | | | qr-sig | | A | Array of Q/R data item signatures. | | | | | Type is "QueryResponseSignature". | | | | | See Section 7.3.2.3.2. | | | | | | | qlist | | A | Array of type "QuestionList". A | | | | | "QuestionList" is an array of | | | | | unsigned integers, indexes to | | | | | "Question" items in the "qrr" array. | | | | | |
+-------------------+---+---+---------------------------------------+ | Field | M | T | Description | +-------------------+---+---+---------------------------------------+ | ip-address | | A | Array of IP addresses, in network | | | | | byte order (of type byte string). If | | | | | client or server address prefixes are | | | | | set, only the address prefix bits are | | | | | stored. Each string is therefore up | | | | | to 4 bytes long for an IPv4 address, | | | | | or up to 16 bytes long for an IPv6 | | | | | address. See Section 7.3.1.1.1. | | | | | | | classtype | | A | Array of RR CLASS and TYPE | | | | | information. Type is "ClassType". | | | | | See Section 7.3.2.3.1. | | | | | | | name-rdata | | A | Array where each entry is the | | | | | contents of a single NAME or RDATA in | | | | | wire format (of type byte string). | | | | | Note that NAMEs, and labels within | | | | | RDATA contents, are full domain names | | | | | or labels; no name compression (per | | | | | [RFC1035]) is used on the individual | | | | | names/labels within the format. | | | | | | | qr-sig | | A | Array of Q/R data item signatures. | | | | | Type is "QueryResponseSignature". | | | | | See Section 7.3.2.3.2. | | | | | | | qlist | | A | Array of type "QuestionList". A | | | | | "QuestionList" is an array of | | | | | unsigned integers, indexes to | | | | | "Question" items in the "qrr" array. | | | | | |
| qrr | | A | Array of type "Question". Each entry | | | | | is the contents of a single Question, | | | | | where a Question is the second or | | | | | subsequent Question in a Query. See | | | | | Section 7.3.2.3.3. | | | | | | | rrlist | | A | Array of type "RRList". An "RRList" | | | | | is an array of unsigned integers, | | | | | indexes to "RR" items in the "rr" | | | | | array. | | | | | | | rr | | A | Array of type "RR". Each entry is | | | | | the contents of a single RR. See | | | | | Section 7.3.2.3.4. | | | | | | | malformed-message | | A | Array of the contents of malformed | | -data | | | messages. Array of type | | | | | "MalformedMessageData". See Section | | | | | 7.3.2.3.5. | +-------------------+---+---+---------------------------------------+
| qrr | | A | Array of type "Question". Each entry | | | | | is the contents of a single Question, | | | | | where a Question is the second or | | | | | subsequent Question in a Query. See | | | | | Section 7.3.2.3.3. | | | | | | | rrlist | | A | Array of type "RRList". An "RRList" | | | | | is an array of unsigned integers, | | | | | indexes to "RR" items in the "rr" | | | | | array. | | | | | | | rr | | A | Array of type "RR". Each entry is | | | | | the contents of a single RR. See | | | | | Section 7.3.2.3.4. | | | | | | | malformed-message | | A | Array of the contents of malformed | | -data | | | messages. Array of type | | | | | "MalformedMessageData". See Section | | | | | 7.3.2.3.5. | +-------------------+---+---+---------------------------------------+
RR CLASS and TYPE information. A map containing the following:
RR类和类型信息。包含以下内容的地图:
+-------+---+---+--------------------------+ | Field | M | T | Description | +-------+---+---+--------------------------+ | type | X | U | TYPE value [rrtypes]. | | | | | | | class | X | U | CLASS value [rrclasses]. | +-------+---+---+--------------------------+
+-------+---+---+--------------------------+ | Field | M | T | Description | +-------+---+---+--------------------------+ | type | X | U | TYPE value [rrtypes]. | | | | | | | class | X | U | CLASS value [rrclasses]. | +-------+---+---+--------------------------+
Elements of a Q/R data item that are often common between multiple individual Q/R data items. A map containing the following:
通常在多个单独Q/R数据项之间通用的Q/R数据项的元素。包含以下内容的地图:
+--------------------+---+---+--------------------------------------+ | Field | M | T | Description | +--------------------+---+---+--------------------------------------+ | server-address | | U | The index in the "ip-address" array | | -index | | | of the server IP address. See | | | | | Section 7.3.2.3. | | | | | | | server-port | | U | The server port. | | | | | | | qr-transport-flags | C | U | Bit flags describing the transport | | | | | used to service the Query. Same | | | | | definition as "mm-transport-flags" | | | | | in Section 7.3.2.3.5, with an | | | | | additional indicator for trailing | | | | | bytes. See Appendix A. | | | | | Bit 0. IP version. 0 if IPv4, 1 if | | | | | IPv6. See Section 6.2.4. | | | | | Bits 1-4. Transport. 4-bit | | | | | unsigned value where | | | | | 0 = UDP [RFC1035] | | | | | 1 = TCP [RFC1035] | | | | | 2 = TLS [RFC7858] | | | | | 3 = DTLS [RFC8094] | | | | | 4 = HTTPS [RFC8484] | | | | | 15 = Non-standard transport (see | | | | | below) | | | | | Values 5-14 are reserved for future | | | | | use. | | | | | Bit 5. 1 if trailing bytes in Query | | | | | packet. See Section 11.2. | | | | | | | qr-type | | U | Type of Query/Response transaction | | | | | based on the definitions in the | | | | | dnstap schema [dnstap-schema]. | | | | | 0 = Stub. A transaction between a | | | | | stub resolver and a DNS server from | | | | | the perspective of the stub | | | | | resolver. | | | | | 1 = Client. A transaction between a | | | | | client and a DNS server (a proxy or | | | | | full recursive resolver) from the | | | | | perspective of the DNS server. |
+--------------------+---+---+--------------------------------------+ | Field | M | T | Description | +--------------------+---+---+--------------------------------------+ | server-address | | U | The index in the "ip-address" array | | -index | | | of the server IP address. See | | | | | Section 7.3.2.3. | | | | | | | server-port | | U | The server port. | | | | | | | qr-transport-flags | C | U | Bit flags describing the transport | | | | | used to service the Query. Same | | | | | definition as "mm-transport-flags" | | | | | in Section 7.3.2.3.5, with an | | | | | additional indicator for trailing | | | | | bytes. See Appendix A. | | | | | Bit 0. IP version. 0 if IPv4, 1 if | | | | | IPv6. See Section 6.2.4. | | | | | Bits 1-4. Transport. 4-bit | | | | | unsigned value where | | | | | 0 = UDP [RFC1035] | | | | | 1 = TCP [RFC1035] | | | | | 2 = TLS [RFC7858] | | | | | 3 = DTLS [RFC8094] | | | | | 4 = HTTPS [RFC8484] | | | | | 15 = Non-standard transport (see | | | | | below) | | | | | Values 5-14 are reserved for future | | | | | use. | | | | | Bit 5. 1 if trailing bytes in Query | | | | | packet. See Section 11.2. | | | | | | | qr-type | | U | Type of Query/Response transaction | | | | | based on the definitions in the | | | | | dnstap schema [dnstap-schema]. | | | | | 0 = Stub. A transaction between a | | | | | stub resolver and a DNS server from | | | | | the perspective of the stub | | | | | resolver. | | | | | 1 = Client. A transaction between a | | | | | client and a DNS server (a proxy or | | | | | full recursive resolver) from the | | | | | perspective of the DNS server. |
| | | | 2 = Resolver. A transaction between | | | | | a recursive resolver and an | | | | | authoritative server from the | | | | | perspective of the recursive | | | | | resolver. | | | | | 3 = Authoritative. A transaction | | | | | between a recursive resolver and an | | | | | authoritative server from the | | | | | perspective of the authoritative | | | | | server. | | | | | 4 = Forwarder. A transaction | | | | | between a downstream forwarder and | | | | | an upstream DNS server (a recursive | | | | | resolver) from the perspective of | | | | | the downstream forwarder. | | | | | 5 = Tool. A transaction between a | | | | | DNS software tool and a DNS server, | | | | | from the perspective of the tool. | | | | | | | qr-sig-flags | | U | Bit flags explicitly indicating | | | | | attributes of the message pair | | | | | represented by this Q/R data item | | | | | (not all attributes may be recorded | | | | | or deducible). | | | | | Bit 0. 1 if a Query was present. | | | | | Bit 1. 1 if a Response was present. | | | | | Bit 2. 1 if a Query was present and | | | | | it had an OPT RR. | | | | | Bit 3. 1 if a Response was present | | | | | and it had an OPT RR. | | | | | Bit 4. 1 if a Query was present but | | | | | had no Question. | | | | | Bit 5. 1 if a Response was present | | | | | but had no Question (only one | | | | | query-name-index is stored per Q/R | | | | | data item). | | | | | | | query-opcode | | U | Query OPCODE. | | | | | | | qr-dns-flags | | U | Bit flags with values from the Query | | | | | and Response DNS flags. Flag values | | | | | are 0 if the Query or Response is | | | | | not present. | | | | | Bit 0. Query Checking Disabled | | | | | (CD). | | | | | Bit 1. Query Authenticated Data | | | | | (AD). | | | | | Bit 2. Query reserved (Z). |
| | | | 2 = Resolver. A transaction between | | | | | a recursive resolver and an | | | | | authoritative server from the | | | | | perspective of the recursive | | | | | resolver. | | | | | 3 = Authoritative. A transaction | | | | | between a recursive resolver and an | | | | | authoritative server from the | | | | | perspective of the authoritative | | | | | server. | | | | | 4 = Forwarder. A transaction | | | | | between a downstream forwarder and | | | | | an upstream DNS server (a recursive | | | | | resolver) from the perspective of | | | | | the downstream forwarder. | | | | | 5 = Tool. A transaction between a | | | | | DNS software tool and a DNS server, | | | | | from the perspective of the tool. | | | | | | | qr-sig-flags | | U | Bit flags explicitly indicating | | | | | attributes of the message pair | | | | | represented by this Q/R data item | | | | | (not all attributes may be recorded | | | | | or deducible). | | | | | Bit 0. 1 if a Query was present. | | | | | Bit 1. 1 if a Response was present. | | | | | Bit 2. 1 if a Query was present and | | | | | it had an OPT RR. | | | | | Bit 3. 1 if a Response was present | | | | | and it had an OPT RR. | | | | | Bit 4. 1 if a Query was present but | | | | | had no Question. | | | | | Bit 5. 1 if a Response was present | | | | | but had no Question (only one | | | | | query-name-index is stored per Q/R | | | | | data item). | | | | | | | query-opcode | | U | Query OPCODE. | | | | | | | qr-dns-flags | | U | Bit flags with values from the Query | | | | | and Response DNS flags. Flag values | | | | | are 0 if the Query or Response is | | | | | not present. | | | | | Bit 0. Query Checking Disabled | | | | | (CD). | | | | | Bit 1. Query Authenticated Data | | | | | (AD). | | | | | Bit 2. Query reserved (Z). |
| | | | Bit 3. Query Recursion Available | | | | | (RA). | | | | | Bit 4. Query Recursion Desired | | | | | (RD). | | | | | Bit 5. Query TrunCation (TC). | | | | | Bit 6. Query Authoritative Answer | | | | | (AA). | | | | | Bit 7. Query DNSSEC answer OK (DO). | | | | | Bit 8. Response Checking Disabled | | | | | (CD). | | | | | Bit 9. Response Authenticated Data | | | | | (AD). | | | | | Bit 10. Response reserved (Z). | | | | | Bit 11. Response Recursion | | | | | Available (RA). | | | | | Bit 12. Response Recursion Desired | | | | | (RD). | | | | | Bit 13. Response TrunCation (TC). | | | | | Bit 14. Response Authoritative | | | | | Answer (AA). | | | | | | | query-rcode | | U | Query RCODE. If the Query contains | | | | | an OPT RR [RFC6891], this value | | | | | incorporates any EXTENDED-RCODE | | | | | value [rcodes]. | | | | | | | query-classtype | | U | The index in the "classtype" array | | -index | | | of the CLASS and TYPE of the first | | | | | Question. See Section 7.3.2.3. | | | | | | | query-qdcount | | U | The QDCOUNT in the Query, or | | | | | Response if no Query present. | | | | | | | query-ancount | | U | Query ANCOUNT. | | | | | | | query-nscount | | U | Query NSCOUNT. | | | | | | | query-arcount | | U | Query ARCOUNT. | | | | | | | query-edns-version | | U | The Query EDNS version. ("EDNS" | | | | | stands for Extension Mechanisms for | | | | | DNS.) | | | | | | | query-udp-size | | U | The Query EDNS sender's UDP payload | | | | | size. | | | | | |
| | | | Bit 3. Query Recursion Available | | | | | (RA). | | | | | Bit 4. Query Recursion Desired | | | | | (RD). | | | | | Bit 5. Query TrunCation (TC). | | | | | Bit 6. Query Authoritative Answer | | | | | (AA). | | | | | Bit 7. Query DNSSEC answer OK (DO). | | | | | Bit 8. Response Checking Disabled | | | | | (CD). | | | | | Bit 9. Response Authenticated Data | | | | | (AD). | | | | | Bit 10. Response reserved (Z). | | | | | Bit 11. Response Recursion | | | | | Available (RA). | | | | | Bit 12. Response Recursion Desired | | | | | (RD). | | | | | Bit 13. Response TrunCation (TC). | | | | | Bit 14. Response Authoritative | | | | | Answer (AA). | | | | | | | query-rcode | | U | Query RCODE. If the Query contains | | | | | an OPT RR [RFC6891], this value | | | | | incorporates any EXTENDED-RCODE | | | | | value [rcodes]. | | | | | | | query-classtype | | U | The index in the "classtype" array | | -index | | | of the CLASS and TYPE of the first | | | | | Question. See Section 7.3.2.3. | | | | | | | query-qdcount | | U | The QDCOUNT in the Query, or | | | | | Response if no Query present. | | | | | | | query-ancount | | U | Query ANCOUNT. | | | | | | | query-nscount | | U | Query NSCOUNT. | | | | | | | query-arcount | | U | Query ARCOUNT. | | | | | | | query-edns-version | | U | The Query EDNS version. ("EDNS" | | | | | stands for Extension Mechanisms for | | | | | DNS.) | | | | | | | query-udp-size | | U | The Query EDNS sender's UDP payload | | | | | size. | | | | | |
| query-opt-rdata | | U | The index in the "name-rdata" array | | -index | | | of the OPT RDATA. See Section | | | | | 7.3.2.3. | | | | | | | response-rcode | | U | Response RCODE. If the Response | | | | | contains an OPT RR [RFC6891], this | | | | | value incorporates any EXTENDED- | | | | | RCODE value [rcodes]. | +--------------------+---+---+--------------------------------------+
| query-opt-rdata | | U | The index in the "name-rdata" array | | -index | | | of the OPT RDATA. See Section | | | | | 7.3.2.3. | | | | | | | response-rcode | | U | Response RCODE. If the Response | | | | | contains an OPT RR [RFC6891], this | | | | | value incorporates any EXTENDED- | | | | | RCODE value [rcodes]. | +--------------------+---+---+--------------------------------------+
Version 1.0 of C-DNS supports transport values corresponding to DNS transports defined in IETF Standards Track documents at the time of writing. There are numerous non-standard methods of sending DNS messages over various transports using a variety of protocols, but they are out of scope for this document. With the current specification, these can be generically stored using value 15 (Non-standard transport), or implementations are free to use the negative integer map keys to define their own mappings. Such non-standard transports may also be the subject of a future extension to the specification.
C-DNS的1.0版支持与编写时IETF标准跟踪文档中定义的DNS传输相对应的传输值。有许多使用各种协议通过各种传输发送DNS消息的非标准方法,但它们不在本文档的范围内。在当前规范中,这些可以使用值15(非标准传输)进行一般性存储,或者实现可以自由使用负整数映射键来定义自己的映射。此类非标准传输也可能是本规范未来扩展的主题。
Details on individual Questions in a Question section. A map containing the following:
在问题部分中详细介绍各个问题。包含以下内容的地图:
+-----------------+---+---+-----------------------------------------+ | Field | M | T | Description | +-----------------+---+---+-----------------------------------------+ | name-index | X | U | The index in the "name-rdata" array of | | | | | the QNAME. See Section 7.3.2.3. | | | | | | | classtype-index | X | U | The index in the "classtype" array of | | | | | the CLASS and TYPE of the Question. | | | | | See Section 7.3.2.3. | +-----------------+---+---+-----------------------------------------+
+-----------------+---+---+-----------------------------------------+ | Field | M | T | Description | +-----------------+---+---+-----------------------------------------+ | name-index | X | U | The index in the "name-rdata" array of | | | | | the QNAME. See Section 7.3.2.3. | | | | | | | classtype-index | X | U | The index in the "classtype" array of | | | | | the CLASS and TYPE of the Question. | | | | | See Section 7.3.2.3. | +-----------------+---+---+-----------------------------------------+
Details on individual RRs in RR sections. A map containing the following:
在RR章节中详细介绍了各个RRs。包含以下内容的地图:
+-----------------+---+---+-----------------------------------------+ | Field | M | T | Description | +-----------------+---+---+-----------------------------------------+ | name-index | X | U | The index in the "name-rdata" array of | | | | | the NAME. See Section 7.3.2.3. | | | | | | | classtype-index | X | U | The index in the "classtype" array of | | | | | the CLASS and TYPE of the RR. See | | | | | Section 7.3.2.3. | | | | | | | ttl | | U | The RR Time to Live. | | | | | | | rdata-index | | U | The index in the "name-rdata" array of | | | | | the RR RDATA. See Section 7.3.2.3. | +-----------------+---+---+-----------------------------------------+
+-----------------+---+---+-----------------------------------------+ | Field | M | T | Description | +-----------------+---+---+-----------------------------------------+ | name-index | X | U | The index in the "name-rdata" array of | | | | | the NAME. See Section 7.3.2.3. | | | | | | | classtype-index | X | U | The index in the "classtype" array of | | | | | the CLASS and TYPE of the RR. See | | | | | Section 7.3.2.3. | | | | | | | ttl | | U | The RR Time to Live. | | | | | | | rdata-index | | U | The index in the "name-rdata" array of | | | | | the RR RDATA. See Section 7.3.2.3. | +-----------------+---+---+-----------------------------------------+
Details on malformed DNS messages stored in this "Block" item. A map containing the following:
有关此“阻止”项中存储的格式错误的DNS消息的详细信息。包含以下内容的地图:
+--------------------+---+---+--------------------------------------+ | Field | M | T | Description | +--------------------+---+---+--------------------------------------+ | server-address | | U | The index in the "ip-address" array | | -index | | | of the server IP address. See | | | | | Section 7.3.2.3. | | | | | | | server-port | | U | The server port. | | | | | | | mm-transport-flags | C | U | Bit flags describing the transport | | | | | used to service the Query. See | | | | | Section 6.2.4. | | | | | Bits 1-4. Transport. 4-bit | | | | | unsigned value where | | | | | 0 = UDP [RFC1035] | | | | | 1 = TCP [RFC1035] | | | | | 2 = TLS [RFC7858] | | | | | 3 = DTLS [RFC8094] | | | | | 4 = HTTPS [RFC8484] | | | | | 15 = Non-standard transport | | | | | Values 5-14 are reserved for future | | | | | use. |
+--------------------+---+---+--------------------------------------+ | Field | M | T | Description | +--------------------+---+---+--------------------------------------+ | server-address | | U | The index in the "ip-address" array | | -index | | | of the server IP address. See | | | | | Section 7.3.2.3. | | | | | | | server-port | | U | The server port. | | | | | | | mm-transport-flags | C | U | Bit flags describing the transport | | | | | used to service the Query. See | | | | | Section 6.2.4. | | | | | Bits 1-4. Transport. 4-bit | | | | | unsigned value where | | | | | 0 = UDP [RFC1035] | | | | | 1 = TCP [RFC1035] | | | | | 2 = TLS [RFC7858] | | | | | 3 = DTLS [RFC8094] | | | | | 4 = HTTPS [RFC8484] | | | | | 15 = Non-standard transport | | | | | Values 5-14 are reserved for future | | | | | use. |
| | | | | | mm-payload | | S | The payload (raw bytes) of the DNS | | | | | message. | +--------------------+---+---+--------------------------------------+
| | | | | | mm-payload | | S | The payload (raw bytes) of the DNS | | | | | message. | +--------------------+---+---+--------------------------------------+
Details on individual Q/R data items.
个别Q/R数据项的详细信息。
Note that there is no requirement that the elements of the "query-responses" array are presented in strict chronological order.
请注意,“查询响应”数组的元素不需要严格按时间顺序显示。
A map containing the following items:
包含以下项目的地图:
+----------------------+---+---+------------------------------------+ | Field | M | T | Description | +----------------------+---+---+------------------------------------+ | time-offset | | U | Q/R timestamp as an offset in | | | | | ticks (see Section 7.3.1.1.1) from | | | | | "earliest-time". The timestamp is | | | | | the timestamp of the Query, or the | | | | | Response if there is no Query. | | | | | | | client-address-index | | U | The index in the "ip-address" | | | | | array of the client IP address. | | | | | See Section 7.3.2.3. | | | | | | | client-port | | U | The client port. | | | | | | | transaction-id | | U | DNS transaction identifier. | | | | | | | qr-signature-index | | U | The index in the "qr-sig" array of | | | | | the "QueryResponseSignature" item. | | | | | See Section 7.3.2.3. | | | | | | | client-hoplimit | | U | The IPv4 TTL or IPv6 Hoplimit from | | | | | the Query packet. | | | | | | | response-delay | | I | The time difference between Query | | | | | and Response, in ticks. See | | | | | Section 7.3.1.1.1. Only present | | | | | if there is a Query and a | | | | | Response. The delay can be | | | | | negative if the network | | | | | stack/capture library returns | | | | | packets out of order. | | | | | |
+----------------------+---+---+------------------------------------+ | Field | M | T | Description | +----------------------+---+---+------------------------------------+ | time-offset | | U | Q/R timestamp as an offset in | | | | | ticks (see Section 7.3.1.1.1) from | | | | | "earliest-time". The timestamp is | | | | | the timestamp of the Query, or the | | | | | Response if there is no Query. | | | | | | | client-address-index | | U | The index in the "ip-address" | | | | | array of the client IP address. | | | | | See Section 7.3.2.3. | | | | | | | client-port | | U | The client port. | | | | | | | transaction-id | | U | DNS transaction identifier. | | | | | | | qr-signature-index | | U | The index in the "qr-sig" array of | | | | | the "QueryResponseSignature" item. | | | | | See Section 7.3.2.3. | | | | | | | client-hoplimit | | U | The IPv4 TTL or IPv6 Hoplimit from | | | | | the Query packet. | | | | | | | response-delay | | I | The time difference between Query | | | | | and Response, in ticks. See | | | | | Section 7.3.1.1.1. Only present | | | | | if there is a Query and a | | | | | Response. The delay can be | | | | | negative if the network | | | | | stack/capture library returns | | | | | packets out of order. | | | | | |
| query-name-index | | U | The index in the "name-rdata" | | | | | array of the item containing the | | | | | QNAME for the first Question. See | | | | | Section 7.3.2.3. | | | | | | | query-size | | U | DNS Query message size (see | | | | | below). | | | | | | | response-size | | U | DNS Response message size (see | | | | | below). | | | | | | | response-processing | | M | Data on Response processing. Map | | -data | | | of type "ResponseProcessingData". | | | | | See Section 7.3.2.4.1. | | | | | | | query-extended | | M | Extended Query data. Map of type | | | | | "QueryResponseExtended". See | | | | | Section 7.3.2.4.2. | | | | | | | response-extended | | M | Extended Response data. Map of | | | | | type "QueryResponseExtended". See | | | | | Section 7.3.2.4.2. | +----------------------+---+---+------------------------------------+
| query-name-index | | U | The index in the "name-rdata" | | | | | array of the item containing the | | | | | QNAME for the first Question. See | | | | | Section 7.3.2.3. | | | | | | | query-size | | U | DNS Query message size (see | | | | | below). | | | | | | | response-size | | U | DNS Response message size (see | | | | | below). | | | | | | | response-processing | | M | Data on Response processing. Map | | -data | | | of type "ResponseProcessingData". | | | | | See Section 7.3.2.4.1. | | | | | | | query-extended | | M | Extended Query data. Map of type | | | | | "QueryResponseExtended". See | | | | | Section 7.3.2.4.2. | | | | | | | response-extended | | M | Extended Response data. Map of | | | | | type "QueryResponseExtended". See | | | | | Section 7.3.2.4.2. | +----------------------+---+---+------------------------------------+
The "query-size" and "response-size" fields hold the DNS message size. For UDP, this is the size of the UDP payload that contained the DNS message. For TCP, it is the size of the DNS message as specified in the two-byte message length header. Trailing bytes in UDP Queries are routinely observed in traffic to authoritative servers, and this value allows a calculation of how many trailing bytes were present.
“查询大小”和“响应大小”字段保存DNS消息大小。对于UDP,这是包含DNS消息的UDP有效负载的大小。对于TCP,它是双字节消息长度标头中指定的DNS消息的大小。UDP查询中的尾随字节通常在到权威服务器的流量中观察到,该值允许计算存在多少尾随字节。
Information on the server processing that produced the Response. A map containing the following:
有关产生响应的服务器处理的信息。包含以下内容的地图:
+------------------+---+---+----------------------------------------+ | Field | M | T | Description | +------------------+---+---+----------------------------------------+ | bailiwick-index | | U | The index in the "name-rdata" array of | | | | | the owner name for the Response | | | | | bailiwick. See Section 7.3.2.3. | | | | | | | processing-flags | | U | Flags relating to Response processing. | | | | | Bit 0. 1 if the Response came from | | | | | cache. | +------------------+---+---+----------------------------------------+
+------------------+---+---+----------------------------------------+ | Field | M | T | Description | +------------------+---+---+----------------------------------------+ | bailiwick-index | | U | The index in the "name-rdata" array of | | | | | the owner name for the Response | | | | | bailiwick. See Section 7.3.2.3. | | | | | | | processing-flags | | U | Flags relating to Response processing. | | | | | Bit 0. 1 if the Response came from | | | | | cache. | +------------------+---+---+----------------------------------------+
Extended data on the Q/R data item.
Q/R数据项上的扩展数据。
Each item in the map is present only if collection of the relevant details is configured.
仅当配置了相关详细信息的集合时,地图中的每个项目才存在。
A map containing the following items:
包含以下项目的地图:
+------------------+---+---+----------------------------------------+ | Field | M | T | Description | +------------------+---+---+----------------------------------------+ | question-index | | U | The index in the "qlist" array of the | | | | | entry listing any second and | | | | | subsequent Questions in the Question | | | | | section for the Query or Response. | | | | | See Section 7.3.2.3. | | | | | | | answer-index | | U | The index in the "rrlist" array of the | | | | | entry listing the Answer RR sections | | | | | for the Query or Response. See | | | | | Section 7.3.2.3. | | | | | | | authority-index | | U | The index in the "rrlist" array of the | | | | | entry listing the Authority RR | | | | | sections for the Query or Response. | | | | | See Section 7.3.2.3. | | | | | | | additional-index | | U | The index in the "rrlist" array of the | | | | | entry listing the Additional RR | | | | | sections for the Query or Response. | | | | | See Section 7.3.2.3. Note that Query | | | | | OPT RR data can optionally be stored | | | | | in the QuerySignature. | +------------------+---+---+----------------------------------------+
+------------------+---+---+----------------------------------------+ | Field | M | T | Description | +------------------+---+---+----------------------------------------+ | question-index | | U | The index in the "qlist" array of the | | | | | entry listing any second and | | | | | subsequent Questions in the Question | | | | | section for the Query or Response. | | | | | See Section 7.3.2.3. | | | | | | | answer-index | | U | The index in the "rrlist" array of the | | | | | entry listing the Answer RR sections | | | | | for the Query or Response. See | | | | | Section 7.3.2.3. | | | | | | | authority-index | | U | The index in the "rrlist" array of the | | | | | entry listing the Authority RR | | | | | sections for the Query or Response. | | | | | See Section 7.3.2.3. | | | | | | | additional-index | | U | The index in the "rrlist" array of the | | | | | entry listing the Additional RR | | | | | sections for the Query or Response. | | | | | See Section 7.3.2.3. Note that Query | | | | | OPT RR data can optionally be stored | | | | | in the QuerySignature. | +------------------+---+---+----------------------------------------+
Counts of various IP-related events relating to traffic with individual client addresses. A map containing the following:
与单个客户端地址的流量相关的各种IP相关事件的计数。包含以下内容的地图:
+--------------------+---+---+--------------------------------------+ | Field | M | T | Description | +--------------------+---+---+--------------------------------------+ | ae-type | X | U | The type of event. The following | | | | | event types are currently defined: | | | | | 0. TCP reset. | | | | | 1. ICMP time exceeded. | | | | | 2. ICMP destination unreachable. | | | | | 3. ICMPv6 time exceeded. | | | | | 4. ICMPv6 destination unreachable. | | | | | 5. ICMPv6 packet too big. | | | | | | | ae-code | | U | A code relating to the event. For | | | | | ICMP or ICMPv6 events, this MUST be | | | | | the ICMP [RFC792] or ICMPv6 | | | | | [RFC4443] code. For other events, | | | | | the contents are undefined. | | | | | | | ae-transport-flags | C | U | Bit flags describing the transport | | | | | used to service the event. See | | | | | Section 6.2.4. | | | | | Bit 0. IP version. 0 if IPv4, 1 if | | | | | IPv6. | | | | | Bits 1-4. Transport. 4-bit | | | | | unsigned value where | | | | | 0 = UDP [RFC1035] | | | | | 1 = TCP [RFC1035] | | | | | 2 = TLS [RFC7858] | | | | | 3 = DTLS [RFC8094] | | | | | 4 = HTTPS [RFC8484] | | | | | 15 = Non-standard transport | | | | | Values 5-14 are reserved for future | | | | | use. | | | | | | | ae-address-index | X | U | The index in the "ip-address" array | | | | | of the client address. See Section | | | | | 7.3.2.3. | | | | | | | ae-count | X | U | The number of occurrences of this | | | | | event during the Block collection | | | | | period. | +--------------------+---+---+--------------------------------------+
+--------------------+---+---+--------------------------------------+ | Field | M | T | Description | +--------------------+---+---+--------------------------------------+ | ae-type | X | U | The type of event. The following | | | | | event types are currently defined: | | | | | 0. TCP reset. | | | | | 1. ICMP time exceeded. | | | | | 2. ICMP destination unreachable. | | | | | 3. ICMPv6 time exceeded. | | | | | 4. ICMPv6 destination unreachable. | | | | | 5. ICMPv6 packet too big. | | | | | | | ae-code | | U | A code relating to the event. For | | | | | ICMP or ICMPv6 events, this MUST be | | | | | the ICMP [RFC792] or ICMPv6 | | | | | [RFC4443] code. For other events, | | | | | the contents are undefined. | | | | | | | ae-transport-flags | C | U | Bit flags describing the transport | | | | | used to service the event. See | | | | | Section 6.2.4. | | | | | Bit 0. IP version. 0 if IPv4, 1 if | | | | | IPv6. | | | | | Bits 1-4. Transport. 4-bit | | | | | unsigned value where | | | | | 0 = UDP [RFC1035] | | | | | 1 = TCP [RFC1035] | | | | | 2 = TLS [RFC7858] | | | | | 3 = DTLS [RFC8094] | | | | | 4 = HTTPS [RFC8484] | | | | | 15 = Non-standard transport | | | | | Values 5-14 are reserved for future | | | | | use. | | | | | | | ae-address-index | X | U | The index in the "ip-address" array | | | | | of the client address. See Section | | | | | 7.3.2.3. | | | | | | | ae-count | X | U | The number of occurrences of this | | | | | event during the Block collection | | | | | period. | +--------------------+---+---+--------------------------------------+
Details on Malformed Message data items. A map containing the following:
有关格式错误的邮件数据项的详细信息。包含以下内容的地图:
+----------------------+---+---+------------------------------------+ | Field | M | T | Description | +----------------------+---+---+------------------------------------+ | time-offset | | U | Message timestamp as an offset in | | | | | ticks (see Section 7.3.1.1.1) from | | | | | "earliest-time". | | | | | | | client-address-index | | U | The index in the "ip-address" | | | | | array of the client IP address. | | | | | See Section 7.3.2.3. | | | | | | | client-port | | U | The client port. | | | | | | | message-data-index | | U | The index in the "malformed- | | | | | message-data" array of the message | | | | | data for this message. See | | | | | Section 7.3.2.3. | +----------------------+---+---+------------------------------------+
+----------------------+---+---+------------------------------------+ | Field | M | T | Description | +----------------------+---+---+------------------------------------+ | time-offset | | U | Message timestamp as an offset in | | | | | ticks (see Section 7.3.1.1.1) from | | | | | "earliest-time". | | | | | | | client-address-index | | U | The index in the "ip-address" | | | | | array of the client IP address. | | | | | See Section 7.3.2.3. | | | | | | | client-port | | U | The client port. | | | | | | | message-data-index | | U | The index in the "malformed- | | | | | message-data" array of the message | | | | | data for this message. See | | | | | Section 7.3.2.3. | +----------------------+---+---+------------------------------------+
The C-DNS File Preamble includes a file Format Version; a major and minor version number are required fields. This document defines version 1.0 of the C-DNS specification. This section describes the intended use of these version numbers in future specifications.
C-DNS文件前导包括文件格式版本;主要版本号和次要版本号是必填字段。本文档定义了C-DNS规范的1.0版。本节描述了这些版本号在未来规范中的预期用途。
It is noted that version 1.0 includes many optional fields; therefore, consumers of version 1.0 should be inherently robust to parsing files with variable data content.
值得注意的是,版本1.0包含许多可选字段;因此,1.0版的使用者在解析具有可变数据内容的文件时应该具有固有的健壮性。
Within a major version, a new minor version MUST be a strict superset of the previous minor version, with no semantic changes to existing fields. New keys MAY be added to existing maps, and new maps MAY be added. A consumer capable of reading a particular major.minor version MUST also be capable of reading all previous minor versions of the same major version. It SHOULD also be capable of parsing all subsequent minor versions, ignoring any keys or maps that it does not recognize.
在主版本中,新的次要版本必须是前一次要版本的严格超集,对现有字段没有语义更改。可以将新键添加到现有地图,也可以添加新地图。能够阅读特定主要版本的使用者。次要版本还必须能够阅读相同主要版本的所有以前次要版本。它还应该能够解析所有后续的次要版本,忽略任何它无法识别的键或映射。
A new major version indicates changes to the format that are not backwards compatible with previous major versions. A consumer capable of only reading a particular major version (greater than 1) is neither required nor expected to be capable of reading a previous major version.
新的主要版本表示对格式的更改与以前的主要版本不向后兼容。仅能阅读特定主要版本(大于1)的使用者既不要求也不期望能够阅读以前的主要版本。
It is usually possible to reconstruct PCAP files from the C-DNS format in a lossy fashion. Some of the issues with reconstructing both the DNS payload and the full packet stream are outlined here.
通常可以以有损方式从C-DNS格式重建PCAP文件。这里概述了重建DNS有效负载和完整数据包流的一些问题。
The reconstruction of well-formed DNS messages depends on two factors:
格式良好的DNS消息的重建取决于两个因素:
1. Whether or not a particular subset of the optional fields were captured in the C-DNS file, specifically the data fields necessary to reconstruct a valid IP header and DNS payload for both Query and Response (see Appendix D.1). Clearly, if not all these data fields were captured, the reconstruction is likely to be imperfect even if reasonable defaults are provided for the reconstruction.
1. C-DNS文件中是否捕获了可选字段的特定子集,特别是为查询和响应重建有效IP报头和DNS有效负载所需的数据字段(见附录D.1)。显然,如果没有捕获所有这些数据字段,即使为重建提供了合理的默认值,重建也可能不完美。
2. Whether or not at least one field was captured that unambiguously identifies the Query/Response data item as containing just a Query, just a Response, or a Query/Response pair. Obviously, the qr-sig-flags defined in Section 7.3.2.3.2 is such a field; however, this field is optional. For more details, see Appendix D.2.
2. 是否至少捕获了一个字段,该字段明确地将查询/响应数据项标识为仅包含查询、响应或查询/响应对。显然,第7.3.2.3.2节中定义的qr sig标志就是这样一个字段;但是,此字段是可选的。有关更多详细信息,请参见附录D.2。
It is noted again that simply having hints that indicate that certain data fields were not omitted does not guarantee that those data fields were actually captured. Therefore, the ability to reconstruct PCAP data (in the absence of defaults) can in principle vary for each record captured in a C-DNS file, and between Blocks that have differing hints.
需要再次指出的是,仅仅具有指示未省略某些数据字段的提示并不保证实际捕获了这些数据字段。因此,对于C-DNS文件中捕获的每个记录,以及具有不同提示的块之间,重构PCAP数据的能力(在没有默认值的情况下)原则上可能会有所不同。
Even if all sections of the Response were captured, one cannot reconstruct the DNS Response payload exactly, due to the fact that some DNS names in the message on the wire may have been compressed. Section 9.1 discusses this in more detail.
即使捕获了响应的所有部分,也无法准确地重构DNS响应有效负载,这是因为线路上消息中的某些DNS名称可能已被压缩。第9.1节对此进行了更详细的讨论。
Some transport information is not captured in the C-DNS format. For example, the following aspects of the original packet stream cannot be reconstructed from the C-DNS format:
某些传输信息不是以C-DNS格式捕获的。例如,不能从C-DNS格式重构原始分组流的以下方面:
o IP fragmentation
o IP分片
o TCP stream information:
o TCP流信息:
* Multiple DNS messages may have been sent in a single TCP segment
* 在单个TCP段中可能发送了多个DNS消息
* A DNS payload may have been split across multiple TCP segments
* DNS有效负载可能已被分割到多个TCP段
* Multiple DNS messages may have been sent on a single TCP session
* 在一个TCP会话上可能发送了多个DNS消息
o TLS session information:
o TLS会话信息:
* TLS version or cipher suites
* TLS版本或密码套件
* TLS-related features such as TCP Fast Open (TFO) [RFC7413] or TLS session resumption [RFC5077]
* TLS相关功能,如TCP快速打开(TFO)[RFC7413]或TLS会话恢复[RFC5077]
o DNS-over-HTTPS [RFC8484] message details:
o HTTPS上的DNS[RFC8484]消息详细信息:
* Whether the message used POST or GET
* 消息是使用POST还是GET
* HTTPS Headers
* HTTPS标头
o Malformed DNS messages if the wire format is not recorded
o 如果未记录wire格式,则DNS消息格式不正确
o Any non-DNS messages that were in the original packet stream, e.g., ICMP
o 原始数据包流中的任何非DNS消息,例如ICMP
Simple assumptions can be made on the reconstruction: fragmented and DNS-over-TCP messages can be reconstructed into single packets, and a single TCP session can be constructed for each TCP packet.
可以对重构进行简单的假设:碎片化和DNS over TCP消息可以重构为单个数据包,并且可以为每个TCP数据包构建单个TCP会话。
Additionally, if malformed messages and non-DNS packets are captured separately, they can be merged with packet captures reconstructed from C-DNS to produce a more complete packet stream.
此外,如果分别捕获格式错误的消息和非DNS数据包,则可以将它们与从C-DNS重构的数据包捕获合并,以生成更完整的数据包流。
All the names stored in the C-DNS format are full domain names; no name compression (per [RFC1035]) is used on the individual names within the format. Therefore, when reconstructing a packet, name compression must be used in order to reproduce the on-the-wire representation of the packet.
以C-DNS格式存储的所有名称都是完整的域名;未对格式中的单个名称使用名称压缩(根据[RFC1035])。因此,在重构数据包时,必须使用名称压缩来再现数据包的在线表示。
Name compression per [RFC1035] works by substituting trailing sections of a name with a reference back to the occurrence of those sections earlier in the message. Not all name server software uses the same algorithm when compressing domain names within the Responses. Some attempt maximum recompression at the expense of runtime resources, others use heuristics to balance compression and speed, and others use different rules for what is a valid compression target.
根据[RFC1035]进行的名称压缩是通过将名称的尾随部分替换为消息中前面出现的那些部分的引用来实现的。在响应中压缩域名时,并非所有名称服务器软件都使用相同的算法。一些人以牺牲运行时资源为代价尝试最大程度的重新压缩,另一些人使用启发式来平衡压缩和速度,还有一些人使用不同的规则来确定什么是有效的压缩目标。
This means that Responses to the same Query from different name server software that match in terms of DNS payload content (header, counts, RRs with name compression removed) do not necessarily match byte for byte on the wire.
这意味着,不同名称服务器软件对相同查询的响应在DNS有效负载内容(标题、计数、删除名称压缩的RRs)方面匹配,不一定在线路上逐字节匹配。
Therefore, it is not possible to ensure that the DNS Response payload is reconstructed byte for byte from C-DNS data. However, it can at least, in principle, be reconstructed to have the correct payload length (since the original Response length is captured) if there is enough knowledge of the commonly implemented name compression algorithms. For example, a simplistic approach would be to try each algorithm in turn to see if it reproduces the original length, stopping at the first match. This would not guarantee that the correct algorithm has been used, as it is possible to match the length whilst still not matching the on-the-wire bytes; however, without further information added to the C-DNS data, this is the best that can be achieved.
因此,无法确保从C-DNS数据逐字节重建DNS响应有效负载。然而,如果对常用的实现的名称压缩算法有足够的了解,则至少在原则上可以将其重构为具有正确的有效负载长度(因为捕获了原始响应长度)。例如,一种过于简单的方法是依次尝试每种算法,看看它是否复制原始长度,并在第一次匹配时停止。这不能保证使用了正确的算法,因为可能会匹配长度,但仍然不匹配在线字节;但是,如果不向C-DNS数据中添加更多信息,这是可以实现的最好结果。
Appendix B presents an example of two different compression algorithms used by well-known name server software.
附录B给出了知名名称服务器软件使用的两种不同压缩算法的示例。
This section describes a non-normative proposed algorithm for the processing of a captured stream of DNS Queries and Responses and production of a stream of Q/R data items, matching Queries and Responses where possible.
本节描述了一种非规范性的拟议算法,用于处理捕获的DNS查询和响应流,并生成Q/R数据项流,在可能的情况下匹配查询和响应。
For the purposes of this discussion, it is assumed that the input has been preprocessed such that:
在本讨论中,假设输入已经过预处理,以便:
1. All IP fragmentation reassembly, TCP stream reassembly, and so on, have already been performed.
1. 所有IP碎片重组、TCP流重组等都已执行。
2. Each message is associated with transport metadata required to generate the Primary ID (see Section 10.2.1).
2. 每条消息都与生成主ID所需的传输元数据相关联(参见第10.2.1节)。
3. Each message has a well-formed DNS Header of 12 bytes, and (if present) the first Question in the Question section can be parsed to generate the Secondary ID (see below). As noted earlier, this requirement can result in a malformed Query being removed in the preprocessing stage, but the correctly formed Response with RCODE of FORMERR being present.
3. 每条消息都有一个格式良好的12字节DNS头,并且(如果存在)可以解析问题部分中的第一个问题以生成辅助ID(见下文)。如前所述,此要求可能导致在预处理阶段删除格式错误的查询,但存在RCODE为FORMERR的正确格式响应。
DNS messages are processed in the order they are delivered to the implementation.
DNS消息按照它们传递到实现的顺序进行处理。
It should be noted that packet capture libraries do not necessarily provide packets in strict chronological order. This can, for example, arise on multi-core platforms where packets arriving at a network device are processed by different cores. On systems where this behavior has been observed, the timestamps associated with each packet are consistent; Queries always have a timestamp prior to the Response timestamp. However, the order in which these packets appear in the packet capture stream is not necessarily strictly chronological; a Response can appear in the capture stream before the Query that provoked the Response. For this discussion, this non-chronological delivery is termed "skew".
应该注意的是,数据包捕获库不一定按照严格的时间顺序提供数据包。例如,这可能出现在多核平台上,其中到达网络设备的数据包由不同的核处理。在观察到这种行为的系统上,与每个数据包相关联的时间戳是一致的;查询总是在响应时间戳之前有一个时间戳。然而,这些分组在分组捕获流中出现的顺序不一定严格按时间顺序排列;响应可以出现在引发响应的查询之前的捕获流中。在本次讨论中,这种不按时间顺序的交付被称为“扭曲”。
In the presence of skew, Response packets can arrive for matching before the corresponding Query. To avoid generating false instances of Responses without a matching Query, and Queries without a matching Response, the matching algorithm must take the possibility of skew into account.
在存在倾斜的情况下,响应包可以在相应查询之前到达以进行匹配。为了避免在没有匹配查询的情况下生成错误的响应实例,以及在没有匹配响应的情况下生成错误的响应实例,匹配算法必须考虑倾斜的可能性。
A schematic representation of the algorithm for matching Q/R data items is shown in Figure 3. It takes individual DNS Query or Response messages as input, and it outputs matched Q/R data items. The numbers in the figure identify matching operations listed in Table 1. Specific details of the algorithm -- for example, queues, timers, and identifiers -- are given in the following sections.
匹配Q/R数据项的算法示意图如图3所示。它将单个DNS查询或响应消息作为输入,并输出匹配的Q/R数据项。图中的数字表示表1中列出的匹配操作。算法的具体细节——例如,队列、计时器和标识符——在以下部分中给出。
.----------------------. | Process next message |<------------------+ `----------------------' | | | +------------------------------+ | | Generate message identifiers | | +------------------------------+ | | | Response | Query | +--------------< >---------------+ | | | | +--------------------+ +--------------------+ | | Find earliest QR | | Create QR item (2) | | | item in OFIFO (1) | +--------------------+ | +--------------------+ | | | +---------------+ | Match | No match | Append new QR | | +--------< >------+ | item to OFIFO | | | | +---------------+ | +-----------+ +--------+ | | | Update QR | | Add to | +-------------------+ | | item (3) | | RFIFO | | Find earliest QR | | +-----------+ +--------+ | item in RFIFO (1) | | | | +-------------------+ | +-----------------+ | | | | | | +----------------+ Match | No match | | | Remove R |-------< >-----+ | | | from RFIFO (3) | | | | +----------------+ | | | | | | +--------------+-----------------------+ | | | +----------------------------------------------+ | | Update all timed-out (QT) OFIFO QR items (4) | | +----------------------------------------------+ | | | +--------------------------------+ | | Remove all timed-out (ST) R | | | from RFIFO, create QR item (5) | | +--------------------------------+ | ____________________|_______________________ | / / | / Remove all consecutive done entries from /-------+ / front of OFIFO for further processing / /____________________________________________/
.----------------------. | Process next message |<------------------+ `----------------------' | | | +------------------------------+ | | Generate message identifiers | | +------------------------------+ | | | Response | Query | +--------------< >---------------+ | | | | +--------------------+ +--------------------+ | | Find earliest QR | | Create QR item (2) | | | item in OFIFO (1) | +--------------------+ | +--------------------+ | | | +---------------+ | Match | No match | Append new QR | | +--------< >------+ | item to OFIFO | | | | +---------------+ | +-----------+ +--------+ | | | Update QR | | Add to | +-------------------+ | | item (3) | | RFIFO | | Find earliest QR | | +-----------+ +--------+ | item in RFIFO (1) | | | | +-------------------+ | +-----------------+ | | | | | | +----------------+ Match | No match | | | Remove R |-------< >-----+ | | | from RFIFO (3) | | | | +----------------+ | | | | | | +--------------+-----------------------+ | | | +----------------------------------------------+ | | Update all timed-out (QT) OFIFO QR items (4) | | +----------------------------------------------+ | | | +--------------------------------+ | | Remove all timed-out (ST) R | | | from RFIFO, create QR item (5) | | +--------------------------------+ | ____________________|_______________________ | / / | / Remove all consecutive done entries from /-------+ / front of OFIFO for further processing / /____________________________________________/
OFIFO = output FIFO containing Q/R data items (Section 10.6) RFIFO = Response FIFO containing unmatched Response items (Section 10.6) QT = Query Timeout (Section 10.3) ST = Skew Timeout (Section 10.3)
OFIFO=包含Q/R数据项的输出FIFO(第10.6节)RFIFO=包含不匹配响应项的响应FIFO(第10.6节)QT=查询超时(第10.3节)ST=倾斜超时(第10.3节)
Figure 3: Query/Response Matching Algorithm
图3:查询/响应匹配算法
+-----------+-------------------------------------------+ | Reference | Operation | +-----------+-------------------------------------------+ | (1) | Find earliest QR item in FIFO where: | | | * QR.done = false | | | * QR.Q.PrimaryID == R.PrimaryID | | | and, if both QR.Q and R have SecondaryID: | | | * QR.Q.SecondaryID == R.SecondaryID | | | | | (2) | Set: | | | QR.Q := Q | | | QR.R := nil | | | QR.done := false | | | | | (3) | Set: | | | QR.R := R | | | QR.done := true | | | | | (4) | Set: | | | QR.done := true | | | | | (5) | Set: | | | QR.Q := nil | | | QR.R := R | | | QR.done := true | +-----------+-------------------------------------------+
+-----------+-------------------------------------------+ | Reference | Operation | +-----------+-------------------------------------------+ | (1) | Find earliest QR item in FIFO where: | | | * QR.done = false | | | * QR.Q.PrimaryID == R.PrimaryID | | | and, if both QR.Q and R have SecondaryID: | | | * QR.Q.SecondaryID == R.SecondaryID | | | | | (2) | Set: | | | QR.Q := Q | | | QR.R := nil | | | QR.done := false | | | | | (3) | Set: | | | QR.R := R | | | QR.done := true | | | | | (4) | Set: | | | QR.done := true | | | | | (5) | Set: | | | QR.Q := nil | | | QR.R := R | | | QR.done := true | +-----------+-------------------------------------------+
Table 1: Operations Used in the Matching Algorithm
表1:匹配算法中使用的操作
A Primary ID is constructed for each message. It is composed of the following data:
为每条消息构造一个主ID。它由以下数据组成:
1. Source IP Address
1. 源IP地址
2. Destination IP Address
2. 目的地址
3. Source Port
3. 源端口
4. Destination Port
4. 目的港
5. Transport
5. 运输
6. DNS Message ID
6. DNS消息ID
If present, the first Question in the Question section is used as a Secondary ID for each message. Note that there may be well-formed DNS Queries that have a QDCOUNT of 0, and some Responses may have a QDCOUNT of 0 (for example, Responses with RCODE=FORMERR or NOTIMP). In this case, the Secondary ID is not used in matching.
如果存在,问题部分中的第一个问题将用作每条消息的辅助ID。请注意,可能存在QDCOUNT为0的格式良好的DNS查询,某些响应的QDCOUNT为0(例如,RCODE=FORMERR或NOTIMP的响应)。在这种情况下,辅助ID不用于匹配。
1. Query Timeout (QT). A Query arrives with timestamp t1. If no Response matching that Query has arrived before other input arrives timestamped later than (t1 + QT), a Q/R data item containing only a Query is recorded. The QT value is typically on the order of 5 seconds.
1. 查询超时(QT)。查询到达时带有时间戳t1。如果在时间戳晚于(t1+QT)的其他输入到达之前,没有与该查询匹配的响应到达,则记录仅包含查询的Q/R数据项。QT值通常为5秒左右。
2. Skew Timeout (ST). A Response arrives with timestamp t2. If a Response has not been matched by a Query before input arrives timestamped later than (t2 + ST), a Q/R data item containing only a Response is recorded. The ST value is typically a few microseconds.
2. 倾斜超时(ST)。响应以时间戳t2到达。如果在输入到达时间戳晚于(t2+ST)之前,响应没有被查询匹配,则记录仅包含响应的Q/R数据项。ST值通常为几微秒。
The algorithm is designed to handle the following input data:
该算法设计用于处理以下输入数据:
1. Multiple Queries with the same Primary ID (but different Secondary ID) arriving before any Responses for these Queries are seen.
1. 具有相同主ID(但不同的辅助ID)的多个查询在看到这些查询的任何响应之前到达。
2. Multiple Queries with the same Primary and Secondary ID arriving before any Responses for these Queries are seen.
2. 具有相同主ID和辅助ID的多个查询在看到这些查询的任何响应之前到达。
3. Queries for which no later Response can be found within the specified timeout.
3. 在指定的超时时间内找不到后续响应的查询。
4. Responses for which no previous Query can be found within the specified timeout.
4. 在指定的超时时间内找不到以前查询的响应。
For cases 1 and 2 listed in the above requirements, it is not possible to unambiguously match Queries with Responses. This algorithm chooses to match to the earliest Query with the correct Primary and Secondary ID.
对于上述要求中列出的情况1和2,不可能明确地将查询与响应匹配。该算法选择使用正确的主ID和辅助ID匹配最早的查询。
The algorithm employs two FIFO queues:
该算法采用两个FIFO队列:
o OFIFO: an output FIFO containing Q/R data items in chronological order.
o OFIFO:一种按时间顺序包含Q/R数据项的输出FIFO。
o RFIFO: a FIFO holding Responses without a matching Query in order of arrival.
o RFIFO:按到达顺序保留响应的FIFO,没有匹配的查询。
The output is a list of Q/R data items. Both the Query and Response elements are optional in these items; therefore, Q/R data items have one of three types of content:
输出是Q/R数据项的列表。查询和响应元素在这些项中都是可选的;因此,Q/R数据项具有三种类型的内容之一:
1. A matched pair of Query and Response messages
1. 一对匹配的查询和响应消息
2. A Query message with no Response
2. 没有响应的查询消息
3. A Response message with no Query
3. 没有查询的响应消息
The timestamp of a list item is that of the Query for cases 1 and 2 and that of the Response for case 3.
列表项的时间戳是案例1和案例2的查询时间戳和案例3的响应时间戳。
When ending a capture, all items in the RFIFO are timed out immediately, generating Response only entries to the OFIFO. These and all other remaining entries in the OFIFO should be treated as timed-out Queries.
在结束捕获时,RFIFO中的所有项目都会立即超时,从而生成仅响应OFIFO的条目。OFIFO中的这些项和所有其他剩余项应视为超时查询。
Whilst this document makes no specific recommendations with respect to "Canonical CBOR" (see Section 3.9 of [RFC7049]), the following guidance may be of use to implementers.
虽然本文件未就“规范CBOR”(见[RFC7049]第3.9节)提出具体建议,但以下指南可供实施者使用。
Adherence to the first two rules given in Section 3.9 of [RFC7049] will minimize file sizes.
遵守[RFC7049]第3.9节中给出的前两条规则将最小化文件大小。
Adherence to the last two rules given in Section 3.9 of [RFC7049] for all maps and arrays would unacceptably constrain implementations -- for example, in the use case of real-time data collection in constrained environments where outputting Block Tables after Q/R data items and allowing indefinite-length maps and arrays could reduce memory requirements.
对于所有映射和数组,遵守[RFC7049]第3.9节中给出的最后两条规则将不可接受地限制实现——例如,在受限环境中实时数据采集的用例中,在Q/R数据项之后输出块表并允许无限长映射和数组可以减少内存需求。
It is recommended that implementations that have fundamental restrictions on what data fields they can collect SHOULD always store hints with the bits unset for those fields, i.e., they unambiguously indicate that those data fields will be omitted from captured C-DNS.
建议对可收集的数据字段有基本限制的实现应始终存储未设置这些字段位的提示,即,它们明确指示将从捕获的C-DNS中忽略这些数据字段。
When decoding C-DNS data, some of the items required for a particular function that the consumer wishes to perform may be missing. Consumers should consider providing configurable default values to be used in place of the missing values in their output.
当解码C-DNS数据时,消费者希望执行的特定功能所需的一些项目可能丢失。消费者应该考虑提供可配置的默认值来代替其输出中的缺失值。
A DNS Query message in a UDP or TCP payload can be followed by some additional (spurious) bytes, which are not stored in C-DNS.
UDP或TCP有效负载中的DNS查询消息后面可以跟一些附加(虚假)字节,这些字节不存储在C-DNS中。
When DNS traffic is sent over TCP, each message is prefixed with a two-byte length field, which gives the message length, excluding the two-byte length field. In this context, trailing bytes can occur in two circumstances, with different results:
当DNS通信通过TCP发送时,每条消息的前缀都是一个双字节长度字段,该字段给出消息长度,不包括双字节长度字段。在此上下文中,尾随字节可能出现在两种情况下,结果不同:
1. The number of bytes consumed by fully parsing the message is less than the number of bytes given in the length field (i.e., the length field is incorrect and too large). In this case, the surplus bytes are considered trailing bytes in a manner analogous to UDP and recorded as such. If only this case occurs, it is possible to process a packet containing multiple DNS messages where one or more have trailing bytes.
1. 完全解析消息所消耗的字节数小于长度字段中给定的字节数(即,长度字段不正确且太大)。在这种情况下,多余的字节被视为以类似于UDP的方式的尾随字节,并被记录为尾随字节。如果仅发生这种情况,则可以处理包含多个DNS消息的数据包,其中一个或多个DNS消息具有尾随字节。
2. There are surplus bytes between the end of a well-formed message and the start of the length field for the next message. In this case, the first of the surplus bytes will be processed as the first byte of the next length field, and parsing will proceed from there, almost certainly leading to the next and any subsequent messages in the packet being considered malformed. This will not generate a trailing-bytes record for the processed well-formed message.
2. 格式良好的消息的结尾和下一条消息的长度字段的开头之间有多余的字节。在这种情况下,剩余字节中的第一个字节将被处理为下一个长度字段的第一个字节,解析将从那里开始,几乎可以肯定会导致数据包中的下一个和任何后续消息被视为格式错误。这不会为已处理的格式良好的消息生成尾随字节记录。
Implementations should consider providing a configurable maximum RDATA size for captures -- for example, to avoid memory issues when confronted with large zone transfer records.
实现应该考虑为捕获提供可配置的最大RDATA大小,例如,在面对大区域传输记录时,避免内存问题。
The preamble to each block includes a timestamp of the earliest record in the Block. As described in Section 7.3.2.1, the timestamp is an array of two unsigned integers. The first is a POSIX "time_t" [posix-time]. Consumers of C-DNS should be aware of this, as it excludes leap seconds and therefore may cause minor anomalies in the data, e.g., when calculating Query throughput.
每个块的前导包括块中最早记录的时间戳。如第7.3.2.1节所述,时间戳是两个无符号整数的数组。第一个是POSIX“时间”[POSIX时间]。C-DNS的使用者应该意识到这一点,因为它不包括闰秒,因此可能会导致数据出现轻微异常,例如,在计算查询吞吐量时。
IANA has created a registry "C-DNS DNS Capture Format" containing the subregistries defined in Sections 12.1 to 12.4 inclusive.
IANA已创建一个注册表“C-DNS DNS捕获格式”,其中包含第12.1节至第12.4节(含第12.1节)中定义的子区域。
In all cases, new entries may be added to the subregistries by Expert Review as defined in [RFC8126]. Experts are expected to exercise their own expert judgment and should consider the following general guidelines in addition to any provided guidelines that are particular to a subregistry.
在所有情况下,可通过[RFC8126]中定义的专家评审将新条目添加到次区域。专家预计将行使自己的专家判断,并应考虑以下一般准则,除了任何提供的指导方针,特别是一个子注册。
o There should be a real and compelling use for any new value.
o 对于任何新的价值,都应该有一个真正的、令人信服的用途。
o Values assigned should be carefully chosen to minimize storage requirements for common cases.
o 应仔细选择分配的值,以尽量减少常见情况下的存储要求。
IANA has created a registry "C-DNS Transports" of C-DNS transport type identifiers. The primary purpose of this registry is to provide unique identifiers for all transports used for DNS Queries.
IANA已创建C-DNS传输类型标识符的“C-DNS传输”注册表。此注册表的主要目的是为用于DNS查询的所有传输提供唯一标识符。
The following note is included in this registry: "In version 1.0 of C-DNS [RFC8618], there is a field to identify the type of DNS transport. This field is 4 bits in size."
此注册表中包含以下注释:“在C-DNS[RFC8618]的1.0版中,有一个字段用于标识DNS传输的类型。此字段的大小为4位。”
The initial contents of the registry are as follows. See Sections 7.3.2.3.2, 7.3.2.3.5, and 7.3.2.5 of this document:
登记处的初步内容如下。见本文件第7.3.2.3.2、7.3.2.3.5和7.3.2.5节:
+------------+------------------------+-----------+ | Identifier | Name | Reference | +------------+------------------------+-----------+ | 0 | UDP | RFC 8618 | | 1 | TCP | RFC 8618 | | 2 | TLS | RFC 8618 | | 3 | DTLS | RFC 8618 | | 4 | HTTPS | RFC 8618 | | 5-14 | Unassigned | | | 15 | Non-standard transport | RFC 8618 | +------------+------------------------+-----------+
+------------+------------------------+-----------+ | Identifier | Name | Reference | +------------+------------------------+-----------+ | 0 | UDP | RFC 8618 | | 1 | TCP | RFC 8618 | | 2 | TLS | RFC 8618 | | 3 | DTLS | RFC 8618 | | 4 | HTTPS | RFC 8618 | | 5-14 | Unassigned | | | 15 | Non-standard transport | RFC 8618 | +------------+------------------------+-----------+
Expert reviewers should take the following point into consideration: Is the requested DNS transport described by a Standards Track RFC?
专家评审员应考虑以下几点:请求的DNS传输是否由标准跟踪RFC描述?
IANA has created a registry "C-DNS Storage Flags" of C-DNS data storage flags. The primary purpose of this registry is to provide indicators giving hints on processing of the data stored.
IANA已创建C-DNS数据存储标志的注册表“C-DNS存储标志”。此注册表的主要目的是提供指示符,提示处理存储的数据。
The following note is included in this registry: "In version 1.0 of C-DNS [RFC8618], there is a field describing attributes of the data recorded. The field is a CBOR [RFC7049] unsigned integer holding bit flags."
此注册表中包含以下注释:“在C-DNS[RFC8618]的1.0版中,有一个字段描述记录的数据属性。该字段是一个CBOR[RFC7049]无符号整数,包含位标志。”
The initial contents of the registry are as follows. See Section 7.3.1.1.1 of this document:
登记处的初步内容如下。见本文件第7.3.1.1.1节:
+------+------------------+-----------------------------+-----------+ | Bit | Name | Description | Reference | +------+------------------+-----------------------------+-----------+ | 0 | anonymized-data | The data has been | RFC 8618 | | | | anonymized. | | | | | | | | 1 | sampled-data | The data is sampled data. | RFC 8618 | | | | | | | 2 | normalized-names | Names in the data have been | RFC 8618 | | | | normalized. | | | | | | | | 3-63 | Unassigned | | | +------+------------------+-----------------------------+-----------+
+------+------------------+-----------------------------+-----------+ | Bit | Name | Description | Reference | +------+------------------+-----------------------------+-----------+ | 0 | anonymized-data | The data has been | RFC 8618 | | | | anonymized. | | | | | | | | 1 | sampled-data | The data is sampled data. | RFC 8618 | | | | | | | 2 | normalized-names | Names in the data have been | RFC 8618 | | | | normalized. | | | | | | | | 3-63 | Unassigned | | | +------+------------------+-----------------------------+-----------+
IANA has created a registry "C-DNS Response Flags" of C-DNS response-processing flags. The primary purpose of this registry is to provide indicators giving hints on the generation of a particular Response.
IANA已创建C-DNS响应处理标志的注册表“C-DNS响应标志”。此注册表的主要目的是提供指标,提示生成特定响应。
The following note is included in this registry: "In version 1.0 of C-DNS [RFC8618], there is a field describing attributes of the Responses recorded. The field is a CBOR [RFC7049] unsigned integer holding bit flags."
此注册表中包含以下注释:“在C-DNS[RFC8618]的1.0版中,有一个字段描述记录的响应的属性。该字段是一个包含位标志的CBOR[RFC7049]无符号整数。”
The initial contents of the registry are as follows. See Section 7.3.2.4.1 of this document:
登记处的初步内容如下。见本文件第7.3.2.4.1节:
+------+------------+-------------------------------+-----------+ | Bit | Name | Description | Reference | +------+------------+-------------------------------+-----------+ | 0 | from-cache | The Response came from cache. | RFC 8618 | | 1-63 | Unassigned | | | +------+------------+-------------------------------+-----------+
+------+------------+-------------------------------+-----------+ | Bit | Name | Description | Reference | +------+------------+-------------------------------+-----------+ | 0 | from-cache | The Response came from cache. | RFC 8618 | | 1-63 | Unassigned | | | +------+------------+-------------------------------+-----------+
IANA has created a registry "C-DNS Address Event Types" of C-DNS AddressEvent types. The primary purpose of this registry is to provide unique identifiers of different types of C-DNS address events and so specify the contents of the optional companion field "ae-code" for each type.
IANA已创建C-DNS地址事件类型的注册表“C-DNS地址事件类型”。此注册表的主要目的是提供不同类型的C-DNS地址事件的唯一标识符,从而为每种类型指定可选伴随字段“ae代码”的内容。
The following note is included in this registry: "In version 1.0 of C-DNS [RFC8618], there is a field identifying types of the events related to client addresses. This field is a CBOR [RFC7049] unsigned integer. There is a related optional field "ae-code", which, if present, holds an additional CBOR unsigned integer giving additional information specific to the event type."
此注册表中包含以下注释:“在C-DNS[RFC8618]的1.0版中,有一个字段标识与客户端地址相关的事件类型。该字段是CBOR[RFC7049]无符号整数。有一个相关的可选字段“ae代码”,它(如果存在)包含一个附加的CBOR无符号整数,提供特定于事件类型的附加信息
The initial contents of the registry are as follows. See Section 7.3.2.5 of this document:
登记处的初步内容如下。见本文件第7.3.2.5节:
+------------------------+---------------+--------------+-----------+ | Identifier | Event Type | ae-code | Reference | | | | Contents | | +------------------------+---------------+--------------+-----------+ | 0 | TCP reset | None | RFC 8618 | | | | | | | 1 | ICMP time | ICMP code | RFC 8618 | | | exceeded | [icmpcodes] | | | | | | | | 2 | ICMP | ICMP code | RFC 8618 | | | destination | [icmpcodes] | | | | unreachable | | | | | | | | | 3 | ICMPv6 time | ICMPv6 code | RFC 8618 | | | exceeded | [icmp6codes] | | | | | | | | 4 | ICMPv6 | ICMPv6 code | RFC 8618 | | | destination | [icmp6codes] | | | | unreachable | | | | | | | | | 5 | ICMPv6 packet | ICMPv6 code | RFC 8618 | | | too big | [icmp6codes] | | | | | | | | 6-18446744073709551615 | Unassigned | | | +------------------------+---------------+--------------+-----------+
+------------------------+---------------+--------------+-----------+ | Identifier | Event Type | ae-code | Reference | | | | Contents | | +------------------------+---------------+--------------+-----------+ | 0 | TCP reset | None | RFC 8618 | | | | | | | 1 | ICMP time | ICMP code | RFC 8618 | | | exceeded | [icmpcodes] | | | | | | | | 2 | ICMP | ICMP code | RFC 8618 | | | destination | [icmpcodes] | | | | unreachable | | | | | | | | | 3 | ICMPv6 time | ICMPv6 code | RFC 8618 | | | exceeded | [icmp6codes] | | | | | | | | 4 | ICMPv6 | ICMPv6 code | RFC 8618 | | | destination | [icmp6codes] | | | | unreachable | | | | | | | | | 5 | ICMPv6 packet | ICMPv6 code | RFC 8618 | | | too big | [icmp6codes] | | | | | | | | 6-18446744073709551615 | Unassigned | | | +------------------------+---------------+--------------+-----------+
Expert reviewers should take the following point into consideration: "ae-code" contents must be defined for a type or, if not appropriate, specified as "None". A specification of "None" requires less storage and is therefore preferred.
专家评审员应考虑以下几点:“ae代码”内容必须针对某一类型进行定义,如果不合适,则指定为“无”。“无”规格需要较少的存储空间,因此是首选。
Any control interface MUST perform authentication and encryption.
任何控制接口都必须执行身份验证和加密。
Any data upload MUST be authenticated and encrypted.
任何数据上传都必须经过身份验证和加密。
Storage of DNS traffic by operators in PCAP and other formats is a long-standing and widespread practice. Section 2.5 of [DNS-Priv-Cons] provides an analysis of the risks to Internet users regarding the storage of DNS traffic data in servers (recursive resolvers, authoritative servers, and rogue servers).
运营商以PCAP和其他格式存储DNS流量是一种长期且广泛的做法。[DNS Priv Cons]第2.5节分析了互联网用户在服务器(递归解析程序、权威服务器和恶意服务器)中存储DNS流量数据的风险。
Section 5.2 of [DNS-Priv-Svc] describes mitigations for those risks for data stored on recursive resolvers (but that could by extension apply to authoritative servers). These include data-handling practices and methods for data minimization, IP address pseudonymization, and anonymization. Appendix C of [DNS-Priv-Svc] presents an analysis of seven published anonymization processes. In addition, the ICANN Root Server System Advisory Committee (RSSAC) have recently published [RSSAC04] ("Recommendations on Anonymization Processes for Source IP Addresses Submitted for Future Analysis").
[DNS Priv Svc]的第5.2节描述了存储在递归解析器上的数据的风险缓解措施(但扩展后,这可能适用于权威服务器)。其中包括数据处理实践和数据最小化、IP地址假名化和匿名化方法。[DNS Priv Svc]的附录C对七个已发布的匿名化过程进行了分析。此外,ICANN根服务器系统咨询委员会(RSSAC)最近发布了[RSSAC04](“提交供未来分析的源IP地址匿名化过程建议”)。
The above analyses consider full data capture (e.g., using PCAP) as a baseline for privacy considerations; therefore, this format specification introduces no new user privacy issues beyond those of full data capture (which are quite severe). It does provide mechanisms to selectively record only certain fields at the time of data capture, to improve user privacy and to explicitly indicate that data is sampled, anonymized, or both. It also provides flags to indicate if data normalization has been performed; data normalization increases user privacy by reducing the potential for fingerprinting individuals. However, a trade-off is the potential reduction of the capacity to identify attack traffic via Query name signatures. Operators should carefully consider their operational requirements and privacy policies and SHOULD capture at the source the minimum user data required to meet their needs.
上述分析考虑全数据捕获(例如,使用pCAP)作为隐私考虑的基线;因此,除了完整数据捕获(非常严重)之外,此格式规范不会引入任何新的用户隐私问题。它确实提供了在数据捕获时仅选择性地记录某些字段的机制,以改善用户隐私,并明确指出数据是采样的、匿名的或两者兼而有之。它还提供标志来指示是否执行了数据规范化;数据规范化通过降低对个人进行指纹识别的可能性来提高用户隐私。然而,一个折衷方案是,通过查询名称签名识别攻击流量的能力可能会降低。运营商应仔细考虑其运营需求和隐私政策,并应在源头捕获满足用户需求所需的最小用户数据。
[pcap-filter] tcpdump.org, "Manpage of PCAP-FILTER", November 2017, <https://www.tcpdump.org/manpages/pcap-filter.7.html>.
[pcap filter]tcpdump.org,“pcap-filter手册”,2017年11月<https://www.tcpdump.org/manpages/pcap-filter.7.html>.
[pcap-options] tcpdump.org, "Manpage of PCAP", July 2018, <https://www.tcpdump.org/manpages/pcap.3pcap.html>.
[pcap选项]tcpdump.org,“pcap手册”,2018年7月<https://www.tcpdump.org/manpages/pcap.3pcap.html>.
[posix-time] The Open Group, "IEEE Standard for Information Technology--Portable Operating System Interface (POSIX(R)) Base Specifications, Issue 7", IEEE Standard 1003.1-2017, Section 4.16, DOI 10.1109/IEEESTD.2018.8277153.
[posix time]开放组,“IEEE信息技术标准——便携式操作系统接口(posix(R))基本规范,第7期”,IEEE标准1003.1-2017,第4.16节,DOI 10.1109/IEEESTD.2018.8277153。
[RFC792] Postel, J., "Internet Control Message Protocol", STD 5, RFC 792, DOI 10.17487/RFC0792, September 1981, <https://www.rfc-editor.org/info/rfc792>.
[RFC792]Postel,J.,“互联网控制消息协议”,STD 5,RFC 792,DOI 10.17487/RFC0792,1981年9月<https://www.rfc-editor.org/info/rfc792>.
[RFC1035] Mockapetris, P., "Domain names - implementation and specification", STD 13, RFC 1035, DOI 10.17487/RFC1035, November 1987, <https://www.rfc-editor.org/info/rfc1035>.
[RFC1035]Mockapetris,P.,“域名-实现和规范”,STD 13,RFC 1035,DOI 10.17487/RFC1035,1987年11月<https://www.rfc-editor.org/info/rfc1035>.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, <https://www.rfc-editor.org/info/rfc2119>.
[RFC2119]Bradner,S.,“RFC中用于表示需求水平的关键词”,BCP 14,RFC 2119,DOI 10.17487/RFC2119,1997年3月<https://www.rfc-editor.org/info/rfc2119>.
[RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform Resource Identifier (URI): Generic Syntax", STD 66, RFC 3986, DOI 10.17487/RFC3986, January 2005, <https://www.rfc-editor.org/info/rfc3986>.
[RFC3986]Berners Lee,T.,Fielding,R.,和L.Masinter,“统一资源标识符(URI):通用语法”,STD 66,RFC 3986,DOI 10.17487/RFC3986,2005年1月<https://www.rfc-editor.org/info/rfc3986>.
[RFC4443] Conta, A., Deering, S., and M. Gupta, Ed., "Internet Control Message Protocol (ICMPv6) for the Internet Protocol Version 6 (IPv6) Specification", STD 89, RFC 4443, DOI 10.17487/RFC4443, March 2006, <https://www.rfc-editor.org/info/rfc4443>.
[RFC4443]Conta,A.,Deering,S.和M.Gupta,Ed.“互联网协议版本6(IPv6)规范的互联网控制消息协议(ICMPv6)”,STD 89,RFC 4443,DOI 10.17487/RFC4443,2006年3月<https://www.rfc-editor.org/info/rfc4443>.
[RFC6891] Damas, J., Graff, M., and P. Vixie, "Extension Mechanisms for DNS (EDNS(0))", STD 75, RFC 6891, DOI 10.17487/RFC6891, April 2013, <https://www.rfc-editor.org/info/rfc6891>.
[RFC6891]Damas,J.,Graff,M.,和P.Vixie,“DNS的扩展机制(EDNS(0)),STD 75,RFC 6891,DOI 10.17487/RFC68911913年4月<https://www.rfc-editor.org/info/rfc6891>.
[RFC7049] Bormann, C. and P. Hoffman, "Concise Binary Object Representation (CBOR)", RFC 7049, DOI 10.17487/RFC7049, October 2013, <https://www.rfc-editor.org/info/rfc7049>.
[RFC7049]Bormann,C.和P.Hoffman,“简明二进制对象表示法(CBOR)”,RFC 7049,DOI 10.17487/RFC7049,2013年10月<https://www.rfc-editor.org/info/rfc7049>.
[RFC7858] Hu, Z., Zhu, L., Heidemann, J., Mankin, A., Wessels, D., and P. Hoffman, "Specification for DNS over Transport Layer Security (TLS)", RFC 7858, DOI 10.17487/RFC7858, May 2016, <https://www.rfc-editor.org/info/rfc7858>.
[RFC7858]Hu,Z.,Zhu,L.,Heidemann,J.,Mankin,A.,Wessels,D.,和P.Hoffman,“DNS传输层安全规范(TLS)”,RFC 7858,DOI 10.17487/RFC7858,2016年5月<https://www.rfc-editor.org/info/rfc7858>.
[RFC8094] Reddy, T., Wing, D., and P. Patil, "DNS over Datagram Transport Layer Security (DTLS)", RFC 8094, DOI 10.17487/RFC8094, February 2017, <https://www.rfc-editor.org/info/rfc8094>.
[RFC8094]Reddy,T.,Wing,D.,和P.Patil,“数据报传输层安全(DTLS)上的DNS”,RFC 8094,DOI 10.17487/RFC8094,2017年2月<https://www.rfc-editor.org/info/rfc8094>.
[RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 8126, DOI 10.17487/RFC8126, June 2017, <https://www.rfc-editor.org/info/rfc8126>.
[RFC8126]Cotton,M.,Leiba,B.,和T.Narten,“在RFC中编写IANA考虑事项部分的指南”,BCP 26,RFC 8126,DOI 10.17487/RFC8126,2017年6月<https://www.rfc-editor.org/info/rfc8126>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[RFC8174]Leiba,B.,“RFC 2119关键词中大写与小写的歧义”,BCP 14,RFC 8174,DOI 10.17487/RFC8174,2017年5月<https://www.rfc-editor.org/info/rfc8174>.
[RFC8484] Hoffman, P. and P. McManus, "DNS Queries over HTTPS (DoH)", RFC 8484, DOI 10.17487/RFC8484, October 2018, <https://www.rfc-editor.org/info/rfc8484>.
[RFC8484]Hoffman,P.和P.McManus,“HTTPS(DoH)上的DNS查询”,RFC 8484,DOI 10.17487/RFC8484,2018年10月<https://www.rfc-editor.org/info/rfc8484>.
[RFC8610] Birkholz, H., Vigano, C., and C. Bormann, "Concise Data Definition Language (CDDL): A Notational Convention to Express Concise Binary Object Representation (CBOR) and JSON Data Structures", RFC 8610, DOI 10.17487/RFC8610, June 2019, <https://www.rfc-editor.org/info/rfc8610>.
[RFC8610]Birkholz,H.,Vigano,C.,和C.Bormann,“简明数据定义语言(CDDL):表示简明二进制对象表示(CBOR)和JSON数据结构的符号约定”,RFC 8610,DOI 10.17487/RFC8610,2019年6月<https://www.rfc-editor.org/info/rfc8610>.
[Avro] The Apache Software Foundation, "Apache Avro(TM)", 2019, <https://avro.apache.org/>.
[Avro ] Apache软件基金会,“Apache AVro(TM)”,2019,<https://avro.apache.org/>.
[ditl] DNS-OARC, "DITL", 2018, <https://www.dns-oarc.net/oarc/data/ditl>.
[ditl]DNS-OARC,“ditl”,2018年<https://www.dns-oarc.net/oarc/data/ditl>.
[DNS-Priv-Cons] Bortzmeyer, S. and S. Dickinson, "DNS Privacy Considerations", Work in Progress, draft-ietf-dprive-rfc7626-bis-00, July 2019.
[DNS隐私权]Bortzmeyer,S.和S.Dickinson,“DNS隐私注意事项”,正在进行中的工作,草案-ietf-dprive-rfc7626-bis-00,2019年7月。
[DNS-Priv-Svc] Dickinson, S., Overeinder, B., van Rijswijk-Deij, R., and A. Mankin, "Recommendations for DNS Privacy Service Operators", Work in Progress, draft-ietf-dprive-bcp-op-03, July 2019.
[DNS Priv Svc]Dickinson,S.,Overfinder,B.,van Rijswijk Deij,R.,和A.Mankin,“DNS隐私服务运营商建议”,正在进行中的工作,草案-ietf-dprive-bcp-op-032019年7月。
[dnscap] DNS-OARC, "DNSCAP", 2018, <https://www.dns-oarc.net/tools/dnscap>.
[dnscap]DNS-OARC,“dnscap”,2018年<https://www.dns-oarc.net/tools/dnscap>.
[dnstap] "dnstap", 2016, <https://dnstap.info/>.
[dnstap]“dnstap”,2016年<https://dnstap.info/>.
[dnstap-schema] "dnstap schema", commit d860ec1, November 2016, <https://github.com/dnstap/dnstap.pb/blob/master/ dnstap.proto>.
[dnstap模式]“dnstap模式”,提交D860EC12016年11月<https://github.com/dnstap/dnstap.pb/blob/master/ dnstap.proto>。
[dnsxml] Daley, J., Ed., Morris, S., and J. Dickinson, "dnsxml - A standard XML representation of DNS data", Work in Progress, draft-daley-dnsxml-00, July 2013.
[dnsxml]Daley,J.,Ed.,Morris,S.,和J.Dickinson,“dnsxml-DNS数据的标准XML表示”,正在进行的工作,草稿-Daley-dnsxml-00,2013年7月。
[dsc] Wessels, D. and J. Lundstrom, "DSC", 2016, <https://www.dns-oarc.net/tools/dsc>.
[dsc]Wessels,D.和J.Lundstrom,“dsc”,2016年<https://www.dns-oarc.net/tools/dsc>.
[gzip] "gzip", <https://www.gzip.org/>.
[gzip]“gzip”<https://www.gzip.org/>.
[icmp6codes] IANA, "ICMPv6 "Code" Fields", <https://www.iana.org/assignments/icmpv6-parameters/>.
[icmp6codes]IANA,“ICMPv6”代码“字段”<https://www.iana.org/assignments/icmpv6-parameters/>.
[icmpcodes] IANA, "Code Fields", <https://www.iana.org/assignments/icmp-parameters/>.
[icmpcodes]IANA,“代码字段”<https://www.iana.org/assignments/icmp-parameters/>.
[IEEE802.1Q] IEEE, "IEEE Standard for Local and Metropolitan Area Networks--Bridges and Bridged Networks", IEEE Standard 802.1Q.
[IEEE802.1Q]IEEE,“局域网和城域网的IEEE标准——网桥和桥接网络”,IEEE标准802.1Q。
[Knot] "Knot DNS", <https://www.knot-dns.cz/>.
[结]“结DNS”<https://www.knot-dns.cz/>.
[lz4] "LZ4", <https://lz4.github.io/lz4/>.
[lz4]“lz4”<https://lz4.github.io/lz4/>.
[mmark] Gieben, M., "mmark", commit de69698, May 2019, <https://github.com/mmarkdown/mmark>.
[mmark]Gieben,M.,“mmark”,提交日期:2019年5月,第69698号<https://github.com/mmarkdown/mmark>.
[NSD] NLnet Labs, "NSD", 2019, <https://www.nlnetlabs.nl/projects/nsd/about/>.
[NSD]NLnet实验室,“NSD”,2019年<https://www.nlnetlabs.nl/projects/nsd/about/>.
[opcodes] IANA, "DNS OpCodes", <https://www.iana.org/assignments/dns-parameters/>.
[操作码]IANA,“DNS操作码”<https://www.iana.org/assignments/dns-parameters/>.
[packetq] .SE - The Internet Infrastructure Foundation, "PacketQ", commit c9b2e89, February 2019, <https://github.com/DNS-OARC/PacketQ>.
Se-互联网基础设施“PACETETQ”,提交C9B2E89C,2019年2月,<https://github.com/DNS-OARC/PacketQ>.
[pcap] "PCAP", 2019, <https://www.tcpdump.org/>.
[pcap]“pcap”,2019年<https://www.tcpdump.org/>.
[pcapng] "pcapng: PCAP next generation file format specification", commit 3c35b6a, March 2019, <https://github.com/pcapng/pcapng>.
[pcapng]“pcapng:PCAP下一代文件格式规范”,提交3c35b6a,2019年3月<https://github.com/pcapng/pcapng>.
[Protocol-Buffers] Google LLC, "Protocol Buffers", <https://developers.google.com/protocol-buffers/>.
[协议缓冲区]谷歌有限责任公司,“协议缓冲区”<https://developers.google.com/protocol-buffers/>.
[rcodes] IANA, "DNS RCODEs", <https://www.iana.org/assignments/dns-parameters/>.
[rcodes]IANA,“DNS rcodes”<https://www.iana.org/assignments/dns-parameters/>.
[RFC5077] Salowey, J., Zhou, H., Eronen, P., and H. Tschofenig, "Transport Layer Security (TLS) Session Resumption without Server-Side State", RFC 5077, DOI 10.17487/RFC5077, January 2008, <https://www.rfc-editor.org/info/rfc5077>.
[RFC5077]Salowey,J.,Zhou,H.,Eronen,P.,和H.Tschofenig,“无服务器端状态的传输层安全(TLS)会话恢复”,RFC 5077,DOI 10.17487/RFC5077,2008年1月<https://www.rfc-editor.org/info/rfc5077>.
[RFC7413] Cheng, Y., Chu, J., Radhakrishnan, S., and A. Jain, "TCP Fast Open", RFC 7413, DOI 10.17487/RFC7413, December 2014, <https://www.rfc-editor.org/info/rfc7413>.
[RFC7413]Cheng,Y.,Chu,J.,Radhakrishnan,S.,和A.Jain,“TCP快速开放”,RFC 7413,DOI 10.17487/RFC74132014年12月<https://www.rfc-editor.org/info/rfc7413>.
[RFC8259] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data Interchange Format", STD 90, RFC 8259, DOI 10.17487/RFC8259, December 2017, <https://www.rfc-editor.org/info/rfc8259>.
[RFC8259]Bray,T.,Ed.“JavaScript对象表示法(JSON)数据交换格式”,STD 90,RFC 8259,DOI 10.17487/RFC8259,2017年12月<https://www.rfc-editor.org/info/rfc8259>.
[RFC8427] Hoffman, P., "Representing DNS Messages in JSON", RFC 8427, DOI 10.17487/RFC8427, July 2018, <https://www.rfc-editor.org/info/rfc8427>.
[RFC8427]Hoffman,P.,“用JSON表示DNS消息”,RFC 8427,DOI 10.17487/RFC8427,2018年7月<https://www.rfc-editor.org/info/rfc8427>.
[rrclasses] IANA, "DNS CLASSes", <https://www.iana.org/assignments/dns-parameters/>.
[rrclasses]IANA,“DNS类”<https://www.iana.org/assignments/dns-parameters/>.
[rrtypes] IANA, "Resource Record (RR) TYPEs", <https://www.iana.org/assignments/dns-parameters/>.
[rrtypes]IANA,“资源记录(RR)类型”<https://www.iana.org/assignments/dns-parameters/>.
[RSSAC04] ICANN, "Recommendations on Anonymization Processes for Source IP Addresses Submitted for Future Analysis", August 2018, <https://www.icann.org/en/system/files/files/ rssac-040-07aug18-en.pdf>.
[RSSAC04]ICANN,“提交供未来分析的源IP地址匿名化流程建议”,2018年8月<https://www.icann.org/en/system/files/files/ rssac-040-07aug18-en.pdf>。
[snappy] "snappy", <https://google.github.io/snappy/>.
[snappy]“snappy”<https://google.github.io/snappy/>.
[snzip] "Snzip, a compression/decompression tool based on snappy", commit 809c6f2, October 2018, <https://github.com/kubo/snzip>.
[snzip]“snzip,一种基于snappy的压缩/解压缩工具”,提交809c6f2,2018年10月<https://github.com/kubo/snzip>.
[xz] "XZ Utils", <https://tukaani.org/xz/>.
[xz] “XZ Utils”<https://tukaani.org/xz/>.
[zstd] "Zstandard - Real-time data compression algorithm", <https://facebook.github.io/zstd/>.
[zstd]“ZS标准-实时数据压缩算法”<https://facebook.github.io/zstd/>.
This appendix gives a CDDL [RFC8610] specification for C-DNS.
本附录给出了C-DNS的CDDL[RFC8610]规范。
CDDL does not permit a range of allowed values to be specified for a bitfield. Where necessary, those values are given as a CDDL group, but the group definition is commented out to prevent CDDL tooling from warning that the group is unused.
CDDL不允许为位字段指定允许值的范围。必要时,这些值作为CDDL组给出,但组定义被注释掉,以防止CDDL工具警告该组未使用。
; CDDL specification of the file format for C-DNS, ; which describes a collection of DNS messages and ; traffic metadata.
; CDDL specification of the file format for C-DNS, ; which describes a collection of DNS messages and ; traffic metadata.
; ; The overall structure of a file. ; File = [ file-type-id : "C-DNS", file-preamble : FilePreamble, file-blocks : [* Block], ]
; ; The overall structure of a file. ; File = [ file-type-id : "C-DNS", file-preamble : FilePreamble, file-blocks : [* Block], ]
; ; The File Preamble. ; FilePreamble = { major-format-version => 1, minor-format-version => 0, ? private-version => uint, block-parameters => [+ BlockParameters], } major-format-version = 0 minor-format-version = 1 private-version = 2 block-parameters = 3
; ; The File Preamble. ; FilePreamble = { major-format-version => 1, minor-format-version => 0, ? private-version => uint, block-parameters => [+ BlockParameters], } major-format-version = 0 minor-format-version = 1 private-version = 2 block-parameters = 3
BlockParameters = { storage-parameters => StorageParameters, ? collection-parameters => CollectionParameters, } storage-parameters = 0 collection-parameters = 1
BlockParameters = { storage-parameters => StorageParameters, ? collection-parameters => CollectionParameters, } storage-parameters = 0 collection-parameters = 1
IPv6PrefixLength = 1..128 IPv4PrefixLength = 1..32 OpcodeRange = 0..15 RRTypeRange = 0..65535
IPv6PrefixLength=1..128 IPv4PrefixLength=1..32操作码范围=0..15 RRTypeRange=0..65535
StorageParameters = { ticks-per-second => uint, max-block-items => uint, storage-hints => StorageHints, opcodes => [+ OpcodeRange], rr-types => [+ RRTypeRange], ? storage-flags => StorageFlags, ? client-address-prefix-ipv4 => IPv4PrefixLength, ? client-address-prefix-ipv6 => IPv6PrefixLength, ? server-address-prefix-ipv4 => IPv4PrefixLength, ? server-address-prefix-ipv6 => IPv6PrefixLength, ? sampling-method => tstr, ? anonymization-method => tstr, } ticks-per-second = 0 max-block-items = 1 storage-hints = 2 opcodes = 3 rr-types = 4 storage-flags = 5 client-address-prefix-ipv4 = 6 client-address-prefix-ipv6 = 7 server-address-prefix-ipv4 = 8 server-address-prefix-ipv6 = 9 sampling-method = 10 anonymization-method = 11
StorageParameters = { ticks-per-second => uint, max-block-items => uint, storage-hints => StorageHints, opcodes => [+ OpcodeRange], rr-types => [+ RRTypeRange], ? storage-flags => StorageFlags, ? client-address-prefix-ipv4 => IPv4PrefixLength, ? client-address-prefix-ipv6 => IPv6PrefixLength, ? server-address-prefix-ipv4 => IPv4PrefixLength, ? server-address-prefix-ipv6 => IPv6PrefixLength, ? sampling-method => tstr, ? anonymization-method => tstr, } ticks-per-second = 0 max-block-items = 1 storage-hints = 2 opcodes = 3 rr-types = 4 storage-flags = 5 client-address-prefix-ipv4 = 6 client-address-prefix-ipv6 = 7 server-address-prefix-ipv4 = 8 server-address-prefix-ipv6 = 9 sampling-method = 10 anonymization-method = 11
; A hint indicates whether the collection method will always omit ; the item from the file. StorageHints = { query-response-hints => QueryResponseHints, query-response-signature-hints => QueryResponseSignatureHints, rr-hints => RRHints, other-data-hints => OtherDataHints, } query-response-hints = 0 query-response-signature-hints = 1 rr-hints = 2 other-data-hints = 3
; A hint indicates whether the collection method will always omit ; the item from the file. StorageHints = { query-response-hints => QueryResponseHints, query-response-signature-hints => QueryResponseSignatureHints, rr-hints => RRHints, other-data-hints => OtherDataHints, } query-response-hints = 0 query-response-signature-hints = 1 rr-hints = 2 other-data-hints = 3
QueryResponseHintValues = &( time-offset : 0, client-address-index : 1, client-port : 2, transaction-id : 3, qr-signature-index : 4, client-hoplimit : 5,
QueryResponseHintValues = &( time-offset : 0, client-address-index : 1, client-port : 2, transaction-id : 3, qr-signature-index : 4, client-hoplimit : 5,
response-delay : 6, query-name-index : 7, query-size : 8, response-size : 9, response-processing-data : 10, query-question-sections : 11, ; Second & subsequent ; Questions query-answer-sections : 12, query-authority-sections : 13, query-additional-sections : 14, response-answer-sections : 15, response-authority-sections : 16, response-additional-sections : 17, ) QueryResponseHints = uint .bits QueryResponseHintValues
响应延迟:6,查询名称索引:7,查询大小:8,响应大小:9,响应处理数据:10,查询问题段:11;第二次及以后;问题查询回答部分:12,查询权限部分:13,查询附加部分:14,响应回答部分:15,响应权限部分:16,响应附加部分:17,)QueryResponseHints=uint.bits QueryResponseHintValues
QueryResponseSignatureHintValues = &( server-address-index : 0, server-port : 1, qr-transport-flags : 2, qr-type : 3, qr-sig-flags : 4, query-opcode : 5, qr-dns-flags : 6, query-rcode : 7, query-classtype-index : 8, query-qdcount : 9, query-ancount : 10, query-nscount : 11, query-arcount : 12, query-edns-version : 13, query-udp-size : 14, query-opt-rdata-index : 15, response-rcode : 16, ) QueryResponseSignatureHints = uint .bits QueryResponseSignatureHintValues
QueryResponseSignatureHintValues = &( server-address-index : 0, server-port : 1, qr-transport-flags : 2, qr-type : 3, qr-sig-flags : 4, query-opcode : 5, qr-dns-flags : 6, query-rcode : 7, query-classtype-index : 8, query-qdcount : 9, query-ancount : 10, query-nscount : 11, query-arcount : 12, query-edns-version : 13, query-udp-size : 14, query-opt-rdata-index : 15, response-rcode : 16, ) QueryResponseSignatureHints = uint .bits QueryResponseSignatureHintValues
RRHintValues = &( ttl : 0, rdata-index : 1, ) RRHints = uint .bits RRHintValues
RRHintValues = &( ttl : 0, rdata-index : 1, ) RRHints = uint .bits RRHintValues
OtherDataHintValues = &( malformed-messages : 0, address-event-counts : 1, )
OtherDataHintValues = &( malformed-messages : 0, address-event-counts : 1, )
OtherDataHints = uint .bits OtherDataHintValues
OtherDataHints=uint.bits OtherDataHintValue
StorageFlagValues = &( anonymized-data : 0, sampled-data : 1, normalized-names : 2, ) StorageFlags = uint .bits StorageFlagValues
StorageFlagValues = &( anonymized-data : 0, sampled-data : 1, normalized-names : 2, ) StorageFlags = uint .bits StorageFlagValues
; Metadata about data collection VLANIdRange = 1..4094
; 关于数据收集VLANIdRange=1..4094的元数据
CollectionParameters = { ? query-timeout => uint, ; Milliseconds ? skew-timeout => uint, ; Microseconds ? snaplen => uint, ? promisc => bool, ? interfaces => [+ tstr], ? server-addresses => [+ IPAddress], ? vlan-ids => [+ VLANIdRange], ? filter => tstr, ? generator-id => tstr, ? host-id => tstr, } query-timeout = 0 skew-timeout = 1 snaplen = 2 promisc = 3 interfaces = 4 server-addresses = 5 vlan-ids = 6 filter = 7 generator-id = 8 host-id = 9
CollectionParameters = { ? query-timeout => uint, ; Milliseconds ? skew-timeout => uint, ; Microseconds ? snaplen => uint, ? promisc => bool, ? interfaces => [+ tstr], ? server-addresses => [+ IPAddress], ? vlan-ids => [+ VLANIdRange], ? filter => tstr, ? generator-id => tstr, ? host-id => tstr, } query-timeout = 0 skew-timeout = 1 snaplen = 2 promisc = 3 interfaces = 4 server-addresses = 5 vlan-ids = 6 filter = 7 generator-id = 8 host-id = 9
; ; Data in the file is stored in Blocks. ; Block = { block-preamble => BlockPreamble, ? block-statistics => BlockStatistics, ; Much of this ; could be derived ? block-tables => BlockTables, ? query-responses => [+ QueryResponse], ? address-event-counts => [+ AddressEventCount], ? malformed-messages => [+ MalformedMessage], }
; ; Data in the file is stored in Blocks. ; Block = { block-preamble => BlockPreamble, ? block-statistics => BlockStatistics, ; Much of this ; could be derived ? block-tables => BlockTables, ? query-responses => [+ QueryResponse], ? address-event-counts => [+ AddressEventCount], ? malformed-messages => [+ MalformedMessage], }
block-preamble = 0 block-statistics = 1 block-tables = 2 query-responses = 3 address-event-counts = 4 malformed-messages = 5
块前导码=0块统计数据=1块表=2个查询响应=3个地址事件计数=4个格式错误的消息=5
; ; The (mandatory) preamble to a Block. ; BlockPreamble = { ? earliest-time => Timestamp, ? block-parameters-index => uint .default 0, } earliest-time = 0 block-parameters-index = 1
; ; The (mandatory) preamble to a Block. ; BlockPreamble = { ? earliest-time => Timestamp, ? block-parameters-index => uint .default 0, } earliest-time = 0 block-parameters-index = 1
; Ticks are sub-second intervals. The number of ticks in a second is ; file/block metadata. Signed and unsigned tick types are defined. ticks = int uticks = uint
; 滴答声是亚秒的间隔。一秒钟内的滴答声数为;文件/块元数据。定义了有符号和无符号记号类型。滴答声=整数=整数
Timestamp = [ timestamp-secs : uint, ; POSIX time timestamp-ticks : uticks, ]
Timestamp = [ timestamp-secs : uint, ; POSIX time timestamp-ticks : uticks, ]
; ; Statistics about the Block contents. ; BlockStatistics = { ? processed-messages => uint, ? qr-data-items => uint, ? unmatched-queries => uint, ? unmatched-responses => uint, ? discarded-opcode => uint, ? malformed-items => uint, } processed-messages = 0 qr-data-items = 1 unmatched-queries = 2 unmatched-responses = 3 discarded-opcode = 4 malformed-items = 5
; ; Statistics about the Block contents. ; BlockStatistics = { ? processed-messages => uint, ? qr-data-items => uint, ? unmatched-queries => uint, ? unmatched-responses => uint, ? discarded-opcode => uint, ? malformed-items => uint, } processed-messages = 0 qr-data-items = 1 unmatched-queries = 2 unmatched-responses = 3 discarded-opcode = 4 malformed-items = 5
; ; Tables of common data referenced from records in a Block. ; BlockTables = { ? ip-address => [+ IPAddress], ? classtype => [+ ClassType], ? name-rdata => [+ bstr], ; Holds both names ; and RDATA ? qr-sig => [+ QueryResponseSignature], ? QuestionTables, ? RRTables, ? malformed-message-data => [+ MalformedMessageData], } ip-address = 0 classtype = 1 name-rdata = 2 qr-sig = 3 qlist = 4 qrr = 5 rrlist = 6 rr = 7 malformed-message-data = 8
; ; Tables of common data referenced from records in a Block. ; BlockTables = { ? ip-address => [+ IPAddress], ? classtype => [+ ClassType], ? name-rdata => [+ bstr], ; Holds both names ; and RDATA ? qr-sig => [+ QueryResponseSignature], ? QuestionTables, ? RRTables, ? malformed-message-data => [+ MalformedMessageData], } ip-address = 0 classtype = 1 name-rdata = 2 qr-sig = 3 qlist = 4 qrr = 5 rrlist = 6 rr = 7 malformed-message-data = 8
IPv4Address = bstr .size (0..4) IPv6Address = bstr .size (0..16) IPAddress = IPv4Address / IPv6Address
IPv4Address=bstr.size(0..4)IPv6Address=bstr.size(0..16)IPAddress=IPv4Address/IPv6Address
ClassType = { type => uint, class => uint, } type = 0 class = 1
ClassType = { type => uint, class => uint, } type = 0 class = 1
QueryResponseSignature = { ? server-address-index => uint, ? server-port => uint, ? qr-transport-flags => QueryResponseTransportFlags, ? qr-type => QueryResponseType, ? qr-sig-flags => QueryResponseFlags, ? query-opcode => uint, ? qr-dns-flags => DNSFlags, ? query-rcode => uint, ? query-classtype-index => uint, ? query-qdcount => uint, ? query-ancount => uint, ? query-nscount => uint, ? query-arcount => uint,
QueryResponseSignature = { ? server-address-index => uint, ? server-port => uint, ? qr-transport-flags => QueryResponseTransportFlags, ? qr-type => QueryResponseType, ? qr-sig-flags => QueryResponseFlags, ? query-opcode => uint, ? qr-dns-flags => DNSFlags, ? query-rcode => uint, ? query-classtype-index => uint, ? query-qdcount => uint, ? query-ancount => uint, ? query-nscount => uint, ? query-arcount => uint,
? query-edns-version => uint, ? query-udp-size => uint, ? query-opt-rdata-index => uint, ? response-rcode => uint, } server-address-index = 0 server-port = 1 qr-transport-flags = 2 qr-type = 3 qr-sig-flags = 4 query-opcode = 5 qr-dns-flags = 6 query-rcode = 7 query-classtype-index = 8 query-qdcount = 9 query-ancount = 10 query-nscount = 11 query-arcount = 12 query-edns-version = 13 query-udp-size = 14 query-opt-rdata-index = 15 response-rcode = 16
? query-edns-version => uint, ? query-udp-size => uint, ? query-opt-rdata-index => uint, ? response-rcode => uint, } server-address-index = 0 server-port = 1 qr-transport-flags = 2 qr-type = 3 qr-sig-flags = 4 query-opcode = 5 qr-dns-flags = 6 query-rcode = 7 query-classtype-index = 8 query-qdcount = 9 query-ancount = 10 query-nscount = 11 query-arcount = 12 query-edns-version = 13 query-udp-size = 14 query-opt-rdata-index = 15 response-rcode = 16
; Transport gives the values that may appear in bits 1..4 of ; TransportFlags. There is currently no way to express this in ; CDDL, so Transport is unused. To avoid confusion when used ; with CDDL tools, it is commented out. ; ; Transport = &( ; udp : 0, ; tcp : 1, ; tls : 2, ; dtls : 3, ; https : 4, ; non-standard : 15, ; )
; Transport gives the values that may appear in bits 1..4 of ; TransportFlags. There is currently no way to express this in ; CDDL, so Transport is unused. To avoid confusion when used ; with CDDL tools, it is commented out. ; ; Transport = &( ; udp : 0, ; tcp : 1, ; tls : 2, ; dtls : 3, ; https : 4, ; non-standard : 15, ; )
TransportFlagValues = &( ip-version : 0, ; 0=IPv4, 1=IPv6 ) / (1..4) TransportFlags = uint .bits TransportFlagValues
TransportFlagValues = &( ip-version : 0, ; 0=IPv4, 1=IPv6 ) / (1..4) TransportFlags = uint .bits TransportFlagValues
QueryResponseTransportFlagValues = &( query-trailingdata : 5, ) / TransportFlagValues QueryResponseTransportFlags = uint .bits QueryResponseTransportFlagValues
QueryResponseTransportFlagValues = &( query-trailingdata : 5, ) / TransportFlagValues QueryResponseTransportFlags = uint .bits QueryResponseTransportFlagValues
QueryResponseType = &( stub : 0, client : 1, resolver : 2, auth : 3, forwarder : 4, tool : 5, )
QueryResponseType = &( stub : 0, client : 1, resolver : 2, auth : 3, forwarder : 4, tool : 5, )
QueryResponseFlagValues = &( has-query : 0, has-response : 1, query-has-opt : 2, response-has-opt : 3, query-has-no-question : 4, response-has-no-question: 5, ) QueryResponseFlags = uint .bits QueryResponseFlagValues
QueryResponseFlagValues = &( has-query : 0, has-response : 1, query-has-opt : 2, response-has-opt : 3, query-has-no-question : 4, response-has-no-question: 5, ) QueryResponseFlags = uint .bits QueryResponseFlagValues
DNSFlagValues = &( query-cd : 0, query-ad : 1, query-z : 2, query-ra : 3, query-rd : 4, query-tc : 5, query-aa : 6, query-do : 7, response-cd: 8, response-ad: 9, response-z : 10, response-ra: 11, response-rd: 12, response-tc: 13, response-aa: 14, ) DNSFlags = uint .bits DNSFlagValues
DNSFlagValues = &( query-cd : 0, query-ad : 1, query-z : 2, query-ra : 3, query-rd : 4, query-tc : 5, query-aa : 6, query-do : 7, response-cd: 8, response-ad: 9, response-z : 10, response-ra: 11, response-rd: 12, response-tc: 13, response-aa: 14, ) DNSFlags = uint .bits DNSFlagValues
QuestionTables = ( qlist => [+ QuestionList], qrr => [+ Question] )
QuestionTables = ( qlist => [+ QuestionList], qrr => [+ Question] )
QuestionList = [+ uint] ; Index of Question
QuestionList = [+ uint] ; Index of Question
Question = { ; Second and subsequent Questions name-index => uint, ; Index to a name in the ; name-rdata table classtype-index => uint, } name-index = 0 classtype-index = 1
Question = { ; Second and subsequent Questions name-index => uint, ; Index to a name in the ; name-rdata table classtype-index => uint, } name-index = 0 classtype-index = 1
RRTables = ( rrlist => [+ RRList], rr => [+ RR] )
RRTables = ( rrlist => [+ RRList], rr => [+ RR] )
RRList = [+ uint] ; Index of RR
RRList = [+ uint] ; Index of RR
RR = { name-index => uint, ; Index to a name in the ; name-rdata table classtype-index => uint, ? ttl => uint, ? rdata-index => uint, ; Index to RDATA in the ; name-rdata table } ; Other map key values already defined above. ttl = 2 rdata-index = 3
RR = { name-index => uint, ; Index to a name in the ; name-rdata table classtype-index => uint, ? ttl => uint, ? rdata-index => uint, ; Index to RDATA in the ; name-rdata table } ; Other map key values already defined above. ttl = 2 rdata-index = 3
MalformedMessageData = { ? server-address-index => uint, ? server-port => uint, ? mm-transport-flags => TransportFlags, ? mm-payload => bstr, } ; Other map key values already defined above. mm-transport-flags = 2 mm-payload = 3
MalformedMessageData = { ? server-address-index => uint, ? server-port => uint, ? mm-transport-flags => TransportFlags, ? mm-payload => bstr, } ; Other map key values already defined above. mm-transport-flags = 2 mm-payload = 3
; ; A single Query/Response data item. ; QueryResponse = { ? time-offset => uticks, ; Time offset from ; start of Block ? client-address-index => uint, ? client-port => uint, ? transaction-id => uint, ? qr-signature-index => uint, ? client-hoplimit => uint, ? response-delay => ticks, ? query-name-index => uint, ? query-size => uint, ; DNS size of Query ? response-size => uint, ; DNS size of Response ? response-processing-data => ResponseProcessingData, ? query-extended => QueryResponseExtended, ? response-extended => QueryResponseExtended, } time-offset = 0 client-address-index = 1 client-port = 2 transaction-id = 3 qr-signature-index = 4 client-hoplimit = 5 response-delay = 6 query-name-index = 7 query-size = 8 response-size = 9 response-processing-data = 10 query-extended = 11 response-extended = 12
; ; A single Query/Response data item. ; QueryResponse = { ? time-offset => uticks, ; Time offset from ; start of Block ? client-address-index => uint, ? client-port => uint, ? transaction-id => uint, ? qr-signature-index => uint, ? client-hoplimit => uint, ? response-delay => ticks, ? query-name-index => uint, ? query-size => uint, ; DNS size of Query ? response-size => uint, ; DNS size of Response ? response-processing-data => ResponseProcessingData, ? query-extended => QueryResponseExtended, ? response-extended => QueryResponseExtended, } time-offset = 0 client-address-index = 1 client-port = 2 transaction-id = 3 qr-signature-index = 4 client-hoplimit = 5 response-delay = 6 query-name-index = 7 query-size = 8 response-size = 9 response-processing-data = 10 query-extended = 11 response-extended = 12
ResponseProcessingData = { ? bailiwick-index => uint, ? processing-flags => ResponseProcessingFlags, } bailiwick-index = 0 processing-flags = 1
ResponseProcessingData = { ? bailiwick-index => uint, ? processing-flags => ResponseProcessingFlags, } bailiwick-index = 0 processing-flags = 1
ResponseProcessingFlagValues = &( from-cache : 0, ) ResponseProcessingFlags = uint .bits ResponseProcessingFlagValues
ResponseProcessingFlagValues = &( from-cache : 0, ) ResponseProcessingFlags = uint .bits ResponseProcessingFlagValues
QueryResponseExtended = { ? question-index => uint, ; Index of QuestionList ? answer-index => uint, ; Index of RRList ? authority-index => uint, ? additional-index => uint, } question-index = 0 answer-index = 1 authority-index = 2 additional-index = 3
QueryResponseExtended = { ? question-index => uint, ; Index of QuestionList ? answer-index => uint, ; Index of RRList ? authority-index => uint, ? additional-index => uint, } question-index = 0 answer-index = 1 authority-index = 2 additional-index = 3
; ; Address event data. ; AddressEventCount = { ae-type => &AddressEventType, ? ae-code => uint, ae-address-index => uint, ? ae-transport-flags => TransportFlags, ae-count => uint, } ae-type = 0 ae-code = 1 ae-address-index = 2 ae-transport-flags = 3 ae-count = 4
; ; Address event data. ; AddressEventCount = { ae-type => &AddressEventType, ? ae-code => uint, ae-address-index => uint, ? ae-transport-flags => TransportFlags, ae-count => uint, } ae-type = 0 ae-code = 1 ae-address-index = 2 ae-transport-flags = 3 ae-count = 4
AddressEventType = ( tcp-reset : 0, icmp-time-exceeded : 1, icmp-dest-unreachable : 2, icmpv6-time-exceeded : 3, icmpv6-dest-unreachable: 4, icmpv6-packet-too-big : 5, )
AddressEventType = ( tcp-reset : 0, icmp-time-exceeded : 1, icmp-dest-unreachable : 2, icmpv6-time-exceeded : 3, icmpv6-dest-unreachable: 4, icmpv6-packet-too-big : 5, )
; ; Malformed messages. ; MalformedMessage = { ? time-offset => uticks, ; Time offset from ; start of Block ? client-address-index => uint, ? client-port => uint, ? message-data-index => uint, } ; Other map key values already defined above. message-data-index = 3
; ; Malformed messages. ; MalformedMessage = { ? time-offset => uticks, ; Time offset from ; start of Block ? client-address-index => uint, ? client-port => uint, ? message-data-index => uint, } ; Other map key values already defined above. message-data-index = 3
The basic algorithm, which follows the guidance in [RFC1035], is simply to collect each name, and the offset in the packet at which it starts, during packet construction. As each name is added, it is offered to each of the collected names in order of collection, starting from the first name. If (1) labels at the end of the name can be replaced with a reference back to part (or all) of the earlier name and (2) the uncompressed part of the name is shorter than any compression already found, the earlier name is noted as the compression target for the name.
基本算法遵循[RFC1035]中的指导,只需在数据包构造过程中收集每个名称以及数据包开始时的偏移量。在添加每个名称时,将按照收集顺序向每个收集的名称提供该名称,从第一个名称开始。如果(1)名称末尾的标签可以替换为对早期名称的部分(或全部)的引用,并且(2)名称的未压缩部分比已找到的任何压缩短,则早期名称将作为名称的压缩目标。
The following tables illustrate the step-by-step process of adding names and performing name compression. In an example packet, the first name added is foo.example, which cannot be compressed.
下表说明了添加名称和执行名称压缩的分步过程。在示例数据包中,添加的第一个名称是foo.example,它不能被压缩。
+---+-------------+--------------+--------------------+ | N | Name | Uncompressed | Compression Target | +---+-------------+--------------+--------------------+ | 1 | foo.example | foo.example | None | +---+-------------+--------------+--------------------+
+---+-------------+--------------+--------------------+ | N | Name | Uncompressed | Compression Target | +---+-------------+--------------+--------------------+ | 1 | foo.example | foo.example | None | +---+-------------+--------------+--------------------+
The next name added is bar.example. This is matched against foo.example. The example part of this can be used as a compression target, with the remaining uncompressed part of the name being bar.
添加的下一个名称是bar.example。这与foo.example匹配。其中的示例部分可用作压缩目标,名称的剩余未压缩部分为bar。
+---+-------------+--------------+-----------------------+ | N | Name | Uncompressed | Compression Target | +---+-------------+--------------+-----------------------+ | 1 | foo.example | foo.example | None | | 2 | bar.example | bar | 1 + offset to example | +---+-------------+--------------+-----------------------+
+---+-------------+--------------+-----------------------+ | N | Name | Uncompressed | Compression Target | +---+-------------+--------------+-----------------------+ | 1 | foo.example | foo.example | None | | 2 | bar.example | bar | 1 + offset to example | +---+-------------+--------------+-----------------------+
The third name added is www.bar.example. This is first matched against foo.example, and as before this is recorded as a compression target, with the remaining uncompressed part of the name being www.bar. It is then matched against the second name, which again can be a compression target. Because the remaining uncompressed part of the name is www, this is an improved compression, and so it is adopted.
添加的第三个名称是www.bar.example。这首先与foo.example匹配,与之前一样,这被记录为压缩目标,名称的剩余未压缩部分为www.bar。然后将其与第二个名称匹配,第二个名称也可以是压缩目标。由于名称的剩余未压缩部分是www,因此这是一种改进的压缩,因此被采用。
+---+-----------------+--------------+-----------------------+ | N | Name | Uncompressed | Compression Target | +---+-----------------+--------------+-----------------------+ | 1 | foo.example | foo.example | None | | 2 | bar.example | bar | 1 + offset to example | | 3 | www.bar.example | www | 2 | +---+-----------------+--------------+-----------------------+
+---+-----------------+--------------+-----------------------+ | N | Name | Uncompressed | Compression Target | +---+-----------------+--------------+-----------------------+ | 1 | foo.example | foo.example | None | | 2 | bar.example | bar | 1 + offset to example | | 3 | www.bar.example | www | 2 | +---+-----------------+--------------+-----------------------+
As an optimization, if a name is already perfectly compressed (in other words, the uncompressed part of the name is empty), then no further names will be considered for compression.
作为一种优化,如果名称已经完全压缩(换句话说,名称的未压缩部分为空),则不会考虑进一步压缩名称。
Using the above basic algorithm, the packet lengths of Responses generated by the Name Server Daemon (NSD) [NSD] can be matched almost exactly. At the time of writing, a tiny number (<.01%) of the reconstructed packets had incorrect lengths.
使用上述基本算法,名称服务器守护程序(NSD)[NSD]生成的响应的数据包长度几乎可以精确匹配。在编写本文时,极少数(<.01%)的重构数据包的长度不正确。
The Knot Authoritative name server [Knot] uses different compression behavior, which is the result of internal optimization designed to balance runtime speed with compression size gains. In brief, and omitting complications, Knot Authoritative will only consider the QNAME and names in the immediately preceding RR section in an RRSET as compression targets.
Knot权威名称服务器[Knot]使用不同的压缩行为,这是内部优化的结果,旨在平衡运行速度和压缩大小增益。总之,省略并发症,结权威只考虑在RRSET中作为压缩目标的前一个RR部分中的QNEY和名称。
A set of smart heuristics as described below can be implemented to mimic this, and while not perfect, it produces output nearly, but not quite, as good a match as with NSD. The heuristics are as follows:
可以实现如下所述的一组智能启发法来模拟这一点,虽然不完美,但它产生的输出几乎与NSD匹配,但并不完全匹配。试探法如下:
1. A match is only perfect if the name is completely compressed AND the TYPE of the section in which the name occurs matches the TYPE of the name used as the compression target.
1. 只有当名称被完全压缩并且名称所在的节的类型与用作压缩目标的名称的类型匹配时,匹配才是完美的。
2. If the name occurs in RDATA:
2. 如果名称出现在RDATA中:
* If the compression target name is in a Query, then only the first RR in an RRSET can use that name as a compression target.
* 如果压缩目标名称在查询中,则只有RRSET中的第一个RR可以将该名称用作压缩目标。
* The compression target name MUST be in RDATA.
* 压缩目标名称必须采用RDATA格式。
* The name section TYPE must match the compression target name section TYPE.
* 名称节类型必须与压缩目标名称节类型匹配。
* The compression target name MUST be in the immediately preceding RR in the RRSET.
* 压缩目标名称必须位于RRSET中紧靠前的RR中。
Using this algorithm, less than 0.1% of the reconstructed packets had incorrect lengths.
使用该算法,不到0.1%的重构数据包具有不正确的长度。
In sample traffic collected on a root name server, around 2-4% of Responses generated by Knot had different packet lengths than those produced by NSD.
在根名称服务器上收集的样本流量中,Knot生成的响应中约有2-4%的数据包长度与NSD生成的数据包长度不同。
Several binary serialization formats were considered. For completeness, they were also compared to JSON.
考虑了几种二进制序列化格式。为了完整性,还将它们与JSON进行了比较。
o Apache Avro [Avro]. Data is stored according to a predefined schema. The schema itself is always included in the data file. Data can therefore be stored untagged, for a smaller serialization size, and be written and read by an Avro library.
o apacheavro[Avro]。数据根据预定义的模式存储。模式本身始终包含在数据文件中。因此,数据可以不加标记地存储,用于较小的序列化大小,并由Avro库写入和读取。
* At the time of writing, Avro libraries are available for C, C++, C#, Java, Python, Ruby, and PHP. Optionally, tools are available for C++, Java, and C# to generate code for encoding and decoding.
* 在编写时,AVro库可用于C、C++、C、java、Python、Ruby和PHP。可选地,工具可用于C++、java和C语言生成编码和解码的代码。
o Google Protocol Buffers [Protocol-Buffers]. Data is stored according to a predefined schema. The schema is used by a generator to generate code for encoding and decoding the data. Data can therefore be stored untagged, for a smaller serialization size. The schema is not stored with the data, so unlike Avro, it cannot be read with a generic library.
o 谷歌协议缓冲区[协议缓冲区]。数据根据预定义的模式存储。生成器使用模式生成编码和解码数据的代码。因此,可以不标记地存储数据,以获得较小的序列化大小。模式不与数据一起存储,因此与Avro不同,它不能用通用库读取。
* Code must be generated for a particular data schema to read and write data using that schema. At the time of writing, the Google code generator can currently generate code for encoding and decoding a schema for C++, Go, Java, Python, Ruby, C#, Objective-C, JavaScript, and PHP.
* 必须为特定数据架构生成代码,才能使用该架构读写数据。在编写时,谷歌代码生成器可以生成用于编码和解码用于C++、GO、java、Python、露比、Cype、Objto-C、JavaScript和PHP的模式的代码。
o CBOR [RFC7049]. This serialization format is comparable to JSON but with a binary representation. It does not use a predefined schema, so data is always stored tagged. However, CBOR data schemas can be described using CDDL [RFC8610], and tools exist to verify that data files conform to the schema.
o CBOR[RFC7049]。这种序列化格式类似于JSON,但具有二进制表示。它不使用预定义的模式,因此数据总是标记存储。但是,CBOR数据模式可以使用CDDL[RFC8610]来描述,并且存在用于验证数据文件是否符合模式的工具。
* CBOR is a simple format and is simple to implement. At the time of writing, the CBOR website lists implementations for 16 languages.
* CBOR是一种简单的格式,易于实现。在撰写本文时,CBOR网站列出了16种语言的实施情况。
Avro and Protocol Buffers both allow storage of untagged data, but because they rely on the data schema for this, their implementation is considerably more complex than CBOR. Using Avro or Protocol Buffers in an unsupported environment would require notably greater development effort compared to CBOR.
Avro和协议缓冲区都允许存储未标记的数据,但由于它们依赖于数据模式,因此它们的实现比CBOR复杂得多。与CBOR相比,在不受支持的环境中使用Avro或协议缓冲区需要更大的开发工作量。
A test program was written that reads input from a PCAP file and writes output using one of two basic structures: either a simple structure, where each Query/Response pair is represented in a single record entry, or the C-DNS block structure.
编写了一个测试程序,该程序使用两种基本结构之一从PCAP文件读取输入并写入输出:一种是简单结构,其中每个查询/响应对表示在单个记录条目中,另一种是C-DNS块结构。
The resulting output files were then compressed using a variety of common general-purpose lossless compression tools to explore the compressibility of the formats. The compression tools employed were:
然后使用各种通用无损压缩工具对生成的输出文件进行压缩,以探索格式的可压缩性。使用的压缩工具有:
o snzip [snzip]. A command-line compression tool based on the Google Snappy library [snappy].
o snzip[snzip]。基于Google Snappy库[Snappy]的命令行压缩工具。
o lz4 [lz4]. The command-line compression tool from the reference C LZ4 implementation.
o lz4[lz4]。来自参考C LZ4实现的命令行压缩工具。
o gzip [gzip]. The ubiquitous GNU zip tool.
o gzip[gzip]。无处不在的GNU压缩工具。
o zstd [zstd]. Compression using the Zstandard algorithm.
o zstd[zstd]。使用Zstandard算法进行压缩。
o xz [xz]. A popular compression tool noted for high compression.
o xz[xz]。以高压缩率著称的流行压缩工具。
In all cases, the compression tools were run using their default settings.
在所有情况下,压缩工具都是使用默认设置运行的。
Note that this document does not mandate the use of compression, nor any particular compression scheme, but it anticipates that in practice output data will be subject to general-purpose compression, and so this should be taken into consideration.
请注意,本文件不强制使用压缩,也不强制使用任何特定的压缩方案,但预计在实践中,输出数据将接受通用压缩,因此应考虑到这一点。
"test.pcap", a 662 MB capture of sample data from a root instance, was used for the comparison. The following table shows the formatted size and size after compression (abbreviated to Comp. in the table headers), together with the task Resident Set Size (RSS) and the user time taken by the compression. File sizes are in MB, RSS is in kB, and user time is in seconds.
“test.pcap”是从根实例捕获的662MB样本数据,用于比较。下表显示了格式化的大小和压缩后的大小(在表头中缩写为Comp.),以及任务驻留集大小(RSS)和压缩所花费的用户时间。文件大小以MB为单位,RSS以kB为单位,用户时间以秒为单位。
+-------------+-----------+-------+------------+-------+-----------+ | Format | File Size | Comp. | Comp. Size | RSS | User Time | +-------------+-----------+-------+------------+-------+-----------+ | PCAP | 661.87 | snzip | 212.48 | 2696 | 1.26 | | | | lz4 | 181.58 | 6336 | 1.35 | | | | gzip | 153.46 | 1428 | 18.20 | | | | zstd | 87.07 | 3544 | 4.27 | | | | xz | 49.09 | 97416 | 160.79 | | | | | | | | | JSON simple | 4113.92 | snzip | 603.78 | 2656 | 5.72 | | | | lz4 | 386.42 | 5636 | 5.25 | | | | gzip | 271.11 | 1492 | 73.00 | | | | zstd | 133.43 | 3284 | 8.68 | | | | xz | 51.98 | 97412 | 600.74 | | | | | | | | | Avro simple | 640.45 | snzip | 148.98 | 2656 | 0.90 | | | | lz4 | 111.92 | 5828 | 0.99 | | | | gzip | 103.07 | 1540 | 11.52 | | | | zstd | 49.08 | 3524 | 2.50 | | | | xz | 22.87 | 97308 | 90.34 | | | | | | | | | CBOR simple | 764.82 | snzip | 164.57 | 2664 | 1.11 | | | | lz4 | 120.98 | 5892 | 1.13 | | | | gzip | 110.61 | 1428 | 12.88 | | | | zstd | 54.14 | 3224 | 2.77 | | | | xz | 23.43 | 97276 | 111.48 | | | | | | | | | PBuf simple | 749.51 | snzip | 167.16 | 2660 | 1.08 | | | | lz4 | 123.09 | 5824 | 1.14 | | | | gzip | 112.05 | 1424 | 12.75 | | | | zstd | 53.39 | 3388 | 2.76 | | | | xz | 23.99 | 97348 | 106.47 | | | | | | | | | JSON block | 519.77 | snzip | 106.12 | 2812 | 0.93 | | | | lz4 | 104.34 | 6080 | 0.97 | | | | gzip | 57.97 | 1604 | 12.70 | | | | zstd | 61.51 | 3396 | 3.45 | | | | xz | 27.67 | 97524 | 169.10 | | | | | | | | | Avro block | 60.45 | snzip | 48.38 | 2688 | 0.20 | | | | lz4 | 48.78 | 8540 | 0.22 | | | | gzip | 39.62 | 1576 | 2.92 | | | | zstd | 29.63 | 3612 | 1.25 | | | | xz | 18.28 | 97564 | 25.81 | | | | | | | |
+-------------+-----------+-------+------------+-------+-----------+ | Format | File Size | Comp. | Comp. Size | RSS | User Time | +-------------+-----------+-------+------------+-------+-----------+ | PCAP | 661.87 | snzip | 212.48 | 2696 | 1.26 | | | | lz4 | 181.58 | 6336 | 1.35 | | | | gzip | 153.46 | 1428 | 18.20 | | | | zstd | 87.07 | 3544 | 4.27 | | | | xz | 49.09 | 97416 | 160.79 | | | | | | | | | JSON simple | 4113.92 | snzip | 603.78 | 2656 | 5.72 | | | | lz4 | 386.42 | 5636 | 5.25 | | | | gzip | 271.11 | 1492 | 73.00 | | | | zstd | 133.43 | 3284 | 8.68 | | | | xz | 51.98 | 97412 | 600.74 | | | | | | | | | Avro simple | 640.45 | snzip | 148.98 | 2656 | 0.90 | | | | lz4 | 111.92 | 5828 | 0.99 | | | | gzip | 103.07 | 1540 | 11.52 | | | | zstd | 49.08 | 3524 | 2.50 | | | | xz | 22.87 | 97308 | 90.34 | | | | | | | | | CBOR simple | 764.82 | snzip | 164.57 | 2664 | 1.11 | | | | lz4 | 120.98 | 5892 | 1.13 | | | | gzip | 110.61 | 1428 | 12.88 | | | | zstd | 54.14 | 3224 | 2.77 | | | | xz | 23.43 | 97276 | 111.48 | | | | | | | | | PBuf simple | 749.51 | snzip | 167.16 | 2660 | 1.08 | | | | lz4 | 123.09 | 5824 | 1.14 | | | | gzip | 112.05 | 1424 | 12.75 | | | | zstd | 53.39 | 3388 | 2.76 | | | | xz | 23.99 | 97348 | 106.47 | | | | | | | | | JSON block | 519.77 | snzip | 106.12 | 2812 | 0.93 | | | | lz4 | 104.34 | 6080 | 0.97 | | | | gzip | 57.97 | 1604 | 12.70 | | | | zstd | 61.51 | 3396 | 3.45 | | | | xz | 27.67 | 97524 | 169.10 | | | | | | | | | Avro block | 60.45 | snzip | 48.38 | 2688 | 0.20 | | | | lz4 | 48.78 | 8540 | 0.22 | | | | gzip | 39.62 | 1576 | 2.92 | | | | zstd | 29.63 | 3612 | 1.25 | | | | xz | 18.28 | 97564 | 25.81 | | | | | | | |
| CBOR block | 75.25 | snzip | 53.27 | 2684 | 0.24 | | | | lz4 | 51.88 | 8008 | 0.28 | | | | gzip | 41.17 | 1548 | 4.36 | | | | zstd | 30.61 | 3476 | 1.48 | | | | xz | 18.15 | 97556 | 38.78 | | | | | | | | | PBuf block | 67.98 | snzip | 51.10 | 2636 | 0.24 | | | | lz4 | 52.39 | 8304 | 0.24 | | | | gzip | 40.19 | 1520 | 3.63 | | | | zstd | 31.61 | 3576 | 1.40 | | | | xz | 17.94 | 97440 | 33.99 | +-------------+-----------+-------+------------+-------+-----------+
| CBOR block | 75.25 | snzip | 53.27 | 2684 | 0.24 | | | | lz4 | 51.88 | 8008 | 0.28 | | | | gzip | 41.17 | 1548 | 4.36 | | | | zstd | 30.61 | 3476 | 1.48 | | | | xz | 18.15 | 97556 | 38.78 | | | | | | | | | PBuf block | 67.98 | snzip | 51.10 | 2636 | 0.24 | | | | lz4 | 52.39 | 8304 | 0.24 | | | | gzip | 40.19 | 1520 | 3.63 | | | | zstd | 31.61 | 3576 | 1.40 | | | | xz | 17.94 | 97440 | 33.99 | +-------------+-----------+-------+------------+-------+-----------+
The above results are discussed in the following sections.
下面几节将讨论上述结果。
An important first consideration is whether moving away from PCAP offers significant benefits.
首先要考虑的一个重要问题是,离开PCAP是否会带来显著的好处。
The simple binary formats are typically larger than PCAP, even though they omit some information such as Ethernet Media Access Control (MAC) addresses. But not only do they require less CPU to compress than PCAP, the resulting compressed files are smaller than compressed PCAP.
简单的二进制格式通常比PCAP大,尽管它们省略了一些信息,如以太网媒体访问控制(MAC)地址。但它们不仅需要比PCAP更少的CPU来压缩,而且得到的压缩文件比压缩的PCAP更小。
The intention of the block coding is to perform data deduplication on Query/Response records within the block. The simple and block formats shown above store exactly the same information for each Query/Response record. This information is parsed from the DNS traffic in the input PCAP file, and in all cases each field has an identifier and the field data is typed.
块编码的目的是对块内的查询/响应记录执行重复数据消除。上面显示的简单格式和块格式为每个查询/响应记录存储完全相同的信息。此信息从输入PCAP文件中的DNS流量解析而来,在所有情况下,每个字段都有一个标识符,并键入字段数据。
The data deduplication on the block formats show an order-of-magnitude reduction in the size of the format file size against the simple formats. As would be expected, the compression tools are able to find and exploit a lot of this duplication, but as the deduplication process uses knowledge of DNS traffic, it is able to retain a size advantage. This advantage reduces as stronger compression is applied, as again would be expected, but even with the strongest compression applied the block-formatted data remains around 75% of the size of the simple format and its compression requires roughly a third of the CPU time.
块格式上的重复数据消除显示,与简单格式相比,格式文件大小减少了一个数量级。正如预期的那样,压缩工具能够发现并利用大量这种重复,但由于重复数据消除过程使用了DNS流量的知识,因此它能够保持规模优势。这一优势随着应用更强的压缩而减少,这也是意料之中的,但即使应用了最强的压缩,块格式的数据仍保持简单格式大小的75%左右,其压缩大约需要三分之一的CPU时间。
Text data formats offer many advantages over binary formats, particularly in the areas of ad hoc data inspection and extraction. It was therefore felt worthwhile to carry out a direct comparison, implementing JSON versions of the simple and block formats.
与二进制格式相比,文本数据格式具有许多优势,特别是在特殊数据检查和提取领域。因此,有必要进行直接比较,实现简单格式和块格式的JSON版本。
Concentrating on JSON block format, the format files produced are a significant fraction of an order of magnitude larger than binary formats. The impact on file size after compression is as might be expected from that starting point; the stronger compression produces files that are 150% of the size of similarly compressed binary format and require over 4x more CPU to compress.
专注于JSON块格式,生成的格式文件比二进制格式大一个数量级。压缩后对文件大小的影响与从该起点可以预期的一样;更强大的压缩产生的文件大小是类似压缩二进制格式的150%,需要4倍以上的CPU进行压缩。
Concentrating again on the block formats, all three produce format files that are close to an order of magnitude smaller than the original "test.pcap" file. CBOR produces the largest files and Avro the smallest, 20% smaller than CBOR.
再次将注意力集中在块格式上,这三种格式的文件都比原始的“test.pcap”文件小一个数量级。CBOR生成的文件最大,Avro生成的文件最小,比CBOR小20%。
However, once compression is taken into account, the size difference narrows. At medium compression (with gzip), the size difference is 4%. Using strong compression (with xz), the difference reduces to 2%, with Avro the largest and Protocol Buffers the smallest, although CBOR and Protocol Buffers require slightly more compression CPU.
然而,一旦考虑到压缩,尺寸差异就会缩小。中等压缩(gzip)时,尺寸差异为4%。使用强压缩(使用xz),差异减小到2%,Avro最大,协议缓冲区最小,尽管CBOR和协议缓冲区需要稍多的压缩CPU。
The measurements presented above do not include data on the CPU required to generate the format files. Measurements indicate that writing Avro requires 10% more CPU than CBOR or Protocol Buffers. It appears, therefore, that Avro's advantage in compression CPU usage is probably offset by a larger CPU requirement in writing Avro.
上述测量不包括生成格式文件所需的CPU数据。测量结果表明,编写Avro需要比CBOR或协议缓冲区多10%的CPU。因此,似乎Avro在压缩CPU使用方面的优势可能被编写Avro时更大的CPU需求所抵消。
The above assessments lead us to the choice of a binary format file using blocking.
通过以上评估,我们可以选择使用分块的二进制格式文件。
As noted previously, this document anticipates that output data will be subject to compression. There is no compelling case for one particular binary serialization format in terms of either final file size or machine resources consumed, so the choice must be largely based on other factors. CBOR was therefore chosen as the binary serialization format for the reasons listed in Section 5.
如前所述,本文档预期输出数据将受到压缩。对于一种特定的二进制序列化格式,无论是最终文件大小还是所消耗的机器资源,都没有令人信服的理由,因此选择必须在很大程度上基于其他因素。因此,出于第5节列出的原因,选择CBOR作为二进制序列化格式。
Given the choice of a CBOR format using blocking, the question arises of what an appropriate default value for the maximum number of Query/Response pairs in a block should be. This has two components:
考虑到使用块的CBOR格式的选择,问题是块中最大查询/响应对数的适当默认值应该是多少。这包括两个部分:
1. What is the impact on performance of using different block sizes in the format file?
1. 在格式化文件中使用不同的块大小对性能有什么影响?
2. What is the impact on the size of the format file before and after compression?
2. 压缩前后对格式文件的大小有什么影响?
The following table addresses the performance question, showing the impact on the performance of a C++ program converting "test.pcap" to C-DNS. File sizes are in MB, RSS is in kB, and user time is in seconds.
下表讨论了性能问题,显示了C++程序将“测试.pCAP”转换为C-DNS的性能的影响。文件大小以MB为单位,RSS以kB为单位,用户时间以秒为单位。
+------------+-----------+--------+-----------+ | Block Size | File Size | RSS | User Time | +------------+-----------+--------+-----------+ | 1,000 | 133.46 | 612.27 | 15.25 | | 5,000 | 89.85 | 676.82 | 14.99 | | 10,000 | 76.87 | 752.40 | 14.53 | | 20,000 | 67.86 | 750.75 | 14.49 | | 40,000 | 61.88 | 736.30 | 14.29 | | 80,000 | 58.08 | 694.16 | 14.28 | | 160,000 | 55.94 | 733.84 | 14.44 | | 320,000 | 54.41 | 799.20 | 13.97 | +------------+-----------+--------+-----------+
+------------+-----------+--------+-----------+ | Block Size | File Size | RSS | User Time | +------------+-----------+--------+-----------+ | 1,000 | 133.46 | 612.27 | 15.25 | | 5,000 | 89.85 | 676.82 | 14.99 | | 10,000 | 76.87 | 752.40 | 14.53 | | 20,000 | 67.86 | 750.75 | 14.49 | | 40,000 | 61.88 | 736.30 | 14.29 | | 80,000 | 58.08 | 694.16 | 14.28 | | 160,000 | 55.94 | 733.84 | 14.44 | | 320,000 | 54.41 | 799.20 | 13.97 | +------------+-----------+--------+-----------+
Therefore, increasing block size tends to increase maximum RSS a little, with no significant effect (if anything, a small reduction) on CPU consumption.
因此,增加块大小往往会稍微增加最大RSS,而不会对CPU消耗产生显著影响(如果有任何影响的话,也会稍微减少)。
The following table demonstrates the effect of increasing block size on output file size for different compressions.
下表演示了增加块大小对不同压缩的输出文件大小的影响。
+------------+--------+-------+-------+-------+-------+-------+ | Block Size | None | snzip | lz4 | gzip | zstd | xz | +------------+--------+-------+-------+-------+-------+-------+ | 1,000 | 133.46 | 90.52 | 90.03 | 74.65 | 44.78 | 25.63 | | 5,000 | 89.85 | 59.69 | 59.43 | 46.99 | 37.33 | 22.34 | | 10,000 | 76.87 | 50.39 | 50.28 | 38.94 | 33.62 | 21.09 | | 20,000 | 67.86 | 43.91 | 43.90 | 33.24 | 32.62 | 20.16 | | 40,000 | 61.88 | 39.63 | 39.69 | 29.44 | 28.72 | 19.52 | | 80,000 | 58.08 | 36.93 | 37.01 | 27.05 | 26.25 | 19.00 | | 160,000 | 55.94 | 35.10 | 35.06 | 25.44 | 24.56 | 19.63 | | 320,000 | 54.41 | 33.87 | 33.74 | 24.36 | 23.44 | 18.66 | +------------+--------+-------+-------+-------+-------+-------+
+------------+--------+-------+-------+-------+-------+-------+ | Block Size | None | snzip | lz4 | gzip | zstd | xz | +------------+--------+-------+-------+-------+-------+-------+ | 1,000 | 133.46 | 90.52 | 90.03 | 74.65 | 44.78 | 25.63 | | 5,000 | 89.85 | 59.69 | 59.43 | 46.99 | 37.33 | 22.34 | | 10,000 | 76.87 | 50.39 | 50.28 | 38.94 | 33.62 | 21.09 | | 20,000 | 67.86 | 43.91 | 43.90 | 33.24 | 32.62 | 20.16 | | 40,000 | 61.88 | 39.63 | 39.69 | 29.44 | 28.72 | 19.52 | | 80,000 | 58.08 | 36.93 | 37.01 | 27.05 | 26.25 | 19.00 | | 160,000 | 55.94 | 35.10 | 35.06 | 25.44 | 24.56 | 19.63 | | 320,000 | 54.41 | 33.87 | 33.74 | 24.36 | 23.44 | 18.66 | +------------+--------+-------+-------+-------+-------+-------+
There is obviously scope for tuning the default block size to the compression being employed, traffic characteristics, frequency of output file rollover, etc. Using a strong compression scheme, block sizes over 10,000 Query/Response pairs would seem to offer limited improvements.
显然,可以根据所采用的压缩、流量特征、输出文件滚动频率等调整默认块大小。使用强压缩方案,超过10000个查询/响应对的块大小似乎只能提供有限的改进。
This section specifies the data fields that would need to be captured in order to perform the fullest PCAP traffic reconstruction for well-formed DNS messages that is possible with C-DNS.
本节指定了需要捕获的数据字段,以便对格式良好的DNS消息执行最完整的PCAP流量重建,这在C-DNS中是可能的。
o All data fields in the QueryResponse type except response-processing-data.
o QueryResponse类型中的所有数据字段(响应处理数据除外)。
o All data fields in the QueryResponseSignature type except qr-type.
o QueryResponseSignature类型中的所有数据字段(qr类型除外)。
o All data fields in the RR TYPE.
o RR类型中的所有数据字段。
At the other extreme, an interesting corner case arises when opting to perform captures with a smaller data set than that recommended above. The following list specifies a subset of the above data fields; if only these data fields are captured, then even a minimal traffic reconstruction is problematic because there is not enough information to determine if the Query/Response data item contained just a Query, just a Response, or a Query/Response pair.
在另一个极端,当选择使用比上述建议更小的数据集执行捕获时,会出现一个有趣的情况。以下列表指定了上述数据字段的子集;如果只捕获这些数据字段,那么即使是最小的流量重建也是有问题的,因为没有足够的信息来确定查询/响应数据项是否只包含查询、响应或查询/响应对。
o The following data fields from the QueryResponse type:
o QueryResponse类型中的以下数据字段:
* time-offset
* 时间偏移
* client-address-index
* 客户端地址索引
* client-port
* 客户端端口
* transaction-id
* 事务id
* query-name-index
* 查询名称索引
o The following data fields from the QueryResponseSignature type:
o QueryResponseSignature类型中的以下数据字段:
* server-address-index
* 服务器地址索引
* server-port
* 服务器端口
* qr-transport-flags
* qr运输标志
* query-classtype-index
* 查询类类型索引
In this case, simply also capturing the qr-sig-flags will provide enough information to perform a minimal traffic reconstruction (assuming that suitable defaults for the remaining fields are provided). Additionally, capturing response-delay, query-opcode, and response-rcode will avoid having to rely on potentially misleading defaults for these values and should result in a PCAP that represents the basics of the real traffic flow.
在这种情况下,简单地捕获qr sig标志将提供足够的信息来执行最小流量重建(假设为其余字段提供了合适的默认值)。此外,捕获响应延迟、查询操作码和响应rcode将避免依赖这些值的潜在误导性默认值,并应生成表示实际交通流基本信息的PCAP。
Acknowledgements
致谢
The authors wish to thank CZ.NIC -- in particular, Tomas Gavenciak -- for many useful discussions on binary formats, compression, and packet matching. Thanks also to Jan Vcelak and Wouter Wijngaards for discussions on name compression, and Paul Hoffman for a detailed review of this document and the C-DNS CDDL.
作者希望感谢CZ.NIC——特别是Tomas Gavenciak——对二进制格式、压缩和数据包匹配进行了许多有益的讨论。还要感谢Jan Vcelak和Wouter Wijngaards对名称压缩的讨论,以及Paul Hoffman对本文档和C-DNS CDDL的详细审查。
Thanks also to Robert Edmonds, Jerry Lundstrom, Richard Gibson, Stephane Bortzmeyer, and many other members of DNSOP for review.
还要感谢罗伯特·爱德蒙兹、杰瑞·伦德斯特伦、理查德·吉布森、斯蒂芬·波茨迈耶和DNSOP的许多其他成员的审查。
Also, thanks to Miek Gieben for [mmark].
另外,感谢Miek Gieben的[mmark]。
Authors' Addresses
作者地址
John Dickinson Sinodun IT Magdalen Centre Oxford Science Park Oxford OX4 4GA United Kingdom Email: jad@sinodun.com
John Dickinson Sinodun IT Magdalen Centre牛津科技园牛津OX4 4GA英国电子邮件:jad@sinodun.com
Jim Hague Sinodun IT Magdalen Centre Oxford Science Park Oxford OX4 4GA United Kingdom Email: jim@sinodun.com
Jim Hague Sinodun IT Magdalen中心牛津科技园牛津OX4 4GA英国电子邮件:jim@sinodun.com
Sara Dickinson Sinodun IT Magdalen Centre Oxford Science Park Oxford OX4 4GA United Kingdom Email: sara@sinodun.com
Sara Dickinson Sinodun IT Magdalen中心牛津科技园牛津OX4 4GA英国电子邮件:sara@sinodun.com
Terry Manderson ICANN 12025 Waterfront Drive Suite 300 Los Angeles, CA 90094-2536 United States of America Email: terry.manderson@icann.org
Terry Manderson ICANN 12025 Waterfront Drive Suite 300加利福尼亚州洛杉矶90094-2536美利坚合众国电子邮件:Terry。manderson@icann.org
John Bond Wikimedia Foundation, Inc. 1 Montgomery Street Suite 1600 San Francisco, CA 94104 United States of America Email: ietf-wikimedia@johnbond.org
约翰邦德维基米达基金会,公司1蒙哥马利街套房1600旧金山,CA 94104美利坚合众国电子邮件:IETF-wikimedia@johnbond.org