This repo contains the daily reports generated by the DFZ Name and Shame ("DNAS") parser.
The daily reports generated by DNAS, which are published in this repo, contain only the worst data found within the DFZ. The reports are the Top Trumps of garbage being advertised into the global BGP DFZ. It is not all the unwanted/bad information advertised into the DFZ. There is a large amount of unwanted data in the DFZ, reporting all of it wouldn’t be very interesting or entertaining.
To produce a report of only the "worst" data in the DFZ, the statistics from the parsed MRT files are aggregated based on a property of the BGP UPDATE message. The aggregating property depends on the kind of statistic being reported. For example, not every bogon prefix found in the DFZ is listed in the daily report, only the bogon prefixes with the highest number of unique origin ASNs are listed. In this example, detecting a higher number of unique origin ASNs for a bogon prefix can be used as a proxy metric, to infer that the prefix was more "visible" across the DFZ than another bogon prefix with fewer origin ASNs.
The report currently exist in a plain-text human readable format only. Additional formats such as a JSON parseable format are coming soon.
The DNAS parser is running continuously, parsing MRT archives of BGP UPDATES from various sources. The same stats (which are explained below) are extracted from each MRT file from each source.
Note: Each MRT source is one perspective of the DFZ. It's important to note that there is no single DFZ, only different perspectives of the same phenomina.
Each day a report is generated for the previous day. The stats from each source are compared and the worst stats from each source are merged into a list of "the worst of the worst" across all sources. This means that in the report, the prefix with the most BOGON source ASNs may have been seen by $source1, and the prefix withe the highest number of updates may have been seen from $source2.
Only the worst stats are collected and shown in the report. The definition of "worst" varies by the type of statistic being reported. For example, a prefix originated by more origin ASNs is more "visible" in the DFZ than a prefix with fewer origin ASNs. Thus, a BOGON prefix announced by multiple origin ASNs is considered worse than a BOGON prefix announced by a single origin, because it is more like to propagate across the DFZ the more ASNs originate the prefix. This is because there is no single view of the DFZ, "pollution" seen in the DFZ might not have been seen by many networks or even any networks at all, if good filtering is in place.
As a further example, if the statistic is the highest MED seen, 20 prefixes may have been seen with a MED of 12M (out of a maximum of 16.7M), but even if only 1 prefix was seen with a MED of 15M (leaving just a small percentage of the total metric space still usable), the 1 prefix would be shown in the report, not the 20 prefixes (because the 1 prefix leaves even less usable metric space than the 20 for onward propagation without hitting the max MED).
All of this means that there is lots of DFZ "noise" not shown in the report. Showing it all would result in a huge report and some of it isnt very interesting or much of a problem. This is why only the worst data from the past 24 hours is reported. The report serves as a kind of warning of stuff that should probably be fixed.
Additionally, some statistics are cumulative across MRT sources while some are not, depending on whether it makes sense for each specific statistic (this is explained below).
The report is address family agnostic, statistics are independant of IP version.
All the unique BOGON origin ASNs seen for each prefix across all MRT sources.
The prefixes listed may have been seen from different MRT sources.
Prefixes with more BOGON origin ASNs are considered worse.
All the unique origin ASNs seen for each BOGON prefix across all MRT sources
The prefixes listed may have been seen from different MRT sources.
BOGON prefixes with more origin ASNs are considered worse.
When the origin ASN is a BOGON ASN, walk up the AS PATH until the first non-BOGON ASN is found. This is a list of those ASNs propagating routes with a BOGON origin ASN.
The ASNs listed may have been seen from different MRT sources.
ASNs originating more prefixes with a BOGON origin ASN are considered worse.
The list of prefixes seen with the same highest MED.
The prefixes listed may have been seen from different MRT sources.
A higher MED is considered worse.
A list of prefixes seen with the same highest AS Path length (they often all have a path length of 255 ASNs because this is the maximum BGPv4 supports).
The prefixes listed may have been seen from different MRT sources.
A longer AS Path is considered worse.
A list of prefixes all with the same highest number of communities attached. This includes standard and large communities. When the MRT source is operating at an IXP, any communities which match the IXP community prefix e.g.,65535:* are stripped before the communities on the UPDATE are counted. This is because the local IXP communities should not be forwarded and are expected on UPDATES at IXPs. This means that prefixes shown in these reports with hundreds of communities attached, which look like typical IXP communities might have come from another IXP the prefix has passed through and not been stripped.
The prefixes listed may have been seen from different MRT sources.
A higher number of communities is considered worse.
A list of prefixes whose length is < /8 or > /24 for IPv4, < /16 or > /56 for IPv6, with the most origin ASNs. An IPv4 /32 from 1 origin ASN is less visible than an IPv6 /64 that has 3 origin ASNs, in this case the /32 wouldn't be shown in the report.
The prefixes listed may have been seen from different MRT sources.
A higher number of origin ASNs is considered worse.
A list of prefixes which were included in the most BGP messages (UPDATEs and WITHDRAWs). The prefixes shown were included in the same (highest) number of BGP messages.
The prefixes listed and number of messages which contained those prefixes is the highest number seen from a single MRT source. This is because it's not possible to determine if a BGP message seen by two MRT sources is the same message or not (deduplication).
A higher number of messages which contain the same prefix is considered worse.
A list of prefixes which were included in the most BGP UPDATEs.
The prefixes listed and number of UPDATEs which contained those prefixes is the highest number seen from a single MRT source. This is because it's not possible to determine if a BGP UPDATE seen by two MRT sources is the same UPDATE or not (deduplication).
A higher number of UPDATEs which contain the same prefix is considered worse.
A list of prefixes which were included in the most BGP WITHDRAWs.
The prefixes listed and number of WITHDRAWs which contained those prefixes is the highest number seen from a single MRT source. This is because it's not possible to determine if a BGP WITHDRAW seen by two MRT sources is the same WITHDRAW or not (deduplication).
A higher number of WITHDRAWs which contain the same prefix is considered worse.
A list of origin ASNs which were included in the most BGP messages (UPDATEs and WITHDRAWs). The origin ASNs shown were included in the same (highest) number of BGP messages.
The origin ASNs listed and number of messages which contained those ASNs is the highest number seen from a single MRT source. This is because it's not possible to determine if a BGP message seen by two MRT sources is the same message or not (deduplication).
A higher number of messages which contain the same origin ASN is considered worse.
A list of peer ASNs (1st ASN in the path) which sent the most BGP messages (UPDATEs and WITHDRAWs) to the MRT collector. The peer ASNs shown sent the same (highest) number of BGP messages.
The peer ASNs listed and number of messages sent by those peer ASNs is the highest number seen from a single MRT source. This is because it's not possible to determine if a BGP message seen by two MRT sources is the same message or not (deduplication).
A higher number of messages which contain the same peer ASN is considered worse.
A list of peer ASNs which sent the most BGP UPDATEs to the MRT collector.
The peer ASNs listed and number of UPDATEs which came from those peers is the highest number seen from a single MRT source. This is because it's not possible to determine if a BGP UPDATE seen by two MRT sources is the same UPDATE or not (deduplication).
A higher number of UPDATEs which came from the same peer is considered worse.
A list of peer ASNs which sent the most BGP WITHDRAWs to the MRT collector.
The peer ASNs listed and number of WITHDRAWs which came from those peers is the highest number seen from a single MRT source. This is because it's not possible to determine if a BGP WITHDRAW seen by two MRT sources is the same WITHDRAW or not (deduplication).
A higher number of WITHDRAWs which came from the same peer is considered worse.
A list of prefixes which have the same highest number of origin ASNs.
The prefixes listed may have been seen from different MRT sources.
Prefixes with more origin ASNs are considered worse.
A list of prefixes which have the same highest number of unknown BGP attributes attached.
The prefixes listed may have been seen from different MRT sources.
Prefixes with more unknown attributes are considered worse.
If the ASN you operate or prefixes you originate are in the report and you need help, or want to understand what this might mean, you can contact James at the following address: (jwbensley) [@] (gmail) dot com.