Removal of bogon route objects
Friends, As previously discussed, the decision was made awhile back to remove from the data base all RIPE-NONAUTH route objects which refer to currently unallocated IP space: https://www.ripe.net/ripe/mail/archives/db-wg/2021-May/006952.html In recent days I stumbled upon one such object that was still present in the data base. Upon seeing that object, I necessarily assumed that it was most likely not a "one off", and that it probably had siblings. In the spirit of trying to help out, and after a fair bit of fooling around on my part, I managed to concoct a rather simple method for finding all such bogon route objects that remain in the data base, and I wrote a small Perl script to do that exact thing. A copy of the script may be obtained here, for anyone who may be interested: https://pastebin.com/raw/LsvZ05tX Notes: *) The script assumes the presence of two external programs and their presence on the user's current $PATH, i.e. the widely used wget progarm, and also John Levine's version of a thing called grepcidr, sources for which may be obtained here: https://github.com/jrlevine/grepcidr3 *) The initial line of the script may have to be adjusted depending on the location, in the filesystem, of your local Perl interpreter. *) The script computes its results based upon the following two input data files, both of which are automagically fetched by the script using wget: https://www.nro.net/wp-content/uploads/delegated-stats/nro-extended-stats ftp://ftp.ripe.net/ripe/dbase/ripe-nonauth.db.gz (It is my understanding that each of these is updated on a daily basis.) *) The script DOES NOT verify that the entirety of any given RIPE-NONAUTH route object in fact refers to bogon space. Rather the script only checks to see whether or not the first IPv4 address of each current RIPE-NONAUTH route object does or does not refer to current bogon space. If it does, then the entire IPv4 CIDR of the route object is printed to stdout. *) Assuming that the script is installed as "ripebogonroutes" and in one's path, it may be invoked simply as: ripebogonroutes > bogon-route-cidrs *) At present there are 55,964 RIPE-NONAUTH objects in the data base. Of these, the first address of 857 of them refers to current bogon space. It is my fervent hope that the corresponding RIPE-NONAUTH route objects will be removed from the data base as soon as reasonably practical. Regards, rfg
Hi Ronald, Could I request that you provide a pastebin of the actual list of prefixes that you think are at issue here? -Cynthia On Thu, Jun 10, 2021, 21:43 Ronald F. Guilmette via db-wg <db-wg@ripe.net> wrote:
Friends,
As previously discussed, the decision was made awhile back to remove from the data base all RIPE-NONAUTH route objects which refer to currently unallocated IP space:
https://www.ripe.net/ripe/mail/archives/db-wg/2021-May/006952.html
In recent days I stumbled upon one such object that was still present in the data base. Upon seeing that object, I necessarily assumed that it was most likely not a "one off", and that it probably had siblings.
In the spirit of trying to help out, and after a fair bit of fooling around on my part, I managed to concoct a rather simple method for finding all such bogon route objects that remain in the data base, and I wrote a small Perl script to do that exact thing. A copy of the script may be obtained here, for anyone who may be interested:
https://pastebin.com/raw/LsvZ05tX
Notes:
*) The script assumes the presence of two external programs and their presence on the user's current $PATH, i.e. the widely used wget progarm, and also John Levine's version of a thing called grepcidr, sources for which may be obtained here:
https://github.com/jrlevine/grepcidr3
*) The initial line of the script may have to be adjusted depending on the location, in the filesystem, of your local Perl interpreter.
*) The script computes its results based upon the following two input data files, both of which are automagically fetched by the script using wget:
https://www.nro.net/wp-content/uploads/delegated-stats/nro-extended-stats ftp://ftp.ripe.net/ripe/dbase/ripe-nonauth.db.gz
(It is my understanding that each of these is updated on a daily basis.)
*) The script DOES NOT verify that the entirety of any given RIPE-NONAUTH route object in fact refers to bogon space. Rather the script only checks to see whether or not the first IPv4 address of each current RIPE-NONAUTH route object does or does not refer to current bogon space. If it does, then the entire IPv4 CIDR of the route object is printed to stdout.
*) Assuming that the script is installed as "ripebogonroutes" and in one's path, it may be invoked simply as:
ripebogonroutes > bogon-route-cidrs
*) At present there are 55,964 RIPE-NONAUTH objects in the data base. Of these, the first address of 857 of them refers to current bogon space. It is my fervent hope that the corresponding RIPE-NONAUTH route objects will be removed from the data base as soon as reasonably practical.
Regards, rfg
In message <CAKw1M3PuvtmemW9pCqdSXwKPQjt99A-tEstNRXoCc-vaVjBZLA@mail.gmail.com> =?UTF-8?Q?Cynthia_Revstr=C3=B6m?= <me@cynthia.re> wrote:
Could I request that you provide a pastebin of the actual list of prefixes that you think are at issue here?
Your wish is my command. Sorted by IP address and NOT de-duped (857): https://pastebin.com/raw/yEWTq3pP Sorted by IP address and de-duped (700): https://pastebin.com/raw/CpUydPTH Regards, rfg
Hello Ronald,
On 10 Jun 2021, at 22:54, Ronald F. Guilmette via db-wg <db-wg@ripe.net> wrote:
In message <CAKw1M3PuvtmemW9pCqdSXwKPQjt99A-tEstNRXoCc-vaVjBZLA@mail.gmail.com> =?UTF-8?Q?Cynthia_Revstr=C3=B6m?= <me@cynthia.re> wrote:
Could I request that you provide a pastebin of the actual list of prefixes that you think are at issue here?
Your wish is my command.
Sorted by IP address and NOT de-duped (857): https://pastebin.com/raw/yEWTq3pP
Sorted by IP address and de-duped (700): https://pastebin.com/raw/CpUydPTH
Regards, rfg
The DB team is implementing the NONAUTH cleanup job according to the proposal: https://www.ripe.net/ripe/mail/archives/db-wg/2021-March/006876.html <https://www.ripe.net/ripe/mail/archives/db-wg/2021-March/006876.html> I ran a script using those rules (against the latest split files and delegated stats from each RIR) and found the following unregistered prefixes: https://pastebin.com/aeyhUKeH <https://pastebin.com/aeyhUKeH> My list appears to agree with your list (I didn't de-duplicate prefixes). Also for unregistered IPv6 prefixes referenced by NONAUTH route6 objects: https://pastebin.com/unK958cE <https://pastebin.com/unK958cE> In total I found 825 route objects and 57 route6 objects with unregistered prefixes, and which are now eligible for deletion. We will announce to the DB-WG when our implementation is complete. Regards Ed Shryane RIPE NCC
In message <032287F0-AD5A-48B9-AFEB-079D70F76726@ripe.net>, Edward Shryane <eshryane@ripe.net> wrote:
The DB team is implementing the NONAUTH cleanup job according to the proposal: https://www.ripe.net/ripe/mail/archives/db-wg/2021-March/006876.html <https://www.ripe.net/ripe/mail/archives/db-wg/2021-March/006876.html>
I see. That message is very interesting. There are a couple of parts of it that I have trouble interpreting however, to wit: There are approximately 64 routes and 37 route6's with a prefix not listed in any RIR's delegated stats. These will not be affected. Can you explain the meaning of that, in small words? Also, there's this: Only route(6) prefixes that are "allocated" or "assigned" in any RIRs delegated stats should remain... I'm just looking at the nro-extended-stats file. In that one, the only things marked as "allocated" are all just ASNs and they are all just "allocated" to IANA. So I'm not clear why you said ""allocated" or "assigned". The former set seems to be effectively a null set. Last but not least, there is this very perplexing line: The origin AS status is not considered, only the IPv4/IPv6 prefix. Why? Maybe it's just me, but I don't think that RIPE should be effectively condoning the use of unregistered bogon AS numbers. (And by the way, I just did some work on that. There are 82 route objects in the "normal" (non-RIPE-NONAUTH) data base at present that refer to bogon ASNs. As long as there is a cleanup occurring, why not clean these up also? (See list at end.)
I ran a script using those rules (against the latest split files and delegated stats from each RIR) and found the following unregistered prefixes: https://pastebin.com/aeyhUKeH <https://pastebin.com/aeyhUKeH>
Well, so I guess I need to get all those separate per-RIR delegated stats files, because I guess what the NRO has isn't actually accurate (?)
My list appears to agree with your list (I didn't de-duplicate prefixes).
Ummm... no. As I had noted, I found 857 (non-deduped) bogon routes using my methodology. Your list appears to contain only 820. After de-duping, the differences seem to come down to just these CIDRs: 41.217.144.0/21 192.31.196.0/24 192.88.99.0/24 192.175.48.0/24 That last one appears to be an error on my part... and I will be trying to figure out how & why that one got into my set. The other three however seems to be ones that you missed. Hopefully our two lists will convergeat some point.
Also for unregistered IPv6 prefixes referenced by NONAUTH route6 objects: https://pastebin.com/unK958cE <https://pastebin.com/unK958cE>
I'm not even looking at any IPv6 stuff myself.
We will announce to the DB-WG when our implementation is complete.
Any ETA for that? Regards, rfg P.S. Route objects in the regular data base that refer to bogon ASNs: 194.165.87.0/24 AS21389 83.216.160.0/19 AS31419 213.228.244.0/24 AS31453 86.110.128.0/19 AS31419 83.136.64.0/21 AS8871 83.216.160.0/20 AS31419 83.216.176.0/20 AS31419 86.110.128.0/20 AS31419 86.110.144.0/20 AS31419 194.99.204.0/22 AS36895 83.216.160.0/21 AS31419 83.216.168.0/21 AS31419 83.216.176.0/21 AS31419 83.216.184.0/21 AS31419 91.102.168.0/21 AS8295 91.102.168.0/24 AS8295 91.102.169.0/24 AS8295 91.102.170.0/24 AS8295 91.102.171.0/24 AS8295 91.102.172.0/24 AS8295 91.102.173.0/24 AS8295 91.102.174.0/24 AS8295 91.102.175.0/24 AS8295 193.33.110.0/23 AS42796 193.201.204.0/24 AS14880 193.201.205.0/24 AS14880 86.110.128.0/21 AS31419 86.110.136.0/21 AS31419 86.110.144.0/21 AS31419 86.110.152.0/21 AS31419 91.220.83.0/24 AS46414 5.1.112.0/21 AS198738 5.1.112.0/24 AS198738 5.1.113.0/24 AS198738 81.52.165.0/24 AS36879 81.52.166.0/24 AS36879 81.52.167.0/24 AS36879 81.52.168.0/24 AS36879 81.52.169.0/24 AS36879 185.48.220.0/22 AS199724 194.201.253.0/24 AS25568 5.160.196.0/24 AS62280 5.160.197.0/24 AS62280 5.160.240.0/24 AS62280 5.160.241.0/24 AS62280 77.246.74.0/24 AS37631 185.106.44.0/24 AS200968 158.255.59.0/24 AS203655 185.148.138.0/23 AS198738 82.98.78.0/24 AS394935 185.164.160.0/22 AS14076 185.200.32.0/22 AS33431 185.225.136.0/22 AS33431 185.244.144.0/22 AS204634 185.194.4.0/22 AS205740 93.92.104.0/22 AS1975505 185.62.220.0/22 AS1975505 185.62.221.0/24 AS1975505 217.68.70.0/24 AS214359 45.14.212.0/22 AS2209013 91.107.84.0/24 AS202532 185.148.136.0/22 AS198738 185.250.112.0/22 AS20180320 147.28.44.0/24 AS558363 147.28.45.0/24 AS558363 147.28.46.0/24 AS558363 147.28.47.0/24 AS558363 45.8.92.0/24 AS46723 45.8.95.0/24 AS46723 45.8.94.0/24 AS46723 45.8.93.0/24 AS46723 45.8.92.0/24 AS397796 45.8.95.0/24 AS397796 45.8.94.0/24 AS397796 141.98.12.0/24 AS208469 188.92.121.0/24 AS40178 193.135.119.0/24 AS46723 45.95.0.0/24 AS46723 45.95.1.0/24 AS46723 45.95.2.0/24 AS46723 45.95.3.0/24 AS46723 193.110.94.0/24 AS16712
Hi Ronald,
On 11 Jun 2021, at 10:19, Ronald F. Guilmette via db-wg <db-wg@ripe.net> wrote:
In message <032287F0-AD5A-48B9-AFEB-079D70F76726@ripe.net>, Edward Shryane <eshryane@ripe.net> wrote:
The DB team is implementing the NONAUTH cleanup job according to the proposal: https://www.ripe.net/ripe/mail/archives/db-wg/2021-March/006876.html <https://www.ripe.net/ripe/mail/archives/db-wg/2021-March/006876.html>
I see. That message is very interesting. There are a couple of parts of it that I have trouble interpreting however, to wit:
There are approximately 64 routes and 37 route6's with a prefix not listed in any RIR's delegated stats. These will not be affected.
Can you explain the meaning of that, in small words?
The cleanup job will use all of the RIRs delegated stats. If the route prefix is not defined in any delegated stats, then it's skipped. The count above didn't consider prefixes split between different RIRs, but now all stats are combined. I now found 28 routes and 37 route6 objects with a prefix not listed in any RIRs delegated stats (which I downloaded today). These prefixes were not listed in any delegated stats: 192.31.196.0/24 192.88.99.0/24 2001::/32 2001:4:112::/48 2002::/16 2011:4188::/48
Also, there's this:
Only route(6) prefixes that are "allocated" or "assigned" in any RIRs delegated stats should remain...
I'm just looking at the nro-extended-stats file. In that one, the only things marked as "allocated" are all just ASNs and they are all just "allocated" to IANA. So I'm not clear why you said ""allocated" or "assigned". The former set seems to be effectively a null set.
I'm not using the NRO extended stats, but rather I'm combining each RIRs delegated stats. Each RIRs delegated stats are more specific about the prefix status, including "assigned".
Last but not least, there is this very perplexing line:
The origin AS status is not considered, only the IPv4/IPv6 prefix.
Why?
The origin AS is not validated either in the RIPE database when creating or updating a route(6) object (apart from excluding AS numbers reserved by IANA). Also, this cleanup job was created out of discussion about deregistered address space: "196.52.0.0/14 revoked, cleanup efforts needed" in January.
Maybe it's just me, but I don't think that RIPE should be effectively condoning the use of unregistered bogon AS numbers. (And by the way, I just did some work on that. There are 82 route objects in the "normal" (non-RIPE-NONAUTH) data base at present that refer to bogon ASNs.
As long as there is a cleanup occurring, why not clean these up also? (See list at end.)
If the DB-WG agrees that we should, and can define *which* AS numbers should be cleaned up, the job can be extended to include AS numbers. If so, should *both* the prefix and origin be considered, or separately (i.e. should we cleanup if either is unregistered)?
I ran a script using those rules (against the latest split files and delegated stats from each RIR) and found the following unregistered prefixes: https://pastebin.com/aeyhUKeH <https://pastebin.com/aeyhUKeH>
Well, so I guess I need to get all those separate per-RIR delegated stats files, because I guess what the NRO has isn't actually accurate (?)
I used the per-RIR delegated stats as I'm more familiar with them (they are already used in Whois for the mirror databases).
My list appears to agree with your list (I didn't de-duplicate prefixes).
Ummm... no. As I had noted, I found 857 (non-deduped) bogon routes using my methodology. Your list appears to contain only 820.
After de-duping, the differences seem to come down to just these CIDRs:
41.217.144.0/21
From the AFRINIC delegated stats, 41.217.144.0/21 is composed of: afrinic|ZZ|ipv4|41.217.144.0|1024||reserved| afrinic|CM|ipv4|41.217.148.0|1024|20090529|allocated|F367BA6A From the cleanup proposal: If a prefix is partially "available" or "reserved" then it is also considered to be unregistered. So this prefix *is* included.
192.31.196.0/24
192.31.196.0/24 is not listed in any RIRs delegated stats, according to the NRO's delegated stats it's reserved for the IETF. I see this prefix is listed as an anycast address for DNAME: https://datatracker.ietf.org/doc/html/rfc7534 https://www.as112.net/well-known.html So I believe this prefix is eligible for cleanup, but it's not listed in the RIR delegated stats so it will be skipped.
192.88.99.0/24
This prefix is also listed in the NRO delegated stats as reserved for the IETF, and not in the RIR delegated stats.
192.175.48.0/24
That last one appears to be an error on my part... and I will be trying to figure out how & why that one got into my set.
This prefix is listed in ARIN's delegated stats arin|US|ipv4|192.175.48.0|256|19960105|assigned|9064d089391b6dcc310a2188da60c437 But in the NRO delegated stats it's listed as IETF reserved: iana|ZZ|ipv4|192.175.48.0|256|20150501|reserved|ietf|iana I don't know why.
The other three however seems to be ones that you missed.
The first can be explained by the "reserved" status, and the other two don't appear in the RIR delegated stats.
Hopefully our two lists will convergeat some point.
I think there will always be some differences between the NRO delegated stats and the RIR delegated stats (e.g. the IETF reserved prefixes).
Also for unregistered IPv6 prefixes referenced by NONAUTH route6 objects: https://pastebin.com/unK958cE <https://pastebin.com/unK958cE>
I'm not even looking at any IPv6 stuff myself.
IPv6 will be included in the cleanup job (about 94 route6 objects will be affected), the methodology should be the same as for IPv4.
We will announce to the DB-WG when our implementation is complete.
Any ETA for that?
The DB team are currently working on it, I don't have an ETA but I expect it will be ready in the coming weeks. Also thanks to you for checking NONAUTH for bogon route objects, your questions and comments are very welcome. Regards Ed Shryane RIPE NCC
Regards, rfg
P.S. Route objects in the regular data base that refer to bogon ASNs:
194.165.87.0/24 AS21389 83.216.160.0/19 AS31419 213.228.244.0/24 AS31453 86.110.128.0/19 AS31419 83.136.64.0/21 AS8871 83.216.160.0/20 AS31419 83.216.176.0/20 AS31419 86.110.128.0/20 AS31419 86.110.144.0/20 AS31419 194.99.204.0/22 AS36895 83.216.160.0/21 AS31419 83.216.168.0/21 AS31419 83.216.176.0/21 AS31419 83.216.184.0/21 AS31419 91.102.168.0/21 AS8295 91.102.168.0/24 AS8295 91.102.169.0/24 AS8295 91.102.170.0/24 AS8295 91.102.171.0/24 AS8295 91.102.172.0/24 AS8295 91.102.173.0/24 AS8295 91.102.174.0/24 AS8295 91.102.175.0/24 AS8295 193.33.110.0/23 AS42796 193.201.204.0/24 AS14880 193.201.205.0/24 AS14880 86.110.128.0/21 AS31419 86.110.136.0/21 AS31419 86.110.144.0/21 AS31419 86.110.152.0/21 AS31419 91.220.83.0/24 AS46414 5.1.112.0/21 AS198738 5.1.112.0/24 AS198738 5.1.113.0/24 AS198738 81.52.165.0/24 AS36879 81.52.166.0/24 AS36879 81.52.167.0/24 AS36879 81.52.168.0/24 AS36879 81.52.169.0/24 AS36879 185.48.220.0/22 AS199724 194.201.253.0/24 AS25568 5.160.196.0/24 AS62280 5.160.197.0/24 AS62280 5.160.240.0/24 AS62280 5.160.241.0/24 AS62280 77.246.74.0/24 AS37631 185.106.44.0/24 AS200968 158.255.59.0/24 AS203655 185.148.138.0/23 AS198738 82.98.78.0/24 AS394935 185.164.160.0/22 AS14076 185.200.32.0/22 AS33431 185.225.136.0/22 AS33431 185.244.144.0/22 AS204634 185.194.4.0/22 AS205740 93.92.104.0/22 AS1975505 185.62.220.0/22 AS1975505 185.62.221.0/24 AS1975505 217.68.70.0/24 AS214359 45.14.212.0/22 AS2209013 91.107.84.0/24 AS202532 185.148.136.0/22 AS198738 185.250.112.0/22 AS20180320 147.28.44.0/24 AS558363 147.28.45.0/24 AS558363 147.28.46.0/24 AS558363 147.28.47.0/24 AS558363 45.8.92.0/24 AS46723 45.8.95.0/24 AS46723 45.8.94.0/24 AS46723 45.8.93.0/24 AS46723 45.8.92.0/24 AS397796 45.8.95.0/24 AS397796 45.8.94.0/24 AS397796 141.98.12.0/24 AS208469 188.92.121.0/24 AS40178 193.135.119.0/24 AS46723 45.95.0.0/24 AS46723 45.95.1.0/24 AS46723 45.95.2.0/24 AS46723 45.95.3.0/24 AS46723 193.110.94.0/24 AS16712
Edward Shryane via db-wg wrote on 11/06/2021 13:44:
192.175.48.0/24
That last one appears to be an error on my part... and I will be trying to figure out how & why that one got into my set.
This prefix is listed in ARIN's delegated stats
arin|US|ipv4|192.175.48.0|256|19960105|assigned|9064d089391b6dcc310a2188da60c437
But in the NRO delegated stats it's listed as IETF reserved:
iana|ZZ|ipv4|192.175.48.0|256|20150501|reserved|ietf|iana
I don't know why.
https://www.iana.org/assignments/iana-ipv4-special-registry ARIN handles special purpose address registrations. Nick
In message <4282F6EF-99C9-4306-817B-95382487CBBA@ripe.net>, Edward Shryane <eshryane@ripe.net> wrote:
I see. That message is very interesting. There are a couple of parts of it that I have trouble interpreting however, to wit:
There are approximately 64 routes and 37 route6's with a prefix not listed in any RIR's delegated stats. These will not be affected.
Can you explain the meaning of that, in small words?
The cleanup job will use all of the RIRs delegated stats. If the route prefix is not defined in any delegated stats, then it's skipped.
Umm... I thought that it was the definition of a "bogon" that it is some IP space (or ASN) that has not been assigned, by any RIR, to any other party. And isn't the project here to detect and eliminate bogon route objects? We seem to have a communications/terminology gap at some level.
I'm not using the NRO extended stats, but rather I'm combining each RIRs delegated stats.
Each RIRs delegated stats are more specific about the prefix status, including "assigned".
Could you please post the 5 URLs for those 5 RIR delegated stats files? I'm sure that I could find them on my own, if pressed, but it would save me a small amount of time if you would just post them. (It is a curious oddity that the NRO stats file is apparently not quite a perfect representation of the sum total of those 5 files.)
Last but not least, there is this very perplexing line:
The origin AS status is not considered, only the IPv4/IPv6 prefix.
Why?
The origin AS is not validated either in the RIPE database when creating or updating a route(6) object (apart from excluding AS numbers reserved by IANA).
Also, this cleanup job was created out of discussion about deregistered address space: "196.52.0.0/14 revoked, cleanup efforts needed" in January.
OK, but as long as a cleanup is happening anyway, I would like to propose that all (stale?) route objects which reference bogon ASN should be elided also. It seems quite entirely half fast for you/NCC to be doing all of this work to eliminate JUST the bogus route objects that reference bogon IP address space, while leaving alone all of the route objects that refer to bogon ASNs. I mean, is it just me, or isn't that obvious?
If the DB-WG agrees that we should, and can define *which* AS numbers should be cleaned up, the job can be extended to include AS numbers.
I have stated my own preference. I certainly would be happy to hear the opinions of others in the WG.
If so, should *both* the prefix and origin be considered, or separately (i.e. should we cleanup if either is unregistered)?
I'm not sure that I am following you here. I mean I'm not even sure your question makes any sense to me. I will just reiterate what I believe I have already expressed, i.e. that in my opinion a given route object is, by definition, "bogus" (and should be removed from the DB) if *either* it references bogon IP space *or* it references an unassigned (bogon) ASN.
Also thanks to you for checking NONAUTH for bogon route objects, your questions and comments are very welcome.
Just trying to be helpful, if I can be. I should be thanking *you* for your work on and involvement in this rather elaborate cleanup process. It's a tough and probably thankless job, but somebody's got to do it. I wish that I could wake up tomorrow and find that the whole world had totally and universally embraced RPKI, but until then, the route registries of the 5 RIRs are still being relied upon by a lot of folks, and it just won't do to have these route registries oozing bogosity. Regards, rfg
Ronald F. Guilmette via db-wg wrote on 11/06/2021 22:19:
Could you please post the 5 URLs for those 5 RIR delegated stats files?
There's a convention. Each RIR has a short name: afrinic apnic arin lacnic ripencc and a stats ftp directory: ftp://ftp.afrinic.net ftp://ftp.apnic.net/public/apnic ftp://ftp.arin.net/pub ftp://ftp.lacnic.net/pub ftp://ftp.ripe.net/pub Inside each of these stats directories, the delegated files are in: $ftpdir/stats/$shortname/delegated-$shortname-extended-latest The transfer files are in: $ftpdir/stats/$shortname/transfers/transfers_latest.json e.g. for APNIC, the extended stats file is in:
ftp://ftp.apnic.net/public/apnic/stats/apnic/delegated-apnic-extended-latest
for RIPE NCC, the transfer file is:
ftp://ftp.ripe.net/pub/stats/ripencc/transfers/transfers_latest.json
Nick
Hi Ronald,
On 11 Jun 2021, at 23:19, Ronald F. Guilmette via db-wg <db-wg@ripe.net> wrote:
In message <4282F6EF-99C9-4306-817B-95382487CBBA@ripe.net>, Edward Shryane <eshryane@ripe.net> wrote:
I see. That message is very interesting. There are a couple of parts of it that I have trouble interpreting however, to wit:
There are approximately 64 routes and 37 route6's with a prefix not listed in any RIR's delegated stats. These will not be affected.
Can you explain the meaning of that, in small words?
The cleanup job will use all of the RIRs delegated stats. If the route prefix is not defined in any delegated stats, then it's skipped.
Umm... I thought that it was the definition of a "bogon" that it is some IP space (or ASN) that has not been assigned, by any RIR, to any other party.
This particular cleanup job is deleting route(6) objects referencing *unregistered* space ("available" or "reserved" in delegated stats), rather than any "bogons". The aim is to cleanup route(6) objects after an RIR deregisters space (e.g. 196.52.0.0/14, as discussed in January).
And isn't the project here to detect and eliminate bogon route objects?
We seem to have a communications/terminology gap at some level.
Not for now. I suggest we define "bogon" space and cleanup any referencing route(6) objects as a separate effort. Note that Whois already blocks the creation of new route(6) objects with a "bogon" prefix, as defined here: http://www.team-cymru.com/bogon-reference.html
I'm not using the NRO extended stats, but rather I'm combining each RIRs delegated stats.
Each RIRs delegated stats are more specific about the prefix status, including "assigned".
Could you please post the 5 URLs for those 5 RIR delegated stats files? I'm sure that I could find them on my own, if pressed, but it would save me a small amount of time if you would just post them.
Thanks Nick for explaining the location of the RIR delgated stats files.
(It is a curious oddity that the NRO stats file is apparently not quite a perfect representation of the sum total of those 5 files.)
I'm using the RIR delegated stats as they are already in use by Whois, I haven't checked for differences with the NRO delegated stats.
Last but not least, there is this very perplexing line:
The origin AS status is not considered, only the IPv4/IPv6 prefix.
Why?
The origin AS is not validated either in the RIPE database when creating or updating a route(6) object (apart from excluding AS numbers reserved by IANA).
Also, this cleanup job was created out of discussion about deregistered address space: "196.52.0.0/14 revoked, cleanup efforts needed" in January.
OK, but as long as a cleanup is happening anyway, I would like to propose that all (stale?) route objects which reference bogon ASN should be elided also.
It seems quite entirely half fast for you/NCC to be doing all of this work to eliminate JUST the bogus route objects that reference bogon IP address space, while leaving alone all of the route objects that refer to bogon ASNs.
I mean, is it just me, or isn't that obvious?
We can extend the cleanup job to also validate the origin AS, if the rules are clear and if the DB-WG agrees.
If the DB-WG agrees that we should, and can define *which* AS numbers should be cleaned up, the job can be extended to include AS numbers.
I have stated my own preference. I certainly would be happy to hear the opinions of others in the WG.
If so, should *both* the prefix and origin be considered, or separately (i.e. should we cleanup if either is unregistered)?
I'm not sure that I am following you here. I mean I'm not even sure your question makes any sense to me. I will just reiterate what I believe I have already expressed, i.e. that in my opinion a given route object is, by definition, "bogus" (and should be removed from the DB) if *either* it references bogon IP space *or* it references an unassigned (bogon) ASN.
Thanks, that was my question, should we validate that *either* the IP or AS is unregistered, or *both* IP and AS are unregistered (and you indicated *either*). Regards Ed Shryane RIPE NCC
Also thanks to you for checking NONAUTH for bogon route objects, your questions and comments are very welcome.
Just trying to be helpful, if I can be.
I should be thanking *you* for your work on and involvement in this rather elaborate cleanup process.
It's a tough and probably thankless job, but somebody's got to do it.
I wish that I could wake up tomorrow and find that the whole world had totally and universally embraced RPKI, but until then, the route registries of the 5 RIRs are still being relied upon by a lot of folks, and it just won't do to have these route registries oozing bogosity.
Regards, rfg
In message <46AAC114-7B46-4D17-A024-56E82872797C@ripe.net>, Edward Shryane <eshryane@ripe.net> wrote:
(It is a curious oddity that the NRO stats file is apparently not quite a perfect representation of the sum total of those 5 files.)
I'm using the RIR delegated stats as they are already in use by Whois...
As of now, I am also. (The NRO status file contains *many* divergences from the sum of the 5 RIR status files. IT is apparent that some of these, at least, are caused by "version skew", i.e. the NRO stats file is apparently not as fresh as what can be gotten from the 5 RIRs directly.)
It seems quite entirely half fast for you/NCC to be doing all of this work to eliminate JUST the bogus route objects that reference bogon IP address space, while leaving alone all of the route objects that refer to bogon ASNs.
I mean, is it just me, or isn't that obvious?
We can extend the cleanup job to also validate the origin AS, if the rules are clear and if the DB-WG agrees.
Wouldn't it make your task easier if you were to tackle both of these sets of bogus route objects at the same time, rather than processing each set sequentially? I'm just saying that I think it would be Good to move forward with deleting both sets of bogus route objects ASAP. (I mean it's all garbage anyway, and I really doubt that anyone is going to miss any of this gunk.)
Thanks, that was my question, should we validate that *either* the IP or AS is unregistered, or *both* IP and AS are unregistered (and you indicated *either*).
I have made my own personal preference clear. What do others on the list think? Regards, rfg
Edward Shryane via db-wg wrote on 14/06/2021 15:16:
Thanks, that was my question, should we validate that*either* the IP or AS is unregistered, or*both* IP and AS are unregistered (and you indicated*either*). definitely if the prefix is unregistered.
If there's a nonauth route object registered with a valid address range, but invalid AS, is it currently possible to update the route object to reference a correctly registered ASN? If the answer is that it's not possible (i.e. because the db primary key is a concatenation of the prefix + as), then the route object should be deleted because it's basically unfixable. If there are no immediate plans to investigate recently reregistered space, would it be worth running the invalid prefix analysis for all of the extended delegation files every month since 2018-09-04? The unallocated / reserved address pools change, and it should be easy enough to script this up. Nick
Hi, On Fri, Jun 11, 2021 at 02:44:37PM +0200, Edward Shryane via db-wg wrote:
192.88.99.0/24 2002::/16
These two are 6to4 anycast relays (v4 and v6 side). While that is deprecated, it's still "special".
2001::/32
This is Teredo.
2001:4:112::/48
This one seems to be intentional route6: 2001:4:112::/48 descr: RFC7535 AS112 DNAME prefix descr: See: http://www.as112.net/ and is indeed visible and 2001:4:112::1 accepts DNS queries.
2011:4188::/48
This seems to be a typo... to my knowledge, nothing was ever allocated out of 2011: Gert Doering -- NetMaster -- have you enabled IPv6 on something today...? SpaceNet AG Vorstand: Sebastian v. Bomhard, Michael Emmer Joseph-Dollinger-Bogen 14 Aufsichtsratsvors.: A. Grundner-Culemann D-80807 Muenchen HRB: 136055 (AG Muenchen) Tel: +49 (0)89/32356-444 USt-IdNr.: DE813185279
With regards to the 192.175.48.0/24 block, the route for which I had incorrectly listed as a bogon, it seems that someone @ NRO somehow managed to muck this one up. Here is the relevant line from the latest nro-extended-stats file: iana|ZZ|ipv4|192.175.48.0|256|20150501|reserved|ietf|iana In reality the block isn't reserved and it doesn't belong to IANA. It's a regular old (assigned) ARIN block. I guess that I have to start distrusting the nro-extended-stats since it apparently contains some bogus information. Regards, rfg
Hi Ronald, On 11/06/2021 11:24, Ronald F. Guilmette via db-wg wrote:
iana|ZZ|ipv4|192.175.48.0|256|20150501|reserved|ietf|iana
In reality the block isn't reserved and it doesn't belong to IANA. It's a regular old (assigned) ARIN block.
I think it once *was* regular. and then: https://datatracker.ietf.org/doc/html/rfc7534#section-7.2.3 I think AS112 has a little bit a "special" history. Route Objects for 192.175.48.0 and AS112 are a good thing, I hope not only in ARIN IRR. Frank
In message <28bdc761-e318-2402-c90f-1881c1d0310b@geier.ne.tz>, Frank Habicht <geier@geier.ne.tz> wrote:
Hi Ronald,
On 11/06/2021 11:24, Ronald F. Guilmette via db-wg wrote:
iana|ZZ|ipv4|192.175.48.0|256|20150501|reserved|ietf|iana
In reality the block isn't reserved and it doesn't belong to IANA. It's a regular old (assigned) ARIN block.
I think it once *was* regular. and then: https://datatracker.ietf.org/doc/html/rfc7534#section-7.2.3
I think AS112 has a little bit a "special" history. Route Objects for 192.175.48.0 and AS112 are a good thing, I hope not only in ARIN IRR.
I made no judgement on either the block or the relevant ASN. The block has an ARIN WHOIS record. It should be listed in the NRO file in a manner consistant with that. Code hates exceptions, and so do I. Regards, rfg
Edward Shryane via db-wg wrote on 11/06/2021 08:35:
In total I found 825 route objects and 57 route6 objects with unregistered prefixes, and which are now eligible for deletion.
Taking both non-merger transfers and unregistered / reserved prefixes into account, I figure 1555 ipv4 prefixes and 59 ipv6 route objects: ipv4:
https://pastebin.ibn.ie/?b14b8aea397ddf06#4Z7GfmAm1EevbzvkkvzfFAN6hkwubRiaKm...
ipv6:
https://pastebin.ibn.ie/?fede4d799e615f4f#DiwQK7EGnkCDRyLf5TUUjURReGGTZNyWzb...
This doesn't include transfers / reregistrations which happened before 2018-09-04. This would add to the total. Also, some care needs to be taken with transfers from the lacnic / apnic regions because they don't appear to differentiate between RESOURCE_TRANSFER and MERGER_ACQUISITION transfer types (or maybe they do but they don't have any, which would be odd). This would take away some entries. The code is available here: https://github.com/nickhilliard/irrslurp Seems to run fine on macos and ubuntu. Nick
In message <ca608a29-078d-e4be-33bf-15432f37964e@foobar.org>, Nick Hilliard <nick@foobar.org> wrote:
Edward Shryane via db-wg wrote on 11/06/2021 08:35:
In total I found 825 route objects and 57 route6 objects with unregistered prefixes, and which are now eligible for deletion.
Taking both non-merger transfers and unregistered / reserved prefixes into account, I figure 1555 ipv4 prefixes and 59 ipv6 route objects:
Now I'm confused. My understanding was that the plan was -just- to remove route object that referenced bogon space. Assuming that that is the current plan, then Ed's results and mine are in reasonably close agreement with regards to the number of IPv4 route objects that need to be purged, i.e. some 800+. Your figure is nearly double that, so obviously, you are fishing out a rather different set of IPv4 route objects than Ed and I are. Not saying you're wrong. Just saying that now I'm a bit confused about the actual scope of this cleanup project. Regards, rfg
Ronald F. Guilmette via db-wg wrote on 13/06/2021 23:30:
Assuming that that is the current plan, then Ed's results and mine are in reasonably close agreement with regards to the number of IPv4 route objects that need to be purged, i.e. some 800+.
Your figure is nearly double that, so obviously, you are fishing out a rather different set of IPv4 route objects than Ed and I are.
yep that's correct - the previous figures included internal RIR resource re-registrations and resource transfers, which will need to be dealt with at some point in future. You can get comparable output for the current discussion as follows:
% perl -I lib bin/bogonsearch.pl --protocol 4 --no-checkregtime --authsource delegated | gron - | grep route | wc -l 834 % perl -I lib bin/bogonsearch.pl --protocol 6 --no-checkregtime --authsource delegated | gron - | grep route | wc -l 58
The output of these is here: ipv4: https://pastebin.com/7dSHSnHa ipv6: https://pastebin.com/XegepuBZ This turns up several additional prefixes. 156.38.0.0/24 156.38.1.0/24 156.38.2.0/24 156.38.3.0/24 169.239.104.0/22 196.43.234.0/24 41.57.124.0/22 41.57.124.0/23 41.57.124.0/24 41.57.125.0/24 41.57.126.0/23 41.57.126.0/24 41.57.127.0/24 2c0f:f0a8::/32 Nick
participants (6)
-
Cynthia Revström -
Edward Shryane -
Frank Habicht -
Gert Doering -
Nick Hilliard -
Ronald F. Guilmette