List of Atlas probes subjected to DNS traffic interception (MITM)
Hi, I am looking for a list of Atlas probes that suffer from DNS traffic interception, to exclude them from my measurements. What I mean by "traffic interception" is that DNS queries from the probe to a third-party DNS server do not reach the server, but are intercepted and answered by a middle-box instead. I started building this list myself, but it's a long and potentially error-prone process. It seems that the "DNS Root Instances" map could be used for that purpose, because DNS traffic interception shows up as if the probe was contacting an "Unknown" root instance. To get the list of probes, I ended up using an URL like the following, showing probes for all possible "unknown" root instance hostnames: https://atlas.ripe.net/results/maps/root-instances/?server=1&question=10300&... ... However, there seems to be a limit on the size of the URL so I cannot get all probes, and they are just displayed on the map without any obvious way to get the raw list of probes instead. Is there a way to get the raw list of probes from this map? Or has anybody already done this classification work independently? I also looked for DNS-related tags on probes, but could not find anything useful. Thanks, Baptiste
On Fri, Sep 29, 2017 at 02:56:12PM +0200, Baptiste Jonglez <baptiste.jonglez@imag.fr> wrote a message of 56 lines which said:
What I mean by "traffic interception" is that DNS queries from the probe to a third-party DNS server do not reach the server, but are intercepted and answered by a middle-box instead.
Many interceptors (for instance the GFC) do so only when the request matches some criteria. "Intercepting" is not all-or-nothing.
It seems that the "DNS Root Instances" map could be used for that purpose, because DNS traffic interception shows up as if the probe was contacting an "Unknown" root instance.
There are many rogue root instances (with anycast, it can be difficult to be sure of talking to a real root) so a strange instance is not always DNS interception.
I also looked for DNS-related tags on probes, but could not find anything useful.
System tag "clean DNS" would certainly be useful but, as the two examples above show, it is difficult to define precisely.
Hi Baptiste
It seems that the "DNS Root Instances" map could be used for that purpose, because DNS traffic interception shows up as if the probe was contacting an "Unknown" root instance. To get the list of probes, I ended up using an URL like the following, showing probes for all possible "unknown" root instance hostnames:
You're right. We've done the same in a study on the Roots[1]. On that time, we found 74 probes with this issue.
Or has anybody already done this classification work independently?
Root Servers return a standard answer for chaos queries. So you can use the Ripe measurements to the roots for that. Lemme illustrate that with B-Root. B-Root CHAOS IPv4 measurement is https://atlas.ripe.net/measurements/10310. The chaos answer should either be b*-lax or b*-mia (it has two anycast sites, Miami and LA). Here's how you can do it: 1. Download part of the dataset from the measurement on B-root (https://atlas.ripe.net/measurements/10310/#!download). Start with the last 30 min or so. 2. Parse the json and extract the answers [2], you'll need to decode the abuf field [3] 3. See which probes do not give the standard answers (b*-mia or b*-lax). Another indicator I found is that usually is that hijacked probes tend to have *very short RTTs*. Imagine a probe in Eastern Europe connecting on b-root in LA with a RTT of 3ms.... just physically impossible. So by coupling the chaos answers with rtt you'll be fine. Heads-up: be aware that the list of hijacked probes may change as probes can change their locations, or ISPs change their configurations. So make sure you use the right time frame you're interested. good luck, /giovane [1] https://www.sidnlabs.nl/downloads/papers-reports/imc2016.pdf [2] https://github.com/RIPE-NCC/ripe.atlas.sagan [3] https://atlas.ripe.net/docs/code/#decoding_dns_abuf
Hi Giovane, On Fri, Sep 29, 2017 at 03:55:21PM +0200, Giovane C. M. Moura wrote:
It seems that the "DNS Root Instances" map could be used for that purpose, because DNS traffic interception shows up as if the probe was contacting an "Unknown" root instance. To get the list of probes, I ended up using an URL like the following, showing probes for all possible "unknown" root instance hostnames:
You're right. We've done the same in a study on the Roots[1]. On that time, we found 74 probes with this issue.
Thanks for the pointer! Quoting the relevant part of your IMC paper: Moreover, we also discard measurements of a few VPs where traffic to a root appears to be served by third parties. We identify hijacking in 74 VPs (less than 1%) by the combination of a CHAOS reply that does not match that letter’s known patterns and unusually short RTTs (less than 7 ms), following prior work [23]. Did you discard probes that match *both* criteria, or just one of the criteria? In my preliminary experiments I did notice the very low RTT, but just filtering on unusual CHAOS replies seemed to be enough.
Or has anybody already done this classification work independently?
Root Servers return a standard answer for chaos queries. So you can use the Ripe measurements to the roots for that.
Lemme illustrate that with B-Root. B-Root CHAOS IPv4 measurement is https://atlas.ripe.net/measurements/10310.
The chaos answer should either be b*-lax or b*-mia (it has two anycast sites, Miami and LA).
I was running UDM towards a public resolver to perform CHAOS queries, but re-using the root measurements is quite clever, thanks for the idea!
Here's how you can do it:
1. Download part of the dataset from the measurement on B-root (https://atlas.ripe.net/measurements/10310/#!download). Start with the last 30 min or so.
2. Parse the json and extract the answers [2], you'll need to decode the abuf field [3]
3. See which probes do not give the standard answers (b*-mia or b*-lax).
Another indicator I found is that usually is that hijacked probes tend to have *very short RTTs*. Imagine a probe in Eastern Europe connecting on b-root in LA with a RTT of 3ms.... just physically impossible. So by coupling the chaos answers with rtt you'll be fine.
Heads-up: be aware that the list of hijacked probes may change as probes can change their locations, or ISPs change their configurations. So make sure you use the right time frame you're interested.
good luck,
/giovane
[1] https://www.sidnlabs.nl/downloads/papers-reports/imc2016.pdf [2] https://github.com/RIPE-NCC/ripe.atlas.sagan [3] https://atlas.ripe.net/docs/code/#decoding_dns_abuf
Did you discard probes that match *both* criteria, or just one of the criteria? In my preliminary experiments I did notice the very low RTT, but just filtering on unusual CHAOS replies seemed to be enough.
We only used chaos queries, since RTT suggests it but does not confirm it The thing is that the most roots are configured with IP anycast, so the same IP is shared among machines across the globe, meaning that the RTT varies depend on the relationship between the networks of the probe and the roots. What we did is not to run only for B-root, but for all letters (13 in total). That's what did, to double check.Reference 46 in the paper gives you measurement IDs for the other root letters.
I was running UDM towards a public resolver to perform CHAOS queries, but re-using the root measurements is quite clever, thanks for the idea!
Sure thing. The cool thing about these chaos measurements to the Roots is that they are "built-in" measurement. That means it uses *all* atlas probes, so you're covered. /giovane
Have you also looked at this project from the last RIPE DNS hackaton? https://recdnsfp.github.io/ Follow-up at https://www.ietf.org/proceedings/99/slides/slides-99-maprg-fingerprint-based... Cheers, Andrea ----- Original Message ----- From: "Baptiste Jonglez" <baptiste.jonglez@imag.fr> To: ripe-atlas@ripe.net Sent: Friday, September 29, 2017 1:56:12 PM Subject: [atlas] List of Atlas probes subjected to DNS traffic interception (MITM) Hi, I am looking for a list of Atlas probes that suffer from DNS traffic interception, to exclude them from my measurements. What I mean by "traffic interception" is that DNS queries from the probe to a third-party DNS server do not reach the server, but are intercepted and answered by a middle-box instead. I started building this list myself, but it's a long and potentially error-prone process. It seems that the "DNS Root Instances" map could be used for that purpose, because DNS traffic interception shows up as if the probe was contacting an "Unknown" root instance. To get the list of probes, I ended up using an URL like the following, showing probes for all possible "unknown" root instance hostnames: https://atlas.ripe.net/results/maps/root-instances/?server=1&question=10300&... ... However, there seems to be a limit on the size of the URL so I cannot get all probes, and they are just displayed on the map without any obvious way to get the raw list of probes instead. Is there a way to get the raw list of probes from this map? Or has anybody already done this classification work independently? I also looked for DNS-related tags on probes, but could not find anything useful. Thanks, Baptiste
On Fri, Sep 29, 2017 at 04:42:37PM +0200, Andrea Barberio wrote:
Have you also looked at this project from the last RIPE DNS hackaton? https://recdnsfp.github.io/
Follow-up at https://www.ietf.org/proceedings/99/slides/slides-99-maprg-fingerprint-based...
Yes, I had a look thanks to Vesna: it's interesting but too elaborate for my needs! The goal here is just to filter out "misbehaving" probes, and Giovane's method is simple and effective for this. Thanks, Baptiste
----- Original Message ----- From: "Baptiste Jonglez" <baptiste.jonglez@imag.fr> To: ripe-atlas@ripe.net Sent: Friday, September 29, 2017 1:56:12 PM Subject: [atlas] List of Atlas probes subjected to DNS traffic interception (MITM)
Hi,
I am looking for a list of Atlas probes that suffer from DNS traffic interception, to exclude them from my measurements. What I mean by "traffic interception" is that DNS queries from the probe to a third-party DNS server do not reach the server, but are intercepted and answered by a middle-box instead.
I started building this list myself, but it's a long and potentially error-prone process.
It seems that the "DNS Root Instances" map could be used for that purpose, because DNS traffic interception shows up as if the probe was contacting an "Unknown" root instance. To get the list of probes, I ended up using an URL like the following, showing probes for all possible "unknown" root instance hostnames:
https://atlas.ripe.net/results/maps/root-instances/?server=1&question=10300&... ...
However, there seems to be a limit on the size of the URL so I cannot get all probes, and they are just displayed on the map without any obvious way to get the raw list of probes instead.
Is there a way to get the raw list of probes from this map? Or has anybody already done this classification work independently? I also looked for DNS-related tags on probes, but could not find anything useful.
Thanks, Baptiste
participants (4)
-
Andrea Barberio -
Baptiste Jonglez -
Giovane C. M. Moura -
Stephane Bortzmeyer