On Thu, Jan 24, 2019 at 3:52 AM Shane Kerr <shane@time-travellers.org> wrote:
Chris,
On 24/01/2019 06.39, Christopher Morrow wrote:
howdy! So, if I have a dns server somewhere and I want to make a bunch of measurements: "how does NYC see my dns?" (pick N probes near NYC) "How does DEN see my dns?" (pick N probes near DEN) "How does LHR see my dns?" (pick N probes near LHR) "how does ...."
...you get the idea... I'm trying to characterize my service from a 'user' perspective by large Metropolis, well within N km of that metropolis anyway :) but across ~70 or so metros, in v4 and v6 and with authoritative and non-authoritative queries with each test running for 4 days so i can see some daily cycles in traffic patterns/etc. The number of measurements is large in total, breaking up by 3-4 metropolis chunks is super tedious :(
Can I request an API key (or some other thing) which can make that sort of request happen for all my measurements in one go? The thing I'm probing is very able to handle a few extra thousand queries per second.. and I'd only be hurting myself i get my estimate wrong :)
I was part of a team on a hackathon a while back which needed to compute distances between points on the globe, including Atlas probes:
https://github.com/shane-kerr/ripe-atlas-anycast-work
I see that the openflights project has since moved to GitHub, and so you can get the IATA airport data here:
https://github.com/jpatokal/openflights/blob/master/data/airports.dat
The geo data wasn't problematic for me, once I found: github.com/kelvins/geocoder which apparently uses: https://raw.githubusercontent.com/jpatokal/openflights/master/data/airports.... oh hai! It looks like I didn't do it the 'insane way', mostly :) (I should make the repository of code I wrote available, oops) I guess I should update the README.md...
The repository is basically a last-point-in-time snapshot of the hackathon work, so it a bit rough. 😬 Anyway, this code is probably the closest to what you want:
https://github.com/shane-kerr/ripe-atlas-anycast-work/blob/master/add-dist.p...
i think you are just sorting through measurement response data to find things near/around/at your points of interest here, right? that seems useful.
Turning the great_circle_dist() function into a filter across the set of all probes (meta-probes.json, which you can download in advance) should get you something like what you want.
looks neat ;)
Note that this was all done 4 years ago according to GitHub, so maybe Atlas provides better tools for this nowadays. 😊
they do have a method to request probes based on geo data and a radius from that point, which is what I am doing. They also have filters (which I've apparently sucked at reading completely!) to limit based on type/ip-version/activity/etc. mostly you can query for: "probes near here with these sets of attributes" which works pretty well ;) my main problem was/is (I think) that I want to say: "For these M metros, please find 10 (or so) probes with working address family (v4||v6) and probe dns server (V4||V6 address) with a (non-)authoritative request" my M number is ~70+. I didn't want (for some reason?) to try and gather the probes across the globe into less measurements but more probes... huh, I could try that I guess, then I'd only have 4 measurements not M*4. Cheers,
-- Shane