Hello,

The REST API query to get probes is documented here:

https://atlas.ripe.net/docs/apis/rest-api-reference/probes

If you do a lookup which would return too many results, it sets the
"next" field in the return, like this:

--------------------------------------------------------------------
{
   "count": 55125,
   "next": "https://atlas.ripe.net/api/v2/probes/?page=2",
   "previous": null,
    ...
--------------------------------------------------------------------

The idea is the REST API client will then make a call to the "next" URL
and get the next batch of data; this is basic paging.

There is no kind of transaction or anything, so the status of the probes
can change between each call. This is fine since the status of the
probes changes independently and you cannot count on their status really.

One problem is that using "page" for this means it is possible to get
the same probe twice.

For example, if you are looking for "Connected" probes, and returning
the probes in order of probe ID, and a probe that you have already
returned becomes "Connected", then the answers will all shift up by one,
and you will see the last probe twice.

This can be avoided by instead of using "page" for the "next" field
using whatever the sorting is for the returned values in the query. I
think by default this is probe ID, so the "next" field could be:

--------------------------------------------------------------------
{
   "count": 55125,
   "next": "https://atlas.ripe.net/api/v2/probes/?id__gt=101",
   "previous": null,
    ...
--------------------------------------------------------------------

I suspect this would be faster on the back-end too.

I ended up doing this logic on the client side and it works fine,
although that is for a specific sorting order. It would be more
complicated to do this for all orderings on the server side.

Cheers,

--
Shane
-----
To unsubscribe from this mailing list or change your subscription options, please visit: https://mailman.ripe.net/mailman3/lists/ripe-atlas.ripe.net/
As we have migrated to Mailman 3, you will need to create an account with the email matching your subscription before you can change your settings.
More details at: https://www.ripe.net/membership/mail/mailman-3-migration/