mat-wg

Download

mat-wg@ripe.net

September 2024

3 participants
2 discussions

Some historical RIPEstat data will not be available from 24 September
by Felipe Silveira 30 Sep '24

30 Sep '24

Dear colleagues, As we shared back in May, we have been working to reduce our data centre footprint by managing the data for RIS and RIPEstat more efficiently[1]. We have experienced some problems which mean that parts of the historical RIPEstat data will not be available for an extended period. To reduce our data centre usage, we have been exporting RIS-related datasets from the database that contains our largest datasets (HBase) to a new environment, which allows us to decommission our old Hadoop cluster. During the migration, we encountered timeouts and memory limitations with old servers and software that held historical data. Here we faced a difficult choice: we could either extend our data centre contracts at considerable additional cost, or we could accept that certain historical data would not be available to users for an extended period. We decided on the second approach. It is important to be clear that we haven’t lost anything here. However, this does mean that we will not be able to provide aggregated historical data for a number of API calls and their corresponding widgets for some time after we complete our migration. These include: as-path-length, as-routing-consistency, asn-neighbours, asn-neighbours-history, bgp-update-activity, bgp-state, bgp-updates, bgplay, ris-peerings, routing-status, visbility. As mentioned, this has been a difficult process. We may still encounter other issues when we make the switch to the new environment tomorrow (24 September). My team is doing their utmost to keep any disruption to a minimum, but there are no guarantees everything will go to plan. We will be looking at the most efficient way to rebuild access to the historical data in the coming months, and we will share more details in an article on RIPE Labs soon. Kind regards, Felipe Victolla Silveira Chief Technology Officer RIPE NCC [1] For more on this, refer to the following: - Reducing our Data Centre Footprint (RIPE Labs): https://labs.ripe.net/author/felipe_victolla_silveira/reducing-the-ripe-ncc… - RIPE 88 Technology Update (slides 10-20): https://ripe88.ripe.net/wp-content/uploads/presentations/89-RIPE-88-RIPE-NC…

3 2

RIPEstat database migration postponed
by Felipe Silveira 26 Sep '24

26 Sep '24

Dear colleagues, Following Monday’s announcement, we have decided to postpone the move of our largest datasets (HBase) to a new environment by one month. As I mentioned in my last email, this has been an unexpectedly complex process that has required a massive effort from our engineering teams. To ensure that RIPEstat will continue operating, we did not feel confident going ahead when this could result in the service being severely disrupted. I am sure many of you have questions. We plan to publish a RIPE Labs article with more details soon, including which historical data will be delayed when the migration is fully completed. Kind regards, Felipe Victolla Silveira Chief Technology Officer RIPE NCC

1 0