- db-wg - mailman.ripe.net

Object Tags in the RIPE Database
by Johan Åhlén 19 Jun '13

19 Jun '13

Dear colleagues, The recent RIPE Database software rewrite has enabled us to offer new and valuable functionality to the community with minimal effort. One of these features is the Object Tags functionality, initially introduced by Kaveh Ranjbar at RIPE 66 and since then communicated to the community and the Database Working Group Chairs. We are now happy to announce that the Object Tags feature is fully deployed. Object Tags is publicly visible metadata related to an object in the RIPE Database. The tags do not appear by default and users can enable them by supplying the --show-tag-info option. The tags will appear in the comment section and it does not change the default behaviour of your queries. The main goal of the tags is to improve the data quality in the RIPE Database by, for example, better notifying users of potential issues with an object. As a secondary benefit, query results can now be filtered based on tags, making it easier for users to find exactly what they need. The RIPE NCC will maintain a list of the available tags on a webpage referenced in the Help section and in related documentation. Currently, there are two types of tags: - Objects due to be deleted by the automated clean-up are tagged: Unreferenced # will be deleted in n days - All RIPE and GRS resource data is tagged with either: {SOURCE}-REGISTRY-RESOURCE or {SOURCE}-USER-RESOURCE For more information and examples of how the new feature can be used, I'd like to refer you to Kaveh's RIPE Labs article at: https://labs.ripe.net/Members/kranjbar/ripe-database-object-tags We hope that you can come up with creative ways of using this functionality. We are eager to hear how you use this feature and what we can do to make it better. I would also like to take this opportunity to introduce myself as not everyone here knows who I am. My name is Johan Åhlén and I'm the new Assistant Database Team Manager at the RIPE NCC. I'm taking over Kaveh's role of managing the Database Team as Kaveh has accepted the new position of Head of Science Division. Before managing the Database Team, I worked as a Senior Software Engineer in the Business Applications Department where I was involved in the development of both internal and external RIPE NCC applications. I look forward to working with you all. Kind regards, Johan Åhlén Assistant Database Team Manager RIPE NCC

1 0

Historical data in ripe-db. whois with "--list-versions"
by Bengt Gördén 17 Jun '13

17 Jun '13

Hi, I've been using this service and I like it. But I ran into a problem, I think. I'm trying to see the history for 194.228.90.192/26. whois -h whois.ripe.net 194.228.90.192/26 <-------- Works. I get an inetnum for the network whois -h whois.ripe.net -- "--list-versions 194.228.90.192/26" <--------- Doesn't work. Provides no revision list whois -h whois.ripe.net 194.228.0.0/16 <-------- Works. I get an inetnum for the network whois -h whois.ripe.net -- "--list-versions 194.228.0.0/16" <--------- Works. Gives revision list Am I doing something wrong here? I suspect that the /26 inetnum hasn't changed since 2002 but shouldn't there be at least a rev. 1 for it? Or is it as it says in the proposal "there might have been cases were some changes are missing in the history, mainly for technical reasons"? regards, /bengan

2 2

API - INET6NUM Object Creation Bug
by Tim Garrison 10 Jun '13

10 Jun '13

Greetings, I have understood that doing an HTTP POST to https://apps.db.ripe.net/whois/create will respond with an HTTP201 status code and a 'Location' header pointing to the created object. This seems to work great for 'inetnum' and 'person' objects, but I've discovered that for 'inet6num' objects, the parent network block's handle is used in this header. This is creating problems for me in my current project, and I was wondering if someone could possibly take a look. Thanks! -- Tim Garrison Software Engineer II tgarrison(a)softlayer.com 281.714.4213 Direct 713.540.4325 Cell 866.398.7638 Toll-Free 281.714.4657 Fax SoftLayer Technologies, Inc. 315 Capitol Street Suite 205 Houston, TX 77002 http://www.softlayer.com/

2 2

[Training] New E-Learning Videos: Role Object and Abuse-C Set up
by Sandra Bras 10 Jun '13

10 Jun '13

[Apologies for duplicates] Dear colleagues, We are pleased to announce the launch of two new RIPE Database videos as part of our online training. The topics are: - Create a Role Object https://www.ripe.net/lir-services/training/e-learning/ripe-database/contact… - Set up Abuse Contact Information https://www.ripe.net/lir-services/training/e-learning/ripe-database/contact… For more videos go to https://www.ripe.net/lir-services/training/e-learning. Stay tuned! We plan to announce more exciting E-Learning news in the coming months. RIPE NCC E-Learning Tutorials are a free service available to everyone. If you have any questions or comments, feel free to contact us at <e-learning(a)ripe.net>. Happy learning! Sandra Brás E-Learning Project Manager

1 0

[training] RIPE NCC Training Courses July-September 2013
by Training Team 27 May '13

27 May '13

Dear Colleagues, Our training team travels the RIPE NCC service region to deliver training courses to our members without any additional cost. Over the next few months, we'll be in Rome, Chisinau, Munich, Moscow, Prague, London, Ljubljana, Yerevan, Malmö, Istanbul, Amsterdam. Visit the following page to register and to check which training courses we are giving in your area: https://lirportal.ripe.net/training/courses The RIPE NCC delivers the following training courses: - LIR Training Course - Database Training Course (new) - IPv6 for LIRs Training Course - Routing Security Training Course For more information visit: http://www.ripe.net/lir-services/training/courses With kind regards, Rumy Spratley-Kanis Training Services Manager

1 0

Reminder:Cosmetic Surgery Project, new revised RIPE Policy, document for IPv6 database objects
by Emilio Madaio 21 May '13

21 May '13

Dear colleagues, As part of the Cosmetic Surgery Project, the RIPE NCC is moving forward with a review of the policy document ripe-513, "Value of the "status:" and "assignment-size:" attributes in INET6NUM objects for sub-assigned PA space". At the request of the proposers of the original policy proposal that produced ripe-513, we’re sending a reminder about the document review and inviting the IPv6 Working Group Mailing List and Database Working Group Mailing List subscribers to participate in the review process. A draft of the policy document is now online and ready for community review at: https://www.ripe.net/ripe/readability/improving-the-readability-of-ripe-doc… We encourage you to read the edited draft of ripe-513 and send any comments to <address-policy-wg at ripe.net> by 10 June 2013. Kind Regards, Emilio Madaio Policy Development Officer RIPE NCC

1 0

RIPE 66 Draft minutes
by Nigel Titley 15 May '13

15 May '13

Dear WG For your delight and delectation, please find the draft minutes for the DB-WG meeting at RIPE 66 are attached Comments and corrections to me please Nigel

3 2

Removing personal data from bulk output from the RIPE Database
by Denis Walker 14 May '13

14 May '13

Dear colleagues, We are proposing an improvement to the algorithm used for removing personal data from the bulk provisioning of RIPE Database data. Below you will find an explanation of the current process, why we’re proposing to change it and how the new process would work. ------------------------ Current process ------------------------ At the moment this data is published in three forms: • Complete dump, accessible from: <ftp://ftp.ripe.net/ripe/dbase/ripe.db.gz> • Split dump of each object type, accessible from: <ftp://ftp.ripe.net/ripe/dbase/split/> • A live feed using NRTM protocol, described at: < https://www.ripe.net/data-tools/db/nrtm-mirroring> These data dumps and streams are mainly used for keeping an up-to-date local copy of the RIPE Database or for research and analytical purposes. To prevent the bulk publishing of personal data, for example names, email addresses and phone numbers, we remove all personal data from these datasets. We call this process "dummification of personal data". At the moment it consists of replacing all PERSON and ROLE objects with a single dummy placeholder object as well as changing all references to the personal data objects from all other objects to refer to this dummy placeholder object. ------------------------------------------------- Rationale for Improving the Process -------------------------------------------------- We’ve received feedback from different users and researchers that we are overdoing the dummification. For example, one can obtain all references to personal objects without hitting any personal object result limits by querying the live RIPE Database with proper flags (like -r). This makes the "dummification" of these references in the data dumps meaningless. For more information, see: <http://www.ripe.net/data-tools/support/documentation/aup> ----------------------------------------------------- Proposal for New Dummification Algorithm ----------------------------------------------------- In order to improve the usability of the data dumps and streams, we are proposing to change the "dummification" algorithm to keep the actual personal objects and all references to them and only obfuscate the fields with personal data (for example real names, phone numbers and addresses). The new algorithm will also try to preserve data that is useful for researchers, while not revealing any data that might expose the identity of the date subject. For example, we are proposing to keep the first half of phone number digits or to keep the domain part of email addresses. The proposed changes should not affect any End User scripts currently running because the data will still be in the valid RIPE RPSL format. We propose to run the two datasets in parallel for some time to ensure a smooth transition. Further details and examples of how the dummified objects will change are detailed in a RIPE Labs article at: <https://labs.ripe.net/Members/kranjbar/proposed-improvements-to-dummificati…> Please let us know if you agree with the proposed improvements. Regards, Denis Walker Business Analyst RIPE NCC Database Group

4 4

Re: [db-wg] Removing personal data from bulk output from the RIPE Database
by Tore Anderson 10 May '13

10 May '13

* denis(a)ripe.net > Everyone is correct in what they have said. But when it comes to the > "changed:" attribute you have to rely on assumptions, estimates and 'good > enough' because the dates are not precisely defined. It is possible 90% of > changed dates are spot on, but you don't know that. > > No one would suggest that anyone would deliberately enter the wrong date. > And of course if you don't add a date the software will generate the > correct date for you. But if you wrote a template some time ago for > creating assignments and that template has a date in it, then everytime > you fill in the template to create a new assignment, it will have the > wrong date. Or if you 'change' the date on a single "changed:" attribute > every time you update the object, it will only show the 'last updated' > date. Or as Kaveh pointed out, maybe the object was created 10 years ago, > but you updated the information last week, without altering the changed > date. Do you consider the object to be stale data? There are so many ways > this date can represent something different to what you assume it to be. So while I have no illusions about that I'd get an exact number from looking at the "changed:" attribute, what I was really trying to say is that tried to use the database dumps on ftp.ripe.net for some purpose, but due to excessive dummification I couldn't. Hence I support the proposal to reduce the amount of dummification. Whether or not my desired query would stand up to any "scientific" scrutiny is really besides the point. That said, going slightly off topic wrt. the proposal itself: I'm still no closer to finding an answer to my question, which could be stated as: «How many IPv4 PA Assignments have been made by RIPE NCC LIRs since the 14th of September 2012? (If possible, but less important: How many addresses did these assignments consist of, in total?)» Even a very rough estimation is useful to me. Even just the order of magnitude would be interesting. "A dozen", "about five thousands", or "hundreds of thousands" - these are all results I could put in my slides to underline the point I want to make. While I think the use of the "changed:" attribute I originally planned would give me a useful enough result, I'm in no way married to that particular approach. So if there's some other way to get the number, possibly by having any of you RIPE NCC guys fetch it for me from some internal data source, that would be much appreciated! :-) I'd need it before Wednesday (well, preferably ASAP) if I'm going to be able to make use of it though. > As Kaveh also pointed out, you can query an object with the > '--list-versions' option and get a more accurate, reliable list of dates > for when an object was created and each update. But if you want to find > which PA assignments were created within a time period, you would have to > query them all to get the accurate historical dates. There are 3.7 million > assignment objects. For this type of analysis it might be easier if the > split file had a well defined date(s) you could rely on. No, I really wouldn't want to script 3.7M queries to your whois server...having an auto-generated "created:" attribute in the dumps (and in the whois service too for that matter) would certainly be better. Tore

2 2

Re: [db-wg] Removing personal data from bulk output from the RIPE Database
by James A. T. Rice 09 May '13

09 May '13

On Thu, 9 May 2013, denis(a)ripe.net wrote: > The "changed:" attribute is almost useless. Disagree, it gave a good indication of how stale an object was likely to be. If an object is is 15 years old, one wouldn't be too surprised if it's inaccurate. If it's from last month, then it's a lot more likely to be accurate. Cheers James

3 2