August 2015 - db-wg - mailman.ripe.net

<auto-dbm@ripe.net> is slow this afternoon.
by RIPE Database Administration 17 May '18

17 May '18

Dear Colleagues, Because of a very high level of use this afternoon, the response time of <auto-dbm(a)ripe.net> is longer than normal. I have made contact with those who are responsible and I expect the situation to improve as the machine works through the queue. Please be patient. Obviously we at the RIPE NCC Database Group are continuously working to make the performance of the database as good as possible. However, we cannot make improvements in zero time. We thank you for your co-operation up to now and we ask for your continued co-operation so that everyone receives the best possible service. If you have anymore questions, please contact <ripe-dbm(a)ripe.net>. Regards, Ambrose Magee ____________________________ RIPE Database Administration.

2 2

[training] RIPE NCC Training Courses October-December 2015
by Training Services 27 Aug '15

27 Aug '15

Dear colleagues, Our training team travels the RIPE NCC service region to deliver training courses to our members without any additional cost. Over the next few months, we'll be in Amsterdam, Athens, Odessa, Naples, Cardiff, Barcelona, Oslo, Bucharest, Budapest, Istanbul, Zagreb, Doha, Berlin. Visit the following page to register and to check which training courses we are giving in your area: https://lirportal.ripe.net/training/courses The RIPE NCC delivers the following training courses: - LIR Training Course - RIPE Database Training Course - Basic IPv6 Training Course - 2 days Advanced IPv6 Training Course - Routing Security Training Course - DNSSEC Training Course For more information visit: https://www.ripe.net/support/training/courses With kind regards, Rumy Spratley-Kanis Training Services Manager

1 0

Updating maintainer objects with filtered auth lines
by Daniel Stolpe 24 Aug '15

24 Aug '15

I just had a very specific experience but this is a general problem. What I wanted to do was update a maintainer object in the database. No problems since I was authorized via a PGP key. The problem was that one of the lines looked like: auth: MD5-PW # Filtered And although I have no personal use for the md5 hash I did not want to disturb users who might have. I asked the NCC for advice and the answers were like "if you have an access account" (yes) you can add a line "auth: SSO xxxx(a)xxxx.xx" (yes). Still not a solution to my problem since adding the SSO line would just break the md5 anyway. The solution this time was that I eventually found a local copy of the object with the md5 hash unfiltered. I know the md5 hashes are a security problem and I do not recommend anyone using them but as long as they are there the filtering causes trouble. And by the way, we now see a lot of auth: SSO # Filtered What I am looking for is a way to retrieve the whole unfiltered object for anyone authorized, or, at least, a possibility to updated the object without touching the filtered lines. Maybe you could send a PGP signed request, not for updating but just for viewing the complete object? Best Regards, Daniel Stolpe _________________________________________________________________________________ Daniel Stolpe Tel: 08 - 688 11 81 stolpe(a)resilans.se Resilans AB Fax: 08 - 55 00 21 63 http://www.resilans.se/ Box 45 094 556741-1193 104 30 Stockholm

6 9

call for application authorisation ideas
by Job Snijders 24 Aug '15

24 Aug '15

Hi group, I know all of you have automated their interactions with the RIPE database entirely. :-) As it currently stands, there are two ways to automate interaction with the RIPE DB: a) send GPG signed emails (and wait for the dumb greylisting to clear) b) call the http://rest.db.ripe.net/{source}/{objecttype}/{key}?password=XXX restful API Option A works, but is not very nice if you want something done immediatly and require a confirmation that the operation succeeded. The advantage of option A is that you don't need any auth: MD5-PW lines in your mntner object and RIPE stores no sensitive data. Option B goes very well with modern programming paradigms, but the disadvantage is that you need to have an "auth: MD5-PW XXX" line in your maintainer, and should that pass ever leak somehow it could be used through webupdates / email / api, from any location. I think all in the group agree that it would be very nice if you can perform 100% of all operations without the need for any MD5-PW. I'll even go as far as stating that we should not be looking at successors such as SHA3-PW, let's leap forward and make the PW auth concept entirely obsolete. :-) CALL FOR IDEAS ============== Now that personalised authorisation is covered and in progress, what about our poor applications? How should they authenticate with the RIPE DB? I see value in stuff like signalling to RIPE "this token can only be used for the API from this source IP address", I've also heard that OAUTH2 is magic and populair for app2app auth. Or maybe all of this is is overkill and we just need to GPG sign the payload of the requests to rest.db.ripe.net and call it a day? DB-WG, please speak up and voice your ideas! Kind regards, Job

5 10

Re: [db-wg] [opensource-wg] Question for RPSL Parser
by Stavros Konstantaras 20 Aug '15

20 Aug '15

Shane is right, I don’t have the intention to shuffle the project with Bison and Flex because 1) I don’t have the knowledge of these technologies 2) I don’t have the time to learn them 3) I want to Pythonize as much as I can and be independent as much as possible (OK maybe a little bit dependent from other Python Libraries but this is the boundary). I am sure that you guys made a great job 14 years before but now or soon RPSL will be replaced with something else so I still want my toolset to be alive by easily and quickly replacing 1-2 Python libraries. What my prototype does now is extract the attributes and values by using Regular Expressions, a solution that has already been proposed in this discussion and I think this is the way to go. In theory, it can be an easy parser based on Python’s RE but complexity starts when -for example- RPSL says “accept as-set BLABLA” instead of “accept 10.100.200.0/16" and that as-set needs to be resolved (another query to RIPE DB). But anyway, thanks for your ideas. Crazy or not, I don’t mind everyone is welcome. If I still need to reply to Tomas last e-mail but I am quite busy today. Kind Regards Stavros > On 20 Aug 2015, at 13:05, Shane Kerr <shane(a)time-travellers.org> wrote: > > Denis, > > There are two problems with this: > > 1. Lex and YACC (actually flex and bison for the RIPE Database) are > tightly integrated into C (or C++ if you are feeling brave). > > 2. Stavros does state a goal of avoiding reuse of past or legacy > software, which surely means the RIPE Database code. (Okay, I think > it was only 12 or 14 years ago when we wrote these parts, but that > surely counts as "past"...) > > :) > > Cheers, > > -- > Shane > > On Thu, 20 Aug 2015 10:56:20 +0000 (UTC) > denis walker <dw100uk(a)yahoo.co.uk> wrote: > >> HI >> The RIPE Database I believe still uses LEX and YACC to parse the routing policy information from the RPSL attribute/value pairs. Is it possible to just 'drop' this into a python script and extend it to do any extra parsing you want? (I know nothing about python so I may be way off the mark here :) but it is just a thought) >> cheersdenis >> From: Shane Kerr <shane(a)time-travellers.org> >> To: Stavros Konstantaras <stavros(a)nlnetlabs.nl> >> Cc: db-wg(a)ripe.net >> Sent: Thursday, 20 August 2015, 5:21 >> Subject: Re: [db-wg] [opensource-wg] Question for RPSL Parser >> >> Stavros, >> >> On Wed, 19 Aug 2015 16:00:59 +0200 >> Stavros Konstantaras <stavros(a)nlnetlabs.nl> wrote: >> >>> To make things more clear, our project at NLnet Labs is the >>> development of "a modern IRRtoolset" written in Python and targeting >>> any operator, not just another homemade tool for our own needs. Or to >>> say in more detail the creation of a tool that is able to configure >>> BGP routers directly+automatically by extracting the policies from >>> RIPE DB. This means that I am struggling to avoid the (re)use of any >>> past or legacy software that exists, otherwise we loose independency >>> and inherit restrictions from the past. >> >> I don't know of any Python RPSL parser you can use. >> >> Your goals make a certain amount of sense. Unfortunately RPSL is an >> ugly language... it's actually more like several languages in one, all >> bad. :P >> >> * Start with simple attribute/value, one per line >> >> * Oh, but then "RPSL-ize", which allows line continuation and >> end-of-line comments >> >> * Oh, actually we want to make a separate grammar for each attribute >> >> * But be sure to make all of this "extensible", to insure maximum >> confusion! >> >> My approach in the past has been to start with a simple generic parser >> that split text into objects, objects into attributes, extracts >> attribute name & values (handling line continuation and end-of-line >> comments). You can do all of this with Python regular expressions, >> something like r"\n(?![ \t+])" to split objects into attributes; >> cleaning end-of-line comments & line continuation characters is >> straightforward. Then you can deal with the grammar for the attributes >> you know, and ignore the rest. You can extend the parsing to be more >> and more comprehensive until you are able to parse the data set you >> worry about. >> >> I'm not sure this is worth doing... but if there was a good, >> easy-to-use Python library then maybe interesting things would result? >> >> >> >> Cheers, >> >> -- >> Shane >> >> >> >> >

2 1

Re: [db-wg] [opensource-wg] Question for RPSL Parser
by Stavros Konstantaras 20 Aug '15

20 Aug '15

Hi Thomas and others, thanks for your quick replies. To make things more clear, our project at NLnet Labs is the development of "a modern IRRtoolset" written in Python and targeting any operator, not just another homemade tool for our own needs. Or to say in more detail the creation of a tool that is able to configure BGP routers directly+automatically by extracting the policies from RIPE DB. This means that I am struggling to avoid the (re)use of any past or legacy software that exists, otherwise we loose independency and inherit restrictions from the past. Currently, I do have a working proof of concept with a simple RPSL parser written in Python which is able to parse simple policies with (v6)import(s)-(v6)export(s)-accept-announce-action-pref-med-coomunity attributes. Convenient I would say for many existing policies like KPN’s, RIPE’s etc. but definitely it will crash in complex policy files like GRNET’s. My current solution uses also BGPq3 to translate AS, AS-SET and ROUTE-SET and this is also something that I want to ged rid of. So I was looking for the existence of a Python Library which can do a complete job and translate any existing policy in an intermediate data structure like XML (this is what I do now). I was hoping to find one, adopt it and present our (completed) work in the next RIPE meeting (yes I know I am very ambitious, I need to slow down). * Masimo, I already checked rpsltool and discarded it. But thanks for your super-fast reply. * Tomas, thanks for pointing me to your library. I will have a look at it and check if it is convenient for our project. * George, yes you are right. This should be the #1 step, take corresponding RPSL RFCs and implement them in Python. However, that would take too much time and will not allow us to work on other missing functionalities of the prototype. And we would like to have something ready at the beginning of November. However, I will check your solution as well. I hope now that I made myself more clear. Big thanks to everyone, every help, info or idea is kindly appreciated. Kind Regards Stavros ---- Stavros Konstantaras Internet Research Engineer, NLnet Labs Science Park 400, 1098 XH, Amsterdam Fingerprint: 4502 7D16 2DF8 ADB0 4AA6 21A9 BF9E EFFF 2744 FE5D > On 19 Aug 2015, at 14:39, Tomas Hlavacek <tomas.hlavacek(a)nic.cz> wrote: > > Hi Stavros! > > On 08/19/2015 02:15 PM, Ondrej Filip wrote: >> >> Vis o necem? >> >> -------- Forwarded Message -------- >> Subject: [opensource-wg] Question for RPSL Parser >> Date: Wed, 19 Aug 2015 14:13:58 +0200 >> From: Stavros Konstantaras <stavros(a)nlnetlabs.nl> >> To: db-wg(a)ripe.net >> CC: opensource-wg(a)ripe.net >> >> >> >> >> Hi all WG members, >> >> For my project at NLnet Labs I would like to ask you if you know any >> (relatively) complete RPSL parser written in *Python*. I searched >> around and I didnât find anything. Does someone know any repository >> where I can get one or maybe contact someone who has written one? > > It depends on what you need... I wrote one - > https://github.com/tmshlvck/bgpcrunch , but the aim was rather data > analysis than "authoritative" RPSL translation. When it comes to filter > interpretation my approach is quite "heuristic" and it is focused on > speed and ability to take short-cuts which wouldn't be possible with > full parsing the RPSL objects according to EBNF metasyntax. > > Have you considered creating simple Python interface for peval from > IRRToolSet? It might do the job if you don't need any special features. > > Cheers, > Tomas

4 4

Question for RPSL Parser
by Stavros Konstantaras 19 Aug '15

19 Aug '15

Hi all WG members, For my project at NLnet Labs I would like to ask you if you know any (relatively) complete RPSL parser written in Python. I searched around and I didn’t find anything. Does someone know any repository where I can get one or maybe contact someone who has written one? Thanks in advance. Kind Regards Stavros ---- Stavros Konstantaras Internet Research Engineer, NLnet Labs Science Park 400, 1098 XH, Amsterdam Fingerprint: 4502 7D16 2DF8 ADB0 4AA6 21A9 BF9E EFFF 2744 FE5D

3 2

NCC still enforcing descr: content
by Jan Saner 15 Aug '15

15 Aug '15

Hello Since the NCC still seems to heavy-handedly revert changes to descr: in AUT-NUM, INETNUM and INETNUM6 I’m wondering what the current status of the implementation mentioned by Tim in a post to this list from February is: https://www.ripe.net/ripe/mail/archives/db-wg/2015-February/004496.html Quote: ----------------------------------------------------------- "> When will this change go into effect? We were thinking of mid April, for two reasons: 1) This gives the working group enough time to comment. 2) While the RIPE NCC has finished the effort to ensure that the org-id (and name) are set up correctly for all INETNUM and INET6NUM objects, we are still finalising this work for AUT-NUM objects. We expect that we will finish this effort in about 6 weeks. For consistency it would be easiest to change this for all resource types simultaneously." ----------------------------------------------------------- I personally know a case where the NCC went as far as to ignore German “umlauts” in the name and deliberately spell it wrong in the descr: field of the guy’s AUT-NUM object. When he tried to set it right (ae (ä) instead of a), the NCC reverted the change. Then there’s a case where someone tried to remove his middle name from the descr: field —> reverted. Another post on the same subject: https://www.ripe.net/ripe/mail/archives/members-discuss/2015-June/001737.ht… Looks like there’s still a lot of energy wasted in those descr fields now that the org pointer is mandatory. Cheers

10 17

Upcoming work by the RIPE NCC Database team
by Tim Bruijnzeels 14 Aug '15

14 Aug '15

Dear working group, Following working group discussions in different threads we would like to give you a quick overview of current and planned work. 1= SSO integration in Webupdates As you may know we have been working on this in the background for a while. The objective of this effort is to improve the integration of SSO with Webupdates. Where previously we were asking users to provide plain text (MD5) passwords, and storing them in the session, we now want to encourage everyone to use their SSO account instead. We expect that we will finish this work 2-3 weeks from now, and plan to deploy a new release 1.81 to the RC environment then. 2= Changed phase 3 - deprecating "changed:" We plan to work on this as soon as the SSO release is deployed to the RC environment. We we expect that we can have a release 1.82 for this ready for testing in the RC environment by the end of September. As mentioned before, because the impact of this change can be bigger than usual, we plan to have this release in RC for testing for 6 weeks before deploying it to production. 3= Descr We don't foresee major technical issues with the options being discussed currently, but can only present a definite implementation plan when consensus is reached on this. If consensus is reached in time, it may be an option to postpone the release of 1.82 above and have a bigger release that also includes this work - to avoid delays because of the long RC period required for that release. 4= Personalized Authorization We are very happy to have a mandate from the chairs to work on a proof of concept for personalised authentication. We look forward to implementing this as soon as we have the opportunity and working closely with the working group using the RC environment. In addition to core whois changes we will also deploy improvements to Webupdates in the RC environment when we work on this - to exemplify how we can make use of this to improve usability. Kind regards Tim Bruijnzeels Assistant Manager Software Engineering RIPE NCC

1 0

Proposal about personalised authorisation
by Tim Bruijnzeels 12 Aug '15

12 Aug '15

The RIPE NCC has discussed the concept of personalised authorisation on various occasions, most recently at the DB WG session at RIPE 69. Following discussions and input from the working group we would now like to propose the following additions to the RIPE Database: = Extend the person object template with "auth:" as an optional, multiple attribute, with all current authentication methods. = Extend the mntner object "auth:" attribute with a new method that allows a reference to a person object that has at least one "auth:" attribute. There is no requirement to use these extensions, but users that choose to do so can benefit from the following advantages: = Currently it is is not always clear who uses the credentials in each "auth:" attribute on a mntner. There are two main problems with this. First of all this can result in authorisation being given to the wrong users, and second this makes it difficult to keep these objects up to date. For example it is not easy to remove the password for a specific user from a mntner without knowing, or deducing, which password hash is theirs. Having explicit references to person objects will make this clear. = Users who are authorised for multiple mntners will be able to maintain their own credentials in one place. = Audit logging can be improved. It will become possible to show to authorised users of the database which person made which change to their object. = The security of regaining lost access to a maintainer can be improved by letting users of the database reset the credentials on their own person object, rather than on a maintainer. Especially when the person object is linked with a Single Sign-On account. Allowing "auth:" attributes on person objects also allows us to make it easier for users to manage their person object in the RIPE Database in combination with their Single Sign-On (SSO) account on RIPE NCC Access as a single identity. To facilitate this we propose the following: = Being they are part of the same identity, person objects can only be associated with one SSO account, and vice versa. = Associated person objects and SSO accounts are automatically kept in sync. = SSO accounts can associated with new person objects, or existing person objects, provided that the user has access to both the SSO account and at least one maintainer of the person object. = Users will be able to decouple their own SSO account and person object.

4 4

Help needed to delete an object via email
by Hank Nussbacher 07 Aug '15

07 Aug '15

I believe I have encountered a bug with email processing of updates via auto-dbm@ripe and have not been able to engage the people there via ticket NCC#2015080034. I reported it and the ticket was quickly closed without being solved. I therefore ask for help from the hive-mind. The recently added "remarks:" line is causing parsing problems. The line looks like this: remarks: For information on "status:" attribute read https://www.ripe.net/data-tools/db/faq/faq-status-values-legacy-resources First I tried editing an object but the parser kept chopping the line as follows: >remarks: For information on "status:" attribute read >https: >//www.ripe.net/data-tools/db/faq/faq-status-values-legacy-resources >***Error: "https" is not a known RPSL attribute So I just deleted the line and was able to update the object. But now I want to delete an intenum object with the "remarks" line. When I do that I get: >remarks: For information on "status:" attribute read >https: >//www.ripe.net/data-tools/db/faq/faq-status-values-legacy-resources >... >created: 1970-01-01T00:00:00Z >last-modified: 2014-05-27T13:18:49Z >source: RIPE > >***Error: Object [inetnum] 192.114.48.0 - 192.114.63.255 doesn't match >version > in database The parser again chops the line at https and auto-starts a new line and therefore flags the entry as not macthing the version in the DB. Is there a workaround for this? Some escape character that will cause it not to break the line? Thanks, Hank

5 11

RIPE DB syntax check for mp-import
by Tomas Hlavacek 06 Aug '15

06 Aug '15

Hello! I just noticed that aut-num object for AS2852 in RIPE DB contains line mp-import: afi ipv6.unicastfrom AS39790 with 'afi ipv6.unicast' and 'from' keywords merged, which causs some confusion of my parsing scripts. Maybe I am missing something but I think this is not syntactically valid, is it? And if it is not, how could that get into RIPE DB? Tomas

3 2