Options for extending "abuse-c:"
Dear colleagues, At RIPE 67 in Athens, the RIPE NCC agreed to take another look at the implementation of RIPE Document ripe-563, "Abuse Contact Management in the RIPE Database." Two issues have been identified that are seen to be difficult to handle with the current model - partitioned subnets within one organisation and adding abuse contacts to more specifics for End Users. The RIPE NCC has considered these two issues and found what we believe to be practical solutions, available within the current model. More information about these solutions and the implementation of "abuse-c:" is available on RIPE Labs: https://labs.ripe.net/Members/denis/suggestions-for-improving-abuse-handling This topic will also be raised during the Anti-Abuse Working Group session at RIPE 68 in Warsaw. Regards, Denis Walker Business Analyst RIPE NCC Database Team
Hello, On 05/06/2014 06:01 PM, Denis Walker wrote:
Two issues have been identified that are seen to be difficult to handle with the current model - partitioned subnets within one organisation and adding abuse contacts to more specifics for End Users. The RIPE NCC has considered these two issues and found what we believe to be practical solutions, available within the current model.
More information about these solutions and the implementation of "abuse-c:" is available on RIPE Labs: https://labs.ripe.net/Members/denis/suggestions-for-improving-abuse-handling
I find this suggestion clumsy. It adds hard to parse extraneous information to simple objects. The organization object for a very large organization would become unmanageable and unintelligible quickly. I would much rather like to see a new inetnum and inet6num object status "INFORMATIONAL" that only requires authorization of the immediately larger enclosing inet(6)num object, and doesn't represent an assignment or allocation at all. Such objects can then be used to redirect technical, administrative and abuse contacts to the proper places, as well as present their own remarks and descriptions. This solution would cover PI, PA and all other styles of address blocks equally well. I know this has been suggested many times before, but I still think it would be a much more elegant solution to this problem. Yours, -- Aleksi Suhonen
Dear Aleksi Thank you for your comments. I have replied in more detail in line below. Sorry for another long email, but there really are a lot more discussion points about abuse-c implementation than you think...so I highlighted the important bit which was the last paragraph below and moved it to here: ****** The important bit ******* I may be going off at a tangent here, but I am trying to explain some of the background thinking as to why we implemented the abuse-c the way we did. We are trying to centralise the 'management' data in the database and link it to the organisation and allow it to be inherited by other data. In the long term this should reduce the amount of the management data in the database. We don't want to go the other way and increase the amount of duplicated management data. Right now the database has far more management data in it than operational data. With proper use of inheritance and better management tools, there could be a massive reduction of this management data. ***************************** Regards Denis Walker Business Analyst RIPE NCC Database Team On 07/05/2014 03:27, Aleksi Suhonen wrote:
Hello,
On 05/06/2014 06:01 PM, Denis Walker wrote:
Two issues have been identified that are seen to be difficult to handle with the current model - partitioned subnets within one organisation and adding abuse contacts to more specifics for End Users. The RIPE NCC has considered these two issues and found what we believe to be practical solutions, available within the current model.
More information about these solutions and the implementation of "abuse-c:" is available on RIPE Labs: https://labs.ripe.net/Members/denis/suggestions-for-improving-abuse-handling
I find this suggestion clumsy. It adds hard to parse extraneous information to simple objects. The organization object for a very large organization would become unmanageable and unintelligible quickly.
Who do you believe is going to parse this object for this information? The RIPE NCC already has an Abuse Finder tool which can be accessed directly or via RIPEstat. As I said in the last paragraph of my article, people should start to move away from the old fashioned idea of digging directly into the RIPE Database themselves to find data, parse it and interpret it. If you need information the RIPE NCC will provide web tools and API calls to supply that information. We will do all the digging, parsing and interpretation for you. We will also provide tools for maintainers of the data to provide you with a neat overview of who handles abuse throughout your whole network. We also proposed a wizard for adding and removing abuse contact details for your End Users. We can also add functionality to the overview page to add/remove further abuse details for your subnets. In the end it should not matter to you where this data is stored in the database. You deal in information, we deal in data storage and retrieval. To be honest we don't even need to put this data into any object. We can just store it as meta data associated with your organisation. As long as we provide you with tools to view and manage the information and for the public to find what they need....leave the storage to us.
I would much rather like to see a new inetnum and inet6num object status "INFORMATIONAL" that only requires authorization of the immediately larger enclosing inet(6)num object, and doesn't represent an assignment or allocation at all. Such objects can then be used to redirect technical, administrative and abuse contacts to the proper places, as well as present their own remarks and descriptions.
I believe this is going in completely the wrong direction. This means creating more 'management' objects and replicating even more data all over the database. We already have 3.8 million INETNUM objects all with a mandatory admin-c and tech-c. That is 7.6M bits of replicated data! We only have 10k members. These members manage the majority of the end user resources as well as their own networks. What this database is really missing is inheritance. Most of this management information could be stored with your organisation, as the abuse-c is, and then inherited by most of the operational data. Just did some quick stats...we have 7.4M objects in the database and 1.96M unique nic-hdls used in the admin-c, tech-c and zone-c attributes. We know some large users have a business model to create a nic-hdl for every customer. They certainly account for a few hundred thousand of these nic-hdls. So probably we have about 1.5M nic-hdls replicated over 7.4 million objects. That is a lot of data duplication.
This solution would cover PI, PA and all other styles of address blocks equally well. I know this has been suggested many times before, but I still think it would be a much more elegant solution to this problem.
Yours,
Hello dbwg, On 05/07/2014 01:52 PM, Denis Walker wrote:
Sorry for another long email, but there really are a lot more discussion points about abuse-c implementation than you think...so I highlighted the important bit which was the last paragraph below and moved it to here:
Thank you.
On 07/05/2014 03:27, Aleksi Suhonen wrote:
I find this suggestion clumsy. It adds hard to parse extraneous information to simple objects. The organization object for a very large organization would become unmanageable and unintelligible quickly.
Who do you believe is going to parse this object for this information?
Anyone who *wants* to? Isn't that what open source, open data and open interfaces are for?
The RIPE NCC already has an Abuse Finder tool which can be accessed directly or via RIPEstat. As I said in the last paragraph of my article, people should start to move away from the old fashioned idea of digging directly into the RIPE Database themselves to find data, parse it and interpret it. If you need information the RIPE NCC will provide web tools and API calls to supply that information. We will do all the digging, parsing and interpretation for you.
Oh dear, I'm feeling a huge ideological rift has developed here. Let me get this straight: What you want is to hide away the whole RIPE database and provide a point and click interface which only dispenses those morsels of information you think people should have? This may be fine for casual end users. However, I object at the idea of having it as the central design paradigm of the whole RIPE db.
In the end it should not matter to you where this data is stored in the database. You deal in information, we deal in data storage and retrieval. To be honest we don't even need to put this data into any object. We can just store it as meta data associated with your organisation. As long as we provide you with tools to view and manage the information and for the public to find what they need....leave the storage to us.
Um ... and the ISPs who use some integrated provisioning and IPAM software to automatically manage all their RIPE data should go and hire more secretaries to point and click at the wizard instead? How do I get back to the 1990s where all the other people who think like me used to live?
I would much rather like to see a new inetnum and inet6num object status "INFORMATIONAL" that only requires authorization of the immediately larger enclosing inet(6)num object, and doesn't represent an assignment or allocation at all. Such objects can then be used to redirect technical, administrative and abuse contacts to the proper places, as well as present their own remarks and descriptions.
I believe this is going in completely the wrong direction. This means creating more 'management' objects and replicating even more data all over the database. We already have 3.8 million INETNUM objects all with a mandatory admin-c and tech-c. That is 7.6M bits of replicated data! We only have 10k members. These members manage the majority of the end user resources as well as their own networks. What this database is really missing is inheritance. Most of this management information could be stored with your organisation, as the abuse-c is, and then inherited by most of the operational data.
This line of reasoning seems to lead to completely removing all inetnums and simply listing all address space in the organization object instead?
Just did some quick stats...we have 7.4M objects in the database and 1.96M unique nic-hdls used in the admin-c, tech-c and zone-c attributes. We know some large users have a business model to create a nic-hdl for every customer. They certainly account for a few hundred thousand of these nic-hdls. So probably we have about 1.5M nic-hdls replicated over 7.4 million objects. That is a lot of data duplication.
What if, instead, you just make the contact attributes of most object types optional? If they are missing they get inherited from either the organization or the maintainer object. And then to clean up the database, you can remove all those contact attributes that are the same as what inheritance would yield. As an unexpected added bonus, you will gain all the problems of multiple inheritance too! High five for modern concepts! By the way, am I the only one that thinks 10 million RIPE objects is not a lot of data in 2014? Yours, -- Aleksi Suhonen `What I need,' shouted Ford, by way of clarifying his previous remarks, `is a strong drink and a peer-group.' -- Douglas Adams, Life the Universe and Everything
Dear Aleksi We do seem to be drifting a long way off topic as far as abuse-c is concerned. But since you mentioned ideology I thought I would reply with some thoughts. I am not saying I have all the answers, or even any of the answers. I am not even saying all my ideas are practical or doable. I just want to raise some questions in people's minds about the complexity of the RIPE Database and the ongoing problems that so many new users have when trying to understand how to use it. On 07/05/2014 15:22, Aleksi Suhonen wrote:
Hello dbwg,
On 05/07/2014 01:52 PM, Denis Walker wrote:
Sorry for another long email, but there really are a lot more discussion points about abuse-c implementation than you think...so I highlighted the important bit which was the last paragraph below and moved it to here:
Thank you.
On 07/05/2014 03:27, Aleksi Suhonen wrote:
I find this suggestion clumsy. It adds hard to parse extraneous information to simple objects. The organization object for a very large organization would become unmanageable and unintelligible quickly.
Who do you believe is going to parse this object for this information?
Anyone who *wants* to? Isn't that what open source, open data and open interfaces are for?
The main goal of the database team is to manage, maintain and develop a modern, efficient, secure database service with features and interfaces that allows members to easily perform their daily tasks that interact with this database and provide access to the operational and registration details to the public. Although we make the software available as open source and accept patches from members who request additional features or fix known bugs, this is not the main goal.
The RIPE NCC already has an Abuse Finder tool which can be accessed directly or via RIPEstat. As I said in the last paragraph of my article, people should start to move away from the old fashioned idea of digging directly into the RIPE Database themselves to find data, parse it and interpret it. If you need information the RIPE NCC will provide web tools and API calls to supply that information. We will do all the digging, parsing and interpretation for you.
Oh dear, I'm feeling a huge ideological rift has developed here. Let me get this straight:
What you want is to hide away the whole RIPE database and provide a point and click interface which only dispenses those morsels of information you think people should have?
This is definitely not what I want, nor is it what I said. But it does no harm to occasionally ask questions of this 15 year old design and see if it still works "for everyone". In terms of hiding data away, it was agreed to hide the MD5 hash for security reasons. Let me ask you a question....does it benefit you, or me or the public that everyone can see your MNTNER object? What data exists in your MNTNER that I need to know? Do I need to know who, in your organisation, receives notifications of failed updates because of authorisation failures? Do I need to know how you have partitioned the management of your resources within your organisation? The database contains two basic types of data - public operational/registration data and a whole lot of data concerning the way you manage that operational/registration data. This management data is all about you and your organisation and the way you arrange your data. Does all of that really need to be public? Does any of the management data need to be public?
This may be fine for casual end users. However, I object at the idea of having it as the central design paradigm of the whole RIPE db.
There may be many more database users than you think that would love to have simple interfaces to this data/information. We run a full 1 day training course to teach people only the basics of using this database. Why - because it is not intuitive and the design is over complicated.
In the end it should not matter to you where this data is stored in the database. You deal in information, we deal in data storage and retrieval. To be honest we don't even need to put this data into any object. We can just store it as meta data associated with your organisation. As long as we provide you with tools to view and manage the information and for the public to find what they need....leave the storage to us.
Um ... and the ISPs who use some integrated provisioning and IPAM software to automatically manage all their RIPE data should go and hire more secretaries to point and click at the wizard instead?
Unless this provisioning software was written in the 90s, I am sure it can handle API calls.
How do I get back to the 1990s where all the other people who think like me used to live?
I would much rather like to see a new inetnum and inet6num object status "INFORMATIONAL" that only requires authorization of the immediately larger enclosing inet(6)num object, and doesn't represent an assignment or allocation at all. Such objects can then be used to redirect technical, administrative and abuse contacts to the proper places, as well as present their own remarks and descriptions.
I believe this is going in completely the wrong direction. This means creating more 'management' objects and replicating even more data all over the database. We already have 3.8 million INETNUM objects all with a mandatory admin-c and tech-c. That is 7.6M bits of replicated data! We only have 10k members. These members manage the majority of the end user resources as well as their own networks. What this database is really missing is inheritance. Most of this management information could be stored with your organisation, as the abuse-c is, and then inherited by most of the operational data.
This line of reasoning seems to lead to completely removing all inetnums and simply listing all address space in the organization object instead?
I am referring to separating out the operational data from the management of that data. The management data does not need to be public and in most organisations a simple set of management data can easily be inherited by all their resources. Web forms 'AND' API calls can return this data. It could be returned in many ways....even re-composed into RPSL objects.
Just did some quick stats...we have 7.4M objects in the database and 1.96M unique nic-hdls used in the admin-c, tech-c and zone-c attributes. We know some large users have a business model to create a nic-hdl for every customer. They certainly account for a few hundred thousand of these nic-hdls. So probably we have about 1.5M nic-hdls replicated over 7.4 million objects. That is a lot of data duplication.
What if, instead, you just make the contact attributes of most object types optional? If they are missing they get inherited from either the organization or the maintainer object. And then to clean up the database, you can remove all those contact attributes that are the same as what inheritance would yield.
This is one way of starting to separate management data from operational data. But you would still need management tools to see an overview of where you have added the optional attributes. In a large network you could easily miss where you have added them and be working with the wrong contact. It might still be better to group the additional attributes together, which would make it easier to manage for those who prefer to look at RPSL objects themselves instead of using purpose made tools.
As an unexpected added bonus, you will gain all the problems of multiple inheritance too! High five for modern concepts!
By the way, am I the only one that thinks 10 million RIPE objects is not a lot of data in 2014?
It is not a question of how many objects there are in the database. There are no tools to view the entire set and there is no use case for it. What matters is what (not how many) objects your organisation has, what is the relationship between them, where have you put optional data or how is it inherited, how do you manage them and how do you view your data. Also for the routing registry you need to be able to view a collection of operational data across many users and to be able to manipulate that data. We have been asked to look at server side expansion of set objects. This means we interpret your request, manipulate data and provide you with a view of what you want....not a stream of raw RPSL objects. Do you think it is a bad thing that we don't just give you a 90s stream of raw data objects and let you do all the parsing and manipulation yourself? There was community consensus that a default query for address space will return the abuse contact details. This is another view of information that does not include raw RPSL objects. So the precedence already exists for providing views of information directly from the RIPE Database. Regards Denis
Yours,
On Wed, May 07, 2014 at 07:26:59PM +0200, Denis Walker wrote: Dear Denis
What you want is to hide away the whole RIPE database and provide a point and click interface which only dispenses those morsels of information you think people should have?
This is definitely not what I want, nor is it what I said. But it does no harm to occasionally ask questions of this 15 year old design and see if it still works "for everyone". In terms of hiding data away, it was agreed to hide the MD5 hash for security reasons. Let me ask you a question....does it benefit you, or me or the public that everyone can see your MNTNER object? What data exists in your MNTNER that I need to know? Do I need to know who, in your organisation, receives notifications of failed updates because of authorisation failures? Do I need to know how you have partitioned the management of your resources within your organisation?
At this point of time it helps me that some MNTNERs are visible to the public. Unfortunately admin-c and/or tech-c of MNTNERs are/were used by https://stat.ripe.net/special/abuse as a last resort of finding abuse contact. I have pointed that already both to the RIPE NCC staff members and on one of the RIPE mailing lists. Piotr -- gucio -> Piotr Strzyżewski E-mail: Piotr.Strzyzewski@polsl.pl
On Wed, May 07, 2014 at 12:52:59PM +0200, Denis Walker wrote: Dear Denis
I find this suggestion clumsy. It adds hard to parse extraneous information to simple objects. The organization object for a very large organization would become unmanageable and unintelligible quickly.
Who do you believe is going to parse this object for this information? The
Users. ;-)
RIPE NCC already has an Abuse Finder tool which can be accessed directly or
RIPE NCC already has two Abuse Contact Finders, which already misleads the users. :| And yes, I know that one of them will be obsoleted.
via RIPEstat. As I said in the last paragraph of my article, people should start to move away from the old fashioned idea of digging directly into the RIPE Database themselves to find data, parse it and interpret it. If you need information the RIPE NCC will provide web tools and API calls to supply that information. We will do all the digging, parsing and interpretation for you.
And sadly speaking, what I understand here, between the lines, is let's abandon whois in some point of time. I hope I misunderstood you. Piotr -- gucio -> Piotr Strzyżewski E-mail: Piotr.Strzyzewski@polsl.pl
Dear Piotr We are not suggesting anything, certainly not abandoning whois. What we are trying to do is start to raise questions. This database design/model is 15 years old. We can say for sure it is not efficient, relationships are not good, there is massive duplication of data. But does it still do what people want? Could it be (much) better? Could we provide alternative ways to interface with it (and keeping the old ways)? Could we provide better features and services? Can we make your daily/regular tasks with the database easier and quicker and less error prone? There are many people within the community who have 'grown up' with this database and RPSL. They understand it, know how to use it, have work arounds for it's limitations, have lots of software that integrates with it. But so many new users struggle to do all these things. We see the same problems on training courses. We see the same questions being asked so many times in support tickets. We hear the same issues being raised in the background at meetings. We know these are big issues and nothing is going to be fixed/improved in one single step. But there is a big knowledge/usability gap between long term/experienced users and new users. In general many of the experienced users don't appreciate the way new users struggle with the complexity of the RIPE Database. So we are trying to raise awareness of this and find a way to move forward. Regards Denis Walker Business Analyst RIPE NCC Database Team On 08/05/2014 11:57, Piotr Strzyzewski wrote:
On Wed, May 07, 2014 at 12:52:59PM +0200, Denis Walker wrote:
Dear Denis
I find this suggestion clumsy. It adds hard to parse extraneous information to simple objects. The organization object for a very large organization would become unmanageable and unintelligible quickly. Who do you believe is going to parse this object for this information? The Users. ;-)
RIPE NCC already has an Abuse Finder tool which can be accessed directly or RIPE NCC already has two Abuse Contact Finders, which already misleads the users. :| And yes, I know that one of them will be obsoleted.
via RIPEstat. As I said in the last paragraph of my article, people should start to move away from the old fashioned idea of digging directly into the RIPE Database themselves to find data, parse it and interpret it. If you need information the RIPE NCC will provide web tools and API calls to supply that information. We will do all the digging, parsing and interpretation for you. And sadly speaking, what I understand here, between the lines, is let's abandon whois in some point of time. I hope I misunderstood you.
Piotr
On Thu, May 08, 2014 at 12:25:01PM +0200, Denis Walker wrote: Dear Denis
We are not suggesting anything, certainly not abandoning whois. What we are
Good to know.
trying to do is start to raise questions. This database design/model is 15 years old. We can say for sure it is not efficient, relationships are not good, there is massive duplication of data. But does it still do what people want? Could it be (much) better? Could we provide alternative ways to interface with it (and keeping the old ways)? Could we provide better features and services? Can we make your daily/regular tasks with the database easier and quicker and less error prone?
There are many people within the community who have 'grown up' with this database and RPSL. They understand it, know how to use it, have work arounds for it's limitations, have lots of software that integrates with it. But so many new users struggle to do all these things. We see the same problems on training courses. We see the same questions being asked so many times in support tickets. We hear the same issues being raised in the background at meetings.
We know these are big issues and nothing is going to be fixed/improved in one single step. But there is a big knowledge/usability gap between long term/experienced users and new users. In general many of the experienced users don't appreciate the way new users struggle with the complexity of the RIPE Database. So we are trying to raise awareness of this and find a way to move forward.
Please excuse me for not being effusive here. I just want to state that I highly appreciate the way you are trying to clean up all this mess. Piotr -- gucio -> Piotr Strzyżewski E-mail: Piotr.Strzyzewski@polsl.pl
On Tue, May 06, 2014 at 05:01:52PM +0200, Denis Walker wrote: Dear Denis
At RIPE 67 in Athens, the RIPE NCC agreed to take another look at the implementation of RIPE Document ripe-563, "Abuse Contact Management in the RIPE Database."
Two issues have been identified that are seen to be difficult to handle with the current model - partitioned subnets within one organisation and adding abuse contacts to more specifics for End Users. The RIPE NCC has considered these two issues and found what we believe to be practical solutions, available within the current model.
More information about these solutions and the implementation of "abuse-c:" is available on RIPE Labs: https://labs.ripe.net/Members/denis/suggestions-for-improving-abuse-handling
This topic will also be raised during the Anti-Abuse Working Group session at RIPE 68 in Warsaw.
First of all thanks for the proposed solution. I would like to comment both issues: 1. A solution to the subnet issue I perceive this proposed solution as a way of making a lot of mess whenever some customer marked with additional abuse-c leaves LIR. 2. A solution to the End User issue I like this idea. However I'm not sure why at the third screen there is RIPE-NCC-MNT mentioned, contrary to the LIR-MNT put on the fourth screen. Piotr -- gucio -> Piotr Strzyżewski E-mail: Piotr.Strzyzewski@polsl.pl
Dear Piotr Looking at your reply and others I think either I am misunderstanding the problem, or everyone is misunderstanding my proposed solutions. I understood the subnet issue to mean an organisation has more than one default abuse handling team within their organisation. For example they may have three allocations and have a different abuse team for each allocation. I did not expect an organisation to have hundreds of abuse teams, so I don't think this solution would create too much of a problem. The ORGANISATION object is not going to grow too large. For End User customers who are handling abuse, they are taking over part of the management of that internet resource. They should therefore have their own ORGANISATION object referenced from that resource and an "abuse-c:" referenced from the ORGANISATION object. For this we are offering the wizard solution that will create and delete these extra objects as required. We will also provide a management tool that will provide an overview of all additional "abuse-c:" setups within your network. Regards Denis Walker Business Analyst RIPE NCC Database Team On 08/05/2014 11:51, Piotr Strzyzewski wrote:
On Tue, May 06, 2014 at 05:01:52PM +0200, Denis Walker wrote:
Dear Denis
At RIPE 67 in Athens, the RIPE NCC agreed to take another look at the implementation of RIPE Document ripe-563, "Abuse Contact Management in the RIPE Database."
Two issues have been identified that are seen to be difficult to handle with the current model - partitioned subnets within one organisation and adding abuse contacts to more specifics for End Users. The RIPE NCC has considered these two issues and found what we believe to be practical solutions, available within the current model.
More information about these solutions and the implementation of "abuse-c:" is available on RIPE Labs: https://labs.ripe.net/Members/denis/suggestions-for-improving-abuse-handling
This topic will also be raised during the Anti-Abuse Working Group session at RIPE 68 in Warsaw. First of all thanks for the proposed solution. I would like to comment both issues:
1. A solution to the subnet issue
I perceive this proposed solution as a way of making a lot of mess whenever some customer marked with additional abuse-c leaves LIR.
2. A solution to the End User issue
I like this idea. However I'm not sure why at the third screen there is RIPE-NCC-MNT mentioned, contrary to the LIR-MNT put on the fourth screen.
Piotr
On Thu, May 08, 2014 at 12:03:02PM +0200, Denis Walker wrote: Dear Denis
Looking at your reply and others I think either I am misunderstanding the problem, or everyone is misunderstanding my proposed solutions.
I understood the subnet issue to mean an organisation has more than one default abuse handling team within their organisation. For example they may have three allocations and have a different abuse team for each allocation. I did not expect an organisation to have hundreds of abuse teams, so I don't think this solution would create too much of a problem. The ORGANISATION object is not going to grow too large.
I get it. I have mixed up both issues into one. Please excuse me, there is students party next to my office window ;-) Piotr -- gucio -> Piotr Strzyżewski E-mail: Piotr.Strzyzewski@polsl.pl
Hi, On Thu, May 08, 2014 at 12:03:02PM +0200, Denis Walker wrote:
I understood the subnet issue to mean an organisation has more than one default abuse handling team within their organisation. For example they may have three allocations and have a different abuse team for each allocation. I did not expect an organisation to have hundreds of abuse teams, so I don't think this solution would create too much of a problem. The ORGANISATION object is not going to grow too large.
For End User customers who are handling abuse, they are taking over part of the management of that internet resource. They should therefore have their own ORGANISATION object referenced from that resource and an "abuse-c:" referenced from the ORGANISATION object. For this we are offering the wizard solution that will create and delete these extra objects as required.
I pointed out in the past that creation of an extra organization object just to get an abuse-c: referenced is something I consider "too much hassle". It's a "database people think so" solution. Having an optional abuse-c: in the more-specific inet(6)num: would be a nice and low-effort solution.
We will also provide a management tool that will provide an overview of all additional "abuse-c:" setups within your network.
Nice and shiny query tools are also missing the point. Creation of a needless object just to fulfill "abuse-c: may only be referenced from an organization: object" designs is not being made less effort by nice query tools. Gert Doering -- NetMaster -- have you enabled IPv6 on something today...? SpaceNet AG Vorstand: Sebastian v. Bomhard Joseph-Dollinger-Bogen 14 Aufsichtsratsvors.: A. Grundner-Culemann D-80807 Muenchen HRB: 136055 (AG Muenchen) Tel: +49 (0)89/32356-444 USt-IdNr.: DE813185279
On 09/05/2014 19:26, Gert Doering wrote:
Hi,
On Thu, May 08, 2014 at 12:03:02PM +0200, Denis Walker wrote:
I understood the subnet issue to mean an organisation has more than one default abuse handling team within their organisation. For example they may have three allocations and have a different abuse team for each allocation. I did not expect an organisation to have hundreds of abuse teams, so I don't think this solution would create too much of a problem. The ORGANISATION object is not going to grow too large.
For End User customers who are handling abuse, they are taking over part of the management of that internet resource. They should therefore have their own ORGANISATION object referenced from that resource and an "abuse-c:" referenced from the ORGANISATION object. For this we are offering the wizard solution that will create and delete these extra objects as required. I pointed out in the past that creation of an extra organization object just to get an abuse-c: referenced is something I consider "too much hassle". It's a "database people think so" solution.
Having an optional abuse-c: in the more-specific inet(6)num: would be a nice and low-effort solution.
I think you are missing the point in both cases here. This is not "just to get an abuse-c referenced". The ORGANISATION object was added to the database back in 2004 for a reason. The database always showed 'what'. This was to show 'who'. The ORGANISATION object was designed to provide some information about 'who' manages and has control of and is accountable for Internet resources. When this responsibility is shared between organisations there should be ORGANISATION objectSSS to show the organisation details for these parties. But as with so many ideas/designs/philosophies with this database no one thinks about how these principles can be enforced. The ORGANISATION (and ROLE) objects were introduced with certain ideas in mind. But no business rules were added to enforce those ideas. It is very easy to mis-use them. Then it becomes too much hassle to do it right. If there is a low effort way to do something, without any constraints, people will do it - even if it breaks the database model. If you are not concerned with accountability in the database for management of Internet resources, we can do anything. Right now we have a default abuse handler for the LIR. We know who you are and you are accountable. Once you start adding abuse-c attributes all over a large network, referencing ROLE objects that we don't know who maintains (as MNTNER objects are pretty much anonymous), then we have lost a degree of accountability. It is very easy to hide behind an email address when you don't have to provide any other information in the public domain. What you want means all we have in the public domain for these End Users who manage the abuse reports for a resource is an email address. So we don't have any information about the End User organisation who is now managing one aspect of this resource - abuse handling. We only have an abuse email address. How do you want the Abuse Finder service to work? What do we do when this End User does not respond? We can't return any further certain information about this organisation, like alternative email address, real address, phone number, company name, as we don't have anything. So do we return the LIRs abuse email? Do we start following chains of references of other objects (like we did before and get it wrong). For example the abuse ROLE object, it's MNTNER object, any referenced PERSON objects, their MNTNERs.....Should we give out the LIRs default abuse contact with the End User's contact at the same time, in case the End User address does not work? When someone is held publicly accountable and has to provide additional information, which the LIR can validate as you know who the End User is, they are more likely to provide a working email address. We can always provide a low effort solution....but they have consequences.
We will also provide a management tool that will provide an overview of all additional "abuse-c:" setups within your network. Nice and shiny query tools are also missing the point. Creation of a needless object just to fulfill "abuse-c: may only be referenced from an organization: object" designs is not being made less effort by nice query tools.
Again you are missing the point here. Whatever solution we end up with means there will be many abuse contacts distributed over a network. The number of levels of indirection does not matter. The point is there are many of them spread across a large network. If you want to know where localised abuse contacts have been set up in your network (by whatever method we end up with), how are you going to find them? How can you get a clear overview of abuse handling in your network? Currently there is no query that gives you this overview. That is because queries return low level data, not high level information. The reason abuse-c was introduced is because what we had before was an unmanageable mixture of earlier solutions. If we allow abuse-c to be distributed across a network without proper tools to manage it, then people simply will not manage it. It will become too much hassle. Contact references will be left when not needed because they have been forgotten and can't be easily seen. Reports will go to the wrong places and be ignored. In a couple of years time we will start again with a new idea to clean up the next problem. Modern interfaces and tools to manage information in a system like the RIPE Database are pretty much standard these days. That is why other systems don't need a full day training course just to teach the basics of how to enter data into a database, protect it and retrieve it. Regards Denis Walker Business Analyst RIPE NCC Database Team
Gert Doering -- NetMaster
Hello, Thanks for looking into the 'more specific abuse-c', and presenting a solution. Technically it would solve the "subnet issue" in our case. However, the proposed solution strikes me as really weird: it does not scale, is hard to parse, and breaks with almost everything I'd expect from the RIPE DB syntax- and usage-wise. Scaling might not be important for the projected use-case - but with every extension you could run into trouble (cf. extending the abuse role project, from your upcoming presentation). If I understood your earlier mails correctly, having stronger heritage and less different objects is intentional (and I can certainly agree with that idea). But I would apply that thinking rather to all contact objects, and not single the abuse-c out. Please keep the contact syntax coherent. On 05/10/2014 05:46 AM, Denis Walker wrote:
On 09/05/2014 19:26, Gert Doering wrote:
Having an optional abuse-c: in the more-specific inet(6)num: would be a nice and low-effort solution.
And I'd add: low-effort for everyone: the one running with the default org-attached abuse-c, the one needing more specific, and the data-requester with the whois client. I'd also believe that it is an additional workload NOT to use the default, and that's why I find it difficult to share you concern of bad data. Besides, why is the abuse-c so special that is warrants all the added technical barriers around it? After all, it is just a contact. Yes, it is mandatory, but that alone does not make it special. And whatever rules you put around it, you cannot force an abuse-c to be useful - which by the end of the day is the only thing that matters.
If you are not concerned with accountability in the database for management of Internet resources, we can do anything. Right now we have a default abuse handler for the LIR. We know who you are and you are accountable. Once you start adding abuse-c attributes all over a large network, referencing ROLE objects that we don't know who maintains (as MNTNER objects are pretty much anonymous), then we have lost a degree of accountability. It is very easy to hide behind an email address when you don't have to provide any other information in the public domain. What you want means all we have in the public domain for these End Users who manage the abuse reports for a resource is an email address.
Personally I don't feel strongly about the described accountability in the database. I'd rather have flexible ease of use, along with a comprehensive toolset for the RIPE NCC to enforce policy. You can put useless data in the DB anyway, so if any data is so important that it deserves special care or even validation, make the validation out of band: you'd have to anyway, so you can be more liberal on the business logic and not annoy the rest of the rule-abiding world.
When someone is held publicly accountable and has to provide additional information, which the LIR can validate as you know who the End User is, they are more likely to provide a working email address. We can always provide a low effort solution....but they have consequences.
[...]
If you want to know where localised abuse contacts have been set up in your network (by whatever method we end up with), how are you going to find them? How can you get a clear overview of abuse handling in your network? Currently there is no query that gives you this overview.
Which, in turn, would be nice feature for the lirportal. Along with the 'do it right'-wizard. Actually, the proposed solution for the "subnet-issue" is very appealing to do the "end-user issue" wrong: what would prevent a LIR to put an abuse-c per end-user assignment in it's org-object? It would probably be easier than to create the organisation objects... All in all, fragmenting the syntax of objects depending on their intented use seems a too high cost for the presented benefits... Best regards, Gilles -- Fondation RESTENA - DNS-LU 6, rue Coudenhove-Kalergi L-1359 Luxembourg tel: (+352) 424409 fax: (+352) 422473
Dear Gilles Sorry, another long email. Maybe at some point we have to accept that email discussions are not going to resolve this matter :) To take your last point first - "All in all, fragmenting the syntax of objects depending on their intented use seems a too high cost for the presented benefits...". We have already done this in a way, which is why we are having this whole discussion. The ROLE and ORGANISATION objects had an intended use. But no business rules were added to the software to enforce the use and there were not even any best common practise rules documented. 10+ years down the line and most people have forgotten why these objects are there. Now we are trying to build a case to 'do things right' and it's seen as too much hassle. More comments in line below.... On 12/05/2014 16:09, Gilles Massen wrote:
Hello,
Thanks for looking into the 'more specific abuse-c', and presenting a solution. Technically it would solve the "subnet issue" in our case.
However, the proposed solution strikes me as really weird: it does not scale, is hard to parse, and breaks with almost everything I'd expect from the RIPE DB syntax- and usage-wise. Scaling might not be important for the projected use-case - but with every extension you could run into trouble (cf. extending the abuse role project, from your upcoming presentation).
As I understand it, the subnet case is not a large usage requirement, so scaling is not an issue. The same syntax is used by "mnt-routes:" so there is a precedent for the syntax and we already have the software to parse it. If scaling became an issue we could solve that with proper tooling. I know we have not yet put a good case for tools to help you work with the RIPE Database and experienced users don't see a need for 'simple' tools. After all, you can do everything you want with a few keystrokes on a command line, right? I come from a generation that just about remembers writing software in assembler. Then we moved to high level (C like) languages. At the time programmers thought they were losing some power and control over the processor and it's internal registers. Then high level languages became the norm and now we have fully integrated development environments.....but you can still write in assembler if you want. We can always present you with a view of your low level data in RPSL format and accept input from you in that way. But don't forget what the 'RP' stands for - 'Routing Policy'. It does not mean 'Address Management'. To finally sum up this point for now - we can build and provide you with many tools to help you with address management information that don't have to work with RPSL format data (but can do).
If I understood your earlier mails correctly, having stronger heritage and less different objects is intentional (and I can certainly agree with that idea). But I would apply that thinking rather to all contact objects, and not single the abuse-c out. Please keep the contact syntax coherent.
I agree absolutely with this and have said so on many occasions. A small organisation may only have one technical or admin team, but may still have many resources in the database. There is no need to duplicate these contact details every where. They can be managed in the same way as abuse contacts.
On 05/10/2014 05:46 AM, Denis Walker wrote:
On 09/05/2014 19:26, Gert Doering wrote:
Having an optional abuse-c: in the more-specific inet(6)num: would be a nice and low-effort solution. And I'd add: low-effort for everyone: the one running with the default org-attached abuse-c, the one needing more specific, and the data-requester with the whois client. I'd also believe that it is an additional workload NOT to use the default, and that's why I find it difficult to share you concern of bad data.
Besides, why is the abuse-c so special that is warrants all the added technical barriers around it? After all, it is just a contact. Yes, it is mandatory, but that alone does not make it special. And whatever rules you put around it, you cannot force an abuse-c to be useful - which by the end of the day is the only thing that matters.
Again you are right. The abuse contact is not special. It only seems special because we have applied the principles of the database model to it. These principles are sadly missing from so many other areas of the data. Although it has been forgotten after 10+ years the basic database model is - an organisation is the core of your data. That organisation has human resources and Internet resources. The human resources are grouped into roles and these roles manage the Internet resources. That is it in a nutshell. It sounds simple, but so many layers have been built on and around these principles, that the original principles have been partially lost. That core organisation is anyone/thing that manages (some aspect of) an Internet resource. If it is an outsourced 24/7 team, an abuse handler or an End User doing their own routing. There should be an ORGANISATION object to identify them. Everything else hangs off that object. For example, if an End User manages the routing for their resource, who do you want to contact if their is a routing problem? The End User who manages the routing or their LIR who assigned the resource? How are you going to contact that End User? From the MNTNER in the "mnt-routes:" of the assignment? Which email address should you use from the MNTNER or referenced PERSON objects? Or maybe the ROUTE object where all contact information is optional. Following the basic principles there should be an ORGANISATION object for the End User with clearly defined contact details. It makes sense to follow the basic design. It does not make sense to take short cuts and break the basic model.
If you are not concerned with accountability in the database for management of Internet resources, we can do anything. Right now we have a default abuse handler for the LIR. We know who you are and you are accountable. Once you start adding abuse-c attributes all over a large network, referencing ROLE objects that we don't know who maintains (as MNTNER objects are pretty much anonymous), then we have lost a degree of accountability. It is very easy to hide behind an email address when you don't have to provide any other information in the public domain. What you want means all we have in the public domain for these End Users who manage the abuse reports for a resource is an email address. Personally I don't feel strongly about the described accountability in the database. I'd rather have flexible ease of use, along with a comprehensive toolset for the RIPE NCC to enforce policy. You can put useless data in the DB anyway, so if any data is so important that it deserves special care or even validation, make the validation out of band: you'd have to anyway, so you can be more liberal on the business logic and not annoy the rest of the rule-abiding world.
We can provide tools to make it easy to manage data without breaking the model. Anyone who agrees to the Terms and Conditions of use of the RIPE Database agrees to enter valid and correct information.
When someone is held publicly accountable and has to provide additional information, which the LIR can validate as you know who the End User is, they are more likely to provide a working email address. We can always provide a low effort solution....but they have consequences. [...]
If you want to know where localised abuse contacts have been set up in your network (by whatever method we end up with), how are you going to find them? How can you get a clear overview of abuse handling in your network? Currently there is no query that gives you this overview. Which, in turn, would be nice feature for the lirportal. Along with the 'do it right'-wizard.
Actually, the proposed solution for the "subnet-issue" is very appealing to do the "end-user issue" wrong: what would prevent a LIR to put an abuse-c per end-user assignment in it's org-object? It would probably be easier than to create the organisation objects...
Yes it would be easier, but wrong. Some things can be managed by software business rules, other things need to rely on members following the rules. Regards Denis Walker Business Analyst RIPE NCC Database Team
All in all, fragmenting the syntax of objects depending on their intented use seems a too high cost for the presented benefits...
Best regards, Gilles
Hi Denis, On 05/12/2014 05:32 PM, Denis Walker wrote:
Sorry, another long email. Maybe at some point we have to accept that email discussions are not going to resolve this matter :)
I'd have loved to be in Warsaw and bug you directly :) Thanks for enlighten me, and from your answer I think we are not in wild disagreement.
To take your last point first - "All in all, fragmenting the syntax of objects depending on their intented use seems a too high cost for the presented benefits...". We have already done this in a way, which is why we are having this whole discussion. The ROLE and ORGANISATION objects had an intended use. But no business rules were added to the software to enforce the use and there were not even any best common practise rules documented. 10+ years down the line and most people have forgotten why these objects are there. Now we are trying to build a case to 'do things right' and it's seen as too much hassle.
So your wish is to take the current abuse-c implementation as a model for other contacts, is that correct? I would see merit in that - but I'm still not wild about the fragmentation: either it is the way forward, then it should have been said so, and be brought explicitly before the DB-WG. Before the implementation. Because if it is not (or there is too much resistance against applying the model to other contacts) then you're stuck with the two kinds of contacts: to me as a user of the database (reader or writer) as well as part time DBA that is hugely annoying. Supposing I did not misread your intentions, and the abuse-c model is the right:
As I understand it, the subnet case is not a large usage requirement, so scaling is not an issue. The same syntax is used by "mnt-routes:" so there is a precedent for the syntax and we already have the software to parse it.
Ok, I didn't know about mnt-routes, and I agree that the subnet case _should_ be limited. However, it needs to scale if applied to admin-c and tech-c's, at least much more than for the abuse-c. Maybe the proposal is enough - you have better data on that than I do - but in any case we should end up with the same for all contacts. And keep the necessary flexibility to allow database user to represent their reality. For example, the possible copyright-abuse-mailbox you are about to present (I had a peak at the slides): great idea. But in our case (the 'subnet case') that would be a company wide contact, whereas the 'real' abuse-c is network specific. We are small enough that it does not really matter (i.e. I'd duplicate the data and be done with it) - but the use case for flexibility exists. [...]
To finally sum up this point for now - we can build and provide you with many tools to help you with address management information that don't have to work with RPSL format data (but can do).
In my particular case we can live with about anything: not many objects, infrequent updates. But even then I'd really love coherence in the DB structure and logic.
Although it has been forgotten after 10+ years the basic database model is - an organisation is the core of your data. That organisation has human resources and Internet resources. The human resources are grouped into roles and these roles manage the Internet resources. That is it in a nutshell. It sounds simple, but so many layers have been built on and around these principles, that the original principles have been partially lost.
Sounds like interesting times ahead :) What's wrong with trying to get back there?
That core organisation is anyone/thing that manages (some aspect of) an Internet resource. If it is an outsourced 24/7 team, an abuse handler or an End User doing their own routing. There should be an ORGANISATION object to identify them. Everything else hangs off that object.
As not all aspects of an organisation are alike (subnet case), and duplicating organisations is a mess (and even impossible in cases like anycast assignments) sub-organisations or departments would be an way to solve this, within the basic database model?
For example, if an End User manages the routing for their resource, who do you want to contact if their is a routing problem? The End User who manages the routing or their LIR who assigned the resource? How are you going to contact that End User? From the MNTNER in the "mnt-routes:" of the assignment? Which email address should you use from the MNTNER or referenced PERSON objects?
I'd always follow basic logic: the most specific contact adapted to my request. In this case: the tech-c related to the internet resource or it's closest parent, because that seems obvious without reading the database documentation. Any other address only in case of no-reaction / despair - and then all bets are off anyway.
Or maybe the ROUTE object where all contact information is optional. Following the basic principles there should be an ORGANISATION object for the End User with clearly defined contact details. It makes sense to follow the basic design. It does not make sense to take short cuts and break the basic model.
I completely agree. On the other hand, and that might be personal preference, about the only thing I'd value above the database model is coherence in the entries. If only because it is easier to migrate a coherent mess than half a mess :)
Actually, the proposed solution for the "subnet-issue" is very appealing to do the "end-user issue" wrong: what would prevent a LIR to put an abuse-c per end-user assignment in it's org-object? It would probably be easier than to create the organisation objects...
Yes it would be easier, but wrong. Some things can be managed by software business rules, other things need to rely on members following the rules.
My point exactly. And I'll always value flexibility about over-tight rules. If life has taught me anything it's that if the 'wrong' way is too easy compared to the 'right' (and doesn't come with serious inconvenience) it will be used. Sometimes only by ignorance. But that's not something I have to tell the RIPE NCC database team :) cheers, Gilles -- Fondation RESTENA - DNS-LU 6, rue Coudenhove-Kalergi L-1359 Luxembourg tel: (+352) 424409 fax: (+352) 422473
On Mon, May 12, 2014 at 05:32:19PM +0200, Denis Walker wrote: Dear Denis
Again you are right. The abuse contact is not special. It only seems special because we have applied the principles of the database model to it. These principles are sadly missing from so many other areas of the data.
Although it has been forgotten after 10+ years the basic database model is - an organisation is the core of your data. That organisation has human resources and Internet resources. The human resources are grouped into roles and these roles manage the Internet resources. That is it in a nutshell. It sounds simple, but so many layers have been built on and around these principles, that the original principles have been partially lost.
That core organisation is anyone/thing that manages (some aspect of) an Internet resource. If it is an outsourced 24/7 team, an abuse handler or an End User doing their own routing. There should be an ORGANISATION object to identify them. Everything else hangs off that object.
So, do we have to start thinking about making admin-c/tech-c of INET(6)NUM (and others) optional and then deprecated at some point in time? Do we have to start thinking about moving whole contact details to the ORGANISATION objects? Piotr -- gucio -> Piotr Strzyżewski E-mail: Piotr.Strzyzewski@polsl.pl
participants (5)
-
Aleksi Suhonen -
Denis Walker -
Gert Doering -
Gilles Massen -
Piotr Strzyzewski