On 27/12/2012 15:44, Joao Damas wrote:
- client side resources are more plentiful than server-side (e.g. CPU)
this will always be true.
- the client can apply smart filters rather than do full expansion
- you can keep the connection open using the -k flag and issue commands in quick succession. A big part of the slowness was due to server process (or thread in the new one) initiation and the -l flag allowed persistent connections.
this sounds like a problem associated with fork() efficiency before the widespread use of vfork() and copy-on-write memory handling techniques.
- you can also narrow down the record types that are returned by the server to those who are relevant to policy so that you don't have to parse all the irrelevant info when using RPSL policy (like all the *-c info)
this is effectively available using the -K option.
The server was designed to make combined use of these last two features very efficient.
Of course hardware limitations on servers were quite different than they are now (though a DoS is still very easy if you push more work onto the server).
I suspect things have moved on with more recent versions of the front / back-end split in the whois server mechanism. The back-end mechanism uses SQL and it should be easy to implement recursive as-set expansion using either direct sql joins or else a set of efficient
I would however encourage people to try out the above before putting more crud into the server-side
irrtoolset / bgpq3 / rpsltool and everything else already use these techniques. The problem looks like it's related almost entirely to serialisation delays, not client side delays or raw server performance delays. Looking at the 10 minute / 4 second figures, the only way to get around this with the ripe whois server is to open up multiple connections to the whois server and issue parallel whois queries - this technique will reduce the overall latency by a factor equal to the number of parallel open sessions. I don't think anyone in the NCC is going to thank me for opening up 175 parallel connections in order to get my end-user performance similar to merit irrd levels :-) Nick