RE: RE: HELP! 'networkupdate', diskussion on short names of attributes
Hi all ! Contents: --------- 1. Thanksgiving for quick response 2. More about the problem with 'networkupdate' 3. Comments on short names of attributes 4. Comments on whois -i option 5. Question on destination for emails in 'RIPE DB' context 1. Thanksgiving for quick response ================================== At first I would like to thank - Wilfried Woeber, UniVie/ACOnet - Geert Jan de Groot, RIPE NCC - Carol Orange, RIPE NCC and specially - David Kessens who gave me quick response to my 'cry for help'. Thank you very much for your suggestions and for taking time to think about that problem with the 'networkupdate' client programm. Below you can read more about the progress in that case. And no, there is no reason to say sorry for a 'delayed response'. I'm pleased about your quick and helpful reactions. Thank you again! 2. More about the problem with 'networkupdate' ============================================= Below i repeat parts of David Kessens answer ...
Hi Markus,
Markus Bock writes :
For writing database front ends which manipulate and visualisize the stored data we want to use the 'networkupdate' program to have access to the database from different workstations via LAN.
This function is indeed present for this purpose.
Yes, great. I think with the direct network update-method one has many potentialities to develope client-server based applications with a two task architecture like you see in the sketch below (just an example). ----------------- client-area | application | ----------- ----------------- | networkupdate | ----------------- | ^ v | communication drivers & protokolls // // network ======================================== \\ \\ // || VV communication drivers & protokolls | ^ v | ----------------- server-area | DB-server | ----------- | whoisd | ----------------- | | ----------------- / \ / \ ________ ___________ (________) (___________) | DB | | DB | | source | | source | | XXXX | | OWN | -------- -----------
Maybe I make something wrong or this effect has something to do with
running under Slowaris 2.5 ... ?
May be, may be not. Could you try the C version of networkupdate (available from ftp.ripe.net/tools/whois-tools*) and see if the same problem happens ?
Yes I did it. The same problem happens NOT. I performed several tests with the C version and everything is working fine giving the estimated results. Just for information: --------------------- I had to do the following changes inside the makefile of the C versions of the ripe whois tools to compile it successful at our site under Solaris 2.5 with gcc: - define '-DSYSV' added - additional LIBS 'LIBS= -lresolv -lsocket -lnsl' added - changing the syntax to call install # install: ${PROG} # $(INSTALL) -c ${DESTDIR} -s -m 755 ${PROG} networkupdate # $(INSTALL) -c ${MANDIR}/whois.1 -m 444 whois.man
Could you try 'networkupdate -h whois.ripe.net'. You should get a message back that tells you that you are not authorized to network updates. The problem is problably in the client side of the software if you don't get this message.
Yes, I did it. - With the perl version I didn't received the message that tells me that I'm not authorized to do network updates. There is an analogical problem like I described in my 'cry for help'. 'networkupdate' terminates immediately. - With the C version I received the estimated message % ERROR: ***You are not authorized to do network updates***. With this results I agree with you that the problem is problably in the client side of the software. I tried to find out what is potentially wrong in the software by taking the examples for a client and a server from L. Walls camel book and comparing it with the RIPE DB SW (I assume that the same mechanisms are used by the RIPE DB SW). But I was still not able to change it in that way that these examples are able to handle the input of multiple lines (I'm not familar with 'command buffering or not' stuff) or to produce an analogical failure like decribed above. So I have no further result with this case. Has anyone tested the perl version of the networkupdate programm succesfully at his site ? If yes, in which enviroment (OS, version, HW etc.) ?
Please contact <ripe-dbm@ripe.net> and see if they can help you with extra debugging code in the whoisd daemon/whois client to do a more detailed trace of what is happening in the server software.
Yes, thank you. I'll do it.
I hope this helps,
David K.
PS Please don't expect any fast responses right now since I am still busy with moving & am not working yet. I have, however, already built a new computer & got a ppp account at ISI so I am not completely disconnected anymore ;-).
From a view of data modelling I think the short names are redundant even if I know that from a view of implementing
Yes, thank you again. Your help enables me to go sucessfully on with my work. Also this gives me the opportunity to wish you success and good luck for your future at ISI. 3. Comments on short names of attributes ======================================== First: ------ While testing the RIPE DB for use to store own data objects I examined the following cases: Assume that you have to store 30 or more new database objects with about 300 (or more) new attributes in a (SPLIT) RIPE database (I already did it). Because you will have problems to find sensefull short names for attributes you use a numbering scheme like this: # DB-object dummy A1 ATTR A1 dummy_longname_1 ATTR A2 dummy_longname_2 ATTR A3 dummy_longname_3 # DB-object dummy A1 OBJ A1 ATSQ A1 A2 A3 OBJ A1 MAND A1 OBJ A1 MULT A2 A3 OBJ A1 UNIQ A1 OBJ A1 KEYS A1 Seems ok. *** But be careful. Don't use uppercase letters *** for naming database objects and attributes *** in a SPLIT database. The side effects are that at first everything is working fine (indexing, dbupdate updates via email etc.). The data will be stored correctly. But if you query the database via whois and whoisd you ever get the message that will be displayed if no match was found (have a look in your ripedb-config file). What happened ? After I had a look at the source code of the whoisd I think one can find the explanation in this section # sub findsplitdbs { # local(*keys, @preselection)=@_; # local(@searchdb)=(); # # print STDERR join(" ", keys %CLASSLESSDBS), "\n"; # @preselection=grep($CLASSLESSDBS{$_}, @preselection) if (scalar(grep(/^$VALIDPREFIXKEY$/o, @keys))==scalar(@keys)); # # # # and now we are going to do something really smart... # # and trickey things # local(@upperkeys)=@keys; # grep(tr/a-z/A-Z/, @upperkeys); # # # # see comment below why we need this # local(@tmpkeys,@tmpupperkeys); # local($tmp); and so on. I think that with this the software is not able to find the datafiles, i.e. which are named by this konvention /whereever/it/is/SOURCE.db.OBJECT /whereever/it/is/SOURCE.db.OBJECT.pag /whereever/it/is/SOURCE.db.OBJECT.dir where OBJECT is in uppercase letters. Does somebody agree with me ? Second: ------- Above I wrote about a problem to find sensefull short names if you have to store many new database objects with many (many) attributes. What you have at least for this is a two letter code with all alphabetic characters (in lowercase) and all digits. So you have (26+10) * (26+10) = 1296 (I hope it is correct...) possibilities for short names. This seems to be enough. But you have still the problem to find sensefull short names for a large amount of database objects and attributes. Maybe I see a problem 'where no problem is' but I can assume that in future even the RIPE database (if it will growth by objects and attributes) will reache the border finding sensefull names for short attributes. So my questions are: - Is it possible to use in future only the long names of attributes and objects or - needs this change too much manpower and time ? the database and writing code they are useful (i.e. for doing syntax checks). 4. Comments on whois -i option ============================== Wow ! I really like this option. If you have a somewhat clever data modelling you can realize something like a 'where clause' which is known from SQL. Another thing is that you can realize something like a 'join' over two database objects. You just have to play a little bit around with the 'KEY' and 'REC' terms inside the definition of your database objects. 5. Question on destination for emails ===================================== in 'RIPE DB' context ==================== I don't know if the database working group is the right address to report everthing in context RIPE database. Should one better report only for i.e. problems with implementation and porting the software directly to the database maintainer at RIPE NCC <ripe-dbm@ripe.net> ? and on the other side Should one better report only i.e. issues about the database concepts directly to the database working group ? Is there somewhat like a rule for this ? Thanks again ! have a nice day, Markus
Hi Markus,
Markus Bock writes :
who gave me quick response to my 'cry for help'. Thank you very much for your suggestions and for taking time to think about that problem with the 'networkupdate' client programm. Below you can read more about the progress in that case. And no, there is no reason to say sorry for a 'delayed response'. I'm pleased about your quick and helpful reactions. Thank you again!
Thanks!
2. More about the problem with 'networkupdate' =============================================
Yes, great. I think with the direct network update-method one has many potentialities to develope client-server based applications with a two task architecture like you see in the sketch below (just an example).
Agreed.
Maybe I make something wrong or this effect has something to do with
running under Slowaris 2.5 ... ?
May be, may be not. Could you try the C version of networkupdate (available from ftp.ripe.net/tools/whois-tools*) and see if the same problem happens ?
Yes I did it. The same problem happens NOT. I performed several tests with the C version and everything is working fine giving the estimated results.
Good, then the problem is not in the server. I did some tests with the perl version at RIPE and it seems to work under Linux & SUNOS. I leave it to you & the new database maintainers to see if you can find a solution for this Solaris specific behavior (no need to CC the db-wg list anymore ...)
Just for information: --------------------- I had to do the following changes inside the makefile of the C versions of the ripe whois tools to compile it successful at our site under Solaris 2.5 with gcc:
- define '-DSYSV' added
This one is in the documentation I believe...
- additional LIBS 'LIBS= -lresolv -lsocket -lnsl' added - changing the syntax to call install # install: ${PROG} # $(INSTALL) -c ${DESTDIR} -s -m 755 ${PROG} networkupdate # $(INSTALL) -c ${MANDIR}/whois.1 -m 444 whois.man
Could you send a message with a copy of your Makefile to the new database maintainers <ripe-dbm@ripe.net> so that they can update the whois source code?
Yes, I did it. - With the perl version I didn't received the message that tells me that I'm not authorized to do network updates. There is an analogical problem like I described in my 'cry for help'. 'networkupdate' terminates immediately.
- With the C version I received the estimated message % ERROR: ***You are not authorized to do network updates***.
With this results I agree with you that the problem is problably in the client side of the software. I tried to find out what
Correct.
is potentially wrong in the software by taking the examples for a client and a server from L. Walls camel book and comparing it with the RIPE DB SW (I assume that the same mechanisms are used by the RIPE DB SW).
Yes, but a bit more sophisticated.
that these examples are able to handle the input of multiple lines (I'm not familar with 'command buffering or not' stuff) or to produce an analogical failure like decribed above.
This stuff is indeed very, very trickey in perl. Best advise is usually to unbuffer all output & input although that is certainly not always most efficient.
Has anyone tested the perl version of the networkupdate programm succesfully at his site ?
If yes, in which enviroment (OS, version, HW etc.) ?
Yes, see above.
3. Comments on short names of attributes ========================================
First: ------
While testing the RIPE DB for use to store own data objects I examined the following cases:
Assume that you have to store 30 or more new database objects with about 300 (or more) new attributes in a (SPLIT) RIPE database (I already did it). Because you will have problems to find sensefull short names for attributes you use a numbering scheme like this:
# DB-object dummy A1 ATTR A1 dummy_longname_1 ATTR A2 dummy_longname_2 ATTR A3 dummy_longname_3
# DB-object dummy A1 OBJ A1 ATSQ A1 A2 A3 OBJ A1 MAND A1 OBJ A1 MULT A2 A3 OBJ A1 UNIQ A1 OBJ A1 KEYS A1
Seems ok.
*** But be careful. Don't use uppercase letters *** for naming database objects and attributes *** in a SPLIT database.
The side effects are that at first everything is working fine (indexing, dbupdate updates via email etc.). The data will be stored correctly. But if you query the database via whois and whoisd you ever get the message that will be displayed if no match was found (have a look in your ripedb-config file). What happened ? After I had a look at the source code of the whoisd I think one can find the explanation in this section
# sub findsplitdbs {
I think that with this the software is not able to find the datafiles, i.e. which are named by this konvention
/whereever/it/is/SOURCE.db.OBJECT /whereever/it/is/SOURCE.db.OBJECT.pag /whereever/it/is/SOURCE.db.OBJECT.dir
where OBJECT is in uppercase letters.
Does somebody agree with me ?
I think that it will go wrong already earlier in the code. The database code is more or less case-insensitive and no attempt has been made to preserve caase of the attribute names. This can be changed of course, but I don't know if it's a good idea to make a difference between attribute 'A1:' and 'a1:'.
Second: -------
Above I wrote about a problem to find sensefull short names if you have to store many new database objects with many (many) attributes. What you have at least for this is a two letter code with all alphabetic characters (in lowercase) and all digits. So you have
(26+10) * (26+10) = 1296 (I hope it is correct...)
possibilities for short names. This seems to be enough. But you have still the problem to find sensefull short names for a large amount of database objects and attributes. Maybe I see a problem 'where no problem is' but I can assume that in future even the RIPE database (if it will growth by objects and attributes) will reache the border finding sensefull names for short attributes. So my questions are:
- Is it possible to use in future only the long names of attributes and objects or
I agree with this. There are also better/other reasons: the datasize differences are not that much, the code will be simplified (less bugs) and faster.
- needs this change too much manpower and time ?
It doesn't need that much manpower. The bigger problem is the backwards compatibility. That certainly needs some work for converting back long->short names when doing -F queries.
4. Comments on whois -i option ==============================
Wow ! I really like this option.
If you have a somewhat clever data modelling you can realize something like a 'where clause' which is known from SQL.
Another thing is that you can realize something like a 'join' over two database objects.
You just have to play a little bit around with the 'KEY' and 'REC' terms inside the definition of your database objects.
Yup, this feature allows for a whole buch of new ideas and proposals ;-).
5. Question on destination for emails ===================================== in 'RIPE DB' context ====================
I don't know if the database working group is the right address to report everthing in context RIPE database.
Should one better report only for i.e. problems with implementation and porting the software directly to the database maintainer at RIPE NCC <ripe-dbm@ripe.net> ?
and on the other side
Should one better report only i.e. issues about the database concepts directly to the database working group ?
Is there somewhat like a rule for this ?
<ripe-dbm@ripe.net> is the RIPE mail box for bug reports & questions I think that the database lists need some reorganizing (may be good topic for next RIPE meeting!): db-wg - database working group list for design & working group discussions rr-impl - not very much used db-beta - not very much used But I leave this subject for the working group to decide ... David K. ---
participants (2)
-
davidk＠ISI.EDU -
Markus Bock