Back Midas Rome Roody Rootana
  Midas DAQ System  Not logged in ELOG logo
Entry  13 Apr 2017, Andreas Suter, Bug Report, stop form odbedit broken 
    Reply  13 Apr 2017, Andreas Suter, Bug Report, stop form odbedit broken 
       Reply  15 Apr 2017, Konstantin Olchanski, Bug Report, stop form odbedit broken 
          Reply  15 Apr 2017, Konstantin Olchanski, Bug Report, MAX_STRING_LENGTH, stop form odbedit broken 
             Reply  15 Apr 2017, Konstantin Olchanski, Bug Report, MAX_STRING_LENGTH, stop form odbedit broken 
                Reply  19 Apr 2017, Stefan Ritt, Bug Report, MAX_STRING_LENGTH, stop form odbedit broken 
                   Reply  22 Apr 2017, Konstantin Olchanski, Bug Report, MAX_STRING_LENGTH, stop form odbedit broken 
                      Reply  06 Jun 2017, Konstantin Olchanski, Bug Report, MAX_STRING_LENGTH, stop form odbedit broken 
             Reply  19 Apr 2017, Stefan Ritt, Bug Report, MAX_STRING_LENGTH, stop form odbedit broken 
                Reply  22 Apr 2017, Konstantin Olchanski, Bug Report, MAX_STRING_LENGTH, stop form odbedit broken 
             Reply  02 May 2017, Konstantin Olchanski, Bug Report, mhttpd inline-editor and web MAX_STRING_LENGTH, stop form odbedit broken 
       Reply  24 Apr 2017, Stefan Ritt, Bug Report, stop form odbedit broken 
    Reply  15 Apr 2017, Konstantin Olchanski, Bug Report, where to report bugs, stop form odbedit broken 
    Reply  15 Apr 2017, Konstantin Olchanski, Bug Report, stop form odbedit broken 
Message ID: 1274     Entry time: 19 Apr 2017     In reply to: 1271     Reply to this: 1277
Author: Stefan Ritt 
Topic: Bug Report 
Subject: MAX_STRING_LENGTH, stop form odbedit broken 
ODB name lengths (the name of a key) are limited to 256 characters, the length of strings in the ODB should NOT be limited. At some point we wanted to have complete web pages inside the ODB, 
which for sure are longer than 256 characters. While this was the idea, I see now that db_paste & co. is hopelessly broken. To fix it, everything should be changed to std::string which is in my opinion 
the only 'clean' solution. That would also remove the cumbersome strlcpy and strlcat.

But looking at odb.c, replacing everything with std::string would probably take a brave programmer a couple of weeks. Not sure if we should dive into that adventure right now. The quick fix would be:

a) The strings "CSS File", "Sqlite dir" etc. reported below get reduced to 256 characters (MAX_STRING_LENGTH). The value of 256 characters came from the file system limitation in linux (some many 
years ago), where a full path of a file could not exceed 256 characters. Not sure if this limit is still valid today, but having all file names in the ODB limited to 256 characters is maybe not a bad idea 
anyhow (who wants to type in file names with more than 256 characters ???).

b) Change the max string length in db_paste to 1024 to cover the few exceptions above.


If we go with a), KO has to change his ODB file names, in case of b) I can do the change.

So what is your opinion?

Best regards,
Stefan

> > 
> > I shall check on the use of MAX_STRING_LENGTH at least in odb itself...
> > 
> 
> Ok, I looked at the use of MAX_STRING_LENGTH in ODB (odb.c):
> 
> a) it is not used in any critical places for the database itself, so it is not a limit on maximum length of TID_STRING data. good.
> b) it is used in the code for saving/loading odb from .odb files (old format), not sure how it works against overlong strings, but probably 
> truncates/corrupts/crashes.
> c) it is used in the code for saving odb to odb.xml files. Overlong strings are truncated (I added a message about it).
> d) code for loading/saving to json files handles overlong strings okey.
> e) odbedit "ls" truncates overlong strings, mhttpd has some oddities against overlong strings.
> f) db_sprintf() truncates string text to MAX_STRING_LENGTH to avoid output buffer overflow (should use db_snprintf() instead).
> 
> Conclusion, overlong strings should be okey, but do not use the old .odb and .xml save files. (mlogger saves odb to output .mid file in xml 
> format, we should switch it to use json format).
> 
> > > CSS File = STRING : [1024] mhttpd.css
> > > Sqlite dir = STRING : [1024]
> > > History dir = STRING : [1024]
> > > Sound = STRING : [1000] alarm.mp3
> > > are exceeding the MAX_STRING_LENGTH 256 (defined in msystem.h)
> 
> So these should not cause any corruption or problem unless actual content length exceeds 255 bytes,
> even then they are okey if odb is only saved and loaded into json files.
> 
> > > 1) I get the error message that some strings are too long (exceeding
> > > MAX_STRING_LENGTH). Unfortunately the underlying routine doesn't tell which ODB
> > > variables this is.
> 
> this is in db_check_record(), where it compares odb content with user-supplied data descriptions (there is no system-supplied
> data descriptions with strings longer than MAX_STRING_LENGTH).
> 
> so I think what happened is you created a data structure with overlong strings, passed it to db_paste() or something,
> db_check_record() complained about it, and db_paste() corrupted memory.
> 
> > > 
> > > 2) After this reload, essentially nothing is working anymore. Any client I tried to start just crashed.
> > > 
> 
> Somebody corrupted some shared memory, most likely it was db_paste() corrupted odb shared memory.
> 
> K.O.
ELOG V3.1.4-2e1708b5