This is maybe a bug report that happens reading long lines in a text database.
I am currently handling large data base files for linear fit analysis, basically I have to handle a 828 x 828 
matrix. Writing in the text db 828 lines containing each 828 comma separated values with a single 
new-line at the end does not work, maybe the read buffer is too large. 
So I propose the following modification to ROMETextDataBase.cpp:
Suppose you have a field named "coefficients" which is declared "array", say of four numbers,  in the 
xml file. In the text data base it appears as:

<coefficients>
one,two,three,four
one,two,three,four
one,two,three,four
...
</coefficients>

One should be able to write the same code as follows:

<coefficients>
one,
two,
three,
four                  <----- note there is NO COMMA here!
one,
two,
three,
four
one,two,three,four
...
</coefficients>

or

<coefficients>
one,two,
three,four
one,two,three,
four
one,
two,three,four
...
</coefficients>

In this way I can write a single column of (comma separated) values. Can somebody (Ryu?) take care of 
this (I hope simple) modification?
Thanks.

 

> This is maybe a bug report that happens reading long lines in a text database.

This does not only affect ROME, but also the MXML library which has to read the long lines. I haven't carefully
checked, but I presume that MXML can handle long lines. So it might maybe be better to keep the long lines in a
text database, and modify ROME such that there is no buffer overflow. This maybe makes it easier to read the
database and to import it in programs like a spreadsheet.

#! /bin/bash

BASEDIR=/data/0/PostRunQA		# Base directory for output files
ROMEDIR=${BASEDIR}/dance-rome		# Location of danceanalyzer

NRUN=7581
ROMECFG="romeConfig_Continuous.xml";

# Set PATH and LD_LIBRARY_PATH for correct versions of ...
# This also sets up the *SYS variables
source /opt/wnr/new/bin/newVersion --new root rome wnr

cd ${ROMEDIR};

# Check that ROOT works correctly
echo " Running ROOT batch mode test ";
root -b -q ../jmodScript.C

# Check that ROME works correctly
echo " Running danceanalyzer batch mode test ";
./danceanalyzer.exe -b -ns -q -i ${ROMECFG} -r ${NRUN}

 Running ROOT batch mode test
  *******************************************
  *                                         *
  *        W E L C O M E  to  R O O T       *
  *                                         *
  *   Version   5.10/00      1 March 2006   *
  *                                         *
  *  You are welcome to visit our Web site  *
  *          http://root.cern.ch            *
  *                                         *
  *******************************************
                                                                                                                                  
Compiled on 3 April 2006 for linux with thread support.
                                                                                                                                  
CINT/ROOT C/C++ Interpreter version 5.16.8, February 9, 2006
Type ? for help. Commands must be C++ statements.
Enclose multiple statements between { }.
 Executing rootlogon.C (31-MAY-2006)...  Finished setup
                                                                                                                                  
Processing ../jmodScript.C...
hello world
This is the end of ROOT -- Goodbye

 ...
 Running danceanalyzer batch mode test
 Executing rootlogon.C (31-MAY-2006)...  Finished setup
reading configuration from romeConfig_Continuous.xml
[bredeweg@enlil PostRunQA]$

 Running danceanalyzer batch mode test
 Executing rootlogon.C (31-MAY-2006)...  Finished setup
                                                                                                                                  
 *** Break *** segmentation violation
 Generating stack trace...
 0x081ec18b in main + 0x21b from ./danceanalyzer.exe
 0x0734678a in __libc_start_main + 0xda from /lib/tls/libc.so.6
 0x080dcd31 in TFile::TFile(char const*, char const*, char const*, int) + 0x51 from ./danceanalyzer.exe
/bin/bash: line 22: 28096 Aborted                 ./danceanalyzer.exe -b -ns -q -i ${ROMECFG} -r ${NRUN}

(sorry for top-reply- I am trying to avoid elcode garbling my answer)

In the past, I have seen problems like yours, there are at least two issues:

1) "non interactive" ROOT running from cron or some other incomplete user
environment (no DISPLAY, no tty, no user)- a hello world script runs, but large
app does not, because some silly class somewhere requires an X11 connection. I
once had to fake it by running Xvnc and telling the cron jobs to use it as a
DISPLAY. I know ROOT now have a "dummy" graphics module exactly for this
purpose. I do not know how to activate it from the Rome environement, but I am
sure it is documented somewhere on the ROOT web site.

2) some programs coming from the MIDAS family assume existance of a keyboard and
display. Sometimes they can be faked by using "program < /dev/null", sometimes
not, I do not know what Rome does.

K.O.


[quote="Todd Bredeweg"]I would like to use a bash script run from cron to
analyze new midas event files automatically. Below is an abbreviated example
script to test the way ROOT and ROME handle true batch mode

[CODE]
#! /bin/bash

BASEDIR=/data/0/PostRunQA		# Base directory for output files
ROMEDIR=${BASEDIR}/dance-rome		# Location of danceanalyzer

NRUN=7581
ROMECFG="romeConfig_Continuous.xml";

# Set PATH and LD_LIBRARY_PATH for correct versions of ...
# This also sets up the *SYS variables
source /opt/wnr/new/bin/newVersion --new root rome wnr

cd ${ROMEDIR};

# Check that ROOT works correctly
echo " Running ROOT batch mode test ";
root -b -q ../jmodScript.C

# Check that ROME works correctly
echo " Running danceanalyzer batch mode test ";
./danceanalyzer.exe -b -ns -q -i ${ROMECFG} -r ${NRUN}
[/CODE]

The results for the ROOT section are the same when the script is run from the
command line (full interactive environment) or when using at or crontab
(non-interactive).

[CODE]
 Running ROOT batch mode test
  *******************************************
  *                                         *
  *        W E L C O M E  to  R O O T       *
  *                                         *
  *   Version   5.10/00      1 March 2006   *
  *                                         *
  *  You are welcome to visit our Web site  *
  *          http://root.cern.ch            *
  *                                         *
  *******************************************
                                                                               
                                                  
Compiled on 3 April 2006 for linux with thread support.
                                                                               
                                                  
CINT/ROOT C/C++ Interpreter version 5.16.8, February 9, 2006
Type ? for help. Commands must be C++ statements.
Enclose multiple statements between { }.
 Executing rootlogon.C (31-MAY-2006)...  Finished setup
                                                                               
                                                  
Processing ../jmodScript.C...
hello world
This is the end of ROOT -- Goodbye
[/CODE]

jmodScript.C is merely a classic "hello world" script. The ROME test, on the
other hand, returns quite different results for the two cases. When run from the
command line it works as expected

[CODE]
 ...
 Running danceanalyzer batch mode test
 Executing rootlogon.C (31-MAY-2006)...  Finished setup
reading configuration from romeConfig_Continuous.xml
[bredeweg@enlil PostRunQA]$
[/CODE]

but when run from at or crontab we get 

[CODE] Running danceanalyzer batch mode test
 Executing rootlogon.C (31-MAY-2006)...  Finished setup
                                                                               
                                                  
 *** Break *** segmentation violation
 Generating stack trace...
 0x081ec18b in main + 0x21b from ./danceanalyzer.exe
 0x0734678a in __libc_start_main + 0xda from /lib/tls/libc.so.6
 0x080dcd31 in TFile::TFile(char const*, char const*, char const*, int) + 0x51
from ./danceanalyzer.exe
/bin/bash: line 22: 28096 Aborted                 ./danceanalyzer.exe -b -ns -q
-i ${ROMECFG} -r ${NRUN}[/CODE]

One question that we have been unable to answer as yet is whether rome is
requiring a real TTY, which I do not believe is provided by cron.

We are using the following setup:
    Redhat Enterprise Linux v3
    ROOT v5.10.00
    ROME RELEASE_2_4_R1004

I can provide additional information if needed.

Thanks in advance.

tab[/quote]

I have made modification on batch mode and daemon mode at rev.1129, 1136 and 1137.
The difference of daemon and batch mode is written in ROME homepage
http://midas.psi.ch/rome/usersGuide.html#fwcommandline

You can disable all graphics with -ng option explicitly. And when ROME failed to open display, it goes into 
no graphics mode automatically.

Please try. And please report when you still have a problem.

> (sorry for top-reply- I am trying to avoid elcode garbling my answer)
> 
> In the past, I have seen problems like yours, there are at least two issues:
> 
> 1) "non interactive" ROOT running from cron or some other incomplete user
> environment (no DISPLAY, no tty, no user)- a hello world script runs, but large
> app does not, because some silly class somewhere requires an X11 connection. I
> once had to fake it by running Xvnc and telling the cron jobs to use it as a
> DISPLAY. I know ROOT now have a "dummy" graphics module exactly for this
> purpose. I do not know how to activate it from the Rome environement, but I am
> sure it is documented somewhere on the ROOT web site.
> 
> 2) some programs coming from the MIDAS family assume existance of a keyboard and
> display. Sometimes they can be faked by using "program < /dev/null", sometimes
> not, I do not know what Rome does.
> 
> K.O.
> 
> 
> [quote="Todd Bredeweg"]I would like to use a bash script run from cron to
> analyze new midas event files automatically. Below is an abbreviated example
> script to test the way ROOT and ROME handle true batch mode
> 
> [CODE]
> #! /bin/bash
> 
> BASEDIR=/data/0/PostRunQA		# Base directory for output files
> ROMEDIR=${BASEDIR}/dance-rome		# Location of danceanalyzer
> 
> NRUN=7581
> ROMECFG="romeConfig_Continuous.xml";
> 
> # Set PATH and LD_LIBRARY_PATH for correct versions of ...
> # This also sets up the *SYS variables
> source /opt/wnr/new/bin/newVersion --new root rome wnr
> 
> cd ${ROMEDIR};
> 
> # Check that ROOT works correctly
> echo " Running ROOT batch mode test ";
> root -b -q ../jmodScript.C
> 
> # Check that ROME works correctly
> echo " Running danceanalyzer batch mode test ";
> ./danceanalyzer.exe -b -ns -q -i ${ROMECFG} -r ${NRUN}
> [/CODE]
> 
> The results for the ROOT section are the same when the script is run from the
> command line (full interactive environment) or when using at or crontab
> (non-interactive).
> 
> [CODE]
>  Running ROOT batch mode test
>   *******************************************
>   *                                         *
>   *        W E L C O M E  to  R O O T       *
>   *                                         *
>   *   Version   5.10/00      1 March 2006   *
>   *                                         *
>   *  You are welcome to visit our Web site  *
>   *          http://root.cern.ch            *
>   *                                         *
>   *******************************************
>                                                                                
>                                                   
> Compiled on 3 April 2006 for linux with thread support.
>                                                                                
>                                                   
> CINT/ROOT C/C++ Interpreter version 5.16.8, February 9, 2006
> Type ? for help. Commands must be C++ statements.
> Enclose multiple statements between { }.
>  Executing rootlogon.C (31-MAY-2006)...  Finished setup
>                                                                                
>                                                   
> Processing ../jmodScript.C...
> hello world
> This is the end of ROOT -- Goodbye
> [/CODE]
> 
> jmodScript.C is merely a classic "hello world" script. The ROME test, on the
> other hand, returns quite different results for the two cases. When run from the
> command line it works as expected
> 
> [CODE]
>  ...
>  Running danceanalyzer batch mode test
>  Executing rootlogon.C (31-MAY-2006)...  Finished setup
> reading configuration from romeConfig_Continuous.xml
> [bredeweg@enlil PostRunQA]$
> [/CODE]
> 
> but when run from at or crontab we get 
> 
> [CODE] Running danceanalyzer batch mode test
>  Executing rootlogon.C (31-MAY-2006)...  Finished setup
>                                                                                
>                                                   
>  *** Break *** segmentation violation
>  Generating stack trace...
>  0x081ec18b in main + 0x21b from ./danceanalyzer.exe
>  0x0734678a in __libc_start_main + 0xda from /lib/tls/libc.so.6
>  0x080dcd31 in TFile::TFile(char const*, char const*, char const*, int) + 0x51
> from ./danceanalyzer.exe
> /bin/bash: line 22: 28096 Aborted                 ./danceanalyzer.exe -b -ns -q
> -i ${ROMECFG} -r ${NRUN}[/CODE]
> 
> One question that we have been unable to answer as yet is whether rome is
> requiring a real TTY, which I do not believe is provided by cron.
> 
> We are using the following setup:
>     Redhat Enterprise Linux v3
>     ROOT v5.10.00
>     ROME RELEASE_2_4_R1004
> 
> I can provide additional information if needed.
> 
> Thanks in advance.
> 
> tab[/quote]

In online mode, ROME often stops before finishing to analyze all events of runs. 
 When it happens, gAnalyzer->GetCurrentEventNumber() starts not from 1 but from analyzed event 
number at previous run. And unanalyzed events at the previous run are mixed with the new run.

In mana.c of MIDAS, I found a part to wait until all events in buffers are analyzed when analyzer gets 
TR_STOP signal, while ROME does not have.

I personaly don't need that all events are analyzed, but it is really problem that data of two runs are 
mixed.

One way is to take similar way as midas analyzer, and the another way is just discard unanalyzed 
events.

> In online mode, ROME often stops before finishing to analyze all events of runs. 
>  When it happens, gAnalyzer->GetCurrentEventNumber() starts not from 1 but from analyzed event 
> number at previous run. And unanalyzed events at the previous run are mixed with the new run.
> 
> In mana.c of MIDAS, I found a part to wait until all events in buffers are analyzed when analyzer gets 
> TR_STOP signal, while ROME does not have.
> 
> I personaly don't need that all events are analyzed, but it is really problem that data of two runs are 
> mixed.
> 
> One way is to take similar way as midas analyzer, and the another way is just discard unanalyzed 
> events.

done

I will summarize some points which g++ warned.  
  
(1) I fixed following points.  
*Made destructor of ROMEConfig and ROMEDataBase virtual.  
  
*Removed or comment out unused variables.  
    int i                            in ROMEPATH::~ROMEPath()  
    char* defaultRow                 in ROMESQL::ReadField  
    char* c                          in ROMEString::SetFormatted  
     int istart,iend,iValue,jValue   in ROMEXMLDataBase::Read  
    int istep,iValue,jValue          in ROMEXMLDataBase::Write  
    ROMEStrArray *array              in ROMEXMLDataBase::Write  
  
(2) Following points are not fixed yet.  
* There can be unused "int i" in several codes which builder generates. (ex. XXAnalyzer::InitMidasBanks  
)  
  This is not a problem, but a little bit annoying. If it is very easy, please fix it. 
  
* Please make sure this is not a problem.  
   in ROMEAnalyzer:: ResponseFunction  
      while (obj = iterFolders->Next()) {  
            ROMEString str;  
            str.SetFormatted("%s",obj->GetName());  
            names->Add(new TObjString(str.Data()));  
      }  
  
*ROMEString.cpp:74: warning: `int numberOfDigits' might be used uninitialized in this function  
*ROMEXMLDataBase.cpp:80: warning: `ROMEXML*xml' might be  used uninitialized in this function  
  
  
(3) This is not warning from g++, but request from me.  
  If it is very easy, please make ROMEString::SetFormatted,AppendFormatted to check that number of  
arguments is correct.  

I think that fUniqueID and fBits in TObject is not useful to store in TTree.
I found in ROOT users guide that we can disable that with
MyClass::Class()->IgnoreTObjectStreamer(); 
.

If you also think these are useless information, please implement it in ROME.



second topic.
This is just a matter of favor.
We sometimes use TTree interactively. At that time, I feel fRunNumber, fEventNumber, fTimeStamp, 
fSequentialNumber are long to type.
I prefer shorter without upper case letter.
If many people agree it, I would like to ask to change them run,event,timestamp,number like MIDAS analyzer.

> I think that fUniqueID and fBits in TObject is not useful to store in TTree.
> I found in ROOT users guide that we can disable that with
> MyClass::Class()->IgnoreTObjectStreamer(); 
> .
> 
> If you also think these are useless information, please implement it in ROME.
> 
> 
> 
> second topic.
> This is just a matter of favor.
> We sometimes use TTree interactively. At that time, I feel fRunNumber, fEventNumber, fTimeStamp, 
> fSequentialNumber are long to type.
> I prefer shorter without upper case letter.
> If many people agree it, I would like to ask to change them run,event,timestamp,number like MIDAS analyzer.

MyClass::Class()->IgnoreTObjectStreamer(); is implemented.

> This is just a matter of favor.
> We sometimes use TTree interactively. At that time, I feel fRunNumber, fEventNumber, fTimeStamp, 
> fSequentialNumber are long to type.
> I prefer shorter without upper case letter.
> If many people agree it, I would like to ask to change them run,event,timestamp,number like MIDAS analyzer.
changed to run, event, time and number.

Recently, default values of steering parameters in romeConfig.xml are always 0. They are not as same 
as values in SPFieldInitialization of definition XML file.

This is not so strong suggestion. I am just wondering.

Currently ROME include headers of midas, xml2 and mysql.
There are libraries for windows but no libraries for linux.

I am afraid that there can be inconsequence between headers and libraries in futrue updates of these 
libraries, if compiler reads headers from rome/include instead of correct one.

So, my suggestion is that move midas.h,libxml2 and mysql outside of header files search path for linux.

for instance
  Make new directory rome/include/win.
  Put midas.h,libxml2 and mysql into it.
  Add $ROMESYS/include/win into header files search path in case of Windows.


P.S.
If midas.h is already very different from midas.h in MIDAS. This may be difficult.

> This is not so strong suggestion. I am just wondering.
> 
> Currently ROME include headers of midas, xml2 and mysql.
> There are libraries for windows but no libraries for linux.
> 
> I am afraid that there can be inconsequence between headers and libraries in futrue updates of these 
> libraries, if compiler reads headers from rome/include instead of correct one.
> 
> So, my suggestion is that move midas.h,libxml2 and mysql outside of header files search path for linux.
> 
> for instance
>   Make new directory rome/include/win.
>   Put midas.h,libxml2 and mysql into it.
>   Add $ROMESYS/include/win into header files search path in case of Windows.
> 
> 
> P.S.
> If midas.h is already very different from midas.h in MIDAS. This may be difficult.

I looked into gcc, then I found the header file which is found first is supposed to be used.
In ROME's case $(shell xml2-config --cflags) and $(shell mysql_config --cflags) are located before -I$(ROMESYS)/include/, so in principle 
it is safe.

But about midas.h there still can be inconsequence, because -I$(ROMESYS)/include/ is located before -I$(MIDASSYS)/include/.
It this done on purpose to enable special features for ROME ?


---
One more thing. about libpaths.
If libpaths will be used only for specify the place where midas libraries were put. Followings can also work and more simple.
And I added midascflags, because to use same way as root,xml2 and mysql seems more beautiful. 
And I moved -DHAVE_SQL and -DHAVE_MIDAS to sqlcflags and midascflags.

rootlibs := $(shell root-config --libs)
rootglibs := $(shell root-config --glibs)
rootcflags := $(shell root-config --cflags)
rootthreadlibs := -lThread
xmllibs :=  $(shell xml2-config --libs)
xmlcflags :=  $(shell xml2-config --cflags)
sqllibs := $(shell mysql_config --libs)
sqlcflags := $(shell mysql_config --cflags) -DHAVE_SQL
midaslibs := -L$(MIDASSYS)/lib -lmidas
midascflags := -I$(MIDASSYS)/include -DHAVE_MIDAS
clibs :=-lpthread -lHtml $(SYSLIBS)
clibs += -lutil

Libraries := $(libpaths) $(rootlibs) $(rootglibs) $(rootthreadlibs) $(xmllibs) $(clibs) $(sqllibs) $(midaslibs)
Flags := $(lpuserflags) $(rootcflags) $(xmlcflags) $(sqlcflags) $(midascflags)
Includes := -I$(ROMESYS)/include/ -I. -Iinclude/ -Iinclude/tasks/ -Iinclude/framework/

When I built my application with romebuilder.
It complained like,
Error : File './/src/tasks/LPTXEGainCalib.cpp' has an invalid header !!!
Error : File './/src/tasks/LPTXETimeCalib.cpp' has an invalid header !!!
Error : File './/src/tasks/LPTUTTask.cpp' has an invalid header !!!

Headers looks like,
//// Author: Ryu Sawada
/////////////^@//////////////////////////////////////////////////////////////////
//                                                                            //

Thre first time to build Just after I removed ^@, builder stopped warning. 
But after that  ^@ appeared again and builder starts warning.

I have several task files, but builder adds ^@ always these three files.

My source files are here.
cvs -d :ext:pc4466.psi.ch:/usr/local/cvsroot co lpframework

When I made task description longer than about 80 characters. The description was not written in html 
file propery. It was shorten.

I guess following lines in ROMEBuilder.cpp are relating to this.


-----------
      pos = (char*)taskDescription[iTask].Data();
      lenTot = taskDescription[iTask].Length();
      while (pos-taskDescription[iTask].Data() < lenTot) {
         if (lenTot+(taskDescription[iTask].Data()-pos)<74) 
            i=TMath::Min(75,lenTot);
         else for (i=74;pos[i]!=32&&i>0;i--) {}
         if (i<=0)
            i=TMath::Min(75,lenTot);
         pos[i] = 0;
         buffer.AppendFormatted("// %-74.74s   \n",pos);
         pos = pos+i+1;
      }

> When I made task description longer than about 80 characters. The description was not written in html 
> file propery. It was shorten.
> 
> I guess following lines in ROMEBuilder.cpp are relating to this.
> 
> 
> -----------
>       pos = (char*)taskDescription[iTask].Data();
>       lenTot = taskDescription[iTask].Length();
>       while (pos-taskDescription[iTask].Data() < lenTot) {
>          if (lenTot+(taskDescription[iTask].Data()-pos)<74) 
>             i=TMath::Min(75,lenTot);
>          else for (i=74;pos[i]!=32&&i>0;i--) {}
>          if (i<=0)
>             i=TMath::Min(75,lenTot);
>          pos[i] = 0;
>          buffer.AppendFormatted("// %-74.74s   \n",pos);
>          pos = pos+i+1;
>       }

done. thanks.