<Online> in configuration file is MIDAS specific thing. Non MIDAS users don't need it.
And it can be used when MIDAS is linked.

So it is better that <Online> appears only when MIDAS is linked.

At first there must be several opinions and discussions. Following is just my opinion.
Probably asking opinion some users before changing ROME is better.

There are some properties for histograms and databases.

* database name
* database type and connection
* field database name and path
* histogram properties

I assume that definition file is shared by several persons in experimental group and configuration files 
are created for each users at the first execution. If you assume that we should share also configuration 
file, story written below is different.

My basic concept is that
 *Parameters which is complicated or will be shared among several users should be in definition file.
 *Parameters which can be often changed at run time should be in configuration file.

The good point to specify values in definition file is that they are flexible, because we can use 
ROMEString::SetFormatted.
It is outstanding point of ROME that we can design framework very flexibly due to builder.

Other point is that parameters in definition file will be written by programer while those in 
configuration file can be written by non-experts. It is better that the application can work with few 
changes of configuration file as possible. In my feeling, to specify database paths and histogram 
parameters in configuration file correctly is too tough for normal users.

In my opinion, they are separated like,
Definition file:
* field database name and path
* histogram properties

Configuration file:
* database type and connection
* (histogram properties)

-- about database path
Normally, users share the same database structure. So I think it should be in definition file. I don't think 
paths are changed so often.

-- about database type and connection
At first type and connection can be written in one line like,
   xml://directory_of_database/database_name
   mysql://[user_name]:[passwd]@[hostname]:[port]/database_name
   none://
      (  odb://[host_name]/[experiment] or odb://experiment@host_name  )
(parameters in [] are optional)

These parameters will be changed user by user, because they can have xml files in different place, and 
they can use replicated SQL servers.
Password can be written in connection. So it should be in configuration file.

-- about histogram parameters
We can specify them with calculation if they are in definition file. I think it should be in definition file.
And to allow to overwrite in configuration file is also attractive. In my opinion overwriting with normal 
type like Float_t,Int_t... is enough.

---------
In result, my opinion is like following.
Database name, database paths and histogram parameters are in definition file.
In default configuration file, config class writes database entries appeared in definition file.
Config class writes histogram parameters entries with empty values if the histogram is <Configurable>. 
Only when 
some value is written there, analyzer overwrite these parameters.

As I wrote above I think normal number like following are enough as values of histogram parameters in 
configuration file.
   <HistXmax>1000</HistXmax>
I don't think following is necessary.
   </HistXmax>/GSP/parameter</HistXmax>


Definition file
    <Folder>
        <FolderName>foldername</FolderName>
        <DataBaseAccess>true</DataBaseAccess>
        <Field>
          <FieldName>fieldname</FieldName>
          <FieldType>Float_t</FieldType>
          <DataBaseName>first_database</DataBaseName>
          <DataBasePath>"/Path/to/record"</DataBasePath>
        </Field>
       <Field>
          <FieldName>fieldname2</FieldName>
          <FieldType>Float_t</FieldType>
          <DataBaseName>second_database</DataBaseName>
          <DataBasePath>"/Path/to/record2"</DataBasePath>
        </Field>
   </Folder>
...
     <Histogram>
        <HistName>histname</HistName>
        <Configurable>true</Configurable>
        <HistType>TH1F</HistType>
        <HistTitle>ADC0 Bank</HistTitle>
        <HistFolderTitle>ADC</HistFolderTitle>
        <HistArraySize>gAnalyzer->GetGSP()->GetNumberOfADC()</HistArraySize>
        <HistXNbins>4094</HistXNbins>
        <HistXmin>1</HistXmin>
        <HistXmax>4095</HistXmax>
      </Histogram>

Default configuration file
    <DataBases>
      <DataBase>
        <Name>first_database</Name>
        <Connection>none://</Connection>
      </DataBase>
     <DataBase>
        <Name>second_database</Name>
        <Connection>none://</Connection>
      </DataBase>
    </DataBases> 
...
     <Histogram>
        <HistName>histname</HistName>
        <HistFolderTitle></HistFolderTitle>
        <HistArraySize></HistArraySize>
        <HistXNbins></HistXNbins>
        <HistXmin></HistXmin>
        <HistXmax></HistXmax>
      </Histogram>

  There is no check if the pointer returned by GetDataBase,GetActiveDAQ is valid. This can result 
segmentation fault.
  For instance, if one specifies DAQ system as "none", or one does not specify DataBaseName, rome 
stops without error message.

It is better to modify this behavior.


* One way is to stop application like.
ROMEDAQSystem* GetActiveDAQ() {
   if(!fActiveDAQ){
      gAnalyzer->Println("some error message");
      fApplication->Terminate(1);
   }
   return fActiveDAQ;
};

*Other way is to use a dummy DAQSystem and DataBase which does nothing when user does not 
specify DAQ or DataBase.  When dummy is used, warning message must be shown, because it is 
dangerous that users do not realize they are using dummy.

>   There is no check if the pointer returned by GetDataBase,GetActiveDAQ is valid. This can result 
> segmentation fault.
>   For instance, if one specifies DAQ system as "none", or one does not specify DataBaseName, rome 
> stops without error message.
> 
> It is better to modify this behavior.
> 
> 
> * One way is to stop application like.
> ROMEDAQSystem* GetActiveDAQ() {
>    if(!fActiveDAQ){
>       gAnalyzer->Println("some error message");
>       fApplication->Terminate(1);
>    }
>    return fActiveDAQ;
> };
> 
> *Other way is to use a dummy DAQSystem and DataBase which does nothing when user does not 
> specify DAQ or DataBase.  When dummy is used, warning message must be shown, because it is 
> dangerous that users do not realize they are using dummy.


done

I am Daniele Barbareschi, I am a Corrado Gatto's student 
I work to interfacing between ALIROOT and MYSQL for CONSTANT CALIBRATION of 
subdetector.

There's a bug in compiling...

NAME of file "ROMEXML.h,ROMEXML.cpp" in the ultimate version ROME is lowercase.
The gmake program return error.

Maybe ROMESQL too.
See you those files and similar.

D.B.

Followings are warning message from compiler, please check if they are correct.
When I changed line-490 like,
     490       while ((obj = iterFolders->Next())) {
warning disappeared.

ROMEAnalyzer.cpp:490: warning: suggest parentheses around assignment used as truth value
//get folder names
     485       TObject *obj;
     486       TObjArray *names = new TObjArray(100);
     487 
     488       TCollection *folders = folder->GetListOfFolders();
     489       TIterator *iterFolders = folders->MakeIterator();
     490       while (obj = iterFolders->Next()) {
     491          ROMEString str;
     492          str.SetFormatted("%s",obj->GetName());
     493          names->Add(new TObjString(str.Data()));
     494       }
     495 


ROMEString.cpp:74: warning: `int numberOfDigits' might be used uninitialized in this function
ROMEXMLDataBase.cpp:83: warning: `ROMEXML*xml' might be used uninitialized in this function

In the current cvs version of ROME several things have changed that require 
changes in the user code.

- Most of the histogram parameters have been moved form the definition file to 
the configuration file. Please consult the users guide.

- Database paths have also been moved to the configuration file. Furthermore 
the format has changed for string replacements. It looks now like this :

<DataBasePath>"/AAA/BBB[0,%s]",/GSP/NumberOfBBB</DataBasePath>
For folders write : '/FolderName/FieldName'.

The format statement must always be %s!

- The tag 'InputDataFormat' has changed to 'DAQSystem' in the configuration 
file.

- All bank access methods have to be accessed by gAnalyzer->GetMidas() not only 
gAnalyzer.

Matthias

I recommended to my friend trying ROME, but he couldn't download it, because he didn't set CVS_RSH.
There is no instruction for setting CVS_RSH in ROME download page.

You had better to write about it.

> I recommended to my friend trying ROME, but he couldn't download it, because he didn't set CVS_RSH.
> There is no instruction for setting CVS_RSH in ROME download page.
> 
> You had better to write about it.

Yes, thanks

I am going to make a new ROME project to analyze MC data. Input file will be ROOT which  is created by 
h2root.
And my friend who is working on dark matter search at Columbia university in NY is going to use 
ROME. His file format is special binary format.

If I understand correctly, currently ROME can read midas and ROOT format.
Midas file names are always assumed as "run99999.mid". (99999 is run number)
ROOT files need to have structure defined in definition xml file. 
I guess this is not enough to analyze special binary format or ROOT files created by h2root.

What I propose is to allow users to have own daq class. It might be something like

<DAQs>
  <DAQ>
    <DAQName>daqname</DAQName>
  </DAQ>
</DAQs>

Then builder creates [shortcut][daqname].h, [shortcut][daqname].cpp.
[shortcut][daqname] class inherits ROMEDAQSystem class, and users must fill GetTimeStamp(), 
Initialize(), Connect(), ReadEvent(int event), Disconnect(), Termination()

When "InputDataFormat" in configuration xml file is "daqname" analyzer uses this class.


 

It does not seem that fHistoNameTitleOverWritten,fHistoNameFolderTitleOverWritten and so on are not 
initialized.

I'm afraid it is not safe
,because when ***OverWritten is "true" parameters are not supposed to be initialized.

In my case, framework tries to make a histogram whose number of bins is 1701143923.

This is my definition of the histogram. Do I need to add something?
<Histogram>
  <HistName>hXeQsum</HistName>
  <HistTitle>Qsum</HistTitle>
  <HistFolderName>Xenon</HistFolderName>
  <HistFolderTitle>Xenon</HistFolderTitle>
  <HistType>TH1F</HistType>
  <HistXNbins>4096</HistXNbins>
  <HistXmin>0</HistXmin>
  <HistXmax>5e5</HistXmax>
</Histogram>

After changing xml library from libxml2 to mxml, examples (sample and MEG) can not be compiled.

My project does not work too. What is necessary to make it work?

In ROMEEventLoop.cpp::ReadEvent
this->ResetFolders is called before fActiveDAQ->ReadEvent, so data in event headers are not updated 
properly.

> In ROMEEventLoop.cpp::ReadEvent
> this->ResetFolders is called before fActiveDAQ->ReadEvent, so data in event headers are not updated 
> properly.

done

> > In ROMEEventLoop.cpp::ReadEvent
> > this->ResetFolders is called before fActiveDAQ->ReadEvent, so data in event headers are not updated 
> > properly.
> 
> done
I wonder if this is the reason why Jan couldn't read scaler events.
Event ID was shifted by two events, so when scaler event comes, ROME can always recognize it as trigger events.

> > > In ROMEEventLoop.cpp::ReadEvent
> > > this->ResetFolders is called before fActiveDAQ->ReadEvent, so data in event headers are not updated 
> > > properly.
> > 
> > done
> I wonder if this is the reason why Jan couldn't read scaler events.
> Event ID was shifted by two events, so when scaler event comes, ROME can always recognize it as trigger events.

No the reason for Jan's problem was that rome never requested scaler events from the frontend. This is fixed now.

I realized that some unix system distribution does not provide xmlwriter.h in standard libxml2-devel 
package.

For instance, scientific linux (also PSI linux) does not.
I installed it from source.

Macintosh has xmlreader.h by default, and does not xmlwriter.h.
I installed it by using FINK.
(FINK is a unix package porter like CYGWIN.)

Someone may ask about it someday. 
A problem is that compiler will find xmlwriter.h even if the system does not have it, because ROME has.
This may make it hard that users find out the problem.


How to install libxml2.
----  UNIX OS ----
In most cases, to use standard way of the system is good. But some system does not provide xmlwriter.h. In this case you may need to 
install it from source.
1. download source from http://xmlsoft.org/.
2. decompress it and go to created directory.
3. % ./configure [options as you like];
4. % make;
5. become super user, if you are going to install it in system.
6. % make install

---- MacOSX ----
By default, Macintosh does not have xmlwriter.h. You can follow the same way as other UNIX OS. And other way is to use FINK. (I hope 
many users already installed it.)
1. install FINK (http://fink.sourceforge.net)
2. become super user.
3. apt-get update;
4. apt-get install libxml2 libxml2-shlibs libxml2-bin
If you are goint to use also MySQL.
5. apt-get install mysql12-dev mysql12-shlibs
(* package name can be changed in future, you may find it with "apt-cache search libxml2" or "apt-cache search mysql")

> I realized that some unix system distribution does not provide xmlwriter.h in standard libxml2-devel 
> package.
> 
> For instance, scientific linux (also PSI linux) does not.
> I installed it from source.
> 
> Macintosh has xmlreader.h by default, and does not xmlwriter.h.
> I installed it by using FINK.
> (FINK is a unix package porter like CYGWIN.)
> 
> Someone may ask about it someday. 
> A problem is that compiler will find xmlwriter.h even if the system does not have it, because ROME has.
> This may make it hard that users find out the problem.
> 
> 
> How to install libxml2.
> ----  UNIX OS ----
> In most cases, to use standard way of the system is good. But some system does not provide xmlwriter.h. In this case you may need to 
> install it from source.
> 1. download source from <a href="http://xmlsoft.org/">http://xmlsoft.org/</a>.
> 2. decompress it and go to created directory.
> 3. % ./configure [options as you like];
> 4. % make;
> 5. become super user, if you are going to install it in system.
> 6. % make install
> 
> ---- MacOSX ----
> By default, Macintosh does not have xmlwriter.h. You can follow the same way as other UNIX OS. And other way is to use FINK. (I hope 
> many users already installed it.)
> 1. install FINK (<a href="http://fink.sourceforge.net">http://fink.sourceforge.net</a>)
> 2. become super user.
> 3. apt-get update;
> 4. apt-get install libxml2 libxml2-shlibs libxml2-bin
> If you are goint to use also MySQL.
> 5. apt-get install mysql12-dev mysql12-shlibs
> (* package name can be changed in future, you may find it with "apt-cache search libxml2" or "apt-cache search mysql")

I have replaced libxml with mxml (written by Stefan) which is now included in the rome distribution.
So there's no install problem on linux anymore.

In online mode, ROME often stops before finishing to analyze all events of runs. 
 When it happens, gAnalyzer->GetCurrentEventNumber() starts not from 1 but from analyzed event 
number at previous run. And unanalyzed events at the previous run are mixed with the new run.

In mana.c of MIDAS, I found a part to wait until all events in buffers are analyzed when analyzer gets 
TR_STOP signal, while ROME does not have.

I personaly don't need that all events are analyzed, but it is really problem that data of two runs are 
mixed.

One way is to take similar way as midas analyzer, and the another way is just discard unanalyzed 
events.

> In online mode, ROME often stops before finishing to analyze all events of runs. 
>  When it happens, gAnalyzer->GetCurrentEventNumber() starts not from 1 but from analyzed event 
> number at previous run. And unanalyzed events at the previous run are mixed with the new run.
> 
> In mana.c of MIDAS, I found a part to wait until all events in buffers are analyzed when analyzer gets 
> TR_STOP signal, while ROME does not have.
> 
> I personaly don't need that all events are analyzed, but it is really problem that data of two runs are 
> mixed.
> 
> One way is to take similar way as midas analyzer, and the another way is just discard unanalyzed 
> events.

done

We usually use ROME framework without specifying run number in romeConfig.xml and command line 
in online mode.

Then it warns that "Error in <TArrayI::At>: index 0 out of bounds" when gROME-
>GetRunNumberStringAt(runNumberString,0) is called in ROMEEventLoop::Initialize.

> We usually use ROME framework without specifying run number in romeConfig.xml and command line 
> in online mode.
> 
> Then it warns that "Error in <TArrayI::At>: index 0 out of bounds" when gROME-
> >GetRunNumberStringAt(runNumberString,0) is called in ROMEEventLoop::Initialize.

done

About <EventHeader> in xml definition file.

In description page in ROME homepage, It is wirtten that <EventId> specifies the field.
But It seems that 'd' shoud be capital.

> About <EventHeader> in xml definition file.
> 
> In description page in ROME homepage, It is wirtten that <EventId> specifies the field.
> But It seems that 'd' shoud be capital.

done

I will summarize some points which g++ warned.  
  
(1) I fixed following points.  
*Made destructor of ROMEConfig and ROMEDataBase virtual.  
  
*Removed or comment out unused variables.  
    int i                            in ROMEPATH::~ROMEPath()  
    char* defaultRow                 in ROMESQL::ReadField  
    char* c                          in ROMEString::SetFormatted  
     int istart,iend,iValue,jValue   in ROMEXMLDataBase::Read  
    int istep,iValue,jValue          in ROMEXMLDataBase::Write  
    ROMEStrArray *array              in ROMEXMLDataBase::Write  
  
(2) Following points are not fixed yet.  
* There can be unused "int i" in several codes which builder generates. (ex. XXAnalyzer::InitMidasBanks  
)  
  This is not a problem, but a little bit annoying. If it is very easy, please fix it. 
  
* Please make sure this is not a problem.  
   in ROMEAnalyzer:: ResponseFunction  
      while (obj = iterFolders->Next()) {  
            ROMEString str;  
            str.SetFormatted("%s",obj->GetName());  
            names->Add(new TObjString(str.Data()));  
      }  
  
*ROMEString.cpp:74: warning: `int numberOfDigits' might be used uninitialized in this function  
*ROMEXMLDataBase.cpp:80: warning: `ROMEXML*xml' might be  used uninitialized in this function  
  
  
(3) This is not warning from g++, but request from me.  
  If it is very easy, please make ROMEString::SetFormatted,AppendFormatted to check that number of  
arguments is correct.  

I added new make target "build" in ARGUS. It may be useful also for ROME users.

in WriteMakefile(char* xmlFile)
.
.
.
  int pdnameend = 0;
   int pbnamestart = 0;
   ROMEString xmlfile = xmlFile;
   while((pdnameend = xmlfile.Index("/",1,pbnamestart,TString::kExact))!=-1)
      pbnamestart = pdnameend+1;
   ROMEString xmlbasename = xmlfile(pbnamestart,xmlfile.Length());
   buffer.AppendFormatted("build::\n");
   buffer.AppendFormatted("     $(ROMESYS)/bin/romebuilder.exe -i %s -o .",xmlbasename.Data());
   if (makeOutput)
      buffer.AppendFormatted(" -v");
   if(noLink)
      buffer.AppendFormatted(" -nl");
   if(midas)
      buffer.AppendFormatted(" -midas");
   if(!sql)
      buffer.AppendFormatted(" -nosql");
   buffer.AppendFormatted("\n");
.
.
.

Recently, default values of steering parameters in romeConfig.xml are always 0. They are not as same 
as values in SPFieldInitialization of definition XML file.

We sometimes want to use trees in ROME.

One way is to read it by TSocket connection. But treees are larger than histograms in general, and it 
takes time to download. 
 And I need to make shared library when I changed the structure of folders.

So, I want to propose is to prepare Cint mode with continueing analysis in background.

Currently, when user press "i", ROME stops analysis and goes into cint mode. This is good for offline 
mode, but this is not good for online mode. So I propose that when user press "I", ROME goes cint 
mode, and continue analysis in background.

Then we can see Trees at run time.

In addition, If you are going to implement this feature, please prepare short way to call trees in cint mode.
If I understand correctly, the way to call a tree is like,
gAnalyzer->GetTreeObjectAt(0)->GetTree()->Draw("");
but it is too long to use in interactively and we need to remeber the number of trees.
gAnalyzer->GetTree("trigger")->Draw();  is better.

About warnings,errors and informations output in online mode.

Probably we lauch ROME from web interface of MIDAS. And we don't see console of ROME.
But we need to hear what ROME is saying.

Normally midas analyzer and other applications use cm_msg function. This function shows messages 
proper place like web interface or ODBEdit windows and so on.

I do not have much knowledge about it. Stefan may know better way.

> About warnings,errors and informations output in online mode.
> 
> Probably we lauch ROME from web interface of MIDAS. And we don't see console of ROME.
> But we need to hear what ROME is saying.
> 
> Normally midas analyzer and other applications use cm_msg function. This function shows messages 
> proper place like web interface or ODBEdit windows and so on.
> 
> I do not have much knowledge about it. Stefan may know better way.

I implemented the cm_msg calls.

> I implemented the cm_msg calls.

You should implement a central "message dispatcher". So all errors etc. should go to someting linke
rome_msg(). If compiled with midas online support, rome_msg() should call cm_msg(). If rome is compiled
without midas libraries, a printf() there would be enough. If someone uses rome in a different
environment, only can function needs to be changed to reroute all messages to somewhere else.

> > I implemented the cm_msg calls.
> 
> You should implement a central "message dispatcher". So all errors etc. should go to someting linke
> rome_msg(). If compiled with midas online support, rome_msg() should call cm_msg(). If rome is compiled
> without midas libraries, a printf() there would be enough. If someone uses rome in a different
> environment, only can function needs to be changed to reroute all messages to somewhere else.

Of course, that's the way I implemented that.