> Yes your right.
> 
> I added that.

The compression level should not be "hard-wired" in the builder. Some people might consider not
using compression in favour of higher speed. So maybe an option in the XML File like

<Tree>
  <TreeName>Test</TreeName>
  <TreeTitle>Test Data</TreeTitle>
  <CompressionLevel>1</CompressionLevel>
  ...
</Tree>

> > Yes your right.
> > 
> > I added that.
> 
> The compression level should not be "hard-wired" in the builder. Some people might consider not
> using compression in favour of higher speed. So maybe an option in the XML File like
> 
> <Tree>
>   <TreeName>Test</TreeName>
>   <TreeTitle>Test Data</TreeTitle>
>   <CompressionLevel>1</CompressionLevel>
>   ...
> </Tree>

Ok, that's done.

> In my case, in online mode, I don't need to wirte TTree.
> But I want to fill it, because we use it to see data with using TSocket connection.
> 
> How do you think about filling TTree even if write flag is not toggled ?

I implemented a fill flag for the trees in the romeConfig file.

However, if your fill flag is true and your write flag false, then the tree will be 
filled in memory completely. Which means that, if the tree is large, a memory overflow 
occurs. Therefore you should only do this for circular trees.

in ROMEEventLoop::ReadEvent(), if  gROME->isOffline() and gROME->isMidas(),
ROME checks if bank size is greater than 0.

But It can happen that the bank size is 0 in normal file.
(Actually in my midas file bank size of "environment data" is 0 at this moment)

---------------------------------------------
 else if (gROME->isOffline()&&gROME->isMidas()) {
      // read event header
      EVENT_HEADER *pevent = (EVENT_HEADER*)gROME->GetMidasEvent();
      bool readError = false;
               .
               .
               .
         if (pevent->data_size <= 0) readError = true;
         else {
            n = read(fMidasFileHandle, pevent+1, pevent->data_size);
            if (n != (int) pevent->data_size) readError = true;
	    if ((int) ((BANK_HEADER*)(pevent+1))->data_size < 0) readError = true;    <-- I changed here
         }
      }
      // check input
      if (readError) {
         if (n > 0) cout << "Unexpected end of file\n";
         this->SetEndOfRun();
         return true;

> in ROMEEventLoop::ReadEvent(), if  gROME->isOffline() and gROME->isMidas(),
> ROME checks if bank size is greater than 0.
> 
> But It can happen that the bank size is 0 in normal file.
> (Actually in my midas file bank size of "environment data" is 0 at this moment)

Point may be that data format of "environment data" in my midas file is "FIXED".
According to MIDAS document, MIDAS analyzer cannnot work with this format too.
I guess it can happen that bank size seems negative when ROME reads FIXED data.

In Makefile xmlibs and sqllibs is defined like this. 
xmllibs := -lxml2 -lz
sqllibs := -lmysql
But this is not correct sometimes.
Actually sqllibs for my environment should be -L'/usr/lib/mysql' -lmysqlclient

If you use the same way as rootlibs. It may be better.

xmllibs := $(shell xml2-config --libs)
xmlcflags := $(shell xml2-config --cflags)
sqllibs := $(shell mysql_config --libs)
sqlcflags := $(shell mysql_config --cflags)

User may want to read database in inverse order.

To add new rule to specify wheather the order is normal or opposit may be useful.

For instance following path gives the latest version of pedestal in database.
"/LPPedestal(id=%d)/version;LPPedestal/version(-)",gAnalyzer->GetCurrentRunNumber()

Concerning SQL database. To sort opposite is easy. Just put keyword "DESC" after "ORDER BY" phrase.

I finished Macintosh porting.
I checked it in offline mode , but not yet online mode.

I needed to add byte swapping code.
For bank swapping, if HAVE_MIDAS is defined it uses bk_swap in midas.h, in the other case it uses 
ROMEEventLoop::bk_swap.
Byte swapping will be done if cpu is PowerPC (MacOSX or Linux on Macintosh). Data should be written 
on little-endian machine like Intel.

Important point is If there is banks whose type is not MIDAS, rome may not work on Macintosh.
I want to propose a new MidasBanks scheme of xml file to fix this problem. Structure of EventDefinition 
is according to EVENT_DEF in mana.c.

When rome reads Midas file(offline) or ODB(odb), if the event ID is listed in xml configuration file and 
"Disabled" is false and Format is MIDAS, rome reads banks listed for the event ID, in the other case rome skips the event and goes to next 
event.

Probably this way will help statistics handling in online mode for also Intel platform. Because at this 
moment, rome assumes "Equipment" in ODB is always "Trigger" and "Scaler". But of coarse there can be 
other equipments. (There is possibility that ODB does not have "Scaler" too)

<MidasBanks>
   <EventHeader>
      <Folder>Event</Folder>
      <EventId>ID</EventId>
      <TriggerMask>Mask</TriggerMask>
      <SerialNumber>Eventnumber</SerialNumber>
      <TimeStamp>Time</TimeStamp>
   </EventHeader>
   <EventDefinition>
      <EventName>Trigger</EventName>
      <EventId>1</EventId>
      <Format>MIDAS</Format>
      <Disabled>false</Disabled>
      <Banks>
         <Bank>
            <BankName>ADC0</BankName>
            <BankType>unsigned short</BankType>
         </Bank>
         <Bank>
            <BankName>TDC0</BankName>
            <BankType>unsigned short</BankType>
         </Bank>
      </Banks>
   </EventDefinition>
   <EventDefinition>
      <EventName>Scaler</EventName>
      <EventId>2</EventId>
         <Format>YBOS</Format>
         <Disabled>true</Disabled>
         <Banks>
            <Bank>
               <BankName>SCLR</BankName>
               <BankType>unsigned long</BankType>
            <Bank>
         </Banks>	
      </EventDefinition>
      <EventDefinition>
         <EventName>HV</EventName>
         <EventId>3</EventId>
         <Format>FIXED</Format>
         <Disabled>true</Disabled>
      </EventDefinition>
</MidasBanks>

P.S.
If there is no plan to support YBOS and FIXED format. <Format> is not needed definitly.

I was asked to put change log of my rome application(LPframewok). I think the best place to put it is the 
html file which builder created.  But of course to modify it is not good, because it will be overwritten by 
builder again.

So, I want to propose to allow user to add links in the html file with the way like followings.

(i) User defines links.
------   in LPframewok.xml   ------
<DocumentLinks>
	<DocumentLink>
		<Href>html/changelog.html</Href>
		<Text>Change log</Text>
	</DocumentLink>
</DocumentLinks>
-----------------------------------

(ii)And html files looks like
-----------------------------------
Table of Contents

  • Introduction
  • Objects in the LPframework
      ? Tasks
      ? Folders
      ? Trees
      ? Midas Banks
  • Access Methods to Objects in the LPframework
      ? Folders
      ? Data Base
      ? Steering Parameters
      ? Histograms
      ? Midas Banks
      ? General
  • Class Overview
  • Links
      ? Change log
-----------------------------------

(iii)Users must write the target(html/changelog.html).

> I was asked to put change log of my rome application(LPframewok). I think the best place to put it is the 
> html file which builder created.  But of course to modify it is not good, because it will be overwritten by 
> builder again.
> 
> So, I want to propose to allow user to add links in the html file with the way like followings.
> 
> (i) User defines links.
> ------   in LPframewok.xml   ------
> <DocumentLinks>
> 	<DocumentLink>
> 		<Href>html/changelog.html</Href>
> 		<Text>Change log</Text>
> 	</DocumentLink>
> </DocumentLinks>
> -----------------------------------
> 
> (ii)And html files looks like
> -----------------------------------
> Table of Contents
> 
>   • Introduction
>   • Objects in the LPframework
>       ? Tasks
>       ? Folders
>       ? Trees
>       ? Midas Banks
>   • Access Methods to Objects in the LPframework
>       ? Folders
>       ? Data Base
>       ? Steering Parameters
>       ? Histograms
>       ? Midas Banks
>       ? General
>   • Class Overview
>   • Links
>       ? Change log
> -----------------------------------
> 
> (iii)Users must write the target(html/changelog.html).

There will be a new link available "Additional Info" after "Class Overview".

This points to a file named "UserHTML.html" which can be modified by the user.

This is not so strong suggestion. I am just wondering.

Currently ROME include headers of midas, xml2 and mysql.
There are libraries for windows but no libraries for linux.

I am afraid that there can be inconsequence between headers and libraries in futrue updates of these 
libraries, if compiler reads headers from rome/include instead of correct one.

So, my suggestion is that move midas.h,libxml2 and mysql outside of header files search path for linux.

for instance
  Make new directory rome/include/win.
  Put midas.h,libxml2 and mysql into it.
  Add $ROMESYS/include/win into header files search path in case of Windows.


P.S.
If midas.h is already very different from midas.h in MIDAS. This may be difficult.

About warnings,errors and informations output in online mode.

Probably we lauch ROME from web interface of MIDAS. And we don't see console of ROME.
But we need to hear what ROME is saying.

Normally midas analyzer and other applications use cm_msg function. This function shows messages 
proper place like web interface or ODBEdit windows and so on.

I do not have much knowledge about it. Stefan may know better way.

> This is not so strong suggestion. I am just wondering.
> 
> Currently ROME include headers of midas, xml2 and mysql.
> There are libraries for windows but no libraries for linux.
> 
> I am afraid that there can be inconsequence between headers and libraries in futrue updates of these 
> libraries, if compiler reads headers from rome/include instead of correct one.
> 
> So, my suggestion is that move midas.h,libxml2 and mysql outside of header files search path for linux.
> 
> for instance
>   Make new directory rome/include/win.
>   Put midas.h,libxml2 and mysql into it.
>   Add $ROMESYS/include/win into header files search path in case of Windows.
> 
> 
> P.S.
> If midas.h is already very different from midas.h in MIDAS. This may be difficult.

I looked into gcc, then I found the header file which is found first is supposed to be used.
In ROME's case $(shell xml2-config --cflags) and $(shell mysql_config --cflags) are located before -I$(ROMESYS)/include/, so in principle 
it is safe.

But about midas.h there still can be inconsequence, because -I$(ROMESYS)/include/ is located before -I$(MIDASSYS)/include/.
It this done on purpose to enable special features for ROME ?


---
One more thing. about libpaths.
If libpaths will be used only for specify the place where midas libraries were put. Followings can also work and more simple.
And I added midascflags, because to use same way as root,xml2 and mysql seems more beautiful. 
And I moved -DHAVE_SQL and -DHAVE_MIDAS to sqlcflags and midascflags.

rootlibs := $(shell root-config --libs)
rootglibs := $(shell root-config --glibs)
rootcflags := $(shell root-config --cflags)
rootthreadlibs := -lThread
xmllibs :=  $(shell xml2-config --libs)
xmlcflags :=  $(shell xml2-config --cflags)
sqllibs := $(shell mysql_config --libs)
sqlcflags := $(shell mysql_config --cflags) -DHAVE_SQL
midaslibs := -L$(MIDASSYS)/lib -lmidas
midascflags := -I$(MIDASSYS)/include -DHAVE_MIDAS
clibs :=-lpthread -lHtml $(SYSLIBS)
clibs += -lutil

Libraries := $(libpaths) $(rootlibs) $(rootglibs) $(rootthreadlibs) $(xmllibs) $(clibs) $(sqllibs) $(midaslibs)
Flags := $(lpuserflags) $(rootcflags) $(xmlcflags) $(sqlcflags) $(midascflags)
Includes := -I$(ROMESYS)/include/ -I. -Iinclude/ -Iinclude/tasks/ -Iinclude/framework/

> About warnings,errors and informations output in online mode.
> 
> Probably we lauch ROME from web interface of MIDAS. And we don't see console of ROME.
> But we need to hear what ROME is saying.
> 
> Normally midas analyzer and other applications use cm_msg function. This function shows messages 
> proper place like web interface or ODBEdit windows and so on.
> 
> I do not have much knowledge about it. Stefan may know better way.

I implemented the cm_msg calls.

> I implemented the cm_msg calls.

You should implement a central "message dispatcher". So all errors etc. should go to someting linke
rome_msg(). If compiled with midas online support, rome_msg() should call cm_msg(). If rome is compiled
without midas libraries, a printf() there would be enough. If someone uses rome in a different
environment, only can function needs to be changed to reroute all messages to somewhere else.

> > I implemented the cm_msg calls.
> 
> You should implement a central "message dispatcher". So all errors etc. should go to someting linke
> rome_msg(). If compiled with midas online support, rome_msg() should call cm_msg(). If rome is compiled
> without midas libraries, a printf() there would be enough. If someone uses rome in a different
> environment, only can function needs to be changed to reroute all messages to somewhere else.

Of course, that's the way I implemented that.

We sometimes want to use trees in ROME.

One way is to read it by TSocket connection. But treees are larger than histograms in general, and it 
takes time to download. 
 And I need to make shared library when I changed the structure of folders.

So, I want to propose is to prepare Cint mode with continueing analysis in background.

Currently, when user press "i", ROME stops analysis and goes into cint mode. This is good for offline 
mode, but this is not good for online mode. So I propose that when user press "I", ROME goes cint 
mode, and continue analysis in background.

Then we can see Trees at run time.

In addition, If you are going to implement this feature, please prepare short way to call trees in cint mode.
If I understand correctly, the way to call a tree is like,
gAnalyzer->GetTreeObjectAt(0)->GetTree()->Draw("");
but it is too long to use in interactively and we need to remeber the number of trees.
gAnalyzer->GetTree("trigger")->Draw();  is better.

I am going to make a new ROME project to analyze MC data. Input file will be ROOT which  is created by 
h2root.
And my friend who is working on dark matter search at Columbia university in NY is going to use 
ROME. His file format is special binary format.

If I understand correctly, currently ROME can read midas and ROOT format.
Midas file names are always assumed as "run99999.mid". (99999 is run number)
ROOT files need to have structure defined in definition xml file. 
I guess this is not enough to analyze special binary format or ROOT files created by h2root.

What I propose is to allow users to have own daq class. It might be something like

<DAQs>
  <DAQ>
    <DAQName>daqname</DAQName>
  </DAQ>
</DAQs>

Then builder creates [shortcut][daqname].h, [shortcut][daqname].cpp.
[shortcut][daqname] class inherits ROMEDAQSystem class, and users must fill GetTimeStamp(), 
Initialize(), Connect(), ReadEvent(int event), Disconnect(), Termination()

When "InputDataFormat" in configuration xml file is "daqname" analyzer uses this class.


 

This is not strong request, just a proposal because the same thing can be done by configuration file.

It may be useful if users can have his command line options.

For instance.
<CommandOptions>
  <CommandOption>
    <OptionFlag>m<OptionFlag>
    <OptionName>isMC<OptionName>
    <OptionType>Bool_t<OptionType>
    <OptionInitialization>false<OptionInitialization>
  </CommandOption>
  <CommandOption>
    <OptionFlag>t<OptionFlag>
    <OptionName>TriggerType<OptionName>
    <OptionType>Int_t<OptionType>
    <OptionInitialization>0<OptionInitialization>
  </COmmandOption>
</CommandOptions>

Then user can change the variable with command line option like
./megframework -m -t 2;

Then builder creates new method.
(Bool_t) MEGAnalyzer::GetisMC()
(Int_t) MEGAnalyzer::GetTriggerType()

  There is no check if the pointer returned by GetDataBase,GetActiveDAQ is valid. This can result 
segmentation fault.
  For instance, if one specifies DAQ system as "none", or one does not specify DataBaseName, rome 
stops without error message.

It is better to modify this behavior.


* One way is to stop application like.
ROMEDAQSystem* GetActiveDAQ() {
   if(!fActiveDAQ){
      gAnalyzer->Println("some error message");
      fApplication->Terminate(1);
   }
   return fActiveDAQ;
};

*Other way is to use a dummy DAQSystem and DataBase which does nothing when user does not 
specify DAQ or DataBase.  When dummy is used, warning message must be shown, because it is 
dangerous that users do not realize they are using dummy.