Hi,

I am having some trouble with the data in my Midas bank. I am filling a midas
bank in my frontend (one of several in my system), and this bank is then being
added to the overall event by the event builder.

I check the data as it enters the bank, and also check again after I close the
bank in my frontend (using pdata's original value), and in both cases my data is
as I expect.

However, when I view the data in the .mid file (using mdump), there is a
problem. The correct number of words are there, and the values are correct up
until the 148th word. However, all subsequent words are 0.

I have also noticed that if I change my word size from 32bit (DWORD) to 16bit
(WORD), I observe the same behaviour except that now the first 296 words are
correct and all others are zero.

Note that other frontends in the system are not suffering this issue.

Does anyone have any ideas about how to solve this problem?

I've seen that a similar question has been asked in 2011 but I'll ask again in 
case there are any updates. Is there any way to write 64-bit data words to MIDAS 
banks (other than breaking them up in to two 32-bit words, such as 2 DWORDs) 
currently? And if not, is there any plan to introduce this feature in the future?

Many thanks,
Tom

> > I've seen that a similar question has been asked in 2011 but I'll ask again in 
> > case there are any updates. Is there any way to write 64-bit data words to MIDAS 
> > banks (other than breaking them up in to two 32-bit words, such as 2 DWORDs) 
> > currently? And if not, is there any plan to introduce this feature in the future?
> 
> There is no "breaking them up" as such, you can treat a midas bank as a char* array
> and store arbitrary data inside. In this sense, "there is no need" for a special 64-bit bank type.
> 
> For endian-ness conversion (if such things still matter, big-endian PPC CPUs still exist), single 64-bit 
> word converts the same as two 32-bit words, so here also "there is no need", once can use banks of 
> DWORD with equal effect.
> 
> The above applies equally to 64-bit integers and 64-bit double-precision IEEE-754 floating point 
> numbers.
> 
> But specifically for 64-bit values, such as float64, there is a big gotcha.
> 
> The MIDAS banks structure goes to great lengths to make sure each data type is correctly aligned,
> and gets it exactly wrong for 64-bit quantities - all because the bank header is three 32-bit words.
> 
> bankhheader1
> bh2
> bh3
> bankdata1 <--- misaligned
> ...
> bankdataN
> bh1
> bh2
> bh3
> banddata1 <--- aligned
> ... etc
> 
> So we could introduce QWORD banks today, but inside the midas file, they will be misaligned defeating 
> the only purpose of adding them.
> 
> I guess the misalignement could be cured by adding dummy words, dummy banks, dummy bank 
> headers, etc.
> 
> I figure this problem dates all the way bank where alignement to 16-bits was just getting important. 
> Today, in the VME word, I have to align things on 128-bit boundaries (for 2eSST 2x2 DWORD transfers).
> 
> So back to your question, what advantage do you see in using a QWORD bank instead of putting the 
> same data in a DWORD bank?
> 
> K.O.

> > I've seen that a similar question has been asked in 2011 but I'll ask again in 
> > case there are any updates. Is there any way to write 64-bit data words to MIDAS 
> > banks (other than breaking them up in to two 32-bit words, such as 2 DWORDs) 
> > currently? And if not, is there any plan to introduce this feature in the future?
> 
> There is no "breaking them up" as such, you can treat a midas bank as a char* array
> and store arbitrary data inside. In this sense, "there is no need" for a special 64-bit bank type.
> 
> For endian-ness conversion (if such things still matter, big-endian PPC CPUs still exist), single 64-bit 
> word converts the same as two 32-bit words, so here also "there is no need", once can use banks of 
> DWORD with equal effect.
> 
> The above applies equally to 64-bit integers and 64-bit double-precision IEEE-754 floating point 
> numbers.
> 
> But specifically for 64-bit values, such as float64, there is a big gotcha.
> 
> The MIDAS banks structure goes to great lengths to make sure each data type is correctly aligned,
> and gets it exactly wrong for 64-bit quantities - all because the bank header is three 32-bit words.
> 
> bankhheader1
> bh2
> bh3
> bankdata1 <--- misaligned
> ...
> bankdataN
> bh1
> bh2
> bh3
> banddata1 <--- aligned
> ... etc
> 
> So we could introduce QWORD banks today, but inside the midas file, they will be misaligned defeating 
> the only purpose of adding them.
> 
> I guess the misalignement could be cured by adding dummy words, dummy banks, dummy bank 
> headers, etc.
> 
> I figure this problem dates all the way bank where alignement to 16-bits was just getting important. 
> Today, in the VME word, I have to align things on 128-bit boundaries (for 2eSST 2x2 DWORD transfers).
> 
> So back to your question, what advantage do you see in using a QWORD bank instead of putting the 
> same data in a DWORD bank?
> 
> K.O.


Thanks very much for your reply. I have implemented your suggestion of treating the 64-bit array as a 32-bit 
array for the bank write/read and this solution is working for me.

Thanks again for your help.

I have experienced a problem building MIDAS from the head of the SVN repository (rev 4458) when 
specifying the shared library flag. Whie the shared library appears to compile and link OK, the 
subsequent compilation of mserver fails as follows:

$ make ROOTSYS= NEED_SHLIB=1

<... snipped some lines ...>

ld -shared -o linux/lib/libmidas.so linux/lib/midas.o linux/lib/system.o linux/lib/mrpc.o 
linux/lib/odb.o linux/lib/ybos.o linux/lib/ftplib.o linux/lib/mxml.o linux/lib/history_midas.o 
linux/lib/history_sql.o linux/lib/history.o linux/lib/alarm.o linux/lib/elog.o linux/lib/strlcpy.o -lutil -
lpthread -lz -lc
cc -c -g -O2 -Wall -Wuninitialized -Iinclude -Idrivers -I../mxml -Llinux/lib -DINCLUDE_FTPLIB   -
D_LARGEFILE64_SOURCE -DHAVE_ZLIB -DOS_LINUX -fPIC -Wno-unused-function -o linux/lib/mana.o 
src/mana.c
cc -c -g -O2 -Wall -Wuninitialized -Iinclude -Idrivers -I../mxml -Llinux/lib -DINCLUDE_FTPLIB   -
D_LARGEFILE64_SOURCE -DHAVE_ZLIB -DOS_LINUX -fPIC -Wno-unused-function -o 
linux/lib/cnaf_callback.o src/cnaf_callback.c
cc -c -g -O2 -Wall -Wuninitialized -Iinclude -Idrivers -I../mxml -Llinux/lib -DINCLUDE_FTPLIB   -
D_LARGEFILE64_SOURCE -DHAVE_ZLIB -DOS_LINUX -fPIC -Wno-unused-function -o linux/lib/mfe.o 
src/mfe.c
g++ -Dextname -DMANA_LITE -c -g -O2 -Wall -Wuninitialized -Iinclude -Idrivers -I../mxml -
Llinux/lib -DINCLUDE_FTPLIB   -D_LARGEFILE64_SOURCE -DHAVE_ZLIB -DOS_LINUX -fPIC -Wno-
unused-function -o linux/lib/fal.o src/fal.c
cc -g -O2 -Wall -Wuninitialized -Iinclude -Idrivers -I../mxml -Llinux/lib -DINCLUDE_FTPLIB   -
D_LARGEFILE64_SOURCE -DHAVE_ZLIB -DOS_LINUX -fPIC -Wno-unused-function -o 
linux/bin/mserver src/mserver.c -lmidas -Wl,-rpath,/usr/local/lib -lutil -lpthread -lz
/usr/bin/ld: linux/bin/mserver: hidden symbol `__dso_handle' in /usr/lib/gcc/x86_64-redhat-
linux/4.1.2/crtbegin.o is referenced by DSO
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: ld returned 1 exit status
make: *** [linux/bin/mserver] Error 1

Having googled the error, it appears to be solved by modifying the linker statement for the shared 
library in the Makefile at line 464 to use g++ rather than ld:

463c463
< 	ld -shared -o $@ $^ $(LIBS) -lc
---
> 	$(CXX) -shared -o $@ $^ $(LIBS) -lc

Presumably this is because g++ knows better how to link in the appropriate system libraries required 
for some of the recently added C++ code? 

This was on Scientific Linux SL5.2 x86_64, gcc version 4.1.2, glibc version 2.5-24.

Tim

We have a MIDAS frontend that provides both the readout of raw events 
and the processing of raw events into several distinct derived datasets. 
For one type of derived dataset there is a derived event for 
each raw event. For other types of derived datasets there's a
derived event for every N raw events. We'd like to have the different 
derived event types sent to different buffers / shared memory segments 
and stored in different midas files.

I was thinking of defining a separate equipment for each type of 
derived data. Each equipment would have a different buffer name so
the data would go to different buffers and thereafter to different 
midas files. I was also thinking of defining each equipment as
a "polled event" but with a unique "source ID". I believe the user 
poll_event() function is passed the "source ID" of the equipment
type and therefore could return success/fail based on whether or
not the particular derived event with that source ID is available 
for readout. Each equipment for each derived dataset would have 
a unique readout routine to create and fill the midas databanks 
for that derived event type.

The above scheme is similar to the midas documentation example
of a frontend with a trigger equipment and a scaler equipment
However, the scaler / trigger example uses two different event
types - EQ_POLLED for trigger and EQ_PERIODIC for scaler. I'd like
to use several EQ_POLLED equipments that are distinguished by
their source ID's

Is this a sensible scheme for make different data streams of
different derived event types from a single frontend? Has anyone
tried something similar? 

Hi Midas forum,

I'm having a problem with odb hotlinks after increasing sub-directories in an 
odb. I now get the error code DB_NO_MEMORY after some db_open_record() calls. I 
tried 

1) increasing the parameter DEFAULT_ODB_SIZE in midas.h and make clean, make
but got the same error

2) increasing the parameter  MAX_OPEN_RECORDS in midas.h and make clean, make
but got fatal errors from odbedit and my midas FE and couldnt run anything

3) deleting my expts SHM files and starting odbedit with "odbedit -e SLAC -s 
0x1000000" to increse the odb size but got the same error?

4) I tried a different computer and got the same error code DB_NO_MEMORY

Maybe I running into some system limit that restricts the humber of open records? 
Or maybe I've not increased the correct midas parameter?

Best ,Tim.

oOne additional comment. I was able to trace the setting of the error code DB_NO_MEMORY 
to a call to the db_add_open_record() by mserver that is initiated during the start-up 
of my frontend via an RPC call. I checked with a debug printout that I have indeed 
reached the number of MAX_OPEN_RECORDS

> Hi Midas forum,
> 
> I'm having a problem with odb hotlinks after increasing sub-directories in an 
> odb. I now get the error code DB_NO_MEMORY after some db_open_record() calls. I 
> tried 
> 
> 1) increasing the parameter DEFAULT_ODB_SIZE in midas.h and make clean, make
> but got the same error
> 
> 2) increasing the parameter  MAX_OPEN_RECORDS in midas.h and make clean, make
> but got fatal errors from odbedit and my midas FE and couldnt run anything
> 
> 3) deleting my expts SHM files and starting odbedit with "odbedit -e SLAC -s 
> 0x1000000" to increse the odb size but got the same error?
> 
> 4) I tried a different computer and got the same error code DB_NO_MEMORY
> 
> Maybe I running into some system limit that restricts the humber of open records? 
> Or maybe I've not increased the correct midas parameter?
> 
> Best ,Tim.

[quote="Stefan Ritt"][quote="Thorsten Lux"]In addition now I cannot start
anymore the mlogger from the web interface but only manually. However, I can
stop it from the web interface.[/quote]

At least that one can be fixed easily. Each program has a certain command with
which one can start it. This has to be put into the ODB under
/Programs/<program>. In your case you probably need

/Programs/Logger/Start command = mlogger -D

to start the logger from the Web page. To debug your run stop problems, I would
recommend to start all programs in a terminal window and look which one crashes
on the run end.

/Stefan[/quote]


Hi Stefan,

under /Programs/Logger/Start command I have
/home/next/MIDAS/midas/linux/bin/mlogger -D . This command does not work if I
press the "Start Logger" button on the mhttpd webpage but when I copy and paste
this command to a terminal window, it does the job. 

Well, thanks to you both for the fast response. I wrote Konstantin an email with
the results of the tests he suggested me to do.

Ciao

> 
> > Hi Stefan,
> > 
> > under /Programs/Logger/Start command I have
> > /home/next/MIDAS/midas/linux/bin/mlogger -D . This command does not work if I
> > press the "Start Logger" button on the mhttpd webpage but when I copy and paste
> > this command to a terminal window, it does the job. 
> > 
> > Well, thanks to you both for the fast response. I wrote Konstantin an email with
> > the results of the tests he suggested me to do.
> > 
> > Ciao
> 
> Let me guess: mhttpd is started under root (to be able to connect to port 80), and for root the mlogger program 
> is not in the path. Try to put into the odb the full path:
> 
> /Programs/Logger Start command = /usr/local/bin/mlogger -D

Yes, mhttpd is started as sudo, but I have the full path in the start command. And every user has the right to
execute mlogger. But okay, I will concentrate first to get the rest working again and then I will fight this problem.

Thanks!

Hi,

it seems that I solved the problem in a quite brutal way.
I opened the database with odbedit and saved first the whole database as a ASCII
file and then I did the same for each section separately. Then I closed odbedit.
Afterwards I deleted all .*.SHM files including .ODB.SHM and rebooted the system.
After the restart I opened odbedit and started mhttpd. With this blank system
the problem had disappeared. Afterwards I loaded section by section from the
previous created ASCII files. After each section I tested if I can start and
stop runs and it worked without problems. At the end I also loaded the ASCII
file which contained the whole database. In this case I got the following error
message:
[odb.c:6038:db_paste,ERROR] found string exceeding MAX_STRING_LENGTH

However, after a reboot everything worked fine. I can start and stop runs, with
and without frontend, without any error message. Only the mlogger resisted to
work again. 

But also this problem we solved. It seems it was related to a missing library
path. It is strange since while in the mhttpd web page the command does not work
and is not giving any error message, copying the same command to a terminal and
to start it manually does the job. We solved it by putting the start of mlogger
in a simple shell script and to execute it then from the mhttpd web page.
Probably not an elegant solution but it does the job.

Well, with this I can enjoy my weekend to start over with data taking next week!

Thanks a lot!

Thorsten

Hello,

We observed a strange behavior, from our point of view:
After some issues with with a 100% full database and recovering from this by 
creating a new odb file from a previous copy, the Midas status page started to 
reload/refresh every second while on the other hand it solved the 100% full 
issue.
There are no error messages and the pages looks like normal but it is impossible 
to start a new run due to the permanent reloading.
It is possible to click for example on odb and check the settings. 

Any idea what could be the problem and what the solution? 

Thanks 

Thorsten 

I am sorry, yesterday evening I must have been a bit tired after a long day with a lot of 
problems and error messages, so that I did not realize that yes, the frontend was finally 
starting well again but by recovering the odb file from an old one, it was stuck in the 
transition "stopping run" and this caused the continuous reloading of the status page. 

A "obdedit -C stop" solved the problem. 

Sorry for this! 





 > We observed a strange behavior, from our point of view:
> > ... the Midas status page started to reload/refresh every second
> 
> What version of midas is this? Run the odbedit "ver" command please. Also which
> browser on what OS is this? (chrome->about google chrome, firefox->about firefox).
> 
> The current versions of midas do not reload the status page ever, and I think
> all the page-reload code has been removed and they cannot reload automatically.
> 
> Old versions of the midas status page were designed to reload every 60 seconds or so.
> The reload interval is adjustable, but I do not think it was stored in ODB. It was
> accessed from the status page "config" button and I think it stored the reload
> period is a browser cookie.
> 
> This reload value may have gotten confused, and in this case, to fix it,
> you can try to clear all the web cookies from the web page. Another test for this
> would be to try an alternate web browser, which would presumable not have the bad cookie
> and will not suffer from the reload problem.
> 
> You can also open the web page debugger (google chrome -> right click menu -> inspect ->> 
> console & etc) and see if anything shows up there. I think you can set a break point
> on the page reload function and catch the place that causes the reload.
> 
> K.O.

Hi, I would like to restructure our sequencer scripts and the paths. Until now many things are not generic at all. I would like to ask if it is possible to include files through a relative path for example something like 
INCLUDE ../chip/global_basic_functions
Maybe I just did not found how to do it.

Hello there, 
in our setup we use many variables with many different exceptions. Would it be possible to implement something like an 
IF or/and IF statement? I believe that this is currently not possible.
Best regards, 
Thomas Senger

I find that new mhttpd has strange behaviour for ODB strings.

- I create a new STRING variable in ODB through mhttpd.  It defaults to size 32.

- I then edit the STRING variable through mhttpd, writing a new string larger
than 32 characters.

- Initially everything looks fine; it seems as if the new string value has been
accepted.

- But if you reload the page, or navigate back to the page, you realize that
mhttpd has silently truncated the string back to 32 characters.

You can reproduce this problem on a test page here:

http://midptf01.triumf.ca:8081/AnnMessage

Older versions of mhttpd (I'm testing one from 2 years ago) don't have this
'feature'.  For older mhttpd the string variable would get resized when a larger
string was inputted.  That definitely seems like the right behavior to me.

I am using fresh copy of midas from bitbucket as of this morning. (How do I get
a particular tag/hash of the version of midas that I am using?)