ELOG Midas

Back Midas Rome Roody Rootana

Midas DAQ System, Page 6 of 145

Not logged in

Find | Login | Help

New entries since:

Wed Dec 31 16:00:00 1969

Full | Summary | Threaded | Hide attachments

2896 Entries

Goto page Previous 1, 2, 3 ... 5, 6, 7 ... 143, 144, 145 Next

Params not initialized when starting sequencer

The problem here was that the JS code did not wait to msequencer to finish preparing the "/Sequencer/Param" in the ODB, so I had to change to code to wait for "/Sequencer/Command/Load new file" to be false before proceeding.

As for your problem I recommend that you handle in the following way:

mjsonrpc_db_paste(paths,values).then(function (rpc) {
if (rpc.result.status.every(status => status === 1) {
// do something
} else {
// failed to set values, do something else
}
}).catch(function (error) {
console.error(error);
});

alternatively (for a single ODB) you can use the checkODBValue() function in sequencer.js. This function monitors a specific ODB path until it reaches a specific value and then calls funcCall with args.

var NcheckValue = 0;
// What for ODB in path to have value
// If value is not reached, give up after 10s
function checkODBValue(path,value,funcCall,args) {
/* Arguments:
path - ODB path to monitor for value
value - the value to be reached and return success
funcCall - function name to call when value is reached
args - argument to pass to funcCall
*/
// Call the mjsonrpc_db_get_values function
mjsonrpc_db_get_values([path]).then(function(rpc) {
if (rpc.result.status[0] === 1 && rpc.result.data[0] !== value) {
console.log("Value not reached yet", NcheckValue);
NcheckValue++;
if (NcheckValue < 100) {
// Wait 0.1 second and then call checkODBValue again
// Time out after 10 s
setTimeout(() => {
checkODBValue(path,value,funcCall,args);
}, 100);
}
} else {
if (funcCall) funcCall(args);
console.log("Value reached, proceeding...");
// reset counter
NcheckValue = 0;
}
}).catch(function(error) {
console.error(error);
});
}

Lukas Gerritzen wrote:

I think I have had similar issues in a custom page, where I wrote values to the ODB and they were not ready when I needed them. If you found a fix to such race conditions, could you maybe share how to properly treat this issue? If the solution reliably works, we could also consider including it in the documentation (midaswiki or example.html).

Zaher Salman wrote:

The issue with the parameters should be fixed now. Please test and let me know if it still happens.

Params not initialized when starting sequencer

Zaher Salman wrote:

The issue with the parameters should be fixed now. Please test and let me know if it still happens.

Multiple issues with mhist

Hi,

I am having some trouble with mhist. I suppose that the problems are at least partially due to our specific needs which might exceed what has been tested. For context, in MEG II we have some 10^4 history variables in ~30 different events.

1. mhist -l crashes. After displaying around 7000 lines, I get the following error message:

[mhist,ERROR] [midas.cxx:5949:bm_validate_client_index,ERROR] My client index 10 in buffer 'SYSMSG' 
is invalid: client name '', pid 0 should be my pid 3773321
[mhist,ERROR] [midas.cxx:5952:bm_validate_client_index,ERROR] Maybe this client was removed by a 
timeout. See midas.log. Cannot continue, aborting...
Aborted (core dumped)

Timing the execution shows around 33 seconds before the process is aborted.

I'm not sure if this would actually fix the problem, but while trying to circumvent the issue, I tried the
following:

mhist -e "Xenon" -l

This doesn't seem to be implemented. Listing only the variables of a single event would be nice
regardless of our specific issue.

2. mhist and history files.
We have a directory directory with about 2500 history files (mhf_...dat) for the past 1.5 years. Older
history files are archived in other directories with similar numbers of files. When trying to access them, I
encountered two issues:
It seems like it is not possible to pass a "history directory" as an argument. To dump the history for a full
year in the archive directory, I would need to run mhist many times with -f and then combine all the dumps.

If it really does not work, please consider this a feature request.
Also, even using single files does not work at the moment:

$ mhist -e "Xenon" -v "Det XeTmp 0-0" -t 100000 -s 200101 -p 250101 -f 
/data2/history/2022/mhf_1644698398_20220212_xenon.dat
ID 980316009, Aug 13 19:10:56, size 1851749486

This command was supposed to show me the rough time frame covered in this particular history file. I was
informed that the history files are in the new "FILE" format and mhist might not work with them properly.

tl;dr

Bug: mhist -l crashes
Bug: mhist -f does not work with "FILE" history format
Feature request: mhist -e "Name" -l to only show variables of event "Name"
Feature request: Set temporary history dir with a flag

Lukas

Improve Event Documentation

> My overall idea here is to connect directly to midas so having some frontend features to analyze the data etc. do 
> you also have already a library for this? I can also extend your stuff.

No, sadly I don't have something like this yet. It has been on my "fun things to do at some point" list for too
long, but I haven't had the time.

If you start working on something like this, please keep me in the loop/link a repo here. I would be interested
on keeping an eye/contributing to something like this :)

Improve Event Documentation

> > I am writing a Rust based midas file reader
> 
> You might find this library I wrote useful: https://crates.io/crates/midasio
> 
> It should "just work", and if it doesn't, I would be interested to know.

Nice! I did not know about this. I have now also one simple reader but yours looks much more advanced. My 
overall idea here is to connect directly to midas so having some frontend features to analyze the data etc. do 
you also have already a library for this? I can also extend your stuff.

Best,
Marius

Improve Event Documentation

> I am writing a Rust based midas file reader

You might find this library I wrote useful: https://crates.io/crates/midasio

It should "just work", and if it doesn't, I would be interested to know.

Improve Event Documentation

> > Hi,
> > 
> > I am writing a Rust based midas file reader however it was kind of hard to understand the full midas file 
> > structure from the documentation.
> > 
> > Only at the end of the page 
> > https://daq00.triumf.ca/MidasWiki/index.php/Event_Structure#MIDAS_Format_Event one finds under the 
> > headline “tape format” that there are special events which mark the start and the end of the run. It would 
> > be better to place this information more prominent maybe we a headline: “Special Events”. Maybe a link to 
> > this section at the top of the page could help. Also at the mlogger page there is no information about this.
> > 
> > Best,
> > Marius
> 
> Ben was so kind to update the event documentation:
> 
>   https://daq00.triumf.ca/MidasWiki/index.php/Event_Structure
> 
> Please have a look and let us know if that's better now.
> 
> Best,
> Stefan

Thank you Ben! Now its super clear!

Improve Event Documentation

> Hi,
> 
> I am writing a Rust based midas file reader however it was kind of hard to understand the full midas file 
> structure from the documentation.
> 
> Only at the end of the page 
> https://daq00.triumf.ca/MidasWiki/index.php/Event_Structure#MIDAS_Format_Event one finds under the 
> headline “tape format” that there are special events which mark the start and the end of the run. It would 
> be better to place this information more prominent maybe we a headline: “Special Events”. Maybe a link to 
> this section at the top of the page could help. Also at the mlogger page there is no information about this.
> 
> Best,
> Marius

Ben was so kind to update the event documentation:

  https://daq00.triumf.ca/MidasWiki/index.php/Event_Structure

Please have a look and let us know if that's better now.

Best,
Stefan

Params not initialized when starting sequencer

The issue with the parameters should be fixed now. Please test and let me know if it still happens.

Thomas Senger wrote:

Hi Zaher,
thanks for your help.
I just tried the bug fix, but it still seems not to work properly.
It seems that if the script is short, it will work, but if many SUBROUTINES are integrated, it does not work and the parameter are initialized empty.
Best regards,
Thomas

Improve Event Documentation

Hi,

I am writing a Rust based midas file reader however it was kind of hard to understand the full midas file 
structure from the documentation.

Only at the end of the page 
https://daq00.triumf.ca/MidasWiki/index.php/Event_Structure#MIDAS_Format_Event one finds under the 
headline “tape format” that there are special events which mark the start and the end of the run. It would 
be better to place this information more prominent maybe we a headline: “Special Events”. Maybe a link to 
this section at the top of the page could help. Also at the mlogger page there is no information about this.

Best,
Marius

Help parsing scdms_v1 data?

Stefan Ritt wrote:

The MIDAS event format is described here:

https://daq00.triumf.ca/MidasWiki/index.php/Event_Structure

All banks are aligned on a 8-byte boundary, so that one has effective 64-bit CPU access.

If you have sections of 168 or 192 bytes, this must be something else, like another bank (scaler event, slow control event, ...).

The easiest for you is to check how this events got created using the bk_create() function.

Best,
Stefan

Upon further investigation, the sections I'm looking at appear to be clusters of headers for empty banks.

Thank you!

Help parsing scdms_v1 data?

Help parsing scdms_v1 data?

Hi! I'm working on creating a ksy file to help with parsing some data, but I'm having trouble finding some information. Right now, I have it set up very rudimentary - it grabs the event header and then uses the data bank size to grab the size of the data, but then I'm needing additional padding after the data bank to reach the next event.
However, there's some irregularity in the "padding" between data banks that I haven't been able to find any documentation for. For some reason, after the data banks, there's sections of data of either 168 or 192 bytes, and it's seemingly arbitrary which size is used.
I'm just wondering if anyone has any information about this so that I'd be able to make some more progress in parsing the data.
The data I'm working with can be found at https://github.com/det-lab/dataReaderWriter/blob/master/data/07180808_1735_F0001.mid.gz
And the ksy file that I've created so far is at https://github.com/det-lab/dataReaderWriter/blob/master/kaitai/ksy/scdms_v1.ksy

There's also a block of data after the odb that runs for 384 bytes that I'm unsure the purpose of, if anyone could point me to some information about that.

Thank you!

Ok, no relevant complains so far, so I removed mana and rmana from the CMake build 
process, but left the file mana.cxx still in the repository for educational 
purposes ;-)

Stefan

"Safe" abort of sequencer scripts

> This request came more than once in the past. One thing I could implement is a "atexit" function similarly to the C funciton atexit().
> 
> Then we would have a function in the script which gets called whenever one does "stop immediately". This function can then restore
> some ODB values or do whatever is necessary. 
> 
> If the sequencer gets killed in the middle, it can safely be restarted since the complete sequencer state is kept in the ODB under
> /Sequencer/State. After the restart, the sequencer continues exactly where it has been killed before.
> 
> Would that solve your problem?
> 
> Stefan

Yes, an "atexit" functionality within the Midas Sequencer Language would be useful for us with this issue.  Is this easy for you to implement?

Thanks,
Scott Oser

kernel-module-universe updated to -KO7

> > The linux kernel driver for the Universe-II VME to PCI bridge is updated to 
> > version -KO7. It now builds and runs with Debian-12 stock kernel 6.1.0-22-686.

Ahem, and the location is:

git clone https://daq00.triumf.ca/~olchansk/git/kernel-module-universe.git

K.O.

kernel-module-universe updated to -KO7

> The linux kernel driver for the Universe-II VME to PCI bridge is updated to 
> version -KO7. It now builds and runs with Debian-12 stock kernel 6.1.0-22-686.

I have a report that this driver might work on 64-bit VME CPUs (minus a bug in the 
MIDAS VME library). I do not have such hardware, cannot test, cannot confirm. (All our 
64-bit VME CPUs have the tsi148 bridge and run Ubuntu kernels and userland).

https://daq00.triumf.ca/elog-midas/Midas/2566

K.O.

kernel-module-universe updated to -KO7

The linux kernel driver for the Universe-II VME to PCI bridge is updated to 
version -KO7. It now builds and runs with Debian-12 stock kernel 6.1.0-22-686.

I pxe boot (isolinux/pxelinux) the linux kernel and NFS-mount the stock 32-bit 
Debian-12 userland. Userland tarball is available by request. PXE and NFS-Root 
configuration is written up on the wiki at daq.triumf.ca, example config files 
are available on request.

https://daq00.triumf.ca/DaqWiki/index.php/Ubuntu#setup_diskless_network_booting
https://daq00.triumf.ca/DaqWiki/index.php/VME-CPU

The Debian-11 kernel also works (use -KO6 driver is -KO7 bombs), but Debian-11 
kernel with Debian-12 userland and Ubuntu-22 NFS server fails with "file too 
big" errors, the best I can tell this has to do with old 32-bit kernels getting 
unhappy about 64-bit NFS inode numbers.

Cross-compilation from 64-bit Ubuntu-22 to 32-bit VME processors running 32-bit 
Debian-12 is written up here:
https://daq00.triumf.ca/DaqWiki/index.php/Ubuntu#32-bit_intel_cross-compiler

To cross-build 32-bit MIDAS for 32-bit VME processor use "make linux32" or build 
natively (pretty slow on 1 GHz Pentium-III).

K.O.

"Safe" abort of sequencer scripts

This request came more than once in the past. One thing I could implement is a "atexit" function similarly to the C funciton atexit().

Then we would have a function in the script which gets called whenever one does "stop immediately". This function can then restore
some ODB values or do whatever is necessary. 

If the sequencer gets killed in the middle, it can safely be restarted since the complete sequencer state is kept in the ODB under
/Sequencer/State. After the restart, the sequencer continues exactly where it has been killed before.

Would that solve your problem?

Stefan

"Safe" abort of sequencer scripts

We often use the MIDAS sequencer to temporarily control detector settings, such as:

* <change some setting>
* WAIT 60 seconds
* <revert setting to original value>

The question arises of what happens if the sequencer scripts gets aborted during that wait, preventing the value from being reset.  Depending on the setting, this could be undesirable or even damage something if left uncorrected for too long.

Is there any way to have a "safe abort" from the sequencer so that the "Stop immediately" button will call some cleanup script to leave things in a safe state?  Or what about if the sequencer process itself gets killed in the middle of a script?

How have other experiments using MIDAS protected themselves from unplanned terminations of sequencer scripts?

Goto page Previous 1, 2, 3 ... 5, 6, 7 ... 143, 144, 145 Next

ELOG V3.1.4-2e1708b5