ID |
Date |
Author |
Topic |
Subject |
2022
|
25 Nov 2020 |
Marco Francesconi | Suggestion | ODBSET wildcards with array keys in Sequencer files | Hi,
I guess the issue is in the "[?]" part of the command, the indexing is handled differently from the odb path and does not
support "?".
Are you trying to set only the first 9 channels?
Could you try with "[*]" or "[0-9]" instead?
Marco
> I'm interested in using the matching feature for ODBSET explained on
> https://midas.triumf.ca/MidasWiki/index.php/Sequencer for settings that are in an
> array, like:
>
> COMMENT "Ground the detectors"
> ODBSET "/Detectors/Det*/Settings/Charge/Bias (V)[?]" 0
>
> Currently I get an error when I try to run this script. Is this expected? Would it
> be possible to implement matching for array values?
>
> Thanks! |
2021
|
24 Nov 2020 |
Amy Roberts | Suggestion | ODBSET wildcards with array keys in Sequencer files | I'm interested in using the matching feature for ODBSET explained on
https://midas.triumf.ca/MidasWiki/index.php/Sequencer for settings that are in an
array, like:
COMMENT "Ground the detectors"
ODBSET "/Detectors/Det*/Settings/Charge/Bias (V)[?]" 0
Currently I get an error when I try to run this script. Is this expected? Would it
be possible to implement matching for array values?
Thanks! |
2020
|
24 Nov 2020 |
Isaac Labrie Boulay | Forum | Invalid name "Analyzer/Tests" | Hi everyone,
I've recently took the analyzer template from $MIDASSYS/examples/experiment and
modified it to be able to use Roody on a very simple frontend setup. The
analyzer works fine and I am able to view the online histograms but my console
prints out this error:
[Analyzer,ERROR] [odb.cxx:919:db_validate_name,ERROR] Invalid name
"/Analyzer/Tests/Always true/Rate [Hz]" passed to db_create_key_wlocked: should
not contain "["
[Analyzer,ERROR] [odb.cxx:919:db_validate_name,ERROR] Invalid name
"/Analyzer/Tests/low_sum/Rate [Hz]" passed to db_create_key_wlocked: should not
contain "["
[Analyzer,ERROR] [odb.cxx:919:db_validate_name,ERROR] Invalid name
"/Analyzer/Tests/high_sum/Rate [Hz]" passed to db_create_key_wlocked: should not
contain "["
The error keeps getting printed even after stopping the run.
I do have these 3 keys under Analyzer/Tests/ in my ODB but I do not know where
they come from. Any suggestions on what the root of the issue is?
Thanks for the help!
Isaac |
2019
|
20 Nov 2020 |
Pierre-Andre Amaudruz | Info | Equipment "common" settings in ODB | Indeed this "mirror" of the ODB in settings option can cause frustration in
particular when we think the ODB is empty but is not.
In the other hand, over time the settings are adjusted to a particular
configuration or touched or not by the individual run preset parameters. Later, if
a bug or code correction requires multiple restart of the fe, for every start of
the application, you loose the latest configuration. This can be frustrating as
well until you force a post-setting or report the specifics parameters in the fe
code.
BTW I believe, we originally went for the ODB priority for that specific reason.
I would be in favour for having a general flag (FALSE) in /experiment which would
define this global behaviour.
PAA
> Today I addressed a topic which bugged me since long time. The ODB contains
> settings under /Equipment/<name>/Common which are a "mirror" of the equipment[]
> setting in a frontend (using the mfe.cxx framework). If the "Common" entry in
> the ODB is not present (fresh experiment), the equipment[] settings from the
> frontend are copied to the ODB. But if it exists, it takes precedence over the
> equipment[] entries, which is wrong in my opinion. Like if you change some
> settings in equipment[] (like the logging period of the history), then recompile
> and restart the frontend, the old values in the ODB are kept and your
> modification in the frontend code has no effect.
>
> Starting on commit c3017c6c on Nov. 17th 2020 I reversed the precedence: Now, on
> each start of the frontend program, the values from equipment[] are written to
> the ODB. They are still "live". If one changes them when the frontend is
> running, that change takes effect immediately. But on the next restart of the
> frontend, the old values from equipment[] is put back there.
>
> I fell too many times into this trap, and I hope the modification helps
> everybody. If there are however experiments which rely on the fact that the
> common settings in the ODB are NOT overwritten by the frontend, please let me
> know and I can put a flag "EQUIPMENT_FE_PRECEDENCE = FALSE" somewhere to restore
> the old behaviour.
>
> Stefan |
2018
|
20 Nov 2020 |
Stefan Ritt | Forum | History plot consuming too much memory | > Taking this down a tangent, I have a mild concern that a user could temporarily
> flood our gigabit network if we do have faster disks to read the history data. Have
> there been any plans or thoughts on limiting the bandwidth users can pull from
> mhttpd?
I guess this will not be network limiting but CPU limiting of the mhttpd process. But I'm
not 100% sure, depends on the actual hardware. But even if we improve the history
retrieval to "window only", the user could request all data form 2010 to 2020. So one
would need some code which estimates the amount of data, then tell the user "do you really
want that?". But still, a novice user can simply click "yes" without much of a thought. So
in conclusion I believe proper user training is better than software limits. Like the
other guy "I did 'rm -rf /', and now nothing works any more, can you help?".
Stefan |
2017
|
20 Nov 2020 |
Joseph McKenna | Forum | History plot consuming too much memory | Poking at the behavior of this, its fairly clear the slow response is from the data
being loaded off an HDD, when we upgrade this system we will allocate enough SSD
storage for the histories.
Using Firefox has resolved this issue for the user's project here
Taking this down a tangent, I have a mild concern that a user could temporarily
flood our gigabit network if we do have faster disks to read the history data. Have
there been any plans or thoughts on limiting the bandwidth users can pull from
mhttpd? I do not see this as a critical item as I can plan the future network
infrastructure at the same time as the next system upgrade (putting critical data
taking traffic on a separate physical network).
> Of course one can only
> load that specific window, but when the user then scrolls right, one has to
> append new data to the "right side" of the array stored in the browser. If the
> user jumps to another location, then the browser has to keep track of which
> windows are loaded and which windows not, making the history code much more
> complicated. Therefore I'm only willing to spend a few days of solid work
> if this really becomes a problem.
For now the user here has retrieved all the data they need, and I can direct others
towards mhist in the near future. Being able to load just a specific window would be
very useful in the future, but I comprehend how it would be a spike in complexity. |
2016
|
19 Nov 2020 |
Stefan Ritt | Forum | History plot consuming too much memory | The history code is right now programmes in such a way that when you request
an old time window, then all data from that window until the present date
gets loaded. When we implemented that, this worked fine for data ranges of
several years with a delay of just a few seconds. Of course one can only
load that specific window, but when the user then scrolls right, one has to
append new data to the "right side" of the array stored in the browser. If the
user jumps to another location, then the browser has to keep track of which
windows are loaded and which windows not, making the history code much more
complicated. Therefore I'm only willing to spend a few days of solid work
if this really becomes a problem.
Are you sure that the delay comes from the browser or actually from mhttpd
digging through GBytes of history data? I realized that you need solid state
disks to get a real quick response.
Stefan |
2015
|
19 Nov 2020 |
Joseph McKenna | Forum | History plot consuming too much memory |
A user reported an issue that if they were to plot some history data from
2019 (a range of one day), the plot would spend ~4 minutes loading then
crash the browser tab. This seems to effect chrome (under default settings)
and not firefox
I can reproduce the issue, "Data Being Loaded" shows, then the page and
canvas loads, then all variables get a correct "last data" timestamp, then
the 'Updating data ...' status shows... then the tab crashes (chrome)
It seems that the browser is loading all data until the present day (maybe 4
Gb of data in this case). In chrome the tab then crashes. In firefox, I do
not suffer the same crash, but I can see the single tab is using ~3.5 Gb of
RAM
Tested with midas-2020-08-a up until the HEAD of develop
I could propose the user use firefox, or increase the memory limit in
chrome, however are there plans to limit the data loaded when specifically
plotting between two dates? |
2014
|
17 Nov 2020 |
Stefan Ritt | Info | Equipment "common" settings in ODB | Today I addressed a topic which bugged me since long time. The ODB contains
settings under /Equipment/<name>/Common which are a "mirror" of the equipment[]
setting in a frontend (using the mfe.cxx framework). If the "Common" entry in
the ODB is not present (fresh experiment), the equipment[] settings from the
frontend are copied to the ODB. But if it exists, it takes precedence over the
equipment[] entries, which is wrong in my opinion. Like if you change some
settings in equipment[] (like the logging period of the history), then recompile
and restart the frontend, the old values in the ODB are kept and your
modification in the frontend code has no effect.
Starting on commit c3017c6c on Nov. 17th 2020 I reversed the precedence: Now, on
each start of the frontend program, the values from equipment[] are written to
the ODB. They are still "live". If one changes them when the frontend is
running, that change takes effect immediately. But on the next restart of the
frontend, the old values from equipment[] is put back there.
I fell too many times into this trap, and I hope the modification helps
everybody. If there are however experiments which rely on the fact that the
common settings in the ODB are NOT overwritten by the frontend, please let me
know and I can put a flag "EQUIPMENT_FE_PRECEDENCE = FALSE" somewhere to restore
the old behaviour.
Stefan |
2013
|
06 Nov 2020 |
Isaac Labrie Boulay | Forum | Building an experiment using CAEN VME interface - unknown type name 'VARIANT_BOOL' | Yes, you are right. That fixed it and my frontend is compiling.
Thanks Pierre-Andre.
Isaac
> Hi,
>
> You're building under Linux like. You want to define the LINUX and skip the VARIANT_BOOL all together.
> PAA
>
> > Hi everyone,
> >
> > I have been building an experiment using the v1718 CAEN interface to talk to my modules and I am using the CAENVMElib Linux Library (2.50). I've managed to deal with data type issues by including additional libraries to my driver code but there is one type error
> that persists:
> >
> >
> > In file included from /usr/include/CAENVMElib.h:27:0,
> > from include/v1718.h:25,
> > from v1718.c:26:
> > /usr/include/CAENVMEtypes.h:323:9: error: unknown type name ‘VARIANT_BOOL’
> > CAEN_BOOL cvDS0; /* Data Strobe 0 signal */
> >
> >
> > The header file used to defined the CAEN types (CAENVMEtypes.h) defines 'CAEN_BOOL' like this:
> >
> >
> > #ifdef LINUX
> > #define CAEN_BYTE unsigned char
> > #define CAEN_BOOL int
> > #else
> > #define CAEN_BYTE byte
> > #define CAEN_BOOL VARIANT_BOOL
> > #endif
> >
> >
> > Has anyone ever ran into that problem when setting up an experiment using the CAEN standard?
> >
> > Thanks for your help.
> >
> > Isaac |
Draft
|
06 Nov 2020 |
Isaac Labrie Boulay | Forum | Building an experiment using CAEN VME interface - unknown type name 'VARIANT_BOOL' |
> Hi,
>
> You're building under Linux like. You want to define the LINUX and skip the VARIANT_BOOL all together.
> PAA
>
> > Hi everyone,
> >
> > I have been building an experiment using the v1718 CAEN interface to talk to my modules and I am using the CAENVMElib Linux Library (2.50). I've managed to deal with data type issues by including additional libraries to my driver code but there is one type error
> that persists:
> >
> >
> > In file included from /usr/include/CAENVMElib.h:27:0,
> > from include/v1718.h:25,
> > from v1718.c:26:
> > /usr/include/CAENVMEtypes.h:323:9: error: unknown type name ‘VARIANT_BOOL’
> > CAEN_BOOL cvDS0; /* Data Strobe 0 signal */
> >
> >
> > The header file used to defined the CAEN types (CAENVMEtypes.h) defines 'CAEN_BOOL' like this:
> >
> >
> > #ifdef LINUX
> > #define CAEN_BYTE unsigned char
> > #define CAEN_BOOL int
> > #else
> > #define CAEN_BYTE byte
> > #define CAEN_BOOL VARIANT_BOOL
> > #endif
> >
> >
> > Has anyone ever ran into that problem when setting up an experiment using the CAEN standard?
> >
> > Thanks for your help.
> >
> > Isaac |
2011
|
06 Nov 2020 |
Alexandr Kozlinskiy | Suggestion | cmake build fixes | hi,
there are several problems with current cmake build files in midas:
- not all systems have cuda libs in /usr/local/cuda
- not all cmake version like when redefining vars
(i.e. redefining ROOT_CXX_FLAGS)
- c++ standard not matching the one used to build ROOT
- ROOTSYS is not needed to find ROOT (it is enough to have root in PATH)
I have posted pull request 'https://bitbucket.org/tmidas/midas/pull-requests/17'
which tries to fix some of the problems.
Tests and comments are welcome. |
2010
|
05 Nov 2020 |
Pierre-Andre Amaudruz | Forum | Building an experiment using CAEN VME interface - unknown type name 'VARIANT_BOOL' | Hi,
You're building under Linux like. You want to define the LINUX and skip the VARIANT_BOOL all together.
PAA
> Hi everyone,
>
> I have been building an experiment using the v1718 CAEN interface to talk to my modules and I am using the CAENVMElib Linux Library (2.50). I've managed to deal with data type issues by including additional libraries to my driver code but there is one type error
that persists:
>
>
> In file included from /usr/include/CAENVMElib.h:27:0,
> from include/v1718.h:25,
> from v1718.c:26:
> /usr/include/CAENVMEtypes.h:323:9: error: unknown type name ‘VARIANT_BOOL’
> CAEN_BOOL cvDS0; /* Data Strobe 0 signal */
>
>
> The header file used to defined the CAEN types (CAENVMEtypes.h) defines 'CAEN_BOOL' like this:
>
>
> #ifdef LINUX
> #define CAEN_BYTE unsigned char
> #define CAEN_BOOL int
> #else
> #define CAEN_BYTE byte
> #define CAEN_BOOL VARIANT_BOOL
> #endif
>
>
> Has anyone ever ran into that problem when setting up an experiment using the CAEN standard?
>
> Thanks for your help.
>
> Isaac |
2009
|
05 Nov 2020 |
Isaac Labrie Boulay | Forum | Building an experiment using CAEN VME interface - unknown type name 'VARIANT_BOOL' | Hi everyone,
I have been building an experiment using the v1718 CAEN interface to talk to my modules and I am using the CAENVMElib Linux Library (2.50). I've managed to deal with data type issues by including additional libraries to my driver code but there is one type error that persists:
In file included from /usr/include/CAENVMElib.h:27:0,
from include/v1718.h:25,
from v1718.c:26:
/usr/include/CAENVMEtypes.h:323:9: error: unknown type name ‘VARIANT_BOOL’
CAEN_BOOL cvDS0; /* Data Strobe 0 signal */
The header file used to defined the CAEN types (CAENVMEtypes.h) defines 'CAEN_BOOL' like this:
#ifdef LINUX
#define CAEN_BYTE unsigned char
#define CAEN_BOOL int
#else
#define CAEN_BYTE byte
#define CAEN_BOOL VARIANT_BOOL
#endif
Has anyone ever ran into that problem when setting up an experiment using the CAEN standard?
Thanks for your help.
Isaac |
2008
|
21 Oct 2020 |
Soichiro Kuribayashi | Info | About remote control of front end part of MIDAS on chip | Dear Stefan,
Thank you very much for your help.
I have already contacted someone who has used ZYNQ in that order and It's working fine for now.
But, I'll let you know if something goes wrong.
Best regards,
Soichiro |
2007
|
20 Oct 2020 |
Stefan Ritt | Info | About remote control of front end part of MIDAS on chip | We also use a Zynq chip and boot in the following order:
1. SD Card
a. First Stage Bootloader
b. PL Firmware
c. UBOOT
2. NFS over Ethernet
a. Linux kernel
b. RootFS
c. Mounting home directories
If you need details I can bring you in contact with the person who actually implemented that.
Best,
Stefan |
2006
|
13 Oct 2020 |
Soichiro Kuribayashi | Info | About remote control of front end part of MIDAS on chip | Dear Konstantin,
Thank you very much for your reply and detailed information.
I would appreciate if you could help us.
> I can also suggest that on your frontend SOC/FPGA machine, you boot linux
> using the "nfs-root" method. This way, the local flash memory only
> contains a boot loader (and maybe the linux kernel image, depending on
> bootloader limitations). The rest of the linux rootfs can be on your
> central development machine. This way management of flash cards,
> confusion with different contents of local flash and need to make backups
> of frontend machines is much reduced.
As you said, we can run complete Linux (Ubuntu 16) on ZYNQ and I'm using common NFS
system now. However, I didn't know "nfs-root" method which you mentioned and this method
seems to be reasonable way to just share linux rootfs.
First of all, I will try this method for simpler system.
> If you use a fast SSD and ZFS with deduplication, you will also have good
> performance gain (NFS over 1gige network to server with fast SSD works
> so much better compared to the very slow SD/MMC/NAND flash).
>
> I can point you to some of my documentation how we do this.
I'm concerned about such performance and I have checked the performance with common NFS
over gige network and my DAQ PC roughly(data transfer rate ~ O(10) MByte/sec). However, I
didn't know the ZFS and also how we can have performance gain with a fast SSD and ZFS.
Please let me know your documentation how to do it if possible.
> I would say you are on a good track. For early development on just one board,
> pretty much any way you do it will work, but once you start scaling up
> beyound 3-4-5 frontends, you will start seeing benefits from common NFS-mounted
> home directories, NFS-root booted linux, etc.
I'm developing with just one board and common NFS-mounted now. I'm looking forward to
seeing such benefits when I will use multiple frontends.
> And of course you may want to study the existing ND280/FGD DAQ. I hope you
> have access to the running system at Jparc. If not, I have a copy of
> pretty much everything (except for running hardware, it is stored in the basement,
> dead) and I can give you access.
I don't have access to the system at Jparc, but Nick has told us where FGD DAQ code is.
Is bellow URL everything of code of FGD DAQ?
https://git.t2k.org/hastings/fgddaq/-/tree/master
Best regards,
Soichiro |
2005
|
13 Oct 2020 |
Konstantin Olchanski | Info | About remote control of front end part of MIDAS on chip | > My name is Soichiro Kuribayashi and I am a Ph.D. student at Kyoto University.
> I'm a T2K collaborator and working for Super FGD which is new detector in ND280.
Hi! I did much of the DAQ software for the original FGD. I hope I can help.
> For the DAQ of Super FGD, we will run remotely front end part of MIDAS on ZYNQ
> which is system on chip.
This would be the same as the existing FGD. Inside the FGD DCC is a Virtex4 FPGA
with a 300MHz PPC CPU running Linux from a CompactFlash card (Kentaro-san did this
part). On this linux system runs the FGD DCC midas frontend. It connects
to the FGD midas instance using the mserver. This frontend executable is
copied to the DCC using "scp", there is no common nfs mounted home directory.
> For this remote control of front end part with mserver, we have to mount home
> directory of DAQ PC(Cent OS8) on that of Linux on ZYNQ.
> So I wonder if we should use NFS(Network file system) + NIS(Network information
> service) + autofs for the mounting. Is it correct?
Since you have a bigger SOC and you can run pretty much a complete linux,
I do recommend that you go this route. During development it is very convenient
to have common home directories on the main machine and on the frontend fpga
machines.
But this is not necessary. the midas mserver connection does not require
common (nfs-mounted) home directory, you can copy the files to the frontend
fpga using scp and rsync and you can use the gdb "remote debugger" function.
I can also suggest that on your frontend SOC/FPGA machine, you boot linux
using the "nfs-root" method. This way, the local flash memory only
contains a boot loader (and maybe the linux kernel image, depending on
bootloader limitations). The rest of the linux rootfs can be on your
central development machine. This way management of flash cards,
confusion with different contents of local flash and need to make backups
of frontend machines is much reduced.
If you use a fast SSD and ZFS with deduplication, you will also have good
performance gain (NFS over 1gige network to server with fast SSD works
so much better compared to the very slow SD/MMC/NAND flash).
I can point you to some of my documentation how we do this.
>
> If you have any information or any suggestion for the remote control on chip,
> please let me know.
>
I would say you are on a good track. For early development on just one board,
pretty much any way you do it will work, but once you start scaling up
beyound 3-4-5 frontends, you will start seeing benefits from common NFS-mounted
home directories, NFS-root booted linux, etc.
And of course you may want to study the existing ND280/FGD DAQ. I hope you
have access to the running system at Jparc. If not, I have a copy of
pretty much everything (except for running hardware, it is stored in the basement,
dead) and I can give you access.
P.S. This reminds me that the cascade software from ND280 (they key part
for connecting the FGD, the TPC, the slow controls & etc into one experiment)
was never merged into the midas repository. I opened a ticket for this,
now we will not forget again:
https://bitbucket.org/tmidas/midas/issues/291/import-cascase-frontend-from-t2k-
nd280-fgd
K.O. |
2004
|
13 Oct 2020 |
Soichiro Kuribayashi | Info | About remote control of front end part of MIDAS on chip | Hello!
My name is Soichiro Kuribayashi and I am a Ph.D. student at Kyoto University.
I'm a T2K collaborator and working for Super FGD which is new detector in ND280.
I'm a beginner of MIDAS and I've just started to develop the DAQ software with
MIDAS for Super FGD.
For the DAQ of Super FGD, we will run remotely front end part of MIDAS on ZYNQ
which is system on chip.
For this remote control of front end part with mserver, we have to mount home
directory of DAQ PC(Cent OS8) on that of Linux on ZYNQ.
So I wonder if we should use NFS(Network file system) + NIS(Network information
service) + autofs for the mounting. Is it correct?
If you have any information or any suggestion for the remote control on chip,
please let me know.
Best regards,
Soichiro |
2002
|
06 Oct 2020 |
Konstantin Olchanski | Forum | using python client to start and stop run | > The ODB variable "/Runinfo/State" is a symptom of starting/stopping a run, rather than the cause.
>
> In C++, one uses `cm_transition()` to start/stop runs.
>
> In python code you can use the `start_run()` and `stop_run()` functions from `midas.client`: https://bitbucket.org/tmidas/midas/src/00ff089a836100186e9b26b9ca92623e672f0030/python/midas/client.py#lines-793:808
one can also run an external command: "mtransition START" and "mtransition STOP"
K.O. |
|