ID |
Date |
Author |
Topic |
Subject |
848
|
17 Dec 2012 |
Vinzenz Bildstein | Suggestion | Midas + Elog with SSL | > > I've been trying to set up midas to create an automatic elog entry at the end of
> > each run and I've run into a problem. I've setup an elog on our server which
> > uses SSL and it seems that the melog provided by midas to create logbook entries
> > doesn't know any SSL.
> >
> > My solution to this was to copy the crypt.c from the elog package to the
> > computer running midas and changed melog.c and the makefile to use SSL if a flag
> > -s is used. Does this seem like a sensible solution or did I oversee the obvious
> > and/or right way to do this?
>
> Indeed melog.c is an old version of the elog.c utility in the elog package, which has not been maintained since a
> long time. Can't you just use the recent elog.c utility from the elog package?
Well, that's essentially what I did, I just didn't want to install the whole elog package on the midas server. Whether
the utility is called elog or melog doesn't really matter. I just wanted to make sure that this is the right way to do
it.
Thanks! |
851
|
04 Jan 2013 |
Nabin Poudyal | Suggestion | how to start using midas | Please, tell me how to choose a value of a "key" like DCM, pulser period,
presamples, upper thresholds to run a experiment? where can I find the related
informations? |
969
|
27 Feb 2014 |
Andreas Suter | Suggestion | runlog is "ugly" | I have a couple of questions and suggestions concerning the "new" CSS style of the mhttpd, especially related to the runlog
- If I am not mistaken, the mhttpd.css is hard coded (path/name) into the mhttpd. Wouldn't it be beneficial to have ODB entries where to get is from? This way people could change the look and feel more freely.
- Especially the look and feel of the runlog is unsatisfactorily from my point of view. See . The old style was much more readable. I could recover the old style look and feel by slightly changing the mhttpd.cxx where I changed in show_rawfile(const char*) "dialogTable" to "runlogTable" in the table class. This way I could tinker around with the mhttpd.css by adding the following stuff there:
- adding .runlogTable in line 289 :
- adding some style information for the runlogTable :
This way the "old" runlog look and feel recovered : , which I think is much more readable.
If possible, I would love to have alternating background colors between the runs for readability reasons, but I am not sure how easy it would be to add something like this.
I not much experience with HTML/CSS yet, though a concrete implementation might be different. |
970
|
27 Feb 2014 |
Konstantin Olchanski | Suggestion | runlog is "ugly" | > If I am not mistaken, the mhttpd.css is hard coded (path/name) into the mhttpd.
mhttpd.css is served from $MIDASSYS/resources/mhttpd.css. The actual path is reported on the mhttpd
"help" page.
(I think the internal mhttpd.css and mhttpd.js should be removed as no longer useful - nothing will work
right if the real mhttpd.js and mhttpd.css cannot be served).
> Especially the look and feel of the runlog is unsatisfactorily from my point of view.
persons in charge of implementing the CSS stuff failed to convert quite a few pages, for example, the elog
and the history editor pages were left completely broken. (mostly fixed now).
so thank you for reporting the runlog breakage, I hope Stefan & co can fix it quickly. (I cannot do - I have
have no runlog pages on any of my test experiments).
> the old style was much more readable.
I think the new style is not too bad, except for a few visual artefacts here and there, the general comment
that CSS is too complicated and hard to debug and the fact that over-subtle colouring yields inconsistent
visuals between different monitors and ambient lighting conditions. (persons who select the colours always
respond that "but to me, it looks just fine on my laptop", making it hard to resolve any issues).
> I could recover the old style look and feel by slightly changing the mhttpd.cxx
If you post the patches that fix it for you, I can commit them to midas. (git diff | mail olchansk@triumf.ca).
K.O. |
973
|
28 Feb 2014 |
Andreas Suter | Suggestion | runlog is "ugly" | Understand me right, I mostly like the new style, except the runlog as reported.
Attached you will find the diff's you were asking for. But as pointed out, I
haven't worked so far on CSS and hence this should be checked!!
I understand that the mhttpd.js needs to be the default one, however, mhttpd.css
might be left to the end-user to adopt to their specific needs. I shortly
checked in the mhttpd demon. It checks for the resources path in the ODB. If it
also would check for a CSS name, mhttpd.css could be changed/adopted by the
end-users without breaking things (at least it would then be their one business).
> > If I am not mistaken, the mhttpd.css is hard coded (path/name) into the mhttpd.
>
> mhttpd.css is served from $MIDASSYS/resources/mhttpd.css. The actual path is
reported on the mhttpd
> "help" page.
>
> (I think the internal mhttpd.css and mhttpd.js should be removed as no longer
useful - nothing will work
> right if the real mhttpd.js and mhttpd.css cannot be served).
>
> > Especially the look and feel of the runlog is unsatisfactorily from my point
of view.
>
> persons in charge of implementing the CSS stuff failed to convert quite a few
pages, for example, the elog
> and the history editor pages were left completely broken. (mostly fixed now).
>
> so thank you for reporting the runlog breakage, I hope Stefan & co can fix it
quickly. (I cannot do - I have
> have no runlog pages on any of my test experiments).
>
> > the old style was much more readable.
>
> I think the new style is not too bad, except for a few visual artefacts here
and there, the general comment
> that CSS is too complicated and hard to debug and the fact that over-subtle
colouring yields inconsistent
> visuals between different monitors and ambient lighting conditions. (persons
who select the colours always
> respond that "but to me, it looks just fine on my laptop", making it hard to
resolve any issues).
>
> > I could recover the old style look and feel by slightly changing the mhttpd.cxx
>
> If you post the patches that fix it for you, I can commit them to midas. (git
diff | mail olchansk@triumf.ca).
>
> K.O. |
Attachment 1: mhttpd.cxx.diff
|
diff --git a/src/mhttpd.cxx b/src/mhttpd.cxx
index 7437897..3940b96 100755
--- a/src/mhttpd.cxx
+++ b/src/mhttpd.cxx
@@ -3445,7 +3445,7 @@ void show_rawfile(const char *path)
rsprintf("</table>");
//main table:
- rsprintf("<table class=\"dialogTable\">");
+ rsprintf("<table class=\"runlogTable\">");
/*---- menu buttons ----*/
rsprintf("<tr><td colspan=2>\n");
rsprintf("<input type=submit name=cmd value=\"ELog\">\n");
|
Attachment 2: mhttpd.css.diff
|
diff --git a/resources/mhttpd.css b/resources/mhttpd.css
index 71c73a4..066cfe6 100644
--- a/resources/mhttpd.css
+++ b/resources/mhttpd.css
@@ -286,7 +286,7 @@ div.wrapper{
margin: 0 auto -3em;
}
-.historyConfigTable, .dialogTable, .messageBox, .genericTable, .sequencerTable{
+.historyConfigTable, .runlogTable, .dialogTable, .messageBox, .genericTable, .sequencerTable{
border-radius: 12px;
border: 2px solid #00B26B;
background-color: #EEEEEE;
@@ -304,6 +304,22 @@ table.sequencerTable td table{
margin: 0px;
}
+table.genericTable tr:nth-child(even){
+ background: #EEEEEE;
+}
+table.genericTable tr:nth-child(odd){
+ background: #FAFAFA;
+}
+
+table.runlogTable td{
+ border:none;
+ text-align: left;
+ font-family: monospace;
+}
+
+table.runlogTable pre{
+ line-height: 125%;
+}
table.dialogTable td{
border:none;
|
974
|
28 Feb 2014 |
Stefan Ritt | Suggestion | runlog is "ugly" | > If I am not mistaken, the mhttpd.css is hard coded (path/name) into the mhttpd.
I agree that this should be removed, Unfortunately I'm away right now, so I will fix it next week. Also will put in
Andreas' diffs.
/Stefan |
977
|
07 Mar 2014 |
Stefan Ritt | Suggestion | runlog is "ugly" | I put mhttpd.css and mhttpd.js into the ODB, so every experiment can change it. I put also Andreas' modifications of the CSS file for the runlog table and
committed the changes.
/Stefan |
999
|
26 May 2014 |
Dan Melconian | Suggestion | "Edit-on-end" would be nice | We use the "Edit-on-start" and it's great. But sometimes, something breaks
during the run, or you didn't realize you forgot to plug in a cable, or
whatever. It'd be nice to have an "Edit-on-end" where you could prompt the user
to answer simple questions (like "Was this a good run? [y/n]" or "Was the data
polarized? [y/n]") and/or add a quick summary of what happened that run.
Thanks in advance,
Dan |
1000
|
26 May 2014 |
Stefan Ritt | Suggestion | "Edit-on-end" would be nice | We have similar demands, and we solve it in the following:
We use a run database. In the simplest case, this can be a text file which gets written at the end of the file. The
mlogger has a built in SQL interface, so one can keep that table even inside a SQL interface. The per-run-
information then contains the run number, start/stop time, number of events, some run parameters and a "junk"
flag. So if a run has a problem, one can set the junk flag by accessing the database (or text file) and setting this
flag. In many cases you see that a run had a problem not at the end of the run, but a bit later. You mayby realize
that the last two or three runs had the problem. With the run database approach, you can flag any run as "junk"
later, which we need often, An edit-on-end would not make this possible.
So technically putting and edit-on-end is not a problem, but your life might be much easier if you use a run
database as outlined above.
Best regards,
Stefan |
1004
|
27 May 2014 |
Scott Oser | Suggestion | Saving ODB values in a sequencer script | I have a possibly simple feature request for the MIDAS sequencer. It would be
helpful to be able to save an ODB key's value to a variable, for later use, and
would be the analogue of the ODBSET command. I had in mind an application where
a user wants to temporarily change some settings in the ODB, then restore the
ODB to its original values. Maybe something like on ODBRead command:
<ODBRead path="/Path/ODBkey">varname</ODBRead>
<ODBSet path="/Path/ODBkey">0</ODBRead>
<Wait for="events">3000</Wait>
<ODBSet path="/Path/ODBkey">$varname</ODBRead>
(In which the key's value is saved to variable varname, then later written back
to the ODB.)
I'm open to other suggestions for simple ways to do this through the sequencer.
Thanks! |
1007
|
12 Jun 2014 |
Stefan Ritt | Suggestion | Saving ODB values in a sequencer script | > I have a possibly simple feature request for the MIDAS sequencer. It would be
> helpful to be able to save an ODB key's value to a variable, for later use, and
> would be the analogue of the ODBSET command. I had in mind an application where
> a user wants to temporarily change some settings in the ODB, then restore the
> ODB to its original values. Maybe something like on ODBRead command:
I implemented your request, committed the changed to GIT and updated the documentation. Now you can run
things like:
ODBSET /System/tmp/test 1234
ODBGET /System/tmp/test v
MESSAGE $v
(first you must create the key in the ODB manually).
Best regards,
Stefan |
1008
|
12 Jun 2014 |
Scott Oser | Suggestion | Saving ODB values in a sequencer script | Thanks, this seems very helpful, and we'll give it a try.
> > I have a possibly simple feature request for the MIDAS sequencer. It would be
> > helpful to be able to save an ODB key's value to a variable, for later use, and
> > would be the analogue of the ODBSET command. I had in mind an application where
> > a user wants to temporarily change some settings in the ODB, then restore the
> > ODB to its original values. Maybe something like on ODBRead command:
>
> I implemented your request, committed the changed to GIT and updated the documentation. Now you can run
> things like:
>
> ODBSET /System/tmp/test 1234
> ODBGET /System/tmp/test v
> MESSAGE $v
>
> (first you must create the key in the ODB manually).
>
> Best regards,
> Stefan |
1057
|
14 May 2015 |
Konstantin Olchanski | Suggestion | checksums for midas data files | I am adding LZ4 and LZO compression the mlogger and as part of this work, I would like to add
computation of checksums for the midas files.
On one side, such checksums help me confirm that uncompressed data contents is the same as original
data (compression/decompression is okey).
On the other side, such checksums can confirm to the end user that today's contents of the midas file is
the same as originally written by mlogger (maybe years ago) - there was no bit rot, no file corruption, no
accidental or intentional modification of contents.
There are several choices of checksums available:
crc32 - as implemented by zlib (already written inside mid.gz files)
crc32c - improved and hardware accelerated version of CRC32 (http://tools.ietf.org/html/rfc3309)
md5 - cryptographically strong checksum, but obsolete
sha1 - same, also obsolete
sha256 - currently considered to be cryptographically strong
Of these checksums, only sha256 (sha512, etc) are presently considered to be cryptographically strong,
meaning that they can detect intentional file modifications. As opposed to (for example) crc32 where
it is easy to construct 2 files with different contents but the same checksum. Both md5 and sha1 are
presently considered to be similarly cryptographically broken. But all of them are still usable
as checksums - as they will detect non-intentional data modifications (bit rot, etc) with
very high probability.
(Of course the strongest checksum is also the most expensive to compute).
I will probably implement crc32 (already in zlib), crc32c (easy to find hardware-accelerated
implementations) and sha256 (cryptographically strong).
I can write the computed checksums into midas.log, or into runNNN.crc32, runNNN.sha256, etc files. (or
both).
Any thoughts on this?
K.O. |
1058
|
14 May 2015 |
Stefan Ritt | Suggestion | checksums for midas data files | > Any thoughts on this?
We use binary midas files now for ~20 years and never felt the necessity to put any checksums or even encryption on these files. The reason for that is the following: Data on
modern hard disks is already protected by CRC code or even ERC on the lower level, so it's very unlikely that single bytes change. If something happens, then it's a
corruption of the file system, so a few sectors of a file are missing or wrong. In that case a CRC won't help you much, just tells you that the files are corrupt. But you see that
also in the midas event structure. Each event has a header with the size of the event, so you can follow the file event by event. If something is missing, the next event header
is no event header but something in the middle of the date, and you recognise this immediately since the header does not make any send (date is off by many years, event ID
is arbitrary, event size is very different). So this redundancy in the midas event structure helps you to identify any corrupt files as good in my opinion as a CRC code will. I
would not want to waste a single CPU cycle on lengthy CRC or even SHA algorithms, unless I see single bytes change inside events. But in this case this can even happen at
the network level between frontend and backends. So we should add the CRC/SHA code at the frontend level. This could increase the dead time of the experiment which is
bad. And what about VME transfer? While hard disks and Ethernet networks have already built-in CRC checks, VME transfer doesn't. So how can you be sure that no bits
get corrupt between your ADC and your frontend computer?
If people insist of having CRC or SHA protection/encryption for some reason I do not understand yet, we should make this optional, so that I can turn it off, since I don't
need it.
/Stefan |
1059
|
15 May 2015 |
Konstantin Olchanski | Suggestion | checksums for midas data files | > > Any thoughts on this?
>
> We use binary midas files now for ~20 years and never felt the necessity to put any checksums or even encryption on these files ...
>
"I have never seen a corrupted file, therefore nobody should ever need checksums". Well,
1) actually if you write mid.gz files, you get gzip checksums "for free" (but the checksums are not recorded anywhere, so 5 years later you cannot confirm that the file did not change).
2) I had a defective computer once where reading the same file several times yielded different data. (the defect was on the motherboard, not in the disks)
3) I am presently testing the btrfs filesystem which (like ZFS) keeps checksums for all data. For these tests I am using 3rd quality disks and I see btrfs regularly detect (and correct) "data corruption" events - where data on disk has changed.
4) there was a report from CERN(?) where they checked the checksums on a large number of data files and found a good number of corrupted files.
So bit rot does exist.
In more practical terms:
a) CRC32C is "free" to compute (hardware accelerated on latest CPUs), but does not detect malicious file modifications
b) SHA256 does detect that (but for how long?), but probably too expensive to compute (speed measurement TBD).
c) gzip compressed files have internal whole-file CRC32
d) bzip2 compressed files have internal per-block CRC32
e) lz4 compressed files have internal per-block xxhash checksums
Personally, when dealing with compressed files, I prefer to have a checksum recoded somewhere that I can check against after I decompress the file.
I think there is no need to add checksums to the MIDAS data files format itself (see c,d,e above).
K.O. |
1119
|
28 Sep 2015 |
Anthony Villano | Suggestion | Feature Request: MIDAS sequencer abort. |
I am working for the SuperCDMS collaboration on some DAQ issues for our upcoming
SNOLAB installation. So far, the MIDAS sequencer seems to be a good paradigm
for us to do procedural tasks for our detectors and data running interspersed
with other protocols.
In our testing we've found that the sequencer works very well for this kind of
activity, although it would be useful to have a kind of scripted "abort" for
when something goes wrong -- especially if the user selects to abort a run
sequence.
Because the sequencer is setting various detector parameters to a certain value
before performing the tasks, the values will never be restored if the user
aborts the sequence. Instead, perhaps there can be a portion of a MIDAS
sequence script which is instructed to happen on an abort. Perhaps something
like all commands after a given tag like:
ON ABORT:
get run on a user-initiated abort? |
1123
|
22 Oct 2015 |
Konstantin Olchanski | Suggestion | Feature Request: MIDAS sequencer abort. | > it would be useful to have a kind of scripted "abort" for when something goes wrong ...
How about having the sequencer switching from the aborted sequence file to the special "abort" sequence file? That
should be simple to implement if it is not already there.
K.O. |
1124
|
22 Oct 2015 |
Stefan Ritt | Suggestion | Feature Request: MIDAS sequencer abort. | > > it would be useful to have a kind of scripted "abort" for when something goes wrong ...
>
> How about having the sequencer switching from the aborted sequence file to the special "abort" sequence file? That
> should be simple to implement if it is not already there.
>
> K.O.
It's about the same effort if we jump to a specific label in a script or to a separate script. I just have to find some time to implement it.
Stefan |
1125
|
24 Oct 2015 |
Stefan Ritt | Suggestion | Feature Request: MIDAS sequencer abort. | > It would be useful to have it be specified for each script. Reason is that it's simpler, some scripts might only
> change a few sensitive settings, then on abort it only has to set back to "normal" what it touched to begin with.
> Also, the "normal" values are usually stored in local variables, so it's important to have those similarly accessible
> to the "Abort" portion of the script.
Agree. So I will put a special optional label, which will be accessed upon abort.
Stefan |
1150
|
10 Dec 2015 |
Amy Roberts | Suggestion | script command limited to 256 characters; remove limit? | Both the /Script and /CustomScript trees in the ODB allow users to trigger a
script via Midas - which silently truncates command strings longer than
256 characters.
I'd prefer that Midas place no limit on string length. Failing that, it would be
helpful to have character limits called out in the documentation
(https://midas.triumf.ca/MidasWiki/index.php//Script_ODB_tree#.3Cscript-name.3E_key_or_subtree,
https://midas.triumf.ca/MidasWiki/index.php//Customscript_ODB_tree).
As far as I can tell, odb.c allows arbitrarily large strings in the ODB data.
(Although key *names* are restricted to 256 characters.) I've submitted one
possible version of an arbitrary-length exec_script() as a pull request
(https://bitbucket.org/tmidas/midas/pull-requests/).
Am I misunderstanding any critical pieces? Does Midas intentionally treat
strings in the ODB as limited to 256 characters? |
|