 05 Jun 2020, Stefan Ritt, Suggestion, ODB++ API - documantion updates and odb view after key creation
|
Hi Marius,
your fix is good. Thanks for digging out this deep-lying issue, which would have haunted us if we would not fix it.
The problem is that in midas, the "BOOL" type is 4 Bytes long, actually modelled after MS Windows. Now I realized
that in c++, the "bool" type is only 1 Byte wide. So if we do the memcopy from a "c++ bool" to a "MIDAS BOOL", we
always copy four bytes, meaning that we copy three Bytes beyond the one-byte value of the c++ bool. So your fix
is absolutely correct, and I added it in one more space where we deal with bool arrays, where we need the same.
What I don't understand however is the fact why this fails for you. The ODB values are stored in the C union under
union {
...
bool m_bool;
double m_double;
std::string *m_string;
...
}
Now the C compiler puts all values at the lowest address, so m_bool is at offset zero, and the string pointer reaches
over all eight bytes (we are on 64-bit OS).
Now when I initialize this union in odbxx.h:66, I zero the string pointer which is the widest object:
u_odb() : m_string{} {};
which (at least on my Mac) sets all eight bytes to zero. If I then use the wrong code to set the bool value to the ODB
in odbxx.cxx:756, I do
db_set_data_index(... &u, rpc_tid_size(m_tid), ...);
so it copies four bytes (=rpc_tid_size(TID_BOOL)) to the ODB. The first byte should be the c++ bool value (0 or 1),
and the other three bytes should be zero from the initialization above. Apparently on your system, this is not
the case, and I would like you to double check it. Maybe there is another underlying problem which I don't understand
at the moment but which we better fix.
Otherwise the change is committed and your code should work. But we should not stop here! I really want to understand
why this is not working for you, maybe I miss something.
Best,
Stefan
> Hi Stefan,
>
> your test program was only working for me after I changed the following lines inside the odbxx.cpp
>
> diff --git a/src/odbxx.cxx b/src/odbxx.cxx
> index 24b5a135..48edfd15 100644
> --- a/src/odbxx.cxx
> +++ b/src/odbxx.cxx
> @@ -753,7 +753,12 @@ namespace midas {
> }
> } else {
> u_odb u = m_data[index];
> - status = db_set_data_index(m_hDB, m_hKey, &u, rpc_tid_size(m_tid), index, m_tid);
> + if (m_tid == TID_BOOL) {
> + BOOL ss = bool(u);
> + status = db_set_data_index(m_hDB, m_hKey, &ss, rpc_tid_size(m_tid), index, m_tid);
> + } else {
> + status = db_set_data_index(m_hDB, m_hKey, &u, rpc_tid_size(m_tid), index, m_tid);
> + }
> if (m_debug) {
> std::string s;
> u.get(s);
>
> Likely not the best fix but otherwise I was always getting after running the test program:
>
> [ODBEdit,INFO] Program ODBEdit on host localhost started
> [local:Default:S]/>cd Equipment/Test/Settings/Test_odb_api/
> key not found
> makoeppe@office ~/mu3e/online/online (git)-[odb++_api] % test_connect
> Created ODB key /Equipment/Test/Settings
> Created ODB key /Equipment/Test/Settings/Test_odb_api
> Created ODB key /Equipment/Test/Settings/Test_odb_api/Divider
> Set ODB key "/Equipment/Test/Settings/Test_odb_api/Divider" = 1000
> Created ODB key /Equipment/Test/Settings/Test_odb_api/Enable
> Set ODB key "/Equipment/Test/Settings/Test_odb_api/Enable" = false
> Get definition for ODB key "/Equipment/Test/Settings/Test_odb_api"
> Get definition for ODB key "/Equipment/Test/Settings/Test_odb_api/Divider"
> Get ODB key "/Equipment/Test/Settings/Test_odb_api/Divider": 1000
> Get definition for ODB key "/Equipment/Test/Settings/Test_odb_api/Enable"
> Get ODB key "/Equipment/Test/Settings/Test_odb_api/Enable": false
> Get definition for ODB key "/Equipment/Test/Settings/Test_odb_api/Divider"
> Get ODB key "/Equipment/Test/Settings/Test_odb_api/Divider": 1000
> Get definition for ODB key "/Equipment/Test/Settings/Test_odb_api/Enable"
> Get ODB key "/Equipment/Test/Settings/Test_odb_api/Enable": false
> Datagenerator Enable is Get ODB key "/Equipment/Test/Settings/Test_odb_api/Enable": false
> false
> makoeppe@office ~/mu3e/online/online (git)-[odb++_api] % odbedit
> [ODBEdit,INFO] Program ODBEdit on host localhost started
> [local:Default:S]/>cd Equipment/Test/Settings/Test_odb_api/
> [local:Default:S]Test_odb_api>ls
> Divider 1000
> Enable y
>
> > > I am getting back false. Which looks nice but when I look into the odb via the browser the value is actually "y" meaning true which is stange.
> > > I added my frontend where I cleaned all function leaving only the frontend_init() one where I create this key. Its a cuda program but since
> > > I clean everything no cuda function is called anymore. |
|
08 Jun 2020, Marius Koeppel, Suggestion, ODB++ API - documantion updates and odb view after key creation
|
Hi Stefan,
I agree with your explanation about the size of BOOL and bool.
I checked the program also on my Raspberry and there the old code works like on your mac. I don't really understand
why the behavior is different for my system. The initializing of the union should also work for my system.
At the moment I am using:
Arch Linux
Linux office 5.4.42-1-lts #1 SMP Wed, 20 May 2020 20:42:53 +0000 x86_64 GNU/Linux
gcc version 10.1.0 (GCC)
One thing which makes me a bit suspicious is that if I do:
u_odb u = m_data[index];
char dest[rpc_tid_size(m_tid)];
memcpy(dest, &u, rpc_tid_size(m_tid));
Clion tells me "Clang-Tidy: Undefined behavior, source object type 'midas::u_odb' is not TriviallyCopyable".
I am not sure if this is the problem since I am not so familiar with this TriviallyCopyable. I need to further investigate here.
So fare the update from my side.
Cheers,
Marius
> Hi Marius,
>
> your fix is good. Thanks for digging out this deep-lying issue, which would have haunted us if we would not fix it.
> The problem is that in midas, the "BOOL" type is 4 Bytes long, actually modelled after MS Windows. Now I realized
> that in c++, the "bool" type is only 1 Byte wide. So if we do the memcopy from a "c++ bool" to a "MIDAS BOOL", we
> always copy four bytes, meaning that we copy three Bytes beyond the one-byte value of the c++ bool. So your fix
> is absolutely correct, and I added it in one more space where we deal with bool arrays, where we need the same.
>
> What I don't understand however is the fact why this fails for you. The ODB values are stored in the C union under
>
> union {
> ...
> bool m_bool;
> double m_double;
> std::string *m_string;
> ...
> }
>
> Now the C compiler puts all values at the lowest address, so m_bool is at offset zero, and the string pointer reaches
> over all eight bytes (we are on 64-bit OS).
>
> Now when I initialize this union in odbxx.h:66, I zero the string pointer which is the widest object:
>
> u_odb() : m_string{} {};
>
> which (at least on my Mac) sets all eight bytes to zero. If I then use the wrong code to set the bool value to the ODB
> in odbxx.cxx:756, I do
>
> db_set_data_index(... &u, rpc_tid_size(m_tid), ...);
>
> so it copies four bytes (=rpc_tid_size(TID_BOOL)) to the ODB. The first byte should be the c++ bool value (0 or 1),
> and the other three bytes should be zero from the initialization above. Apparently on your system, this is not
> the case, and I would like you to double check it. Maybe there is another underlying problem which I don't understand
> at the moment but which we better fix.
>
> Otherwise the change is committed and your code should work. But we should not stop here! I really want to understand
> why this is not working for you, maybe I miss something.
>
> Best,
> Stefan
>
> > Hi Stefan,
> >
> > your test program was only working for me after I changed the following lines inside the odbxx.cpp
> >
> > diff --git a/src/odbxx.cxx b/src/odbxx.cxx
> > index 24b5a135..48edfd15 100644
> > --- a/src/odbxx.cxx
> > +++ b/src/odbxx.cxx
> > @@ -753,7 +753,12 @@ namespace midas {
> > }
> > } else {
> > u_odb u = m_data[index];
> > - status = db_set_data_index(m_hDB, m_hKey, &u, rpc_tid_size(m_tid), index, m_tid);
> > + if (m_tid == TID_BOOL) {
> > + BOOL ss = bool(u);
> > + status = db_set_data_index(m_hDB, m_hKey, &ss, rpc_tid_size(m_tid), index, m_tid);
> > + } else {
> > + status = db_set_data_index(m_hDB, m_hKey, &u, rpc_tid_size(m_tid), index, m_tid);
> > + }
> > if (m_debug) {
> > std::string s;
> > u.get(s);
> >
> > Likely not the best fix but otherwise I was always getting after running the test program:
> >
> > [ODBEdit,INFO] Program ODBEdit on host localhost started
> > [local:Default:S]/>cd Equipment/Test/Settings/Test_odb_api/
> > key not found
> > makoeppe@office ~/mu3e/online/online (git)-[odb++_api] % test_connect
> > Created ODB key /Equipment/Test/Settings
> > Created ODB key /Equipment/Test/Settings/Test_odb_api
> > Created ODB key /Equipment/Test/Settings/Test_odb_api/Divider
> > Set ODB key "/Equipment/Test/Settings/Test_odb_api/Divider" = 1000
> > Created ODB key /Equipment/Test/Settings/Test_odb_api/Enable
> > Set ODB key "/Equipment/Test/Settings/Test_odb_api/Enable" = false
> > Get definition for ODB key "/Equipment/Test/Settings/Test_odb_api"
> > Get definition for ODB key "/Equipment/Test/Settings/Test_odb_api/Divider"
> > Get ODB key "/Equipment/Test/Settings/Test_odb_api/Divider": 1000
> > Get definition for ODB key "/Equipment/Test/Settings/Test_odb_api/Enable"
> > Get ODB key "/Equipment/Test/Settings/Test_odb_api/Enable": false
> > Get definition for ODB key "/Equipment/Test/Settings/Test_odb_api/Divider"
> > Get ODB key "/Equipment/Test/Settings/Test_odb_api/Divider": 1000
> > Get definition for ODB key "/Equipment/Test/Settings/Test_odb_api/Enable"
> > Get ODB key "/Equipment/Test/Settings/Test_odb_api/Enable": false
> > Datagenerator Enable is Get ODB key "/Equipment/Test/Settings/Test_odb_api/Enable": false
> > false
> > makoeppe@office ~/mu3e/online/online (git)-[odb++_api] % odbedit
> > [ODBEdit,INFO] Program ODBEdit on host localhost started
> > [local:Default:S]/>cd Equipment/Test/Settings/Test_odb_api/
> > [local:Default:S]Test_odb_api>ls
> > Divider 1000
> > Enable y
> >
> > > > I am getting back false. Which looks nice but when I look into the odb via the browser the value is actually "y" meaning true which is stange.
> > > > I added my frontend where I cleaned all function leaving only the frontend_init() one where I create this key. Its a cuda program but since
> > > > I clean everything no cuda function is called anymore. |
|
16 Jun 2020, Marius Koeppel, Suggestion, ODB++ API - documantion updates and odb view after key creation
|
Hi Stefan,
I played around with the code a bit more and I found out that if I do:
midas::odb test_settings = {{"Enable", false}};
test_settings.connect("/Equipment/Test/Test", true);
The correct value ends up in the odb. In this case an u_odb instance is created
with a clean m_string. But if I run the other code an odb instanceo is created and
the values of m_data are set in
odbxx.h:
odb(std::initializer_list<std::pair<const char *, midas::odb>> list) : odb() {...
This values are comming from u_odb instances since the code does:
odbxx.h:
auto o = new midas::odb(element.second);
and then
odbxx.h:
odb(T v):odb() {
m_num_values = 1;
m_data = new u_odb[1]{v};
m_tid = m_data[0].get_tid();
m_data[0].set_parent(this);
}
and looking at
odbxx.h:
u_odb(bool v) : m_bool{v}, m_tid{TID_BOOL}, m_parent_odb{nullptr} {};
only m_bool is set for this instance meaning that only the first byte gets a value
(still having only 1 byte for bool in c++). If I check m_string inside the u_odb::get function
of this instance I am getting for a bool (I set false) stuff like 0x7f6633f67a00 and for an int
(I set the int to 1000) 0x7f66000003e8. Since the size of BOOL is larger I am getting the
wrong value. I checked this also on openSUSE having the same behavior.
Like you I am not getting this problem on my Mac. What compiler flags do you use on your Mac?
Cheers,
Marius |
|
23 Jun 2020, Stefan Ritt, Suggestion, ODB++ API - documantion updates and odb view after key creation
|
Hi Marius,
thanks for your help, you identified the problematic location. I changed that to
u_odb(bool v) : m_tid{TID_BOOL}, m_parent_odb{nullptr} {m_string = nullptr; m_bool = v;};
which should initialize the full 8 bytes of the u_odb union. I committed to develop. Can you
please give it a try?
Best,
Stefan
> and looking at
>
> odbxx.h:
> u_odb(bool v) : m_bool{v}, m_tid{TID_BOOL}, m_parent_odb{nullptr} {};
>
> only m_bool is set for this instance meaning that only the first byte gets a value
> (still having only 1 byte for bool in c++). If I check m_string inside the u_odb::get function
> of this instance I am getting for a bool (I set false) stuff like 0x7f6633f67a00 and for an int
> (I set the int to 1000) 0x7f66000003e8. Since the size of BOOL is larger I am getting the
> wrong value. I checked this also on openSUSE having the same behavior. |
|
24 Jun 2020, Marius Koeppel, Suggestion, ODB++ API - documantion updates and odb view after key creation
|
Hi Stefan,
now everything works well (Tested on: OpenSuse and Arch Linux) :)
Thank you for the fix.
Cheers,
Marius
> Hi Marius,
>
> thanks for your help, you identified the problematic location. I changed that to
>
> u_odb(bool v) : m_tid{TID_BOOL}, m_parent_odb{nullptr} {m_string = nullptr; m_bool = v;};
>
> which should initialize the full 8 bytes of the u_odb union. I committed to develop. Can you
> please give it a try?
>
> Best,
> Stefan
>
>
> > and looking at
> >
> > odbxx.h:
> > u_odb(bool v) : m_bool{v}, m_tid{TID_BOOL}, m_parent_odb{nullptr} {};
> >
> > only m_bool is set for this instance meaning that only the first byte gets a value
> > (still having only 1 byte for bool in c++). If I check m_string inside the u_odb::get function
> > of this instance I am getting for a bool (I set false) stuff like 0x7f6633f67a00 and for an int
> > (I set the int to 1000) 0x7f66000003e8. Since the size of BOOL is larger I am getting the
> > wrong value. I checked this also on openSUSE having the same behavior. |
 30 Sep 2023, Gennaro Tortone, Bug Report, ODB page and hex values 
|
Hi,
I was playing with MIDAS devel branch and I realized that
if I set an ODB INT32 key to a value using new ODB web interface
it is reported in parenthesis always as (0xFFFFFFFF);
I tested with different browser and result is the same while this
never happens in OldODB web interface...
Cheers,
Gennaro |
|
01 Oct 2023, Stefan Ritt, Bug Report, ODB page and hex values
|
Thanks for reporting this bug, I fixed it in the last commit.
Best,
Stefan |
|
27 Nov 2007, Stefan Ritt, Info, ODB links to array elements implemented
|
In revision 4090 I implemented ODB links to individual array elements. Now you
can have for example:
Key name Type #Val Size Last Opn Mode Value
---------------------------------------------------------------------------
array INT 10 4 2m 0 RWD
[0] 0
[1] 0
[2] 123
[3] 0
[4] 0
[5] 0
[6] 0
[7] 0
[8] 0
[9] 0
element2 -> /array[2] INT 1 4 3m 0 RWD 123
In this case, the link "element2" points to the third element of "array", but is
treated like a single value. This links are very useful for example for the
"Edit on start" parameters, which can now point to individual array elements.
The same is true for the "Links BOR" when the logger writes to a MySQL database.
This modification required major modifications in the ODB. I have carefully
tested the example experiment from the distribution to verify that everything is
fine, but I'm not 100% sure that I covered all possible situations. So if you
update to revision 4090+ and you observe some strange behavior related to links
in the ODB, please report.
There are following two new functions related to this change:
db_get_link()
db_get_link_data()
They are counterparts of db_get_key() and db_get_data(), respectively, but
without following links in the ODB. These functions are probably not of much use
outside odbedit and mhttpd, which are supposed to display links explicitly. Most
user applications want to follow links without even knowing that these are links. |
|
18 Jun 2020, Ruslan Podviianiuk, Forum, ODB key length
|
Hello,
I have a question about length of the name of ODB key.
Is it possible to create an ODB key containing more than 32 characters?
Thanks.
Ruslan |
|
18 Jun 2020, Stefan Ritt, Forum, ODB key length
|
No. But if you need more than 32 characters, you do something wrong. The
information you want to put into the ODB key name should probably be stored in
another string key or so.
Stefan
> Hello,
>
> I have a question about length of the name of ODB key.
> Is it possible to create an ODB key containing more than 32 characters?
>
> Thanks.
> Ruslan |
|
23 Mar 2022, Hunter Lowe, Forum, ODB has issue with example analyzer
|
Trying to play with midas file but I get error:
[Analyzer,ERROR] [odb.cxx:845:db_validate_name,ERROR] Invalid name "/Analyzer/Tests/low_sum/Rate [Hz]" passed to db_create_key_wlocked: should not contain "["
I'm not sure what sets the name so I'm not sure how to fix this.
Thanks |
|
04 May 2018, Francesco Renga, Forum, ODB full
|
Dear expert,
I'm developing a frontend and I'm getting this kind of error at each event:
10:14:56.564 2018/05/04 [Sample Frontend,ERROR] [odb.c:5911:db_set_data1,ERROR]
online database full
If I run the mem command in odbedit I get the result at the end of this post.
Notice that I need to use an event size which is significantly larger than the
default one. I don't know if it is relevant for this error. I have in the ODB:
/Experiment/MAX_EVENT_SIZE = 900000000
and in the frontend code:
/* maximum event size produced by this frontend */
INT max_event_size = 300000000;
/* maximum event size for fragmented events (EQ_FRAGMENTED) */
INT max_event_size_frag = 5 * 1024 * 1024;
/* buffer size to hold events */
INT event_buffer_size = 600000000;
Events seem to be properly stored in the output files, but I'm afraid I could
get some other problem.
Thank you for your help,
Francesco
-------------------------------------------------------------------------
Database header size is 0x21040, all following values are offset by this!
Key area 0x00000000 - 0x0007FFFF, size 524288 bytes
Data area 0x00080000 - 0x00100000, size 524288 bytes
Keylist:
--------
Free block at 0x00000B58, size 0x00000008, next 0x000053E0
Free block at 0x000053E0, size 0x00000008, next 0x00006560
Free block at 0x00006560, size 0x00079AA0, next 0x00000000
Free Key area: 498352 bytes out of 524288 bytes
Data:
-----
Free block at 0x000847F0, size 0x0007B810, next 0x00000000
Free Data area: 505872 bytes out of 524288 bytes
Free: 498352 (95.1%) keylist, 505872 (96.5%) data |
|
04 May 2018, Stefan Ritt, Forum, ODB full
|
Two options:
1) Do NOT send your events into the ODB. This is controlled via the flag RO_ODB in your frontend setting. For simple experiments with small events, it might make sense to copy each
event into the ODB for debugging, but if you have large events, this does not make sense. Use the "mdump" utility to check your events instead.
2) Increase the size of the ODB. See the first FAQ here: https://midas.triumf.ca/MidasWiki/index.php/FAQ
Stefan
> Dear expert,
> I'm developing a frontend and I'm getting this kind of error at each event:
>
> 10:14:56.564 2018/05/04 [Sample Frontend,ERROR] [odb.c:5911:db_set_data1,ERROR]
> online database full
>
> If I run the mem command in odbedit I get the result at the end of this post.
>
> Notice that I need to use an event size which is significantly larger than the
> default one. I don't know if it is relevant for this error. I have in the ODB:
>
> /Experiment/MAX_EVENT_SIZE = 900000000
>
> and in the frontend code:
>
> /* maximum event size produced by this frontend */
> INT max_event_size = 300000000;
>
> /* maximum event size for fragmented events (EQ_FRAGMENTED) */
> INT max_event_size_frag = 5 * 1024 * 1024;
>
> /* buffer size to hold events */
> INT event_buffer_size = 600000000;
>
> Events seem to be properly stored in the output files, but I'm afraid I could
> get some other problem.
>
> Thank you for your help,
> Francesco
>
> -------------------------------------------------------------------------
>
> Database header size is 0x21040, all following values are offset by this!
> Key area 0x00000000 - 0x0007FFFF, size 524288 bytes
> Data area 0x00080000 - 0x00100000, size 524288 bytes
>
> Keylist:
> --------
> Free block at 0x00000B58, size 0x00000008, next 0x000053E0
> Free block at 0x000053E0, size 0x00000008, next 0x00006560
> Free block at 0x00006560, size 0x00079AA0, next 0x00000000
>
> Free Key area: 498352 bytes out of 524288 bytes
>
> Data:
> -----
> Free block at 0x000847F0, size 0x0007B810, next 0x00000000
>
> Free Data area: 505872 bytes out of 524288 bytes
>
> Free: 498352 (95.1%) keylist, 505872 (96.5%) data |
|
20 Jul 2018, Konstantin Olchanski, Forum, ODB full
|
Concurrence.
Normally, MIDAS data events are saved to ODB (via RO_ODB into /eq/xxx/variables) to make them go into the midas history (/eq/xxx/common/history > 0).
If you do not want events to go into the history, but still want them saved to ODB, it should work (as long as ODB itself
is big enough), but you may run into other problems, specifically ODB free space fragmentation, when no matter how big ODB is, there is never
enough continuous free space for saving a large event. If it happens you will also see random "odb full" errors.
K.O.
> Two options:
>
> 1) Do NOT send your events into the ODB. This is controlled via the flag RO_ODB in your frontend setting. For simple experiments with small events, it might make sense to copy each
> event into the ODB for debugging, but if you have large events, this does not make sense. Use the "mdump" utility to check your events instead.
>
> 2) Increase the size of the ODB. See the first FAQ here: https://midas.triumf.ca/MidasWiki/index.php/FAQ
>
> Stefan
>
>
> > Dear expert,
> > I'm developing a frontend and I'm getting this kind of error at each event:
> >
> > 10:14:56.564 2018/05/04 [Sample Frontend,ERROR] [odb.c:5911:db_set_data1,ERROR]
> > online database full
> >
> > If I run the mem command in odbedit I get the result at the end of this post.
> >
> > Notice that I need to use an event size which is significantly larger than the
> > default one. I don't know if it is relevant for this error. I have in the ODB:
> >
> > /Experiment/MAX_EVENT_SIZE = 900000000
> >
> > and in the frontend code:
> >
> > /* maximum event size produced by this frontend */
> > INT max_event_size = 300000000;
> >
> > /* maximum event size for fragmented events (EQ_FRAGMENTED) */
> > INT max_event_size_frag = 5 * 1024 * 1024;
> >
> > /* buffer size to hold events */
> > INT event_buffer_size = 600000000;
> >
> > Events seem to be properly stored in the output files, but I'm afraid I could
> > get some other problem.
> >
> > Thank you for your help,
> > Francesco
> >
> > -------------------------------------------------------------------------
> >
> > Database header size is 0x21040, all following values are offset by this!
> > Key area 0x00000000 - 0x0007FFFF, size 524288 bytes
> > Data area 0x00080000 - 0x00100000, size 524288 bytes
> >
> > Keylist:
> > --------
> > Free block at 0x00000B58, size 0x00000008, next 0x000053E0
> > Free block at 0x000053E0, size 0x00000008, next 0x00006560
> > Free block at 0x00006560, size 0x00079AA0, next 0x00000000
> >
> > Free Key area: 498352 bytes out of 524288 bytes
> >
> > Data:
> > -----
> > Free block at 0x000847F0, size 0x0007B810, next 0x00000000
> >
> > Free Data area: 505872 bytes out of 524288 bytes
> >
> > Free: 498352 (95.1%) keylist, 505872 (96.5%) data |
|
13 Jan 2020, Peter Kunz, Forum, ODB dump format: json - events 0x8000 and 0x8001 missing
|
MIDAS version: 2.1
GIT revision: Tue Dec 31 17:40:14 2019 +0100 - midas-2019-09-i-1-gd93944ce-dirty on branch develop
/Logger/Channels/0/Settings
| ODB dump |
y |
| ODB dump format |
json |
With the settings above the file last.json generated for a new run is empty and the events 0x8000 and 0x8001 are missing in the .mid file.
When setting "ODB dump format" to "xml", events 0x8000 and 0x8001 are included in the .mid file, however, the file last.xml is not created.
|
|
13 Jan 2020, Konstantin Olchanski, Forum, ODB dump format: json - events 0x8000 and 0x8001 missing
|
(Please post messages in "plain" mode, they are much easier to answer)
Thank you for reporting this problem. I will try to reproduce it.
In addition, I will say a few words about your version of midas:
> GIT revision: midas-2019-09-i-1-gd93944ce-dirty on branch develop
I recommend that for production systems one used the tagged release versions of midas.
(i.e. see https://midas.triumf.ca/elog/Midas/1750).
(Your midas is "1 commit after the latest tag" - the "-1" in the git revision).
I apply bug fixes to both the release branch and the develop branch, but for you to get
these fixes, on the develop branch you will also "get" all the unrelated changes that may
come with new bugs. On the release branch, you will only get the bug fixes.
In your midas version it says "-dirty" which means that you have local modifications to the
midas sources. With luck those changes are not related to the bug that you see. (but I
cannot tell). You can do "git status" and "git diff" to see what the local changes are.
It is much better if bugs are reported against "clean" builds of MIDAS (no "-dirty").
K.O.
<p> </p>
<table align="center" cellspacing="1" style="border:1px solid #486090; width:98%">
<tbody>
<tr>
<td style="background-color:#486090">Peter Kunz wrote:</td>
</tr>
<tr>
<td style="background-color:#FFFFB0">
<p>MIDAS version: 2.1<br />
GIT revision: Tue Dec 31 17:40:14 2019 +0100 -
midas-2019-09-i-1-gd93944ce-dirty on branch develop</p>
<p>/Logger/Channels/0/Settings</p>
<table border="3" cellpadding="1" class="dialogTable">
<tbody>
<tr>
<td>ODB dump</td>
<td>y</td>
</tr>
<tr>
<td>ODB dump format</td>
<td>json</td>
</tr>
</tbody>
</table>
<p>With the settings above the file last.json generated for a new run is
empty and the events 0x8000 and 0x8001 are missing in the .mid file.</p>
<p>When setting "ODB dump format" to "xml", events
0x8000 and 0x8001 are included in the .mid file, however, the file last.xml is not created.
</p>
<p> </p>
<p> </p>
<p> </p>
</td>
</tr>
</tbody>
</table>
<p> </p> |
|
13 Jan 2020, Peter Kunz, Forum, ODB dump format: json - events 0x8000 and 0x8001 missing
|
Re: MIDAS versions
Thanks for pointing that out. I wasn't actually aware that there is a release branch and a development branch.
I was just following the installation instructions using
git clone https://bitbucket.org/tmidas/midas --recursive
Apparently that gave me the development branch. How can I get the release version?
(I think my version showed up a "dirty" because I played around with one of the examples. I didn't touch the actual source code.)
> (Please post messages in "plain" mode, they are much easier to answer)
>
> Thank you for reporting this problem. I will try to reproduce it.
>
> In addition, I will say a few words about your version of midas:
>
> > GIT revision: midas-2019-09-i-1-gd93944ce-dirty on branch develop
>
> I recommend that for production systems one used the tagged release versions of midas.
> (i.e. see https://midas.triumf.ca/elog/Midas/1750).
>
> (Your midas is "1 commit after the latest tag" - the "-1" in the git revision).
>
> I apply bug fixes to both the release branch and the develop branch, but for you to get
> these fixes, on the develop branch you will also "get" all the unrelated changes that may
> come with new bugs. On the release branch, you will only get the bug fixes.
>
> In your midas version it says "-dirty" which means that you have local modifications to the
> midas sources. With luck those changes are not related to the bug that you see. (but I
> cannot tell). You can do "git status" and "git diff" to see what the local changes are.
>
> It is much better if bugs are reported against "clean" builds of MIDAS (no "-dirty").
>
>
> K.O.
>
>
> <p> </p>
>
> <table align="center" cellspacing="1" style="border:1px solid #486090; width:98%">
> <tbody>
> <tr>
> <td style="background-color:#486090">Peter Kunz wrote:</td>
> </tr>
> <tr>
> <td style="background-color:#FFFFB0">
> <p>MIDAS version: 2.1<br />
> GIT revision: Tue Dec 31 17:40:14 2019 +0100 -
> midas-2019-09-i-1-gd93944ce-dirty on branch develop</p>
>
> <p>/Logger/Channels/0/Settings</p>
>
> <table border="3" cellpadding="1" class="dialogTable">
> <tbody>
> <tr>
> <td>ODB dump</td>
> <td>y</td>
> </tr>
> <tr>
> <td>ODB dump format</td>
> <td>json</td>
> </tr>
> </tbody>
> </table>
>
> <p>With the settings above the file last.json generated for a new run is
> empty and the events 0x8000 and 0x8001 are missing in the .mid file.</p>
>
> <p>When setting "ODB dump format" to "xml", events
> 0x8000 and 0x8001 are included in the .mid file, however, the file last.xml is not created.
> </p>
>
> <p> </p>
>
> <p> </p>
>
> <p> </p>
> </td>
> </tr>
> </tbody>
> </table>
>
> <p> </p> |
|
13 Jan 2020, Konstantin Olchanski, Forum, ODB dump format: json - events 0x8000 and 0x8001 missing
|
For using the release branch read the messages in this thread. Most of the time, the develop branch is fine, except when we are developing something
new, and the only way to tell is to watch the git activity on bitbucket or see the release branch announcements I post on the midas forum.
https://midas.triumf.ca/elog/Midas/1706
K.O.
> Re: MIDAS versions
>
> Thanks for pointing that out. I wasn't actually aware that there is a release branch and a development branch.
> I was just following the installation instructions using
>
> git clone https://bitbucket.org/tmidas/midas --recursive
>
> Apparently that gave me the development branch. How can I get the release version?
>
> (I think my version showed up a "dirty" because I played around with one of the examples. I didn't touch the actual source code.)
>
> > (Please post messages in "plain" mode, they are much easier to answer)
> >
> > Thank you for reporting this problem. I will try to reproduce it.
> >
> > In addition, I will say a few words about your version of midas:
> >
> > > GIT revision: midas-2019-09-i-1-gd93944ce-dirty on branch develop
> >
> > I recommend that for production systems one used the tagged release versions of midas.
> > (i.e. see https://midas.triumf.ca/elog/Midas/1750).
> >
> > (Your midas is "1 commit after the latest tag" - the "-1" in the git revision).
> >
> > I apply bug fixes to both the release branch and the develop branch, but for you to get
> > these fixes, on the develop branch you will also "get" all the unrelated changes that may
> > come with new bugs. On the release branch, you will only get the bug fixes.
> >
> > In your midas version it says "-dirty" which means that you have local modifications to the
> > midas sources. With luck those changes are not related to the bug that you see. (but I
> > cannot tell). You can do "git status" and "git diff" to see what the local changes are.
> >
> > It is much better if bugs are reported against "clean" builds of MIDAS (no "-dirty").
> >
> >
> > K.O.
> >
> >
> > <p> </p>
> >
> > <table align="center" cellspacing="1" style="border:1px solid #486090; width:98%">
> > <tbody>
> > <tr>
> > <td style="background-color:#486090">Peter Kunz wrote:</td>
> > </tr>
> > <tr>
> > <td style="background-color:#FFFFB0">
> > <p>MIDAS version: 2.1<br />
> > GIT revision: Tue Dec 31 17:40:14 2019 +0100 -
> > midas-2019-09-i-1-gd93944ce-dirty on branch develop</p>
> >
> > <p>/Logger/Channels/0/Settings</p>
> >
> > <table border="3" cellpadding="1" class="dialogTable">
> > <tbody>
> > <tr>
> > <td>ODB dump</td>
> > <td>y</td>
> > </tr>
> > <tr>
> > <td>ODB dump format</td>
> > <td>json</td>
> > </tr>
> > </tbody>
> > </table>
> >
> > <p>With the settings above the file last.json generated for a new run is
> > empty and the events 0x8000 and 0x8001 are missing in the .mid file.</p>
> >
> > <p>When setting "ODB dump format" to "xml", events
> > 0x8000 and 0x8001 are included in the .mid file, however, the file last.xml is not created.
> > </p>
> >
> > <p> </p>
> >
> > <p> </p>
> >
> > <p> </p>
> > </td>
> > </tr>
> > </tbody>
> > </table>
> >
> > <p> </p> |
|
24 Mar 2005, Stefan Ritt, Info, ODB dump format switched to XML
|
Dear midas users,
I have changed the ODB dump format to XML. As you might know, the logger writes
a special begin-of-run event to the .mid file which includes an ASCII dump of
the ODB. The same at the end-of-run. To read these ODB dumps back in offline
analysis, this requires setting up a ODB just to read back these values. In
order to avoid this, we switched the format to XML instead of the old ASCII
format. That way ROME can read the ODB dump and extract individual values from
it without setting up a shared memory.
A similar thing has been made for the ODB dumps to separate .odb files, which
are controlled by "/Logger/ODB Dump" and "/Logger/ODB Dump file". If the dump
file has the extension .xml, the file is dumped in XML format as well.
All the XML functionality is implemented in the new mxml.c/h library, which has
been added to the distribution, and which can be used in other projects as well
(XML configuration of ROODY?). It has already been successfully implemented in
ROME, so ROME is no longer dependent on libxml.
- Stefan |
|
29 Mar 2005, Stefan Ritt, Info, ODB dump format switched to XML
|
> All the XML functionality is implemented in the new mxml.c/h library, which has
> been added to the distribution, and which can be used in other projects as well
> (XML configuration of ROODY?). It has already been successfully implemented in
> ROME, so ROME is no longer dependent on libxml.
Since mxml.c/h is used in several projects (midas, ROME, elog), I separated it's
CVS tree. So in order to compile midas from scratch, you have to check out midas
AND mxml like
cd ~
cvs -d :ext:cvs@midas.psi.ch:/usr/local/cvsroot checkout midas
cvs -d :ext:cvs@midas.psi.ch:/usr/local/cvsroot checkout mxml
cd midas
make
so the "mxml" tree is ABOVE the "midas" tree. The midas Makefile has been adjusted
accordingly. If you decide to put the mxml somwhere else, you have to change
MXML_DIR in the Makefile accordingly.
- Stefan |
|