Seems to me like the problem happens less frequently, but I still see it (1 out of 5 or so). The fact that /Sequencer/Params/Value is empty tells me that the GUI 
has the problem and not the sequencer side.

Stefan

Thanks for the hint Stefan. I pushed a possible fix but I cannot test it since I cannot reproduce the issue.

> Ahh, that rings a bell:
> 
> 1) JS opens start dialog box
> 2) User enters parameters and presses start
> 3) JS writes parameters
> 4) JS starts sequencer
> 5) Sequencer copies parameters to variables
> 
> Now how do you handle 3) and 4). Just issue two mjsonrpc commands together? What then could happen is that 4) is executed before 3) and we get the garbage.
> You have to do 3) and WAIT for the return ("then" in the JS promise), and only then issue 4) from there.
> 
> Stefan

Ahh, that rings a bell:

1) JS opens start dialog box
2) User enters parameters and presses start
3) JS writes parameters
4) JS starts sequencer
5) Sequencer copies parameters to variables

Now how do you handle 3) and 4). Just issue two mjsonrpc commands together? What then could happen is that 4) is executed before 3) and we get the garbage.
You have to do 3) and WAIT for the return ("then" in the JS promise), and only then issue 4) from there.

Stefan

I have been able to reproduce the problem only once. From what I see, it seems that the Variables ODB tree is not initialized properly from the Param tree. Below are the messages from the failed run compared to a successful one. As far as I could see, the javascript code does not change anything in the Variables ODB tree (only monitors it). The actual changes are done by the sequencer program, or am I wrong?

Failed run:
16:14:25.849 2024/05/03 [Sequencer,INFO]  + 3 * 
16:14:24.722 2024/05/03 [Sequencer,INFO]  + 2 * 
16:14:23.594 2024/05/03 [Sequencer,INFO]  + 1 * 
16:14:23.592 2024/05/03 [Sequencer,INFO] Start of looping
16:14:23.591 2024/05/03 [Sequencer,INFO] (( - )/)
16:14:23.591 2024/05/03 [Sequencer,INFO] 
16:14:23.590 2024/05/03 [Sequencer,INFO] 
16:14:23.590 2024/05/03 [Sequencer,INFO] 
16:14:23.589 2024/05/03 [Sequencer,INFO] Parameters:
16:14:23.562 2024/05/03 [Sequencer,TALK] Sequencer started with script "testpars.msl".


Successful run:
16:15:37.472 2024/05/03 [Sequencer,INFO] 1.820000
16:15:37.471 2024/05/03 [Sequencer,INFO] Start of looping
16:15:37.471 2024/05/03 [Sequencer,INFO] 15
16:15:37.470 2024/05/03 [Sequencer,INFO] 0.020000
16:15:37.470 2024/05/03 [Sequencer,INFO] 2.100000
16:15:37.469 2024/05/03 [Sequencer,INFO] 1.800000
16:15:37.469 2024/05/03 [Sequencer,INFO] Parameters:
16:15:37.450 2024/05/03 [Sequencer,TALK] Sequencer started with script "testpars.msl".

The tinyexpr library I use to evaluate expressions does not support boolean operations. I would have to switch to the newer 
tineyexpr-plusplus version, which also has much richer functionality:

https://github.com/Blake-Madden/tinyexpr-plusplus/blob/tinyexpr%2B%2B/TinyExprChanges.md

Unfortunately it requires C++17, and at the moment we limit MIDAS to C++11, meaning we would break this requirement. I 
believe at the moment there are still some experiments (mainly at TRIUMF) which are stuck to older OS and therefore cannot 
switch to C++17, but hopefully this will change over time.

Stefan

Hello there, 
in our setup we use many variables with many different exceptions. Would it be possible to implement something like an 
IF or/and IF statement? I believe that this is currently not possible.
Best regards, 
Thomas Senger

It is pretty much the same as Stefan, I attach here the screenshots. Also in my case it works sometimes, and sometimes partially (one or 2 params, like in 
attachment 3).

> Could you please export and send me the /Sequencer ODB tree (or just /Sequencer/Param and /Sequencer/Variables) in both cases while the sequence is running. 
> 
> thanks,
> Zaher
> 
> 
> > Good afternoon,
> > 
> > After updating Midas to the latest develop commit 
> > (0f5436d901a1dfaf6da2b94e2d87f870e3611cf1) we found out a bug when starting 
> > sequencer. If we have a simple loop from start value to stop value and step 
> > size, just printing the value at each iteration, we see everything good (see 
> > first attachment). Then we included another script though, which contains 
> > several subroutines we defined for our detector, and we try to run the same 
> > script. Unfortunately after this the parameters seem uninitialized, and the 
> > value at each loop does not make sense (see second attachment). Also, sometimes 
> > when pressing run the set parameter window would pop-up, but sometimes not.
> > 
> > The script is this one:
> > 
> > >>>
> > COMMENT Test script to check for a specific bug
> > 
> > INCLUDE global_basic_functions
> > 
> > #CALL setup_paths
> > #CALL generate_DUT_params
> > 
> > PARAM lv_start, "Start of LV", 1.8
> > PARAM lv_stop, "Stop of LV", 2.1
> > PARAM lv_step, "Step of LV", 0.02
> > 
> > n_iterations = (($lv_stop - $lv_start)/$lv_step)
> > 
> > MSG "Parameters:"
> > MSG $lv_start
> > MSG $lv_stop
> > MSG $lv_step
> > MSG $n_iterations
> > 
> > MSG "Start of looping"
> > 
> > LOOP n, $n_iterations
> >    lv_now = $lv_start + $n * $lv_step
> >    MSG $lv_now
> >    WAIT SECONDS, 1
> > ENDLOOP
> > <<<
> > 
> > and the only difference comes from commenting the line:
> > 
> > >>>
> > INCLUDE global_basic_functions
> > <<<
> > 
> > as global_basic_functions is defined as a LIBRARY and it includes 75 (!) 
> > subroutines...
> > 
> > Is it possible that when loading a large script it messes up the loading of 
> > parameters?
> > 
> > Thank you very much,
> > Regards,
> > Luigi.

Ok, here is the complete code to reproduce the problem. Load parameter_test.msl which includes functions.msl. From the screenshot you see the variables containing 
garbage, and you also see that from the ODB screenshot. For completeness, I added Sequencer.json which contains the whole sequencer tree.

The interesting thing is that this works sometimes, and sometimes not. I'm not sure if this in the GUI or in the sequencer program, so we have to sort out who can 
fix it ;-)

Best,
Stefan

Could you please export and send me the /Sequencer ODB tree (or just /Sequencer/Param and /Sequencer/Variables) in both cases while the sequence is running. 

thanks,
Zaher


> Good afternoon,
> 
> After updating Midas to the latest develop commit 
> (0f5436d901a1dfaf6da2b94e2d87f870e3611cf1) we found out a bug when starting 
> sequencer. If we have a simple loop from start value to stop value and step 
> size, just printing the value at each iteration, we see everything good (see 
> first attachment). Then we included another script though, which contains 
> several subroutines we defined for our detector, and we try to run the same 
> script. Unfortunately after this the parameters seem uninitialized, and the 
> value at each loop does not make sense (see second attachment). Also, sometimes 
> when pressing run the set parameter window would pop-up, but sometimes not.
> 
> The script is this one:
> 
> >>>
> COMMENT Test script to check for a specific bug
> 
> INCLUDE global_basic_functions
> 
> #CALL setup_paths
> #CALL generate_DUT_params
> 
> PARAM lv_start, "Start of LV", 1.8
> PARAM lv_stop, "Stop of LV", 2.1
> PARAM lv_step, "Step of LV", 0.02
> 
> n_iterations = (($lv_stop - $lv_start)/$lv_step)
> 
> MSG "Parameters:"
> MSG $lv_start
> MSG $lv_stop
> MSG $lv_step
> MSG $n_iterations
> 
> MSG "Start of looping"
> 
> LOOP n, $n_iterations
>    lv_now = $lv_start + $n * $lv_step
>    MSG $lv_now
>    WAIT SECONDS, 1
> ENDLOOP
> <<<
> 
> and the only difference comes from commenting the line:
> 
> >>>
> INCLUDE global_basic_functions
> <<<
> 
> as global_basic_functions is defined as a LIBRARY and it includes 75 (!) 
> subroutines...
> 
> Is it possible that when loading a large script it messes up the loading of 
> parameters?
> 
> Thank you very much,
> Regards,
> Luigi.

> Ok, I implemented the float second wait function. Internally it works in ms, so the maximum resolution is 0.001 s.
> 
> Best,
> Stefan

Thank you, we will test this soon and let you know if we see any issues (but we're not expecting any).

Ok, I implemented the float second wait function. Internally it works in ms, so the maximum resolution is 0.001 s.

Best,
Stefan

> While I will do it, i'm not sure if this is what you want. If I understand correctly, some process gets triggered and then writes some values to the ODB, then the sequencer 
> should continue. Putting a wait there is dangerous. Maybe your process always takes like 10-20 ms, so you put a wait of let's say 100ms, and things are fine with you. Your 
> script runs many days, but then once in a while your machine is on heavy load because someone starts a web browser, and your process takes 110ms, and you script crashes.
> 
> I would rather go following path: put a "done" flag in the ODB, which is the last one which gets set by your process. Then the sequencer does a 
> 
> WAIT ODBvalue, /path/value, =, 1
> 
> which will work always, independend of the delay of your process.
> 
> Stefan

Our use case is pretty simple and I don't think is affected by the scenario you envision.  We want to turn on a setting in our equipment, and turn it off again some 0.2 s later.  We don't need msec timing.  So something like:

ODBSET /somekey 1   # this will cause a front-end to flip a bit in our hardware     
WAIT seconds, 0.2
ODBSET /somekey 0   # this will cause a front-end to reset a bit in our hardware 

It is true that if the load is high there could be a little delay, and the time that the bit is set will not be 0.2 seconds, but on average it should work, 
and it should be good enough we think.

Yes, we could also check an ODB key to see that something is done, but we'd still need the ability to wait for time intervals less than 1 second, which
right now doesn't exist.

While I will do it, i'm not sure if this is what you want. If I understand correctly, some process gets triggered and then writes some values to the ODB, then the sequencer 
should continue. Putting a wait there is dangerous. Maybe your process always takes like 10-20 ms, so you put a wait of let's say 100ms, and things are fine with you. Your 
script runs many days, but then once in a while your machine is on heavy load because someone starts a web browser, and your process takes 110ms, and you script crashes.

I would rather go following path: put a "done" flag in the ODB, which is the last one which gets set by your process. Then the sequencer does a 

WAIT ODBvalue, /path/value, =, 1

which will work always, independend of the delay of your process.

Stefan

> > This would work fine in principle, but isn't implemented in the current MIDAS sequencer as we understand it.  (We tried!) Is your proposal to rewrite the sequencer 
> > to allow fractional waits?  Right now the code seems to store the start_time as a DWORD and uses atoi to parse the wait time, and uses ss_time (which seems only get
> > the time to the nearest second) to fetch the time.
> 
> No it's not implemented, was just my idea. If it would work for you, I can implement it in the next couple of days.
> 
> Stefan

Yes, please!  Something like WAIT seconds, 0.01 would be perfect.

> This would work fine in principle, but isn't implemented in the current MIDAS sequencer as we understand it.  (We tried!) Is your proposal to rewrite the sequencer 
> to allow fractional waits?  Right now the code seems to store the start_time as a DWORD and uses atoi to parse the wait time, and uses ss_time (which seems only get
> the time to the nearest second) to fetch the time.

No it's not implemented, was just my idea. If it would work for you, I can implement it in the next couple of days.

Stefan

> I guess the simplest way to do that without breaking with existing code is to change the 
> second number to a float. So a
> 
> WAIT SECONDS, 1
> 
> will still work, and you can then write
> 
> WAIT SECONDS, 0.01
> 
> to get a 10 ms wait. Would that work for you?

This would work fine in principle, but isn't implemented in the current MIDAS sequencer as we understand it.  (We tried!) Is your proposal to rewrite the sequencer 
to allow fractional waits?  Right now the code seems to store the start_time as a DWORD and uses atoi to parse the wait time, and uses ss_time (which seems only get
the time to the nearest second) to fetch the time.

Good afternoon,

After updating Midas to the latest develop commit 
(0f5436d901a1dfaf6da2b94e2d87f870e3611cf1) we found out a bug when starting 
sequencer. If we have a simple loop from start value to stop value and step 
size, just printing the value at each iteration, we see everything good (see 
first attachment). Then we included another script though, which contains 
several subroutines we defined for our detector, and we try to run the same 
script. Unfortunately after this the parameters seem uninitialized, and the 
value at each loop does not make sense (see second attachment). Also, sometimes 
when pressing run the set parameter window would pop-up, but sometimes not.

The script is this one:

>>>
COMMENT Test script to check for a specific bug

INCLUDE global_basic_functions

#CALL setup_paths
#CALL generate_DUT_params

PARAM lv_start, "Start of LV", 1.8
PARAM lv_stop, "Stop of LV", 2.1
PARAM lv_step, "Step of LV", 0.02

n_iterations = (($lv_stop - $lv_start)/$lv_step)

MSG "Parameters:"
MSG $lv_start
MSG $lv_stop
MSG $lv_step
MSG $n_iterations

MSG "Start of looping"

LOOP n, $n_iterations
   lv_now = $lv_start + $n * $lv_step
   MSG $lv_now
   WAIT SECONDS, 1
ENDLOOP
<<<

and the only difference comes from commenting the line:

>>>
INCLUDE global_basic_functions
<<<

as global_basic_functions is defined as a LIBRARY and it includes 75 (!) 
subroutines...

Is it possible that when loading a large script it messes up the loading of 
parameters?

Thank you very much,
Regards,
Luigi.

I guess the simplest way to do that without breaking with existing code is to change the 
second number to a float. So a

WAIT SECONDS, 1

will still work, and you can then write

WAIT SECONDS, 0.01

to get a 10 ms wait. Would that work for you?

Stefan

Hello there,

I am working on a task that involves some ODB changes that happen within 20-500 
ms. The wait command for Midas Sequencer only works for > 1 second. As a 
workaround, I tried calling a python script that has a time.sleep() command, but 
the sequencer doesn't wait for the python script to terminate before moving to the 
next command. Obviously, I could just move the entire script to python, but that 
would cause further issues to us. Is there a way to have a wait that has precision 
in order of milliseconds from within the Midas Sequencer? If there is no Midas-
native methods for doing this, what workaround will you suggest to get this to 
work?

I now fully understand the MIDAS RPC code, had to add some debugging printfs, 
write some test code (odbedit test_rpc), catch and fix a few bugs.

Fixes for the bugs are now committed.

Small refactor of rpc_execute() should be committed soon, this removes the 
"goto" in the memory allocation of output buffer. Stefan's original code used a 
fixed size buffer, I later added allocation "as-neeed" but did not fully 
understand everything and implemented it as "if buffer too small, make it 
bigger, goto start over again".

After that, I can implement support for std::string and std::vector<char>.

The way it looks right now, the on-the-wire data format is flexible enough to 
make this change backward-compatible and allow MIDAS programs built with old 
MIDAS to continue connecting to the new MIDAS and vice-versa.

MIDAS RPC support for std::string should let us improve security by removing 
even more uses of fixed-size string buffers.

Support for std::vector<char> will allow removal of last places where 
MAX_EVENT_SIZE is used and simplify memory allocation in other "give me data" 
RPC calls, like RPC_JRPC and RPC_BRPC.

K.O.