Back Midas Rome Roody Rootana
  Midas DAQ System, Page 106 of 150  Not logged in ELOG logo
New entries since:Wed Dec 31 16:00:00 1969
ID Date Author Topic Subjectdown
  1714   29 Sep 2019 Thomas LindnerForumMIDAS interface for WAGASCI online monitor
Hi Pintaudi Giorgio,

I think that the ROOT THttpServer is an option. The ROOT tools are not perfect, but it is relatively easy to embed plots in custom MIDAS pages. I have a description of one way of doing this here:

https://midas.triumf.ca/MidasWiki/index.php/Rootana_javascript_displays

Thomas
  1716   29 Sep 2019 Konstantin OlchanskiForumMIDAS interface for WAGASCI online monitor
> online monitor would show (almost in real-time) the 
> gain, the dark noise, and the pedestal for all the channels, the 2D tracks 
> inside the detectors for each spill and so on.

Hmm... I now realize that the midas distribution does not include an example web page that 
integrates all these elements into one easy to understand html file.

I think an example page that would answer your questions and the questions from the other 
thread about starting/stopping runs, should include following elements:

- the general midas web page framework (the midas left-hand side menu, the top side 
status display, Stefan's new odb tags)
- buttons to start and stop runs (javascript code to call the run transition RPCs)
- embedded images for history display (old style gif and new style canvas)
- embedded images for ROOT histograms (via the ROOT http server and jsroot)
- code to live-update all these elements independantly from each other (to allow history 
plots and ROOT histograms to update at different frequencies).

As for the web page code for showing a mini-event-display, I think we do not know yet how 
to do - the event data lives inside the analyzer as C++ data structures, so somehow it 
needs to be encoded as json (this code is missing - but one can use the ROOT C++ to json 
encoder/streamer), needs to be transported to the web browser (we know how to do this) 
and at the end, plotting the json data on a canvas is the easy part.

I know some experiments have done all of this, and I think we should have such a pipeline 
available as part of the ROOTANA package. Maybe some day...

K.O.
  1717   29 Sep 2019 Pintaudi GiorgioForumMIDAS interface for WAGASCI online monitor
Dear Thomas and Konstantin,

thank you very much for the feedback. I found the ROOTANA javascript display a good source of 
information and references.

As Thomas said, maybe the simplest thing would be to use the ROOT THttpServer. Honestly, I do 
not think that ROOT was ever meant to act as an online monitor due to its wacky memory 
management and abysmal multithread support. In other words, I think that by using ROOT we would 
inevitably lose some performance. 

Perhaps there are better ways of achieving the same goal. For example, I was leaning towards a 
plotly.js based approach where I would encode a series of vectors in base64 strings (for better 
transmission performance), send them to the client through the MJSONRPC mechanism, decode them 
and then feed them to plotly.js. But in this case, I should study many new libraries 
(plotly.js, the library for the base64 encoding, the Gaussian fitting, etc...) and I do not 
have the time to do that now: "beam is coming".

So ROOT it is. I will use the ROOTANA javascript display as a reference. Do you happen to know 
who wrote that part? 

In that example, you have some "static" histograms that you keep always in memory, while in our 
case the number of channels is so big that we have to dynamically generate the histograms only 
when needed (when the user select a single channel).

Best regards
Giorgio
  1718   30 Sep 2019 Konstantin OlchanskiForumMIDAS interface for WAGASCI online monitor
> 
> As Thomas said, maybe the simplest thing would be to use the ROOT THttpServer. Honestly, I do 
> not think that ROOT was ever meant to act as an online monitor due to its wacky memory 
> management and abysmal multithread support. In other words, I think that by using ROOT we would 
> inevitably lose some performance.
>

Yes. The previous data analysis frameworks - PAW/PAW++/CERNlib (CERN), NOVA (TRIUMF) - certainly had
support for online monitoring. In CERNlib/PAW the histograms were stored in shared memory,
the analyzer running in the background was filling them, the PAW/PAW++ display was displaying
them "live". I was very surprised to find this function removed/not implemented in ROOT, given
that the same people were behind both projects (Rene Brun & co).

We tried to roll our own implementation of this in ROOTANA/ROODY, with mixed success.

I am glad the JSROOT project finally gained traction and web based "I can see the data" is now
available in ROOT.

> 
> Perhaps there are better ways of achieving the same goal. For example, I was leaning towards a 
> plotly.js based approach where 
>

There are many web/javascript graphics libraries out there, all have the weak spot - how do you
get your data into them?

Going forward, I see us standardizing on JSROOT: https://root.cern.ch/js/

>
> ... send them to the client through the MJSONRPC mechanism ...
>

I am not sure JSROOT have any support for interacting from the web page to the back-end analyzer. Perhaps
we can use the MIDAS MJSONRPC library for this. Hmm... (Note that the ROOT HTTP server is a derivative
of the mongoose web server library, which we use in mhttpd, so I already know how to work it)

>
> I would encode a series of vectors in base64 strings (for better transmission performance)
>

We looked into this when deciding on the data encoding for the midas history data. There is a tradeoff
between network use and cpu use - to save on the network, you try to reduce the data size by using
compressed binary data - to save on the CPU you try to minimize data encoding.

For history data, we gave up on binary json (extra decoding needed), gave up on text json (extra decoding 
needed), gave up on compression (extra cpu use for decompression) and use javascript native binary processing
("arraybuffer").

Our thinking is that network bandwidth is usually quite big and is getting bigger, but cpu resource is limited
and is expensive. (mobile devices seems to be stuck with ~2 GHz CPUs; cpu use means battery use and
battery capacity is limited, not improving quickly)

> 
> So ROOT it is. I will use the ROOTANA javascript display as a reference. Do you happen to know 
> who wrote that part? 
> 

Yes. See "Contact" at https://root.cern.ch/js/

>
> In that example, you have some "static" histograms that you keep always in memory, while in our 
> case the number of channels is so big that we have to dynamically generate the histograms only 
> when needed (when the user select a single channel).
> 

This requires interaction with the analyzer, requires some kind of RPC mechanism. I am now curious what jsroot 
have, also it would not be too hard to add the mjsonrpc library to rootana. Cooperation from ROOT multithreading
is not required: I can queue the RPC requests in a separate (thread safe, non-ROOT) buffer, then process
them in the ROOT main event loop (this is how the ROOTANA histogram server worked in the days when
ROOT had no multithread support at all).

K.O.
  831   30 Aug 2012 Raquel CastilloForumMIDAS in Windows
Hi,

I need to install MIDAS on a Windows system (Microsoft Windows Server 2003). 
The computer has the Microsoft Visual C++ 2010 Express version.
I have downloaded the MIDAS packages using the tarball mechanism. I have create 
the environment variables without problems and I have create the file           
%SystemRoot%\system32\exptab 
But when I try to build MIDAS and I do 
nmake -f makefile.nt
I have the following problem:
Microsoft (R) Program Maintenance Utility Version 10.00.30319.01
Copyright (C) Microsoft Corporation.  All rights reserved.

NMAKE : fatal error U1073: don't know how to make 'src/mhttpd.c'
Stop.

I don't understand this problem. Can anybody help me, please?

Thanks in advance!!!
  832   31 Aug 2012 Pierre-Andre AmaudruzForumMIDAS in Windows
Hi Raquel,

The makefile.nt has been corrected.
Obviously Midas on Windows has not been updated for quite a while.
mhttpd.c has been converted to c++ (mhttpd.cxx) as well as a couple of other 
applications.

Please give a try,  PAA

> Hi,
> 
> I need to install MIDAS on a Windows system (Microsoft Windows Server 2003). 
> The computer has the Microsoft Visual C++ 2010 Express version.
> I have downloaded the MIDAS packages using the tarball mechanism. I have create 
> the environment variables without problems and I have create the file           
> %SystemRoot%\system32\exptab 
> But when I try to build MIDAS and I do 
> nmake -f makefile.nt
> I have the following problem:
> Microsoft (R) Program Maintenance Utility Version 10.00.30319.01
> Copyright (C) Microsoft Corporation.  All rights reserved.
> 
> NMAKE : fatal error U1073: don't know how to make 'src/mhttpd.c'
> Stop.
> 
> I don't understand this problem. Can anybody help me, please?
> 
> Thanks in advance!!!
  840   23 Oct 2012 Raquel CastilloForumMIDAS in Windows
Hi Pierre-André, 

sorry for the long delay, another things keep me out of this computer.
Thanks a lot for correcting makefile.nt and the other applications!

Now I have try, downloading the MIDAS packages from the tarball mechanism, as
before,
and now it seems that the previous problems are solved. It remains only one small
problem, it is related to the odbedit.

I attach here the figure with the error that is reported by the computer. Is it
possible that is another file that needs to be updated? Can you help me with that?

Thanks a lot in advance!!!!



> Hi Raquel,
> 
> The makefile.nt has been corrected.
> Obviously Midas on Windows has not been updated for quite a while.
> mhttpd.c has been converted to c++ (mhttpd.cxx) as well as a couple of other 
> applications.
> 
> Please give a try,  PAA
> 
> > Hi,
> > 
> > I need to install MIDAS on a Windows system (Microsoft Windows Server 2003). 
> > The computer has the Microsoft Visual C++ 2010 Express version.
> > I have downloaded the MIDAS packages using the tarball mechanism. I have create 
> > the environment variables without problems and I have create the file           
> > %SystemRoot%\system32\exptab 
> > But when I try to build MIDAS and I do 
> > nmake -f makefile.nt
> > I have the following problem:
> > Microsoft (R) Program Maintenance Utility Version 10.00.30319.01
> > Copyright (C) Microsoft Corporation.  All rights reserved.
> > 
> > NMAKE : fatal error U1073: don't know how to make 'src/mhttpd.c'
> > Stop.
> > 
> > I don't understand this problem. Can anybody help me, please?
> > 
> > Thanks in advance!!!
Attachment 1: MIDAS_odbedit.bmp
  2936   01 Feb 2025 Pavel MuratBug ReportMIDAS history system not using the event timestamps ?
> I have a time series of slow control measurements in an ASCII format - 
> data records in a format (run_number, time, temperature, voltage1, ..., voltageN), 
> and, if possible, would like to convert them into a MIDAS history format. 
> 
> Making MIDAS events out of that data is easy, but is it possible to preserve 
> the time stamps?  - Logically, this boils down to whether it is possible to have  
> the event time set by a user frontend

It looks that the original question was not as naive as I expected and may be pointing to a subtle bug. 
I have implemented a python frontend - essentially a clone of 

https://bitbucket.org/tmidas/midas/src/develop/python/midas/frontend.py

reading the old slow control data and setting the event.header.timestamp's to some dates from the year of 2022. 

When I run MIDAS and read the "old slow control events", one event in 10 seconds, 
the MIDAS Event Dump utility shows the data with the correct event timestamps, from the year of 2022. 

However the history plots show the event parameters with the timestamps from Feb 01 2025 and the adjacent 
data points separated by 10 sec.

Is it possible that the history system uses its own timestamp setting instead of using timestamps from the event headers? 
- Under normal circumstances, the two should be very close, and that could've kept the issue hidden... 

-- thanks, regards, Pasha

UPDATE:  I attached the frontend code and the input data file it is reading. The data file should reside in the local directory
- the frontend code doesn't have everything fully automated for the test, 
  -- an integer field "/Mu2e/Offline/Ops/LastTime" would need to be created manually
  -- the history plots would need to be declared manually
Attachment 1: test_frontend.py
#!/usr/bin/env python

"""
Example of a basic midas frontend that has one periodic equipment.

See `examples/multi_frontend.py` for an example that uses more
features (frontend index, polled equipment, ODB settings etc). 
"""

import midas
import midas.frontend
import midas.event
import time, os, sys
from   datetime import datetime

#import TRACE
#TRACE_NAME = 'test_frontend.py'

class MyPeriodicEquipment(midas.frontend.EquipmentBase):
    """
    We define an "equipment" for each logically distinct task that this frontend
    performs. For example, you may have one equipment for reading data from a
    device and sending it to a midas buffer, and another equipment that updates
    summary statistics every 10s.
    
    Each equipment class you define should inherit from 
    `midas.frontend.EquipmentBase`, and should define a `readout_func` function.
    If you're creating a "polled" equipment (rather than a periodic one), you
    should also define a `poll_func` function in addition to `readout_func`.
    """
    def __init__(self, client):
        # The name of our equipment. This name will be used on the midas status
        # page, and our info will appear in /Equipment/MyPeriodicEquipment in
        # the ODB.
        equip_name = "MyPeriodicEquipment"
        
        # Define the "common" settings of a frontend. These will appear in
        # /Equipment/MyPeriodicEquipment/Common. The values you set here are
        # only used the very first time this frontend/equipment runs; after 
        # that the ODB settings are used.
        
        default_common              = midas.frontend.InitialEquipmentCommon()
        default_common.equip_type   = midas.EQ_PERIODIC
        default_common.buffer_name  = "SYSTEM"
        default_common.trigger_mask = 0
        default_common.event_id     = 1
        default_common.period_ms    = 100
        default_common.read_when    = midas.RO_ALWAYS
        default_common.log_history  = 1
        
        # You MUST call midas.frontend.EquipmentBase.__init__ in your equipment's __init__ method!
        midas.frontend.EquipmentBase.__init__(self, client, equip_name, default_common)
        
        # You can set the status of the equipment (appears in the midas status page)
        self.set_status("Initialized")
        self._verbose = 1;
        
# ---------------------------------------------------------------------
    def Print(self,Name,level,Message):
        if(level>self._verbose): return 0;
        now     = time.strftime('%Y/%m/%d %H:%M:%S',time.localtime(time.time()))
        message = now+' [ GridSubmit::'+Name+' ] '+Message
        print(message)
        
    def readout_func(self):
        """
        For a periodic equipment, this function will be called periodically
        (every 100ms in this case). It should return either a `cdms.event.Event`
        or None (if we shouldn't write an event).
        """

        last_time = self.client.odb_get('/Mu2e/Offline/Ops/LastTime')
        self.Print('readout_func',1,f'last_time:{last_time}')
        
#        cmd = 'dqmTool print-numbers --source 5 --value 6 --expand'
        cmd = 'cat val_nightly.csv'
        self.Print('readout_func',1,'executing cmd:%s'%cmd)
        out=os.popen(cmd).readlines()

        event = None;
        for line in out:
            # print(line)
            words = line.strip().split(',')
#------------------------------------------------------------------------------
# ['91', '5228', '0.0', '0  5', 'valNightly', 'reco', 'day', '0  6', 'ops', 'stats', 'CPU  11', '4', '0', '0', '0', '0', '2022-01-11 00:01:00-06:00', '2022-01-11 00:01:00-06:00']
#------------------------------------------------------------------------------
            # In this example, we just make a simple event with one bank.
            dt = datetime.strptime(words[17], "%Y-%m-%d %H:%M:%S%z");
            ts = dt.timestamp();
            if (ts > last_time) :
                data            = []
                print(words);
                event           = midas.event.Event()
                data.append(float(words[0]));
                data.append(float(words[1]));
                data.append(float(words[2]));

                event.create_bank("OFLN", midas.TID_FLOAT, data)
                event.header.timestamp = int(ts);
                self.client.odb_set('/Mu2e/Offline/Ops/LastTime',ts)
                print(f'SET_NEW last_time:{ts} dt:{dt}')
                
                # self.Print('readout_func',1,f'SET_NEW last_time:{ts} dt:{dt}')
                break;
        
        return event

class MyFrontend(midas.frontend.FrontendBase):
    """
    A frontend contains a collection of equipment.
    You can access self.client to access the ODB etc (see `midas.client.MidasClient`).
    """
    def __init__(self):
        # You must call __init__ from the base class.
        midas.frontend.FrontendBase.__init__(self, "myfe_name")
        
        # You can add equipment at any time before you call `run()`, but doing
        # it in __init__() seems logical.
        self.add_equipment(MyPeriodicEquipment(self.client))

        self._verbose = 1
        
    def begin_of_run(self, run_number):
        """
        This function will be called at the beginning of the run.
        You don't have to define it, but you probably should.
        You can access individual equipment classes through the `self.equipment`
        dict if needed.
        """
        self.set_all_equipment_status("Running", "greenLight")
        self.client.msg("Frontend has seen start of run number %d" % run_number)
        return midas.status_codes["SUCCESS"]
        
    def end_of_run(self, run_number):
        self.set_all_equipment_status("Finished", "greenLight")
        self.client.msg("Frontend has seen end of run number %d" % run_number)
        return midas.status_codes["SUCCESS"]
    
    def frontend_exit(self):
        """
        Most people won't need to define this function, but you can use
        it for final cleanup if needed.
        """
        print("Goodbye from user code!")
        
if __name__ == "__main__":
    # The main executable is very simple - just create the frontend object,
    # and call run() on it.
    print("EMOE");
#    TRACE.TRACE(7,'EMOEM',TRACE_NAME)
    with MyFrontend() as my_fe:
        my_fe.run()
Attachment 2: val_nightly.csv
21,4219,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  6,4,0,0,0,0,2022-01-06 00:01:00-06:00,2022-01-06 00:01:00-06:00
35,4999,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  7,4,0,0,0,0,2022-01-07 00:01:00-06:00,2022-01-07 00:01:00-06:00
49,4187,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  8,4,0,0,0,0,2022-01-08 00:01:00-06:00,2022-01-08 00:01:00-06:00
63,3875,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  9,4,0,0,0,0,2022-01-09 00:01:00-06:00,2022-01-09 00:01:00-06:00
77,4498,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  10,4,0,0,0,0,2022-01-10 00:01:00-06:00,2022-01-10 00:01:00-06:00
91,5228,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  11,4,0,0,0,0,2022-01-11 00:01:00-06:00,2022-01-11 00:01:00-06:00
105,4682,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  12,4,0,0,0,0,2022-01-12 00:01:00-06:00,2022-01-12 00:01:00-06:00
119,4364,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  13,4,0,0,0,0,2022-01-13 00:01:00-06:00,2022-01-13 00:01:00-06:00
133,5069,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  14,4,0,0,0,0,2022-01-14 00:01:00-06:00,2022-01-14 00:01:00-06:00
147,3848,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  15,4,0,0,0,0,2022-01-15 00:01:00-06:00,2022-01-15 00:01:00-06:00
161,5248,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  16,4,0,0,0,0,2022-01-16 00:01:00-06:00,2022-01-16 00:01:00-06:00
175,5276,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  17,4,0,0,0,0,2022-01-17 00:01:00-06:00,2022-01-17 00:01:00-06:00
189,5089,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  18,4,0,0,0,0,2022-01-18 00:01:00-06:00,2022-01-18 00:01:00-06:00
203,4789,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  19,4,0,0,0,0,2022-01-19 00:01:00-06:00,2022-01-19 00:01:00-06:00
217,4652,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  20,4,0,0,0,0,2022-01-20 00:01:00-06:00,2022-01-20 00:01:00-06:00
232,5343,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  23,5,0,0,0,0,2022-01-21 00:01:00-06:00,2022-01-21 00:01:00-06:00
246,4896,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  30,5,0,0,0,0,2022-01-22 00:01:00-06:00,2022-01-22 00:01:00-06:00
296,4812,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  46,5,0,0,0,0,2022-01-23 00:01:00-06:00,2022-01-23 00:01:00-06:00
310,4994,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  53,5,0,0,0,0,2022-01-24 00:01:00-06:00,2022-01-24 00:01:00-06:00
324,4397,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  60,5,0,0,0,0,2022-01-25 00:01:00-06:00,2022-01-25 00:01:00-06:00
338,5242,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  67,5,0,0,0,0,2022-01-26 00:01:00-06:00,2022-01-26 00:01:00-06:00
352,4915,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  74,5,0,0,0,0,2022-01-27 00:01:00-06:00,2022-01-27 00:01:00-06:00
366,4937,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  81,5,0,0,0,0,2022-01-28 00:01:00-06:00,2022-01-28 00:01:00-06:00
380,5096,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  88,5,0,0,0,0,2022-01-29 00:01:00-06:00,2022-01-29 00:01:00-06:00
394,5139,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  95,5,0,0,0,0,2022-01-30 00:01:00-06:00,2022-01-30 00:01:00-06:00
408,5434,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  102,5,0,0,0,0,2022-01-31 00:01:00-06:00,2022-01-31 00:01:00-06:00
422,5212,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  109,5,0,0,0,0,2022-02-01 00:01:00-06:00,2022-02-01 00:01:00-06:00
436,5144,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  116,5,0,0,0,0,2022-02-02 00:01:00-06:00,2022-02-02 00:01:00-06:00
450,4229,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  123,5,0,0,0,0,2022-02-03 00:01:00-06:00,2022-02-03 00:01:00-06:00
464,4760,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  130,5,0,0,0,0,2022-02-04 00:01:00-06:00,2022-02-04 00:01:00-06:00
478,4770,0.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  137,5,0,0,0,0,2022-02-05 00:01:00-06:00,2022-02-05 00:01:00-06:00
492,5171,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  144,5,0,0,0,0,2022-02-06 00:01:00-06:00,2022-02-06 00:01:00-06:00
650,4740,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  153,5,0,0,0,0,2022-02-10 00:01:00-06:00,2022-02-10 00:01:00-06:00
700,4923,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  160,5,0,0,0,0,2022-02-11 00:01:00-06:00,2022-02-11 00:01:00-06:00
750,0.0,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  167,5,0,0,0,0,2022-02-12 00:01:00-06:00,2022-02-12 00:01:00-06:00
800,0.0,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  174,5,0,0,0,0,2022-02-13 00:01:00-06:00,2022-02-13 00:01:00-06:00
850,4250,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  181,5,0,0,0,0,2022-02-14 00:01:00-06:00,2022-02-14 00:01:00-06:00
900,4668,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  188,5,0,0,0,0,2022-02-15 00:01:00-06:00,2022-02-15 00:01:00-06:00
950,4638,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  195,5,0,0,0,0,2022-02-16 00:01:00-06:00,2022-02-16 00:01:00-06:00
1000,5111,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  202,5,0,0,0,0,2022-02-17 00:01:00-06:00,2022-02-17 00:01:00-06:00
1050,3788,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  209,5,0,0,0,0,2022-02-18 00:01:00-06:00,2022-02-18 00:01:00-06:00
1100,5291,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  216,5,0,0,0,0,2022-02-19 00:01:00-06:00,2022-02-19 00:01:00-06:00
1150,4563,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  223,5,0,0,0,0,2022-02-20 00:01:00-06:00,2022-02-20 00:01:00-06:00
1200,4649,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  230,5,0,0,0,0,2022-02-21 00:01:00-06:00,2022-02-21 00:01:00-06:00
1250,0.0,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  237,5,0,0,0,0,2022-02-22 00:01:00-06:00,2022-02-22 00:01:00-06:00
1300,0.0,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  244,5,0,0,0,0,2022-02-23 00:01:00-06:00,2022-02-23 00:01:00-06:00
1350,0.0,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  251,5,0,0,0,0,2022-02-24 00:01:00-06:00,2022-02-24 00:01:00-06:00
1400,0.0,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  258,5,0,0,0,0,2022-02-25 00:01:00-06:00,2022-02-25 00:01:00-06:00
1450,0.0,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  265,5,0,0,0,0,2022-02-26 00:01:00-06:00,2022-02-26 00:01:00-06:00
1500,0.0,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  272,5,0,0,0,0,2022-02-27 00:01:00-06:00,2022-02-27 00:01:00-06:00
1550,0.0,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  279,5,0,0,0,0,2022-02-28 00:01:00-06:00,2022-02-28 00:01:00-06:00
1600,254,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  286,5,0,0,0,0,2022-03-01 00:01:00-06:00,2022-03-01 00:01:00-06:00
1650,203,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  293,5,0,0,0,0,2022-03-02 00:01:00-06:00,2022-03-02 00:01:00-06:00
1700,259,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  300,5,0,0,0,0,2022-03-03 00:01:00-06:00,2022-03-03 00:01:00-06:00
1750,255,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  307,5,0,0,0,0,2022-03-04 00:01:00-06:00,2022-03-04 00:01:00-06:00
1800,259,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  314,5,0,0,0,0,2022-03-05 00:01:00-06:00,2022-03-05 00:01:00-06:00
1850,284,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  321,5,0,0,0,0,2022-03-06 00:01:00-06:00,2022-03-06 00:01:00-06:00
1900,258,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  328,5,0,0,0,0,2022-03-07 00:01:00-06:00,2022-03-07 00:01:00-06:00
1950,284,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  335,5,0,0,0,0,2022-03-08 00:01:00-06:00,2022-03-08 00:01:00-06:00
2000,220,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  342,5,0,0,0,0,2022-03-09 00:01:00-06:00,2022-03-09 00:01:00-06:00
2050,197,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  349,5,0,0,0,0,2022-03-10 00:01:00-06:00,2022-03-10 00:01:00-06:00
2100,263,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  356,5,0,0,0,0,2022-03-11 00:01:00-06:00,2022-03-11 00:01:00-06:00
2150,283,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  363,5,0,0,0,0,2022-03-12 00:01:00-06:00,2022-03-12 00:01:00-06:00
2200,276,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  370,5,0,0,0,0,2022-03-13 00:01:00-06:00,2022-03-13 00:01:00-06:00
2250,212,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  377,5,0,0,0,0,2022-03-14 00:01:00-05:00,2022-03-14 00:01:00-05:00
2300,255,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  384,5,0,0,0,0,2022-03-15 00:01:00-05:00,2022-03-15 00:01:00-05:00
2350,212,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  391,5,0,0,0,0,2022-03-16 00:01:00-05:00,2022-03-16 00:01:00-05:00
2400,231,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  398,5,0,0,0,0,2022-03-17 00:01:00-05:00,2022-03-17 00:01:00-05:00
2450,209,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  405,5,0,0,0,0,2022-03-18 00:01:00-05:00,2022-03-18 00:01:00-05:00
2500,253,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  412,5,0,0,0,0,2022-03-19 00:01:00-05:00,2022-03-19 00:01:00-05:00
2550,233,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  419,5,0,0,0,0,2022-03-20 00:01:00-05:00,2022-03-20 00:01:00-05:00
2600,1661,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  426,5,0,0,0,0,2022-03-21 00:01:00-05:00,2022-03-21 00:01:00-05:00
2650,1940,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  433,5,0,0,0,0,2022-03-22 00:01:00-05:00,2022-03-22 00:01:00-05:00
2700,1177,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  440,5,0,0,0,0,2022-03-23 00:01:00-05:00,2022-03-23 00:01:00-05:00
2750,1537,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  447,5,0,0,0,0,2022-03-24 00:01:00-05:00,2022-03-24 00:01:00-05:00
2800,1908,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  454,5,0,0,0,0,2022-03-25 00:01:00-05:00,2022-03-25 00:01:00-05:00
2850,1290,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  461,5,0,0,0,0,2022-03-26 00:01:00-05:00,2022-03-26 00:01:00-05:00
2900,1286,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  468,5,0,0,0,0,2022-03-27 00:01:00-05:00,2022-03-27 00:01:00-05:00
2950,1944,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  475,5,0,0,0,0,2022-03-28 00:01:00-05:00,2022-03-28 00:01:00-05:00
3000,1365,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  482,5,0,0,0,0,2022-03-29 00:01:00-05:00,2022-03-29 00:01:00-05:00
3050,1252,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  489,5,0,0,0,0,2022-03-30 00:01:00-05:00,2022-03-30 00:01:00-05:00
3100,2010,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  496,5,0,0,0,0,2022-03-31 00:01:00-05:00,2022-03-31 00:01:00-05:00
3150,1998,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  503,5,0,0,0,0,2022-04-01 00:01:00-05:00,2022-04-01 00:01:00-05:00
3200,1342,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  510,5,0,0,0,0,2022-04-02 00:01:00-05:00,2022-04-02 00:01:00-05:00
3250,2158,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  517,5,0,0,0,0,2022-04-03 00:01:00-05:00,2022-04-03 00:01:00-05:00
3300,2033,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  524,5,0,0,0,0,2022-04-04 00:01:00-05:00,2022-04-04 00:01:00-05:00
3350,1935,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  531,5,0,0,0,0,2022-04-05 00:01:00-05:00,2022-04-05 00:01:00-05:00
3400,1860,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  538,5,0,0,0,0,2022-04-06 00:01:00-05:00,2022-04-06 00:01:00-05:00
3450,1279,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  545,5,0,0,0,0,2022-04-07 00:01:00-05:00,2022-04-07 00:01:00-05:00
3500,1340,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  552,5,0,0,0,0,2022-04-08 00:01:00-05:00,2022-04-08 00:01:00-05:00
3550,1746,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  559,5,0,0,0,0,2022-04-09 00:01:00-05:00,2022-04-09 00:01:00-05:00
3600,1312,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  566,5,0,0,0,0,2022-04-10 00:01:00-05:00,2022-04-10 00:01:00-05:00
3650,1802,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  573,5,0,0,0,0,2022-04-11 00:01:00-05:00,2022-04-11 00:01:00-05:00
3700,1997,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  580,5,0,0,0,0,2022-04-12 00:01:00-05:00,2022-04-12 00:01:00-05:00
3750,1657,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  587,5,0,0,0,0,2022-04-13 00:01:00-05:00,2022-04-13 00:01:00-05:00
3800,1684,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  594,5,0,0,0,0,2022-04-14 00:01:00-05:00,2022-04-14 00:01:00-05:00
3850,1811,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  601,5,0,0,0,0,2022-04-15 00:01:00-05:00,2022-04-15 00:01:00-05:00
3900,1959,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  608,5,0,0,0,0,2022-04-16 00:01:00-05:00,2022-04-16 00:01:00-05:00
3950,1501,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  615,5,0,0,0,0,2022-04-17 00:01:00-05:00,2022-04-17 00:01:00-05:00
4000,1434,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  622,5,0,0,0,0,2022-04-18 00:01:00-05:00,2022-04-18 00:01:00-05:00
4050,1286,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  629,5,0,0,0,0,2022-04-19 00:01:00-05:00,2022-04-19 00:01:00-05:00
4100,1598,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  636,5,0,0,0,0,2022-04-20 00:01:00-05:00,2022-04-20 00:01:00-05:00
4150,1747,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  643,5,0,0,0,0,2022-04-21 00:01:00-05:00,2022-04-21 00:01:00-05:00
4200,1254,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  650,5,0,0,0,0,2022-04-22 00:01:00-05:00,2022-04-22 00:01:00-05:00
4250,2049,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  657,5,0,0,0,0,2022-04-23 00:01:00-05:00,2022-04-23 00:01:00-05:00
4300,1905,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  664,5,0,0,0,0,2022-04-24 00:01:00-05:00,2022-04-24 00:01:00-05:00
4350,1423,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  671,5,0,0,0,0,2022-04-25 00:01:00-05:00,2022-04-25 00:01:00-05:00
4400,2151,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  678,5,0,0,0,0,2022-04-26 00:01:00-05:00,2022-04-26 00:01:00-05:00
4450,2005,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  685,5,0,0,0,0,2022-04-28 00:01:00-05:00,2022-04-28 00:01:00-05:00
4500,1831,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  692,5,0,0,0,0,2022-04-29 00:01:00-05:00,2022-04-29 00:01:00-05:00
4550,1820,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  699,5,0,0,0,0,2022-04-30 00:01:00-05:00,2022-04-30 00:01:00-05:00
4600,1827,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  706,5,0,0,0,0,2022-05-01 00:01:00-05:00,2022-05-01 00:01:00-05:00
4650,1891,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  713,5,0,0,0,0,2022-05-02 00:01:00-05:00,2022-05-02 00:01:00-05:00
4700,1265,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  720,5,0,0,0,0,2022-05-03 00:01:00-05:00,2022-05-03 00:01:00-05:00
4750,2047,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  727,5,0,0,0,0,2022-05-04 00:01:00-05:00,2022-05-04 00:01:00-05:00
4800,1646,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  734,5,0,0,0,0,2022-05-05 00:01:00-05:00,2022-05-05 00:01:00-05:00
4850,1940,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  741,5,0,0,0,0,2022-05-06 00:01:00-05:00,2022-05-06 00:01:00-05:00
4900,1831,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  748,5,0,0,0,0,2022-05-07 00:01:00-05:00,2022-05-07 00:01:00-05:00
4950,1217,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  755,5,0,0,0,0,2022-05-08 00:01:00-05:00,2022-05-08 00:01:00-05:00
5000,1623,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  762,5,0,0,0,0,2022-05-09 00:01:00-05:00,2022-05-09 00:01:00-05:00
5050,1183,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  769,5,0,0,0,0,2022-05-10 00:01:00-05:00,2022-05-10 00:01:00-05:00
5100,1702,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  776,5,0,0,0,0,2022-05-11 00:01:00-05:00,2022-05-11 00:01:00-05:00
5150,1623,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  783,5,0,0,0,0,2022-05-12 00:01:00-05:00,2022-05-12 00:01:00-05:00
5200,1328,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  790,5,0,0,0,0,2022-05-13 00:01:00-05:00,2022-05-13 00:01:00-05:00
5250,1530,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  797,5,0,0,0,0,2022-05-14 00:01:00-05:00,2022-05-14 00:01:00-05:00
5300,1792,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  804,5,0,0,0,0,2022-05-15 00:01:00-05:00,2022-05-15 00:01:00-05:00
5350,1989,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  811,5,0,0,0,0,2022-05-16 00:01:00-05:00,2022-05-16 00:01:00-05:00
5400,1723,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  818,5,0,0,0,0,2022-05-17 00:01:00-05:00,2022-05-17 00:01:00-05:00
5450,1740,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  825,5,0,0,0,0,2022-05-18 00:01:00-05:00,2022-05-18 00:01:00-05:00
5500,1822,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  832,5,0,0,0,0,2022-05-19 00:01:00-05:00,2022-05-19 00:01:00-05:00
5550,1812,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  839,5,0,0,0,0,2022-05-20 00:01:00-05:00,2022-05-20 00:01:00-05:00
5600,1310,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  846,5,0,0,0,0,2022-05-21 00:01:00-05:00,2022-05-21 00:01:00-05:00
5650,1574,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  853,5,0,0,0,0,2022-05-22 00:01:00-05:00,2022-05-22 00:01:00-05:00
5700,1371,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  860,5,0,0,0,0,2022-05-23 00:01:00-05:00,2022-05-23 00:01:00-05:00
5750,1822,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  867,5,0,0,0,0,2022-05-24 00:01:00-05:00,2022-05-24 00:01:00-05:00
5800,1375,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  874,5,0,0,0,0,2022-05-25 00:01:00-05:00,2022-05-25 00:01:00-05:00
5850,1737,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  881,5,0,0,0,0,2022-05-26 00:01:00-05:00,2022-05-26 00:01:00-05:00
5900,1407,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  888,5,0,0,0,0,2022-05-27 00:01:00-05:00,2022-05-27 00:01:00-05:00
5950,2086,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  895,5,0,0,0,0,2022-05-28 00:01:00-05:00,2022-05-28 00:01:00-05:00
6000,1937,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  902,5,0,0,0,0,2022-05-29 00:01:00-05:00,2022-05-29 00:01:00-05:00
6050,1145,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  909,5,0,0,0,0,2022-05-30 00:01:00-05:00,2022-05-30 00:01:00-05:00
6100,1588,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  916,5,0,0,0,0,2022-05-31 00:01:00-05:00,2022-05-31 00:01:00-05:00
6150,1775,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  923,5,0,0,0,0,2022-06-01 00:01:00-05:00,2022-06-01 00:01:00-05:00
6200,1083,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  930,5,0,0,0,0,2022-06-02 00:01:00-05:00,2022-06-02 00:01:00-05:00
6250,1914,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  937,5,0,0,0,0,2022-06-03 00:01:00-05:00,2022-06-03 00:01:00-05:00
6300,1789,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  944,5,0,0,0,0,2022-06-04 00:01:00-05:00,2022-06-04 00:01:00-05:00
6350,1231,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  951,5,0,0,0,0,2022-06-05 00:01:00-05:00,2022-06-05 00:01:00-05:00
6400,1848,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  958,5,0,0,0,0,2022-06-06 00:01:00-05:00,2022-06-06 00:01:00-05:00
6450,1907,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  965,5,0,0,0,0,2022-06-07 00:01:00-05:00,2022-06-07 00:01:00-05:00
6500,2039,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  972,5,0,0,0,0,2022-06-08 00:01:00-05:00,2022-06-08 00:01:00-05:00
6550,1972,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  979,5,0,0,0,0,2022-06-09 00:01:00-05:00,2022-06-09 00:01:00-05:00
6600,1816,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  986,5,0,0,0,0,2022-06-10 00:01:00-05:00,2022-06-10 00:01:00-05:00
6650,1983,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  993,5,0,0,0,0,2022-06-11 00:01:00-05:00,2022-06-11 00:01:00-05:00
6700,1695,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1000,5,0,0,0,0,2022-06-12 00:01:00-05:00,2022-06-12 00:01:00-05:00
6750,1762,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1007,5,0,0,0,0,2022-06-13 00:01:00-05:00,2022-06-13 00:01:00-05:00
6800,1828,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1014,5,0,0,0,0,2022-06-14 00:01:00-05:00,2022-06-14 00:01:00-05:00
6850,1911,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1021,5,0,0,0,0,2022-06-15 00:01:00-05:00,2022-06-15 00:01:00-05:00
6900,1690,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1028,5,0,0,0,0,2022-06-16 00:01:00-05:00,2022-06-16 00:01:00-05:00
6950,1895,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1035,5,0,0,0,0,2022-06-17 00:01:00-05:00,2022-06-17 00:01:00-05:00
7000,2086,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1042,5,0,0,0,0,2022-06-23 00:01:00-05:00,2022-06-23 00:01:00-05:00
7050,1775,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1049,5,0,0,0,0,2022-06-24 00:01:00-05:00,2022-06-24 00:01:00-05:00
7100,1992,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1056,5,0,0,0,0,2022-06-25 00:01:00-05:00,2022-06-25 00:01:00-05:00
7150,1435,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1063,5,0,0,0,0,2022-06-26 00:01:00-05:00,2022-06-26 00:01:00-05:00
7200,1273,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1070,5,0,0,0,0,2022-06-27 00:01:00-05:00,2022-06-27 00:01:00-05:00
7250,2321,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1077,5,0,0,0,0,2022-06-28 00:01:00-05:00,2022-06-28 00:01:00-05:00
7300,1666,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1084,5,0,0,0,0,2022-06-29 00:01:00-05:00,2022-06-29 00:01:00-05:00
7350,1598,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1091,5,0,0,0,0,2022-06-30 00:01:00-05:00,2022-06-30 00:01:00-05:00
7400,2090,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1098,5,0,0,0,0,2022-07-01 00:01:00-05:00,2022-07-01 00:01:00-05:00
7450,2066,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1105,5,0,0,0,0,2022-07-02 00:01:00-05:00,2022-07-02 00:01:00-05:00
7500,1253,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1112,5,0,0,0,0,2022-07-03 00:01:00-05:00,2022-07-03 00:01:00-05:00
7550,1367,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1119,5,0,0,0,0,2022-07-04 00:01:00-05:00,2022-07-04 00:01:00-05:00
7600,1737,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1126,5,0,0,0,0,2022-07-05 00:01:00-05:00,2022-07-05 00:01:00-05:00
7650,2065,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1133,5,0,0,0,0,2022-07-06 00:01:00-05:00,2022-07-06 00:01:00-05:00
7700,2065,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1140,5,0,0,0,0,2022-07-07 00:01:00-05:00,2022-07-07 00:01:00-05:00
7750,1548,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1147,5,0,0,0,0,2022-07-08 00:01:00-05:00,2022-07-08 00:01:00-05:00
7800,1849,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1154,5,0,0,0,0,2022-07-09 00:01:00-05:00,2022-07-09 00:01:00-05:00
7850,1175,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1161,5,0,0,0,0,2022-07-10 00:01:00-05:00,2022-07-10 00:01:00-05:00
7900,1430,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1168,5,0,0,0,0,2022-07-11 00:01:00-05:00,2022-07-11 00:01:00-05:00
7950,1716,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1175,5,0,0,0,0,2022-07-12 00:01:00-05:00,2022-07-12 00:01:00-05:00
8000,1854,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1182,5,0,0,0,0,2022-07-13 00:01:00-05:00,2022-07-13 00:01:00-05:00
8050,1338,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1189,5,0,0,0,0,2022-07-14 00:01:00-05:00,2022-07-14 00:01:00-05:00
8100,1474,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1196,5,0,0,0,0,2022-07-15 00:01:00-05:00,2022-07-15 00:01:00-05:00
8150,0.0,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1203,5,0,0,0,0,2022-07-16 00:01:00-05:00,2022-07-16 00:01:00-05:00
8200,1288,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1210,5,0,0,0,0,2022-07-17 00:01:00-05:00,2022-07-17 00:01:00-05:00
8250,1784,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1217,5,0,0,0,0,2022-07-18 00:01:00-05:00,2022-07-18 00:01:00-05:00
8300,1981,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1224,5,0,0,0,0,2022-07-19 00:01:00-05:00,2022-07-19 00:01:00-05:00
8350,1332,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1231,5,0,0,0,0,2022-07-20 00:01:00-05:00,2022-07-20 00:01:00-05:00
8400,1680,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1238,5,0,0,0,0,2022-07-21 00:01:00-05:00,2022-07-21 00:01:00-05:00
8450,1384,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1245,5,0,0,0,0,2022-07-22 00:01:00-05:00,2022-07-22 00:01:00-05:00
8500,1929,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1252,5,0,0,0,0,2022-07-23 00:01:00-05:00,2022-07-23 00:01:00-05:00
8550,1994,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1259,5,0,0,0,0,2022-07-24 00:01:00-05:00,2022-07-24 00:01:00-05:00
8600,1976,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1266,5,0,0,0,0,2022-07-25 00:01:00-05:00,2022-07-25 00:01:00-05:00
8650,1719,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1273,5,0,0,0,0,2022-07-26 00:01:00-05:00,2022-07-26 00:01:00-05:00
8700,1752,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1280,5,0,0,0,0,2022-07-27 00:01:00-05:00,2022-07-27 00:01:00-05:00
8750,1264,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1287,5,0,0,0,0,2022-07-28 00:01:00-05:00,2022-07-28 00:01:00-05:00
8800,1649,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1294,5,0,0,0,0,2022-07-29 00:01:00-05:00,2022-07-29 00:01:00-05:00
8850,1656,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1301,5,0,0,0,0,2022-07-30 00:01:00-05:00,2022-07-30 00:01:00-05:00
8900,1283,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1308,5,0,0,0,0,2022-07-31 00:01:00-05:00,2022-07-31 00:01:00-05:00
8950,1242,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1315,5,0,0,0,0,2022-08-01 00:01:00-05:00,2022-08-01 00:01:00-05:00
9000,1586,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1322,5,0,0,0,0,2022-08-02 00:01:00-05:00,2022-08-02 00:01:00-05:00
9050,1396,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1329,5,0,0,0,0,2022-08-03 00:01:00-05:00,2022-08-03 00:01:00-05:00
9100,2035,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1336,5,0,0,0,0,2022-08-04 00:01:00-05:00,2022-08-04 00:01:00-05:00
9150,1774,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1343,5,0,0,0,0,2022-08-05 00:01:00-05:00,2022-08-05 00:01:00-05:00
9200,1878,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1350,5,0,0,0,0,2022-08-06 00:01:00-05:00,2022-08-06 00:01:00-05:00
9250,1922,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1357,5,0,0,0,0,2022-08-07 00:01:00-05:00,2022-08-07 00:01:00-05:00
9300,2277,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1364,5,0,0,0,0,2022-08-08 00:01:00-05:00,2022-08-08 00:01:00-05:00
9350,1589,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1371,5,0,0,0,0,2022-08-09 00:01:00-05:00,2022-08-09 00:01:00-05:00
9400,0.0,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1378,5,0,0,0,0,2022-08-10 00:01:00-05:00,2022-08-10 00:01:00-05:00
9450,1689,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1385,5,0,0,0,0,2022-08-11 00:01:00-05:00,2022-08-11 00:01:00-05:00
9500,1995,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1392,5,0,0,0,0,2022-08-12 00:01:00-05:00,2022-08-12 00:01:00-05:00
9514,0.0,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1399,5,0,0,0,0,2022-08-13 00:01:00-05:00,2022-08-13 00:01:00-05:00
9564,1247,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1406,5,0,0,0,0,2022-08-14 00:01:00-05:00,2022-08-14 00:01:00-05:00
9614,1614,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1413,5,0,0,0,0,2022-08-15 00:01:00-05:00,2022-08-15 00:01:00-05:00
9664,1489,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1420,5,0,0,0,0,2022-08-16 00:01:00-05:00,2022-08-16 00:01:00-05:00
9714,1330,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1427,5,0,0,0,0,2022-08-17 00:01:00-05:00,2022-08-17 00:01:00-05:00
9764,1289,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1434,5,0,0,0,0,2022-08-18 00:01:00-05:00,2022-08-18 00:01:00-05:00
9814,1877,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1441,5,0,0,0,0,2022-08-19 00:01:00-05:00,2022-08-19 00:01:00-05:00
9864,1271,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1448,5,0,0,0,0,2022-08-20 00:01:00-05:00,2022-08-20 00:01:00-05:00
9923,1753,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1455,5,0,0,0,0,2022-08-21 00:01:00-05:00,2022-08-21 00:01:00-05:00
9974,1372,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1463,5,0,0,0,0,2022-08-22 00:01:00-05:00,2022-08-22 00:01:00-05:00
10025,1308,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1471,5,0,0,0,0,2022-08-23 00:01:00-05:00,2022-08-23 00:01:00-05:00
10076,1240,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1479,5,0,0,0,0,2022-08-24 00:01:00-05:00,2022-08-24 00:01:00-05:00
10127,1837,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1487,5,0,0,0,0,2022-08-25 00:01:00-05:00,2022-08-25 00:01:00-05:00
10178,1422,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1495,5,0,0,0,0,2022-08-26 00:01:00-05:00,2022-08-26 00:01:00-05:00
10229,1281,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1503,5,0,0,0,0,2022-08-27 00:01:00-05:00,2022-08-27 00:01:00-05:00
10280,1449,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1511,5,0,0,0,0,2022-08-28 00:01:00-05:00,2022-08-28 00:01:00-05:00
10331,1258,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1519,5,0,0,0,0,2022-08-29 00:01:00-05:00,2022-08-29 00:01:00-05:00
10382,1521,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1527,5,0,0,0,0,2022-08-30 00:01:00-05:00,2022-08-30 00:01:00-05:00
10433,1794,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1535,5,0,0,0,0,2022-08-31 00:01:00-05:00,2022-08-31 00:01:00-05:00
10484,1020,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1543,5,0,0,0,0,2022-09-01 00:01:00-05:00,2022-09-01 00:01:00-05:00
10535,1141,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1551,5,0,0,0,0,2022-09-02 00:01:00-05:00,2022-09-02 00:01:00-05:00
10586,1060,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1559,5,0,0,0,0,2022-09-03 00:01:00-05:00,2022-09-03 00:01:00-05:00
10637,1579,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1567,5,0,0,0,0,2022-09-04 00:01:00-05:00,2022-09-04 00:01:00-05:00
10688,961,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1575,5,0,0,0,0,2022-09-05 00:01:00-05:00,2022-09-05 00:01:00-05:00
10739,1251,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1583,5,0,0,0,0,2022-09-06 00:01:00-05:00,2022-09-06 00:01:00-05:00
10790,1748,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1591,5,0,0,0,0,2022-09-07 00:01:00-05:00,2022-09-07 00:01:00-05:00
10841,1475,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1599,5,0,0,0,0,2022-09-08 00:01:00-05:00,2022-09-08 00:01:00-05:00
10892,1229,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1607,5,0,0,0,0,2022-09-09 00:01:00-05:00,2022-09-09 00:01:00-05:00
10943,1082,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1615,5,0,0,0,0,2022-09-10 00:01:00-05:00,2022-09-10 00:01:00-05:00
10994,1768,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1623,5,0,0,0,0,2022-09-11 00:01:00-05:00,2022-09-11 00:01:00-05:00
11045,1111,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1631,5,0,0,0,0,2022-09-12 00:01:00-05:00,2022-09-12 00:01:00-05:00
11096,1221,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1639,5,0,0,0,0,2022-09-13 00:01:00-05:00,2022-09-13 00:01:00-05:00
11147,1649,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1647,5,0,0,0,0,2022-09-14 00:01:00-05:00,2022-09-14 00:01:00-05:00
11198,1565,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1655,5,0,0,0,0,2022-09-15 00:01:00-05:00,2022-09-15 00:01:00-05:00
11249,1492,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1663,5,0,0,0,0,2022-09-16 00:01:00-05:00,2022-09-16 00:01:00-05:00
11300,1215,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1671,5,0,0,0,0,2022-09-17 00:01:00-05:00,2022-09-17 00:01:00-05:00
11351,990,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1679,5,0,0,0,0,2022-09-18 00:01:00-05:00,2022-09-18 00:01:00-05:00
11402,1220,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1687,5,0,0,0,0,2022-09-19 00:01:00-05:00,2022-09-19 00:01:00-05:00
11453,1737,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1695,5,0,0,0,0,2022-09-20 00:01:00-05:00,2022-09-20 00:01:00-05:00
11504,1786,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1703,5,0,0,0,0,2022-09-21 00:01:00-05:00,2022-09-21 00:01:00-05:00
11555,1658,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1711,5,0,0,0,0,2022-09-22 00:01:00-05:00,2022-09-22 00:01:00-05:00
11606,1580,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1719,5,0,0,0,0,2022-09-23 00:01:00-05:00,2022-09-23 00:01:00-05:00
11657,1003,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1727,5,0,0,0,0,2022-09-24 00:01:00-05:00,2022-09-24 00:01:00-05:00
11708,1630,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1735,5,0,0,0,0,2022-09-25 00:01:00-05:00,2022-09-25 00:01:00-05:00
11759,1535,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1743,5,0,0,0,0,2022-09-26 00:01:00-05:00,2022-09-26 00:01:00-05:00
11810,1242,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1751,5,0,0,0,0,2022-09-27 00:01:00-05:00,2022-09-27 00:01:00-05:00
11861,1601,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1759,5,0,0,0,0,2022-09-28 00:01:00-05:00,2022-09-28 00:01:00-05:00
11912,1831,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1767,5,0,0,0,0,2022-09-29 00:01:00-05:00,2022-09-29 00:01:00-05:00
11963,1830,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1775,5,0,0,0,0,2022-09-30 00:01:00-05:00,2022-09-30 00:01:00-05:00
12014,1819,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1783,5,0,0,0,0,2022-10-01 00:01:00-05:00,2022-10-01 00:01:00-05:00
12065,1533,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1791,5,0,0,0,0,2022-10-02 00:01:00-05:00,2022-10-02 00:01:00-05:00
12116,1663,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1799,5,0,0,0,0,2022-10-03 00:01:00-05:00,2022-10-03 00:01:00-05:00
12167,1513,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1807,5,0,0,0,0,2022-10-04 00:01:00-05:00,2022-10-04 00:01:00-05:00
12218,1235,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1815,5,0,0,0,0,2022-10-05 00:01:00-05:00,2022-10-05 00:01:00-05:00
12269,1385,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1823,5,0,0,0,0,2022-10-06 00:01:00-05:00,2022-10-06 00:01:00-05:00
12320,1521,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1831,5,0,0,0,0,2022-10-07 00:01:00-05:00,2022-10-07 00:01:00-05:00
12371,1613,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1839,5,0,0,0,0,2022-10-08 00:01:00-05:00,2022-10-08 00:01:00-05:00
12422,1282,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1847,5,0,0,0,0,2022-10-09 00:01:00-05:00,2022-10-09 00:01:00-05:00
12473,1174,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1855,5,0,0,0,0,2022-10-10 00:01:00-05:00,2022-10-10 00:01:00-05:00
12524,1511,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1863,5,0,0,0,0,2022-10-11 00:01:00-05:00,2022-10-11 00:01:00-05:00
12575,1510,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1871,5,0,0,0,0,2022-10-12 00:01:00-05:00,2022-10-12 00:01:00-05:00
12626,1110,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1879,5,0,0,0,0,2022-10-13 00:01:00-05:00,2022-10-13 00:01:00-05:00
12677,1107,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1887,5,0,0,0,0,2022-10-14 00:01:00-05:00,2022-10-14 00:01:00-05:00
12728,1129,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1895,5,0,0,0,0,2022-10-15 00:01:00-05:00,2022-10-15 00:01:00-05:00
12779,1194,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1903,5,0,0,0,0,2022-10-16 00:01:00-05:00,2022-10-16 00:01:00-05:00
12830,1028,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1911,5,0,0,0,0,2022-10-17 00:01:00-05:00,2022-10-17 00:01:00-05:00
12881,1036,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1919,5,0,0,0,0,2022-10-18 00:01:00-05:00,2022-10-18 00:01:00-05:00
12932,1160,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1927,5,0,0,0,0,2022-10-19 00:01:00-05:00,2022-10-19 00:01:00-05:00
12983,1488,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1935,5,0,0,0,0,2022-10-20 00:01:00-05:00,2022-10-20 00:01:00-05:00
13034,1693,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1943,5,0,0,0,0,2022-10-21 00:01:00-05:00,2022-10-21 00:01:00-05:00
13085,1658,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1951,5,0,0,0,0,2022-10-22 00:01:00-05:00,2022-10-22 00:01:00-05:00
13136,1130,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1959,5,0,0,0,0,2022-10-23 00:01:00-05:00,2022-10-23 00:01:00-05:00
13187,1623,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1967,5,0,0,0,0,2022-10-24 00:01:00-05:00,2022-10-24 00:01:00-05:00
13238,1528,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1975,5,0,0,0,0,2022-10-25 00:01:00-05:00,2022-10-25 00:01:00-05:00
13289,1714,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1983,5,0,0,0,0,2022-10-26 00:01:00-05:00,2022-10-26 00:01:00-05:00
13340,1441,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1991,5,0,0,0,0,2022-10-28 00:01:00-05:00,2022-10-28 00:01:00-05:00
13391,1478,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  1999,5,0,0,0,0,2022-10-29 00:01:00-05:00,2022-10-29 00:01:00-05:00
13442,1346,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  2007,5,0,0,0,0,2022-10-30 00:01:00-05:00,2022-10-30 00:01:00-05:00
13493,1568,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  2015,5,0,0,0,0,2022-10-31 00:01:00-05:00,2022-10-31 00:01:00-05:00
13544,1076,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  2023,5,0,0,0,0,2022-11-01 00:01:00-05:00,2022-11-01 00:01:00-05:00
13595,1462,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  2031,5,0,0,0,0,2022-11-02 00:01:00-05:00,2022-11-02 00:01:00-05:00
13646,1365,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  2039,5,0,0,0,0,2022-11-03 00:01:00-05:00,2022-11-03 00:01:00-05:00
13697,1491,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  2047,5,0,0,0,0,2022-11-04 00:01:00-05:00,2022-11-04 00:01:00-05:00
13748,1486,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  2055,5,0,0,0,0,2022-11-05 00:01:00-05:00,2022-11-05 00:01:00-05:00
13799,1576,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  2063,5,0,0,0,0,2022-11-06 00:01:00-05:00,2022-11-06 00:01:00-05:00
13850,1121,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  2071,5,0,0,0,0,2022-11-07 00:01:00-06:00,2022-11-07 00:01:00-06:00
13901,0.0,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  2079,5,0,0,0,0,2022-11-08 00:01:00-06:00,2022-11-08 00:01:00-06:00
13952,1401,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  2087,5,0,0,0,0,2022-11-09 00:01:00-06:00,2022-11-09 00:01:00-06:00
14003,1099,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  2095,5,0,0,0,0,2022-11-10 00:01:00-06:00,2022-11-10 00:01:00-06:00
14054,1134,150.0,0  5,valNightly,reco,day,0  6,ops,stats,CPU  2103,5,0,0,0,0,2022-11-11 00:01:00-06:00,2022-11-11 00:01:00-06:00
... 783 more lines ...
  3012   01 Apr 2025 Pavel MuratBug ReportMIDAS history system not using the event timestamps ?
Dear MIDAS experts, 

I confirm that when writing out history files corresponding to the slow control event data, 
MIDAS history system timestamps the data not with the event time coming from the event data, 
but with the current time determined by the program - 

https://bitbucket.org/tmidas/midas/src/293d27fad0c87c80c4ed7b94b5c40ba1e150bea4/progs/mlogger.cxx#lines-5321

where 'now' is defined as  

time_t now = time(NULL);

I'm looking for a way to timestamp the history data with the event time - that is important 
for HEP applications outside the DAQ domain. Yes, MIDAS infrastructure is very well suited for that, 
there could have a number of such applications, and experiments could significantly benefit from that.

So I'm wondering whether the implementation is a design choice made or it could be changed. 

The change itself and especially its validation may require a non-negligible amount of work - I'd be happy to contribute.

Any insight much appreciated. 

-- thanks, regards, Pasha
  3020   01 Apr 2025 Konstantin OlchanskiBug ReportMIDAS history system not using the event timestamps ?
> I confirm that when writing out history files corresponding to the slow control event data, 
> MIDAS history system timestamps the data not with the event time coming from the event data, 
> but with the current time determined by [mlogger].

This is correct. The timestamp in the history file is the mlogger timestamp.

In theory we could use the ODB "last_written" timestamp, but in practice,
timestamps are 1 second granularity and the difference between the two
timestamps normally would be less than 1 second. (time to react to db_watch()).

But ODB last_written also is not the data timestamp. For remote connected clients
it includes the mserver communication delay.

What is the data timestamp, only the user knows - for some FPGA based equipments,
I can see the data timestamp being read from an FPGA register together with the data.

But back to earth.

For making history plots, 1 second granularity with a small (a few seconds) delay should be okey,
and I think the mserver timestamp is good enough.

For data analysis, you are reading history data from a history data file and you are
not constrained to using the MIDAS timestamp.

You can always include your "true" data timestamp as the first value in your data.

We do this in felaview for writing labview data to midas history in the ALPHA antihydrogen experiment at CERN.

This also anticipates your next request, can we have millisecond, microsecond, nanosecond history timestamps:
since you define your "true" data timestamp, you an make it anything you want. (I use "double" time in seconds,
64-bit IEEE-754 "double" has enough precision for microsecond granularity. FPGA based devices can have timestamps
with 10 ns or 8 ns granularity, in this case a uint64_t clock counter could be more appropriate).

K.O.
  3022   02 Apr 2025 Pavel MuratBug ReportMIDAS history system not using the event timestamps ?
> You can always include your "true" data timestamp as the first value in your data.

Are you saying that if the first data word of a history event were a timestamp, 
the MIDAS history system, when plotting the time dependencies, would use that timestamp 
instead of the mlogger timestamp?  

if that is true, what tells MIDAS that the first data word is the timestamp? 

I couldn't find a discussion of that on the page describing the history system - 

 https://daq00.triumf.ca/MidasWiki/index.php/History_System#Frontend_history_event

- perhaps I should be looking at a different page?

-- thanks again, regards, Pasha 
  3023   02 Apr 2025 Konstantin OlchanskiBug ReportMIDAS history system not using the event timestamps ?
> > You can always include your "true" data timestamp as the first value in your data.
> 
> Are you saying that if the first data word of a history event were a timestamp, 
> the MIDAS history system, when plotting the time dependencies, would use that timestamp 
> instead of the mlogger timestamp?
>

you are correct, midas knows nothing about what you put in the history data.

what I suggested is: if you want your true data timestamp recorded in the history,
you can put it into the history data yourself, and I suggested using the 1st value,
but you can also make it the last value or the 10th value, it is up to you.

for making history plots, the history timestamp is used, as you wrote and I confirmed,
this timestamp is generated by mlogger.

what is not clear to me is why this is a problem? do you see a big difference between the 
true data timestamp and the mlogger data timestamp? bigger than 1 second? (this would change 
the shape of "last 10 minutes" plots (600 seconds). bigger than 1 minute? (this would change 
the shape of "last 1 hour plots" (60 minutes, 3600 seconds).

that said, note that we currently store the timestamp as a DWORD 32-bit UNIX time value 
which will overflow in 2038 and which is quickly becoming incompatible with the ongoing 
switch to 64-bit time_t. Ubuntu-24 already build a large number of system libraries with 64-
bit time_t and building MIDAS with 32-bit time_t may soon become as difficult as building 
32-bit MIDAS for 32-bit i686 VME processors. we have to move with the times.

what it means is that the history system data format will have to be updated to 64-bit 
time_t and at the same time, we may try to change the timestamp from mlogger-generated to 
frontend-generated.

but it is still not clear to me how that helps you, because the frontend-generated timestamp 
is not the true data timestamp that you wanted. (and only you know what the true data 
timestamp is and where it comes from and how to tell it to MIDAS).

K.O.
  1014   11 Jul 2014 Konstantin OlchanskiInfoMIDAS high speed test
We have tested operation of MIDAS using a 10GigE network connection. Using a dummy frontend 
generating fake data, we can record MIDAS data to disk at at least 700 Mbytes/sec as reported by 
the MIDAS status page.

Two configurations were tested, both run at at least 700Mbytes/sec sustained:

1) MIDAS mhttpd, mserver, mlogger running on the disk server machine (mlogger writes to local 
disk), frontend running on remote machine (10GigE mserver connection).
2) MIDAS mhttpd, mserver, mlogger, frontend running on remote machine (mlogger writes data to 
an NFS-mounted disk over a 10GigE connection).

In addition, for configuration (2), I simulated online analysis reading fresh midas files at the same 
time as MIDAS writes new data. The resulting observation is that Linux seems to be giving main 
priority to disk write traffic (700 Mbytes/sec) with the remaining disk bandwidth given to read traffic 
(50-100Mbytes/sec). In other words, when running online data analysis on fresh data files, mlogger 
continues to run at full speed (analysis does not slow down data taking).

A few problems with MIDAS were observed during this test:

a) mlogger data compression using gzip-1 has to be turned off (limits data rate to about 
200Mbytes/sec). We plan to implement high speed LZO/LZ4 data compression that we expect to 
keep up with a 10GigE network interface.
b) CPU use by mserver and mlogger is rather high (about 40% CPU)
c) when writing to the NFS disk, mlogger has a pause of 1-2 seconds when closing and reopening 
subrun data files. To avoid a interruption in data taking, the SYSTEM event buffer has to be big 
enough to ride through this pause, but stock MIDAS limits the maximum size of event buffer to 1GB 
(too small), this can be easily increased to 2GB (almost big enough) and with some more work it can 
be increased to 4GB, but no more because the buffer length is a 32-bit integer.
d) when writing to the NFS disk, we also see periodic 3-5 second interruptions ("write operation took 
5123 ms") and we had one death of mlogger by a timeout of 60 sec.

Details of the hardware:

1) the disk server machine CPU is 3.4GHz Intel i7-4770, mobo is ASUS Z87 WS (10 SATA, 2xGigE), 
RAM is 32GB DDR3-1600.
2) disk array is 8x4TB Seagate ST4000VN000-1H4168 NAS disks RAID0 (striped) configuration, raw 
data read/write rate is around 1 GByte/sec, disks are directly attached to mobo (no raid card), linux 
software raid.
3) the frontend machine CPU is 3.7GHz Intel i7-4820, mobo is ASUS P9X79 WS, RAM is 32GB DDR3-
1600.
4) 10GigE network is Solarflare Communications SFC9120 (both machines) with a cross-over fiber 
cable (direct connection,no switches)
5) OS is up-to-date SL6.5 (both machines)

K.O.
  1019   06 Aug 2014 Konstantin OlchanskiInfoMIDAS high speed test
> We have tested operation of MIDAS using a 10GigE network connection. Using a dummy frontend 
> generating fake data, we can record MIDAS data to disk at at least 700 Mbytes/sec as reported by 
> the MIDAS status page.
>
> Details of the hardware:
> 
> 1) the disk server machine CPU is 3.4GHz Intel i7-4770, mobo is ASUS Z87 WS (10 SATA, 2xGigE), 
> RAM is 32GB DDR3-1600.
> 2) disk array is 8x4TB Seagate ST4000VN000-1H4168 NAS disks RAID0 (striped) configuration, raw 
> data read/write rate is around 1 GByte/sec, disks are directly attached to mobo (no raid card), linux 
> software raid.
>

These tests were done using a raid0 array (striped), which is not suitable for production use.

For production use, RAID5 and RAID6 is recommended. But their default configuration has severely reduced performance (50% of 
RAID0) this is because internally the raid driver issues disk read operations that compete against and severely slow down the disk write 
requests. This is easy to see with "iostat -x 1" - when writing to the raid array, there should be no reads from the disks. Following 
changes are required to achieve maximum performance:

echo 32000 > /sys/block/md6/md/stripe_cache_size # increase internal memory buffers - because "raid write" is always "read-
modify-write", bigger buffers ensure that the reads are done from cache, not from phsyical disk
mdadm --grow --bitmap=/md6bitmap /dev/md6 # use external bitmap - if bitmap is internal, there is a large number of disk reads 
competing against writes. external bitmap seems to help quite a bit.

With these settings, my RAID6 array can read and write at about 700-900 Mbytes/sec - this is comparable to RAID0 (minus 2 disks).

With this, I repeated the MIDAS performance tests - (but without 10GigE) - MIDAS can write 700 Mbytes/sec of fake data to a local 
RAID6 data array. (hardware configuration is listed above).

K.O.
  2722   08 Mar 2024 Konstantin OlchanskiInfoMIDAS frontend for WIENER L.V. P.S. and VME crates
Our MIDAS frontend for WIENER power supplies is now available as a standalone git repository.

https://bitbucket.org/ttriumfdaq/fewienerlvps/src/master/

This frontend use the snmpwalk and snmpset programs to talk to the power supply.

Also included is a simple custom web page to display power supply status and to turn things on and off.

This frontend was originally written for the T2K/ND280 experiment in Japan.

In addition to controlling Wiener low voltage power supplies, it was also used to control the ISEG MPOD high 
voltage power supplies.

In Japan, ISEG MPOD was (still is) connected to the MicroMegas TPC and is operated in a special "spark counting" 
mode. This spark counting code is still present in this MIDAS frontend and can be restored with a small amount of 
work.

K.o.
  506   13 Oct 2008 Konstantin OlchanskiInfoMIDAS drivers for Tundra tsi148 pci-vme bridge
The latest midas mvmestd.h driver for the Tundra tsi148 pci-vme bridge as used
on GEFANUC VME processors have been commited, revision 4349.

This midas drivers require the "gefvme" Linux kernel driver supplied by GEFANUC
as part of their Linux BSP. (Note that version "v7865-sdk-linux-R01.00" from
GEFANUC is mostly non-functional).

At TRIUMF have the V7865 VME processors and use the kernel driver
v7865-sdk-linux-R01.00-KO6. This driver supports these functions:

1) memory mapped access to full VME A16 and A24 address spaces and window-mapped
access to VME A32 address space. (original gefvme driver does not do
memory-mapped access)
2) DMA directly from vme to user memory, with support for multi-segment chained
transfers (original gefvme driver lacks chained transfers)
3) DMA from user memort to vme should work but is untested
4) no support for interrupts (original gefvme driver does not interrupts).

If you are interested in in using the TRIUMF driver, please contact me directly.

If you already purchased the GEFANUC BSP, I think you can use my drivers
immediately, without objection from GEFANUC.

Otherwise, I will have to do some research into the gefvme code license: since
all of the code appears to have GPL headers and identical code exists on the
internet, I expect to find that my gefvme driver can be freely distributed under
the GPL. But until then, and until it is cleared with TRIUMF management, I
cannot make my gefvme driver available for free download.

K.O.
  845   14 Dec 2012 Robert CaspersonBug ReportMIDAS does not function correctly on F17
When building MIDAS on Fedora 17 64-bit, the default zlib 1.2.5 shared library
is linked to.  When recording data, the "/Logger/Channels/*/Statistics/Bytes
written" value does not get set correctly beyond the first few seconds of the
run.  Occasionally, it appears to not get set at all, and mlogger aborts the run.

Installing zlib 1.2.3 in static form to /usr/local/lib (the default location),
and changing the NEED_ZLIB section of the MIDAS Makefile to the following seems
to function as a workaround:

ifdef NEED_ZLIB
CFLAGS   += -DHAVE-ZLIB
LIBS     += /usr/local/lib/libz.a
endif

Several Fedora 17 libraries expect zlib 1.2.5 specifically, so it seems safest
to not replace the default zlib shared library.

Some extra details are that the VME CPU is an XVB602, and the most recent GE-IP
drivers are being used for VME communication.  Fedora 17 was chosen to avoid a
bug with the VGA output in Fedora 13-16.
  850   20 Dec 2012 Stefan RittBug ReportMIDAS does not function correctly on F17
If is not so easy to get out of zlib how many bytes have been written actually. I used an undocumented function, 
which breaks down on 64-bit systems.

I now rewrote the code in mlogger.cxx to use lseek() to "measure" actually the output file and set the values 
correctly. I tried on a few systems but am not 100% sure if it works everywhere. Can you please double check?

The fix is in SVN revision 5347.

/Stefan
  835   10 Sep 2012 Shaun MeadInfoMIDAS button to display image
Hi,

I've written a python script that reads some data from a file and generates a
.png image. I want to have a button on my MIDAS status page that:

- executes the script and waits for it to finish,
- then displays the image

How can I do that? I tried using the sequencer to just execute the script every
30 seconds, but I can't get it to work, and it would be better to only execute
the script on demand anyway. 

I also am having trouble getting image display to work. I have the ODB keys set:

[local:oven1:S]/Custom>ls
Temperature Map&                /home/deap/ovendaq/online/index.html
Images

[local:oven1:S]/Custom>ls Images/temps.png/           
Background                      /home/deap/ovendaq/online/temps.png

And the HTML file is just this:
<img src="temps.png">

But the image won't display. It shows a "broken" picture, and when I try to view
it directly it says: Invalid custom page: Page not found in ODB.

Any help would be appreciated...

Thanks
Shaun
  836   11 Sep 2012 Stefan RittInfoMIDAS button to display image
> Hi,
> 
> I've written a python script that reads some data from a file and generates a
> .png image. I want to have a button on my MIDAS status page that:
> 
> - executes the script and waits for it to finish,
> - then displays the image
> 
> How can I do that? I tried using the sequencer to just execute the script every
> 30 seconds, but I can't get it to work, and it would be better to only execute
> the script on demand anyway. 
> 
> I also am having trouble getting image display to work. I have the ODB keys set:
> 
> [local:oven1:S]/Custom>ls
> Temperature Map&                /home/deap/ovendaq/online/index.html
> Images
> 
> [local:oven1:S]/Custom>ls Images/temps.png/           
> Background                      /home/deap/ovendaq/online/temps.png
> 
> And the HTML file is just this:
> <img src="temps.png">
> 
> But the image won't display. It shows a "broken" picture, and when I try to view
> it directly it says: Invalid custom page: Page not found in ODB.
> 
> Any help would be appreciated...
> 
> Thanks
> Shaun


If you use the "custom" image system, you need to use GIF images. mhttpd can dynamically create GIF 
images, 
with a background image and overlaid labels, bar graphs etc. But mhttpd just contains a GIF library to do 
that 
in memory, but no PNG library.

Actually I would recommend you not to use a script to create an image, but use the custom image system 
to 
display temperatures. In the attachment you see an page from our experiment which contains a 
background image (the greyish boxes), labels (white temperature boxes), bar graphs (blue level boxes) 
and history pages (left side). This is all dynamically created inside mhttpd using the custom page system 
without any external script. All you have to do is to get the temperatures and levels inside the ODB via the 
slow control system. If you want, I can send you the full code for that page.

Cheers,
Stefan
Attachment 1: Screen_Shot_2012-09-11_at_14.36.56_.png
Screen_Shot_2012-09-11_at_14.36.56_.png
ELOG V3.1.4-2e1708b5