Back Midas Rome Roody Rootana
  Midas DAQ System  Not logged in ELOG logo
Entry  06 Mar 2023, Gennaro Tortone, Forum, pull request for PostgreSQL support 
    Reply  06 Mar 2023, Konstantin Olchanski, Forum, pull request for PostgreSQL support 
       Reply  06 Mar 2023, Gennaro Tortone, Forum, pull request for PostgreSQL support 
       Reply  20 Mar 2023, Gennaro Tortone, Forum, pull request for PostgreSQL support 
          Reply  24 May 2023, Gennaro Tortone, Forum, pull request for PostgreSQL support 
             Reply  18 Jul 2023, Konstantin Olchanski, Forum, pull request for PostgreSQL support 
                Reply  21 Jul 2023, Konstantin Olchanski, Forum, pull request for PostgreSQL support 
                   Reply  21 Jul 2023, Gennaro Tortone, Forum, pull request for PostgreSQL support 
                      Reply  28 Jul 2023, Stefan Ritt, Forum, pull request for PostgreSQL support 
                         Reply  09 Aug 2023, Konstantin Olchanski, Forum, pull request for PostgreSQL support 
Message ID: 2463     Entry time: 06 Mar 2023     In reply to: 2461
Author: Gennaro Tortone 
Topic: Forum 
Subject: pull request for PostgreSQL support 
Hi Konstantin,
thanks for this update |

My main interest for PostgreSQL is usage of TimescaleDB
(https://github.com/timescale/timescaledb) a PostgreSQL extension that
makes possible usage of downsampling functions on time-series...

here at INFN-Napoli we have a large history dataset that we manage
with MIDAS history and MySQL tables. We have a lot of issues 
(wait time, browser hangs, crashes) when we use MIDAS history plot 
pages on large time period because the Javascript web page try to 
download million of records in order to display them on a plot of 
(max) 2000 pixel width...

with native downsampling we can reduce a large dataset keeping the
"shape" of the curve using only the points needed by the plot area;

in TimescaleDB there is "lttb" ( Largest Triangle Three Bucket) a very 
nice and impressive downsampling function that preserve very well the
shape of the series.

If you are interested to see a lttb at work on some data you can open this page:
   https://www.base.is/flot

In next days I will work to add TimescaleDB backend to MIDAS history (it will be
similar to PostgreSQL backend) and we can discuss on how to add these 
downsampling features to history plot web pages, I already developed some
solutions and I will be happy to share them with MIDAS community;

Cheers,
Gennaro

> > some minutes ago I published a PR for PostgreSQL support I developed
> > at INFN-Napoli for Darkside experiment...
> > 
> > I don't know if you receive a notification about this PR and in doubt
> > I wrote this message...
> 
> Hi, Gennaro, thank you for the very useful contribution. I saw the previous version 
> of your pull request and everything looked quite good. But that pull request was 
> for an older version of midas and it would not have applied cleanly to the current 
> version. I will take a look at your updated pull request. In theory it should only 
> add the Postgres class and modify a few other places in history_schema.cxx and have 
> no changes to anything else. (if you need those changes, it should be a separate 
> pull request).
> 
> Also I am curious what benefits and drawbacks of Postgres vs mysql/mariadb you have 
> observed for storing and using midas history data.
> 
> K.O.
ELOG V3.1.4-2e1708b5