Back Midas Rome Roody Rootana
  Midas DAQ System  Not logged in ELOG logo
Entry  04 Jun 2025, Mark Grimes, Bug Report, Memory leak in mhttpd binary RPC code 
    Reply  04 Jun 2025, Konstantin Olchanski, Bug Report, Memory leak in mhttpd binary RPC code 
       Reply  07 Jun 2025, Mark Grimes, Bug Report, Memory leak in mhttpd binary RPC code msysmon-mu3ebe-20250601-042124-20250606-122124.png
Message ID: 3051     Entry time: 07 Jun 2025     In reply to: 3050
Author: Mark Grimes 
Topic: Bug Report 
Subject: Memory leak in mhttpd binary RPC code 

Hi,

We applied an intermediate fix for this locally and it seems to have fixed our issue.  The attached plot shows the percentage memory use on our machine with 128 Gb memory, as a rough proxy for mhttpd memory use.  After applying our fix mhttpd seems to be happy using ~7% of the memory after being up for 2.5 days.

Our fix to mjson was:

diff --git a/mjson.cxx b/mjson.cxx

index 17ee268..2443510 100644

--- a/mjson.cxx

+++ b/mjson.cxx

@@ -654,8 +654,7 @@ MJsonNode::~MJsonNode() // dtor

       delete subnodes[i];

    subnodes.clear();

 

-   if (arraybuffer_size > 0) {

-      assert(arraybuffer_ptr != NULL);

+   if (arraybuffer_ptr != NULL) {

       free(arraybuffer_ptr);

       arraybuffer_size = 0;

       arraybuffer_ptr = NULL;

We also applied the following in midas for good measure, although I don't think it contributed to the leak we were seeing:

diff --git a/src/mjsonrpc.cxx b/src/mjsonrpc.cxx

index 2201d228..38f0b99b 100644

--- a/src/mjsonrpc.cxx

+++ b/src/mjsonrpc.cxx

@@ -3454,6 +3454,7 @@ static MJsonNode* brpc(const MJsonNode* params)

    status = cm_connect_client(name.c_str(), &hconn);

 

    if (status != RPC_SUCCESS) {

+      free(buf);

       return mjsonrpc_make_result("status", MJsonNode::MakeInt(status));

    }

I hope this is useful to someone.  As previously mentioned we make heavy use of binary RPC, so maybe other experiments don't run into the same problem.

Thanks,

Mark.

Attachment 1: msysmon-mu3ebe-20250601-042124-20250606-122124.png  223 kB  Uploaded 07 Jun 2025  | Hide | Hide all
msysmon-mu3ebe-20250601-042124-20250606-122124.png
ELOG V3.1.4-2e1708b5