Back Midas Rome Roody Rootana
  Midas DAQ System  Not logged in ELOG logo
Entry  15 Jul 2015, Konstantin Olchanski, Info, mlogger improvements 
    Reply  23 Jul 2015, Konstantin Olchanski, Info, rootana lz4 support, mlogger improvements 
    Reply  23 Jul 2015, Konstantin Olchanski, Info, mlogger improvements 
    Reply  29 Jul 2015, Konstantin Olchanski, Info, mlogger improvements - CRC32C, SHA-2 
Message ID: 1085     Entry time: 29 Jul 2015     In reply to: 1068
Author: Konstantin Olchanski 
Topic: Info 
Subject: mlogger improvements - CRC32C, SHA-2 
> A set of improvements to mlogger is in:

Preliminary support for CRC32-zlib, CRC32C, SHA-256 and SHA-512 is in. Checksums are computed correctly, but plumbing configuration is 
preliminary. Good enough for testing and benchmarking.

To enable checksums, set channel compression:
100 - no checksum (LZ4 compression)
11100 - CRC32-zlib checksum
22100 - CRC32C
33100 - SHA-256
44100 - SHA-512
checksums for both uncompressed and compressed data will be computed and reported into midas.log.

To compare:

CRC32-zlib is for compatibility with gzip and zlib tools
CRC32C is for maximum speed
SHA-256 and SHA-512 is for maximum data security

To remember:

- CRC32-zlib is the CRC32 computation from gzip/png/zlib library. I believe the technical name of the algorithm is "adler32".
- CRC32C is the most recently improved version of CRC32 family of checksums. Implementation is from Mark Adler (same Adler as adler32) uses 
hardware acceleration on recent Intel CPUs.
- SHA-256 and SHA-512 are checksums currently accepted as cryptographically secure. One of them is supposed to be faster on 64-bit 
machines. I implement both for benchmarking.

"Cryptographically secure" means "nobody has a practical way to construct two different files with the same checksum".

In simpler words, the file contents cannot change without breaking the checksum - by software bug, by hardware fault, by benign or malicious 
intent.

The CRC family of checksum functions were never cryptographically secure.
MD5 and SHA-1 used to be secure but are no longer considered to be so. MD5 was definitely broken as different files with the same checksum 
have been discovered or constructed.

K.O.
ELOG V3.1.4-2e1708b5