I've recently run into issues when using JSON.parse on ODB keys containing
8-bit data.
For JSON.parse to successfully parse a string, (A) the string must be valid
UTF-8, (B) several whitespace characters, control characters, and the
characters " and \ must be escaped, and (C) you've got to follow the key-
value rules laid out in http://www.json.org/.
The web browser takes care of (A), and I verified that for this key Midas
handled (C) correctly. In principle, the function json_write in odb.c
handles (B) - but json_write does not escape control characters.
To manage this problem, I modified json_write (in odb.c) to replace any
control character with the more-inocuous character, 'C'. My default case
now looks like:
default:
{
// if a char is a control character,
// print 'C' in its place
// note that this loses data:
// a more-correct method would be to print
// \uXXXX, where XXXX is the character in hex
if(iscntrl(*s)){
(*buffer)[(*buffer_end)++] = 'C';
s++;
} else {
(*buffer)[(*buffer_end)++] = *s++;
}
}
Where the call to iscntrl(*s) requires the addition of the ctype.h header
file.
I'm guessing a blanket replacement of control characters with 'C' isn't
something all Midas users would want to do. Replacing the control character
with its hex value seems like a good choice - but not without adding bounds
checking!
An alternative to changing odb.c could be to add a regex to Midas response
text which removes all control characters (U+0000 - U+001F):
var resp_lint = req.response.replace(/[\u{0000}-\u{001F}]/gmu, '');
var json_obj = JSON.parse(resp_lint);
Unfortunately, the 'u' regex flax doesn't work on the Firefox version
included in Scientific Linux 6.8. |