Open-ended specifications are partly to blame, researcher suggests
Supposedly benign behaviors exhibited by JSON parsers can introduce a variety of security risks arising from how data is interpreted across multiple parsers, security researchers have found.
Of 49 JSON parsers road-tested by researchers from Bishop Fox, “each language had at least one parser that exhibited a form of potentially risky interoperability behavior”, said Jake Miller, lead researcher at the offensive security outfit, in a technical write-up.
The researchers presented attack scenarios, supported by Docker Compose labs, exploiting inconsistent duplicate key precedence, key collision through character truncation and comments, JSON serialization quirks, float and integer representation, and permissive parsing.
“Other than segmentation faults, these behaviors are harmless in the context of a single parser, which prevents them from [being] classified as vulnerabilities for a particular parser,” said Miller.
Where multiple parsers are involved, however, they can lead to business logic, injection, and type juggling vulnerabilities, among other issues.
“Parsers provided by standard libraries tended to be the most compliant, but they often lacked speed, which is of increasing importance in microservice architectures,” said Miller. “This has prompted developers to choose more performant, third-party parsers.”
Inter(in)operable misbehavior
However, even compliance with standards does not entirely preclude problems, with Miller concluding that inconsistent implementations are fueled by ambiguous or inconsistent specifications, as well as the growing complexity of interoperability in modern, multi-language microservice architectures.
Applications within these architectures “often rely on several separate JSON parsing implementations, each of which has its own quirks”, he says.
READ MORE HTTP request smuggling: HTTP/2 opens a new attack tunnel
He also cites HTTP request smuggling as an example of how “discrepancies across parsers combined with multi-stage request processing can introduce serious vulnerabilities”.
‘Open-ended’ specs
“Even in the best-case implementation” of parsers generally, “there are inevitably minor, unintentional deviations from specifications”.
But JSON specifications themselves are divergent, said Miller, citing IETF JSON RFC (8259 and prior), ECMAScript Standard, JSON5, and HJSON.
Even the official JSON RFC guidance is “open-ended” for “a few topics, such as how to handle duplicate keys and represent numbers”, he notes. “Although this guidance is followed by disclaimers about interoperability, most users of JSON parsers aren’t aware of these caveats.”
Read more of the latest secure development news
Miller suspects the ambiguity is a deliberate attempt “not to break backwards compatibility with pre-specification JSON parsers, including the original JavaScript implementation”.
If true, this is misconceived, he argues.
“Restricting behavior to deterministic outcomes not only improves interoperability but also makes it easier to report bugs and improve our software,” the researcher suggests.
“Breaking by defining previously undefined behavior may cause pushback. But in the modern context of microservice architectures, where interoperability becomes increasingly complex, it may be a worthwhile choice.”
Mitigations for all parties
Miller recommends that parser maintainers mitigate risks by generating fatal parse errors on duplicate keys, and eschewing character truncation in favor of replacing invalid unicode with placeholder characters, among other things.
These “nuanced attacks” are hard to detect externally, so infosec professionals with access to source code should “look for parsers with known quirks”, and “try duplicating keys and using the suggestions in the labs README to try to induce collisions”.
He also says JSON Schema may help mitigate parsing risks like type checking and constraining the range of allowed integers, but are blind to inconsistent parsing.
Security software engineer Claudio Salazar has responded to the research with his own secure development advice.
“Test the corner cases” among “JSON libraries used on your stack” then choose libraries that share behaviors, “use schema validation” and “ share this validation definition among microservices if they are receiving data independently”, and use a comprehensive test suite, which if “a developer wants to use another JSON library”, will “warn you about some differences in the parsing logic that could affect your application security”, he said in a blog post.
Underrated threat
“The simplicity of JSON is often taken for granted”, says Miller. “We don’t usually consider JSON parsing as part of our threat model.”
However, the researcher told The Daily Swig that “when inconsistent implementations begin to break our assumptions, our ability to safely validate business logic breaks along with it. Vulnerabilities like this highlight the importance of research at lower levels of request processing that are often abstracted to application developers.”
RECOMMENDED H2C smuggling named top web hacking technique of 2020
