How simdjson's On Demand Parsing works

Current JSON parsers generally have either ease of use or performance. Very few have both at

once. simdjson's On Demand API bridges that gap with a familiar, friendly DOM API and the

performance of just-in-time parsing on top of the simdjson core's legendary performance.

To achieve ease of use, we mimicked the *form* of a traditional DOM API: you can iterate over

arrays, look up fields in objects, and extract native values like double, uint64_t, string and bool.

To achieve performance, we introduced some key limitations that make the DOM API *streaming*:

array/object iteration cannot be restarted, and fields must be looked up in order, and string/number

values can only be parsed once.

ondemand::parser parser;

auto doc = parser.iterate(json);

for (auto tweet : doc["statuses"]) {

std::string_view text = tweet["text"];

std::string_view screen_name = tweet["user"]["screen_name"];

uint64_t retweets = tweet["retweet_count"];

uint64_t favorites = tweet["favorite_count"];

cout << screen_name << " (" << retweets << " retweets / " << favorites << " favorites): " << text << endl;

}

This streaming approach means that fields or values you don't use don't have to get parsed or

converted, saving space and time.

Further, the On Demand API doesn't parse a value *at all* until you try to convert it to double,

int, string, or bool. Because you have told it the type at that point, it can avoid the the key

"what type is this" branch present in almost all other parsers, avoiding branch misprediction that

cause massive (sometimes 2-4x) slowdowns.

Parsers Today

To understand exactly what's happening here and why it's different, it's helpful to review the major

approaches to parsing and parser APIs in use today.

Many of the most usable, popular JSON APIs deserialize into a **DOM**: an intermediate tree of

objects, arrays and values. The resulting API lets you refer to each array or object separately,

using familiar techniques like iteration (`for (auto value : array)`) or indexing (`object["key"]`).

In some cases the values are even deserialized straight into familiar C++ constructs like vector and

map.

This model is dead simple to use, since it talks in terms of *data types* instead of JSON. It's

often easy enough that many users use the deserialized JSON as-is instead of deserializing into

their own custom structs, saving a ton of development work.

simdjson's DOM parser is one such example. It looks very similar to the ondemand example, except

it calls `parse` instead of `iterate`:

dom::parser parser;

auto doc = parser.parse(json);

for (auto tweet : doc["statuses"]) {

std::string_view text = tweet["text"];

std::string_view screen_name = tweet["user"]["screen_name"];

uint64_t retweets = tweet["retweet_count"];

uint64_t favorites = tweet["favorite_count"];

cout << screen_name << " (" << retweets << " retweets / " << favorites << " favorites): " << text << endl;

}

Pros and cons of generic DOM:

* Straightforward, user-friendly interface (arrays and objects)

* No lifetime concerns (arrays and objects are often independent of JSON text and parser internal state)

* Parses and stores everything, using memory/CPU even on values you don't end up using (cost can be brought down some with lazy numbers/strings and top-level iterators)

* Values stay in memory even if you only use them once

* Heavy performance drain from [type blindness](#type-blindness).

The SAX model ("Streaming API for XML") uses streaming to eliminate the high cost of

parsing and storing the entire JSON. In the SAX model, a core JSON parser parses the JSON document

piece by piece, but instead of stuffing values in a DOM tree, it passes each value to a callback,

letting the user use the value and decide for themselves whether to discard it and where to store

it. or discard it.

This allows users to work with much larger files without running out of memory. Some SAX APIs even

allow the user to skip values entirely, lightening the parsing burden as well.

The big drawback is complexity: SAX APIs generally have you define a single callback for each type

(e.g. `string_field(std::string_view key, std::string_view value)`). Because of this, you suffer

from context blindness: when you find a string you have to check where it is before you know what to

do with it. Is this string the text of the tweet, the screen name, or something else I don't even

care about? Are we even in a tweet right now, or is this from some other place in the document

entirely?

The following is SAX example of the Twitter problem we've seen in the Generic DOM and On Demand

examples. To make it short enough to use as an example at all, it's heavily redacted: it only solves

a part of the problem (doesn't get user.screen_name), it has bugs (it doesn't handle sub-objects

in a tweet at all), and it uses a theoretical, simple SAX API that minimizes ceremony and skips over

the issue of lazy parsing and number types entirely.

struct twitter_callbacks {

bool in_statuses;

bool in_tweet;

std::string_view text;

uint64_t retweets;

uint64_t favorites;

void start_object_field(std::string_view key) {

if (key == "statuses") { in_statuses = true; }

}

void start_object() {

if (in_statuses) { in_tweet = true; }

}

void string_field(std::string_view key, std::string_view value) {

if (in_tweet && key == "text") { text = value; }

}

void number_field(std::string_view key, uint64_t value) {

if (in_tweet) {

if (key == "retweet_count") { retweets = value; }

if (key == "favorite_count") { favorites = value; }

}

}

void end_object() {

if (in_tweet) {

cout << "[redacted] (" << retweets << " retweets / " << favorites << " favorites): " << text << endl;

in_tweet = false;

} else if (in_statuses) {

in_statuses = false;

}

}

};

sax::parser parser;

parser.parse(twitter_callbacks());

for (object tweet : doc["statuses"].get_array()) {

array tweets = doc["statuses"].get_array();

- If the array is empty (followed by `]`), advances the iterator past the `]` and then releases the

iterator (indicating the array is finished iterating).

- If the array is not empty, the iterator remains pointed at the first value (the token just

after `[`).

2. `tweets.begin()`, `tweets.end()`: Returns an `array_iterator`, which just points back at the

3. `while (iter != end)`: Stops if the iterator has been released.

Design Features

When you iterate over an array or object with an error, the error is yielded in the loop