The BSON engine

BSON is MongoDB's binary document format: length-prefixed, typed, and traversable without parsing the whole document. BisonDB implements its own codec (the encoder, decoder, and the Extended JSON text mappings) and validates it against MongoDB's official conformance corpus.

The value model

In memory a document is a Value: a tagged union (std::variant) over the 11 supported BSON types. Documents preserve insertion order. BSON is an ordered map, so the obvious std::map<std::string, Value> would be wrong; internally it is a vector of key/value pairs with linear key lookup.

Type	Tag byte	Payload
Double	0x01	8-byte IEEE 754, little-endian
String	0x02	i32 length including NUL, then UTF-8 bytes, then 0x00
Document	0x03	a nested document (same format, recursively)
Array	0x04	a document whose keys are "0", "1", ...
ObjectId	0x07	12 raw bytes
Bool	0x08	1 byte, must be 0 or 1
DateTime	0x09	i64 ms since epoch
Null	0x0A	nothing
Int32	0x10	i32
Int64	0x12	i64
Decimal128	0x13	16 raw bytes (IEEE 754-2008 BID)

Decoding a real document, byte by byte

The document {"name": "ada", "age": 36} is 27 bytes on the wire:

Offset	Bytes (hex)	Meaning
0-3	1B 00 00 00	total document size = 27, little-endian
4	02	element type: String
5-9	6E 61 6D 65 00	key "name", NUL-terminated cstring
10-13	04 00 00 00	string length 4 (3 chars + trailing NUL)
14-17	61 64 61 00	"ada" + NUL
18	10	element type: Int32
19-22	61 67 65 00	key "age"
23-26	24 00 00 00	36, little-endian
27	00	document terminator

(The terminator is byte 26 zero-indexed; the declared size counts it.) Every structural fact in that table is something a hostile input can lie about, which leads to:

A decoder that assumes hostility

The decoder treats input as adversarial. The same bytes arrive from disk and from the network. Every read is bounds-checked against both the buffer and the document's declared size, and element parsing must land exactly on the terminator:

• declared size < 5 or beyond the available bytes → reject
• string length < 1, or extending past the document → reject
• a sub-document whose declared size eats its parent's terminator → reject
• bool bytes other than 0/1, unknown type tags → reject
• invalid UTF-8 in keys or strings → reject
• nesting beyond 200 levels → reject (this is a stack-overflow guard, not pedantry)

Every failure throws with the byte offset of the problem. There is no lookahead and no "resync": a framing violation means the stream cannot be trusted.

Corpus testing

The decoder/encoder pair runs against the official MongoDB BSON corpus for all 11 types: every canonical document must decode and re-encode to identical bytes, every degenerate encoding must normalize to canonical, and every listed decode error must throw. On top of that, the fixture suite round-trips real mongodump files; 29,470 ZIP-code documents re-encode byte-for-byte.

Extended JSON

Text I/O uses MongoDB Extended JSON v2 in both modes:

• Relaxed: human-friendly: plain numbers, {"$oid": "..."}, ISO-8601 $date.
• Canonical: lossless: {"$numberInt": "36"} wrappers preserve exact types, so bson → JSON → bson reproduces the original bytes. The converter's round-trip tests depend on this.

The JSON parser is a from-scratch RFC 8259 recursive-descent parser (with surrogate-pair handling and the same 200-level depth cap), extended with the $-wrapper folding, and, for the shell, a relaxed mode accepting unquoted keys, single quotes, and trailing commas.

A note on Decimal128: BisonDB stores the 16 bytes opaquely and implements full binary-integer-decimal string conversion in both directions (including the clamping and non-canonical-significand rules), but no arithmetic. You can store and display 19.99 exactly; you cannot ask the database to add to it.