Skip to content

Add decodeSingle() to decode a single object in a buffer that may have extra bytes#293

Open
KAMRONBEK wants to merge 1 commit into
msgpack:mainfrom
KAMRONBEK:feature/decode-single-128
Open

Add decodeSingle() to decode a single object in a buffer that may have extra bytes#293
KAMRONBEK wants to merge 1 commit into
msgpack:mainfrom
KAMRONBEK:feature/decode-single-128

Conversation

@KAMRONBEK

Copy link
Copy Markdown

Problem

There is no public API to decode a single MessagePack object from a buffer that contains trailing non-MessagePack bytes. decode() throws RangeError: Extra N byte(s) found..., and even if you catch it there is no way to learn how many bytes the object occupied. This breaks framed-stream protocols of the form [msgpack object][opaque bytes][msgpack object], where the msgpack object acts as a header describing the opaque payload that follows (see the use cases described in #128 by @mcclure, @covert-encryption, and @ansemjo).

Root cause

Decoder#decode() deliberately calls createExtraByteError() when bytes remain after doDecodeSync(), and both doDecodeSync() (which decodes exactly one object) and pos (which tracks bytes consumed) are private, so callers cannot opt out of the extra-bytes check or find the object boundary.

Fix

Following the direction discussed in the issue thread, this adds a purely additive API — zero behavior change to existing functions:

  • Decoder#decodeSingle(buffer): DecodeSingleResult — same re-entrancy handling as decode(), decodes exactly one object via the existing doDecodeSync(), and returns { value, bytesConsumed } instead of throwing on remaining bytes. Truncated or empty buffers still throw RangeError, and invalid data still throws DecodeError, exactly like decode().
  • decodeSingle(buffer, options?) top-level function in src/decode.ts, mirroring decode()/decodeMulti().
  • DecodeSingleResult type: { value: unknown; bytesConsumed: number }bytesConsumed is the position where the extra bytes begin, so callers can buffer.subarray(bytesConsumed) to continue with the rest of the frame.
  • Re-exported from src/index.ts; README gets a TOC entry, a full API section with a framed-buffer example, and a pointer from the decode() section.

This covers the in-memory-buffer case; a ReadableStream variant (decode one object from an async byte source, as some commenters describe) would be a natural follow-up if wanted.

Tests

test/decodeSingle.test.ts (mocha + assert, matching the existing test style):

  • decodes a single object and reports bytesConsumed equal to the encoded length
  • framed buffer [msgpack object][0xc1 opaque bytes][msgpack object]: decodes the first object without throwing, skips the opaque bytes using bytesConsumed, decodes the second object
  • empty buffer throws RangeError
  • truncated object throws RangeError
  • Decoder#decodeSingle works repeatedly on a reused instance

Full suite: 332 passing; npm run lint clean.

Refs #128

@KAMRONBEK

Copy link
Copy Markdown
Author

@gfx TL;DR: adds the decodeSingle() API discussed in #128 — decodes exactly one object via the existing private doDecodeSync() and returns { value, bytesConsumed } instead of throwing the extra-bytes RangeError, so framed protocols ([msgpack header][opaque payload]…) can find the object boundary and continue with buffer.subarray(bytesConsumed). Purely additive: zero behavior change to decode()/decodeMulti(); truncated/empty buffers still throw RangeError and invalid data still throws DecodeError. Tests + README API section included (full suite 332 passing). A ReadableStream variant for the async use cases in the thread would be a natural follow-up if you want it. Review appreciated when you have time!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant