lib: use utf8 fast path for streaming TextDecoder

[ChALkeR]

Tracking: #61041

A continuation of #61409

This is based on the logic in https://github.com/ExodusOSS/bytes

Unifies stream and non-stream codepath for UTF-8 both for intl and no-intl variants.

Previously, ICU and string_decoder were used on with-intl and without-intl variants correspondingly for streaming UTF-8 implementation

Instead, just do minor quick slices on JS side and use the single stateless native decoding API to be fast on large chunks

This:

1. Improves streaming UTF-8 TextDecoder performance
2. Brings streaming fatal UTF-8 TextDecoder support to without-intl builds
3. Unifies the logic, using a single codepath everywhere for UTF-8

Benchmarks

Previously: #61131

Non-chunked, 25.5.0

Test	Size	Throughput	Mean Time
Latin lipsum (ASCII)	84.902 KiB	36.69 GiB/s	0.002 ms
Arabic lipsum	79.771 KiB	2.03 GiB/s	0.037 ms
Chinese lipsum	68.203 KiB	4.10 GiB/s	0.016 ms
Arabic + 2 * ASCII	249.577 KiB	2.72 GiB/s	0.092 ms
Non-ASCII char + ASCII	84.905 KiB	3.84 GiB/s	0.024 ms

Non-chunked, PR

Test	Size	Throughput	Mean Time
Latin lipsum (ASCII)	84.902 KiB	36.19 GiB/s	0.002 ms
Arabic lipsum	79.771 KiB	2.02 GiB/s	0.038 ms
Chinese lipsum	68.203 KiB	4.00 GiB/s	0.016 ms
Arabic + 2 * ASCII	249.577 KiB	3.32 GiB/s	0.075 ms
Non-ASCII char + ASCII	84.905 KiB	4.71 GiB/s	0.022 ms

(should not be affected)

Chunked (1000 byte chunks), 25.5.0

Test	Size	Throughput	Mean Time
Latin lipsum (ASCII)	84.902 KiB	1.31 GiB/s	0.062 ms
Arabic lipsum	79.771 KiB	0.90 GiB/s	0.084 ms
Chinese lipsum	68.203 KiB	0.99 GiB/s	0.066 ms
Arabic + 2 * ASCII	249.577 KiB	1.14 GiB/s	0.209 ms
Non-ASCII char + ASCII	84.905 KiB	1.31 GiB/s	0.062 ms

Chunked (1000 byte chunks), PR

Test	Size	Throughput	Mean Time
Latin lipsum (ASCII)	84.902 KiB	7.45 GiB/s	0.011 ms
Arabic lipsum	79.771 KiB	1.24 GiB/s	0.062 ms
Chinese lipsum	68.203 KiB	1.54 GiB/s	0.042 ms
Arabic + 2 * ASCII	249.577 KiB	2.67 GiB/s	0.091 ms
Non-ASCII char + ASCII	84.905 KiB	7.26 GiB/s	0.011 ms

The benchmark creates about ~70-256 chunks on each test
The improvement is significant

[anonrig]

@ChALkeR can you run benchmark CI that applies to this pull-request?

[ChALkeR]

@anonrig I can't do anything until #61553

Also, we don't have a stream: true TextDecoder benchmark.
The bench results above are from https://github.com/lemire/jstextdecoderbench with this modification:

function decodeChunked(bytes) {
  const chunk = 1000
  const max = bytes.length - chunk
  let i = 0
  for (; i < max; i += chunk) decoder.decode(bytes.subarray(i, i + chunk), { stream: true })
  decoder.decode(bytes.subarray(i))
}

And using that instead of decoder.decode

[anonrig]

Thank you for the kind responses and amazing work.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.77%. Comparing base (e155415) to head (1559954).
⚠️ Report is 38 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff            @@
##             main   #61549    +/-   ##
========================================
  Coverage   89.77%   89.77%            
========================================
  Files         672      673     +1     
  Lines      203755   203875   +120     
  Branches    39167    39190    +23     
========================================
+ Hits       182922   183038   +116     
- Misses      13164    13172     +8     
+ Partials     7669     7665     -4     

Files with missing lines	Coverage Δ
lib/internal/encoding.js	100.00% <100.00%> (ø)
lib/internal/encoding/util.js	100.00% <100.00%> (ø)

... and 33 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/71082/

[ChALkeR]

@anonrig I added a benchmark for streaming Unicode TextDecoder.
Benchmark CI: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1787

[nodejs-github-bot]

Commit Queue failed

- Loading data for nodejs/node/pull/61549
✔  Done loading data for nodejs/node/pull/61549
----------------------------------- PR info ------------------------------------
Title      lib: use utf8 fast path for streaming TextDecoder (#61549)
Author     Nikita Skovoroda <chalkerx@gmail.com> (@ChALkeR)
Branch     ChALkeR:chalker/decoder/unify/2 -> nodejs:main
Labels     performance, encoding, author ready, needs-ci, needs-benchmark-ci
Commits    2
 - lib: use utf8 fast path for streaming TextDecoder
 - benchmark: add streaming TextDecoder benchmark
Committers 1
 - Nikita Skovoroda <chalkerx@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/61549
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
------------------------------ Generated metadata ------------------------------
PR-URL: https://github.com/nodejs/node/pull/61549
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
--------------------------------------------------------------------------------
   ℹ  This PR was created on Tue, 27 Jan 2026 18:17:16 GMT
   ✔  Approvals: 3
   ✔  - Yagiz Nizipli (@anonrig) (TSC): https://github.com/nodejs/node/pull/61549#pullrequestreview-3713460238
   ✔  - Gürgün Dayıoğlu (@gurgunday): https://github.com/nodejs/node/pull/61549#pullrequestreview-3715962381
   ✔  - Colin Ihrig (@cjihrig): https://github.com/nodejs/node/pull/61549#pullrequestreview-3717219044
   ✔  Last GitHub CI successful
   ℹ  Last Full PR CI on 2026-01-28T15:17:24Z: https://ci.nodejs.org/job/node-test-pull-request/71102/
   ℹ  Last Benchmark CI on 2026-01-27T23:16:20Z: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1787
- Querying data for job/node-test-pull-request/71102/
✔  Build data downloaded
   ✔  Last Jenkins CI successful
--------------------------------------------------------------------------------
   ✔  No git cherry-pick in progress
   ✔  No git am in progress
   ✔  No git rebase in progress
--------------------------------------------------------------------------------
- Bringing origin/main up to date...
From https://github.com/nodejs/node
 * branch                  main       -> FETCH_HEAD
✔  origin/main is now up-to-date
- Downloading patch for 61549
From https://github.com/nodejs/node
 * branch                  refs/pull/61549/merge -> FETCH_HEAD
✔  Fetched commits as 6c44d31e2522..155995438c2a
--------------------------------------------------------------------------------
Auto-merging test/parallel/test-bootstrap-modules.js
[main d495fa4fc6] lib: use utf8 fast path for streaming TextDecoder
 Author: Nikita Skovoroda <chalkerx@gmail.com>
 Date: Tue Jan 27 22:12:49 2026 +0400
 4 files changed, 138 insertions(+), 47 deletions(-)
 create mode 100644 lib/internal/encoding/util.js
[main 8441e884d9] benchmark: add streaming TextDecoder benchmark
 Author: Nikita Skovoroda <chalkerx@gmail.com>
 Date: Wed Jan 28 03:08:19 2026 +0400
 1 file changed, 55 insertions(+)
 create mode 100644 benchmark/util/text-decoder-stream.js
   ✔  Patches applied
There are 2 commits in the PR. Attempting autorebase.
(node:2357) [DEP0190] DeprecationWarning: Passing args to a child process with shell option true can lead to security vulnerabilities, as the arguments are not escaped, only concatenated.
(Use `node --trace-deprecation ...` to show where the warning was created)
Rebasing (2/4)
Executing: git node land --amend --yes
--------------------------------- New Message ----------------------------------
lib: use utf8 fast path for streaming TextDecoder
PR-URL: #61549

Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>

Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>

Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
[detached HEAD d9f44d2f20] lib: use utf8 fast path for streaming TextDecoder

Author: Nikita Skovoroda <chalkerx@gmail.com>

Date: Tue Jan 27 22:12:49 2026 +0400

4 files changed, 138 insertions(+), 47 deletions(-)

create mode 100644 lib/internal/encoding/util.js

Rebasing (3/4)

Rebasing (4/4)

Executing: git node land --amend --yes

--------------------------------- New Message ----------------------------------

benchmark: add streaming TextDecoder benchmark
PR-URL: #61549

Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>

Reviewed-By: Gürgün Dayıoğlu <hey@gurgun.day>

Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
[detached HEAD 7160f13a63] benchmark: add streaming TextDecoder benchmark

Author: Nikita Skovoroda <chalkerx@gmail.com>

Date: Wed Jan 28 03:08:19 2026 +0400

1 file changed, 55 insertions(+)

create mode 100644 benchmark/util/text-decoder-stream.js

Successfully rebased and updated refs/heads/main.
ℹ  Add commit-queue-squash label to land the PR as one commit, or commit-queue-rebase to land as separate commits.

https://github.com/nodejs/node/actions/runs/21576063433

[nodejs-github-bot]

Landed in 6c44d31...8c57908