Skip to content

Jaccl refactor#3412

Merged
angeloskath merged 14 commits into
mainfrom
jaccl-refactor
Apr 15, 2026
Merged

Jaccl refactor#3412
angeloskath merged 14 commits into
mainfrom
jaccl-refactor

Conversation

@angeloskath

Copy link
Copy Markdown
Member

This PR does a few things:

  1. Refactors JACCL to a standalone C++ library installable with a few lines of CMake
    a. Adds README and examples
    b. Adds a NCCL style benchmark
  2. Changes the mesh all reduce to a deterministic one
    a. We pay a small BW penalty and even smaller latency penalty but it does make things more predictable and stable. Eg people don't have to make sure that they always sample the same token or deal with diverging KV caches etc
  3. A few more robustness improvements like doing a double all_gather in the side-channel barrier to ensure no node can run ahead.
  4. Fixes Fix JACCL GID index for multi-node Thunderbolt 5 setups #3389 with a more general solution that checks for IPv4 and also checks all entries.
  5. Left the best for last, compiles the reductions for armv8.6-a and chooses it at runtime if available, which results in much faster reduction for bf16.
    a. Specifically we now reach 12.2 GB/s algorithm bandwidth for bf16 up from ~9 GB/s for a 256 MB all reduce

@angeloskath angeloskath requested review from nastya236 and zcbenz April 15, 2026 00:21

@zcbenz zcbenz left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants