This package contains a high-performance binary serializer for Go that encodes and decodes arbitrary data structures of variable size. It is designed as a simple json.Marshal / json.Unmarshal drop-in for cases where you control both ends and want compact, fast payloads — for example inter-broker message encoding in emitter.
- Simple API that mirrors
encoding/jsonwithMarshal,Unmarshal, andMarshalTo. - Compact payloads using varint encoding for integers and size-prefixed variable-length values.
- Zero-allocation encoding path via
MarshalTowriting directly to anio.Writer. - Reflect-based support for structs, maps, slices, arrays, pointers, and nested types.
- Fast paths for
[]byteand other common slice types. - Custom serialization via
encoding.BinaryMarshaler/BinaryUnmarshaleror a fullCodecthroughGetBinaryCodec. - Field skipping with the
binary:"-"struct tag. - Optional subpackages for sorted, unsafe, and nocopy typed slices when you need smaller payloads or lower decode cost.
Variable-sized values are prefixed with a varint-encoded size and encoded recursively. The format is intentionally not versioned or cross-language — this is for efficient exchange of known Go types between systems you control.
- Quick Start
- Streaming Encode and Decode
- Skipping Fields
- Custom Serialization
- Typed Slice Subpackages
- Benchmarks
- Disclaimer
- Contributing
- License
Define a message and marshal it the same way you would with JSON:
type message struct {
Name string
Timestamp int64
Payload []byte
Ssid []uint32
}
v := &message{
Name: "Roman",
Timestamp: 1242345235,
Payload: []byte("hi"),
Ssid: []uint32{1, 2, 3},
}
encoded, err := binary.Marshal(v)
if err != nil {
panic(err)
}
var out message
err = binary.Unmarshal(encoded, &out)For hot paths, write into a reused buffer with MarshalTo, or use Encoder / Decoder directly against an io.Writer / io.Reader:
var buf bytes.Buffer
if err := binary.MarshalTo(v, &buf); err != nil {
panic(err)
}
dec := binary.NewDecoder(&buf)
var out message
if err := dec.Decode(&out); err != nil {
panic(err)
}Fields tagged with binary:"-" are ignored during encode and decode. Useful for locks, caches, or derived state:
type Cache struct {
mu sync.Mutex `binary:"-"`
Key string
Value []byte
}By default, values are encoded through reflection. You can override that for a type in two ways, checked in this order:
GetBinaryCodec()— return abinary.Codecfor full control over the wire format (no extra length prefix).MarshalBinary/UnmarshalBinary— the standardencoding.BinaryMarshaler/BinaryUnmarshalerpair; the package length-prefixes the returned bytes for you.
Use MarshalBinary when you already have a []byte representation. Use GetBinaryCodec when you want to stream fields through the encoder without an intermediate buffer.
This matches encoding.BinaryMarshaler and encoding.BinaryUnmarshaler. On encode, the returned slice is written as uvarint(length) + bytes. On decode, that framed blob is passed to UnmarshalBinary:
type CompactHeader struct {
Code uint8
}
func (h CompactHeader) MarshalBinary() ([]byte, error) {
return []byte{h.Code}, nil
}
func (h *CompactHeader) UnmarshalBinary(data []byte) error {
if len(data) != 1 {
return fmt.Errorf("CompactHeader: want 1 byte, got %d", len(data))
}
h.Code = data[0]
return nil
}
encoded, err := binary.Marshal(CompactHeader{Code: 0x13})
// encoded == []byte{0x01, 0x13} // length + payload
var out CompactHeader
err = binary.Unmarshal(encoded, &out)This is enough for most custom types, including anything that already implements the standard library interfaces (for example time.Time).
For tighter packing or to avoid the intermediate []byte, implement GetBinaryCodec on a pointer receiver. It must return a binary.Codec:
type Codec interface {
EncodeTo(*Encoder, reflect.Value) error
DecodeTo(*Decoder, reflect.Value) error
}Example: encode a 2D point as two raw float64 values with no struct field metadata:
type Point struct {
X float64
Y float64
}
func (p *Point) GetBinaryCodec() binary.Codec {
return pointCodec{}
}
type pointCodec struct{}
func (pointCodec) EncodeTo(e *binary.Encoder, rv reflect.Value) error {
p := rv.Interface().(Point)
e.WriteFloat64(p.X)
e.WriteFloat64(p.Y)
return nil
}
func (pointCodec) DecodeTo(d *binary.Decoder, rv reflect.Value) error {
x, err := d.ReadFloat64()
if err != nil {
return err
}
y, err := d.ReadFloat64()
if err != nil {
return err
}
rv.Set(reflect.ValueOf(Point{X: x, Y: y}))
return nil
}Because GetBinaryCodec takes priority, a type should not also rely on MarshalBinary for the same purpose — pick one approach per type.
Nested use works automatically: if a struct field's type implements either customization, that field uses the custom path while the rest of the struct uses the default reflect codecs.
type Packet struct {
ID uint64
At time.Time // uses time.Time's BinaryMarshaler
Pos Point // uses GetBinaryCodec above
Raw []byte
}Optional helpers live in subpackages when the default reflect path is not enough:
| Package | Purpose |
|---|---|
sorted |
Delta-encoded sorted integer / timestamp slices for smaller wire size |
unsafe |
Memory-cast numeric slices (faster, not portable across endianness) |
nocopy |
Like unsafe, but decode reuses the input buffer (zero-copy; lifetime tied to the buffer) |
import (
"github.com/kelindar/binary"
"github.com/kelindar/binary/sorted"
)
v := sorted.Int32s{4, 5, 6, 1, 2, 3}
encoded, err := binary.Marshal(&v)
var out sorted.Int32s
err = binary.Unmarshal(encoded, &out)See each subpackage README for trade-offs and warnings.
Numbers from bench/. Run locally with:
cd bench && go run .name time/op ops/s allocs/op
-------------------- ------------ ------------ ------------
binary/enc 139.2 ns 7.2M 2
binary/enc-to 104.7 ns 9.5M 0
binary/dec 175.6 ns 5.7M 5
binary/map-enc 9.5 µs 105.2K 210
binary/map-dec 14.4 µs 69.4K 511
binary/slice-enc 9.4 µs 106.5K 9
binary/slice-dec 14.9 µs 67.2K 502
binary/nest-enc 5.0 µs 201.9K 14
binary/nest-dec 8.3 µs 120.8K 271
binary/bytes-enc 835.2 ns 1.2M 3
binary/bytes-dec 821.2 ns 1.2M 2
binary/u64-enc 75.3 µs 13.3K 11
binary/u64-dec 58.4 µs 17.1K 2
binary/reuse-enc 113.2 ns 8.8M 0
binary/stream-dec 240.8 ns 4.2M 5
nocopy/str-enc 109.7 ns 9.1M 3
nocopy/str-dec 39.2 ns 25.5M 0
nocopy/dict-enc 185.7 ns 5.4M 2
nocopy/dict-dec 154.5 ns 6.5M 2
nocopy/bmap-enc 313.3 ns 3.2M 5
nocopy/bmap-dec 160.9 ns 6.2M 2
nocopy/hmap-enc 307.6 ns 3.3M 5
nocopy/hmap-dec 146.6 ns 6.8M 2
nocopy/bytes-enc 862.0 ns 1.2M 3
nocopy/bytes-dec 41.6 ns 24.0M 0
nocopy/u64-enc 5.3 µs 187.6K 3
nocopy/u64-dec 39.8 ns 25.1M 0
nocopy/col-enc 524.3 ns 1.9M 9
nocopy/col-dec 532.2 ns 1.9M 8
nocopy/struct-enc 162.9 ns 6.1M 3
nocopy/struct-dec 82.7 ns 12.1M 0
sorted/i32-enc 78.9 µs 12.7K 5
sorted/i32-dec 551.8 µs 1.8K 29.8K
sorted/u32-enc 76.2 µs 13.1K 5
sorted/u32-dec 543.5 µs 1.8K 29.8K
sorted/ts-enc 52.5 µs 19.0K 6
sorted/ts-dec 21.8 µs 45.8K 2
sorted/tsz-enc 126.4 µs 7.9K 7
sorted/tsz-dec 119.3 µs 8.4K 3
sorted/tcz-enc 85.9 µs 11.6K 6
sorted/tcz-dec 76.1 µs 13.1K 3
unsafe/u64-enc 459.8 ns 2.2M 3
unsafe/u64-dec 450.1 ns 2.2M 2
This is not a replacement for JSON, protobuf, or other versioned interchange formats. The codec does not maintain schema evolution or cross-language compatibility. Use it to exchange binary data of a known format between Go services where you control both ends.
Contributions are welcome — open a pull request and we will review it as quickly as we can. This library is maintained by Roman Atachiants.
Binary is licensed under the MIT License.