frontiers

Here is 1 public repository matching this topic...

ziegler-ingo / rate-utility-frontiers

Code and resources for the paper "Rate-Utility Frontiers for Language Encodings: Comparing Tokens, Bytes, and Pixels Under Controlled Linguistic Content"

compression transformers frontiers tokenization encoders multiscript multilingual-nlp tokenization-free rate-utility

Updated Jul 20, 2026
Python

Improve this page

Add a description, image, and links to the frontiers topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the frontiers topic, visit your repo's landing page and select "manage topics."

Learn more

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

frontiers

Here is 1 public repository matching this topic...

ziegler-ingo / rate-utility-frontiers

Improve this page

Add this topic to your repo