llama3-tokenizer-js

esm
JS tokenizer for LLaMA 3
Version 1.2.0 License MIT
Keywords
llamallmtokenizer
<script type="module"> import llama3TokenizerJs from 'https://cdn.jsdelivr.net/npm/llama3-tokenizer-js@1.2.0/+esm' </script>
JS tokenizer for LLaMA 3