Wingetlywingetly
← All apps

Tag · microsoft.ml.tokenizers

Microsoft.ml.tokenizers apps on winget

1 microsoft.ml.tokenizers app indexed from microsoft/winget-pkgs. Each one ships with a ready-to-paste winget install command and a silent variant for unattended deployment.

  • T
    TokenizerMicrosoft Corporation
    Microsoft.Tokenizerv1.3.3

    Contains Typescript and C# implementation of byte pair encoding (BPE) tokenizer for OpenAI LLMs, it is based on open sourced rust implementation in the OpenAI tiktoken.