Apps Packs Docs

Pack GitHub SponsorPckgr↗

wingetly

Browse, bundle, install. Every winget app in one place.

An independent index for the Windows Package Manager ecosystem.

Catalog

All apps
Packs
Docs

Project

Privacy
About

Sponsor

Wingetly is sponsored by Pckgr - automated application patching for Microsoft Intune.

© 2026 WingetlyData from microsoft/winget-pkgs

Apps Packs Docs

Pack GitHub SponsorPckgr↗

Results for "tags:Microsoft.DeepDev.TokenizerLib"

1 winget apps indexed. Showing results for tags:Microsoft.DeepDev.TokenizerLib.

Filters1⌄

Search

Sort

Installer type

MSIMSIXEXEInnoNSISWiXZIPPortable

Architecture

x64x86arm64armneutral

Other

Has icon

Showing 1–1 of 1Reset filters

T
TokenizerMicrosoft Corporation
Microsoft.Tokenizerv1.3.3
Contains Typescript and C# implementation of byte pair encoding (BPE) tokenizer for OpenAI LLMs, it is based on open sourced rust implementation in the OpenAI tiktoken.

wingetly

Browse, bundle, install. Every winget app in one place.

An independent index for the Windows Package Manager ecosystem.

Catalog

All apps
Packs
Docs

Project

Privacy
About

Sponsor

Wingetly is sponsored by Pckgr - automated application patching for Microsoft Intune.

© 2026 WingetlyData from microsoft/winget-pkgs