Package uno.anahata.asi.gemini.tokenizer

package uno.anahata.asi.gemini.tokenizer

Provides the custom Byte Pair Encoding (BPE) and merges tokenizer engine for Gemini.

This is a specialized bug fix for a memory leak in Google's GenAI Java SDK. To be removed in future GenAI Java SDK versions, possibly v1.54.

Author:

anahata

Related Packages

Package	Description
uno.anahata.asi.gemini	Provides the provider implementation for Google's Gemini API using the official GenAI Java SDK.
uno.anahata.asi.gemini.adapter	Provides adapter classes for converting between the core domain model and the native Gemini API types.
uno.anahata.asi.gemini.schema	Provides schema generation and sanitization utilities for the Gemini provider.

Classes

Class	Description
LocalTokenizer