Class OpenAiCompatibleModel

java.lang.Object
uno.anahata.asi.agi.provider.AbstractModel
uno.anahata.asi.openai.compatible.OpenAiCompatibleModel
Direct Known Subclasses:
HuggingFaceModel, MistralModel, ModalModel, NvidiaModel, OllamaModel, OpenRouterModel
public class OpenAiCompatibleModel extends AbstractModel
A concrete implementation of AbstractModel that communicates with any OpenAI-compatible Chat Completion API using the standard JDK HttpClient.

  • Field Details

    • modelId

      private final String modelId
      The unique identifier for the model (e.g., 'gpt-4o', 'claude-3-5-sonnet').

    • displayName

      private final String displayName
      The user-friendly name of the model.

    • version

      private String version
      The model version.

    • maxInputTokens

      private Integer maxInputTokens
      The maximum input tokens allowed, or null if unknown.

    • maxOutputTokens

      private Integer maxOutputTokens
      The maximum output tokens allowed, or null if unknown.

    • reasoningStyle

      private OpenAiCompatibleReasoningStyle reasoningStyle
      The reasoning extraction style used by this model.

    • reasoningFieldName

      private String reasoningFieldName
      The specific field name representing thoughts in the JSON response.

    • reasoningTags

      private List<String> reasoningTags
      The start and end tags wrapping reasoning content.

    • supportsFunctionCalling

      private boolean supportsFunctionCalling
      Whether the model supports native function calling.

    • supportsContentGeneration

      private boolean supportsContentGeneration
      Whether the model supports content generation.

    • supportsBatchEmbeddings

      private boolean supportsBatchEmbeddings
      Whether the model supports batch embeddings.

    • supportsEmbeddings

      private boolean supportsEmbeddings
      Whether the model supports simple embeddings.

    • supportsCachedContent

      private boolean supportsCachedContent
      Whether the model supports content caching.

    • HTTP_CLIENT

      private static final HttpClient HTTP_CLIENT
      The shared, thread-safe HTTP Client instance.

  • Constructor Details

    • OpenAiCompatibleModel

      public OpenAiCompatibleModel(OpenAiChatCompletionsProvider provider, String modelId, String displayName)
      Constructs an OpenAiCompatibleModel with explicit metadata parameters.
      Parameters:
      provider - the parent provider instance.
      modelId - the unique model identifier.
      displayName - the human-readable display name.

    • OpenAiCompatibleModel

      public OpenAiCompatibleModel(OpenAiChatCompletionsProvider provider, com.fasterxml.jackson.databind.JsonNode node)
      Constructs a model instance from a JSON node returned by the /models endpoint. Extracts ID, ownership, and creation timestamp for display and versioning.
      Parameters:
      provider - The parent provider.
      node - The JSON node containing model metadata.

  • Method Details

    • getProvider

      public OpenAiChatCompletionsProvider getProvider()
      Returns the parent OpenAiChatCompletionsProvider instance owning this model.
      Returns:
      The OpenAI-compatible provider instance.

    • resolveModelId

      private static String resolveModelId(com.fasterxml.jackson.databind.JsonNode node)
      Resolves the unique model ID from a model JSON node.

      Checks for id, name, or model fields, defaulting to 'unknown' if missing.

      Parameters:
      node - The JSON node containing model metadata.
      Returns:
      The resolved model ID string.

    • resolveDisplayName

      private static String resolveDisplayName(com.fasterxml.jackson.databind.JsonNode node)
      Resolves a human-readable display name from a model JSON node.

      Checks for display_name, name, or owned_by fields, defaulting cleanly to the model ID if no additional metadata is present.

      Parameters:
      node - The JSON node containing model metadata.
      Returns:
      The resolved display name string.

    • countTokens

      public int countTokens(String text)
      Counts the number of tokens in the given text string using this model's specific tokenizer.

      Utilizes TokenizerUtils to perform offline, BPE token counting based on configured TokenizerType.

      Specified by:
      countTokens in class AbstractModel
      Parameters:
      text - The text to count tokens for.
      Returns:
      The token count, or 0 if the text is null or empty.

    • countTokens

      public int countTokens(AbstractToolResponse<?> toolResponse)
      Counts the number of tokens consumed by the given tool execution response.

      Model subclasses override this to serialize the response into its exact wire-format (Protobuf FunctionResponse, JSON, etc.) to ensure 100% accurate billing.

      Serializes the base tool response to its flat JSON representation and adds the token cost of any generated attachments (like images or logs) by delegating to the generic binary tokenizer. This mirrors the exact OpenAI-compatible wire-format, where attachments are sent as a separate message item.

      Specified by:
      countTokens in class AbstractModel
      Parameters:
      toolResponse - The tool response instance to count.
      Returns:
      The precise, billing-identical token count.

    • countTokens

      public int countTokens(AbstractToolCall<?,?> toolCall)
      Counts the number of tokens consumed by the given tool call in the context window.

      Model subclasses override this method to serialize the tool call into its exact wire-format (JSON, Protobuf, etc.) to ensure 100% accurate billing.

      Serializes the tool call into a standard OpenAI function call JSON object containing 'name' and 'arguments' properties, and counts its tokens using the active BPE encoding.

      Specified by:
      countTokens in class AbstractModel
      Parameters:
      toolCall - The tool call to count tokens for.
      Returns:
      The total token count.

    • isSupportsFunctionCalling

      public boolean isSupportsFunctionCalling()
      Checks if this model supports native function calling (tools).
      Specified by:
      isSupportsFunctionCalling in class AbstractModel
      Returns:
      true if supported.

    • isSupportsContentGeneration

      public boolean isSupportsContentGeneration()
      Checks if this model supports content generation.
      Specified by:
      isSupportsContentGeneration in class AbstractModel
      Returns:
      true if supported.

    • isSupportsBatchEmbeddings

      public boolean isSupportsBatchEmbeddings()
      Checks if this model supports batch embedding generation.
      Specified by:
      isSupportsBatchEmbeddings in class AbstractModel
      Returns:
      true if supported.

    • isSupportsEmbeddings

      public boolean isSupportsEmbeddings()
      Checks if this model supports single content embedding generation.
      Specified by:
      isSupportsEmbeddings in class AbstractModel
      Returns:
      true if supported.

    • isSupportsCachedContent

      public boolean isSupportsCachedContent()
      Checks if this model supports content caching.
      Specified by:
      isSupportsCachedContent in class AbstractModel
      Returns:
      true if supported.

    • getAvailableServerTools

      public List<ServerTool> getAvailableServerTools()
      Gets the list of server-side tools available for this model.
      Specified by:
      getAvailableServerTools in class AbstractModel
      Returns:
      A list of available server tools.

    • getDefaultServerTools

      public List<ServerTool> getDefaultServerTools()
      Gets the list of server-side tools that should be enabled by default for this model.
      Specified by:
      getDefaultServerTools in class AbstractModel
      Returns:
      The list of default server tools.

    • getEndpoint

      protected String getEndpoint()
      Gets the relative endpoint URL for Chat Completion requests.
      Returns:
      the sub-resource endpoint string.

    • createModelMessage

      public OpenAiCompatibleModelMessage createModelMessage(Agi agi)
      Instantiates an OpenAI-compatible message container.
      Parameters:
      agi - the parent AGI session.
      Returns:
      a new OpenAiCompatibleModelMessage.

    • generateContent

      public Response generateContent(GenerationRequest request)
      Description copied from class: AbstractModel
      The core method for interacting with an AI model. It takes a configuration object and a list of messages and returns a standardized Response.
      Specified by:
      generateContent in class AbstractModel
      Parameters:
      request - The generation request containing config and history.
      Returns:
      A standardized Response object.

    • generateContentStream

      public void generateContentStream(GenerationRequest request, StreamObserver<Response<? extends AbstractModelMessage>> observer)
      Description copied from class: AbstractModel
      Generates content asynchronously using token streaming.
      Specified by:
      generateContentStream in class AbstractModel
      Parameters:
      request - The generation request containing config and history.
      observer - The observer that will receive the streaming response chunks.

    • routeChunk

      private void routeChunk(com.fasterxml.jackson.databind.JsonNode choice, OpenAiCompatibleModelMessage target)
      Routes a streaming JSON choice chunk to the target message, extracting thoughts or text content dynamically.
      Parameters:
      choice - the JSON choice node from the chunk event.
      target - the target message accumulating the content.

    • getToolDeclarationJson

      public String getToolDeclarationJson(AbstractTool<?,?> tool, RequestConfig config)
      Description copied from class: AbstractModel
      Gets the provider-specific JSON representation of a tool's declaration. This is used by the UI to show exactly what is being sent to the model.
      Specified by:
      getToolDeclarationJson in class AbstractModel
      Parameters:
      tool - The tool to inspect.
      config - The request configuration (e.g. to check useNativeSchemas).
      Returns:
      The JSON string representing the tool declaration.

    • buildParametersNode

      protected com.fasterxml.jackson.databind.node.ObjectNode buildParametersNode(AbstractTool<?,?> tool, boolean strict)
      Builds the JSON Schema parameters node for a tool.
      Parameters:
      tool - The tool to build parameters for.
      strict - Whether to enforce OpenAI 'strict' mode (requires additionalProperties: false).
      Returns:
      The constructed ObjectNode.

    • createResponseWithEstimatedUsage

      protected OpenAiCompatibleResponse createResponseWithEstimatedUsage(Agi agi, String modelId, String jsonPayload, String historyJson, int estimatedPromptTokens, int estimatedCompletionTokens, TokenizerType tokenizerType)
      Creates an OpenAiResponse with estimated usage metadata when the API doesn't provide it. This is used for providers like Modal's GLM-5 that return usage: null in streaming mode.
      Parameters:
      agi - The parent session.
      modelId - The model ID.
      jsonPayload - The request payload JSON.
      historyJson - The history JSON.
      estimatedPromptTokens - Estimated prompt tokens.
      estimatedCompletionTokens - Estimated completion tokens.
      tokenizerType - The tokenizer used for estimation.
      Returns:
      A new OpenAiResponse with estimated usage metadata.

    • preparePayload

      protected com.fasterxml.jackson.databind.node.ObjectNode preparePayload(GenerationRequest request, boolean stream)
      Prepares the final JSON request payload combining system instructions, history, tools, and temperature.
      Parameters:
      request - the current generation request.
      stream - whether the request is streaming.
      Returns:
      the constructed JSON ObjectNode payload.

    • enrichPayload

      protected void enrichPayload(com.fasterxml.jackson.databind.node.ObjectNode payload, GenerationRequest request)
      Hook for subclasses to add or modify payload parameters before sending the request.
      Parameters:
      payload - The JSON payload being constructed.
      request - The generation request.

    • countTokens

      public int countTokens(byte[] data, String mimeType)
      Counts the number of tokens consumed by raw binary data based on its MIME type and model-specific multimodal billing rules.

      This generic signature provides complete decoupling from domain part classes, allowing the model to tokenize any binary payload (such as blob parts or tool attachments).

      Calculates the exact, model-specific multimodal token count for OpenAI-compatible image data. Delegates the header-only image dimension reading to the core ImageMetadataUtils utility, and performs the OpenAI-specific high-detail scaling and tiling calculations.

      Specified by:
      countTokens in class AbstractModel
      Parameters:
      data - The raw binary data of the file or attachment.
      mimeType - The detected MIME type of the binary data (e.g. "image/png").
      Returns:
      The precise, billing-identical multimodal token count.