Class TestCatalog

java.lang.Object
uno.anahata.asi.yam.tools.benchmarks.TestCatalog
Direct Known Subclasses:
Agi1TestCatalog
public abstract class TestCatalog extends Object
Abstract base catalog representing an extensible benchmark suite and repository of test definitions.

Encapsulates the suite identifier, display name, description, standard prompt header/footer templates, customizable results storage directory, and registered TestDefinitions.

Author:
anahata

  • Field Details

    • id

      private String id
      The unique identifier code for the catalog (e.g., "ANAHATA-AGI-1").

    • name

      private String name
      The human-readable name of the catalog.

    • description

      private String description
      The detailed description of the suite's goals and scope.

    • standardHeader

      private String standardHeader
      The standard header prepended to all test prompts in this catalog.

    • standardFooter

      private String standardFooter
      The standard footer appended to all test prompts in this catalog.

    • resultsDirectory

      private Path resultsDirectory
      The customizable filesystem directory where JSON results files are persisted.

    • tests

      private final List<TestDefinition> tests
      The list of registered test definitions in this catalog.

  • Constructor Details

    • TestCatalog

      public TestCatalog(String id, String name, String description, String standardHeader, String standardFooter, Path resultsDirectory)
      Base constructor for a benchmark test catalog.
      Parameters:
      id - The unique catalog identifier.
      name - The human-readable catalog name.
      description - The catalog description.
      standardHeader - The standard prompt header template.
      standardFooter - The standard prompt footer template.
      resultsDirectory - The filesystem directory where results are stored.

  • Method Details

    • getTests

      public List<TestDefinition> getTests()
      Retrieves an unmodifiable view of all registered test definitions.
      Returns:
      List of test definitions.

    • addTest

      public void addTest(TestDefinition test)
      Adds a test definition to this catalog.
      Parameters:
      test - The test definition to register.

    • findByCode

      public Optional<TestDefinition> findByCode(String testCode)
      Finds a test definition by its unique test code (case-insensitive).
      Parameters:
      testCode - The test code to look up.
      Returns:
      An Optional containing the matching test definition if found.

    • formatPrompt

      public String formatPrompt(TestDefinition test)
      Formats the full prompt for a test by applying this catalog's standard header and footer templates.
      Parameters:
      test - The test definition.
      Returns:
      The formatted prompt ready for submission to the candidate AGI.

    • getResultsFileForTest

      public Path getResultsFileForTest(String testCode)
      Resolves the JSON scorecard path for a specific test code within this catalog's results directory.
      Parameters:
      testCode - The test code (e.g. "JAVA-JNA-1").
      Returns:
      The path to the JSON results file.

    • toString

      public String toString()
      Formats this catalog and its test definitions into a clean Markdown representation suitable for RAG message prompt augmentation.
      Overrides:
      toString in class Object
      Returns:
      The Markdown string describing this catalog.