We're excited to announce BoxLang AI v2.1.0, a major release that brings enterprise-grade features to your AI-powered applications. This release focuses on production readiness with multi-tenant usage tracking, scalable OpenSearch vector memory, complete AWS Bedrock integration, and simplified provider configuration.

🎯 What's New

You can find the what's new document here: https://ai.ortusbooks.com/readme/release-history/2.1.0 in our complete documentation book. Also remember that it includes it's own documentation MCP server so you can integrate it with your Agentic Coding IDE: https://ai.ortusbooks.com/~gitbook/mcp

Docker Desktop AI Support

Run local AI models directly from Docker Desktop with zero configuration. BoxLang AI now natively supports Docker Desktop's AI features, making it effortless to develop and test AI applications locally without external API dependencies.

// Use Docker Desktop AI models with automatic detection
result = aiChat(
    provider: "docker",
    messages: "Explain BoxLang closures",
    params: {
        model: "llama3.2:latest"
    }
)

// Stream responses from Docker Desktop models
aiChatStream(
    provider: "docker",
    messages: "Write a BoxLang function",
    params: {
        model: "qwen2.5:0.5b-instruct"
    },
    onChunk: ( chunk ) => {
        writeOutput( chunk.content )
        flush()
    }
)

Perfect for:

• Local development without internet connectivity
• Cost-free testing of AI features
• Privacy-sensitive applications that can't send data externally
• Rapid prototyping with instant model access

Multi-Tenant Usage Tracking & Billing

Track AI usage across customers, cost centers, and projects with provider-agnostic request tagging. Perfect for SaaS platforms and enterprise applications that need per-tenant billing.

// Tag every AI request with tenant context
result = aiChat(
    messages: "Generate quarterly sales report",
    options: {
        tenantId: "customer-123",
        usageMetadata: {
            costCenter: "sales",
            projectId: "q4-2024",
            userId: "john@company.com"
        }
    }
)

Capture token usage in real-time using interceptors:

BoxRegisterInterceptor( {
    class: "BillingInterceptor",
    points: {
        onAITokenCount: ( event, data ) => {
            // Record usage: data.usage, data.tenantId, data.usageMetadata
            queryExecute(
                "INSERT INTO usage_logs (tenant_id, tokens, metadata, timestamp) 
                 VALUES (:tid, :tokens, :meta, :ts)",
                {
                    tid: data.tenantId,
                    tokens: data.usage.total_tokens,
                    meta: serializeJSON( data.usageMetadata ),
                    ts: now()
                }
            )
        }
    }
} )

Works seamlessly across all providers: OpenAI, Bedrock, Ollama, Anthropic, Gemini, DeepSeek, and more.

OpenSearch Vector Memory

Production-grade vector search with AWS OpenSearch or self-hosted OpenSearch clusters. Store millions of conversation messages with semantic search capabilities for intelligent RAG applications.

// AWS OpenSearch with SigV4 authentication
memory = aiMemory(
    provider: "opensearch",
    options: {
        url: "https://search-domain.us-east-1.es.amazonaws.com",
        indexName: "conversations",
        embeddingProvider: "openai",
        vectorDimension: 1536,
        awsRegion: "us-east-1"
    }
)

// Multi-tenant isolation built-in
memory.add(
    messages: messages,
    userId: "user-123",
    conversationId: "support-ticket-456"
)

results = memory.query(
    query: "pricing discussions",
    userId: "user-123",
    limit: 5
)

Customize HNSW parameters for optimal performance:

memory = aiMemory(
    provider: "opensearch",
    options: {
        spaceType: "cosinesimilarity",
        hnswM: 16,
        hnswEfConstruction: 100,
        hnswEfSearch: 100
    }
)

Predefined Provider Configuration

Define providers once in your module configuration, use them everywhere with clean, maintainable code.

// ModuleConfig.bx
settings = {
    providers: {
        "openai": {
            params: { model: "gpt-4" },
            options: { apiKey: getSystemSetting( "OPENAI_KEY" ) }
        },
        "ollama": {
            params: { model: "qwen2.5:0.5b-instruct" },
            options: { baseUrl: "http://ollama.internal:11434/" }
        }
    }
}

Then use providers by name throughout your application:

// Automatically uses configured defaults
result = aiChat( provider: "openai", messages: "Hello" )

// Override with runtime options
model = aiModel( 
    provider: "ollama",
    options: { logRequestToConsole: true }
)

Full AWS Bedrock Support

Complete AWS Bedrock integration with chat, streaming, and embeddings support for all model families including Claude, Titan, Llama, and Mistral.

// Chat completions
result = aiChat(
    provider: "bedrock",
    messages: "Analyze customer sentiment",
    params: { 
        model: "anthropic.claude-3-sonnet-20240229-v1:0" 
    }
)

// Streaming responses
aiChatStream(
    provider: "bedrock",
    messages: "Write a detailed product description",
    params: { 
        model: "anthropic.claude-3-sonnet-20240229-v1:0" 
    },
    onChunk: ( chunk ) => {
        writeOutput( chunk.content )
        flush()
    }
)

// Generate embeddings
embeddings = aiEmbeddings(
    provider: "bedrock",
    input: "Customer feedback analysis",
    params: {
        model: "amazon.titan-embed-text-v1"
    }
)

Use inference profiles for better latency and throughput:

result = aiChat(
    provider: "bedrock",
    messages: "Analyze customer sentiment",
    options: {
        providerOptions: {
            inferenceProfileArn: "arn:aws: bedrock:us-east-1: 123:inference-profile/production"
        }
    }
)

Custom OpenAI-Compatible Embeddings

Point your vector memory to any OpenAI-compatible embedding service - Ollama, LM Studio, or custom endpoints.

// Use Ollama for embeddings
memory = aiMemory(
    provider: "pinecone",
    options: {
        embeddingProvider: "openai",
        embeddingOptions: {
            baseURL: "http://localhost:11434/v1"
        }
    }
)

// Use LM Studio embeddings
memory = aiMemory(
    provider: "qdrant",
    options: {
        embeddingProvider: "openai",
        embeddingOptions: {
            baseURL: "http://localhost:1234/v1"
        }
    }
)

🔧 Improvements & Fixes

• New Event: onMissingAiProvider for graceful handling of missing providers
• Enhanced Configuration: aiModel() now accepts options struct for service seeding
• Request Control: mergeServiceParams() and mergeServiceHeaders() accept override argument
• Ollama Support: Added nomic-embed-text model for local embeddings
• Corrected Events: Chat events now properly named onAIChatRequest, onAIChatRequestCreate, onAIChatResponse
• Header Passthrough: Fixed headers in aiChat() and aiChatStream()
• MCP Compliance: Prompts return arguments key per MCP specification
• Model Retrieval: Fixed AiModel.getModel() with predefined providers
• Docker Stability: Increased Model Runner retry logic for large model loading

📚 Upgrade Guide

This release is fully backward compatible. New features are opt-in:

1. Add multi-tenant tracking - Include tenantId and usageMetadata in your requests
2. Configure providers centrally - Move provider config to module settings
3. Deploy OpenSearch - Switch to OpenSearch for production-scale vector search
4. Enable Bedrock - Add full chat, streaming, and embeddings support for AWS Bedrock
5. Custom embeddings - Point to self-hosted embedding services

🚀 Getting Started

Install or upgrade BoxLang AI:

// Via the OS installer
install-bx-module bx-ai

// or via CommandBox
box install bx-ai

Check out the complete documentation at https://ai.ortusbooks.com

Get Started Today

• Documentation: ai.ortusbooks.com
• Website: ai.boxlang.io
• Bootcamp: github.com/ortus-boxlang/bx-ai/bootcamp
• Examples: github.com/ortus-boxlang/bx-ai/examples
• Professional Services: ai.ortussolutions.com

Join our community and help shape the future of AI on the JVM!