BoxLang AI 3.4.0 is one of the biggest releases in the module's history, and it is built around a single theme: trust. Every agent you ship eventually needs a human to step in, and every agent that reads untrusted content is a target for prompt injection. This release gives you both a proper answer.

At the center of it all is the new Gateway SPI, a full Human-in-the-Loop (HITL) subsystem with durable approval grants, three phases of prompt-injection and data-loss guardrails, batched tool-call approvals, normalized reasoning across every provider, and AWS Bedrock parity with the rest of the pack.

Let's dig in. You can also check out the full What's New Guide Here: https://ai.ortusbooks.com/readme/release-history/3.4.0

🔌 The Gateway SPI: IGateway

A gateway is a bidirectional human-interaction adapter. It turns platform events, a CLI keystroke, an HTTP webhook, a button click on a chat platform, into normalized agent input, and turns agent events (including a suspended approval request) back into a platform-native experience.

Every gateway implements the same IGateway interface. Unsupported capabilities fall back to a safe default, so a gateway only needs to override what it actually does.

// Core gateways resolve by name
cli  = aiGateway( "cli" )
http = aiGateway( "http", { secret: "shared-hmac-secret" } )

// External gateway modules register under their own name
aiGatewayRegistry().register( new MyPlatformGateway(), "my-platform" )
myGateway = aiGateway( "my-platform" )

// Attach any gateway to HITL middleware
aiAgent(
    middleware  : new HumanInTheLoopMiddleware( gateway: aiGateway( "http" ) ),
    checkpointer: aiMemory( "cache" )
)

Three gateways ship in the box:

Gateway	What it does
CliGateway	The reference implementation. Blocking stdin/stdout approval prompt, now with approve_always/approve_session
HttpGateway	HMAC-SHA256 signed requests, timestamp tolerance, nonce dedup, TTL-bounded pending interactions, and atomic decision claims
MockGateway	In-memory reference gateway for tests and examples

External platforms (Slack, Discord, Teams, and others) can ship as their own modules and register a gateway at load time. Your agent code never has to know or care which one it's talking to. In the next few days we will be introducing our new Agent Framework: BxAgents, which brings about tons of gateways.

🧑‍⚖️ HITL, fully extracted, with durable grants

Human-in-the-loop approval used to live inside HumanInTheLoopMiddleware. In 3.4.0 it's been pulled out into a dedicated models/hitl/ package: an IApprovalPolicy decides whether a tool call needs approval, and a HumanInteractionCoordinator owns presenting the request through a gateway and resolving the decision. The middleware itself is now a thin adapter over the two.

import bxModules.bxai.models.middleware.core.HumanInTheLoopMiddleware;

// Simple: match by tool name (default policy)
hitl = new HumanInTheLoopMiddleware( toolsRequiringApproval: [ "deleteRecord" ] )

// Or supply any IApprovalPolicy, risk-based, callback-based, composite, or your own
hitl = new HumanInTheLoopMiddleware(
    policy : new RiskLevelApprovalPolicy( minLevel: "high" ),
    gateway: aiGateway( "http" )
)

agent = aiAgent( middleware: [ hitl ], checkpointer: aiMemory( "cache" ) )

The bigger deal is durable grants. When a human says "always allow this tool", that decision is now backed by a pluggable IDecisionStore (cache, jdbc, or file) and survives past the current run, even across a restart, instead of asking the same question forever.

store = aiDecisionStore( "jdbc", { datasource: "myDSN", table: "ai_decisions" } )
hitl  = new HumanInTheLoopMiddleware( toolsRequiringApproval: [ "placeOrder" ], decisionStore: store )

📦 Batched approvals: one suspension, not one per call

Previously, if a turn asked for multiple tool calls that each needed approval, only the first one suspended and the rest were silently skipped. That's fixed. Now a turn with several pending calls suspends once, as a single checkpoint, and resuming finishes the whole batch directly against the saved assistant message. Nothing runs twice, and there is no replay of the LLM call, which ultimately saves you tokens $$$$.

agent = aiAgent(
    tools       : [ getWeatherTool, sendEmailTool ],
    middleware  : [ new HumanInTheLoopMiddleware( toolsRequiringApproval: [ "get_weather", "send_email" ] ) ],
    checkpointer: aiMemory( "cache" )
)

result = agent.run( "Check the weather in KC and email me the result", {}, { threadId: "t1" } )
// result.isSuspended() == true, with BOTH tool calls pending in ONE checkpoint

// One decision applies to every pending call...
final = agent.resume( "approve", "t1" )

// ...or resolve each one individually
final = agent.resume(
    [
        { decision: "approve" },
        { decision: "reject", reason: "not needed" }
    ],
    "t1"
)

This is consistent across OpenAI, Claude, Bedrock, and Cohere, with streaming batching covering OpenAI and Claude.

🛡️ Security & Guardrails: three phases, all opt-in

This is the heart of the release. BoxLang AI now ships layered, configurable defense against prompt injection and data loss, all controlled through one settings switch (or attached directly as middleware, per agent).

// boxlang.json
{
  "modules": {
    "bxai": {
      "settings": {
        "security": {
          "enabled": false,
          "input": {
            "enabled": true,
            "action": "flag",
            "detectors": [],
            "customPatterns": [],
            "normalizeUnicode": true,
            "stripZeroWidth": true,
            "scanToolResults": true
          },
          "fencing": {
            "enabled": true,
            "fenceContext": true,
            "escapeBindings": true,
            "preamble": ""
          }
        }
      }
    }
  }
}

Turn on security.enabled and it auto-attaches to every chat request across aiChat(), aiModel(), and aiAgent(). Any single request can opt out with { secure: false }. Unicode hygiene, NFKC normalization plus zero-width stripping, applies even when the feature is off. It neutralizes invisible-character injection and carries virtually no risk.

Phase 1, InputSanitizerMiddleware. Heuristic scanning of inbound user (and optionally tool/MCP) content for injection patterns: instruction override, role impersonation, jailbreak framing, invisible Unicode, suspicious base64 blobs, and exfiltration-shaped URLs.

sanitizer = new bxModules.bxai.models.middleware.security.InputSanitizerMiddleware(
    action         : "strip",
    detectors      : [ "instructionOverride", "jailbreak" ],
    customPatterns : [ { name: "internalCodes", regex: "(?i)PROJ-[0-9]{4}" } ],
    scanToolResults: true
)

agent = aiAgent( name: "support-bot", middleware: [ sanitizer ] )

Each detector can block (throw before a single token is spent), strip, flag, or log.

Phase 2, untrusted-content fencing. The number one real-world LLM attack is indirect: an attacker hides instructions inside a document, a web page, or a tool result your app retrieves, and the model can't tell your instructions from that data. Fencing wraps untrusted content in tamper-resistant boundary markers so the model treats it as data, never as instructions.

context = aiFence( retrievedDoc, "knowledge-base" )
answer  = aiChat( "Answer using this context: #context#" )

${Setting: context not found} template bindings are auto-fenced by default, and ${Setting: ... not found} inside binding values is escaped by default to stop template-confusion injection. Both apply even with security.enabled: false.

Phase 3, OutputGuardMiddleware. Guards what comes out of the model: redacts secrets and PII (email, SSN, credit card with Luhn validation, API keys, JWTs) and strips data-exfiltration markdown. Fully offline, no second model, no network call, and it scans the model's reasoning too, not just its final answer.

guard = new bxModules.bxai.models.middleware.security.OutputGuardMiddleware( action: "redact" )
aiAgent( name: "support", middleware: [ guard ] )

And for the attacks the heuristics miss: LLMGuardMiddleware runs a second, typically cheaper or local model as a judge, classifying requests (and optionally responses) as SAFE, INJECTION, or HARMFUL.

guard = new LLMGuardMiddleware( judge: { provider: "ollama", model: "llama-guard3" } )
aiAgent( name: "support-bot", middleware: [ guard ] )

There's also a new mock provider, a deterministic, offline provider for testing, so you can exercise your entire middleware pipeline, HITL and guardrails included, in CI without live credentials.

🧠 Normalized reasoning, everywhere

Reasoning-capable models could always be enabled, params passes straight through to the provider. But the reasoning that came back was parsed out and silently dropped. It now surfaces consistently on message.reasoning (or delta.reasoning while streaming), regardless of which model is behind it.

result = aiChat( "Solve this step by step: ...", params: {
    thinking: { type: "enabled", budget_tokens: 10000 }
}, options: { returnFormat: "raw" } )

reasoning = result.choices[1].message.reasoning ?: ""
answer    = result.choices[1].message.content

Reasoning is kept strictly separate from content and is never persisted to memory. The model's private thinking is never replayed back to it as if it had said it.

🎮 Agent Run control: cancelRun() / steerRun()

Every agent now supports cancelling or steering a run already in flight, addressed purely by threadId.

agent.cancelRun( threadId )                        // stop at the next checkpoint
agent.steerRun( threadId, "actually, focus on X" )  // splice a message into the live turn

This is also what powers aiGatewaySession()'s steer and interrupt policies, covered below, and it's available directly to any caller.

🔌 Gateway Sessions: aiGatewaySession()

Wires one agent to one or more gateways for inbound message handling. A message arrives, the session dispatches it as an agent turn, and relays the output back through whichever gateway it came in on.

session = aiGatewaySession(
    agent   : myAgent,
    gateways: [ "cli", "http" ],
    policy  : "queue"
)
session.start()

Four policies control what happens when a second message hits a busy thread: reject, queue (default), steer (spliced into the live turn), and interrupt (cancels the current run at its next checkpoint, then dispatches next).

☁️ AWS Bedrock provider parity

Bedrock now has the same authentication depth as every other cloud provider: bearer-token auth, the full AWS credential chain (explicit, environment, ECS/EKS container including EKS Pod Identity, EC2 IMDSv2) with expiry-aware caching, Guardrails support, baseURL overrides, Cohere/Titan-v2 embedding shapes, and confirmed tool-use for Claude on Bedrock.

// Bearer token, simplest path, no SigV4 signing required
result = aiChat( "Hello", provider: "bedrock", options: {
    providerOptions: { region: "us-east-1", bearerToken: "..." }
} )

// Or let the default credential chain resolve automatically
result = aiChat( "Hello", provider: "bedrock", options: {
    providerOptions: { region: "us-east-1" }
} )

Batched HITL approvals are now consistent across OpenAI, Claude, Bedrock, and Cohere.

🧠 Memory: token-based summarization

SummaryMemory can now trigger compression by estimated token count instead of message count.

memory = aiMemory(
    memory: "summary",
    config: { maxTokens: 4000, maxMessages: 0, summaryThreshold: 10 }
)

maxTokens and maxMessages are mutually exclusive triggers, set one, not both. And summarize(), previously only on SummaryMemory, is now on the IAiMemory interface and implemented by every built-in memory type.

🐛 Notable fixes

A few of the fixes worth calling out on their own:

• Tool-call middleware hooks (beforeToolCall/afterToolCall/wrapToolCall) never fired for Claude, Bedrock, or Cohere, tools were invoked directly, bypassing middleware entirely. All three now go through the same pipeline as OpenAI.
• Enabling Claude extended thinking broke aiChat() outright. The synchronous path read the answer from the wrong block position, so with thinking enabled the answer silently came back empty. Now correctly selects the first text block.
• approve_always/approve_session grants never persisted for async, non-CLI, gateways.
• MCP tools crashed the Claude and Bedrock providers due to a missing default schema method.
• AWS profile-file credentials never worked at all due to a function-name typo.

See the full release history for the complete list.

🔄 Updated defaults

• Default AI request timeout rises from 45 to 90 seconds.
• Groq's default model moves to openai/gpt-oss-20b after an upstream deprecation.
• OpenAI's default model moves to gpt-5.6-luna.
• Claude's default model moves to claude-sonnet-5.

Migration notes

Everything under settings.security defaults to enabled: false, so there's no behavior change unless you opt in, except unicode hygiene, which is on unconditionally by design. Existing mode: "cli" / mode: "web" HITL configuration keeps working exactly as before; prefer gateway: going forward when you're attaching something specific. agent.resume() still accepts a single decision exactly as before, arrays of per-call decisions are additive. Full details are in the migration guide.

Get it now

box install bx-ai

or

install-bx-module bx-ai

Full docs: ai.ortusbooks.com | Repo: github.com/ortus-boxlang/bx-ai

---

BoxLang AI is built and maintained by Ortus Solutions, the makers of BoxLang, ColdBox, CommandBox, and the broader Box ecosystem.