Luis Majano

May 11, 2023

Spread the word

Share your thoughts

We are excited to announced the release of Testbox version 5, which brings a host of new features and improvements for developers. TestBox is a powerful and flexible tool that helps developers write comprehensive BDD/TDD tests for their applications, ensuring code quality and reducing the likelihood of bugs and errors. With TestBox v5, developers can take advantage of new features such as batch code coverage testing, improved reporting capabilities, method spies, and better integration with other tools in the Ortus suite.

These new features make TestBox even more versatile and user-friendly, and provide developers with a powerful tool for building high-quality, reliable applications. You can read more about TestBox in our comprehensive documentation online:



It is easy to install using CommandBox: install testbox --save-dev. You can also install our new TestBox CLI: install testbox-cli


It is easy to update, just type update testbox and you are done!

What's New With 5.0.0

TestBox 5.x series is a major bump in our library. Here are the major areas of improvement and the full release notes.

Engine Support

We have dropped Adobe 2016 support and added support for Adobe 2023 and Lucee 6+

Batch Test Coverage Reporting

Due to memory limitations in CI environments, larger codebases cannot run all tests as a single testbox run command. Instead, specs are run in a methodical folder-by-folder sequence, separating the testbox run out over many requests and thus working around the Out-Of-Memory exceptions.

While this works, it prevents accurate code coverage reporting since only a small portion of the tests are executed during any request. The generated code coverage report only shows a tiny fraction of the coverage - say, 2% - and not the whole picture

TestBox 5 introduces a CoverageReporter component which

  1. Runs on every TestBox code coverage execution
  2. Loads any previous coverage data from a JSON file
  3. Combines the previous coverage data with the current execution's coverage data (file by file and line by line)
  4. Persists the COMBINED coverage data to a JSON file.
  5. Returns the COMBINED coverage data for the CoverageBrowser.cfc to build as an HTML report

When setting url.isBatched=true and executing the batched test runner, the code coverage report will grow with each sequential testbox run command.

Method Spies!

MockBox now supports a $spy( method ) method that allows you to spy on methods with all the call log goodness but without removing all the methods. Every other method remains intact, and the actual spied method remains active. We decorate it to track its calls and return data via the $callLog() method.

Example of CUT:

void function doSomething(foo){
  // some code here then... = variables.collaborator.callMe(;

Example Test:

function test_it(){
  local.mocked = createMock( " collaborator" )
    .$spy( "callMe" )
    .$spy( "whatever" );
  variables.CUT.$property( "collaborator", "variables", local.mocked );
  assertEquals( 1, local.mocked.$count( "callMe" ) );
  assertEquals( 1, local.mocked.$count( "whatever" ) );

Performance Improvements

We have focused on this release to lazy load everything as much as possible to allow for much better testing performance. Check it out!

Skip it! Skip it -> Good!

You can now use the skip( message ) method to skip any spec or suite a-la-carte instead of as an argument to the function definitions. This lets you programmatically skip certain specs and suites and pass a nice message.

it( "can do something", () => {
    if( condition ){
        skip( "Condition is true, skipping spec" )
} )

Release Notes


  • TESTBOX-341 toHaveLength param should be numeric
  • TESTBOX-354 Element $DEBUGBUFFER is undefined in THIS
  • TESTBOX-356 Don't assume TagContext has length on simple reporter
  • TESTBOX-357 notToThrow() incorrectly passes when no regex is specified
  • TESTBOX-360 full null support not working on Application env test
  • TESTBOX-361 MockBox Suite: Key [aNull] doesn't exist
  • TESTBOX-362 Cannot create subfolders within testing spec directories.


  • TESTBOX-333 Add to the repo
  • TESTBOX-339 full null support automated testing
  • TESTBOX-353 allows globbing path patterns in test bundles argument
  • TESTBOX-355 Add debugBuffer to JSONReporter
  • TESTBOX-366 ANTJunit Reporter better visualization of the failed origin and details
  • TESTBOX-368 Support list of Directories for HTMLRunner to allow a more modular tests structure
  • TESTBOX-370 `toHaveKey` works on queries in Lucee but not ColdFusion


  • TESTBOX-371 Add CoverageReporter for batching code coverage reports
  • TESTBOX-137 Ability to spy on existing methods: $spy()
  • TESTBOX-342 Add development dependencies to box.json
  • TESTBOX-344 Performance optimizations for BaseSpec creations by lazy loading external objects
  • TESTBOX-345 add a skip([message]) like fail() for skipping from inside a spec
  • TESTBOX-365 New build process using CommandBox
  • TESTBOX-372 Adobe 2023 and Lucee 6 Support

Add Your Comment


May 16, 2023 14:20:18 UTC

by Wil Belden

Good looking updates! Esp. the spy(method).

Recent Entries

CBWIRE 3.0.0 Released

CBWIRE 3.0.0 Released

We are very excited to announce the release of version 3 of CBWIRE, our ColdBox module that makes building modern, reactive apps a breeze. This version brings with it a new component syntax, 19 enhancements and bug fixes, and improved documentation. Our biggest goal with this release was to improve the developer experience and to provide a low barrier to entry to getting started with CBWIRE.

Grant Copley
Grant Copley
May 22, 2023
ColdBox 7.0.0 Released

ColdBox 7.0.0 Released

Introducing ColdBox 7: Revolutionizing Web Development with Cutting-Edge Features and Unparalleled Performance

We are thrilled to announce the highly anticipated release of ColdBox 7, the latest version of the acclaimed web development HMVC framework for ColdFusion (CFML). ColdBox 7 introduces groundbreaking features and advancements, elevating the development experience to new heights and empowering developers to create exceptional web applications and APIs.

Designed to meet the evolving needs of modern web development, ColdBox 7 boasts a range of powerful features that streamline the development process and enhance productivity. With its robust HMVC architecture and developer-friendly tools, ColdBox 7 enables developers to deliver high-performance, scalable, and maintainable web applications and APIs with ease.

Esme Acevedo
Esme Acevedo
May 16, 2023
TestBox CLI 1.x Released

TestBox CLI 1.x Released


We're excited to unveil our latest **TestBox CLI ** tool! This robust command-line interface is specifically crafted to assist developers in streamlining their workflows and enhancing their TestBox BDD/TDD development process. Boasting an intuitive syntax and potent functionalities, the TestBox CLI tool empowers developers to create, test, and generate reports on their ColdFusion (CFML) application

Luis Majano
Luis Majano
May 05, 2023