*Part 5 of 5 in our series on TestBox 7.1's 47 new assertions. Read the release announcement.
The first four posts covered the headline features. This finale covers everything else in the release: eight new matchers, five new $assert methods, a class-level @skip, and a handful of fixes and behavior changes you should know about before you upgrade.
Everything in the first two sections works on BoxLang and CFML. According to the official 7.1.0 release notes, these matchers were inspired by JUnit 5 and Jasmine, so if you have used either, some will feel familiar.
The eight new matchers
| Matcher | Asserts |
|---|---|
toBeTruthy() / toBeFalsy() | the value is truthy or falsy |
toHaveSize( n ) | the size of a string, array, struct or query |
toBeSameInstanceAs( other ) | two references point to the exact same object |
toThrowMatching( predicate ) | a function throws an exception that passes your closure |
toIncludeAll( needles ) | a string or array contains every needle |
toIncludeAny( needles ) | a string or array contains at least one needle |
toIncludeNone( needles ) | a string or array contains no needle |
(toBeTruthy and toBeFalsy count as two, which brings the total to eight.) As with every TestBox matcher, each one has a negated form such as notToBeSameInstanceAs().
toBeTruthy and toBeFalsy: "something" versus "nothing"
Sometimes you do not care whether a value is literally true. You care whether there is something there. A truthy value is one that is not false, 0, an empty string or null. A falsy value is one of those.
expect( 42 ).toBeTruthy()
expect( "hello" ).toBeTruthy()
expect( "" ).toBeFalsy()
expect( 0 ).toBeFalsy()
Where this helps: checking that an id was generated, that an error message was set, or that an optional field was filled in, without pinning the exact value.
toBeTrue() still exists and is the right choice when you specifically want a boolean result. Reach for toBeTruthy() when the question is "is there a value?".
toHaveSize: one name for "how big is it"
toHaveSize() is an alias for toHaveLength(). It works on strings, arrays, structs and queries, so you no longer need to remember which collection type takes which matcher:
expect( "abc" ).toHaveSize( 3 )
expect( [ 1, 2 ] ).toHaveSize( 2 )
expect( { a : 1, b : 2 } ).toHaveSize( 2 )
toBeSameInstanceAs: identity, not equality
This one trips people up, so it is worth slowing down. Two objects can be equal (same contents) without being the same instance (the same thing in memory). toBe() checks whether values are equal. toBeSameInstanceAs() checks whether both references point at one object.
var original = { name : "test" }
var copy = duplicate( original )
expect( original ).toBeSameInstanceAs( original ) // same object
expect( copy ).notToBeSameInstanceAs( original ) // equal contents, different object
When does this matter? Whenever the thing you are testing is reuse: a singleton, a cached value, a connection pool, an object handed out by a dependency injection container. "Did I get the cached one, or did it build a new one?" is an identity question, and toBeSameInstanceAs() is how you ask it.
toThrowMatching: inspect the exception, not just its type
toThrow() has always let you check an exception's type and a message pattern. toThrowMatching() hands the thrown exception to a closure, so you can check anything about it:
expect( () => {
throw( type = "FooException" )
} ).toThrowMatching( e => e.type == "FooException" )
The closure receives the exception and returns true or false. Use it when the rule is more than "some exception of this type": for example, the type and a fragment of the detail, or a custom property. If nothing is thrown, or the closure returns false, the expectation fails.
toIncludeAll, toIncludeAny and toIncludeNone
These check a string or an array against a list of needles, with case-insensitive matching:
expect( "hello world" ).toIncludeAll( [ "hello", "world" ] ) // both present
expect( "hello world" ).toIncludeAny( [ "hello", "foo" ] ) // at least one present
expect( "hello world" ).toIncludeNone( [ "foo", "bar" ] ) // none present
toIncludeNone() is the one to remember. Asserting that sensitive words never appear in some output ("no password, no token, no card number") reads far better than a pile of negated toInclude() calls.
The five new assertion BIFs
If you write xUnit style tests with $assert, these bring it in line with the BDD matchers. The last parameter of each is an optional failure message.
| Method | Asserts |
|---|---|
$assert.isTruthy( actual, message ) | value is truthy |
$assert.isFalsy( actual, message ) | value is falsy |
$assert.includesAll( target, needles, message ) | target contains every needle |
$assert.includesAny( target, needles, message ) | target contains at least one needle |
$assert.includesNone( target, needles, message ) | target contains no needle |
$assert.isTruthy( "hello" )
$assert.isFalsy( 0 )
$assert.includesAll( "hello world", [ "hello", "world" ] )
$assert.includesAny( [ "a", "b" ], [ "b", "z" ] )
$assert.includesNone( "hello", [ "x", "y" ] )
$assert.all(), the grouped assertion method, was covered in Part 4.
Skip an entire class
TestBox has long let you skip individual specs and suites. Version 7.1 adds a class-level skip annotation, so you can park a whole test class without editing every describe() or fiddling with runner filters. Add a reason after it if you like:
/**
* @skip Waiting on the sandbox credentials
*/
class extends="testbox.system.BaseSpec" {
function run(){
describe( "Payment gateway", () => {
// none of this runs while @skip is present
} );
}
}
Skipped classes are reported as skipped, not silently dropped, so your totals stay honest and a parked class is visible in the results until someone un-parks it.
Hands-on: a payment service
Let's use all of it against one small service. Save the files below in the same folder of a TestBox harness (see the setup steps in Part 3), for example tests/specs.
The system under test
/**
* PaymentService.bx
*
* A small service that hands out a cached gateway object, charges amounts,
* and throws descriptive exceptions when a charge is invalid.
*/
class {
variables.gateway = "";
variables.lastError = "";
variables.counter = 1000;
/**
* The gateway is built once and then reused (cached).
*/
function getGateway(){
if ( isSimpleValue( variables.gateway ) ) {
variables.gateway = { "name" : "SandboxGateway", "connectedAt" : now() };
}
return variables.gateway;
}
function getSupportedCurrencies(){
return [ "USD", "EUR", "GBP" ];
}
/**
* The last error message, or an empty string if the last call succeeded.
*/
function getLastError(){
return variables.lastError;
}
/**
* Charges an amount and returns a receipt.
* Throws InvalidAmount or UnsupportedCurrency for bad input.
*/
function charge( required numeric amount, string currency = "USD" ){
variables.lastError = "";
if ( arguments.amount <= 0 ) {
variables.lastError = "Amount must be greater than zero";
throw(
type = "InvalidAmount",
message = "Invalid charge amount",
detail = "Amount [#arguments.amount#] is negative or zero"
);
}
if ( !getSupportedCurrencies().contains( arguments.currency ) ) {
variables.lastError = "Unsupported currency";
throw(
type = "UnsupportedCurrency",
message = "Currency not supported",
detail = "Currency [#arguments.currency#] is not supported"
);
}
variables.counter++;
return {
"id" : "ch_#variables.counter#",
"amount" : arguments.amount,
"currency" : arguments.currency,
"status" : "approved",
"description" : "Payment of #arguments.amount# #arguments.currency# approved"
};
}
}
The spec
/**
* PaymentServiceSpec.bx
*
* The new matchers and assertion BIFs in action.
*/
class extends="testbox.system.BaseSpec" {
function run(){
describe( "PaymentService", () => {
beforeEach( () => {
variables.service = new PaymentService();
} );
describe( "toBeTruthy / toBeFalsy", () => {
it( "a successful charge produces an id and no error", () => {
var receipt = service.charge( 42.50, "USD" );
expect( receipt.id ).toBeTruthy();
expect( service.getLastError() ).toBeFalsy();
} );
it( "a failed charge leaves an error message behind", () => {
try {
service.charge( -5 );
} catch ( any e ) {
// we only care that a failure was recorded
}
expect( service.getLastError() ).toBeTruthy();
} );
} );
describe( "toHaveSize", () => {
it( "works on structs and arrays with one matcher", () => {
var receipt = service.charge( 10, "EUR" );
expect( receipt ).toHaveSize( 5 );
expect( service.getSupportedCurrencies() ).toHaveSize( 3 );
expect( receipt.id ).toHaveSize( 7 ); // "ch_1001"
} );
} );
describe( "toBeSameInstanceAs", () => {
it( "returns the cached gateway on every call", () => {
var first = service.getGateway();
var second = service.getGateway();
expect( first ).toBeSameInstanceAs( second );
} );
it( "a copy is a different instance even though it looks identical", () => {
var gateway = service.getGateway();
var copy = duplicate( gateway );
expect( copy ).notToBeSameInstanceAs( gateway );
} );
} );
describe( "toThrowMatching", () => {
it( "rejects a negative amount with a descriptive detail", () => {
expect( () => service.charge( -5 ) ).toThrowMatching(
e => e.type == "InvalidAmount" && e.detail contains "-5"
);
} );
it( "rejects an unknown currency", () => {
expect( () => service.charge( 10, "BTC" ) ).toThrowMatching(
e => e.type == "UnsupportedCurrency" && e.detail contains "BTC"
);
} );
} );
describe( "toIncludeAll / toIncludeAny / toIncludeNone", () => {
it( "checks a string for several words at once (case-insensitive)", () => {
var receipt = service.charge( 25, "GBP" );
expect( receipt.description ).toIncludeAll( [ "payment", "APPROVED", "gbp" ] );
expect( receipt.description ).toIncludeAny( [ "declined", "approved" ] );
expect( receipt.description ).toIncludeNone( [ "declined", "error" ] );
} );
it( "checks an array of currencies", () => {
var currencies = service.getSupportedCurrencies();
expect( currencies ).toIncludeAll( [ "USD", "EUR" ] );
expect( currencies ).toIncludeAny( [ "USD", "JPY" ] );
expect( currencies ).toIncludeNone( [ "BTC", "DOGE" ] );
} );
it( "a serialized receipt never contains card data", () => {
var json = serializeJSON( service.charge( 10, "USD" ) );
expect( json ).toIncludeNone( [ "cardNumber", "cvv", "password" ] );
} );
} );
describe( "the xUnit style $assert methods", () => {
it( "mirror the matchers", () => {
var receipt = service.charge( 15, "USD" );
$assert.isTruthy( receipt.id, "a receipt id should be generated" );
$assert.isFalsy( service.getLastError(), "no error after a good charge" );
$assert.includesAll( receipt.description, [ "payment", "approved" ] );
$assert.includesAny( service.getSupportedCurrencies(), [ "USD", "JPY" ] );
$assert.includesNone( receipt.description, [ "declined", "error" ] );
} );
} );
} );
}
}
And a second, tiny spec that shows the class-level skip. It is discovered by the runner, but none of its specs execute:
/**
* LegacyGatewaySpec.bx
*
* @skip Waiting on the sandbox credentials
*/
class extends="testbox.system.BaseSpec" {
function run(){
describe( "Legacy gateway", () => {
it( "would call the real sandbox", () => {
fail( "this should never run while @skip is present" );
} );
} );
}
}
Running it
testbox run bundles=tests.specs.PaymentServiceSpec,tests.specs.LegacyGatewaySpec
You should see the payment specs pass and LegacyGatewaySpec reported as skipped. Delete the @skip line and the class runs, and its fail() proves it.
Fixes and behavior changes to know about
These are not new assertions, but they affect specs you already have. Read this list before you upgrade.
Code coverage is now opt-in (action may be required)
The CFML test runner's coverageEnabled parameter now defaults to false instead of true. Coverage requires FusionReactor, so having it on by default meant every plain runner hit paid for something most runs did not want. If you rely on coverage, turn it on explicitly:
/tests/runner.cfm?coverageEnabled=true
This is the one behavior change in the release. Check your CI configuration.
isLucee() no longer returns true on BoxLang
BoxLang registers a lucee key in the server scope for compatibility, which made isLucee() return true under BoxLang. Any spec that branched or skipped based on engine took the Lucee path on BoxLang. isLucee() now also requires that the boxlang key is absent. If you had a workaround for this, you can remove it.
MockBox $args() matching is deterministic again
$args() used to build its argument hash in a way that depended on the iteration order of structs. Two structurally identical structs built in a different order could produce different hashes, so a mock would fail to match arguments it should have matched and quietly return null. It was always latent, but Lucee 7.1's new map implementation made it easy to reproduce. $args() now normalizes nested structures deterministically:
var mock = createMock( "PaymentService" )
.$( "charge" )
.$args( { amount : 100, currency : "USD" } )
.$results( true );
// matches, even though the keys were supplied in a different order
mock.charge( { currency : "USD", amount : 100 } );
$args() also understands BoxLang Set and Range objects now.
Dates are compared by instant
Equality assertions now compare date and date/time objects by their instant, instead of calling actual.equals() blindly. Comparing a java.util.Date to a CFML date string, or two date objects of different concrete types, now behaves the way you expect instead of failing on type identity.
Smaller fixes
- Simple reporter: bundle and spec names are HTML-encoded, so a test name containing markup no longer breaks the report layout.
- Full null support: null handling was audited across the runners, coverage service and mock generator, and that configuration is now covered in CI.
- BoxLang CLI runner: three fixes. The runner no longer misreads its own script path as a bundle argument,
KeyNotFoundException [url]no longer crashes every CLI run on BoxLang 1.17 and later, andGetPageContextResponse()works when running BoxLang in Adobe compatibility mode.
Upgrade checklist
| Check | Why |
|---|---|
| Does CI rely on coverage output? | coverageEnabled now defaults to false |
Any engine-detection workarounds for isLucee() on BoxLang? | The bug is fixed, the workaround may now be wrong |
Any flaky mocks that use $args() with structs? | They should now match reliably |
| Any date comparisons that failed on type? | They now compare by instant |
box install testbox@7.1.0
7.1.0 is a minor release and is backward compatible, apart from the coverageEnabled default.
Add Your Comment