Async Your Future

Applications are only sometimes lightweight. There can often be large datasets and complex operations that need to run. To help remedy this, you can tap into the JDK using ColdBox's AsyncManager to bring pipelines, futures, and other tools to help complete your operations.

Scenario

Let's dig deeper into the common need for processing datasets. You may often need to run through datasets and massage data returned from databases, APIs, file contents, and other sources. A simple solution would be looping over that data one record at a time to prepare it accordingly. When dealing with smaller and simpler datasets, that's a viable solution. But more complex and larger datasets can introduce issues like long waits, errors, and timeouts. This is where the AsyncManager's Pipelines & Futures come in.

In Practice

We first inject the asyncManager using WireBox. We'll also include the LogBox logger, as it may be useful for monitoring progress and Exceptions.

property name="async" inject="asyncManager@coldbox";
property name="log" inject="logbox:logger: {this}";

Next, let's assume we have an array of structures named bulkyArray; where we know processing each record is going to take some heavy lifting. To better handle this, we transform our array of structs into an array of Futures. A Future is a task that can run asynchronously, which means we can run many tasks at once without one waiting on another. As a bonus, Futures can have Exception handling. In this example we're going process, log and return the new results (or error details).

var futureArray = bulkyArray.map( (dataset) => {

    return async
        .newFuture( dataset = dataset ) {
            // process and massage dataset, creating newData
            var newData = { a : dataset.a, b : dataset.b };
            log.info('processing record');
            return {
                error : false,
                details : [],
                data : newData
            }
        }
        .onException( ( e ) => {
            // All exceptions in Futures are wrapped in a `CompletionException`.
            if ( "java.util.concurrent.CompletionException" == e.getClass().getName() ) {
                e = e.getCause();
            }
            log.error( "Woops! Error => " & e.getMessage() );
            return {
                error : true,
                details : [ e.getMessage() ]
                data : {}
            };
        } );

});

Note that the processing of Futures has yet to actually be run, instead, we created an array of Future tasks that can then be triggered to asynchronously run.

The AsyncManager provides several different methods to build pipelines and run Futures. Here we choose to utilize the all method that will return a Future, which will return an array of our results when' get' is called.

async
    .all( futureArray )
    .get();

Additionally, handling data can be more than just a one-time passthrough. There may be several steps needed to organize and process the information. For this scenario, we'll choose the thenRun method, where we can further process the data and bulk insert the data into a database, outputting to a filesystem or other needed operations.

async
    .all( futureArray )
    .thenRun( ( data ) => {
        log.info('processing array of structs returned from running the futureArray');
        return data;
    } )
    .get();

Conclusion

The ColdBox AsyncManager is an incredibly useful tool to help asynchronously run tasks. It provides flexible options, built-in exception handling, faster results, and a platform to handle larger tasks in the background without requiring extended timeouts and user wait times.

You can further utilize other ColdBox tools such as QB to handle the bulk inserting of processed data and LogBox to monitor and review the tasks as they are processing.