Batch

Batch state allows you to execute a function, webhook, or circuit in batches as multiple jobs in an iterative manner.

 

While the Parallel state executes multiple states simultaneously using the same input, the Batch state executes the (same set of) State(s) for multiple entries of an array in the state input. This helps to handle a large volume of input or tasks in sets rather than overloading the involved resource.

 

Collection Path

The Collection Path field in the Batch state allows you to specify an array within an input JSON for the state to iterate over. This is particularly useful when you have a JSON input that contains an array of items, and you want to process each item individually within the Batch state. Use JSONPath syntax to specify the location in the input of the JSON array used for iteration.

 

Let's consider the following JSON input:

{
	"employee_list": [
	{
		"name": "John",
		"email": "john@zylker.com",
		"role": "Business Analyst"
	},
	{
		"name": "Helena",
		"email": "helena@zylker.com",
		"role": "Programmer"
	},
	{
		"name": "Mark",
		"email": "mark@zylker.com",
		"role": "QA Analyst"
	}
	]
}

Every index in the above input can be iterated, set to a variable, and sent for processing using the collection path $.employee_list.

 

Collection Variable

The Collection Variable is used to store the JSON values that need to be processed in the associated resource (Circuit, Webhook, or Function states) of the Batch state.

 

For example, if we want to process the employee details of each employee from the 'employee_list' array in the above JSON input, in the Batch state configuration, set the Collection Path as $.employee_list and the Collection Variable as employee. This will extract each employee object within the 'employee_list' array and assign it to the 'employee' variable.

 

If the associated resource is a circuit, the values from 'employee_list' will be processed using the collection variable. The values in the Collection Variable can be used as $.employee.name, $.employee.email, $.employee.role in the circuit.

 

Example Use Case: In the employee onboarding scenario, a circuit named 'New Employee Onboarding' is already designed for the onboarding of a single employee, which has a predefined sequence of steps or actions to be followed during the onboarding process. However, to onboard multiple employees efficiently, we can execute the circuit in batch mode for multiple entries of employee details using the Batch state. Each entry represents an individual employee going through the onboarding process, ensuring that each step of the process is followed consistently for all employees.

 

This enables us to onboard multiple employees simultaneously. By setting the number of jobs in the Batch state, let's say 10, the circuit can be executed concurrently for 10 employees, processing their respective inputs. Once the initial 10 jobs are completed and the first set of employees is processed, the subsequent iteration will initiate the next batch of 10 jobs to onboard the next set of employees. This iterative process continues until all employees are successfully onboarded.

 

Now, let's configure a Batch state in Builder View and Code View for executing multiple employee onboarding circuits as a batch.

Builder View  

To configure a Batch state to your circuit:

1. In Builder View, drag and drop the Batch state from the left pane into your circuit or click the required Batch state in your circuit.

2. Under Configuration, update the common state field attributes.

3. In the Collection Path, enter the key that contains the JSON array with the values to be processed by the Batch state in the input JSON.

4. In the Collection Variable, provide a variable name to hold the JSON value or object extracted from the list specified in the Collection Path. The iterated JSON value or object can be accessed using this variable.

5. Using Bind Type, you can bind the Batch state to a function, webhook, or circuit.
   

In this process, we are binding circuit developed for single employees onboarding and executing as a batch for multiple employees during the employee onboarding process.

6. In the No. of Jobs field, define the number of threads for each batch.
   
7. To configure Input path, Output path and Result Path for Batch state refer to Input & Output Processing. 
   

      

Since multiple instances of a function or a circuit are executed, this ensures that they do not face a timeout error.

‌If you define the Retry option for the batch state, the retry policy applies only if all iterations of the batch state fail. For example, if you configure the Retry field in a batch state with three iterations, the retry policy applies only if all three iterations fail.

Code View

In Code View, JSON details can be provided in fields collectionPath and collectionVariable. You can link a Function, Webhook, or Circuit using bind and define the number of threads for batch using degree.

Batch state example:

"Employee Onboarding Tasks": {
	"type": "batch",
	"next": "End",
	"start": true,
	"collection_path": "$.employee_list",
	"collection_variable": "employee",
	"degree": 10,
	"bind": {
		"type": "circuit",
		"circuit_id": "new_employee_onboarding"
	}
}

Next: Success state

• Related Articles

• Wait

  The Wait state introduces a pause or delay within your circuit, temporarily halting the execution for a specified duration before resuming and progressing to the next state. The delay duration can be configured in seconds. For example, if you set a ...

• Circuit

  Circuit enables you to create automated workflows for IT and business by organizing a sequence of tasks. In certain cases, a process might include complex sub-processes that can be constructed as separate standalone circuits. These circuits can be ...

• Sample Use Cases

  Payment process In a typical payment process, for every purchase made, a payment link with the invoice will be shared with the purchaser. Upon successful payment, a receipt will be generated; in the event of failure, the entire process will have to ...

• Create and Configure a Circuit

  Create a Circuit Log in to Qntrl. Click Circuit from the left pane, then click New Circuit. Specify a unique Name and add a Description for the Circuit. Click Save. A sample circuit with the Pass state and name as 'State1' will be created by default, ...

• Execution Logs

  A record of all tested executions is maintained in Circuit, allowing you to access and re-run specific executions anytime. Follow these steps to access the execution history of a circuit: Access Execution History Go to the left pane and click on the ...