Batch

Run AWS batch jobs without installing software or servers

You can perform AWS batch job processing directly in the DuploCloud Portal without the additional overhead of installed software, allowing you to focus on analyzing results and diagnosing problems.

Step 1: Create batch job scheduling policies (optional)

Create scheduling policies to define when your batch job runs.

1. From the DuploCloud Portal, navigate to Cloud Services -> Batch page, and click the Scheduling Policies tab.

2. Click Add. The Create Batch Scheduling Policy page displays.

3. Create batch job scheduling policies using the AWS documentation. The fields in the AWS documentation map to the fields on the DuploCloud Create Batch Scheduling Policy page.

   

   The Create Batch Scheduling Policy page in the DuploCloud Portal
4. Click Create.

Step 2: Configure compute environments

AWS compute environments (Elastic Compute Cloud [EC2] instances) map to DuploCloud Infrastructures. The settings and constraints in the computing environment define how to configure and automatically launch the instance.

1. In the DuploCloud Portal, navigate to Cloud Services -> Batch.

2. Click the Compute Environments tab.

3. Click Add. The Add Batch Environment page displays.

4. In the Compute Environment Name field, enter a unique name for your environment.

5. From the Type list box, select the environment type (On-Demand, Spot, Fargate, etc.).

6. Modify additional defaults on the page or add configuration parameters in the Other Configurations field, as needed.

   

   The Add Batch Environment page in the DuploCloud Portal
7. Click Create. The compute environment is created.

The Compute Environments window in the DuploCloud Portal showing the configured environment.

Step 3: Create batch job queues

After you define job definitions, create queues for your batch jobs to run in. For more information, see the AWS instructions for creating a job queue.

1. From the DuploCloud Portal, navigate to Cloud Services -> Batch page, and click the Queues tab.

2. Click Add. The Create Batch Queue page displays.

   

   The AWS Batch Create Batch Queue page in the DuploCloud Portal.
3. Create batch job queues using the AWS documentation. The fields in the AWS documentation map to the fields on the DuploCloud Create Batch Queue page.
4. Click Create. The batch queue is created.

In the Priority field, enter a whole number. Job queues with a higher priority number are run before those with a lower priority number in the same compute environment.

Step 4: Create batch job definitions

Before you can run AWS batch jobs, you need to create job definitions specifying how batch jobs are run.

1. From the DuploCloud Portal, navigate to Cloud Services -> Batch, and click the Job Definitions tab.

2. Click Add. The Create Batch Job Definition page displays.

   

   The Create Batch Job Definition page in the DuploCloud Portal.
3. Define your batch jobs using the AWS documentation. The fields in the AWS documentation map to the fields on the DuploCloud Create Batch Job Definition page.
4. Click Create. The batch job definition is created.

   

   The Batch Job Definition configuration is shown in the DuploCloud Portal.

Step 5: Create a batch job

Add a job for AWS batch processing. See the AWS documentation for more information about batch jobs.

1. After you configure your compute environment, navigate to Cloud Services -> Batch and click the Jobs tab.
2. Click Add. The Add Batch Job page displays.

   

   The Add Batch Job page in the DuploCloud Portal.
3. On the Add Batch Job page, fill the Job Name, Job Definition, Job Queue, and Job Properties fields.

4. Optionally, if you created a scheduling policy to apply to this job, paste the YAML code below into the Other Properties field.
5. Click Create. The batch job is created.

Create a batch job with a scheduling policy (optional)

As you create a batch job, paste the following YAML code into the Other Properties field on the Add Batch Job page. Replace the scheduling priority override value ("1" in this example) with an integer representing the job's scheduling priority, and replace SHARE_IDENTIFIER with the job's share identifier. For more information, see the AWS documentation.

  {    

    "schedulingpriorityoverride": 1,

    "ShareIdentifier": "SHARE_IDENTIFIER"

}

View batch jobs

Navigate from the DuploCloud Portal to Cloud Services -> Batch, and click the Jobs tab. The jobs list displays.

The Job list shows the Batch Job in the DuploCloud Portal.

Click the name of the job to view job details such as job status, ID, queue, and definition.

The Jobs window shows the running batch job in the DuploCloud Portal.

Run AWS batch jobs

Use the AWS Best Practices Guide for information about running your AWS batch jobs.