Introduction to using HTCondor to run distributed compute Jobs and Workflows on Servers, Clusters, Grids, or Clouds

by Todd Tannenbaum (University of Wisconsin-Madison) available at https://indico.scc.kit.edu/event/427/contributions/4280/.

Description

HTCondor is an open-source high-throughput computing software framework for coarse-grained distributed parallelization of computationally intensive tasks (jobs). It can be used to manage workloads on a single server, a cluster of computers, public cloud resources, or even national computing grids like the Open Science Grid.

This workshop will introduce the concept of High Throughput Computing and how to submit large batches of jobs and also job workflows (job pipelines) to HTCondor, which will be of interest to end users. We will discuss the architecture of the system, and participants will create a unified compute cluster.

If you wish to participate in the hands-on exercises (not required, but recommended), you will need a laptop with WiFi, an SSH client (such as PuTTY if using Windows), familiarity with the Linux command-line environment (cd, less, cp, rm, mkdir, etc), and the ability to edit a file using a Linux terminal editor like vi, vim, or nano.

Keywords htcondor batch system