Run Multiple Bash Commands in Parallel and Capture Exit Codes

Run multiple Bash commands concurrently by starting them with &, saving each PID with $!, and collecting their exit codes with wait.

Quick answer: run commands in parallel and read exit codes

Append & to each command, then wait for every background job to finish:

sleep 2 &
sleep 1 &
sleep 3 &
wait

All three commands overlap instead of running one after another.

If you also need each command's exit status, save $! after starting each process:

sleep 2 &
p1=$!
sleep 1 &
p2=$!
wait "$p1"; rc1=$?
wait "$p2"; rc2=$?
echo "cmd1=$rc1 cmd2=$rc2"

Sample output:

cmd1=0 cmd2=0

Use & to start commands concurrently, $! to save each PID, and wait "$pid" to retrieve each command's exit status.

For line-based input without a custom Bash loop, -P caps concurrency directly:

printf '%s\n' host1 host2 host3 | xargs -n1 -P3 ping -c1

Set a lower concurrency limit for disk- or network-heavy jobs. CPU-bound work often tracks core count; I/O-bound work usually needs a smaller cap.

Run Multiple Commands with & and wait

The pattern above overlaps independent work. Three sequential sleeps take about six seconds on the wall clock:

TIMEFORMAT='real %R sec'
time { sleep 1; sleep 2; sleep 3; }

Sample output:

real 6.042 sec

The same three commands in the background finish in roughly the time of the longest sleep:

time { sleep 1 & sleep 2 & sleep 3 & wait; }

Sample output:

real 3.029 sec

The three background jobs finish in about three seconds because their sleep periods overlap.

To measure script execution time on your own batches, wrap the loop with time the same way.

wait with no arguments waits for all running background jobs and returns zero when they have all exited, even when a child failed. It does not report which command broke or preserve individual exit codes. Save each PID with $! and call wait "$pid" on each child when per-task status matters. For more on &, $!, and plain wait, see Bash background processes.

Capture Exit Codes from Parallel Commands

After each wait "$pid", store $? before the next command runs:

false & f=$!
true & t=$!
wait "$f"; e1=$?
wait "$t"; e2=$?
echo "fail=$e1 ok=$e2"

Sample output:

fail=1 ok=0

For many tasks, keep task_status[name]=$rc in an associative array. Walk the original task list in stable order when you print a summary:

overall=0
for task in "${tasks[@]}"; do
  rc=${task_status[$task]:-999}
  printf '%-12s %s\n' "$task" "$rc"
  (( rc != 0 )) && overall=1
done
exit "$overall"

Exit code 999 in the summary means a task was never reaped. Under Bash strict mode (set -e), collect expected non-zero statuses in an if wait "$pid"; then … else rc=$?; fi branch rather than using a bare wait "$pid".

See Bash exit codes for how statuses above 128 encode signals.

Identify Which Parallel Command Failed

A PID alone does not tell you which command or task failed. Map each PID to a task name when you launch the child:

declare -A pid_to_task=()
bash -c 'sleep 1; exit 3' & pid_to_task[$!]=alpha
bash -c 'sleep 1; exit 0' & pid_to_task[$!]=beta
for pid in "${!pid_to_task[@]}"; do
  wait "$pid"
  rc=$?
  echo "task=${pid_to_task[$pid]} pid=$pid exit=$rc"
done

Sample output:

task=alpha pid=39102 exit=3
task=beta pid=39103 exit=0

Print order is not guaranteed because associative-array iteration order is unspecified. Use wait -n -p when you need to know which child finished first.

Limit the Number of Parallel Jobs

When you have dozens of tasks, starting them all at once can overload CPU, disk, or network. On Bash 4.3+, a simple cap launches work until max_jobs slots are full, then uses wait -n to wait for one child to finish:

max_jobs=4
running=0
tasks=(task1 task2 task3 task4 task5 task6)

for task in "${tasks[@]}"; do
  echo "start $task" &
  ((running++))
  if (( running >= max_jobs )); then
    wait -n || true
    ((running--))
  fi
done
wait

This simple throttle deliberately discards child exit codes. Use the PID-mapped worker pool below when failures matter.

wait -n requires Bash 4.3 or later; wait -n -p requires Bash 5.1 or later. On older Bash, cap concurrency by waiting on the oldest PID in a FIFO queue.

Avoid polling jobs -p | wc -l as your primary throttle. Job counts and unwaited children can disagree during rapid start and exit bursts.

Build a Worker Pool with wait -n -p

When you need both bounded concurrency and per-task exit codes, keep a pid_to_task map and use wait -n -p as jobs finish. In these non-job-control script examples, -p stores the completed child's PID in finished_pid. The return status of wait -n is the child's exit code; you do not need a second wait "$pid".

reap_one() {
  local finished_pid rc finished_task
  finished_pid=
  if wait -n -p finished_pid; then
    rc=0
  else
    rc=$?
  fi
  [[ -z ${finished_pid:-} ]] && return 1
  finished_task=${pid_to_task[$finished_pid]}
  task_status[$finished_task]=$rc
  unset 'pid_to_task[$finished_pid]'
}

Before starting another background job, call reap_one while ${#pid_to_task[@]} is at or above MAX_JOBS. When the task queue is empty but slots remain busy, keep reaping until the map is clear.

A two-job pool records status as each slot frees:

declare -A pid_to_task=()
declare -A task_status=()
( sleep 0.5; exit 1 ) & pid_to_task[$!]=fast-fail
( sleep 2; exit 0 ) & pid_to_task[$!]=slow-ok
while ((${#pid_to_task[@]} > 0)); do
  finished_pid=
  if wait -n -p finished_pid; then
    rc=0
  else
    rc=$?
  fi
  [[ -z ${finished_pid:-} ]] && break
  task_status[${pid_to_task[$finished_pid]}]=$rc
  unset 'pid_to_task[$finished_pid]'
done
printf 'fast-fail=%s slow-ok=%s\n' "${task_status[fast-fail]}" "${task_status[slow-ok]}"

Sample output:

fast-fail=1 slow-ok=0

The shorter job finishes first even though it started second.

Stop Launching Jobs After a Failure

Decide what should happen when one parallel job fails:

• Finish all: let all tasks finish, then return failure if any task failed.
• Fail fast: stop launching new work after the first failure while workers may still be running; optionally kill running workers from your cleanup trap.

Finish all launches every job before waiting:

declare -A st=()
for label in ok fail ok; do
  bash -c '[[ $1 == fail ]] && exit 2 || exit 0' _ "$label" &
  st[$!]=$label
done
overall=0
for pid in "${!st[@]}"; do
  wait "$pid"
  rc=$?
  echo "${st[$pid]} exit=$rc"
  (( rc != 0 )) && overall=1
done
echo "overall=$overall"

Sample output:

ok exit=0
fail exit=2
ok exit=0
overall=1

Every task runs to completion even after fail exits 2. A fail-fast implementation still needs bounded parallel execution; waiting immediately after each launch makes the loop sequential. Extend the worker pool so a non-zero wait -n -p result sets a flag and stops launching new tasks:

if wait -n -p finished_pid; then
  rc=0
else
  rc=$?
  stop_launching=1
fi

Record the status, remove the PID from pid_to_task, and skip starting new work while stop_launching is set. Workers already in the pool can finish or receive kill from your cleanup trap.

GNU Parallel handles this without a custom loop. --halt soon,fail=1 stops launching new jobs and waits for running ones to finish. --halt now,fail=1 terminates running jobs when one fails.

Keep Parallel Output Readable

Background jobs share stdout and stderr unless you redirect them. Lines interleave unpredictably:

for i in 1 2 3; do
  ( echo "task$i start"; sleep 1; echo "task$i end" ) &
done
wait

Sample output:

task1 start
task2 start
task3 start
task2 end
task1 end
task3 end

Redirect each child to its own log file to keep output coherent:

for i in 1 2 3; do
  ( echo "task$i start"; sleep 1; echo "task$i end" ) >"task-$i.log" 2>&1 &
done
wait

For live output, prefix lines with the task name. For logs, writing one file per task keeps the output separate. See Bash redirection for append vs truncate and merging stderr into the same log.

Clean Up Parallel Jobs on Ctrl+C

Background jobs can continue after the parent script exits unless you clean them up. Add an EXIT trap that kills and waits for tracked workers, and handle INT and TERM by exiting with the corresponding status (same pattern as Bash trap). In a worker pool, iterate the live pid_to_task keys directly:

cleanup() {
  local pid
  for pid in "${!pid_to_task[@]}"; do
    kill "$pid" 2>/dev/null || true
  done
  for pid in "${!pid_to_task[@]}"; do
    wait "$pid" 2>/dev/null || true
  done
}
trap cleanup EXIT
trap 'exit 130' INT
trap 'exit 143' TERM

Ctrl+C runs the INT trap, which calls exit 130. Bash then runs the EXIT trap once. The complete example at the end wires the same handlers into its worker pool.

Run Many Inputs with xargs -P

When each input line drives the same command template, GNU xargs -P n can run the same command over multiple inputs with a fixed concurrency limit. Pass each input as an argument to bash -c instead of interpolating it into the command string:

printf '%s\n' host1 host2 host3 host4 host5 | xargs -P 2 -I{} bash -c 'sleep 1; printf "%s done\n" "$1"' _ {}

Sample output:

host1 done
host2 done
host3 done
host4 done
host5 done

-P 2 keeps at most two children alive. The _ before {} becomes $0, while {} is passed as $1 instead of being embedded in shell code. This avoids shell re-parsing of spaces and metacharacters in the substituted value. For arbitrary filenames, prefer NUL-delimited input with -0.

GNU xargs returns an aggregate status rather than preserving each child's exit code:

When any child fails in a typical batch:

printf '%s\n' ok fail ok | xargs -P 3 -I{} bash -c '[[ $1 == fail ]] && exit 2 || exit 0' _ {}
echo "xargs exit=$?"

Sample output:

xargs exit=123

Use xargs -P when you need throughput over many inputs and only care whether the batch failed. Use a Bash worker pool when you must map each failure to a named task and preserve individual exit codes in a summary table.

When to Use GNU Parallel

GNU Parallel adds job logs, grouped output, and rich input quoting on top of plain xargs. It is a separate package (parallel on most distros).

Run the same worker script over named tasks with at most four concurrent jobs:

parallel -j4 ./worker.sh ::: task1 task2 task3 task4

Stop launching new jobs after the first failure, but let running jobs finish:

parallel -j4 --halt soon,fail=1 ./worker.sh ::: task1 task2 task3 task4

GNU Parallel is useful for grouped output, job logs, and larger batch workflows. A Bash worker pool avoids the extra dependency and works well when tasks are shell functions in the same script.

Complete Example: Bounded Parallel Jobs with Exit Codes

The script below runs ten tasks with at most four concurrent workers. Each task writes to its own log file, records exit status in task_status, prints a summary table, exits non-zero if any task failed (task 7 exits 17 on purpose), and cleans up on interrupt:

#!/usr/bin/env bash
MAX_JOBS=4
declare -A pid_to_task=()
declare -A task_status=()
tasks=()
for i in $(seq 1 10); do
  tasks+=("task-$i")
done

cleanup() {
  local pid
  for pid in "${!pid_to_task[@]}"; do
    kill "$pid" 2>/dev/null || true
  done
  for pid in "${!pid_to_task[@]}"; do
    wait "$pid" 2>/dev/null || true
  done
}
trap cleanup EXIT
trap 'exit 130' INT
trap 'exit 143' TERM

run_task() {
  local id=$1
  sleep $((RANDOM % 2 + 1))
  if (( id % 7 == 0 )); then
    return 17
  fi
  return 0
}

reap_one() {
  local finished_pid rc finished_task
  finished_pid=
  if wait -n -p finished_pid; then
    rc=0
  else
    rc=$?
  fi
  [[ -z ${finished_pid:-} ]] && return 1
  finished_task=${pid_to_task[$finished_pid]}
  task_status[$finished_task]=$rc
  unset 'pid_to_task[$finished_pid]'
}

for task in "${tasks[@]}"; do
  while (( ${#pid_to_task[@]} >= MAX_JOBS )); do
    reap_one || break
  done
  run_task "${task#task-}" >"out-${task}.log" 2>&1 &
  pid=$!
  pid_to_task[$pid]=$task
done

while (( ${#pid_to_task[@]} > 0 )); do
  reap_one || break
done

fail=0
printf '%-10s %s\n' "TASK" "EXIT"
for task in "${tasks[@]}"; do
  rc=${task_status[$task]:-999}
  printf '%-10s %s\n' "$task" "$rc"
  (( rc != 0 )) && fail=1
done
exit "$fail"

Save it as run-pool.sh and run bash run-pool.sh. task-7 exits with status 17, while the script itself exits with status 1 because at least one task failed. Check out-task-7.log for that task's output.

Common Parallel Execution Mistakes

Summary

You can run multiple Bash commands in parallel by appending &, then either call bare wait to synchronize or save each $! and use wait "$pid" to collect exit codes. Bare wait does not report which child failed. Map PIDs to task names, cap concurrency with wait -n or wait -n -p, and decide whether to finish every job or stop launching after the first failure.

Redirect output per task when logs must stay readable, and register traps when interrupts should stop leftover workers. Use xargs -P or GNU Parallel for line-driven batches; use a Bash worker pool when you need a per-task status table. The complete example combines bounded concurrency, exit-code tracking, per-task logs, and interrupt cleanup.

References

• GNU Bash manual — wait builtin
• GNU Findutils — xargs invocation
• GNU Parallel tutorial