Skip to main content
Ionworks distributes optimization work across a pool of worker processes that scales automatically with demand. This page explains how resources are allocated across multistart runs and when distributed evaluation is used to accelerate population-based optimizers.

​
Two Levels of Parallelism

Ionworks applies parallelism at two levels during optimization:

​
Batch-level: Multistart Runs

When you configure multiple multistarts, all runs are coordinated from a single driver and share one worker pool. The driver dispatches each start’s evaluations onto the pool and processes results as workers finish — it does not launch a separate task or reserve a dedicated CPU per start. Concurrency across starts is bounded by the pool size: if total demand exceeds the available workers, evaluations queue and run as workers free up.

​
Optimizer-level: Distributed Evaluation

Population-based optimizers evaluate a population of candidate solutions every generation. These evaluations are dispatched across the shared worker pool, which scales to host them. The following optimizers are population-based and support distributed evaluation: Non-population-based optimizers (e.g. Nelder-Mead) evaluate one candidate at a time, so a single such run cannot parallelize internally. Multiple non-population multistarts still run concurrently across the shared pool — one in-flight evaluation per start.

​
Streaming (async) evaluation

The driver streams results: it feeds each candidate result back to its optimizer as soon as a worker finishes, rather than waiting for the entire generation to complete. This reduces idle time when individual evaluations vary in duration — fast evaluations are processed immediately while slower ones continue in the background. It is automatic for all population-based optimizers (Differential Evolution, CMA-ES, PSO, XNES, SNES).
Streaming evaluation does not change optimization results — it only affects throughput. Each optimizer receives the same candidate-result pairs regardless of the order in which workers finish.

​
Resource Allocation

The worker pool is sized to the fit’s demand: num_starts × generation width × per-point tasks, where the generation width is the population size for population-based optimizers (1 for non-population optimizers) and per-point tasks equals the number of objectives when objective-level parallelism is enabled. This demand is capped by the total worker supply available — i.e. min(demand, supply). There is no fixed per-job worker count. The pool scales up and down as jobs start and finish. When you run against your own machine, the currently-free CPU count is used as the supply cap. The pool is always built with at least one worker; when only one worker is available the fit’s evaluations run effectively sequentially through it.
The pool size is determined automatically from the fit’s demand and available supply. No manual worker-count configuration is required.

​
Scenarios

The following examples illustrate how resources are allocated in different configurations.

​
Scenario 1: Multiple Multistarts with a Population-based Optimizer

4 multistarts with CMA-ES All four starts are coordinated from the driver and share one worker pool sized to the combined demand across all starts — num_starts × population × per-point tasks — capped by the available supply. The pool scales to host the work, and every start’s population evaluations are dispatched across it.

​
Scenario 2: Single Multistart with a Population-based Optimizer

1 multistart with PSO The driver coordinates one run. The worker pool is sized to that run’s demand (1 × population × per-point tasks), capped by the available supply. All workers serve the population evaluations for that single run.

​
Scenario 3: Many Multistarts

32 multistarts with CMA-ES The shared worker pool is sized to the aggregate demand across all 32 starts, capped by the total supply. The pool scales to meet demand; until it has fully scaled, evaluations queue and start as workers become available.

​
Scenario 4: Multiple Multistarts on Your Own Machine

4 multistarts with CMA-ES running locally The pool is sized to the fit’s demand, capped by your machine’s currently-free CPUs (the supply). The starts share whatever pool that yields; if fewer CPUs are free than the fit demands, evaluations queue and run as workers free up.

​
Scenario 5: Single Multistart with a Non-population-based Optimizer

1 multistart with Nelder-Mead The driver coordinates one run. Because Nelder-Mead is not population-based, it evaluates one candidate at a time, so this single run runs effectively sequentially on one worker.

​
Summary

​
Next Steps