> ## Documentation Index > Fetch the complete documentation index at: https://docs.ionworks.com/llms.txt > Use this file to discover all available pages before exploring further. # Data Fitting Overview > Estimate battery model parameters by fitting to experimental data with iws.DataFit. `iws.DataFit` describes a parameter fit: which experiments to compare against, which parameters are free, and how to search. The schema is submitted as one element of a pipeline. For the theory (cost functions, identifiability, multi-start), see the [Data Fitting Guide](/guide/data-fitting/overview). ## A minimal fit ```python theme={null} import pybamm import ionworks_schema as iws from ionworks import Ionworks # Known parameters (everything not fit) known = iws.direct_entries.DirectEntry( parameters={"Ambient temperature [K]": 298.15}, ) # Objective: compare a current-driven SPMe simulation against measured voltage obj_1C = iws.objectives.CurrentDriven( data_input="file:examples/data/chen_synthetic_1C/time_series.csv", options={"model": pybamm.lithium_ion.SPMe()}, ) # Free parameters parameters = { "Negative particle diffusivity [m2.s-1]": iws.Parameter( "Negative particle diffusivity [m2.s-1]", initial_value=2e-14, bounds=(1e-14, 1e-13), ), "Positive particle diffusivity [m2.s-1]": iws.Parameter( "Positive particle diffusivity [m2.s-1]", initial_value=2e-15, bounds=(1e-15, 1e-14), ), } fit = iws.DataFit( objectives={"test_1C": obj_1C}, parameters=parameters, cost=iws.costs.SSE(), optimizer=iws.optimizers.DifferentialEvolution(), ) pipeline = iws.Pipeline({"known": known, "fit": fit}) client = Ionworks() submission = client.pipeline.create(pipeline) client.pipeline.wait_for_completion(submission.id) result = client.pipeline.result(submission.id) print(result.element_results["fit"]) ``` Configuration mistakes inside a `DataFit` (bad parameter names, malformed objectives, …) surface as `UserConfigurationError`. The job classifier maps these to a **Configuration error** in Studio so they're easy to distinguish from solver failures. ## Multiple objectives Pass multiple objectives to fit against several experiments simultaneously (e.g. discharge at different C-rates or temperatures): ```python theme={null} import pybamm fit = iws.DataFit( objectives={ "1C": iws.objectives.CurrentDriven( data_input="file:.../1C.csv", options={"model": pybamm.lithium_ion.SPMe()}, ), "0.5C": iws.objectives.CurrentDriven( data_input="file:.../0.5C.csv", options={"model": pybamm.lithium_ion.SPMe()}, ), }, parameters=parameters, ) ``` Each objective contributes to a single combined cost. ## Optimizers `iws.optimizers` exposes the optimisers available to `DataFit`. Pick the one that fits your problem: | Schema | Best for | | ------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | | `iws.optimizers.ScipyMinimize(method="L-BFGS-B")` | Smooth problems, fast local optimisation | | `iws.optimizers.ScipyLeastSquares()` | Residual-based least-squares; good with priors. Uses an analytic parameter Jacobian when the model supplies one. | | `iws.optimizers.ScipyLsqLinear()` | Bounded **linear** least-squares (single-solve BVLS/TRF). Use as the inner solver of a `Nested` fit when the inner residual is linear in its parameters. | | `iws.optimizers.DifferentialEvolution()` | Global, no gradients required | | `iws.optimizers.CMAES()` | Global, many local minima, well-tested defaults | | `iws.optimizers.PSO()` | Global, parallelisable population search | | `iws.optimizers.BayesianOptimization()` | Expensive evaluations, ≤ \~10 parameters, small budget | | `iws.optimizers.TuRBO()` | Expensive evaluations run in parallel batches; higher-dimensional problems | | `iws.optimizers.SOBER()` | Wide parallel batches using quadrature-style recombination | See [Objective Functions](/pipelines/data-fitting/objective-functions) for the cost-function options. The surrogate optimisers (`BayesianOptimization`, `TuRBO`, `SOBER`) require the optional `surrogate` install extra, which adds `torch`, `botorch`, and `gpytorch`: ```bash theme={null} pip install "ionworkspipeline[surrogate]" ``` They are imported lazily, so installs that only use the population-based or SciPy optimisers do not pay this dependency cost. ### TuRBO for expensive parallel problems When each evaluation is expensive and you have workers to spare, `TuRBO` proposes a batch of candidates per round and adapts a trust region around the current best point. Worker count is now owned by the execution engine (see the note below), so size the warm-up (`n_initial`) to the capacity you give `DataFit.run(execution=ExecutionConfig(...))` — that way the first round fully uses the available workers: ```python theme={null} import ionworks_schema as iws fit = iws.DataFit( objectives=objectives, parameters=parameters, optimizer=iws.optimizers.TuRBO( max_iterations=12, population_size=64, algorithm_options={"noise_floor": "low", "n_initial": 64}, ), ) ``` `DataFit` no longer takes `parallel`, `num_workers`, or `max_batch_size` (and `AskTellOptimizer` no longer takes `async_mode`). Parallelism is owned by the execution engine and is configured at run time — pass an `ExecutionConfig` to `DataFit.run(execution=...)`. Stored configs that still carry the removed fields are migrated automatically at parse time, so no action is required for existing saved fits. Useful `algorithm_options` keys for surrogate optimisers include `n_initial` (warm-up sample count), `noise_floor` (`"low"`, `"standard"`, or a `(lo, hi)` interval), and — for TuRBO — trust-region controls (`tr_length_init`, `tr_length_min`, `tr_length_max`, `tr_success_tolerance`, `tr_failure_tolerance`, `n_candidates`). ### Strict option validation `algorithm_options` is validated against the optimiser you chose. Unknown keys — including typos — are rejected at submission time rather than silently ignored, so misconfigured fits fail fast instead of running with default behaviour: ```python theme={null} # Raises a validation error: "noise_flor" is not a recognised TuRBO option. iws.optimizers.TuRBO(algorithm_options={"noise_flor": "low"}) # AskTellOptimizer is the low-level class behind CMAES(), PSO(), XNES(), etc.; # the named optimisers above are thin wrappers that set `method` for you. # Raises a validation error: BO options passed to a CMAES fit. iws.optimizers.AskTellOptimizer( method="CMAES", algorithm_options=iws.optimizers.BayesianOptimizationOptions(n_initial=32), ) ``` The same check applies whether you pass a raw dict or one of the typed wrappers (`CMAESOptions`, `PSOOptions`, `DEOptions`, `XNESOptions` — for the `XNES` optimizer, available via `iws.optimizers.XNES()` / `AskTellOptimizer(method="XNES")` — `BayesianOptimizationOptions`, `SOBEROptions`, `TuRBOOptions`). Prefer the typed wrappers for editor autocomplete and inline documentation of each option. The only exception is `CMAESOptions`, which remains a passthrough to pycma's own option surface. #### No SciPy-style kwargs on native optimizers Native ask/tell optimizers (`CMAES`, `DifferentialEvolution`, `PSO`, `XNES`, `BayesianOptimization`, `TuRBO`, `SOBER`, and the underlying `AskTellOptimizer`) also reject unknown top-level keyword arguments at construction. SciPy-style keys such as `maxiter`, `popsize`, `seed`, and `tol` are **not** accepted — they previously had no effect and now fail validation immediately, so misconfigured fits surface at construction rather than silently: ```python theme={null} # Raises a validation error — "Extra inputs are not permitted" for each unknown # key (maxiter, popsize, tol). iws.optimizers.DifferentialEvolution(maxiter=10, popsize=5, tol=1e-6) # Correct: use the documented ask/tell parameters (tol -> population_convergence_tol). iws.optimizers.DifferentialEvolution( max_iterations=10, population_size=5, population_convergence_tol=1e-6, ) # Algorithm-specific settings go in `algorithm_options`. iws.optimizers.CMAES(algorithm_options={"seed": 42}) ``` SciPy-style keys still belong on the SciPy passthrough optimizers (`ScipyMinimize`, `ScipyLeastSquares`, `ScipyDifferentialEvolution`), which forward them directly to the underlying SciPy call: ```python theme={null} # OK: ScipyDifferentialEvolution forwards maxiter/popsize/seed/tol to scipy.optimize. iws.optimizers.ScipyDifferentialEvolution(maxiter=10, popsize=5, seed=0) ``` In short: put iteration, population, and tolerance limits in the named ask/tell arguments (`max_iterations`, `population_size`, `population_convergence_tol`); put algorithm internals in `algorithm_options`; and keep SciPy keywords on the `Scipy*` optimizers. ### Strict objective validation `DataFit.objectives` (and `Validation.objectives`) validates each entry against a discriminated union keyed on the objective's `type`. Configuration mistakes are rejected at submission time with a clear `ValidationError`, instead of being silently ignored or surfacing later as an opaque runtime crash. Objective **instances** (e.g. `iws.objectives.CurrentDriven(...)`) keep working unchanged — both the positional and keyword constructors are preserved. The strict-validation rules apply when you pass raw config **dicts**, which is common for configs loaded from JSON or produced by `to_config()`: ```python theme={null} # OK — concrete type, all keys recognised. fit = iws.DataFit( objectives={ "1C": { "type": "CurrentDriven", "data": "file:.../1C.csv", "options": {"model": {"type": "SPMe"}}, }, }, parameters=parameters, ) # Also OK — the legacy `objective` alias and top-level `model` shorthand # are normalised into `type` and `options.model` before validation. fit = iws.DataFit( objectives={ "1C": { "objective": "CurrentDriven", "model": {"type": "SPMe"}, "data": "file:.../1C.csv", }, }, parameters=parameters, ) ``` The following are now rejected up-front: | Mistake | Example | Why it fails | | ---------------------------- | ------------------------------------------------------------ | -------------------------------------------------------------------- | | Unknown objective type | `{"type": "NotAnObjective"}` | Type isn't a member of the union. | | Missing discriminator | `{"electrode": "positive", "data": "x.csv"}` | No `type` (or legacy `objective`) to dispatch on. | | Unknown inner key | `{"type": "OCPHalfCell", "definitely_not_a_field": 1}` | Schemas use `extra="forbid"`; typos are caught early. | | Non-concrete type | A generic/base name instead of a specific objective | Only concrete objective types (e.g. `OCPHalfCell`) validate. | | Conflicting discriminators | `{"type": "OCPHalfCell", "objective": "CurrentDriven", ...}` | Raises `Conflicting objective discriminators` — fix one of the keys. | | Non-dict, non-instance value | `{"bad": "RMSE"}` / `{"bad": 123}` | Objective values must be a config dict or an objective instance. | In a raw config dict the data key is `data`, not `data_input`. The Python constructor exposes it as the `data_input` keyword argument, but `to_config()` serialises it to `data` — so hand-written dicts and JSON configs must use `data`. Passing `data_input` in a dict is now rejected as an unknown key. `DesignObjective` configs are not accepted on the `DataFit` runtime path: they run on the separate design-optimization pipeline. Passing a `DesignObjective` dict to `DataFit.from_schema` raises a `UserConfigurationError` pointing you at the design-optimization pipeline instead of crashing later. ## Multi-start For problems with multiple local minima, run several optimisations from different starting points: ```python theme={null} fit = iws.DataFit( objectives=objectives, parameters=parameters, multistarts=20, ) ``` The pipeline generates initial guesses (Latin Hypercube by default), runs them in parallel, and returns every result sorted by cost. ### Choosing the initial-guess sampler By default multi-start uses Latin Hypercube sampling, which spreads initial guesses evenly across the parameter bounds. Override the sampler with `initial_guess_sampler` when you want a different sampling scheme — for example, plain uniform sampling for a baseline comparison: ```python theme={null} fit = iws.DataFit( objectives=objectives, parameters=parameters, multistarts=20, initial_guess_sampler=iws.distribution_samplers.Uniform(), ) ``` Available samplers: | Sampler | Use for | | -------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | | `iws.distribution_samplers.LatinHypercube()` | Default. Stratified samples that cover the parameter space more evenly than independent draws — recommended for most fits. | | `iws.distribution_samplers.Uniform()` | Independent uniform draws across the bounds. Useful as a baseline or when you want IID samples. | The sampler is validated against a discriminated union at submission time: passing an unknown sampler `type` or an unrecognised key raises a `ValidationError` immediately instead of failing later in the run. ## Nested (variable-projection) fits `iws.optimizers.Nested` is a bi-level optimizer: an **outer** optimizer searches over a chosen subset of parameters, and for every outer trial point an **inner** optimizer is run to optimality over the remaining parameters. This is the variable-projection formulation — the outer sees a concentrated objective $g(x_\text{outer}) = \min_{x_\text{inner}} f(x_\text{outer}, x_\text{inner})$ and the inner absorbs the parameters that are cheap to fit conditionally. Use `Nested` when: * One subset of parameters enters the model **linearly** (or near-linearly) and can be fit in closed form with a bounded least-squares inner solve, while the rest are nonlinear. * The full joint search is ill-conditioned or slow, but the reduced outer problem is well-behaved. * You want a global outer search (e.g. CMA-ES or `Nelder-Mead`) with a fast local inner refinement at every point. ```python theme={null} import ionworks_schema as iws fit = iws.DataFit( objectives=objectives, parameters=parameters, optimizer=iws.optimizers.Nested( parameters=["Negative particle diffusivity [m2.s-1]"], optimizer=iws.optimizers.ScipyMinimize(method="Nelder-Mead"), inner=iws.optimizers.ScipyLeastSquares(), ), ) ``` `parameters` lists the fit-parameter names this level's outer optimizer controls; every other free parameter in the `DataFit` is delegated to `inner`. `inner` can itself be a `Nested` for deeper nesting. The concentrated objective can be non-smooth where the inner optimum is non-unique, so derivative-free or finite-difference outer optimizers (`Nelder-Mead`, `CMAES`, `DifferentialEvolution`) are recommended for the outer slot. Samplers are rejected at construction — both slots must be optimizers. ### Bounded linear inner solves with `ScipyLsqLinear` When the inner parameters enter the residual **linearly** (for example, a linear combination of basis functions with bounded coefficients), pair the `Nested` estimator with `iws.optimizers.ScipyLsqLinear` as the inner solver. It reaches the bounded optimum in a single `scipy.optimize.lsq_linear` call rather than iterating with Gauss-Newton, and warm-starts its active set across successive outer evaluations so the concentrated objective stays smooth: ```python theme={null} optimizer=iws.optimizers.Nested( parameters=["k_nonlinear"], optimizer=iws.optimizers.CMAES(), inner=iws.optimizers.ScipyLsqLinear( method="bvls", # or "trf" warm_start=True, # reuse the previous active set (default) ), ) ``` Use `ScipyLsqLinear` only when the residual really is linear in the inner parameters; for nonlinear inner residuals, keep `ScipyLeastSquares`. ### Analytic parameter Jacobian When the outer or inner slot is a least-squares optimizer (`ScipyLeastSquares` or `ScipyLsqLinear`), the pipeline supplies an **analytic residual Jacobian** $J(x) = \partial r / \partial x$ built from the model's parameter sensitivities. That replaces SciPy's finite-difference Jacobian, so each iteration takes one solve plus one sensitivity evaluation instead of $O(n_\text{params})$ solves. In practice this makes gradient-based least-squares fits substantially faster and more robust to noisy finite-difference steps, especially inside a `Nested` variable-projection loop. No configuration is required — the analytic Jacobian is used automatically when the objective and model support it, and the fit transparently falls back to finite differences otherwise. Look for `using analytic residual Jacobian` in the fit logs to confirm it's active. ## Runtime options `iws.DataFit` accepts an `options` dict that tunes the optimisation loop without changing the schema. All keys are optional. | Key | Default | Description | | -------------------------- | --------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `seed` | `None` | Random seed for reproducible multi-start initial guesses and stochastic optimisers. | | `low_memory` | `False` | Drop log entries that don't improve the best cost by ≥0.1%. Useful for long runs with many iterations. | | `max_iterations` | `None` | Per-job iteration cap. Only applies when the model uses `convert_to_format == 'casadi'`. | | `maxtime` | `None` | Per-job wall-time budget in seconds. With multi-start the total may exceed this since many jobs run. | | `skip_objective_callbacks` | `False` locally / `True` on the cluster | Skip the per-objective callbacks that simulate the model at the initial guess and at the fitted parameters. Improves performance but leaves the initial/final fit results unpopulated. | ```python theme={null} fit = iws.DataFit( objectives=objectives, parameters=parameters, options={ "seed": 42, "low_memory": True, "maxtime": 600, "skip_objective_callbacks": False, }, ) ``` Pipelines submitted to the Ionworks cluster enable `skip_objective_callbacks` by default to reduce simulation cost. Set it explicitly to `False` in `options` if you need the initial- and final-fit simulation results returned with the run. ## Objective-level parallelism `DataFit` exposes `objective_parallelism` as a top-level field — a debugging escape hatch for objective-level scheduling. It is a direct `DataFit` argument, **not** a `Runtime options` key: | Value | Behaviour | | ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------- | | `"auto"` (default) | The execution engine decides. Multiple non-trivial objectives are flattened and evaluated in parallel; single-objective fits stay sequential. | | `"on"` | Force objective-level parallelism. Useful when the auto heuristic underestimates the per-objective cost. | | `"off"` | Force sequential objective evaluation. Useful for reproducing a single-threaded baseline or when debugging an objective in isolation. | ```python theme={null} fit = iws.DataFit( objectives=objectives, parameters=parameters, objective_parallelism="off", # force sequential evaluation while debugging ) ``` Leave it at `"auto"` unless you have a specific reason to override. The setting only affects scheduling — it does not change the cost the optimizer sees. ## Troubleshooting model failures If an objective's model can't be set up or evaluated during a fit — for example because the parameter set is incomplete, a custom model can't be discretised, or a state-of-charge initialisation fails — the pipeline raises a `ModelError` that names the offending objective and the underlying cause: ```text theme={null} Failed to set up the model for objective 'test_1C': Parameter 'Negative electrode thickness [m]' not found. ``` `ModelError` is distinct from a generic `CONFIGURATION_ERROR`: the failed job is tagged with the `MODEL_ERROR` code so the message can be shown to you directly without being routed through internal error monitoring. The fix is almost always to your fit configuration — supply the missing parameter, widen unrealistic bounds, or adjust the model options on the objective — rather than something to report. Already-clear errors pass through unchanged: solver-side numerical failures still surface as `pybamm.SolverError` / `SOLVER_ERROR`, and parameter-lookup failures still surface as `ParameterNotFoundError`. Those continue to be handled by the optimizer's normal fallback (NaN cost, skipped step) when they occur inside an evaluation, so a single bad iteration won't kill the whole fit. ## Retrieving results ```python theme={null} client.pipeline.wait_for_completion(submission.id) result = client.pipeline.result(submission.id) print(result.element_results["fit"]) ``` `result.element_results["fit"]` is a dict keyed by the data-fit's outputs (best parameter values, final cost, and any logged trajectories). See [`packages/ionworks-api/examples/pipeline/datafit.py`](https://github.com/ionworks/ionworks-api/tree/main/examples/pipeline/datafit.py) for an end-to-end example. Cost-function math, identifiability, multi-start strategy. Pick the right cost for your data shape. Stabilise fits with Gaussian priors. Quantify which parameters the fit actually constrains.