Validation

A Validation element takes the parameters produced earlier in the pipeline, simulates the experiments listed in objectives, and compares those simulations to the measured data. It is how you check whether a fit generalises beyond the data it was fit on.

A minimal validation

import pybamm
import ionworks_schema as iws
from ionworks import Ionworks

known = iws.direct_entries.DirectEntry(
    parameters={"Ambient temperature [K]": 298.15},
)

# Held-out experiment(s) to validate against
val = iws.Validation(
    objectives={
        "0.5C": iws.objectives.CurrentDriven(
            data_input="file:examples/data/chen_synthetic_0.5C/time_series.csv",
            options={"model": pybamm.lithium_ion.SPMe()},
        ),
    },
    summary_stats=[iws.costs.RMSE(), iws.costs.MAE(), iws.costs.Max()],
)

pipeline = iws.Pipeline({"known": known, "validate": val})

client = Ionworks()
submission = client.pipeline.create(pipeline)
client.pipeline.wait_for_completion(submission.id)

If summary_stats is omitted, sensible defaults are filled in so the report carries the same physical units as the measurements.

Validation.objectives is validated against the same discriminated type union as DataFit.objectives: unknown objective types, missing or conflicting discriminators, unknown inner keys, and bogus scalar values (e.g. {"bad": "RMSE"}) are rejected at submission time. See Strict objective validation for the full list of rules and examples.

Retrieving validation output

element_results holds the per-objective summary statistics:

result = client.pipeline.result(submission.id)
print(result.element_results["validate"]["summary_stats"])

The full per-point comparison (time-series traces, plot configs) is persisted separately and fetched with get_element_metadata:

metadata = client.pipeline.get_element_metadata(submission.id, "validate")

The element name ("validate" above) is whatever key you used in the pipeline’s elements dict — a pipeline can run several validation elements under names like "validate_pristine" and "validate_aged".

Objectives without validation plots

A few objective types — most notably iws.objectives.Pulse and other feature-extraction objectives — do not produce time-series validation plots, because the “measurement” the fit compares against is a derived feature (a pulse resistance, a fitted slope, …) rather than a continuous trace. When a validation step runs against one of these objectives, the validation element’s result comes back as a flat dict carrying a validation_not_supported flag at the top level instead of plot data — the whole element, not a per-objective entry:

result = client.pipeline.result(submission.id)
# The validation element (named "validate" here, after the key you gave it in
# the pipeline) carries the flag at the top level of its result.
validation = result.element_results["validate"]

if validation.get("validation_not_supported"):
    print(validation["message"])
    # → "Validation plots are not supported for this objective type."

The fit itself still runs and its summary statistics are still reported — only the per-point validation plots are unavailable. In the UI, the validation tab shows this message rather than a blank chart or a generic error.

Validating after a fit

The common pattern is to run the fit and the validation in the same pipeline so they share parameters automatically:

pipeline = iws.Pipeline(
    {
        "known": known,
        "fit": fit,           # iws.DataFit, see /pipelines/data-fitting/overview
        "validate": val,
    },
)

The validation step receives the best-fit parameters from fit and runs the held-out experiments against them.

Inspecting validation plots in Ionworks Studio

Validation and data-fit results render as interactive plots in the Ionworks Studio web app. The plots support:

Zoom: Click and drag to zoom into a region
Pan: Hold shift and drag to pan
Reset: Double-click to reset the view
Hover: Move cursor over data points to see values

The initial view shows a downsampled trace so large jobs render quickly. When you zoom in, the app automatically refetches a higher-resolution slice for the visible x-range. A loading indicator appears while data is being fetched.

This applies to both Validation elements and the validation plots produced by DataFit elements. The trace data and per-plot layout are persisted separately from element_results; the SDK exposes the same payload via client.pipeline.get_element_metadata(submission_id, element_name).

Pipelines overview

How validation chains with direct entries, calculations, and data fits.

Data Fitting overview

Producing the parameters that validation then checks.

Get Started

Core Concepts

Data

Build

Pipelines

Simulate

Optimize

A minimal validation

Retrieving validation output

Objectives without validation plots

Validating after a fit

Inspecting validation plots in Ionworks Studio

Pipelines overview

Data Fitting overview

​A minimal validation

​Retrieving validation output

​Objectives without validation plots

​Validating after a fit

​Inspecting validation plots in Ionworks Studio

Pipelines overview

Data Fitting overview

A minimal validation

Retrieving validation output

Objectives without validation plots

Validating after a fit

Inspecting validation plots in Ionworks Studio