Skip to main content
Sensitivity analysis quantifies how changes in input parameters affect model outputs. The implementation uses variance-based SOBOL indices through the SALib library to identify which parameters most strongly influence your model predictions.

​
Key Concepts

First-Order (S1)

Direct contribution of each parameter to output variance. An S1 of 0.3 means that parameter alone accounts for 30% of variance.

Total-Order (ST)

Direct effects plus all interaction effects involving that parameter. ST - S1 indicates interaction strength.

Second-Order (S2)

Pairwise interactions between parameters. Computationally expensive but reveals parameter coupling.

​
Workflow

Sensitivity analysis is run after fitting a model: the fit defines the parameter ranges, and the analysis samples those ranges and aggregates how much each parameter influences the model output.
To run sensitivity analysis on a data fit, see Pipelines → Data Fitting → Sensitivity Analysis.

​
Understanding Results

The analysis returns SOBOL indices in the form: 
{
    "S1": [0.65, 0.25],           # First-order indices
    "S1_conf": [0.05, 0.04],      # 95% confidence intervals
    "ST": [0.72, 0.35],           # Total-order indices
    "ST_conf": [0.06, 0.05],      # 95% confidence intervals
}

​
Interpretation Example

For S1 = [0.65, 0.25] and ST = [0.72, 0.35]:
ParameterDirect Effect (S1)Interaction Effect (ST - S1)Total Effect (ST)
Parameter 165%7%72%
Parameter 225%10%35%
  • Unaccounted variance: 1 - sum(S1) = 10% from joint interactions
  • Total effects sum > 1: Because interaction contributions involve both parameters
If ST ≈ 0 for a parameter, it has negligible influence on model outputs within the specified bounds. Consider fixing that parameter or checking if bounds are physically realistic.

​
Configuration Options

​
Sample Size

The SOBOL algorithm generates n_samples × (2 × n_params + 2) evaluations. Recommendations:
  • Use powers of 2: 256, 512, 1024
  • Larger values reduce confidence intervals but increase computation time
  • Start with 256 for exploratory analysis

​
Second-Order Indices

Element S2[i,j] quantifies the interaction between parameters i and j beyond their individual effects. Computing it requires more samples but reveals which parameters are coupled.

​
Practical Workflow

1

Initial Parameter Screening

Run with a moderate sample size (256) to identify important parameters. Parameters with ST < 0.05 are typically negligible.
2

Refine Analysis

For important parameters, increase the sample size (e.g. 1024) to tighten confidence intervals.
3

Investigate Interactions

If ST − S1 is large for multiple parameters, enable second-order indices to find which pairs are coupled.

​
Handling Failed Evaluations

If model evaluations fail for some parameter combinations, the analysis will warn — e.g. “15 out of 853 evaluations (1.76%) failed”. If many evaluations fail (>5%):
  • Check parameter bounds for physical validity
  • Verify model stability across the parameter space
  • Consider tightening bounds around the fitted values

​
Log-Transform Option

If the cost landscape has an extremely large range, apply a log transform during the analysis. With a log transform, the sensitivity represents multiplicative changes in cost due to parameters, rather than additive changes.

​
Mathematical Background

SOBOL indices are variance-based measures that decompose the total output variance into contributions from individual parameters and their interactions. This approach treats the model as a black box and samples the entire parameter space, making it a global sensitivity method—in contrast to local methods like gradient-based sensitivity that only characterize behavior near a single point.

​
Variance Decomposition

Consider a model Y=f(X1,X2,…,Xp)Y = f(X_1, X_2, \ldots, X_p) where XiX_i are independent input parameters. The total variance of the output can be decomposed as: V(Y)=∑iVi+∑i<jVij+⋯+V1,2,…,pV(Y) = \sum_i V_i + \sum_{i<j} V_{ij} + \cdots + V_{1,2,\ldots,p} where ViV_i is the variance contribution from parameter ii alone, VijV_{ij} captures the interaction between parameters ii and jj, and so on for higher-order interactions.

​
First-Order Index

The first-order index SiS_i measures the direct contribution of parameter XiX_i to the output variance, excluding any interactions: Si=VXi(EX∼i(Y∣Xi))V(Y)S_i = \frac{V_{X_i}(E_{X_{\sim i}}(Y|X_i))}{V(Y)} The notation X∼iX_{\sim i} means “all parameters except XiX_i”. The inner expectation EX∼i(Y∣Xi)E_{X_{\sim i}}(Y|X_i) averages over all other parameters while holding XiX_i fixed. The outer variance VXi(⋅)V_{X_i}(\cdot) then measures how much this conditional mean varies as XiX_i changes. Interpretation: If Si=0.3S_i = 0.3, then 30% of the output variance is directly attributable to XiX_i alone, without considering interactions.

​
Total-Order Index

The total-order index STiS_{T_i} captures both the direct effect of XiX_i and all interactions involving XiX_i: STi=EX∼i(VXi(Y∣X∼i))V(Y)=1−VX∼i(EXi(Y∣X∼i))V(Y)S_{T_i} = \frac{E_{X_{\sim i}}(V_{X_i}(Y|X_{\sim i}))}{V(Y)} = 1 - \frac{V_{X_{\sim i}}(E_{X_i}(Y|X_{\sim i}))}{V(Y)} The difference STi−SiS_{T_i} - S_i quantifies how much of parameter ii‘s influence comes through interactions with other parameters. Interpretation: If STi≈0S_{T_i} \approx 0, the parameter has negligible influence on the output—even through interactions—and could potentially be fixed without loss of model fidelity.

​
Second-Order Index

The second-order index SijS_{ij} isolates the interaction effect between parameters ii and jj: Sij=VXi,Xj(EX∼ij(Y∣Xi,Xj))V(Y)−Si−SjS_{ij} = \frac{V_{X_i,X_j}(E_{X_{\sim ij}}(Y|X_i,X_j))}{V(Y)} - S_i - S_j This subtracts the individual first-order effects to reveal only the joint contribution that cannot be attributed to either parameter alone. Interpretation: A large SijS_{ij} indicates that the effect of parameter ii depends on the value of parameter jj (and vice versa). This coupling may suggest physical dependencies or identifiability issues.

​
Saltelli Estimators

Computing SOBOL indices requires evaluating high-dimensional integrals. The Saltelli sampling scheme provides efficient Monte Carlo estimators using two independent sample matrices A\mathbf{A} and B\mathbf{B}, each of size n×pn \times p. For each parameter ii, a hybrid matrix AB(i)\mathbf{A}_B^{(i)} is constructed by taking all columns from A\mathbf{A} except column ii, which comes from B\mathbf{B}. First-order index estimator: S^i=1n∑j=1nf(B)j(f(AB(i))j−f(A)j)V(Y)\hat{S}_i = \frac{\frac{1}{n}\sum_{j=1}^{n} f(\mathbf{B})_j(f(\mathbf{A}_B^{(i)})_j - f(\mathbf{A})_j)}{V(Y)} Total-order index estimator: S^Ti=12n∑j=1n(f(A)j−f(AB(i))j)2V(Y)\hat{S}_{T_i} = \frac{\frac{1}{2n}\sum_{j=1}^{n} (f(\mathbf{A})_j - f(\mathbf{A}_B^{(i)})_j)^2}{V(Y)} This scheme requires n(2p+2)n(2p + 2) model evaluations for first and total-order indices, where nn is the base sample size and pp is the number of parameters. The computational cost scales linearly with the number of parameters, making it tractable for moderate-dimensional problems.

​
Comparison with Other Methods

MethodPurposeWhat it Characterizes
Sensitivity AnalysisWhich parameters matter?Parameter importance
Linear Confidence IntervalsHow precisely are parameters estimated?Parameter uncertainty
MCMC SamplingWhat are parameter distributions?Full posterior distributions
Use sensitivity analysis before extensive MCMC sampling to identify which parameters are worth the computational investment.

​
References

  1. Herman, J., & Usher, W. (2017). SALib: An open-source Python library for Sensitivity Analysis. Journal of Open Source Software, 2(9), 97.
  2. Saltelli, A., et al. (2010). Variance based sensitivity analysis of model output. Computer Physics Communications, 181(2), 259-270.