Compare and analyse runs

The other recipe pages tell you to list your runs and then ask what the final phase fractions were. That works, and for two runs it is often enough. But Phases can also calculate the difference for you, with a versioned recipe whose identity and digest travel with the answer, and it is available to every signed-in account.

Three tools do this work, and they are meant to be tried in that order:

All three are chat-only. There is no button anywhere in the product that produces an analysis card, and none of them runs any physics — they read runs you already have.

The three deterministic recipes#

run_analysis_recipe takes a recipe_id from a fixed server allowlist. There are three, and the run counts are exact:

Ask for three runs and the recipe refuses with run_comparison_v1 requires exactly 2 run version(s). There is no three-way comparison; do it as two pairs.

Recipe identity, version and a recipe_sha256 digest are part of the result, which is what makes a standard calculation reproducible without anyone writing code.

Ask for a comparison#

Continue the scenario from Compare two grades: two heat-treatment runs on two grades, in the same conversation.

Compare run 1 and run 2.

Analyse the results of run 3 and run 4 and overlay the hardness curves.

Compare just hardness and martensite fraction between run 1 and run 2.

Naming metrics narrows the table. At most 16 metrics come back per call; omit the list and Phases takes the first 16 compatible metric keys in alphabetical order.

For a single run:

Give me summary statistics for run 5.

Reading the analysis card#

Both compare_runs and run_analysis_recipe render the same card, headed Deterministic engineering analysis with the recipe title underneath, a Recipe v1 badge on the right, and a line giving the simulator and the run versions involved. Below that, in order:

The basis badge row. Five labels: Origin, Execution, Scientific, Uncertainty, Applicability. Analysis output is always Deterministic Derivation for origin and Reproducible for execution. Scientific status is inherited from the weakest source evidence and on most cards today reads Not Assessed. That is honesty, not breakage — see How we validate for where calibrated evidence actually exists. Execution status describes bytes and process; it never raises the scientific status.

The compatibility table — Basis, Baseline, Candidate, Status — with one row each for simulator, material definition, solver revision, model revision and calibration revision. Read this before the numbers.

The metric table — Metric, Baseline, Candidate, Delta, Alignment. The delta column carries the relative change as a percentage in parentheses when the baseline is non-zero.

Overlay charts, up to four, each captioned with the alignment and the point count, for example Exact · 512/8000 points. The baseline run is a solid line and the candidate a dashed line.

Mean tiles for summary statistics, a Compatibility and evidence limits warning block, and up to six claim statements, each with its own Scope: line naming the runs, the alignment used and the phrase "numerical comparison only, with no physical validation implied."

The delta table is in kelvin#

This is the detail that surprises people. Before anything is compared, every column is converted to a canonical quantity and unit so that two series can never be subtracted across incompatible units. Temperatures become kelvin.

A cooling rate in K/s and one in °C/s are numerically identical, so that conversion is a relabelling. A temperature delta is also the same number in kelvin as in °C. An absolute temperature is not — read the column heading before you copy a value into a report.

Same session, completed runs only#

The loader is scoped to the session in the request. You cannot compare a run from today's chat with one from last Tuesday's chat, and neither run number resolves outside its own conversation.

The practical rule: if you intend to compare two simulations, run them in the same conversation. See Reuse and share runs for how sessions and run numbering work.

Only successfully completed runs can be analysed. A run that is still running, was cancelled, or failed is refused with only successfully completed runs can be analyzed. A run number that does not exist in this session returns the requested owned run is unavailable.

Exact axes, interpolation, and sampled overlays#

alignment defaults to exact. Two series are compared point-by-point only when their axis columns are identical in key, quantity kind, unit and values, the two value arrays are the same length, and any categorical dimensions match.

When they do not match and you left alignment on exact, the result is not an error: the comparison falls back to endpoint_only, the Alignment column says so, and a warning appears reading axes were not compatible for exact curve comparison; only explicitly labeled endpoint values were compared. You still get the last-value delta, and nothing else pretends to be a curve difference.

Linear interpolation has to be asked for:

Compare run 1 and run 2 using linear interpolation over the overlapping range.

linear_overlap is permitted only over the two axes' overlapping span, only when both axes are strictly increasing, and never when the dataset has categorical dimensions. If any of that fails, it falls back to endpoint_only too.

Overlay charts are capped at 512 display points per metric. When a series is sampled, the caption discloses it (512/8000 points) and a warning says full-resolution claims require the source runs. A sampled display series is for looking at. It is not the evidence.

What each simulator can feed it#

The loader reads a run's recorded metadata plus a fixed set of known data files, searched up to four levels below the run prefix:

An RSW run therefore reduces to its recorded scalars — peak temperature, final temperature, elapsed time, and nugget diameter and final hardness where the run recorded them. You get deltas between two welds; you do not get an overlaid temperature history. Everything a spot weld actually shows you is in its plot gallery, described in Resistance spot welding.

Bounds worth knowing: at most 12 datasets per run, 32 quantities per dataset, 10,000 rows per artifact, 2 MB per file and 6 MB total. When a limit trims something, the card says so in the warnings block rather than quietly truncating.

When it refuses, and what the message means#

The comparison is designed to fail closed rather than hand you a misleading number.

Different simulator types. cross-simulator comparison is not allowed; compare runs of one simulator type. There is no unit system in which a CCT sweep and a spot weld are the same experiment.

Differing solver, model or calibration revision. The compatibility row reads Incompatible, no deltas are calculated at all, and the warning says Comparison was not calculated because immutable model, solver, or calibration revisions differ. The fix is to rerun both cases under one revision, or read the two runs separately. Phases will not subtract results produced by two different versions of the physics.

A legacy run with no revision recorded. The row reads Unknown, and the whole comparison is marked numerically descriptive and unqualified. Phases does not back-fill today's revision pointers onto an old run to make the table look tidier.

A different material definition. This is not a failure. The row reads Different, the numbers are calculated, and the consequence line says the deltas must be interpreted as a material comparison — which is exactly what you wanted when you set out to compare two grades.

A guest session. Anonymous sessions resolve no workspace, so the tool refuses with Deterministic engineering analysis is disabled for this workspace. Sign in. See Guests and limits.

When a recipe is refused for a bad request, the reply carries the list of available recipes and the line "Use run_python_analysis only if no allowlisted deterministic recipe can express the required derivation." The chat card for that case is headed Analysis could not be compared safely.

Ad-hoc numerical work#

When no recipe fits — you want a ratio the catalog does not compute, a combined figure, a custom table — you can ask Phases to write the code:

Use Python to plot the martensite fraction from runs 2, 3 and 4 on one figure.

The word Python in the request is what asks for this lane; without it, Phases prefers a recipe, and the system prompt tells it to.

What actually happens: Phases writes a program defining run(inputs, context), and the backend executes it in a rootless, network-disabled sandbox with a pinned image, a read-only root filesystem, every Linux capability dropped, one CPU core, 768 MB of memory and a 120-second timeout. Allowed import roots are math, csv, io, json, statistics, NumPy, SciPy, pandas, Matplotlib and the simulator SDK. Nothing else.

The data the code sees is not your storage. Either you supply explicit JSON, or you select two to four owned runs from the same session and the backend builds a bounded canonical snapshot of them, carrying evidence and revision digests but never signed URLs, storage paths or credentials. Tool reference lists the exact row, dataset and byte caps.

A successful analysis becomes its own numbered run in the session, with simulator type python-analysis, storing the source, the sanitized inputs, the result, a provenance file and any generated figures. Signed links to those figures last one hour.

The result card is headed Generated-code analysis basis and carries the same five badges, plus a code digest, a source-snapshot digest, the sandbox image digest, an artifact count, per-run source receipts, and a source-sampling line reading, for example, 3/3 runs, 1536/8000 rows. It closes with a sentence worth quoting: execution integrity "describes code execution only; it does not raise the scientific status."

Two honest limits:

• Sampling disqualifies full-resolution claims. If rows were sampled or datasets omitted, peaks, extrema, integrals and residual statistics that need every source point are explicitly not qualified, and the card says so.
• Generated code is post-processing, never physics. It cannot replace a solver, add a simulator, or upgrade the status of anything it reads. Every Python result is scientifically not_assessed and applicable only to the inputs it was given.

If the sandbox is not reachable in your deployment, you get a plain refusal — the isolated Python analysis runtime is unavailable on this deployment — not a silently degraded answer.

What is not available#

Phases has a separate scientific-validation capability — evaluating a calibration project against holdout experimental receipts and writing an immutable report. It is not enabled for customer workspaces, and nothing in the product turns it on. You may see inspect_validation_report and evaluate_validation_project listed among the tools in Tool reference; treat them as documented, not as available. Asking for one today does not produce a validation report.

What the analysis tools on this page give you is a calculated, reproducible, unit-safe difference between two things you ran. That is a real and useful thing. It is not a validation, and the cards say so on every claim.

Related#

• Compare two grades — getting the two runs in the first place
• Reuse and share runs — run numbering, inspection and sharing
• Run a simulation study — when you want twenty runs compared, not two
• Tool reference — full parameter tables for all three tools
• Sessions, runs and artifacts
• Known limits