Cases / NMR structure task 01
NMR structure elucidation · structure plus evidence explanation

Infer a structure from a published characterization entry

The task gives one compound's complete characterization entry from a paper's experimental section, with the compound name withheld: 1H, 13C and 19F NMR, HRMS and IR. The agent must infer the complete structure and write it as SMILES, plus a separate evidence-based explanation. Two frontier models each received one run under the same protocol.

structure metric: independent graph comparison
explanation: 100-point rubric, reviewed separately
single run per model

The task

Data
One paper's characterization entry, supplied as text and as a local data file
Question
Infer the neutral molecular structure and return it as SMILES
Inputs allowed
Local computational tools (e.g. RDKit) for analysis; no web search, databases, paper lookup or hidden solutions
Outputs
A JSON file with a single SMILES, and a Markdown explanation
Scoring
Structure: correct / incorrect by molecular-graph comparison. Explanation: 7 dimensions, 100 points, reviewed on the submitted text only

Data supplied (verbatim excerpt)

¹H NMR (500 MHz, chloroform-d) δ 7.79 (s, 1H), 7.56 (d, J = 8.8 Hz, 1H),
  6.99−6.96 (m, 1H), 3.87 (s, 3H), 3.04−2.90 (m, 3H), 2.83−2.73 (m, 1H),
  2.37−2.31 (m, 1H), 2.26−2.21 (m, 1H), 1.61 (s, 3H);
¹³C{¹H} NMR (126 MHz, chloroform-d) δ 168.2, 158.4, 157.2, 136.4,
  132.2, 126.3 (q, 278.3 Hz), 120.1, 114.3, 99.7, 56.0, 42.1 (q, 27.6 Hz),
  34.5 (q, 1.7 Hz), 31.0, 30.4, 24.9;
¹⁹F NMR δ −59.22 (t, 10.9 Hz);
HRMS (ESI) [M+H]⁺ calcd for C₁₅H₁₆F₃N₂O₂ 313.1158, found 313.1153

Neutral molecular formula: C₁₅H₁₅F₃N₂O₂. Absolute stereochemistry is not required; grading compares connectivity and bond orders after removing stereo annotations.

Results

RunStructureExplanation
Model A0 / 189.0 / 100
Model B0 / 190.5 / 100

One answer each, no retries, no exceptions. Model identities are available to reviewers on request.

Reference structure versus the structure returned by both models
Left: reference structure, methoxy at the 7-position. Right: the structure returned by both runs, methoxy at the neighbouring 8-position. The only difference is one bond to the aromatic ring.

What happened

Both runs produced a structure identical to the reference except for one aromatic substituent position. Both models generated and checked the correct structure as a candidate during the run, and then submitted the neighbouring isomer. Both explanations leaned on the acylated-nitrogen / carbonyl side of the ring for the aromatic assignment and did not give the imine-type C=N environment comparable weight.

The recurring pattern

“Aromatic position selection relies mainly on qualitative deshielding arguments from the acylated N / carbonyl side, without establishing the relative influence of the imine C=N end.”

Both runs follow the same pattern. This is a stable reasoning shortcut, not a random miss.

Explanation scores were high despite the wrong structure. The rubric is designed that way: it rewards correct fragments, atom accounting and calibrated evidence, and a correct final structure gives no automatic explanation credit. Conversely, a wrong final structure does not erase valid partial reasoning.

Explanation dimensions (out of 100)

DimensionModel AModel B
Formula and unsaturation10 / 1010 / 10
Fragments20 / 2020 / 20
Scaffold assembly20 / 2020 / 20
Aliphatic placement10.5 / 1513.5 / 15
Aromatic placement15 / 2015 / 20
Final consistency check8.5 / 108.5 / 10
Evidence calibration5 / 53.5 / 5

Task validation

Before release, the task package was run end to end with its reference solution and a separate verifier container: the reference structure scored 1, both required output files were collected, and structure and explanation metrics are reported independently. The comparison program treats equivalent atom orderings, Kekulé forms, explicit hydrogens and stereochemical annotations consistently, and the analysis-file checks cover missing, empty, invalid and oversized submissions. The original characterization data and the reference structure are unchanged from the source record.

Availability

The full task record for this case (prompt, model outputs, verification logs, reference solution) is available to reviewers on request. Model identities are disclosed with the record.