Every mapping passed alone—then the ontology hub made five Brick classes impossible

The benchmark’s strongest result appears after every pair passes

A new evaluation of Information Exchange Standard (IES) crosswalks demonstrates a failure mode that pairwise certification cannot detect. The authors built mappings from three IES profiles to 13 target ontologies spanning seven sector groups, then evaluated raw, reviewed, certified, and reviewed-plus-certified variants. Across the 12 pairs with class overlap, review plus certification made 11 coherent and conservative; HQDM was the exception because one retained equivalence pointed to a class already unsatisfiable in HQDM. But when the three individually passing building mappings were loaded together, the combined hub made five Brick classes unsatisfiable: Floor, Storey, Basement, Parking_Level, and Rooftop.[1]

That is the practical benchmark result. A test split into isolated pairs can report success while the deployed composition fails.

Review and certification catch different errors

The study’s raw matcher produced 90 correspondences. Hierarchy-based certification downgraded 56 of them and removed all 526 conservativity violations found across seven pairs, without deleting correspondences. Human review alone removed homonyms but left 73 conservativity violations across five pairs. Certification alone also failed: it retained a BOT Element homonym and left five IES classes unsatisfiable. Only review followed by certification made all three building pairs separately coherent and conservative.[1]

The protocol is unusually explicit. Each alignment ran three times with identical candidate sets; timings are medians of three runs on an Apple M3 Max. The evaluated targets include BOT 0.3.2, Brick 1.4.2, SAREF4BLDG 1.1.2, the Open Energy Ontology 2.14.0, GeoSPARQL 1.1, SOSA/SSN, PROV-O, OWL-Time, HQDM 1.0 and BFO 2020. The paper also names five targets it could not obtain as openly licensed OWL in the offline run rather than silently treating them as evaluated.[1]

The hidden error required a third ontology

The building hub combined IES, BOT, Brick and SAREF4BLDG with 31 reviewed-and-certified crosswalk rows. Its failure came from mapping the IES building extension’s Floor, a structural component, to Brick’s Floor, a building level equivalent to Storey. IES itself has no disjointness axioms, so the IES–Brick pair stayed coherent. BOT supplies the missing contradiction by declaring building elements and zones disjoint. Once all three ontologies were present, Brick’s five level classes fell under both sides and became unsatisfiable.[1]

Rejecting that single row reduced the hub from 31 to 30 rows, brought newly unsatisfiable classes from five to zero, kept conservativity violations at zero, and increased entailed cross-ontology links from 280 to 290. HermiT classification fell from 1.7 to 1.4 seconds.[1]

The core hub exposed a second composition failure: Open Energy Ontology and BFO 2020 were inconsistent together even with every crosswalk removed. Excluding BFO yielded a coherent seven-input hub with 95 inferred cross-links, but classification took 451.4 seconds; excluding Open Energy Ontology yielded none and took 12.8 seconds.[1]

The deployment rule is simple: certify each pair, but rerun coherence and conservativity over the entire hub whenever any ontology or crosswalk changes. Pairwise green checks are necessary evidence—not a release gate.