THE ROSETTA STONE STRATEGY¶
WHAT THIS IS¶
Three real IFC buildings, decomposed into reference DBs. The compiler reads a DSL describing the same building and produces output. The test: does every compiled element land at the same POSITION as the reference?
Not the same dimensions. The same COORDINATES. Position in 3D space. A wall at (5.0, 0.0, 0.0) sized 4000x150x2700 must match a reference wall at (5.0, 0.0, 0.0) sized 4000x150x2700. Same place. Same size.
The reference DB has every answer. Read it. Match it.
PROJECT LAYOUT¶
Compiler source has moved to DAGCompiler/ subfolder (multi-module Maven). Reference (input) and compiled (output) DBs are consolidated under DAGCompiler/lib/. Python scripts used by DAGCompiler are in DAGCompiler/python/.
DAGCompiler/ src/main/java/com/bim/compiler/ — compiler source (Java) lib/ input/ dsl/ — DSL manifests (the .bim files — START of the pipeline) Ifc4_SampleHouse.bim Ifc2x3_Duplex.bim SJTII_Terminal.bim IFC/ — IFC source files (the ground truth) Ifc4_SampleHouse.ifc Ifc2x3_Duplex_Architecture.ifc Ifc2x3_Duplex_MEP.ifc Ifc2x3_Duplex_Federated.ifc — merged ARC+MEP Ifc4_SampleHouse_extracted.db — reference DB (extracted from IFC) Ifc2x3_Duplex_extracted.db — reference DB (extracted from IFC) Terminal_Extracted.db — reference DB (from enhanced_federation_GI.db) output/ — compiled output DBs (generated by E2E tests) python/ — Python scripts used by DAGCompiler extractIFCtoDB.py — unified IFC→reference DB extractor (geometry + materials + surface_styles) reference_schema.sql — canonical CREATE script for reference DBs (incl. surface_styles, material_layers) output_schema.sql — canonical CREATE script for output DBs (incl. surface_styles, material_layers) spatial_checker.py — X-ray fidelity scoring (output vs reference) placement_extractor.py — extract element positions from reference → ad_element_placement pom.xml — dag-compiler module (parent pom at project root)
STONES¶
Stone 1: SampleHouse (UK residential, 1 storey, ARC only, 55 elements) IFC: DAGCompiler/lib/input/IFC/Ifc4_SampleHouse.ifc Extracted: DAGCompiler/lib/input/Ifc4_SampleHouse_extracted.db DSL: DAGCompiler/lib/input/dsl/Ifc4_SampleHouse.bim Output: DAGCompiler/lib/output/ifc4_samplehouse.db
Stone 2: Duplex (US residential, 2 storey, ARC+MEP+STR, 1099 elements) IFC: DAGCompiler/lib/input/IFC/Ifc2x3_Duplex_Federated.ifc (merged ARC+MEP) Extracted: DAGCompiler/lib/input/Ifc2x3_Duplex_extracted.db DSL: DAGCompiler/lib/input/dsl/Ifc2x3_Duplex.bim Output: DAGCompiler/lib/output/ifc2x3_duplex.db
Stone 3: Terminal (MY institutional, 4 storey, 8 disciplines, 48432 elements) IFC: (federated from multiple discipline IFCs → database/enhanced_federation_GI.db) Extracted: DAGCompiler/lib/input/Terminal_Extracted.db DSL: DAGCompiler/lib/input/dsl/SJTII_Terminal.bim Output: DAGCompiler/lib/output/sjtii_terminal.db
ALL THREE must pass. Not 2 of 3. Not "residential only."
THREE VERIFICATION TIERS¶
Tier 1: VOCABULARY — "Do we have the right parts?" Dimensional signature match (category, L, W, H). STATUS: PASSED. Library coverage 100% all 3 stones (Phase 122K). No longer the primary metric.
Tier 2: PLACEMENT — "Are the parts in the right places?" For each compiled element, find nearest reference element of same class. Measure centroid distance. Auto-aligned by wall median. Bands: <50mm (exact), <500mm (near), <2000mm (close), >2000mm (wrong). TARGET: 90%+ elements within 500mm on all 3 stones. THIS IS THE PRIMARY CONVERGENCE METRIC.
Tier 3: INTEGRITY — "Does the building work?" 3a. CONNECTIONS: wall sits on slab, door sits in wall, slab meets wall top. 3b. CLASHES: no solid elements overlapping (excludes hosted: door-in-wall, column-through-slab). TARGET: zero clashes. No reference needed — checks compiled output only.
Run order after every fix (extracted buildings): 1. Tier 2 (positional) — did elements move closer to reference? 2. Tier 3 (clashes) — did clash count decrease? 3. Tier 1 (signature) — informational only, not the target.
THE ROSETTA DICTIONARY — Compositional Verification (S67)¶
The three tiers above verify EXTRACTED buildings: buildings with a full reference DB to compare against (exact sameness). But composed buildings — DemoHouse, BIM Designer creations, C_Project developments — have NO extracted reference. They are assembled from fragments of proven stones.
The insight: the Rosetta Stone was not the full hieroglyphic encyclopedia. It was a short composition — but enough to infer the grammar.
THE DICTIONARY MODEL
When a Rosetta Stone passes exact sameness (G1-G6 ALL GREEN), its BOM becomes a dictionary entry. Every product, every tack offset, every verb pattern in that BOM is a PROVEN WORD. A composed building is a SENTENCE built from proven words.
Dictionary entries (proven by exact sameness): SH_BOM → 55 elements, tack offsets verified, positions exact FK_BOM → 82 elements, same DX_BOM → 1099 elements, same TE_BOM → 48428 elements, same ... (34 stones = 34 dictionary entries)
Composed building = combination of dictionary fragments: DemoHouse = SH structure - original roof (removed) + pitched roof from FK (swapped via WorkOutputDAO.swapProduct) + FP elements from ad_space_type_mep_bom rules + ELEC elements from ad_space_type_mep_bom rules
THE VERIFICATION QUESTION CHANGES:
For extracted buildings: "Does compiled output == reference?" (Exact sameness — Tier 2 positional match)
For composed buildings: "Is each fragment consistent with its proven source, and does the composition satisfy spatial invariants?"
These are different questions. The second one does not need a full reference DB. It needs: (a) Provenance — which stone proved this fragment? (b) Fragment fidelity — does the fragment still match its source? (c) Spatial invariants — does the assembly make geometric sense?
TIER 4: COMPOSITIONAL VERIFICATION — "Are proven words in valid sentences?"
For each C_OrderLine in the composed building:
Step 1: PROVENANCE — trace to source stone
C_OrderLine.family_ref → M_Product → source BOM
The family_ref already encodes which product this is.
The product exists in component_library.db and was proven
correct by at least one Rosetta Stone's exact sameness test.
Step 2: FRAGMENT FIDELITY — verify against source stone's BOM
For structural fragments (from SH_BOM):
- The tack offset (dx/dy/dz) in the composed BOM matches SH_BOM
- The product dimensions match SH's proven M_Product
- The verb pattern (TILE/CLUSTER/FRAME/ROUTE) matches SH's pattern
For swapped fragments (e.g., pitched roof from FK):
- The replacement product exists and was proven in FK_BOM
- The attachment point (locator_ref) is valid for the parent
- The compiled position is consistent with the tack offset
For rule-driven fragments (FP/ELEC from AD rules):
- The product exists in component_library.db
- The placement satisfies the spatial rule (ad_space_type_mep_bom)
- Spacing conforms to the rule's qty/area parameters
Step 3: SPATIAL INVARIANTS — check composition geometry
These are reference-free checks (like Tier 3) but specific to
composition:
- Roof covers structure (AABB containment: roof.z_max >= wall.z_max)
- FP elements are below ceiling (sprinkler.z ≈ ceiling.z - offset)
- ELEC elements are inside rooms (element AABB ∈ room AABB)
- No structural gaps (wall-to-wall adjacency, slab coverage)
- No escapees (all elements inside building envelope)
Step 4: EDGE CASES — fragmented compositions
The balcony chair problem: when a BOM drops furniture into a
balcony, does it end up in the balcony?
- CO_EmptySpaceLine gives spatial slot identity
- Element must be within its slot's AABB (containment check)
- Slot must be within its parent's AABB (recursive containment)
Highly fragmented compositions (many sources mixed):
- Each fragment traces to its source stone (provenance)
- Fragments from different sources don't overlap (clash check)
- Combined envelope is plausible (< site AABB, no negative extents)
HOW IFCtoBOM ALREADY PROVIDES THE DICTIONARY
The extraction pipeline (IFC → IFCtoBOM → BOM.db) decomposes each building into grammar: - WORDS: individual M_Product entries with dimensions - PHRASES: m_bom_line parent-child relationships with tack offsets - SENTENCES: m_bom assembly recipes (BUILDING → FLOOR → ROOM → parts) - GRAMMAR RULES: verb patterns (TILE dx=500, CLUSTER spacing=3000)
Each Rosetta Stone that passes exact sameness CERTIFIES its grammar. The composed building's test is: "every word and phrase used was certified by at least one passing stone, and the sentence is syntactically valid (spatial invariants)."
This is extractable during IFCtoBOM — the dictionary IS the BOM database. No separate "dictionary" table needed. The proof chain:
Stone passes G1-G6 → its BOM entries are CERTIFIED
Composed building uses certified entries → PROVENANCE check
Compiled output satisfies spatial invariants → INTEGRITY check
Both pass → composition is VERIFIED
The test does NOT need to re-verify what the stone already proved. It only needs to verify that the composition is faithful to its sources and geometrically valid.
THE ASI/VIEWPORT MUTATION PATH
When a user drags a wall in the Bonsai viewport: 1. Drag changes M_AttributeSetInstance dimensions (backend, no Bonsai needed) 2. Recompile via CompilationPipeline.run() 3. EYES spatial invariants check the result: - Wall still meets adjacent walls (no gaps) - Doors still inside walls (containment) - Room AABB still encloses furniture (containment) - Building envelope still valid (perimeter closure)
This is a PROPERTY-BASED TEST: for any valid ASI mutation, spatial invariants must hold. Java parameterised tests with ASI variations cover this without launching Bonsai.
The EYES module already has the proof classes: - WallCoverageProof — wall-to-wall adjacency - RoomHasDoorProof — door containment - PerimeterClosureProof — building envelope - SameClassOverlapProof — clash detection
The gap: EYES currently runs only on extracted buildings (with Rosetta Stone data). Running EYES on composed buildings (DemoHouse) would close this — the proofs are reference-free by design.
JAVA ORGANISATION FOR COMPOSITIONAL COMPLEXITY
The existing codebase already has the right patterns. The addition is a verification visitor that walks C_OrderLine and dispatches by fragment type:
Interface: FragmentVerifier.verify(NodeContext ctx, Connection sourceDb) → ProofResult (PASS / FAIL with evidence)
Strategy implementations: ProvenFragmentVerifier — fragment from Rosetta Stone checks: tack offset match, product dimension match, verb pattern match source: the stone's BOM.db (already on disk)
RuleDrivenFragmentVerifier — fragment from AD rules (FP/ELEC/ACMV)
checks: product exists, placement satisfies spatial rule, spacing correct
source: ERP.db AD tables
UserModifiedFragmentVerifier — fragment with ASI override
checks: EYES spatial invariants hold post-mutation
source: output.db compiled result
FreehandFragmentVerifier — fragment from viewport drawing
checks: containment in parent space, adjacency, no clashes
source: output.db + CO_EmptySpaceLine
Walker integration: OrderLineWalker already fires visitor events per C_OrderLine. A CompositionVerifierVisitor walks the order and for each leaf: 1. Determines fragment type (proven / rule-driven / user-modified / freehand) 2. Dispatches to the appropriate FragmentVerifier strategy 3. Collects ProofResults 4. Fails fast on CRITICAL violations, accumulates warnings
Witness claims: W-COMP-PROV-1 — all fragments trace to a certified source W-COMP-FRAG-1 — all fragments match their source's proven offsets W-COMP-SPAT-1 — all spatial invariants hold for the composition W-COMP-CONT-1 — all elements contained within their spatial slots
Gate integration: G7-COMPOSITION gate in RosettaStoneGateTest: Runs ONLY for composed buildings (DM, C_Project children) Requires: G1-G6 GREEN on all source stones first Checks: W-COMP-* witnesses all PASS
RUN ORDER FOR COMPOSED BUILDINGS:
1. Verify source stones pass G1-G6 (prerequisite)
2. Compile the composed building (standard pipeline)
3. Tier 4 Step 1: provenance (all fragments traced)
4. Tier 4 Step 2: fragment fidelity (offsets match source)
5. Tier 4 Step 3: spatial invariants (EYES proofs)
6. Tier 4 Step 4: edge cases (containment checks)
If Step 1 or 2 fails: the BOM is wrong (data issue, not compilation). If Step 3 or 4 fails: the composition has a geometric problem. Steps 1-2 are fast (SQL joins). Steps 3-4 are comprehensive (EYES).
WHAT THE ROSETTA STONE ANALOGY ACTUALLY MEANS HERE
The original Rosetta Stone was a short composition in three scripts. It did not contain all of hieroglyphics. It contained enough grammar to infer the rest. Champollion didn't need a second stone — he needed to trust the first one and extrapolate.
Our Rosetta Stones work the same way: - SH (55 elements) = a simple sentence (1-storey residential) - DX (1099 elements) = a complex sentence (2-storey, 3 disciplines) - TE (48428 elements) = an encyclopedia article (4-storey, 8 disciplines)
Once these pass exact sameness, their grammar is certified. Any new building composed from this grammar does not need its own stone — it needs to confirm it is using certified words in valid sentences.
The DemoHouse merely needs to confirm: - The roof is still where it should be (fragment fidelity) - The FP is aligned to the same ceiling spacing (spatial invariant) - No element has escaped the building (containment)
Once that is done, ALL other extrapolations have their major issues covered. Edge cases (balcony furniture, highly fragmented compositions) are bounded by the containment check — if every element is inside its slot, and every slot is inside its parent, the building is valid.
RULES¶
-
POSITION IS THE METRIC. Not dimensions. Not count. Position. A wall with right dimensions at the wrong position = FAIL. A wall at the right position with slightly wrong dimensions = CLOSE. Fix position first. Then dimensions. Never the reverse.
-
SCORE IS ARBITER. If Tier 2 drops after a change, revert. Check before AND after every change. All 3 stones every time.
-
FIX BUILDING SHAPE BEFORE ELEMENTS. If the building footprint is wrong (Duplex: 20.4x19.1m vs ref 8.8x17.8m), every element inside inherits the error. Fix the footprint. Then elements.
-
OVER-PRODUCTION IS A BUG. If compiled has MORE elements than reference (Terminal walls: 768 vs 330), the compose function is splitting/duplicating when it shouldn't. Remove logic. Check: compiled_count / ref_count per IFC class. Ratio > 1.5 = over-production. Fix by removing splits. Ratio < 0.5 = under-production. Fix by adding dispatch rules.
-
EVERY VALUE FROM THE LIBRARY. No hardcoded dimensions in compose functions. No switch statements on thickness. Every dimension, every dispatch rule, every placement offset reads from an AD table with a profile column. If the table doesn't exist, create it. If the column doesn't exist, add it.
-
CATALOG, DON'T FIX. Every element extracted from a stone goes into component_library.db as a reusable, profile-tagged catalog entry. Not a per-stone hardcode. The library_gap_filler.py tool automates this. Run it on every stone. Apply the INSERTs. Audit with --audit-only.
-
THREE-STONE REGRESSION. After every batch of fixes, all 3 stones must hold or improve. If one stone drops, the fix is overfit. Revert and find the general solution.
-
Rule 5 applies to BOM data too. M_BOM_Line dx/dy/dz are parent-relative per the tack convention (BOMBasedCompilation.md §4). Test: any dx < 0 or dy < 0 or dz < 0 = REJECT.
COMPILED CONSTRUCTION vs REVIT — WHY THIS PROJECT EXISTS¶
Revit is a canvas. You open it, click, place, adjust, repeat. Every element is a manual act. An architect placing 33,324 roof tiles in Revit does it one at a time — or uses a 1D linear array and still manually adjusts thousands. A fire engineer placing 909 sprinkler heads does it one at a time. A structural engineer placing 2,660 rebar does it one at a time. The tool is powerful, but the workflow is manual.
This project is a compiler. You write intent, the compiler produces geometry. The difference is not feature count — Revit has more features. The difference is the fundamental act: AUTHORING vs COMPILING.
AUTHORING (Revit) COMPILING (BIM COBOL) ───────────────────────── ───────────────────────── Place element → see result Write rule → compile all results One element at a time Formula generates thousands Compliance checked AFTER Compliance enforced DURING BOM extracted AFTER BOM generated WITH geometry Different architects → different Same source → identical output files for same building every time Source = the model itself Source = readable text 50K elements = 50K manual acts 50K elements = ~50 formulas + 2K flat
What this means in practice (measured on Terminal — 51,092 elements):
REVIT WORKFLOW COMPILED WORKFLOW ───────────────────────── ───────────────────────── Place 33,324 roof plates TILE 19 panels (auto-expands) Place 909 sprinkler heads ROUTE SPRINKLERS SPACING 3000mm Place 814 light fixtures WIRE LIGHTING per storey Place 2,660 rebar (removed — Bonsai addon script) Route 9,345 pipe/duct segments ROUTE per system (path formula) Position 590 beams + columns FRAME per storey (grid formula) ───────────────────────── ───────────────────────── ~48,969 manual placements ~50 formula statements + 2,123 irregular placements + 2,123 manual placements = days/weeks of work = seconds of compilation
Then separately: Simultaneously: - Extract BOM in CostX/Bluebeam - BOM generated (m_bom + c_orderline) - Check compliance in Solibri - Compliance proven (witness per verb) - Export to IFC (lossy) - IFC is the native output
WHERE REVIT WINS (and will keep winning):
Interactive 3D design — click, drag, see. 20 years of UX. No CLI can match this for creative, bespoke architectural work. Parametric families — 10,000+ manufacturer components. Doors that resize, windows that constrain. We have 122 products. Construction documents — plans, sections, elevations, annotations. We produce none. Multi-user collaboration — worksharing, BIM 360, ACC. We are single-user. Rendering — built-in + cloud. We rely on Bonsai/Blender. Ecosystem — Navisworks, Dynamo, 100+ plugins. We have BIM COBOL + Bonsai. Regulatory acceptance — building departments accept RVT worldwide. Nobody accepts witness proofs yet.
WHERE COMPILED WINS (and Revit cannot catch up):
Compliance at compile time — Revit's Model Checker API was deprecated in 2026. The industry direction is post-hoc checking (Solibri, SMC). BIM COBOL makes non-compliant geometry impossible to produce. BOM → ERP integration — Revit schedules ≠ procurement BOMs. No ERP can import a Revit schedule directly. BIM COBOL outputs iDempiere- compatible M_BOM + C_OrderLine. Import to any ERP. Formula-driven placement — Revit's arrays are 1D (linear/radial, improved in 2026). Bonsai's arrays are 1D (BBIM_Array pset). BIM COBOL has 5 formula patterns covering 95.8% of elements. Deterministic reproduction — same .bimcobol → byte-identical output. Revit: same brief, two architects, two different files. Auditability — fire engineer reads ROUTE SPRINKLERS SPACING 3000mm and signs off. In Revit, they inspect 909 heads visually. Scale efficiency — 51K elements in 2.9K factored rows. Revit: one object per element in memory. Airport terminals choke Revit. Open format + no lock-in — IFC native, SQLite, no subscription. Revit: proprietary RVT, lossy IFC export, $4K/year/seat.
THE STRATEGIC POSITION:
BIM COBOL does not replace Revit. It replaces the MANUAL REPETITIVE WORK inside Revit.
The 95.8% of elements that follow patterns — those are the elements architects spend days placing manually. One by one. Click, place, adjust. Repeat 33,324 times for a roof deck. BIM COBOL generates them from formulas in seconds.
The target is not "all buildings." It is the HIGH-REPETITION, RULE-GOVERNED segment: - Institutional: terminals, hospitals, schools - Residential: mass housing, hundreds of similar units - Infrastructure: repetitive structural bays, MEP systems
These are the projects where 95%+ of elements follow patterns, where manual placement in Revit is the bottleneck, and where compliance checking is the most expensive.
The ceiling: make Revit unnecessary for the pattern-governed majority, while remaining interoperable (via IFC) for the bespoke minority. Bonsai provides the GUI for the 4.2% that needs manual placement. BIM COBOL handles the 95.8% that doesn't.
PLACEMENT DETERMINISM (Phase A — the Tier 2 breakthrough)¶
Tier 1 reached 100% by extracting DIMENSIONS from the stones into metadata. Tier 2 uses the SAME pattern for POSITIONS.
THE PROBLEM: Compose functions were COMPUTING positions from a grid solver. The solver guesses room layout from area constraints, then walls derive from room edges, openings from wall offsets, furniture from room centroids. Every layer compounds the error. This is a convergence problem — iterating towards an answer that already exists in the reference DB.
THE FIX: Extract positions from the reference, store as metadata, compose functions READ coordinates instead of computing them. Same pattern as Tier 1. No solver guesswork. Pure metadata determinism.
HOW IT WORKS: 1. EXTRACT: For each reference element, extract its placement parameters (wall: startX, startY, length, height; door: hostWall, offsetAlongWall; furniture: cx, cy within room; slab: minX, minY, spanX, spanY). 2. STORE: Write to an AD placement table with profile + building_type columns. One row per element instance. Tagged [EXTRACTED: {stone}]. 3. READ: Compose functions read placement params from the AD table instead of computing from grid. The compose function itself doesn't change — only its input source changes. 4. GENERALISE: When the stones are removed, users set those same parameters for their own buildings. The metadata layer becomes a configurable preset. A door at offset 0.3 along a 4m wall is a parameter, not a hardcode.
This turns Tier 2 from a convergence problem into a WIRING problem. Extract → Store → Read. Same as Tier 1.
RULE: Do NOT attempt to make the solver "smarter". The solver is for when there is no reference. When there IS a reference, extract from it.
EXTRACTION PIPELINE — Current Architecture¶
Full pipeline from IFC source to verified output (all Java, no Python):
Stage 1: IFC → component_library.db IfcOpenShell (Bonsai Federation addon) extracts IFC → I_Element_Extraction. ExtractionPopulator.java enriches: storey normalisation, M_Product_ID linkage.
Stage 2: IFC → BOM (Java IFCtoBOM pipeline)
IFCtoBOMMain + DisciplineBomBuilder (CO path: TE) or
ScopeBomBuilder + StructuralBomBuilder (RE path: SH, DX).
Computes tack offsets per BOMBasedCompilation.md §4.
VerbDetector compresses recipes: TILE/CLUSTER/ROUTE/FRAME.
Run: ./scripts/run_RosettaStones.sh classify_sh.yaml
Stage 3: Compile (Java DAGCompiler) BOMWalker traverses BOM tree, PlacementCollectorVisitor accumulates tack offsets. BuildingWriter emits output DB (elements_meta, elements_rtree, geometry).
Stage 4: Verify (Java + Shell) RosettaStoneGateTest: G1-G6 gates (count, volume, digest, tamper, provenance, isolation). TotalityContractTest: per-element spatial diff (output vs reference). Shell: EN-BLOC vs WALK-THRU delta (both must match).
See: TerminalAnalysis.md (TE pipeline detail), WorkOrderGuide.md (YAML config), TestArchitecture.md (gate definitions), SourceCodeGuide.md (code navigation). output DB → Bonsai Full Load → Blender Principled BSDF materials surface_styles.transparency → Alpha (inverted) + blend_method=BLEND surface_styles.specular_exponent → Roughness (inverse mapping) surface_styles.specular_ratio → Specular IOR Level
Format: material_rgba = "R,G,B,A" (0.0-1.0 range, alpha = 1.0 - IFC_Transparency) Glass example: IFC Transparency=0.9 → alpha=0.100 → "0.000,0.502,0.753,0.100" Aluminium example: specular_exponent=128 → roughness≈0.1 (shiny metal)
IFC merge (Duplex): scripts/merge_duplex_ifc.py (IfcOpenShell append_asset) Merges ARC+MEP into Ifc2x3_Duplex_Federated.ifc. Advanced: tools/federation_preprocessor.py (IfcPatch MergeProjects recipe).
COMPOSE FAILURE MODES¶
When Tier 2 score is low, the compose function fails in one of 4 ways:
A. OVER-PRODUCE: more elements than reference. Fix: REMOVE splitting. B. UNDER-PRODUCE: fewer elements than reference. Fix: ADD dispatch rules. C. WRONG POSITION: right count, wrong coordinates. Fix: READ from metadata. D. ABSENT: compose path doesn't exist. Fix: add sub-writer.
Diagnose with: compiled_count vs ref_count per ifc_class → A or B centroid distance per matched pair → C ref elements with no match within 2m → D
Fix priority: C (position) > A (over-produce) > B (under-produce) > D (absent). Position errors cascade to everything. Over-production creates false clashes.
METADATA ARCHITECTURE¶
Every compose function reads from AD tables. Never from constants. Every AD table value has a source. Never invented.
Governing docs (read these): docs/AI_Ground_Truth_Methodology.md — PRIME RULE: extract, don't imagine docs/GLOSSARY.md — provenance tags and definitions docs/ARCHITECTURE.md §1.1, §2 — founding principle + AD pattern
Provenance tags: [EXTRACTED: {stone/source}], [RESEARCHED: {standard §clause}], [PENDING] No other provenance is valid. library_gap_filler.py generates EXTRACTED tags automatically. Manual inserts MUST include provenance inline.
Level 1: BUILDING — profile, template, grid, storey heights Level 2: ROOM — ad_room_slot, ad_space_type_opening, ad_wall_type_rule Level 3: COMPONENT — m_bom_line, m_attribute, ad_floor_type Level 4: PLACEMENT — ad_element_placement (extracted positions per element) Wall: startX, startY, length, orientation, height Opening: host wall ref, offset along wall, offset from floor Furniture: cx, cy relative to room origin Slab: minX, minY, spanX, spanY All keyed by building_type + storey + element_ref. Stone values = preset. User values = override. Framework is the same.
Every AD table that stores element parameters MUST have a profile column. Without profile, it can't distinguish Malaysian from UK from US. Exception: _type definition tables are shared. _rule dispatch tables have profile.
Profile controls WHICH type gets dispatched. The type itself is universal.
span_mode: PER_STOREY (residential) vs CONTINUOUS (institutional) wall_splitting: AT_OPENINGS_ONLY vs AT_OPENINGS_AND_GRID slab_grouping: BY_ROOM vs BY_GRID_BAY
These are profile-level parameters that affect all structural compose functions.
DISCIPLINE ASSIGNMENT¶
Discipline = bom_category on m_bom (a metadata marker, not structure). The BOM walker is discipline-agnostic — same tack convention for all. See BOMBasedCompilation.md §1.1 (disciplines as metadata).
Mapping: YAML disciplines: section maps ifc_class → discipline code.
classify_sh.yaml: ARC only (residential, single discipline)
classify_dx.yaml: ARC, STR, MEP (residential, multi-discipline)
classify_te.yaml: ARC, STR, FP, ACMV, ELEC, SP, CW, LPG (8 disciplines)
Validation per discipline: AD_Val_Rule keyed on bom_category (DocValidate.md).
TOOLS¶
All verification is Java + shell. Python extraction scripts are historical (replaced by Java IFCtoBOM pipeline — ExtractionPopulator, DisciplineBomBuilder).
Full Rosetta Stone run (compile + verify all gates):¶
./scripts/run_RosettaStones.sh classify_sh.yaml # SH only ./scripts/run_RosettaStones.sh # all 3 stones
Tests only (no compile):¶
./scripts/run_tests.sh
Compile only:¶
mvn compile -q
See DEVELOPER_GUIDE.md for build setup, SourceCodeGuide.md for code navigation.
AUDITS (run after every batch of fixes)¶
- Three-stone regression: compile all 3, check all 3. Any drop = revert.
- BomValidator 9 checks + verb fidelity — runs automatically in pipeline.
- Bare constants: every dimension reads from library or YAML. No hardcodes.
- Catalog vs fix: Pattern A (library entry) not Pattern B (hardcode).
SCORE HISTORY¶
Phase DE-5 — Rich Surface Styles (2026-02-17) Scores unchanged (material-only change, no geometry impact). New: surface_styles table (32 SH, 33 DX) + material_layers table (19 SH, 41 DX). extractIFCtoDB.py extracts IfcSurfaceStyle → transparency, specular, reflectance. extractIFCtoDB.py extracts IfcMaterialLayerSet → layer composition. BuildingWriter.java copies surface_styles + material_layers from component_library.db. Bonsai Full Load now creates Principled BSDF materials with: - Glass: transparent (alpha=0.1, blend_method=BLEND) - Aluminium: shiny (roughness≈0.1, specular_exponent=128) - Proper surface RGB from IFC styles (not just flat RGBA) Files: blend_cache.py, stage2_tessellation_loader.py, stage2_gpu_progressive.py Pushed to IfcOpenShell feature/IFC4_DB branch.
Phase DE-4 — Enhanced Terminal Reference (2026-02-16) Stone Recall Precision F1 Output Ref Ratio SampleHouse 100% (55) 100% 100% 55 55 1.00x Duplex 100% (1085) 100% 100% 1085 1085 1.00x Terminal ~100% 100% ~100% 51719 51723 1.00x
Terminal reference upgraded to enhanced_federation_GI.db (Terminal_Extracted.db). 51,723 elements: +33,324 IfcPlate (Metal Deck roof tiles Z=18-28m), +2,660 IfcReinforcingBar, +631 IfcOpeningElement, +292 IfcSlab. Zero code changes — generic emission pipeline handled all new classes. 4 IfcSensor have no spatial coords (metadata-only, no elements_rtree entry). NOTE: IfcSensor is a 7D feature — investigate how they appear in Bonsai without coordinates. May need non-spatial emission path for sensor/IoT elements.
Phase DE-3 — Per-Instance Geometry + Full Precision (2026-02-16) Stone Recall Precision F1 Output Ref Ratio SampleHouse 100% (55) 100% 100% 55 55 1.00x Duplex 100% (1085) 100% 100% 1085 1085 1.00x Terminal 100% (15104)100% 100% 15104 15104 1.00x
Per-instance geometry mapping (DE-3a-c), vertex spot checks (DE-3d), window/door rotation fix (DE-3e), Terminal surplus elimination (DE-3f). All 3 stones at 100%/100%/100% against original 15K Terminal reference.
Tier 1 (Vocabulary — ACHIEVED Phase 122K, no longer primary) Library coverage: 100% all 3 stones.
Cross-Discipline (Phase CD-1 — All Disciplines, 2026-02-15) Stone Discipline Signature Positional<50mm Duplex MEP 98% 100.0% (890/890) Duplex STR 100% 100.0% (12/12) Terminal MEP 99% 100.0% (10844/10844) Terminal STR 98% 100.0% (2860/2860)
ALL DISCIPLINES AT 100% POSITIONAL (<50mm). ARC unchanged. Method: removed ARC-only filter in emitGlobalPlacementElements(), added discipline-aware GUID prefix mapping with class suffix for uniqueness. Same placement metadata pattern that achieved ARC 100%. Element counts: Duplex 1840, Terminal 21,401.
Tier 1+2 Combined (Phase B3 — Exact Fidelity, 2026-02-15) Stone Signature Positional<50mm Positional<500mm SampleHouse 100% 100.0% 100% Duplex 100% 100.0% 100% Terminal 99% 100.0% 100%
ALL 3 STONES AT 100% POSITIONAL (<50mm). Mathematical proof of exact replica. Fix: cladding centering (halfThickness offset) + float32-exact extraction.
Tier 1+2 Combined (Phase B2 — Placement Determinism, 2026-02-15) Stone Signature Positional<50mm Positional<500mm SampleHouse 100% 91.4% 100% Duplex 100% 100.0% 100% Terminal 99% 100.0% 100%
Method: Extract all element bounding boxes from reference DBs → store in ad_element_placement metadata table → compose functions READ exact positions instead of computing them. Turned Tier 2 from convergence problem into wiring problem. Same pattern that achieved Tier 1 100%.
Historical — Spatial Fidelity (Phase A.2, pre-placement) Stone Walls Openings Furniture Slabs X-ray OVERALL SampleHouse 100% 100% 71% 66% 57% 68% Duplex 84% 86% 70% 9% 51% 61% Terminal 92% 51% 0% 0% 33% 44%
Historical — Tier 2 Positional (Phase 122L baseline, pre-placement) Stone <50mm <500mm <2000mm >2000mm SampleHouse 6% 26% 46% 54% Duplex 3% 15% 32% 68% Terminal 1% 3% 5% 95%
Tier 3 (Connections — baseline Phase 122L, to be re-measured) Stone Wall-on-Slab Door-in-Wall Win-in-Wall Slab-Wall SampleHouse 100% 100% 100% 100% Duplex 76% 95% 78% 73% Terminal 19% 85% 81% 5% TARGET 100% 100% 100% 100%
Root cause (RESOLVED Phase B2): ALL THREE STONES: Compose functions COMPUTED positions from grid solver instead of READING them from metadata. Fixed by Placement Determinism — extract coordinates from reference → ad_element_placement → compose functions read them. SH: 68%→100% signature, 0%→100% positional. DX: 61%→100% signature, 6%→100% positional. TM: 44%→99% signature, 0.1%→100% positional.
TERMINAL RECOMPOSITION — THE THIRD STONE BOM CHALLENGE¶
Added 2026-02-28. Maps the full task sequence to create Terminal's {PREFIX}_BOM.db content (m_bom, m_bom_line, c_order, c_orderline).
CURRENT STATE c_order: 1 row (SJTII_Terminal, doc_sub_type=TE, C_DocType_ID=CO_TE) m_bom: ZERO rows for TE (SH has 6, DX has 7, plus 33 generic) c_orderline: ZERO rows for TE (SH has 55, DX has 1,099) Reference: 51,092 elements in Terminal_Extracted.db — 100% positional match TE_BOM.db: Does not exist yet (proposed — see SEPARATE TE_BOM.db below)
Stone 1 SH: 55 elements, 1 storey, 1 discipline (ARC) — DONE Stone 2 DX: 1,085 elements, 2 storeys, 3 disciplines — DONE Stone 3 TE: 51,092 elements, 4+ storeys, 9 disciplines — THIS SECTION
ELEMENT INVENTORY (from Terminal_Extracted.db) Discipline Count Dominant Classes ─────────── ─────── ────────────────────────────────────────────── ARC 34,724 IfcPlate(33,324) Wall(330) Window(236) Furniture(176) Door(135) Member(130) Slab(91) Column(90) Covering(82) BuildingElementProxy(61) Railing(34) StairFlight(32) Roof(2) FP 6,867 PipeFitting(3,146) PipeSegment(2,672) FireSuppression(909) Alarm(80) BuildingElementProxy(31) FlowController(14) REB 2,660 ReinforcingBar(2,660) ACMV 1,621 DuctFitting(713) DuctSegment(568) AirTerminal(289) Proxy(51) CW 1,431 PipeFitting(638) PipeSegment(619) FlowTerminal(106) Valve(57) STR 1,429 Slab(614) Beam(432) Member(312) Column(68) Wall(3) ELEC 1,172 LightFixture(814) BuildingElementProxy(339) Appliance(19) SP 979 PipeSegment(455) PipeFitting(372) FlowTerminal(150) Valve(2) LPG 209 PipeFitting(87) PipeSegment(75) Valve(47)
STOREY STRUCTURE (from spatial_structure) (empty) 34,479 — mostly IfcPlate roof tiles (Z=18-28m) Aras Tanah 4,166 — Ground level Aras 01 2,299 — Level 1 Aras 02 2,765 — Level 2 Aras 03 1,564 — Level 3 Aras 04 400 — Level 4 GROUND FLOOR LEVEL 1,288 — English duplicate of Aras Tanah 02/03/04 FLOOR LEVEL ~1,242 — English duplicates Ceiling levels ~673 — Ceiling levels per storey Aras Kedai/Jalan/Bumbung ~153 — Shop/Road/Roof levels
WHY TERMINAL IS DIFFERENT FROM SH/DX SH/DX are residential. Their BOM hierarchy is: UNIT → FLOOR → ROOM → SET → ITEM Terminal is institutional. There are no "rooms" in the residential sense. Instead there are ZONES: departure hall, check-in counters, boarding gates, retail areas, mechanical rooms, roof structure. The BOM hierarchy must be: BUILDING → STOREY → ZONE → ASSEMBLY → COMPONENT This is a fundamentally different decomposition pattern.
Also: SH/DX had 1-2 IFC source files. Terminal was federated from 9 discipline-specific models. The discipline boundaries are authoritative — they came from separate consultant firms (structural engineer, MEP engineer, fire protection specialist, etc.).
WHAT SH/DX TAUGHT US (foundation advantage) 1. Placement determinism works: extract coords → ad_element_placement → compile. Terminal already has 100% positional match. The position problem is solved. 2. BOM pattern works: m_bom hierarchy + m_bom_line with child_product_id. The pattern is proven. Extending it to 9 disciplines is data, not code. 3. M_Product catalog is extensible: 198 rows currently. Terminal needs ~200 more unique products. Same table, same pattern. 4. Discipline dispatch works: ElementPersistence emits all disciplines. Terminal's 9 disciplines already compile correctly.
The recomposition IS easier because of SH/DX — the patterns exist. The challenge is SCALE and VARIETY, not architecture.
FACTORIZED BOM MODEL — EXPLOITING REPETITION ─────────────────────────────────────────────── THE PRINCIPLE: A naive BOM lists every element as its own m_bom_line row. 51,092 elements → 51,092 rows. But a building is not 51,092 unique things. It is a small number of TYPES arranged in a large number of POSITIONS.
This is algebraic factorization. Instead of:
plate_A + plate_A + plate_A + ... (603 times)
write:
603 × plate_A
The information is identical — nothing is lost, nothing approximated.
The representation is smaller because it factors out the repetition.
A BOM is a RECIPE — "what and how many". A recipe for a cake says
"3 eggs", not "egg, egg, egg". The naive model writes "egg" 603 times.
The factorized model writes "603 × egg". Same cake. Fewer lines.
This is already how iDempiere works: C_OrderLine.QtyOrdered = 100 means
100 of the same product in ONE row. We apply the same pattern to
m_bom_line with a qty column.
WHERE does each instance GO? That's the c_orderline table — one row
per physical element, carrying its unique (x, y, z) position. The BOM
says WHAT and HOW MANY. The order line says WHERE EACH ONE GOES.
Two tables, two concerns:
m_bom_line — RECIPE (type × qty) → factorizable, shrinks
c_orderline — MANIFEST (instance × pos) → irreducible, stays at 51K
THE EVIDENCE — measured repetition in Terminal_Extracted.db:
Discipline Elements Unique Types Factor
────────── ──────── ──────────── ──────
ARC 34,724 617 56×
ELEC 1,172 36 33×
FP 6,863 918 7×
REB 2,660 376 7×
STR 1,429 469 3×
LPG 209 93 2×
ACMV 1,621 778 2×
CW 1,431 683 2×
SP 979 566 2×
────────── ──────── ──────────── ──────
TOTAL 51,088* 4,433 12×
* 4 IfcSensor have metadata only (no spatial coords); 51,092 - 4 = 51,088
"Unique Types" = distinct dimensional signatures (dx × dy × dz rounded to mm).
Each unique type becomes one M_Product row. Each BOM line references a type
with a qty. The factor column shows how many instances share each type on average.
EXTREME CASE — IfcPlate roof tiles: 33,324 plates. Only 14 unique sizes. Top size (500×150×106mm) = 32,203 plates. Spread across 9 Z-bands: Z=18: 1,438 Z=19: 7,356 Z=20: 7,318 Z=21: 6,680 Z=22: 6,846 Z=23: 1,164 Z=25: 291 Z=26: 1,358 Z=27: 873
Level 1 factorization (qty): 33,324 → ~90 m_bom_line rows (type × qty per bay).
LEVEL 2 FACTORIZATION — PARAMETRIC PLACEMENT (the "vector" technique):
Level 1 factors the BOM (what and how many). But the c_orderline table still
needs 33,324 rows — one per plate with its unique (x, y, z). Or does it?
The plates are not scattered randomly. They tile a surface in a GRID.
Measured from Terminal_Extracted.db:
Y-step: 150mm (plate depth, edge-to-edge — 3,774 of 3,819 pairs exact)
X-step: 495mm (plate width, edge-to-edge — 35 of 43 pairs exact)
Each Z-band decomposes into a small number of rectangular PANELS:
Z Panels Total Plates Largest Panel
── ────── ──────────── ─────────────────────────
18 3 1,438 3 strips × 294 rows
19 3 7,356 15 strips × 294 rows
20 3 7,318 14 strips × 294 rows
21 3 6,680 14 strips × 294 rows
22 2 6,846 15 strips × 294 rows
23 2 1,164 5 strips × 174 rows
25 1 291 3 strips × 97 rows
26 1 1,358 14 strips × 97 rows
27 1 873 9 strips × 97 rows
── ────── ────────────
19 33,324
18 of 19 panels are perfect rectangles (every strip same length).
1 panel (Z=20 central) has mixed strip lengths — splittable into sub-rects.
A single panel is fully described by:
origin(x, y, z) + grid(nx, ny) + step(dx, dy) + product_type
EXAMPLE — Z=19 complete roof in 3 formulas:
PANEL origin=(92.49, -42.16, 19.0) grid=(15, 294) step=(0.495, 0.150) → 4,410
PANEL origin=(122.63, -42.16, 19.0) grid=(14, 174) step=(0.495, 0.150) → 2,436
PANEL origin=(141.74, -42.16, 19.0) grid=(15, 34) step=(0.495, 0.150) → 510
total: 7,356
19 formulas replace 33,324 c_orderline rows. 1,754× reduction.
At compile time, each formula expands to individual orderlines with computed
positions: pos(i,j) = origin + (i × dx, j × dy, 0). No lookup needed.
BIM COBOL VERB — TILE: This parametric placement is a natural BIM COBOL verb (verb 11):
TILE SURFACE "ROOF_DECK_Z19" WITH "PLATE_500x150x106"
PANEL "west" ORIGIN (92.49, -42.16, 19.0) GRID 15 x 294 STEP (495mm, 150mm)
PANEL "central" ORIGIN (122.63, -42.16, 19.0) GRID 14 x 174 STEP (495mm, 150mm)
PANEL "east" ORIGIN (141.74, -42.16, 19.0) GRID 15 x 34 STEP (495mm, 150mm)
END-TILE
9 Z-bands × ~2-3 panels = ~20 TILE statements describe the entire roof.
The ScriptRunner expands TILE into c_orderline rows at compile time.
GUI implication: instead of placing 33K tiles, the user draws a surface,
picks a product, sets spacing. The GUI writes one TILE block. That's
parametric design — what Revit curtain walls and ArchiCAD roof arrays do.
FORMULA COVERAGE — ALL 51,092 ELEMENTS (updated 2026-03-04):
Formula Pattern BIM COBOL Verb Elements % Status
──────────────── ────────────────────── ──────── ───── ──────────
TILE (2D grid) TILE SURFACE 33,324 65.2% ★ LIVE v0.9
PATH (1D route) ROUTE SPRINKLERS 9,345 18.3% ★ LIVE
ARRAY (1D linear) ARRAY 2,660 5.2% ★ LIVE v0.9
GRID (ceiling) WIRE LIGHTING / ROUTE 2,012 3.9% ★ LIVE
PERIMETER ENCLOSE / SPAN 1,038 2.0% designed
GRID (structural) FRAME 590 1.2% designed
──────────────── ────────────────────── ──────── ───── ──────────
FORMULA TOTAL 48,969 95.8% 74.4% LIVE
Irregular (flat) manual placement 2,123 4.2%
95.8% of elements follow formula patterns. 74.4% (38,001 elements) are
now covered by LIVE verbs — TILE SURFACE, ARRAY, ROUTE SPRINKLERS, and
WIRE LIGHTING (BIM_COBOL v0.9, 12 verbs, 56 witnesses pass).
BOM SIZE REDUCTION (recipe storage, not compiled output):
Flat c_orderline: 51,092 rows (~20.5 MB)
Factorized verbs: ~50 formulas + ~2,123 flat entries (~349 KB)
Reduction: 58× smaller BOM definition
Verb storage model: PP_Order_Node + PP_Order_NodeProduct tables
(PP_Order_Node pattern from iDempiere Manufacturing). Full schema in
BIM_COBOL.md §15.6. Historical lineage in ADHistory.md.
Pipes follow ROUTE paths. Rebar follows ARRAY spacing. Lights follow
ceiling grids. Columns follow structural grids. Walls follow perimeters.
Only 2,123 elements (furniture, proxies, miscellaneous) need flat
coordinate storage. These are the genuinely irregular positions.
COMBINED FACTORIZATION SUMMARY:
Layer Table Naive Level 1 (qty) Level 2 (formula)
───────── ───────────── ───────── ───────────── ─────────────────
BOM recipe m_bom_line ~51,300 ~700 ~700
Placement c_orderline ~51,092 ~51,092 ~2,200*
───────── ───────────── ───────── ───────────── ─────────────────
TOTAL rows ~102,400 ~51,800 ~2,900
* ~50 formula statements (TILE panels + ROUTE paths + ARRAY hosts +
GRID zones + PERIMETER segments) + ~2,123 flat irregular entries
Level 1 factors the RECIPE. Level 2 factors the PLACEMENT.
Together: 102K → 2.9K rows. 35× total reduction.
The formula IS the placement. Compile = expand formulas to instances.
THE COLUMN: m_bom_line.qty INTEGER DEFAULT 1
Default 1 means every existing SH/DX BOM line works unchanged — one line,
one item, qty=1 (implicit). Terminal sets qty > 1 where types repeat.
BOMWalker expansion:
for (MBOMLine line : children) {
int qty = line.getQty(); // 1 for SH/DX, N for TE
for (int i = 0; i < qty; i++) {
visitor.visitLeaf(line, i); // instance index
}
}
The walker multiplies. Each expansion creates one c_orderline with a unique
position. For TILE rules, position is computed from the formula.
For flat lines, position is read from ad_element_placement.
The BOM is the factored form. Compile = expand.
C_ORDERLINE SEPARATION — ORDER TOPICS vs PRODUCTION DETAIL (2026-03-04)
The PP_Order_Node model (PP_Order_Node) implies a split of c_orderline.
Current c_orderline mixes three concerns:
Concern iDempiere table c_orderline columns (current)
─────────────── ────────────────── ──────────────────────────────
Order topics C_OrderLine building_type, storey, element_ref,
ifc_class, discipline, family_ref
Placement HOW PP_Order_Node host_type, host_ref, position_rule,
position_value ×3, orientation
Material WITH PP_Order_BOMLine width_mm, depth_mm, height_extent_mm,
material_name, material_rgba
After migration:
c_orderline → slim (order topics only): ~134 rows
PP_Order_Node → production steps: ~25 verb lines
PP_Order_NodeProduct → structured placement params
M_Product → material dims (already there)
CO_EmptySpaceLine stays — promoted to spatial workstation (≈ S_Resource).
Verbs TARGET ESLines via co_emptyspace_line_id FK. Multiple verbs per ESLine.
BomCategory unaffected — still drives template composition.
Migration phases (non-breaking):
Phase 1: Add verb tables. New buildings use verbs. Legacy coexists.
Phase 2: VerbStage with fallback to RelationalResolver for legacy.
Phase 3: Migrate extracted buildings. Drop placement columns.
See ConstructionAsERP.md §11.9 for full analysis.
See ADHistory.md §S_Resource Parallel for iDempiere mapping.
SEPARATE TE_BOM.db — STORAGE ARCHITECTURE ─────────────────────────────────────────── Terminal's BOM data does not belong in {PREFIX}_BOM.db. The scale difference alone justifies separation:
Concern {PREFIX}_BOM.db (current) TE_BOM.db (proposed)
────────── ───────────────────── ──────────────────────
Size 1.3 MB (55 BOMs, 1300 OL) ~15-25 MB estimated
Scope Shared dictionary Terminal-specific only
Ownership Multi-building authority Single-building work area
Merge Always read Import into {PREFIX}_BOM.db when proven
WHAT STAYS IN {PREFIX}_BOM.db: - C_DocType (CO_TE already there) - c_order row for SJTII_Terminal (already there) - Generic BOMs (NULL doc_sub_type) reusable by Terminal (33 currently) - M_Product rows (shared catalog — Terminal products added here) - M_BomCategory entries
WHAT GOES IN TE_BOM.db: - Terminal-specific m_bom rows (doc_sub_type = 'TE') - Terminal-specific m_bom_line rows - Terminal c_orderline rows (~51,092) - ad_storey_map (TE-1 normalisation) - Terminal-specific ad_* override tables
PIPELINE CHANGE (future — not this session): CompilationPipeline needs a secondary connection for TE_BOM.db when building_id = SJTII_Terminal. Analogous to how component_library.db is already a second DB connection. Minimal code: // BuildingRegistry or CompilationContext Connection teBomConn = isTe ? open("library/TE_BOM.db") : null;
PHASE TE-1: STOREY MAPPING & SPATIAL NORMALISATION ──────────────────────────────────────────────────── Problem: 34,479 of 51,092 elements have empty storey. Malay and English storey names coexist. Must normalise to a single naming scheme.
Tasks: TE-1a. Audit storey-to-Z mapping. For each named storey, extract Z-range from elements_rtree bboxes. Map unnamed elements (empty storey) to storeys by Z-coordinate. TE-1b. Create storey normalisation map: Aras Tanah = GF, Aras 01 = L01, GROUND FLOOR LEVEL = GF (merge), etc. Store in ad_building table or a new ad_storey_map. TE-1c. Assign the 34,479 roofless elements. The IfcPlate roof tiles are above the highest occupied storey — they form a ROOF zone, not a storey. Assign to ROOF pseudo-storey. TE-1d. c_order row for Terminal already exists (SJTII_Terminal, doc_sub_type=TE, C_DocType_ID=CO_TE, element_count=51088). Verify and update if needed.
Deliverable: Every element has a normalised storey. c_order exists. Witness: COUNT(DISTINCT normalised_storey) matches spatial_structure. Dependency: None. Can start immediately.
PHASE TE-2: ARC ENVELOPE DECOMPOSITION ──────────────────────────────────────── The 34,724 ARC elements break into sub-problems:
TE-2a. ROOF DECK (33,324 IfcPlate). These are metal deck roof tiles — highly repetitive. Group by: - Z-level (panels at same height form a deck section) - X-Y grid bay (panels within same structural bay) Create ROOF_DECK_BAY assembly BOMs. Each bay BOM has N plate children. Estimated: ~50 bay assemblies × ~600 plates each.
TE-2b. WALLS (330 IfcWall). Group by storey. Create WALL_SET BOMs per storey-zone. Include ARC walls only (STR has 3 walls separately). Estimated: 4 storeys × ~80 walls/storey.
TE-2c. OPENINGS (236 IfcWindow + 135 IfcDoor = 371). Host to parent walls via ad_element_placement host_ref. Create OPENING_SET BOMs per wall. Same pattern as SH/DX.
TE-2d. FURNITURE (176 IfcFurniture). Group by storey-zone. Assign to zone BOMs. Terminal furniture: seating, counters, check-in desks.
TE-2e. STRUCTURE-ADJACENT (130 IfcMember + 90 IfcColumn + 91 IfcSlab). These are ARC-discipline structural elements (not STR discipline). Likely curtain wall framing, architectural columns, floor slabs. Group by storey. Create ARC_STRUCT_SET per storey.
TE-2f. MISC (82 Covering + 61 Proxy + 34 Railing + 32 StairFlight + 2 Roof + 1 Ramp). Group by storey. Add to zone BOMs as children.
Deliverable: m_bom tree for all ARC elements. m_bom_line rows link children. Witness: SUM(leaf element count across all ARC BOMs) = 34,724. Dependency: TE-1 (storey mapping).
PHASE TE-3: STRUCTURAL DECOMPOSITION ────────────────────────────────────── 1,429 STR elements + 2,660 REB elements = 4,089 structural.
TE-3a. STRUCTURAL FRAME (432 Beam + 68 Column + 3 Wall + 312 Member). Group by storey. Create STRUCT_FRAME_LXX BOMs per storey. Beams and columns form grid bays — group by bay if grid extractable.
TE-3b. STRUCTURAL SLABS (614 IfcSlab). Group by storey. One SLAB_SET per storey.
TE-3c. REINFORCING (2,660 IfcReinforcingBar). Group by storey and parent element (slab, beam, column). Create REBAR_SET per parent structural element type per storey. This is the largest single-class group after IfcPlate.
Deliverable: m_bom tree for STR+REB disciplines. Witness: SUM(leaf count) = 4,089. Dependency: TE-1.
PHASE TE-4: MEP SYSTEM DECOMPOSITION ────────────────────────────────────── 5 MEP sub-disciplines, each a separate pipe/duct network.
TE-4a. FIRE PROTECTION (6,867 elements). THE LARGEST MEP system. Three sub-assemblies: - PIPE_RUN: PipeSegment(2,672) + PipeFitting(3,146) grouped into runs by storey + spatial proximity. - SPRINKLER_SET: FireSuppression(909) + their feed pipes. - CONTROLS: Alarm(80) + Proxy(31) + FlowController(14) + others. Create FP_SYSTEM_LXX per storey. Each contains pipe runs + sprinkler sets.
TE-4b. ACMV / AIR CONDITIONING (1,621 elements). - DUCT_RUN: DuctSegment(568) + DuctFitting(713) grouped by storey-zone. - AIR_TERMINAL_SET: AirTerminal(289) + Proxy(51) grouped by zone. Create ACMV_SYSTEM_LXX per storey.
TE-4c. COLD WATER (1,431 elements). - PIPE_RUN: PipeSegment(619) + PipeFitting(638). - FIXTURES: FlowTerminal(106) + Valve(57) + FlowController(7). Create CW_SYSTEM_LXX per storey.
TE-4d. ELECTRICAL (1,172 elements). - LIGHTING: LightFixture(814) grouped by storey-zone. - EQUIPMENT: Proxy(339) + Appliance(19). Create ELEC_SYSTEM_LXX per storey.
TE-4e. SANITARY PLUMBING (979 elements). - PIPE_RUN: PipeSegment(455) + PipeFitting(372). - FIXTURES: FlowTerminal(150) + Valve(2). Create SP_SYSTEM_LXX per storey.
TE-4f. LPG / GAS (209 elements). - PIPE_RUN: PipeSegment(75) + PipeFitting(87). - VALVES: Valve(47). Create LPG_SYSTEM (likely single-storey or riser-based).
Deliverable: m_bom tree for all 5 MEP sub-disciplines. Witness: SUM(leaf count across FP+ACMV+CW+ELEC+SP+LPG) = 12,279. Dependency: TE-1.
PHASE TE-5: M_PRODUCT CATALOG EXPANSION ───────────────────────────────────────── Current catalog: 198 M_Product rows (122 original + 65 P0.1-DEDUP + 11 SH). Terminal introduces classes not in SH/DX: - IfcPlate (metal deck) - IfcReinforcingBar - IfcDuctSegment, IfcDuctFitting - IfcFireSuppressionTerminal, IfcAlarm - IfcAirTerminal - IfcFlowTerminal, IfcFlowController - IfcRailing, IfcStairFlight, IfcRampFlight - IfcCovering (ARC discipline)
Tasks: TE-5a. Extract unique dimensional signatures from Terminal reference. Group by (ifc_class, discipline, width±tolerance, height±tolerance). Each unique signature = one M_Product row. TE-5b. INSERT new M_Product rows. Estimated ~150-200 new products. Tag provenance = [EXTRACTED: Terminal]. TE-5c. Assign child_product_id on all new m_bom_line rows.
Deliverable: M_Product catalog covers all Terminal element types. Witness: library_gap_filler.py --audit-only reports 0 gaps for Terminal. Dependency: Can run in parallel with TE-2/3/4.
PHASE TE-6: BOM TREE ASSEMBLY (FACTORIZED) ─────────────────────────────────────────── Combine phases TE-2, TE-3, TE-4 into a single BOM hierarchy. Uses m_bom_line.qty to factor ~51K leaf refs into ~700 BOM lines. Target DB: TE_BOM.db (separate from {PREFIX}_BOM.db).
TE-6a. Create top-level m_bom: BUILDING_TE_STD (bom_type=BUILDING, doc_sub_type=TE). TE-6b. Create storey-level m_bom: TERMINAL_TE_GF, _L01, _L02, _L03, _L04, _ROOF. Each bom_type=FLOOR. TE-6c. Wire discipline BOMs as children of storey BOMs: TERMINAL_TE_GF → [ARC_GF, STR_GF, FP_GF, ACMV_GF, CW_GF, ELEC_GF, SP_GF, LPG_GF] TE-6d. Wire zone/assembly BOMs as children of discipline BOMs. TE-6e. Wire leaf products (M_Product) as children of assembly BOMs via m_bom_line with qty > 1 where vectorization applies.
PREDICTED HIERARCHY (5 levels):
Level 0: BUILDING_TE_STD (RE)
├── Level 1: TERMINAL_TE_GF (FLOOR) — Ground (Aras Tanah), 4166 elements
├── Level 1: TERMINAL_TE_L01 (FLOOR) — Level 1 (Aras 01), 2299 elements
├── Level 1: TERMINAL_TE_L02 (FLOOR) — Level 2 (Aras 02), 2765 elements
├── Level 1: TERMINAL_TE_L03 (FLOOR) — Level 3 (Aras 03), 1564 elements
├── Level 1: TERMINAL_TE_L04 (FLOOR) — Level 4 (Aras 04), 400 elements
├── Level 1: TERMINAL_TE_ROOF (FLOOR) — Roof zone, ~34,479 elements
│ ├── Level 2: DECK_BAY_Z19_G01..G20 (SET) — Z=19m bays, 7356 plates
│ ├── Level 2: DECK_BAY_Z20_G01..G20 (SET) — Z=20m bays, 7318 plates
│ ├── Level 2: DECK_BAY_Z21_G01..G18 (SET) — Z=21m bays, 6680 plates
│ ├── Level 2: DECK_BAY_Z22_G01..G18 (SET) — Z=22m bays, 6846 plates
│ ├── Level 2: DECK_BAY_Z18/Z23/Z25/Z26/Z27 (SET) — remaining, 5124 plates
│ └── Level 2: ROOF_MEP_TE (SET) — MEP at roof height
└── Each FLOOR contains:
├── Level 2: ARC_TE_LXX (DISCIPLINE)
│ ├── Level 3: WALL_SET_TE_LXX — ~80 walls/storey
│ ├── Level 3: OPENING_SET_TE_LXX — doors+windows hosted on walls
│ ├── Level 3: FURNITURE_SET_TE_LXX — zone furniture
│ └── Level 3: MISC_ARC_TE_LXX — coverings, railings, stairs
├── Level 2: STR_TE_LXX (DISCIPLINE)
│ ├── Level 3: FRAME_TE_LXX — beams + columns + members
│ ├── Level 3: SLAB_SET_TE_LXX — structural slabs
│ └── Level 3: REBAR_SET_TE_LXX — reinforcing bars
├── Level 2: FP_TE_LXX (DISCIPLINE)
│ ├── Level 3: FP_PIPE_RUN_TE_LXX — pipe segments + fittings
│ └── Level 3: SPRINKLER_SET_TE_LXX — fire suppression terminals
├── Level 2: ACMV_TE_LXX (DISCIPLINE)
│ ├── Level 3: DUCT_RUN_TE_LXX — duct segments + fittings
│ └── Level 3: AIR_TERM_SET_TE_LXX — air terminals
├── Level 2: ELEC_TE_LXX (DISCIPLINE)
│ ├── Level 3: LIGHTING_SET_TE_LXX — light fixtures (qty factorized)
│ └── Level 3: EQUIP_TE_LXX — proxies + appliances
├── Level 2: CW_TE_LXX / SP_TE_LXX / LPG_TE_LXX
│ └── (same pattern: pipe runs + fixture sets)
└── ...
PREDICTED SIZINGS:
Two tables, two concerns (see FACTORIZED BOM MODEL above):
- m_bom = NODES in the tree (the assembly definitions)
- m_bom_line = EDGES in the tree (parent→child links, with qty)
The factorization acts on the EDGES — the leaf-level m_bom_line rows.
The nodes are just the tree skeleton; they don't grow with element count.
m_bom rows — tree nodes (in TE_BOM.db):
Level What Count
───── ────────────────────────────────────── ─────
L0 BUILDING_TE_STD 1
L1 Storey BOMs (GF, L01-L04, ROOF) 6
L2 Discipline BOMs per storey + roof bays ~140
L3 Assembly/set BOMs per discipline-storey ~120
───── ────────────────────────────────────── ─────
Total ~270
m_bom_line rows — tree edges (FACTORIZED):
Category Lines Naive (without qty)
───────────────────────────── ───── ───────────────────
L0→L1 (unit → storeys) 6 6
L1→L2 (storey → disciplines) ~50 ~50
L2→L3 (discipline → assemblies) ~140 ~140
L3→products (type × qty) ~500 ~51,092
───────────────────────────── ───── ───────────────────
Total ~700 ~51,300
The top 3 levels are structural wiring — same count either way.
The factorization hits at L3→products: 500 factored lines replace
51,092 naive lines. That's where "603 × plate_A" replaces 603 rows.
FACTORIZATION: ~700 BOM lines instead of ~51,300 = 73× reduction
c_orderline rows: ~51,092 (one per element — carries unique position)
M_Product rows to add: ~200 new products (Terminal-specific types)
TE_BOM.db estimated size: ~15-25 MB (mostly c_orderline + ad_element_placement)
Deliverable: Complete m_bom + m_bom_line hierarchy for Terminal in TE_BOM.db. Witness: Tree traversal from BUILDING_TE_STD reaches all 51,092 leaf elements. Dependency: TE-2, TE-3, TE-4, TE-5 complete.
PHASE TE-7: C_ORDERLINE GENERATION ──────────────────────────────────── SH has 55 c_orderline rows. DX has 1,099. Terminal needs ~51,092.
TE-7a. Generate c_orderline rows from the BOM tree. Each leaf element in the BOM gets one c_orderline row with: - building_type = 'TE' - storey (normalised from TE-1) - element_ref (from reference GUID) - bom_line_ref (FK to m_bom_line) - Placement coordinates from ad_element_placement Factorized BOM lines expand: qty=603 → 603 c_orderline rows, each with a unique position from ad_element_placement. TE-7b. Run BOMWalker + AssemblyStructureVisitor on the new BOM tree. Verify element_assemblies table is populated correctly. BOMWalker must honour qty expansion (see FACTORIZED BOM MODEL above). TE-7c. Run StoreyCompiler to verify storey-level aggregation.
Deliverable: 51,092 c_orderline rows in TE_BOM.db. Witness: c_orderline COUNT matches reference element count. Dependency: TE-6.
PHASE TE-8: VERIFICATION & REGRESSION ─────────────────────────────────────── TE-8a. SpatialDigest: compiled output hash must match reference hash. This is the ultimate gate — same as SH and DX. TE-8b. StructuralCrossCheckTest: per-class element count + bbox digest. TE-8c. Three-stone regression: all 3 stones must still pass (225 PASS). TE-8d. ProveStage: run full prove pipeline on Terminal output.
Deliverable: Terminal at 100% fidelity. 3-stone gate holds. Witness: SpatialDigest match. X1 cross-check pass. Dependency: TE-7.
ESTIMATED EFFORT & ORDERING TE-0 (TE_BOM.db creation) — FIRST. Create empty TE_BOM.db with schema. Schema mirrors {PREFIX}_BOM.db tables (m_bom, m_bom_line, c_orderline, ad_storey_map). Add qty column to m_bom_line schema. Data-only — no code changes. TE-1 (storey mapping) — after TE-0. Unlocks everything. TE-5 (M_Product catalog) — parallel with TE-1. No dependency. NOTE: M_Product rows go into {PREFIX}_BOM.db (shared catalog), NOT TE_BOM.db. The product catalog is universal. TE-2 (ARC envelope) — after TE-1. Largest single phase. TE-3 (structural) — after TE-1. Can parallel with TE-2. TE-4 (MEP systems) — after TE-1. Can parallel with TE-2/TE-3. TE-6 (BOM tree assembly) — after TE-2+TE-3+TE-4+TE-5. Into TE_BOM.db. Uses factorized qty on m_bom_line (~700 rows, not ~51K). TE-7 (c_orderline gen) — after TE-6. Into TE_BOM.db (~51,092 rows). TE-8 (verification) — after TE-7.
Critical path: TE-0 → TE-1 → TE-2 → TE-6 → TE-7 → TE-8 (TE-2 is on the critical path because ARC has 34,724 of 51,092 elements.)
Foundation advantage from SH/DX: - Position problem SOLVED (ad_element_placement + 100% match exists) - BOM pattern PROVEN (m_bom + m_bom_line + child_product_id) - Discipline dispatch PROVEN (ElementPersistence handles all 9) - The work is DATA POPULATION, not architecture change - Factorized BOM model reduces m_bom_line from ~51K to ~700 rows (73× factor) - Estimated: 70% SQL INSERTs, 20% spatial analysis (TE-1, TE-2a), 10% verification
SCOPE¶
Three stones. 100% on all three. Nothing else until done.
ROSETTA STONE GATE — Automated Integrity Verification (2026-03-04)¶
Five gates implemented in RosettaStoneGateTest.java, permanent in Maven surefire stage 2.
Gate definitions: G1: element count reference = compiled G2: total AABB volume reference = compiled (±0.1%) G3: per-element spatial SHA256 (SpatialDigest) G4: self-inspection via git history + source regex (12 extensible rules) G5: every output element traced to library (material + geometry + known IFC class)
Source: DAGCompiler/src/test/java/com/bim/compiler/contract/RosettaStoneGateTest.java See also: docs/BOMBasedCompilation.md §6 for gate rationale and methodology.
BENCHMARK RESULTS — Regression Baseline (2026-03-06)¶
Run after every change. Compare against this baseline. Any regression = revert.
GATE RESULTS (mvn test -pl DAGCompiler, surefire stage 2):
Gate RE_SH RE_DX CO_TE Notes ───────────── ───────────── ───────────── ───────────── ────────────────── G1-COUNT PASS (55=55) PASS (1099) FAIL (-4) TE: Phase B scope G2-VOLUME PASS (+0.00%) PASS (+0.00%) PASS Fixed: orphan RTREE JOIN G3-DIGEST PASS (sha256) PASS (sha256) FAIL Cross-mode hash fmt G4-TAMPER PASS (0/12/10) PASS PASS T5 self-ref excluded G5-PROVENANCE PASS PASS FAIL TB/TE/ST not Phase A ───────────── ───────────── ───────────── ───────────── ──────────────────
ELEMENT-LEVEL VERIFICATION:
Metric SH (55 elem) DX (1099 elem) ────────────────────────────── ────────────── ────────────── Per-class count match 8/8 classes 13/13 classes AABB dimension match 55/55 (100%) 1099/1099 (100%) Centroid position match (±1mm) 55/55 (100%) 1099/1099 (100%) material_rgba coverage 51/55 (93%) 139/1099 (13%) material_rgba ref=out match 51/51 (100%) 139/139 (100%) Overall AABB match EXACT EXACT
BOM / ESLine:
Metric SH DX ────────────────────────────── ────────────── ────────────── C_OrderLine rows 0 0 co_empty_space_line rows 8 11 ESLine L0 (building) 1 1 ESLine L1 (structural) 4 6 ESLine L2 (room content) 3 4 C_Order DocStatus CO CO PP_Order_Node rows 0 0
SUITE COUNTS (all modules):
Module Total Pass Fail Notes ────────────── ───── ───── ───── ────────────────── DAGCompiler varies SH+DX CO_TE 2-phase surefire ORMSandbox 36 33 3 w_compose_dx, w_category_2, w_doctype_1 TopologyMaker 19 19 0 BIM_COBOL 63 60 3 CoverWithRoof ×3 ────────────── ───── ───── ───── ──────────────────
DUAL OUTPUT — _s AND _e HELLO WORLD VERIFICATION¶
RosettaStone is the ONLY HelloWorld stage. No other work until _s and _e both pass against the reference extracted DBs.
run_RosettaStones.sh produces TWO output DBs per stone:
_enbloc = EN-BLOC (HelloWorld POC only): - BOM lines are already tacked (dx/dy/dz set from IFC extraction) - DocType is the flag: when AABB and DocType are consistent throughout, the compiler takes each as-is without recalculating through layers - Single C_OrderLine, single CO_EmptySpaceLine - Root: BUILDING_SH_STD / BUILDING_DX_STD (bom_category='RE', bom_type='BUILDING') - Proves data correctness — basic sanity proof - Cross-ref: BOMBasedCompilation.md §3.1, ConstructionAsERP.md §11.1
_walkthru = WALK THRU (precursor to production): - The proper normal path — recalculates by tacking through each BOM layer (BUILDING → FLOOR → SET → BUY) - C_OrderLine per slot, CO_EmptySpaceLine per slot - Root: BUILDING_SH_STD / BUILDING_DX_STD (structured hierarchy, bom_category='RE') - Proves the compilation mechanism works - Cross-ref: BOMBasedCompilation.md §3.2, ConstructionAsERP.md §11.2
Both produce the same result when the data stack is consistent. EN-BLOC proves data. WALK THRU proves the mechanism. Same result is a consequence of both being correct, not a goal.
Verification: each vs reference (never vs each other) 1. _s vs reference — does EN-BLOC (already tacked) match? 2. _e vs reference — does WALK THRU (recalculated) match? When both match the reference, delta between them is zero as consequence.
Future: as the BOM catalog grows, EN-BLOC becomes rare. WALK THRU is the production path. Every new building arrangement that compiles clean can be committed as an EN-BLOC BOM for future singularity matching.
KNOWN GAPS — Honest Audit (2026-03-06, updated 2026-03-07)¶
Gates G1-G6 verify GEOMETRY (count, position, volume, digest) + ISOLATION (scoping, styles, storeys, spaces, containment). The following gaps were identified and resolved via G6-ISOLATION gate (commit history + tests GREEN).
GAP #1 — CROSS-BUILDING CONTAMINATION IN ASSEMBLY_COMPONENTS RESOLVED: loadAllActiveBomIds + lookupDocSubType in BuildingWriter. Scoped by doc_sub_type. G6 check passes.
GAP #2 — SURFACE STYLES GLOBAL DUMP RESOLVED: Filter copySurfaceStyles by usedMaterials from elements_meta. SH:11, DX:1 (only styles actually used).
GAP #3 — MATERIAL LAYERS GLOBAL DUMP RESOLVED: Filter by usedMaterials (same set). SH:9, DX:26.
GAP #4 — GEOMETRY INSTANCE DE-DUPLICATION LOST RESOLVED: computeGeometryHash SHA-256 with 1mm precision in ElementPersistence. GEO_ boxes deduped; LOD_ path 1:1 by design.
GAP #5 — SPATIAL STRUCTURE SEVERELY REDUCED RESOLVED: emitIfcSpaceFromL2 + normalizeStoreyNames in CompilationPipeline. SH:8 rows (4 IfcSpace), DX:12 rows (4 IfcSpace).
GAP #6 — DX rel_contained_in_space EMPTY RESOLVED: populateSpaceContainment two-pass (storey + room centroid-in-AABB). SH:55, DX:1099.
GAP #7 — DX STOREY REDISTRIBUTION [LOW — INTENTIONAL]
DX reference: 915 elements have storey = '' (unknown).
DX output: 25 elements have storey = '' (unknown).
The compiler resolved 890 elements from "Unknown" to real storeys
via Z-coordinate inference. This is an improvement — the reference
extractor couldn't assign storey to MEP elements. But it means a
naive ref-vs-output storey diff will always fail for DX.
Impact: none on geometry. Storey assignment is richer in output.
Gate G3-DIGEST is storey-agnostic (sorts by coordinates, not storey).
GAP #8 — DX STOREY NAME INCONSISTENCY [LOW]
DX spatial_structure uses "Ground" and "Upper" as storey names.
DX elements_meta uses "Level 1" and "Level 2" for the same storeys.
Two naming schemes within the same output DB.
GAP #9 — GEOMETRY HASH SCHEME CHANGED [LOW — DESIGN CHOICE]
Reference: hex hashes from IFC mesh tessellation (0876e13b5edbfb53).
Output: GEO_<int>_<int> for box geometry, LOD_<hex>_<pos>_<scale>
for library meshes. Different format by design — extraction and
compilation use different geometry pipelines.
Impact: geometry_hash cannot be compared between ref and output.
G3-DIGEST handles this by excluding geometry_hash in cross-mode.
No functional impact — vertex/face counts match.
WHAT THE GATES PROVE vs WHAT THEY DON'T¶
PROVEN (by G1-G6 + ExtractedGeometryTruthTest + StructuralCrossCheckTest): ✓ Element count per IFC class: exact match ✓ AABB position per element: ±1mm match (centroid + all 6 faces) ✓ AABB dimensions per element: exact match (mm precision) ✓ Total volume conservation: ±0.1% ✓ material_rgba propagation: ref coverage = output coverage ✓ No test suppression, stubs, or code cheating (G4 regex + reflection) ✓ Geometry traceable to library (G5 provenance) ✓ Per-class SHA256 digest: byte-exact (StructuralCrossCheckTest) ✓ Assembly/BOM tree isolation per building (Gap #1 — RESOLVED) ✓ Surface style scoping per building (Gap #2 — RESOLVED) ✓ Material layer scoping per building (Gap #3 — RESOLVED) ✓ Geometry instance sharing / de-duplication (Gap #4 — RESOLVED) ✓ IFC spatial structure completeness — rooms + storeys (Gap #5 — RESOLVED) ✓ Spatial containment for DX (Gap #6 — RESOLVED)
NOT PROVEN: ✗ Vertex-level mesh fidelity (G7 — WIP, 12/55 SH wrong meshes) ✗ DX calibration (G8 — intentional) ✗ Storey naming consistency (Gap #8 — intentional, normalizeStoreyNames added)
Gates G1-G6 cover both GEOMETRY and METADATA FIDELITY. Position, dimension, count, isolation, styles, spatial structure, and containment are proven. G7 (vertex fidelity) is next.
TESTING CODE DESCRIPTION — For Expert Scrutiny¶
The verification system has 9 layers across 6 Java test classes plus 1 runtime prover. All live in DAGCompiler/src/test/java/ com/bim/compiler/ unless noted. The system runs in two Maven surefire phases enforced by DAGCompiler/pom.xml.
PHASE 1: COMPILATION (surefire execution "compile-buildings") ────────────────────────────────────────────────────────────── Runs ONLY BuildingRegistryTest.java.
BuildingRegistryTest: Reads C_DocType from {PREFIX}_BOM.db (5 building type definitions). For each active building (IsActive=1), runs the full 9-stage CompilationPipeline: Metadata → Parse → Compile → Template → Write → Verb(SPI) → Digest → Geometry → Prove. Each pipeline run produces one output.db under DAGCompiler/lib/output/. Maven aborts here on failure — no gate tests run if compilation itself fails. This is the only test that writes to disk. All other tests are read-only against the databases this test produces.
PHASE 2: VALIDATION (surefire execution "validate-contracts") ────────────────────────────────────────────────────────────── Runs all *Test.java in contract/, arch/, coordinate/, library/ packages EXCEPT BuildingRegistryTest (already ran in Phase 1).
--- TEST CLASS 1: RosettaStoneGateTest.java (contract/) ---
@TestFactory methods generate dynamic tests per building.
Each building with a reference DB gets G1-G3 and G5.
G4 runs once (source-level, not per-building).
G1-COUNT (lines 65-90):
SQL: SELECT COUNT(*) FROM elements_meta
Runs against reference DB and output DB separately.
Compares: assertEquals(refCount, outCount).
Tolerance: ZERO. Exact match required.
Failure message includes delta (+N or -N).
G2-VOLUME (lines 96-122):
SQL: SELECT COALESCE(SUM((r.maxX-r.minX)*(r.maxY-r.minY)
*(r.maxZ-r.minZ)),0) FROM elements_rtree r
JOIN elements_meta m ON r.id = m.id
The JOIN is critical — without it, orphan RTREE rows
(from deleted elements that leave stale spatial index
entries) inflate the volume. This was the G2 fix.
Tolerance: ±0.1% (deltaPct = ((out-ref)/ref)*100).
Rationale: volume is a second-order check. Two buildings
could have the same volume with different element
arrangements — G3 catches that. G2 catches gross errors
like missing storeys or duplicated slabs.
G3-DIGEST (lines 128-177):
Delegates to SpatialDigest.computeWithReport(dbPath, includeGeoHash).
Cross-mode: includeGeoHash=false (because extraction and
compilation use different hash formats — see Gap #9).
SpatialDigest algorithm (SpatialDigest.java lines 148-237):
1. Query: SELECT ifc_class, minX, maxX, minY, maxY,
minZ, maxZ, material_rgba FROM elements_meta em
JOIN elements_rtree r ON em.id = r.id
ORDER BY ifc_class, ROUND(minX*1000), ROUND(minY*1000),
ROUND(minZ*1000), ROUND(maxX*1000), ROUND(maxY*1000),
ROUND(maxZ*1000)
2. For each IFC class (alphabetical), emit header line:
"CLASS={ifc_class} COUNT={n}"
3. For each element in that class (sorted by 1mm-rounded
coordinates with tie-break on max coordinates):
"{minX_mm}|{maxX_mm}|{minY_mm}|{maxY_mm}|{minZ_mm}|
{maxZ_mm}|{material_rgba}|{geometry_hash}"
4. SHA256 of all lines joined by "\n".
Key design: name-agnostic (no element_name in hash),
storey-agnostic (sorted by coordinates, not storey label),
1mm rounding absorbs float-epsilon noise.
Tolerance: ZERO. Byte-for-byte SHA256 match.
Scrutiny point: the ORDER BY uses ROUND(coord*1000) which
maps to integer mm. Two elements at 1.5004m and 1.5006m
both round to 1500mm and are considered co-located. If
they are the same IFC class, their relative order depends
on the next sort column. The tie-break chain (all 6 coords)
makes this deterministic for all practical cases. A
pathological case would require two elements of the same
class at the exact same 1mm-rounded bounding box — which
would indicate a genuine duplicate (flagged by PlacementProver
P05 NO_DUPLICATE_POSITION).
G4-TAMPER (lines 207-291):
Two-phase scan. No database access — pure text analysis.
Phase A — Git diff scan (scanGitDiff, lines 456-487):
Runs: git log --format=%H -1 --skip=10 (get baseline commit)
Then: git diff <base>..HEAD --unified=0 --diff-filter=AM
Parses unified diff output line-by-line. For added lines
(starting with "+", not "+++"), applies regex rules T1-T5.
Tracks current filename from "diff --git" headers.
Self-exclusion: skips violations in RosettaStoneGateTest.java
(line 479) to prevent T5's pattern from matching its own
definition text.
Phase B — Source file scan (scanSourceFiles, lines 492-529):
Walks codebase directory tree (depth 20). Matches files
against glob patterns (e.g., "DAGCompiler/src/test/java/
com/bim/compiler/contract/*.java"). Reads each matching
file line-by-line. Applies regex rules T6-T12.
Self-exclusion: skips RosettaStoneGateTest.java (line 518).
12 tamper rules (TAMPER_RULES list, lines 215-253):
T1 GIT_DIFF @Disabled|@Ignore in *Test.java
T2 GIT_DIFF assertTrue(true)|assertFalse(false) in *Test.java
T3 GIT_DIFF Thread.sleep|System.exit in src/main/**/*.java
T4 GIT_DIFF skipTests|<skip>true in pom.xml
T5 GIT_DIFF --no-verify|--no-gpg-sign|SKIP_HOOKS (all files)
T6 SOURCE @Disabled|@Ignore in contract/*.java
T7 SOURCE @Disabled|@Ignore in arch/*.java
T8 SOURCE return null|0|emptyList|List.of()|Map.of()
in validation/*.java
T9 SOURCE new Random|Math.random in dsl/*.java
T10 SOURCE TODO|FIXME|HACK|XXX in dsl/*.java
T11 SOURCE empty test body (@Test void x(){}) in contract/*.java
T12 SOURCE hardcoded coords >1000 (=[1-9]\d{3,}[;,)])
in dsl/*.java
Tolerance: ZERO violations.
Git depth: 10 commits (GIT_HISTORY_DEPTH constant).
Scrutiny point: T8 checks only validation/*.java — a stub
returning null in dsl/*.java or bom/*.java would evade T8.
T12 only scans dsl/ — hardcoded coords in other packages
are not caught. These are deliberate scope limits (validation
and DSL are the highest-risk packages), but an adversarial
developer could circumvent by moving code to unscanned packages.
G5-PROVENANCE (lines 297-390):
Five sub-checks per building:
Check 1: material_rgba coverage (NULL/empty count).
If reference exists, output missing <= ref missing.
Check 2: geometry instance linkage. Every element_instances
row must have a matching base_geometries row.
Check 3: minimum vertex count. base_geometries with
vertex_count < 8 = degenerate (cube is minimum).
Check 4: null/empty geometry_hash in element_instances.
Check 5: IFC class whitelist (27 known classes). Unknown
class = invented element type.
Tolerance: ZERO issues.
Scrutiny point: Check 2 verifies linkage within the output
DB only — it does not compare output geometries against
reference geometries. A systematically wrong mesh (correct
hash, wrong vertices) would pass G5. Only visual inspection
or vertex-level comparison would catch mesh corruption.
--- TEST CLASS 2: ExtractedGeometryTruthTest.java (contract/) ---
Three-tier class-agnostic geometric truth test. Runs only for
buildings with Provenance=EXTRACTED and a reference DB.
T1 COUNT (class-agnostic):
SQL: SELECT COUNT(*) FROM elements_meta em
JOIN elements_rtree r ON em.id = r.id
No IFC class grouping. Pure count. assertEquals(ref, out).
T2 VOLUME (integer mm³):
SQL: SELECT SUM(CAST(ROUND((maxX-minX)*1000) AS INTEGER)
* CAST(ROUND((maxY-minY)*1000) AS INTEGER)
* CAST(ROUND((maxZ-minZ)*1000) AS INTEGER))
Converts to integer mm³ before summing — eliminates all
float accumulation error. Tolerance: <0.1% difference.
T3 PLACEMENT MATCH (1:1 AABB multiset):
Builds a multiset (HashMap<String, Integer>) of bbox keys:
key = "{minX_mm}|{maxX_mm}|{minY_mm}|{maxY_mm}|{minZ_mm}|{maxZ_mm}"
Each coordinate: Math.round(coord * 1000) → integer mm.
Counts occurrences per key (handles duplicate AABBs).
Compares ref multiset vs output multiset entry by entry.
Reports: missing (in ref, not in output), phantom (in output,
not in ref), count mismatches.
Tolerance: ZERO. Every reference bbox must have a compiled
partner at the same position and dimensions.
Scrutiny point: T3 is class-agnostic. A wall and a slab at
the same bbox would match interchangeably. G3-DIGEST is
class-aware and would catch this swap. T3 is deliberately
weaker to test "did geometry land in the right place"
independently of "is it the right type of geometry."
--- TEST CLASS 3: StructuralCrossCheckTest.java (contract/) ---
Per-IFC-class SHA256 digest. Stricter than G3 — runs per class,
not aggregated. Used for X1-DX (all 13 DX classes verified).
Algorithm per class:
1. Query coords: SELECT minX, maxX, minY, maxY, minZ, maxZ
FROM elements_meta em JOIN elements_rtree r ON em.id = r.id
WHERE em.ifc_class = ?
2. Round each coord to integer mm: Math.round(val * 1000)
3. Format: "{minX_mm}|{maxX_mm}|{minY_mm}|{maxY_mm}|{minZ_mm}|{maxZ_mm}"
4. Sort the formatted strings AFTER rounding (critical — sorting
raw floats causes cross-platform ordering differences at
the sub-mm level, e.g. 1.500000001 vs 1.499999999)
5. Prepend "COUNT={n}" header
6. SHA256 of all lines joined by "\n"
7. Compare ref hash vs output hash. Byte-exact.
Scrutiny point: does not include material_rgba (unlike G3-DIGEST).
Two elements at the same position with different materials would
pass X1 but fail G3. This is by design — X1 tests structural
geometry, G3 tests full spatial identity including visual fidelity.
--- TEST CLASS 4: CompilerContractTest.java (contract/) ---
Uses Java reflection to access package-private classes and verify
invariants that cannot be tested through the public API.
Reflection targets:
- PlacementLoader$Placement (private inner class): constructor access
- BoundElement (package-private): constructor, validateScaleFactors()
- PlacementLoader.getInstance() + getAll(String): placement cache
- BoundElement.MIN_SCALE / MAX_SCALE fields: 0.3 / 3.0
- CompilationPipeline.STAGES field: verify MetadataValidator is first
Geometric assertions (G1-G8, for TB-LKTN building):
Hand-computed expected coordinates verified at 1mm tolerance:
MM1 = 0.001 (constant)
assertEquals(1.550, cx(window), MM1, "centroidX = ...")
Each assertion includes a comment showing the arithmetic derivation
from room boundaries and wall positions. If the derivation is wrong,
the test is wrong — the assertion is only as good as the formula
in the comment.
Contract tests (C1-C7):
C1: BoundElement null rejection (null placement → exception)
C2: Scale factor bounds [0.3, 3.0] (outside → exception)
C3: PlacementProver critical proofs (P01-P03 must be CRITICAL)
C4: Proof criticality classification
C5: DimensionalContractViolation evidence format
C6: Scale constants via reflection (MIN=0.3, MAX=3.0)
C7: MetadataValidator is pipeline stage[0]
--- TEST CLASS 5: OrderLineInterfaceContractTest.java (ORMSandbox) ---
Reflection-based structural guard for the three-concern separation
(§11.9). Verifies X_C_OrderLine has ONLY "WHAT" columns.
6 tests using getDeclaredMethods() and getDeclaredFields():
W-LOCK-1: no placement getters/setters (getHostType, setHostRef,
getPositionValue, etc.)
W-LOCK-2: no material getters/setters (getWidthMm, setGeometryHash,
setMaterialRgba, etc.)
W-LOCK-3: no placement COLUMNNAME_ constants
W-LOCK-4: no material COLUMNNAME_ constants
W-LOCK-5: exactly 8 setters present (setCOrderId, setStorey, setName,
setIfcClass, setDiscipline, setMProductId, setIsActive,
setADBuildingId)
W-LOCK-6: setter count == 8 (no more, no less)
These tests break the build if ANYONE adds a placement or material
column to X_C_OrderLine. The three-concern lock is enforced at the
Java class structure level, not by convention.
Scrutiny point: tests check method NAMES via string matching.
A method named "setPositionData" (not in the forbidden list) could
smuggle placement data through. The forbidden list is hand-maintained
and must be updated if new placement/material concerns emerge.
--- TEST CLASS 6: ArchitectureTest.java + DriftGuardTest.java (arch/) ---
ArchUnit bytecode analysis (not reflection — scans .class files).
ArchitectureTest (3 rules):
A1: Contract types (IBOMChildLine, IBOMContractor, etc.) must be
interfaces, not abstract classes or enums.
A2: BOM concrete classes not accessed outside approved packages
(com.bim.compiler.bom, com.bim.compiler.contract). Must go
through AssemblerFactory.
A3: No absolute coordinate field names (worldX, absoluteX, posX,
flatX) in dsl/bom/placement packages. Derived accessors
(cx(), cy(), cz()) are permitted.
DriftGuardTest (5 rules):
D1: No Map.getOrDefault() in resolver/bom/placement — hides
missing metadata rows.
D2: No bindParametric() outside MeshBinder — silent geometry
fabrication.
D5: No building-ID string constants in pipeline/resolver/compiler
— use BuildingEntry.id(), never "SAMPLE_HOUSE" literals.
D8: No direct WorldCoord construction outside coordinate package
— prevents accumulation chain bypass.
D9: Only ViewAccessLayer may call executeQuery() — SQL isolation.
Scrutiny point: ArchUnit scans compiled bytecode, not source. A
method using reflection to bypass package boundaries would evade
ArchUnit. The rules are structural — they catch accidental drift,
not adversarial circumvention.
--- RUNTIME: PlacementProver.java (validation/) ---
Not a test class — a runtime prover invoked during compilation
(prove() before write) and during E2E tests (proveFromDB()).
14 proofs in 5 tiers:
Tier 1 (per-element):
P01 POSITIVE_EXTENT: dx>0, dy>0, dz>0
P02 FINITE_COORDS: no NaN/Inf, all in [-100m, 1000m]
P03 MIN_DIMENSION: min(dx,dy,dz) >= 1mm
P04 STOREY_Z_BAND: Z range within storey ± 500mm
Tier 2 (pairwise):
P05 NO_DUPLICATE_POSITION: centroid distance >= 1mm (same class)
P06 NO_SAME_CLASS_OVERLAP: overlap volume <= 10mm³ cube
(exempts walls/beams that legitimately overlap at corners)
Tier 3-5 (relational, require library):
P07-P14: openings contained in walls, furniture in rooms,
fixtures on surfaces, etc. SKIPPED if library unavailable.
Criticality: P01, P02, P03, P04, P16, P17, P22 are CRITICAL.
Critical violation → report.criticalViolations() > 0 → test failure.
Advisory violations are logged but don't fail the build.
Scrutiny point: P06 overlap check exempts walls and beams
(ifc_class contains "Wall" or "Beam"). An IfcWallCustom or
similar non-standard class name would not be exempted and could
false-positive. The exemption list is string-based.
OVERALL SCRUTINY NOTES FOR EXPERT REVIEWER:
1. The verification is GEOMETRY-STRONG, METADATA-WEAK. Every
element's position, dimension, and count are proven to 1mm.
But IFC context (rooms, containment, assemblies, styles) has
known gaps (#1-#6 above). An IFC compliance checker (like
Solibri) would flag the output for missing spatial structure.
2. The tamper detection (G4) is TEXT-BASED, not SEMANTIC. It
catches textual patterns (@Disabled, return null) but not
semantic evasions (a validator that always returns an empty
valid result via a complex codepath, or a test that asserts
on a value computed to always be true).
3. Cross-mode digest (G3) deliberately EXCLUDES geometry_hash
and element names. This is necessary (different hash formats
between extraction and compilation), but it means two elements
with identical bounding boxes but completely different mesh
geometry would pass G3. Visual inspection or vertex-level
comparison is the only catch for mesh corruption.
4. All tests use the SAME 1mm rounding. If both reference and
output have a systematic 0.5mm bias in the same direction,
no test would catch it. The 1mm quantum is the resolution
limit of the entire system.
5. G5 provenance checks linkage WITHIN the output DB. It does
not cross-reference geometry content against the reference.
A correct link to a corrupted mesh passes G5.
6. The 2-phase surefire enforces compilation → validation order.
But there is no mechanism to verify the output DBs were
ACTUALLY produced by the compilation just run (vs stale
cached DBs from a previous successful run). A stale output
DB from yesterday would pass all gates today if no code
changed. BuildingRegistryTest overwrites the output files,
so in practice this is unlikely, but not formally guarded.
SYNTHETIC ROSETTA STONE — Two Round-Trip Proofs (2026-03-05)¶
The Rosetta Stone strategy proves ONE loop: 3D IFC → 1D DSL → 3D output. The Synthetic Rosetta Stone extends this to prove a SECOND loop through 2D.
LOOP 1: 3D → 1D → 3D (current proof — SH/DX at 100%)
IFC source (3D)
→ Extract → {PREFIX}_BOM.db + component_library.db
→ 1D DSL (.bim text + C_Order + BOM recipes)
→ Compile (9-stage pipeline)
→ 3D output.db
→ SpatialDigest must match reference
Already demonstrated: SH 100%, DX 100%. Phase A makes this rigorous
(all 5 gates green). The digest verification (G3) is the gate.
LOOP 2: 3D → 2D → 3D (the novel contribution)
3D compiled output.db
→ 2D export (SVG floor plans + elevations + sections)
→ 2D parser (extract rooms, walls, openings, grid)
→ Generate 1D DSL (.bim) or C_OrderLines
→ Compile (9-stage pipeline)
→ 3D output.db (Synthetic Rosetta Stone)
→ SpatialDigest must match original
This closes the second loop. It proves:
- The 2D export preserves enough information for 3D reconstruction
- An architect's 2D drawings can drive compilation (industry's primary medium)
- The compiler is round-trip stable — no information loss in either cycle
BOTH LOOPS CONVERGE AT VERIFICATION: SpatialDigest(SH_original) == SpatialDigest(SH_synthetic) Same elements, same positions, same dimensions. Lossless round-trip.
WHAT MAKES THIS NOVEL: Traditional BIM is one-way (3D model → 2D drawings for the contractor). This compiler goes both ways. The architect's 2D drawings become first-class compilation input. 3D → 1D → 3D proves the BOM model. 3D → 2D → 3D proves the drawing export. Together they prove the entire chain.
SYNTHETIC STONE TASKS: D1. SH 3D→2D→3D — compile SH → export 2D → parse 2D → recompile → verify D2. DX 3D→2D→3D — same for Duplex (multi-storey, multi-discipline) D3. 2D parser — SVG → room boundaries, wall positions, openings, grid → DSL D4. Terminal 3D→2D→3D — 51K elements, 9 disciplines D5. Register Synthetic Rosetta Stones in C_DocType (SH_SYN, DX_SYN, TE_SYN)
GATE: SpatialDigest match for all three synthetic stones. DEPENDENCY: Phase C (2D export), Phase B (Terminal data for D4).
Full roadmap: docs/ACTION_ROADMAP.md
WHY ROSETTA STONES EXIST — The Training Set Thesis¶
The Rosetta Stones are not the product. They are the TRAINING SET.
A stone that round-trips clean is one more "this is how a building gets built" pattern burned into the BOM dictionary. Each proven compilation teaches the compiler a default path — the known-good route from intent to 3D output.
THE DEFAULT PATH (proven by Rosetta Stones): DocType → BOM walk → C_OrderLines → elements → output.db
Once the default path is proven, everything else rides on it:
-
EDITOR RIDES THE DEFAULT PATH The Bonsai editor doesn't start from zero. It starts from a known-good compilation and the user makes MACRO-LEVEL changes: "Move this wall 500mm" → re-resolve affected BOM entries "Add a bedroom" → insert room ESLine → BOM fills it "Change to timber frame" → swap structural BOM → re-compile The user never touches individual elements. They operate at the block level — rooms, assemblies, systems. The compiler resolves everything underneath because it already knows how.
-
2D ROUND-TRIP PROVES THE EDITOR'S INPUT LANGUAGE The 2D export (Phase C) + 2D parser (Phase D) prove that sketch-level input — floor plans and elevations — is sufficient to drive full 3D compilation. That's exactly what the editor needs: the architect draws rooms and walls, the compiler does the rest.
-
BOM DICTIONARY GROWS WITH EACH STONE Every new building type that compiles clean adds its resolutions to the dictionary. EN-BLOC singularity means a whole proven arrangement becomes a single lookup. The compiler gets FASTER and MORE CAPABLE with each stone, not just more verified.
WHY THE 2D ROUND-TRIP WON'T DRIFT: The BOM is the anchor. The 2D drawing doesn't carry material, product dimensions, or assembly structure — the BOM does. The 2D drawing is a SPATIAL INDEX into the same BOM gospel. SH's floor plan says "wall here, door there, kitchen there" — the BOM already knows which wall product, which door product, which kitchen assembly. The 2D is ADDRESSING, not SPECIFYING. Same separation of concerns already built: WHAT (BOM) + WHERE (layout) = deterministic output.
THE SEQUENCE: PROVE (Rosetta Stones) → REPRODUCE (2D round-trip) → EDIT (Bonsai GUI) → ACCUMULATE (BOM library grows) = "Once built, now anything"