StandardInternational professional-learning synthesis · ABET 2026–27 · IEA GAPC v4 · ASCE ethics and AI responsibility · NIST AI RMF 1.0
Source3 source files
Review stateTechnical and publication gates pending
After this chapter, you should be able to
- Create a dataset and provenance card.
- Check units, schema, missingness and impossible values.
- Freeze versions with checksums and licences.
- Build deterministic preprocessing without target leakage.
- Communicate data limitations to an engineering reviewer.
Engineering context and methodSource §Lesson 02 · Engineering context and method · GOV-01 · RES-11 · SW-01 · DATA-01 · DATA-05
Civil data inherit instrument, sampling, maintenance, survey, laboratory and administrative processes. A clean table is not automatically fit for a decision. Record source URL or custodian, release and retrieval date, licence, checksum, schema, units, coordinate reference system, time zone, identifiers, missing-value codes, exclusions and transformations. Freeze raw data read-only; code should produce derived tables and a machine-readable validation report.
Verified worked exampleSource §Lesson 02 · Verified worked example · GOV-01 · RES-11 · SW-01 · DATA-01 · DATA-05
Unit harmonisation with an audit trail
A beam depth is recorded as 0.45 m while the controlled schema requires millimetres.
- Identity
1 m = 1000 mm
conversion factor = 1000 - Conversion
0.45 × 1000
450 mm - Audit
retain originalvalue, originalunit, ruleid
0.45 m → 450 mm
Result. 450 mm, with the original value and conversion rule retained. Never infer an unknown unit from magnitude alone.
Practical lab · 5 h lesson effortSource §Lesson 02 · Practical lab · 5 h lesson effort · GOV-01 · RES-11 · SW-01 · DATA-01 · DATA-05
- Ingest a frozen concrete or synthetic asset-register table without overwriting the raw file.
- Produce schema, data types, units, missingness, duplicates, ranges and impossible-value checks.
- Create a dataset card with release, retrieval date, licence, checksum and exclusions.
- Run a deterministic transformation twice and confirm identical row count, columns and checksum.
Failure modes to investigateSource §Lesson 02 · Failure modes to investigate · GOV-01 · RES-11 · SW-01 · DATA-01 · DATA-05
- Silent unit conversion or mixed unit systems.
- Missing-value code treated as a physical zero.
- Overwriting the source file during cleaning.
- Random row deletion without an exclusion log.
- Dataset metadata mistaken for decision fitness.
Knowledge checksSource §Lesson 02 · Knowledge checks · GOV-01 · RES-11 · SW-01 · DATA-01 · DATA-05
| Question | Answer rationale |
|---|---|
| Why retain a checksum? | It identifies the exact frozen bytes used by the analysis. |
| Can magnitude prove a unit? | No; unit must come from controlled metadata or the data custodian. |
| What belongs in a dataset card? | Provenance, version, licence, schema, units, population, quality, exclusions and intended limits. |
| Why keep raw data read-only? | It preserves auditability and allows transformations to be reproduced. |
| Does public access mean unrestricted reuse? | No; verify the dataset or product licence and any sensitive-data conditions. |
Key points
- Start from the accountable engineering decision and its consequence.
- Compare against a transparent non-AI baseline.
- Validate on a split that represents intended use and retain human authority.
Source references recorded by the supplied chapter
- UCI Machine Learning Repository, Concrete Compressive Strength dataset, DOI 10.24432/C5PK67, CC BY 4.0.
- USGS 3D Elevation Program product and metadata guidance.
- scikit-learn, Common pitfalls and recommended practices.