Data management: imports, duplicates, and cleaning up after someone else

Order of operations, external ids, and upsert. Most import disasters are a relationship loaded before its parent.

1. VideoImport Wizard, Data Loader, or a query tool: choosing by the job6 min
2. VideoExternal ids and upsert, and why record ids are the wrong keyA record id is created by the target org. An external id comes from the source system, so a re-run updates instead of duplicating.Preview7 min
3. ReadingTurning automation off during a load, and the risk of doing so6 min
4. VideoBulk API versus REST, and the batch size that actually matters6 min
5. VideoField mapping, and the column that silently did not load7 min
6. LabLab: load 2,000 accounts and 8,000 contacts, twice, idempotently6 min

A parent-child dataset has exactly one order that works. Everything else fails partway through, usually quietly.

1. VideoLoading a parent-child dataset in the right order7 min
2. VideoSelf-relationships and the load that has to run twice6 min
3. ReadingComposite requests, and loading two objects in one call7 min
4. VideoRollback behaviour, and what a partial failure actually leaves behind6 min
5. VideoLoad logs: what to keep, and for how long7 min
6. LabLab: load a five-object dataset with two circular-looking relationships6 min

Matching rules, duplicate rules and merge. Tuned so they catch the duplicate and do not block the salesperson at 4pm on a Friday.

1. VideoMatching rules: fuzzy, exact, and the fields that actually discriminate6 min
2. VideoDuplicate rules: alert, block, and the report nobody reads7 min
3. ReadingTuning a matching rule so it stops blocking real work6 min
4. VideoMerging records, and what merge does to related lists7 min
5. VideoBulk deduplication, and the merge you cannot undo6 min
6. LabLab: tune a duplicate rule against a seeded dataset7 min

Soft delete, recycle bin, hard delete, and what each one does to a rollup someone downstream is trusting.

1. VideoDeleting safely: soft delete, recycle bin, hard delete, and rollups6 min
2. VideoCascade delete, and the master-detail child you did not mean to lose6 min
3. ReadingMass delete, and the permission that should not be given lightly7 min
4. VideoArchiving instead of deleting, and when that is the honest choice6 min
5. LabLab: delete a batch of records and verify every rollup afterward7 min
6. VideoData retention, and the policy you write down before you need it6 min

A reported lifetime invoicing figure was inflated by phantom data inherited from a migration. This module is where that forensic pass starts.

1. VideoThe reported total that did not reconcileThe headline number came from a rollup. The line items behind it did not sum to it. That gap is where every forensic pass starts.7 min
2. Video31 percent of quote lines carrying no tax code, found by querying rather than asking6 min
3. ReadingWhere a headline number actually comes from, and why that matters6 min
4. VideoA reported lifetime invoicing total, and the gap that did not add up7 min
5. VideoIsolating inherited records: created date, owner, and the load signature6 min
6. LabLab: find the gap between a headline and its own line items7 min

Three-way reconciliation across the old system, the new one and the accounting ledger. 97 percent keyed, and the unkeyed residue itemised rather than hidden.

1. VideoReconciling a total against its own line itemsThree-way reconciliation across the old system, the new one and the accounting ledger. 97% keyed, and the unkeyed residue itemised rather than hidden.6 min
2. VideoTwo mistake invoices, and how they were found7 min
3. ReadingThe corrected figure, across 2,860 invoices, and why the count matters too6 min
4. VideoThree-way reconciliation across two retired systems and a live one7 min
5. VideoThe residue you itemise instead of hiding: the residue, named6 min
6. LabLab: reconcile a total across three systems and itemise what did not key7 min

A clean-up is not finished when the number is fixed. It is finished when someone else can read why it changed.

1. VideoWrite a data dictionary an auditor can read7 min
2. VideoThe proof step: sweeping the table after a fix and reporting zero hits6 min
3. ReadingWhat a clean-up report says to the person who has to sign off on it7 min
4. LabLab: find and quantify the phantom data in a seeded orgThe capstone lab. Reconcile a seeded lifetime invoicing figure against its line items and isolate every inherited record.28 min
5. VideoHanding a corrected number to someone who has to defend it6 min
6. QuizCheck: data management14 min