Weeks 6–7 · Phase 1 Foundations
The Data Fundamentals Lab
Models are only as good as the data flowing into them. Classify it, trace its lifecycle, score its quality, and protect the people inside it.
Structured, semi-structured, or unstructured?
Every dataset you'll ever meet lands in one of three buckets — and the bucket decides how much work stands between it and a model. Sort these nine.
The data lifecycle
From source to model, data passes through five gates. Click each stage.
The quality scorecard
Your Week 7 deliverable is exactly this: score a dataset on five dimensions and decide if it's fit for AI. Drag the sliders and watch the verdict.
PII anonymisation
India's DPDP Act 2023 mandates explicit consent for personal data use. Before this record goes anywhere near a training set or a prompt, the PII goes.
Data for AI, specifically
Six concepts that separate “data work” from “data work for AI”.
Human-in-loop, auto-label, weak supervision. Quality > quantity — 1K clean labels beats 10K noisy ones.
80/10/10, and never let them leak into each other. Temporal splits for time-series; stratified for imbalance.
SMOTE, oversampling, class weights. Imbalanced data → models that ignore the minority class.
Fixed, semantic, or sentence-level. Chunk size affects RAG retrieval quality dramatically — Week 8 proves it.
Dense vectors (OpenAI, Cohere, BGE). The embedding model matters as much as the LLM.
Domain signals, encoding, normalisation. Classic ML still needs hand-crafted features.
Governance & compliance essentials
PII handling
Identify, mask, anonymise, pseudonymise. DPDP Act 2023 (India) mandates explicit consent for PII use.
Regulation landscape
DPDP Act (India), GDPR (EU), CCPA (US). Know which laws apply to your data — ignorance is no defence.
Consent & purpose
Data may be used only for its stated purpose. Build consent tracking into pipelines from day one.
Lineage & audit
Track origin, transformations, and model usage. Required for regulatory audits and for debugging AI decisions.
Data minimisation
Collect only what you need. Less data = smaller attack surface and lower compliance burden.