Scoping ML Projects

• Judge whether a problem is a good ML fit before any training
• Translate a business metric into an ML objective
• Define a baseline and a fail-fast threshold

• The majority of failed ML projects fail before any code — because the problem was unanswerable or the metric was the wrong one.
• A clear baseline (heuristic, rules, 'always predict yesterday') sets the bar a model must clear.
• Scoping forces the labeling question — who provides ground truth and at what cost — early.
• It defines the deployment shape: batch vs real-time, which directly drives the lifecycle choices.

You ask who the customers are, what they'll pay, whether the kitchen can even cook it, and how you'll know it sold well. Scoping is the menu decision before the chefs ever start cooking.

• Write the consumer and the decision: 'risk team uses this prediction to approve or reject a loan application.'
• Define the business metric affected (default rate, revenue) and the ML metric that proxies it (precision, AUC).
• Check data availability and label quality; if ground truth doesn't exist yet, the project is a labeling project first.
• Set a baseline — a simple heuristic — that a model must demonstrably beat.
• Agree on a fail-fast milestone: the point where you kill the project if targets aren't met.

• A logistics company scoped 'predict delay' after confirming historic trip features exist; the baseline 'same truck as last week' was beaten by 12% before any deep model was tried.
• An insurance team killed a claims-fraud project at the fail-fast gate: labels were too noisy and manual review was already 94% precise.
• A bank scoped underwriting around a constrained decision (approve/reject at a fixed cutoff), making monitoring and audit immediately tractable.

• Write the one-page scope doc: decision, consumer, metric, baseline, data need, kill point.
• Prefer a metric the business already trusts over a clever ML one.
• Plan for the cost of labels and mistakes up front — false positives and false negatives have asymmetric prices.
• Scope the smallest thing that delivers value; grow scope only after proving it.

• Picking 'accuracy' when the real cost is a specific error type (false negatives).
• Scoping around the model ('let's build a transformer') instead of the decision.
• Assuming labels exist when they don't; labeling effort becomes the hidden schedule.
• No baseline, so the 'winning' model is never compared to the trivial answer.

• If stakeholders argue about what the model should do, you haven't written the decision down.
• Re-read the success metric after the first data look — it often reveals the metric is unmeasurable.
• Keep a glossary of business terms; a wrong meaning quietly corrupts labels.