Sentiment Analysis
Reading opinion reliably enough to act on it
Sentiment models read opinion: polarity, intensity and aspect. The hard parts aren't the classifier — they're negations, sarcasm, aspects and calibration across domains.
What you will learn
- Distinguish doc-level, sentence-level and aspect-based sentiment
- Handle negations and domain-specific polarity
- Evaluate sentiment with the right granularity
What is this?
Sentiment analysis assigns opinion polarity (positive/neutral/negative, sometimes intensity) to text — a review, tweet, chat turn, or an aspect within it ('screen is good, battery is awful'). Approaches range from lexicon scoring (VADER, domain dictionaries) to ML: TF-IDF/embedding classifiers and transformers fine-tuned on opinion datasets. The real challenge is linguistic: negation ('not bad'), sarcasm, context-dependence ('literally insane' can be praise) and aspect-reversal.
Why do you need it?
- Opinion data is the highest-signal, lowest-cost feedback companies get.
- Triage: routing angry tickets to priority queues needs intensity detection.
- Aspect sentiment drives product decisions ('bench is great, camera mediocre').
- Trended sentiment is an early-warning signal for churn and PR firestorms.
Mental model
Reading a customer's face in a live chat
Sentiment model
A face reads more than good/bad — it distinguishes frustration from courtesy nods and knows the annoyance peaks around billing specifically (aspect). Sentiment models try to capture that same layered reading from text alone.
How it works
- Define the unit: document, sentence, or aspect-value pair?
- Lexicon route: aggregate per-token polarity with negation flipping and intensity modifiers.
- ML route: features (TF-IDF/embeddings, negation-transformed) → classifier with 3–5 classes.
- Transformer route: fine-tune on opinions; handle long docs with segmenting/aggregation.
- Evaluate: macro-F1, per-class confusion, and calibration if you promise confidence.
Sentiment pipeline with negation handling.
In production
- A support triage sorts tickets by negative intensity, fast-tracking strongly negative ones.
- A product team monitors aspect-level sentiment per release note to flag regressions early.
- A marketplace filters and surfaces 'positive for seller reliability' signals in reviews.
Best practices
- Choose granularity to the decision: triage wants doc/sentence; product wants aspect.
- Handle negation explicitly — the single biggest wins beyond basic polarity scoring.
- Retrain/rebuild lexicons per domain; 'crash' is neutral in dev logs, negative in reviews.
- Report calibration alongside accuracy if downstream sorts by 'most negative first'.
Common mistakes
- Evaluating on model-preferred reviews length and missing the 'short angry' tail.
- Ignoring sarcasm entirely — a classic failing on reviews ('Great, ANOTHER bug fixed...').
- Aggregating aspect sentiments into one number and losing the product signal.
- Deploying a model trained on movie reviews to a banking chat corpus.
Common issues
Tweets full of sarcasm always scored wrongSarcasm inverts polarity, and no single-word signal survives
Fine-tune (or few-shot instruct a language model) on a sarcasm-inclusive set; add emoji/pattern features.
Positivity collapses in technical languageDomain terms ('patched', 'fixed') read negative to a generic model
White-list domain-positive patterns or retrain on domain data.
Debugging tips
- Leak-negation test: append 'not ' to 100 test sentences and verify polarity flips where it should.
- Print the top negative examples — reverse polarity usually obvious once you read them.
- Slice by length and emoji usage; emoji-heavy text comporting text differs from formal complaints.
- Use a confidence threshold to send ambiguous texts to a human queue rather than misclassifying.
Interview checkpoint
Quick quiz
The expression 'not terrible' — a naive lexicon likely scores it:
A sentiment model trained on movie reviews transfers cleanly to banking chats.