Worksheet: Sentiment Analysis

Background

You are annotating product reviews for sentiment analysis. The goal is to train a model that understands customer opinions.

Sentiment refers to the attitude, opinion, or feeling expressed in text toward a target (product, service, person, etc.).

Label Options

Basic scheme: Positive, Negative, Neutral

Extended scheme: Very Negative (-2), Negative (-1), Neutral (0), Positive (+1), Very Positive (+2)

Sentiment seems simple, but involves difficult decisions about:

Mixed sentiments within the same text
Sarcasm and irony
Implicit vs. explicit sentiment
Aspect-level vs. document-level sentiment

Part 1: Basic Sentiment (Warm-up)

Review 1

"This laptop is amazing! Fast, lightweight, and the battery lasts forever."

Very Negative

Negative

Neutral

Positive

Very Positive

Review 2

"The product arrived on time."

Very Negative

Negative

Neutral

Positive

Very Positive

Question 1

Is "arrived on time" neutral (just stating a fact) or slightly positive (meeting expectations)?

Part 2: Mixed Sentiment

Review 3

"The camera quality is outstanding, but the battery life is disappointing. For the price, I expected more."

Very Negative

Negative

Neutral

Positive

Very Positive

Question 2

How should mixed sentiment be handled?

Average the sentiments (neutral) Use the dominant/final sentiment Add a "Mixed" label Annotate sentiment per aspect separately

What are the tradeoffs?

Part 3: Sarcasm and Irony

Review 4

"Oh great, another phone that dies by noon. Exactly what I needed."

Very Negative

Negative

Neutral

Positive

Very Positive

Review 5

"Five stars for being consistently inconsistent. The quality varies so much, it's like a surprise every time!"

Very Negative

Negative

Neutral

Positive

Very Positive

Question 3

Should annotation capture the literal meaning or the intended meaning?

Literal (surface) meaning Intended (pragmatic) meaning Both, with separate labels

How would you handle sarcasm in guidelines?

Part 4: Implicit Sentiment

Review 6

"I've already ordered two more for my family members."

Very Negative

Negative

Neutral

Positive

Very Positive

Review 7

"I returned it the same day."

Very Negative

Negative

Neutral

Positive

Very Positive

Question 4

Neither review contains explicit sentiment words. Should implicit sentiment be annotated?

Yes, based on implied meaning No, only explicit sentiment Yes, but mark as "implicit"

What world knowledge is needed to interpret these reviews?

Part 5: Aspect-Based Sentiment

Review 8

"The food was delicious but the service was terrible. The atmosphere was nice though, and prices are reasonable."

Question 5

Annotate sentiment for each aspect:

Aspect	Sentiment
Food quality
Service
Atmosphere
Price
Overall

How did you determine the overall sentiment?

Part 6: Comparative Sentiment

Review 9

"This is better than my old Samsung, but still not as good as the iPhone."

Question 6

What is the sentiment toward each product?

How should comparative sentiment be handled in guidelines?

Part 7: Group Discussion

Question 7

Compare your annotations with your group. Where did you disagree?

Scale position (e.g., +1 vs +2) Sarcasm interpretation Mixed sentiment handling Implicit sentiment inference Neutral vs. slightly positive/negative

Describe one specific disagreement:

Part 8: Reflection

Question 8

Why is sentiment analysis harder than it appears?

Sentiment is inherently subjective Context changes interpretation Much sentiment is implicit Scale granularity is arbitrary Real opinions are often mixed

Key Takeaway

Sentiment is not a property of text—it's an interpretation that depends on context, culture, and task.

The same text can have different sentiments for different targets
Annotation guidelines encode assumptions about what sentiment means
Simple scales hide complex, multi-dimensional opinions