Course materials · AI Applications

Text Classification and Sentiment Analysis

English translation

Create your own copy to run, edit, and record results.

Self-study material

1. Tokens and vocabulary

Convert sentences to Tokens and Indices, then average Embedding vectors to classify overall sentiment. This exercise uses short English Reviews and a trainable Mean Embedding model to inspect preprocessing results and sentence vectors directly. As in the lecture's IMDB example, assign Labels to entire sentences, here using three sentiments. Mean Pooling is a simple model that omits word order, not a required replacement for RNNs. No TorchText installation, Hugging Face login, or external model download is needed. PAD=0 marks empty positions; UNK=1 marks unknown words. Check whether short sentences such as screen size okay receive PAD, and use sentences of different lengths in the comparison cell below as well.

A. Baseline example · Read and run first

import sys, numpy as np, matplotlib.pyplot as plt
import torch
from torch import nn
import torch.nn.functional as F
torch.manual_seed(42)
torch.set_num_threads(2)
print('Python:', sys.version.split()[0], '| PyTorch:', torch.__version__)
texts=['great battery bright screen','bad camera slow app','screen size okay','love fast performance','broken charger bad battery','average design normal speed','great sound love display','slow response hate keyboard','okay price average camera','bright screen fast app','bad screen okay battery','normal package average manual']
y=torch.tensor([2,0,1,2,0,1,2,0,1,2,0,1]);names=['negative','neutral','positive']
words=sorted(set(' '.join(texts).split()));vocab={'<PAD>':0,'<UNK>':1,**{w:i+2 for i,w in enumerate(words)}}
def encode(text_list):
    rows=[[vocab.get(w,1) for w in text.lower().split()] or [1] for text in text_list]
    length=max(map(len,rows));return torch.tensor([r+[0]*(length-len(r)) for r in rows])
X=encode(texts);print(texts[:3],X[:3],X.shape)


# Check where PAD is inserted when lengths differ.
probe_texts = ['great battery', 'great battery bright screen', 'excellent battery']
probe_ids = encode(probe_texts)
for text, row in zip(probe_texts, probe_ids):
    print('text:', text, '| tokens:', text.lower().split(), '| ids:', row.tolist())
assert (probe_ids[0] == 0).sum().item() == 2
assert probe_ids[2, 0].item() == vocab['<UNK>']

B. Hands-on activities

  1. Print two English sentences of different lengths in the order Token → Index → PAD.
  2. Add one word absent from the training vocabulary and locate UNK.
  3. Distinguish Indices for actual words from PAD and UNK.

Your prediction may describe a shape or an increasing/decreasing trend rather than an exact number. Record explanations in the Text Cell below.

# ✍ Write your own: add your code below this cell.
# You may add Code Cells if the code gets long.

C. Your record · Double-click this Text Cell to edit

2. Masked averaging and training

Weeks 12 and 13 predicted the next character at each position; here, predict one sentiment per sentence. From Embedding (N,S,16), compute the mean excluding PAD, (N,16), then use Linear to obtain (N,3) Logits. Embedding and Linear are trained together; this model has no RNN/LSTM.

A. Baseline example · Read and run first

Output captions: PAD 포함 평균과 차이 = maximum difference with PAD included; PAD 제외 평균과 차이 = maximum difference with PAD excluded.

class TextClassifier(nn.Module):
    def __init__(self):
        super().__init__();self.embedding=nn.Embedding(len(vocab),16,padding_idx=0);self.fc=nn.Linear(16,3)
    def forward(self,ids):
        mask=(ids!=0).unsqueeze(-1);embedded=self.embedding(ids)
        pooled=(embedded*mask).sum(1)/mask.sum(1).clamp_min(1)
        return self.fc(pooled)
model=TextClassifier();opt=torch.optim.Adam(model.parameters(),lr=.02);losses=[];accuracies=[]
for epoch in range(200):
    logits=model(X);loss=F.cross_entropy(logits,y)
    opt.zero_grad();loss.backward();opt.step()
    losses.append(loss.item());accuracies.append((logits.argmax(1)==y).float().mean().item())
print('Training accuracy:',accuracies[-1]);print('embedding:',model.embedding(X).shape)
fig,axes=plt.subplots(1,2,figsize=(9,3));axes[0].plot(losses);axes[1].plot(accuracies)
for ax,yl in zip(axes,['Loss','Accuracy']):ax.set(xlabel='Epoch',ylabel=yl);ax.grid()
plt.show()


# The correct mean is the same whether a sentence is processed alone or batched with a longer sentence.
with torch.no_grad():
    single_ids = encode(['great battery'])
    batch_ids = encode(['great battery', 'great battery bright screen'])
    single_vector = model.embedding(single_ids).mean(1)[0]
    batch_embedded = model.embedding(batch_ids)
    mask = (batch_ids != 0).unsqueeze(-1)
    masked_vectors = (batch_embedded * mask).sum(1) / mask.sum(1).clamp_min(1)
    plain_vectors = batch_embedded.mean(1)
    print('PAD 포함 평균과 차이:', (single_vector - plain_vectors[0]).abs().max().item())
    print('PAD 제외 평균과 차이:', (single_vector - masked_vectors[0]).abs().max().item())
    assert torch.allclose(single_vector, masked_vectors[0], atol=1e-6)
    order_logits = model(encode(['great battery', 'battery great']))
    print('단어 순서를 바꾼 두 문장의 Logit:', order_logits)
    assert torch.allclose(order_logits[0], order_logits[1], atol=1e-6)

B. Hands-on activities

  1. Run the mean comparison cell for the two provided sentences. Relative to the Vector from processing the short sentence alone, record the maximum absolute difference for the Vector processed with the long sentence, before and after applying the Mask.
  2. In the comparison cell, change the sentence in single_ids and the first sentence in batch_ids to the same short sentence. Change the second sentence in batch_ids to a longer sentence and repeat the calculation. Record the sentences used and both differences. Keep the trained Embedding weights.
  3. Write the Logits for two sentences with reordered words side by side and compute their maximum absolute difference. Using the Token count, PAD count, and printed maximum difference, explain separately the effect of including PAD in the average and the effect of word order under mean Pooling.

Your prediction may describe a shape or an increasing/decreasing trend rather than an exact number. Record explanations in the Text Cell below.

# ✍ Write your own: add your code below this cell.
# You may add Code Cells if the code gets long.

C. Your record · Double-click this Text Cell to edit

3. Enter new sentences

Replace the three empty strings in my_reviews below with English sentences. First use words from texts, then add one unknown word. Even if the predicted sentiment differs from your intention, record the result and explain why. Predictions are skipped for unfinished entries.

A. Baseline example · Read and run first

If the cell reports that my_reviews is unfinished, fill its three empty strings with your own reviews and rerun the cell.

my_reviews=['','',''] # Enter your own: 1 positive, 1 neutral, 1 negative
if all(s.strip() for s in my_reviews):
    model.eval()
    with torch.no_grad():prob=model(encode(my_reviews)).softmax(1)
    for text,p in zip(my_reviews,prob):print(text,'=>',names[p.argmax().item()],p.tolist())
else:
    print('my_reviews의 빈 문자열 3개를 직접 작성하고 이 셀을 다시 실행하세요.')
print('unknown token example:',encode(['excellent battery']))

B. Hands-on activities

  1. In my_reviews, write one positive, one neutral, and one negative English sentence using words from the training vocabulary.
  2. Record the intended sentiment, prediction, and Tokens, and add one unknown word or punctuation mark to check UNK handling.
  3. If the prediction differs from your intention, explain the cause.

Your prediction may describe a shape or an increasing/decreasing trend rather than an exact number. Record explanations in the Text Cell below.

# ✍ Write your own: add your code below this cell.
# You may add Code Cells if the code gets long.

C. Your record · Double-click this Text Cell to edit

Practice through repetition · Reason before running

Summarize the three conditions compared in the preceding activities in this table. You do not need to repeat experiments already completed. If comparisons are insufficient, supplement them by changing just one condition.

Changed condition Predicted shape/value/result Actual result Cause of any error
⟦Enter your response 1⟧ ⟦Enter your response⟧ ⟦Enter your response⟧ ⟦Enter your response⟧
⟦Enter your response 2⟧ ⟦Enter your response⟧ ⟦Enter your response⟧ ⟦Enter your response⟧
⟦Enter your response 3⟧ ⟦Enter your response⟧ ⟦Enter your response⟧ ⟦Enter your response⟧ 
# ✍ Code for repeated experiments

Optional exercise · Find the cause of an error and fix the code

When Token Indices are passed to Embedding as a Float Tensor

  1. After training the sentence classification model, add a Code Cell at the bottom of the notebook and run the following code.

    for debug_input in [X.float(), X]:
    print("Token Index:", debug_input.shape, debug_input.dtype)
    try:
        with torch.no_grad():
            print("Output:", model(debug_input).shape)
    except (RuntimeError, ValueError) as error:
        print(type(error).__name__, str(error))

  2. In a Text Cell, tabulate each input's dtype and execution success. Explain the error in terms of Embedding inputs being Vocabulary row indices rather than continuous word measurements. Even after calling X.float(), the original X remains integer-valued.

Check before saving

  1. Check the response fields, outputs, and plots, then run all cells in order in a fresh runtime.
  2. Save to your Drive, then download using .ipynb.

Include code, actual outputs and plots, and your own interpretation. Submit through KLAS; check the semester schedule and latest announcements for the deadline.

Filename: Name_StudentID_Week14.ipynb.