Can the classification methods learned earlier also select the next character?
Description
Computing scores for candidate characters and comparing them with the target character is a classification task.
An RNN carries previous information through Hidden State for prediction. In this exercise, inspect each position's Input, Target, and prediction side by side.
Practice
1. Construct a character Batch
Run the data cell and print the Input and Target characters in two rows for each of the three sentences. Check that each Target character is immediately after the Input character in the same column in the original text.
Print X.shape and Y.shape. Label each axis with the appropriate meaning: sentence count, character count, or Vocabulary size.
Print the Vocabulary's character list and record the positions containing spaces and punctuation.
2. Train an RNN
Compare before_pred[0] printed by the training cell and post-training pred[0] character by character against Y[0]. Record incorrect positions before and after training separately.
Feed X[:, :4, :], X[:, :6, :], and X[:, :8, :] separately to the trained model. Run within torch.no_grad() without retraining.
Record input Sequence length, full input shape, and full output shape in a three-row table. Mark the output axis that changes with length.
Submission and saving
Include code, actual outputs and plots, and your own interpretation. Submit through KLAS; check the semester schedule and latest announcements for the deadline.
Filename: Name_StudentID_Week12.ipynb.
Restart the runtime, run all code from the top, then save.
Optional practice
When Batch and Sequence axes are swapped
After training the RNN with three sentences, add a Code Cell at the bottom of the notebook and run the following code.
for debug_input in [X, X.transpose(0, 1)]:
with torch.no_grad():
debug_output = model(debug_input)
print("Input:", debug_input.shape, "Output:", debug_output.shape,
"Target:", Y.shape)
In a Text Cell, record the Batch size and Sequence length interpreted by the model in each call. With batch_first=True, the first axis is Batch. Explain which input treats each of the three sentences as one Sequence, and why the other call also runs without an error.