seq2seq
an archive of posts with this tag
-
The decoder's remaining problem - output context still accumulates
Attention solved the encoder-side information bottleneck. But the decoder itself is still a sequential structure that depends on its previous state. Here is why that becomes the final motivation for the Transformer.
-
Attention Seq2Seq - don't just look at the last state, look directly
How do you fix the vanilla Seq2Seq bottleneck? By letting the decoder reference every encoder hidden state directly, "as needed".
-
Vanilla Seq2Seq - the architecture that leaned on one final state
To understand the basic Seq2Seq architecture you first have to accept one fact: the input is compressed into a single vector. How everything ends up in the last hidden state, and the fundamental limits that follow.