rnn

From Seq2Seq to Transformer: from accumulated context to direct reference · Part 1

Vanilla Seq2Seq - the architecture that leaned on one final state

To understand the basic Seq2Seq architecture you first have to accept one fact: the input is compressed into a single vector. How everything ends up in the last hidden state, and the fundamental limits that follow.

seq2seq  rnn  encoder-decoder  attention  nlp

deep-learning

2023-05-25   ·   3 min read