nlp

ROUGE

Recall-Oriented Understudy for Gisting Evaluation

rouge  nlp  evaluation

deep-learning

2026-04-01   ·   8 min read

How to teach your embedding model new words

A step-by-step guide on adding new vocabulary to a Hugging Face sentence-transformer model, including fine-tuning and using it in LangChain.

embeddings  fine-tuning  nlp

deep-learning

2025-10-25   ·   8 min read

From Seq2Seq to Transformer: from accumulated context to direct reference · Part 3

The decoder's remaining problem - output context still accumulates

Attention solved the encoder-side information bottleneck. But the decoder itself is still a sequential structure that depends on its previous state. Here is why that becomes the final motivation for the Transformer.

attention  seq2seq  bahdanau-attention  query-key-value  nlp

deep-learning

2023-06-28   ·   3 min read

From Seq2Seq to Transformer: from accumulated context to direct reference · Part 2

Attention Seq2Seq - don't just look at the last state, look directly

How do you fix the vanilla Seq2Seq bottleneck? By letting the decoder reference every encoder hidden state directly, "as needed".

attention  seq2seq  bahdanau-attention  query-key-value  nlp

deep-learning

2023-06-02   ·   4 min read

From Seq2Seq to Transformer: from accumulated context to direct reference · Part 1

Vanilla Seq2Seq - the architecture that leaned on one final state

To understand the basic Seq2Seq architecture you first have to accept one fact: the input is compressed into a single vector. How everything ends up in the last hidden state, and the fundamental limits that follow.

seq2seq  rnn  encoder-decoder  attention  nlp

deep-learning

2023-05-25   ·   3 min read