.learn() - page 6
what I have learned so far
-
The decoder's remaining problem - output context still accumulates
Attention solved the encoder-side information bottleneck. But the decoder itself is still a sequential structure that depends on its previous state. Here is why that becomes the final motivation for the Transformer.
-
Attention Seq2Seq - don't just look at the last state, look directly
How do you fix the vanilla Seq2Seq bottleneck? By letting the decoder reference every encoder hidden state directly, "as needed".
-
Vanilla Seq2Seq - the architecture that leaned on one final state
To understand the basic Seq2Seq architecture you first have to accept one fact: the input is compressed into a single vector. How everything ends up in the last hidden state, and the fundamental limits that follow.
-
Putting it together - what happens when you type a URL
From pressing Enter to the page appearing, every layer in the series is called on within a second. The final part traces all five as a single flow.
-
The application layer (DNS/HTTP) - look up a name, then talk
People use names, not IP addresses. DNS, the distributed system that turns names into addresses, and HTTP, the language the web speaks over them.