Sequence-to-Sequence Models

Summary

Encoder-decoder architectures that map one sequence to another (e.g. translation) — the architectural predecessor to the Transformer and still conceptually relevant.

How to Learn This

1Learn the encoder-decoder pattern at a conceptual level with a translation example.
2Understand the information bottleneck problem that motivated attention.
3Trace how seq2seq models evolved into attention-based and then Transformer models.

LSTMs & GRUs

Attention Mechanism

More topics in Deep Learning Architectures

Convolutional Neural Networks (CNNs)Pooling Layers Recurrent Neural Networks (RNNs)LSTMs & GRUs Attention Mechanism Transformer Architecture Positional Encoding Transfer Learning Autoencoders Generative Adversarial Networks (GANs, Overview)

Stuck on this topic? Ask an Insider

Get 1:1 guidance from people who've walked this exact path — free on the InsideEdge app.

Download