PyTorch implementation of Transformer encoder: Transformer encoder for Classification Tasks.
just for learning or research purpose
Transformer model is a neural network that learns context and thus meaning by tracking relationships in sequential data like the words in this sentence, which means Transformer models are good at capturing content-based global interactions.Transformer models apply an evolving set of mathematical techniques, called attention or self-attention, to detect subtle ways even distant data elements in a series influence and depend on each other.
In this repository just transformer encoder was implemented, for more detail further check the paper Attention Is All You Need
- Transformer encoder
- Implemented on Pytorch (version=1.11.0)