Transformer Architecture: Components, Working, and Types

Quick Overview: 

• The Transformer architecture was introduced in June 2017 in the research paper "Attention Is All You Need" by researchers at Google Brain.

• It helps the model to process the inputs based on the relationships between tokens by employing attention mechanisms and is an efficient way of doing many tasks that have been the backbone of most AI applications like LLMs and multimodal AI.

• Three popular variations are encoder-only models like BERT, decoder-only models like GPT-style models, and encoder-decoder models like T5.

• In this blog you will learn transformer architecture in detail along with its components, working, types and challenges in the architecture.

Explore how transformer architecture powers modern LLMs and AI applications. The Executive Post Graduate Certificate in Building AI Products, Systems & Services from IIT Kharagpur helps you develop practical skills to design, build, and deploy AI-powered solutions.

Transformer Architecture Overview

The Transformer Architecture in AI is a deep learning model which was introduced by Ashish Vaswani and seven co-authors in 2017 in their research paper Attention Is All You Need and currently powers models like OpenAI's GPT, Meta's Llama, and Google's Gemini. The research paper introduced a new way to process the sequential data using the attention mechanisms rather than relying on recurrent neural networks (RNNs).

A transformer first converts words into numbers which computers can understand which are called embeddings. It then uses, attention mechanism to identify relationships between those words and understand how they are related to each other.

To understand it more clearly let’s take a look at transformer architecture example:

“The dog chased the ball because it was excited.”

To understand this whole sentence, a language model need to understand that what the word “it” means and belong to. The attention mechanism helps the LLM to connect “it” with “the dog” by understanding the relationship between the sentences.

Also Read: Reranking in RAG: How It Works, Types, Models, and Tools

The Shift from Sequential to Parallel Processing

Before the transformers, recurrent neural networks (RNNs) and long short-term memory networks (LSTMs) were commonly used for processing the sequential data.

These models process sequences step by step. The output at one step can influence the processing at the next step. This makes it harder to parallelise the main sequence computation during training.

These models only process sequence step by step. Below are the key differences between both RNNs, LSTMs and Transformers. 

Transformer Architecture Diagram

Before going into more details of transformer architecture in LLM, let’s take a look at the simplified view of the architecture below:

Also Read: What Is Semantic Search? How It Works, Benefits & Examples

Key Components of Transformer Architecture

The original Transformer architecture, introduced in 2017, used six encoder layers, six decoder layers, and eight attention heads in its base configuration.

The key components used in transformers are:

1. Input Embeddings

Input embeddings convert words or tokens into numerical representations that a computer can understand. These numbers help the Transformer process text and identify relationships between words.

For example, the sentence “I love machine learning” is split into tokens. The model converts each token into a list of numbers that represents information about it.

2. Positional Encoding

Positional encoding helps the Transformer to understand the order of words in a sentence. This matter because changing the word order can also change meaning.

For example:

• “The cat chased the dog.”

• “The dog chased the cat.”

Although‍‍‍ the same words appear in both sentences, the actions involved are different. Through positional encoding the model gets to know the positions of tokens in the sentence.

3. Self-Attention Mechanism

The self attention mechanism help the transformers to identify the relationship between word and sentences. It check and determine the words which are relevant when the model processes a particular word.

For example, “Maya put the ice cream in the freezer because it was melting.” The model needs to connect “it” with “the ice cream” to understand the sentence. 

Caption for every token is generated, and the model works with these captions:

• Query (Q): It indicates the needs of the token.

• Key (K): It contains the data that is used to assess its relevance.

• Value (V): It holds the data that is used to produce the final result.

4. Multi Head Attention

Using a multi-head attention mechanism, transformer models can look at different relationships between words simultaneously with several attention heads.

Every attention head in this system is capable of learning various relationships by looking at different patterns in the text.

In their base model, the original transformers made use of eight attention heads.

Also Read: RAG Architecture Explained: Components, Workflow, Types & Python Implementation

5. Feed Forward Neural Network

The feed forward neural network process the information produced by the attention mechanism. It apply the mathematical operations to help the model to learn patterns and make useful representations.

In the original base Transformer, the feed-forward network used a hidden dimension of 2,048, compared with the model's main representation dimension of 512.

This gives the network more space to process information at each position in the sequence.

6. Residual Connections and Layer Normalization

• Residual connections help information to pass through the network by adding a layer's input to its output. This helps information and gradients flow through deeper networks.

• Layer normalization keeps the values within each layer on a more consistent scale, helping the model train effectively.

In the original Transformer, these 2 techniques were used around the attention and feed-forward sublayers.

7. Encoder

The encoder process the input and build representations that capture the meaning and relationships between its elements.

In the original Transformer, the encoder contained 6 layers. Each layer included a self-attention mechanism and a feed-forward neural network.

8. Decoder

The decoder uses the encoder's output and previously generated tokens to produce the next part of the output.

In the original Transformer, the decoder also contained 6 layers. It used masked self-attention to prevent the model from looking at future output tokens during generation. It also used attention to access information from the encoder.

Not all Transformers use both components. GPT models use a decoder-only architecture, while BERT uses an encoder-only architecture.

9. Linear Layer and Softmax

The linear layer converts the decoder's output into scores for possible next tokens. The softmax function converts these scores into probabilities.

For example, when a model receives the prompt “The sky is”, it might assign probabilities to words such as “blue,” “clear,” or “cloudy.” The model then selects the next token based on its output probabilities and decoding settings.

Now that you understand the core components of a transformer, let’s look that how the transformer architecture works.

Also Read: Retrieval Augmented Generation (RAG): How It Works, Types, and Applications

Step-by-Step Working of Transformer Architecture

Let’s understand how a transformer process a sentence “The cat sat on the mat.” to understand its core working:

Step 1: Tokenization and Embeddings

The transformer breaks the sentence into tokens, such as “The,” “cat,” and “sat,” and converts each token into numbers the model can process.

Step 2: Positional Encoding

The model adds information about each token’s position to understand the word order in the sentence.

Step 3: Self-Attention

The model then identifies relationships between words. For example, it connects “cat” with “sat” to help understand who performed the action.

Step 4: Multi-Head Attention

The model then examine different word relationships at the same time, such as the relationship between “cat” and “sat” and between “on” and “mat.”

Step 5: Feed-Forward Network

The network processes the information collected by attention and helps the model learn patterns in the sentence.

Step 6: Output Generation

The model uses the processed information to understand the sentence or predict the next word. For example, given “The cat sat on the,” it may predict “mat” as the next word.

Also Read: Vector Database: Architecture, Indexing, Search, and Use Cases

Now let’s understand what different types of transformer architectures are there based on the information or input they use and output they produce.

Types of Transformer Architectures

Transformer models are commonly grouped into three main architectural categories: encoder-only, decoder-only, and encoder-decoder. 

Every technology has some limitations, which transformers also have. Let’s discuss them below.

Also Read: What Are Foundation Models? Types, Training, Examples & Uses

Challenges in Transformer Architecture

Transformer architecture has limitations that can affect its cost, speed, and accuracy. Some of the main challenges are:

1. High Computational Requirements

To be effective at processing very large amount of text, transformers are required to have a good computing capability. The reason the computation increases to a great extent in terms of the squared length when the sequence gets longer is that a token will compare representations with other every single token under standard self-attention.

Example: Processing a document that contains 10,000 tokens can consume significantly more attention computation than processing a document that contains 1,000 tokens. This can increase the time and money required to process long documents.

2. High Memory Usage

To process input transformers utilize their memory capacity. Besides, to carry intermediate calculations and attention scores, the models need memory. With the increase in the length of the input handled by a model or in the number of concurrent users served by a model, naturally the required memory and computing resources go up as well.

For example, The capital expenditures for Google's parent company Alphabet stood at $80.6 billion in the first half of 2026. The money was spent on servers, networking equipment, and data center buildings needed to support Alphabet services and AI workloads.

3. Dependence on Training Data

A training data is a source from which transformers identify and recreate underlying patterns. Therefore, when the training data has errors, gaps, the model can inherit those flaws. It may come out in its output responses that are wrong or have a gap.

Example: If a hiring model is trained on historical recruitment data that favors certain groups, it may learn patterns that unfairly disadvantage other applicants. 

4. Hallucinations and Incorrect Answers

When the information is not sufficient, transformer based LLMs have the capability to just guess. In most of the cases, they do not look over the facts before the response, and the answers they deliver do not make a distinction between the factually correct ones and the made ups. In fact, they can even create some sort of fictional information which is so believable but entirely false.

For example, In 2022, Air Canada's chatbot gave a customer incorrect information about bereavement discounts. In 2024, a Canadian tribunal held the airline responsible for the misleading advice and ordered it to pay damages

5. Difficulty Understanding Complex Relationships

When dealing with situations where one is required to track multiple relationships or combine different steps of a reasoning, transformers tend to make it rather difficult to come up with the final answer. 

This was the focus of the study titled Can LLMs Reason Structurally? Benchmarking via the Lens of Data Structures which was presented at the International Conference on Machine Learning (ICML) in ‍‍‍2026.

The researchers evaluated 13 large language models using DSR-Bench, a benchmark containing 4,140 problems across 20 data structures and 35 operations. The best-performing model scored only 0.46 out of 1 on challenging problems.

Conclusion

Till‍‍‍ now you are able to understand that, transformer architecture help the AI models to understand relationships between words and use them to process information or generate text.

Through this blog, we have discussed the various components of the transformers; embeddings, positional encoding, attention mechanisms and feed-forward networks have worked together. Additionally, we also explored how different transformer variants can be used for different purposes and identified some limitations of its usage.