PCIe, what are groups, lanes, gen and how they all relate?

It’s pretty easy to get lost in all the PCIe jargon and, as a non-technical person, figure out what the specs actually mean and what they do.

It’s even more important for a PCB designer to understand all these differences. I was lost too.

So, let’s break down the basics of PCIe (Peripheral Component Interconnect Express) and try to understand what all these terms actually mean.

Table of Contents

• 1. Lanes — the xN notation
• 2. Generation — Gen
  • NRZ vs PAM4
  • GT/s is not the same as Gbps
  • PCIe bandwidth by generation
  • PCIe Gen4 x16
• 3. PCIe slots and link width
• 4. Groups, Root Complex and PCIe Switches
  • Root Complex
  • Groups / Lane bifurcation
• Channel Length
• Why did Gen6 move to PAM4 instead of just increasing the frequency?
  • Rise time makes things even more interesting
  • The downside of PAM4
• PCIe vocabulary

1. Lanes — the xN notation

A lane is the basic unit of communication in PCIe.

One PCIe lane consists of:

• One differential TX pair
• One differential RX pair

So that’s a total of 4 signal wires:

TX+ TX- and RX+ RX-

PCIe supports multiple lanes, with the number of lanes normally being a power of two.

For example:

PCIe x1

One lane:

• 1 × TX differential pair
• 1 × RX differential pair

PCIe x4

Four lanes:

• 4 × TX differential pairs
• 4 × RX differential pairs

PCIe x16

Sixteen lanes:

• 16 × TX differential pairs
• 16 × RX differential pairs

So, pretty simply:

More lanes = more bandwidth.

This is where the x1, x4, x8, x16, etc. notation comes from.

2. Generation — Gen

The PCIe generation refers to the data rate per lane.

Higher generation = higher data rate per lane.

For example, PCIe Gen3 provides 8 GT/s per lane, while Gen5 provides 32 GT/s per lane.

But there is a little more to it than just increasing the frequency.

NRZ vs PAM4

All PCIe generations up to Gen5 use NRZ (Non-Return-to-Zero) signalling.

NRZ effectively carries one bit per symbol.

Gen6 moves to PAM4 (Pulse Amplitude Modulation with 4 levels).

PAM4 has four voltage levels, which means each symbol can represent two bits:

• 00
• 01
• 10
• 11

This allows PCIe Gen6 to achieve 64 GT/s per lane without simply doubling the signalling frequency compared with Gen5.

That distinction is important.

Gen5 runs at 32 GT/s, while Gen6 runs at 64 GT/s, but Gen6 achieves this using PAM4 rather than simply running NRZ at twice the frequency.

GT/s is not the same as Gbps

This is another place where PCIe specifications can get confusing.

PCIe bandwidth is normally specified in GT/s — giga-transfers per second.

The actual usable data rate is lower because of the encoding overhead.

For example, PCIe Gen5 runs at:

32 GT/s per lane

With 128b/130b encoding, the usable data rate is approximately:

31.5 Gbps per lane

or around:

3.94 GB/s per lane

which is usually rounded to 4 GB/s per lane.

PCIe bandwidth by generation

*For NRZ, the signalling frequency is approximately half the transfer rate. Gen6 uses PAM4, so the relationship is different.

So, for example:

PCIe Gen4 x16

Gen4 provides approximately 2 GB/s per lane.

With 16 lanes:

2 GB/s × 16 = 32 GB/s

So a PCIe Gen4 x16 link has approximately 32 GB/s of aggregate bandwidth.

This is why you will often see something like:

PCIe Gen4 x16

That tells you two things:

Gen4 → how fast each lane operates

x16 → how many lanes are being used

3. PCIe slots and link width

This is another place where things can get confusing.

A motherboard can have physical PCIe slots of different sizes:

• x1
• x4
• x8
• x16

But the physical slot width doesn’t necessarily tell you the actual electrical link width.

For example, you could plug a GPU into a physical x16 slot, but the GPU might actually be operating at x8.

Why?

Because the number of PCIe lanes available from the CPU or chipset is limited.

So there are really two things to consider:

Mechanical width — how many lanes the slot physically accepts.

Electrical/link width — how many lanes are actually connected and negotiated.

This is why a motherboard specification might say something like:

PCIe x16 slot — x8 electrical

The slot is physically x16, but only eight lanes are actually connected.

4. Groups, Root Complex and PCIe Switches

Now we get into some of the terminology that confused me when I first started looking at PCIe system architectures.

Root Complex

The Root Complex is essentially the main PCIe controller connecting the PCIe hierarchy to the CPU or SoC.

It sits at the top of the PCIe topology and provides the connection between the processor and the PCIe devices.

From the Root Complex, PCIe links can connect to endpoints, switches, or other PCIe infrastructure.

Groups / Lane bifurcation

Some PCIe implementations allow a larger group of lanes to be split into smaller groups.

For example, a x16 connection can potentially be configured as:

• 1 × x16
• 2 × x8
• 4 × x4

This is commonly called PCIe bifurcation.

This becomes particularly useful in SoM and carrier-board designs.

For example, a SoM might expose a x16 PCIe interface, but the carrier board could use those lanes as multiple independent interfaces depending on the application.

So instead of thinking:

“I have a x16 PCIe interface.”

you sometimes need to think:

“I have 16 PCIe lanes that can potentially be allocated in different groups.”

Channel Length

Now we get into the part that becomes much more interesting for PCB designers.

Channel length is essentially the total electrical path between the PCIe transmitter and receiver.

And that doesn’t necessarily mean just the PCB trace.

The channel can include:

• PCB traces
• Vias
• Connectors
• Cables
• Packages
• Add-in cards
• Other interconnects

As the PCIe generation increases, the channel becomes harder to design.

Higher data rates mean that:

• attenuation increases
• reflections become more significant
• crosstalk becomes more problematic
• jitter becomes more important
• insertion loss becomes a bigger constraint

This means that the maximum practical channel length generally decreases as the PCIe generation increases.

But how do we actually calculate the maximum trace length?

There isn’t one simple formula.

Instead, we work with a loss budget.

The PCIe specification defines the maximum allowable channel loss for a particular generation and frequency.

For example, a Gen5 channel has a defined loss budget around the Nyquist frequency.

The total channel loss can roughly be thought of as:

Total Loss = TX Loss + PCB Trace Loss + Connector Loss + Add-in Card Loss + RX Loss

Most of these losses can be estimated or characterized separately.

The part we’re interested in as PCB designers is often the PCB trace loss.

Once we know how much loss is available for the PCB, we can determine how long the trace can be before we exceed the channel budget.

Why did Gen6 move to PAM4 instead of just increasing the frequency?

This is probably one of the most interesting parts of the whole thing.

PAM4 stands for Pulse Amplitude Modulation with 4 levels.

With NRZ, there are two voltage levels, so each symbol represents either:

0 or 1

With PAM4, there are four voltage levels, so each symbol can represent:

00, 01, 10, or 11

That’s two bits per symbol.

This allows Gen6 to achieve 64 GT/s while keeping the fundamental signalling frequency at around 16 GHz, instead of pushing NRZ to a much higher frequency.

And this is important because simply doubling the NRZ frequency would make the channel significantly harder to design.

If Gen6 used NRZ at 64 GT/s, the fundamental frequency would be approximately 32 GHz.

At that frequency, channel loss becomes enormous.

For a normal FR-4 PCB, the usable channel length could potentially become only a few inches.

With a low-loss material, you could stretch that somewhat further, but you’re still dealing with a very difficult high-speed channel.

At this point, you’re not just doing traditional “high-speed PCB design.”

You’re getting very close to RF engineering territory.

Rise time makes things even more interesting

Gen5 has a signalling frequency of 16 GHz, but frequency alone isn’t enough to understand the signal integrity problem.

The rise time is extremely important.

A fast edge contains significant high-frequency energy well beyond the fundamental frequency.

For example, a Gen5 NRZ signal can have a rise time in the tens of picoseconds.

This is also where the classic:

“When does a PCB trace become a transmission line?”

question comes in.

The downside of PAM4

PAM4 isn’t free.

Instead of two voltage levels, we now have four.

That means the voltage difference between adjacent levels is smaller.

So PAM4 has a smaller noise margin compared with NRZ.

This makes the system more sensitive to:

• noise
• crosstalk
• jitter
• distortion
• channel loss

It also results in a higher raw bit error rate.

To compensate for this, PCIe Gen6 introduces techniques such as:

• FEC — Forward Error Correction
• CRC
• DSP / equalization techniques

The system is therefore doing considerably more work to maintain the required reliability.

PCIe vocabulary

At this point, the terminology starts to make a lot more sense.

When you see something like:

PCIe Gen5 x16

you can break it down into:

Gen5 The generation of the PCIe interface, defining the signalling/data rate.

x16 There are 16 PCIe lanes.

Each lane has:

• one differential TX pair
• one differential RX pair

So a x16 interface has:

16 TX pairs + 16 RX pairs = 32 differential pairs

And each lane provides approximately 4 GB/s of usable bandwidth in each direction.

That gives us roughly:

4 GB/s × 16 = 64 GB/s

of aggregate bandwidth per direction.

But getting those 64 GB/s through a real PCB is where things get interesting.

And that’s where the fun begins.