INTRODUCTION TO CHIPLETS

Chiplets: A set of chips on the same substrate that together act as a

processor. Today, this technology is on everyone's lips. AMD is already
releasing their third generation of Ryzen on the chiplet-based architecture
called Zen. Intel is planning to join the party in the next generation of their
core processors.

● But what made chip makers suddenly so interested in chiplets?

● Is it really a new technology?
● What pitfalls does this technology bring about?

Let's begin with trying to understand what is different between the regular
monolithic CPUs and the chiplet ones. In conventional CPUs, all units such
as cores, cache, and various controllers are located in one silicon die. In
the case of chiplets, such units are distributed across separate dies.

For example: In the Ryzen 3000 series and newer, there may be one or
two dies with cores under the processor lid, as well as an I.O. die
containing PCIe and RAM controllers, and starting with the 7000 series, an
integrated graphics too.
 ● But why would the manufacturers want to complicate things so
much?
After all, it is obvious that designing several dies, their power supply
wiring, and buses for data transfer is not an easy task. Why not just
continue to develop monolithic dies as Intel has been doing?

The answer here will hardly surprise anyone. At a long distance, chiplets
turn out to be more profitable. There are several reasons for this. And the
main one is that with each new process node, silicon wafers, which include
thousands of ready-made dies, are becoming more expensive.

For example: the current 300mm wafers, manufactured under the 5nm
TSMC process node, cost the processor manufacturers $13,000 apiece.
The next stage, 3nm, raises the price to $20,000. And for 2nm wafers, they
predict the price to be over $25,000.

It is obvious that under such conditions the chip makers would want to cut
any corners they can find. And chiplets allow them to do this.

“Not all units inside a chip require the use of the latest process
nodes.”

In the case of the core dies, there is no choice. The finer the process node,
the better. Otherwise, the chips will be hot and with low performance. But
take for example a RAM or PCIe controller. They don't need the expensive
3nm TSMC process node at all. So you can save money by making it a
separate chip, which is what AMD did.
For example: the Ryzen 7000 processor chips are based on a new 5nm
process node, whereas the IO chip in them is based on a cheaper 6nm
node, which allows the manufacturer to use different suppliers for different
parts. For example, according to rumors, TSMC will manufacture the
graphics chiplets for the next Intel processors called Meteor Lake, which
means that the defect rate is lower.

“The thing is that no matter how precise and clean the chip
production is, there are still defective dies on the wafer that do not
meet the requirements.”
For example: a core, cache or integrated graphics may not be operational.
In the case of monolithic processors, if the defect is not critical, some units
may be disabled and the chip is then sold not as say a Core i9, but as an
i7. But if the damage to the chip is serious, the entire die is trashed.

Therefore, it is much more beneficial to produce smaller chips, since the

defective places on the wafer are more or less evenly spaced, and the
chance of fatal damage in smaller dies is less, which means the yield of
usable chips is higher, reducing the final cost of each.

“It's easy to scale.”

AMD, having a standard 8-core die, can easily get any number of cores
from 4 to 16 in desktop CPUs, by both playing around with disabling
individual cores and being able to throw a whole extra die onto the
substrate. Intel, with their older lake, had to develop two completely
different monolithic dies. One for baseline and mid-tier processors with up
to 6 p-cores and a separate one for the top-end CPUs where the limit is 8
p-cores and 8 e-cores.This is especially important in the server segment,
where AMD can easily scatter as many as 12 dies with 8 cores each onto
the substrate, getting a total of 96. Intel is forced to make two different dies
with MCC and XCC with a core count of up to 32 and starting from 32. And
of course at a huge cost, which affects the final price in a bad way.
This is especially important in the server segment, where AMD can easily
scatter as many as 12 dies with 8 cores each onto the substrate, getting a
total of 96. Intel is forced to make two different dies with MCC and XCC
with a core count of up to 32 and starting from 32. And of course at a huge
cost, which affects the final price in a bad way.

And finally, let's talk a bit about geometry. Silicon wafers are round, but
chips are rectangular. Therefore, some part of the precious wafer is
guaranteed to go to trash. And of course, processor manufacturers would
love to reduce the waste as much as possible. And chiplets are here to
help. The smaller the die compared to the wafer, the smaller the wasted
material zone, which makes the benefit in comparison with large monolithic
chips obvious.
 ● But since chiplets are so good, why haven't they been used before?

In fact, they have, but only in some rare cases when their advantages
outweigh the troubles.

The use of chiplets has been limited in the past because their benefits did
not always outweigh the challenges of implementing them. One notable
early example of chiplet use is the Pentium 2 from 1995, in which Intel
incorporated chiplets onto the PCB. The main processor with compute
units, controllers, and L1 cache was produced independently by Intel, but
the company purchased L2 cache chips separately. At that time, it was not
cost-effective to integrate them into the same die with the CPU. Intel
revisited the use of chiplets in the mid-2000s with the introduction of the 4-
core Core 2 quad processors, which incorporated two dies from the dual-
core Core 2 duo.

This approach made sense at the time. The core architecture was
groundbreaking and innovative enough to double the core count in a
relatively inexpensive way. As you can see, AMD was not the first to do
this. It's just that this time AMD realized that producing large monolithic dies
is too expensive faster than the industry giant, and focused on chiplets. On
the other hand, Intel is planning to return to this approach only next year.

However, Intel, with all its expertise in processors, has proven its
capabilities with the older Lake architecture and its heterogeneous cores.
Intel is planning to make a triumphant return to the chiplet market with a
more progressive approach than that of AMD. AMD simply placed
completely separate dies on the PCB, which had to be interconnected by
the Infinity Fabric Bus. This caused inter-die latency and RAM access time
to increase significantly, which had to be mitigated by a large volume of L3
cache. This approach turned out to be quite successful.

Intel decided to take a more complex but technologically advanced

approach by using an additional substrate called Interposa.

This is another silicon chip to which all the chiplets are attached from
above. Data buses and power wires run inside it, and the interposer itself is
then connected to the PCB. There are two advantages at once.
“Firstly, all the chiplets are physically closer to each other, which
solves the latency issue. “

“Secondly, the use of a silicon chip substrate allows for more

accurate control of voltage and data transfer, thereby reducing power
consumption. “

Intel's design turns out to be more flexible in terms of the number of

chiplets that could be added, of which there can be up to four types in
Meteor Lake. The main ones are those with computing units and L1 and L2
cache. Separately, there may be a chiplet with integrated graphics, another
with an L3 cache and a PCIe bus, and finally one with a memory controller
and other IO buses. This makes it easier and cheaper to manufacture using
different processors.

For example: now Intel's F solutions come with integrated graphics

disabled, but physically it is present in the die. With chiplets, the F
processors will simply be devoid of the graphics on the physical level.

● Does it mean that Intel's approach is better?

Maybe, but not cheaper. Yes, the interposer is a much more advanced
thing, but creating a two-layer stand is more expensive than just putting
dies on a PCB as AMD does. In any case, we'll have to wait for the Meteor
Lake tests, then it will become clear whether it makes sense to overpay for
Intel's more expensive approach.

● As we have already understood, the chiplets are the future, but aren't
there any possible pitfalls that will surface later?
Some already have.The modern Ryzen CPUs have eight powerful cores in
tiny dies the size of a fingernail. Dissipating 100 watts of heat from them is
far from a trivial task. Previously, this was solved by having the cores
located in one large die which included other units that didn't heat up as
much. However, now the elements that don't heat up as much are put into
a separate chip, leading to temperatures above 90 degrees on the core
chiplets. The challenge is that heat is transferred from the small surface of
the die to the heat spreader very reluctantly, and this heating problem is not
the only one.

Additionally, the current server Epic Genoa CPU is the size of a palm due
to the presence of 12 chiplets with cores inside them and another one with
various controllers. It's concerning to even think about the upcoming Intel
LGA 7529.The socket can accommodate up to 6 M5 Ryzen CPUs due to
the increase in size. This increase is necessary because it is challenging to
create a monolithic die larger than 1000 square millimeters.

Although having larger chiplets results in a higher percentage of defects, it

is still feasible to fit multiple small chiplets onto a substrate. The
competition in core count between Intel and AMD may result in desktop
PCs using CPUs as large as those currently found in servers. These CPUs
could contain a dozen chiplets, leading to challenges in fitting them into
sockets.

Routine tasks may include fixing bent pins and addressing cooling
difficulties due to the complexity of creating a perfectly flat surface on such
large CPUs and coolers. However, these issues are likely to be problems
for the distant future.

● Why aren't chiplets more commonly used in GPUs?

For example: the RTX 4000 or Intel ARC has a monolithic die and
separate memory chips. Even in the RX 7000 series, chiplets are used but
with limitations.

The reality is that chiplets in GPUs face significant challenges with

reliability and size. Some may recall the AMD FIDG R9 and Vega series
video cards, which featured a GPU die and HBM memory stacks on the
same substrate.

The approach was innovative at that time, enabling a significant reduction

in board size. The Radeon R9 Nano, with its compact dimensions and
single fan, performed comparably to top-end GTX 980.

However, issues soon emerged.

“Firstly, there were reliability problems. “

If any component of the unit - whether it be a memory die or GPU - failed,
the entire unit had to be replaced. Consequently, it is now rare to find any
functioning Vega 56 or 64 units as they have all failed. This is not an issue
with conventional video cards, as individual components can be easily
replaced if they fail.

“Secondly, there were difficulties with cooling.”

Without a heat spreader, all dies had to be the same height.Otherwise,

cooling the slightly shorter ones will be more difficult.

“Lastly, let's talk about the cost. “

The Vega 7 initially costs $700. Despite being slightly slower in games
compared to the RTX 2080, which was available at the same price, it offers
features such as ray tracing and DLSS. As a result, AMD decided to take a
step back and reverted to using regular GDDR memory soldered onto the
PCB instead of the HBM memory located next to the GPU die. However,
Team Red has not completely abandoned the idea of chiplets. In the RX
7000 lineup, they divided the GPU into parts. For instance, the top-end
Navy 31 chip consists of one large die with compute units and L2 cache,
alongside 6 smaller dies with L3 and memory controllers.

It seems that this approach is quite effective. High-end Radeon 7000 series
cards are less expensive than the RTX 4000s while offering similar
performance. Considering the rapid growth in the number of transistors in
GPUs and the fact that the size of their dies is approaching the limits of
production capabilities, it is likely that we will also see chiplets in the
upcoming ARC2 or RTX 50.

This signifies a divide-and-rule policy in the silicon world. It is becoming

evident that the future of some semiconductors lies in chiplets. Of course,
new technologies bring about new problems that did not exist in monolithic
dies, but the benefits of chiplets outweigh the drawbacks. Therefore,
congratulations are in order. We are on the cusp of perhaps the most
significant advancement in the development of certain semiconductors in
the last 20 years.