2022 IEEE Hot Chips 34 Symposium (HCS) | 978-1-6654-6028-6/22/$31.00 ©2022 IEEE | DOI: 10.1109/HCS55958.2022.

9895532

Intel Corporation

Meteor Lake and Arrow Lake

Intel Next-Gen 3D Client
Architecture Platform with
Foveros
Wilfred Gomes, Slade Morgan, Boyd Phelps, Tim Wilson, Erik Hallnor

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
 Pervasive
Intel’s Mission Intelligence Era 1021
We create world-changing technology 100B Edge Connected Devices
that improves the life of every person on the planet

1018

Mobile + Cloud Era

10B Cloud Connected Devices Empower 1015

Everyone
PC Era
1B Internet Connected Devices 109

Connect
Everyone 104

Compute
Digitize
+ x86 Everything + x86 x86

1980 1990 2000 2010 2020

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 2
 Pervasive
Experience Driven Client Intelligence Era 1021
100B Edge Connected Devices

Experience First 1018

Purposeful Performance Experience 1015

Driven
Compute
Dynamic 109

104
Scale

Compute
1980 1990 2000 2010 2020

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 3
 Experience Driven Implications for Client

Experience First Performance Performance, Perf/Watt

Mix & Match Blocks

Purposeful Performance Flexibility and Functions

Dynamic Innovation Pace Time-to-Market

Scale Next Exponential The next generation of devices

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 4
 Monolithic Disaggregated

Highest Performance Lower (tax on latency, power, B/W)

Very Limited Flexibility Limited

Slow (per SOC basis) Innovation Pace Faster (release per new function)

Low Scale Higher

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 5
 Monolithic Disaggregated

Highest
Can we get monolithic Lower (tax on latency, power, B/W)

Very Limited
performance with Limited

Slow (per SOC basis)

disaggregated Faster (release per new function)

Low
architecture benefits? Higher

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 6
 Disaggregation Journey

Haswell Kaby Lake G Lakefield Ponte Vecchio

Ultra Thin & Light Ultra Thin & Perf Graphics Ultra Thin & Light High Density & Performance
2014 2017 2019 2022

CPU/PCH/Memory Hybrid Architecture 47 Tiles

Architecture partitioning
CPU/GFX partitioning
CPU/PCH partitioning Compute/mem/IO partitioning

Packaging 2D 2.5D + 2D 3D 2.5D + 3D

MCP EMIB + MCP 50µm Foveros EMIB + 36µm Foveros

Process

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 7
Process

Transistor Design Target Range

Transistor Diversity
Opportunity
Desktop CPU
SOC Optimized

Frequency Density
Graphics Optimized IO

Memory Optimized
Mobile CPU

Mobile CPU Optimized Graphics

Memory
Desktop CPU Optimized

I/O Optimized Power

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 8
Packaging

Advanced Packaging Foveros Direct

Interconnect Density

Foveros Omni
Bump pitch - < 10 microns
Bump density - > 10,000/mm2
Foveros Power - < 0.05 pJ/bit`

EMIB Bump pitch – 25 µm

Bump density – 1,600/mm2
Power – <0.15 pJ/bit
Bump pitch – 50-25 µm
Bump density – >400-1600/mm2
Power – 0.15 pJ/bit
Standard Package
Bump pitch – 55-45 µm
Bump density - 330- 772/mm2
Power – 0.50 pJ/bit

Bump pitch - 100 µm

Bump density - 100/mm2
Power – 1.7 pJ/bit

Power Efficiency

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 9
 Packaging

Advanced Packaging Foveros Direct

High Yield & High Volume
Manufacturing for large
number of tiles
Interconnect Density

Foveros Omni Die Center Die Edge

Bump pitch - < 10 microns
Bump density - > 10,000/mm2
Foveros Power - < 0.05 pJ/bit`

EMIB Bump pitch – 25 µm

Bump density – 1,600/mm2
Power – <0.15 pJ/bit
Bump pitch – 50-25 µm
Bump density – >400-1600/mm2
Power – 0.15 pJ/bit
Cu
Ni
Bump pitch – 55-45 µm
Bump density - 330- 772/mm2
Power – 0.50 pJ/bit
Ni
Cu

Power Efficiency

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 10
 Packaging

Advanced Packaging Foveros Direct

Interconnect Density

Foveros Omni
Bump pitch - < 10 microns
Bump density - > 10,000/mm2
Foveros Power - < 0.05 pJ/bit`
Base die

EMIB Bump pitch – 25 µm

Bump density – 1,600/mm2
Power – <0.15 pJ/bit
Bump pitch – 50-25 µm ‘Mix and match’ tiles in base die complex
Bump density – >400-1600/mm2
Power – 0.15 pJ/bit

Bump pitch – 55-45 µm 4x higher interconnect bump density vs EMIB

Bump density - 330- 772/mm2
Power – 0.50 pJ/bit

Power Efficiency

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 11
 Packaging

Advanced Packaging Foveros Direct bonded

Cu
inter-
connect
Interconnect Density

top die
Foveros Omni
Bump pitch - < 10 microns
Bump density - > 10,000/mm2
Foveros Power - < 0.05 pJ/bit`

base die
EMIB Bump pitch – 25 µm
Bump density – 1,600/mm2
Power – <0.15 pJ/bit
Bump pitch – 50-25 µm
Bump density – >400-1600/mm2
16x higher interconnect bump density
Power – 0.15 pJ/bit
vs Foveros @ 36 µm pitch

Bump pitch – 55-45 µm

Bump density - 330- 772/mm2
Power – 0.50 pJ/bit Higher B/W at lower latency, power, & die area

Power Efficiency

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 12
Architecture

Architecture Evolution

Ponte Vecchio
Lakefield
Kaby Lake G 2022
Haswell 2019
Core Duo 2017
Pentium 2013
Intel 386
1990
2005 First Commercial 2D+ 3D
1980
Design with
Design with 3D Logic Design Large
Design with 2D Chiplet on Logic
Number of Tiles
Design with IP 2D and 2.5D
Chiplets Stacking
Design with
HDL functional,
Schematics Sea of cells/blocks units/IPs
Sea of transistors/
gates

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 13
 Disaggregation Journey

Kaby Lake G Lakefield Ponte Vecchio

Ultra Thin & Perf Graphics Ultra Thin & Light High Density & Performance
2017 2019 2022

Hybrid Architecture 47 Tiles

Meteor
Architecture CPU/GFX partitioning

Lake
CPU/PCH partitioning Compute/Memory/IO partitioning

Packaging 2.5D + 2D 3D 2.5D + 3D Next Step in our

EMIB + MCP 50µm Foveros EMIB + 36µm Foveros
Disaggregation
Journey
Process

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 14
Architecture
and Design
tradeoffs
New Flexible Tiled Architecture

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 15
 Scalable GPU Tile SOC Tile

Architecture CPU Tile

Base Tile

Goals

Flexibility with core arch, count, process

IO
Flexibility with Graphics cores Extender
Tile

IO modularity

Process node flexibility

Ability to scale graphics and compute

Low power to discrete graphics performance

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 16
Compute Tile
Core Count Core Node Cache
Scalability Generation Scalability Scalability
Scalability
E-core E-core
cluster cluster

P-core P-core E-core E-core

L2$

L2$
Coherent Fabric
cluster cluster

L3$

L3$
P-core P-core

L2$

L2$
P-core P-core

P-core P-core

E-core E-core

L2$

L2$
Coherent Fabric
cluster cluster

L3$

L3$
E-core E-core

L2$

L2$
Cluster cluster P-core P-core

P-core P-core

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 17 17
Graphics Tile
Core Count Node Cache
Scalability Scalability Scalability

Xe-core Xe-core Xe-core Xe-core

Xe-core Xe-core Xe-core Xe-core

Xe-core Xe-core Xe-core Xe-core

Fixed
L3$ L3$
Functions
Fixed
L3$ L3$
Functions
Fixed
L3$ L3$
Functions

Xe-core Xe-core Xe-core Xe-core

Xe-core Xe-core Xe-core Xe-core

Xe-core Xe-core Xe-core Xe-core Xe-core Xe-core Xe-core Xe-core

Xe-core Xe-core Xe-core Xe-core Fixed Fixed

L3$ L3$ L3$ L3$
Functions Functions

Fixed
L3$ L3$
Functions

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 18
SOC Tile
Scalable IP Blocks

Low power IP SRAM High Voltage IO

Memory Memory
Control.
Control.

Media Media

I/O Fabric
I/O Fabric

Low

CoheherentFabric
Low
IO Power IO Power

Memory IO
Memory IO

IP IP

XPU XPU

Imaging Imaging

Display Display

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 19
I/O Extender Tile
Scalable I/O Blocks

Number of Lanes Bandwidth Protocol Speed

IO IO
IO IO
Control Control

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 20
Meteor
Lake
Construction

Base Tile

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 21
 Meteor Lake
Back side
Known good tiles
metallization

GPU SOC Tile CPU Tile

Base Tile

Metal layers for

Base tile
36 µm pitch IO/power
with large
Die2Die delivery and
capacitance
Die2Die routing

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 22
Meteor Lake
Base Tile

3D Die2Die
Capacitors power delivery,
package IO routing

Modularity with
active silicon for
memory and logic

Silicon
Redistribution layers
with active silicon
Package
Bump

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 23
 Meteor Lake
Colors Represent
3D Capacitors,
Voltage Islands
Die 2 Die

SOC Tile

Die 2 Die Die 2 Die

CPU Tile

IOE Tile

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 24
 FDI - Foveros Die Interconnect
Low Voltage CMOS interface
Protocol Layer (optional)

Logical Protocol
High Bandwidth, Low Latency handler handler

PGT D2D IO

Synchronous and µbumps

asynchronous signaling
PGT D2D IO

Logical Protocol
Low area overhead handler handler

Operation @ 2 Ghz, Protocol Layer (optional)

0.15 – 0.3 pJ/bit

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 25
Meteor Lake
Interconnect
Link Mainband width Mainband Protocol

Die 2 Die

SOC Tile
CPU - SoC ∼2K 2x IDI

- SoC
Die 2 Die
∼2K 2x iCXL Die 2 Die

CPU Tile

SoC - IOE ∼1K IOSF, 4x Display Port

IOE Tile

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 26
 Meteor Lake
Thermals System, Software, Silicon
>1X Co-optimization
1X
Turbo Power Capability

Floorplan

Materials

System

Process

Previous Start Innovations Meteor Lake

Architecture
Generation

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 27
 Meteor Lake
Power Delivery System, Software, Silicon
Co-optimization
Capacitors on Base Tile
to implement power delivery
500
solutions for mix/match
Total Capacitance FF/um2

Decoupling/Noise
376

Voltage Regulator
193
IP Mix and Match
141
Form Factor
37
Package Optimization
14 nm 10 nm Intel 7 Intel 4 MTL Base Die

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 28
 Client ( Haswell ) Meteor Lake
2013 2023

Interface OPIO ( On Package IO) FDI ( Foveros Die Interconnect)

Speed 2- 8GT/s 2 GT/s

IO/mm^2 1X ( 110 um) 10X ( 36 µm)

Latency 10- 20 ns < 10 ns

Power 1 pJ/bit 0.2 - 0.3 pJ/bit

Number of Tiles 2 5
Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 29
 Monolithic Disaggregated

Highest
Can we get monolithic Lower (tax on latency, power, B/W)

Very Limited
performance with Limited

Slow (per SOC basis)

disaggregated Faster (release per new function)

Low
architecture benefits? Higher

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 30
 Monolithic vs. Disaggregated
Disaggregated
Custom/Targeted Process

Optimized IP Density

+ IP Refresh Pace
Performance

Form Factor
Monolithic (High Bandwidth interface)

- 2-3% Disaggregation Tax - No Optimization

90 50 36 25 15 10
Bump Pitch (µm)
Organic Package Foveros

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 31
 Disaggregated Transistor
Custom/Targeted Process Performance Uplift1
Performance

Optimized IP Density

+ IP Refresh Pace

Form Factor
5.00 Intel 7

Monolithic High Bandwidth interface Intel 4

6VT

Total Power (Normalized)

4.00
No Optimization
Intel 4
8VT

3.00

2.00

1.00

0.00
2.00 2.50 3.00 3.50 4.00
90 50 36 25 15 10
Bump Pitch (µm) Frequency (GHz)
Organic Package Foveros

1. B. Sell et al., "Intel 4 CMOS Technology Featuring

Authorized Advanced
licensed FinFET
use limited Transistors
to: Transilvania optimized
University for High
of Brasov. Densityon
Downloaded and High-Performance
October Computing,"
30,2023 at 09:55:50 2022
UTC from IEEE IEEE Restrictions
Xplore. Symposium on VLSI Technology and
apply.
Department or Event
Circuits Name and Circuits), 2022, pp. 282-283, doi: 10.1109/VLSITechnologyandCir46769.2022.9830194.
(VLSI Technology Intel Confidential 32
 Disaggregated
Performance
Custom/Targeted Process IP refresh, Process node
Optimized IP Density

+ IP Refresh Pace

Form Factor
Monolithic High Bandwidth interface Meteor Lake xPU

- No Optimization

Arrow Lake xPU

Lunar Lake
and Beyond xPU

90 50 36 25 15 10
Bump Pitch (µm)
Organic Package Foveros

1. B. Sell et al., "Intel 4 CMOS Technology Featuring

Authorized Advanced
licensed FinFET
use limited Transistors
to: Transilvania optimized
University for High
of Brasov. Densityon
Downloaded and High-Performance
October Computing,"
30,2023 at 09:55:50 2022
UTC from IEEE IEEE Restrictions
Xplore. Symposium on VLSI Technology and
apply.
Department or Event
Circuits Name and Circuits), 2022, pp. 282-283, doi: 10.1109/VLSITechnologyandCir46769.2022.9830194.
(VLSI Technology Intel Confidential 33
 Meteor Lake Status

Meteor Lake
Graphics

SOC

booted and Compute IO

in the lab

*Graphics for illustrative purposes only and not to scale.

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 34
 One Architecture – Multiple Performance Points

<10W >100W

• Small Graphics Tile • Max Config GPU Tile

• Efficient CPU Tile • Max Config CPU Tile
• Reduced IO Tile • Expanded IO Tile

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 35
 Scalable Architecture across Multiple Generations

Meteor Lake Arrow Lake Lunar Lake & Beyond

Packaging ▪ Foveros ▪ Foveros ▪ Foveros

▪ 36 µm pitch ▪ 36 µm pitch ▪ 25 µm pitch

Ext- Ext- Ext-

Process ernal
ernal ernal

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 36
 Future of Client
“Experience First” Client
drives New Era of System level integration

Monolithic performance with

disaggregated benefits

Process, packaging and architecture

together make this possible

Meteor Lake first tiled disaggregated

architecture on Intel 4

Architecture is extremely flexible and

scales across design points and into future

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 37
 Future of
Compute
Scalable Architecture,
Construction and Packaging

Large range of Thermal and

Performance Envelope

Flexible across process nodes,

Monolithic benefits

3D Monolithic
Multi-layer, Logic on Logic

Manufacturing at Scale for

the next Billion Devices

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 38
 Q&A
Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.
Department or Event Name Intel Confidential 39
All product and service plans, and roadmaps are subject to change without notice. Any forecasts of goods and services needed for Intel's operations are provided for discussion purposes only. Intel will have no liability to make
any purchase in connection with forecasts published in this document. Code names are often used by Intel to identify products, technologies, or services that are in development and usage may change over time. No license
(express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document. Product and technology performance varies by use, configuration and other factors. Learn more at
www.Intel.com/PerformanceIndex and www.Intel.com/ProcessInnovation.

The products and services described may contain defects or errors which may cause deviation from published specifications. Current characterized errata are available on request. Intel disclaims all express and implied
warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in
trade. Statements in this document that refer to future plans or expectations are forward-looking statements. These statements are based on current expectations and involve many risks and uncertainties that could cause
actual results to differ materially from those expressed or implied in such statements. For more information on the factors that could cause actual results to differ materially, see our most recent earnings release and SEC filings
at www.intc.com.

© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others. This document contains information on
products and technologies in development.

Authorized licensed use limited to: Transilvania University of Brasov. Downloaded on October 30,2023 at 09:55:50 UTC from IEEE Xplore. Restrictions apply.