Exploiting 2.5D/3D Heterogeneous Integration For AI Computing

Table of Contents

References

Exploiting 2.5D/3D Heterogeneous Integration for AI Computing¹

Benchmarking Heterogeneous Integration with 2.5D/3D Interconnect Modeling²

End-to-End Benchmarking of Chiplet-Based In-Memory Computing³

Summary

HISIM⁴, a modeling and benchmarking tool for heterogeneous integration of chiplets by communicating through NoP⁵. Components: partitioning, mapping and placement; computing unit/processing unit; heterogeneous interconnection; network/routing engine; thermal analysis. technology roadmap⁶, power/latency prediction, thermal analysis for electro-thermal co-design and cycle-accurate simulation for design space exploration.

For 3D interconnection, simplified TSV model for RC extraction.

generalize technology roadmap and electro-modeling for 2.5D/3D interconnect, they are comparable with on-chip interconnect.

Novelties

It is the 1st to support hardware performance evaluation of chiplets architecture. handle monolithic, 2.5D and 3D architectures for AI models at the same time to compare the performance, optimize the configuration.
Integrate In-Memory-Computing for AI models.

Source Code

Placement:

place_1: tiers determined by tiles per tier int(math.sqrt(N_tile));

place_5: tiles per tier determined by num of tiers; mesh_edge=int(math.sqrt(N_tile))

Variable	Explanation	Relationships
`computing_data[i][1]`	tiles for the ML layer
`computing_data[i][9]`	specific chiplet	`computing_data`
`computing_data[i][14]`	flops
placement⁷	1: calc tier, 2.5D⁸ 5: calc tiles per tier⁹
placement	Tier Edge to Edge from the bottom to top tier1 the hotspot far from each other put all hotspot in the same place tile-to-tile 3D connection
`layer_start_tile` `layer_end_tile`¹⁰	: tiles per tier given¹¹ :¹²	`computing_data[i][9]` `computing_data[i][1]`¹³ `layer_start_tile_tier`
`each_tile_activation_Q`¹⁴		`computing_data[i][8]` `computing_data[i][1]`
`tile_total`¹⁵	`layer`¹⁶ `tile` array `tile` element	`tile_index` `[[x,y,computing_data[i][9]]]` last item `each_tile_activation_Q`*3
`empty_tile_total`	depends on placement [[[x0,y0,tier], [x0,y1,tier],…]]	tile_index¹⁷

`Q_3d_scatter`¹⁸
`layer_HOP_2d`¹⁹ `layer_HOP_3d`²⁰	→ ↑
Local/Global Routing `layer_Q`²¹ `Q_3d`²² `Q_2d`²² one tile connects to all next level tiles	Local Routing Q_3d+=(tile_total[i][-1][2])*(len(tile_total[i+1])-1) $\frac{Q_{t i e r}}{N_{l a y e r}} \cdot N_{l a y e r_{n} e x t}$ Global Routing Q_3d²³), Q_2d	`each_tile_activation_Q`
`total_router_area`²⁴	Orion²⁵	single_router_area`¹` edge_single_router channel_width
`area_2_5d`	aib edge length²⁶ `layer_Q_2_5d`²⁷	`layer_aib`²⁸ `aib_out`
L_booksim_2d²⁹ L_booksim_3d³⁰ `L_2_5d`³¹	Latecny from Booksim³² delay factors³³	`aib_out[2]` `power_summary_router`
power `total_2d_channel_power` `total_2d_router_power` `total_tsv_channel_power` `total_3d_router_power`	Latecny from Booksim³² `power_summary_router` func³⁴
`tier_2d_hop_list_power` `tier_3d_hop_list_power` `tier_2d_hop_list`³⁵ `tier_3d_hop_list`
`total_energy` `energy_2d`³⁶ `energy_3d`³⁷ `energy_2_5d`³⁸		L_booksim_2d L_booksim_3d

network_model.py

Variable	Explanation	Relationships
`power_router` `power_tsv`	not placement 2³⁹ placement 2⁴⁰ not placement 2⁴¹ placement 2⁴²	`tier_2d_hop_list_power` `tier_3d_hop_list_power`
`imc_size` `r_size`		`area_single_tile` `single_router_area`
thermal 3D	`get_unitsize()` `create_cube()` `load_power()` `conductance_G()` `solver()`
thermal 2.5D

thermal_model.py

Components

Component	State-of-the-Art	This Work

Interconnect		vv ff
Network	NoC: Booksim⁴³, Orion⁴⁴, Nirgram⁴⁵⁴⁶ 3D NoC: Ratatoskr⁴⁷	cycle-accurate link delay

Category	Parameter	Description
mono/2.5D	$l_{w i r e}$	Length of wire
mono/2.5D	$w_{w i r e}$	Width of wire
mono/2.5D	$t_{w i r e}$	thickness of wire
mono/2.5D	$p_{w i r e}$	pitch of wire
3D	$d_{T S V}$	TSV metal diameter
3D	$h_{T S V}$	TSV height
3D	$p_{T S V}$	TSV pitch
3D	$t_{o x_{T} S V}$	Thickness of TSV insulator

2D, 2.5D wire and 3D TSV interconnect parameters

TSV

TSV component lump circuit is based on the paper Modeling and Analysis of Through-Silicon Via (TSV) Noise Coupling and Suppression Using a Guard Ring⁴⁸

For the TSV model in cylinder shape, 4 $C_{TSV}$ on horizontal directions in parallel, 2 $R_{TSV}$ , $L_{TSV}$ , $M_{TSV}$ on vertical directions in series. In Benchmarking Heterogeneous Integration with 2.5D/3D Interconnect Modeling², only resistance and capacitance are considered.

Resistance

\left\{\begin{matrix}<br>R_{\mathrm{TSV}}=\frac{1}{2} \frac{1}{\sigma_{\mathrm{TSV}}} \frac{h_{\text {unit }}}{\pi r_{\mathrm{TSV}}{ }^{2}}\text{ , if }\sigma_{\text{TSV}}\ge r_{\text{TSV}}\\ <br>R_{\mathrm{TSV}}=\frac{1}{2} \frac{1}{\sigma_{\mathrm{TSV}}} \frac{h_{\text {unit }}}{\pi\left(r_{\mathrm{TSV}}^{2}-\left(r_{\mathrm{TSV}}-\delta_{\mathrm{TSV}}\right)^{2}\right)}\text{ , if }\sigma_{\text{TSV}}\lt r_{\text{TSV}}<br>\end{matrix}\right.

Where skin depth is $δ_{TSV} = \frac{1}{\sqrt{π f μ σ_{TSV}}} [m]$

Self Inductance

L_{\mathrm{TSV}}=\frac{1}{2}\left(L\left(r_{\mathrm{TSV}}\right)-L\left(p_{\mathrm{TSV}-\mathrm{TSV}}\right)\right) \frac{h_{\text {unit }}}{h_{\mathrm{TSV}}}[H]

L(x)=\frac{\mu h_{\mathrm{TSV}}}{2 \pi}\left[\ln \left(\left(\frac{h_{\mathrm{TSV}}}{x}\right)+\sqrt{\left(\frac{h_{\mathrm{TSV}}}{x}\right)^{2}+1}\right)+\frac{x}{h_{\mathrm{TSV}}}-\sqrt{\left(\frac{x}{h_{\mathrm{TSV}}}\right)^{2}+1}\right]

Mutual Inductance

M_{\mathrm{TSV}}= \frac{1}{2}\left(L\left(2 r_{\mathrm{TSV}}+d_{\mathrm{TSV}-\mathrm{TSV}}\right)\right) \frac{h_{\text {unit }}}{h_{\mathrm{TSV}}} -\frac{1}{2} L\left(\sqrt{p_{\mathrm{TSV} \_\mathrm{TSV}}^{2}+\left(2 r_{\mathrm{TSV}}+d_{\mathrm{TSV} \_\mathrm{TSV}}\right)^{2}}\right) \frac{h_{\text {unit }}}{h_{\mathrm{TSV}}}

L(x)=\frac{\mu h_{\mathrm{TSV}}}{2 \pi}\left[\ln \left(\left(\frac{h_{\mathrm{TSV}}}{x}\right)+\sqrt{\left(\frac{h_{\mathrm{TSV}}}{x}\right)^{2}+1}\right)+\frac{x}{h_{\mathrm{TSV}}}-\sqrt{\left(\frac{x}{h_{\mathrm{TSV}}}\right)^{2}+1}\right]

Capacitance

C_{\mathrm{TSV}}=\frac{1}{4}\cdot \frac{2 \pi \varepsilon_{0} \varepsilon_{r, \text { ox\_TSV }} h_{\text {sub }}}{\ln \left(\frac{r_{\mathrm{TSV}}+t_{\text {ox\_TSV }}}{r_{\mathrm{TSV}}}\right)} \cdot\frac{h_{\text {unit }}}{h_{\text {sub }}}[F]

#Cu TSV skin depth at 1GHz
import math
freq=1e9
sigma=5.8e7
mu=4e-7*3.14159
TSV_skin_depth=1/(2*3.14159*freq*mu*sigma)**0.5
print("TSV_skin_depth=",TSV_skin_depth)
r_tsv=5e-6
d_tsv=2*r_tsv
h_tsv=100e-6
R_tsv=0.5*(1/TSV_skin_depth)*( (1/math.pi)*h_tsv*1/(r_tsv**2-(r_tsv-TSV_skin_depth)**2) )
print("R_tsv=",R_tsv)