3D OPIN-CHANZ connectivity #3323

soheilshahrouz · 2025-10-28T15:53:20Z

Removes the pin_offset attribute from the architecture file, which was previously used to model 3D OPIN connectivity.
Instead, the <fc_override> tag is now used to specify absolute fc values for connecting OPINs to CHANZ wire segments.

Additionally, several functions have been moved from rr_graph2.cpp to two new files: rr_graph_chan_chan_edges.cpp and rr_graph_opin_chan_edges.cpp.

…raph_sg

AmirhosseinPoolad

What a PR, Soheil. Had some thoughts, mostly on styling. I did not review rr_graph_chan_chan_edges.cpp/h and rr_graph_opin_chan_edges.cpp/h as you said it was only moving the code to a new file.

Also only reviewed add_edges_opin_chanz_per_side and add_edges_opin_chanz_per_block in rr_graph_sg for the same reason.

libs/librrgraph/src/base/get_parallel_segs.h

libs/librrgraph/src/base/rr_spatial_lookup.h

AmirhosseinPoolad · 2025-11-12T20:25:28Z

vpr/src/route/rr_graph_generation/build_scatter_gathers.cpp

                                  scatter_wire_candidates.size());

-                    continue;
+                    // continue;


Remove commented out code

AmirhosseinPoolad · 2025-11-12T20:27:32Z

vpr/src/route/rr_graph_generation/clock_network_builders.cpp

+    seg_idx = get_parallel_seg_index(drive_seg_idx, indices_map, e_parallel_axis::X_AXIS);
+    drive_seg_idx = (seg_idx >= 0) ? seg_idx : drive_seg_idx;

-    get_parallel_seg_index(left_seg_idx, e_parallel_axis::X_AXIS, indices_map);
+    seg_idx = get_parallel_seg_index(left_seg_idx, indices_map, e_parallel_axis::X_AXIS);
+    left_seg_idx = (seg_idx >= 0) ? seg_idx : left_seg_idx;

-    get_parallel_seg_index(right_seg_idx, e_parallel_axis::X_AXIS, indices_map);
+    seg_idx = get_parallel_seg_index(right_seg_idx, indices_map, e_parallel_axis::X_AXIS);
+    right_seg_idx = (seg_idx >= 0) ? seg_idx : right_seg_idx;


Rename drive_seg_idx, left_seg_idx and right_seg_idx to have underlines after them. Not clear in a glance that they're private fields and not consistent with the rest of the code base.

Explain the logic of this code. Not sure why we are doing the set either the reutnred value or the old value as the new value and that looks like new behaviour. It should be explained.

vpr/src/route/rr_graph_generation/rr_graph.cpp

AmirhosseinPoolad · 2025-11-12T22:36:31Z

vpr/src/route/rr_graph_generation/rr_graph_sg.h

+                                   t_rr_edge_info_set& rr_edges_to_create,
+                                   const vtr::NdMatrix<std::vector<t_bottleneck_link>, 2>& interdie_3d_links);
+
+void add_edges_opin_chanz_per_block(const RRGraphView& rr_graph,


vpr/src/route/rr_graph_generation/rr_types.h

AmirhosseinPoolad · 2025-11-12T22:40:56Z

vtr_flow/arch/multi_die/simple_arch/3d_k4_N4_90nm.xml

+        <!-- Gather 30 connections from the end of L4 wires at all four sides of a switchblock location -->
+        <wireconn num_conns="0" from_type="L1" from_switchpoint="1" side="rltb"/>
+      </gather>
+      <scatter>
+        <!-- Scatter 30 connections to the start of L4 wires at all four sides of a switchblock location -->
+        <wireconn num_conns="30" to_type="L1" to_switchpoint="0" side="rtlb"/>
+      </scatter>
+      <sg_link_list>
+        <!-- Link going up one layer, using the 'seg4_inter_die_driver' multiplexer to gather connections from the bottom layer and using the LZ node/wire to move up one layer -->
+        <sg_link name="L_UP" z_offset="1" x_offset="0" y_offset="0" mux="segz_driver" seg_type="LZ"/>
+        <!-- Link going down one layer, using the 'seg4_inter_die_driver' multiplexer to gather connections from the top layer and using the LZ node/wire to down up one layer -->
+        <sg_link name="L_DOWN" z_offset="-1" mux="segz_driver" seg_type="LZ"/>
+      </sg_link_list>
+      <!-- Instantiate 6 'L_UP' and 6 'L_DOWN' sg_links per switchblock location everywhere on the device -->


Update the comments here to represent the architecture.

AmirhosseinPoolad · 2025-11-12T22:42:19Z

vtr_flow/arch/multi_die/stratixiv_3d/3d_full_OPIN_inter_die_stratixiv_arch.timing.xml

+  <!-- Specify a scatter-gather pattern for 3D connections. -->
+  <scatter_gather_list>
+    <sg_pattern name="name" type="unidir">
+      <gather>
+        <!-- Gather 30 connections from the end of L4 wires at all four sides of a switchblock location -->
+        <wireconn num_conns="0" from_type="L4" from_switchpoint="3" side="rltb"/>
+      </gather>
+      <scatter>
+        <!-- Scatter 30 connections to the start of L4 wires at all four sides of a switchblock location -->
+        <wireconn num_conns="30" to_type="L4" to_switchpoint="0" side="rtl"/>
+      </scatter>
+      <sg_link_list>
+        <!-- Link going up one layer, using the 'seg4_inter_die_driver' multiplexer to gather connections from the bottom layer and using the LZ node/wire to move up one layer -->
+        <sg_link name="L_UP" z_offset="1" x_offset="0" y_offset="0" mux="seg4_inter_die_driver" seg_type="LZ"/>
+        <!-- Link going down one layer, using the 'seg4_inter_die_driver' multiplexer to gather connections from the top layer and using the LZ node/wire to down up one layer -->
+        <sg_link name="L_DOWN" z_offset="-1" mux="seg4_inter_die_driver" seg_type="LZ"/>
+      </sg_link_list>
+      <!-- Instantiate 6 'L_UP' and 6 'L_DOWN' sg_links per switchblock location everywhere on the device -->
+      <sg_location type="EVERYWHERE" num="60" sg_link_name="L_UP"/>
+      <sg_location type="EVERYWHERE" num="60" sg_link_name="L_DOWN"/>
+    </sg_pattern>
+  </scatter_gather_list>


Update comments.

Make sure there is a comment at the top of the arch file explaining the 3D part.

vaughnbetz

Nice cleanup and overall well structured code. Needs more commenting though in places.

libs/librrgraph/src/base/get_parallel_segs.h

vpr/src/base/setup_vpr.cpp

vpr/src/route/router_lookahead/router_lookahead_map_utils.cpp

vaughnbetz · 2025-11-14T22:43:04Z

vpr/src/route/rr_graph_generation/clock_network_builders.cpp

+    seg_idx = get_parallel_seg_index(drive_seg_idx, indices_map, e_parallel_axis::X_AXIS);
+    drive_seg_idx = (seg_idx >= 0) ? seg_idx : drive_seg_idx;

-    get_parallel_seg_index(left_seg_idx, e_parallel_axis::X_AXIS, indices_map);
+    seg_idx = get_parallel_seg_index(left_seg_idx, indices_map, e_parallel_axis::X_AXIS);
+    left_seg_idx = (seg_idx >= 0) ? seg_idx : left_seg_idx;

-    get_parallel_seg_index(right_seg_idx, e_parallel_axis::X_AXIS, indices_map);
+    seg_idx = get_parallel_seg_index(right_seg_idx, indices_map, e_parallel_axis::X_AXIS);
+    right_seg_idx = (seg_idx >= 0) ? seg_idx : right_seg_idx;


Explain the logic of this code. Not sure why we are doing the set either the reutnred value or the old value as the new value and that looks like new behaviour. It should be explained.

vpr/src/route/rr_graph_generation/rr_graph_sg.cpp

vaughnbetz · 2025-11-15T00:06:15Z

vtr_flow/arch/multi_die/simple_arch/3d_k4_N4_90nm.xml

+    <sg_pattern name="name" type="unidir">
+      <gather>
+        <!-- Gather 30 connections from the end of L4 wires at all four sides of a switchblock location -->
+        <wireconn num_conns="0" from_type="L1" from_switchpoint="1" side="rltb"/>


Should num_conns be 30 instead of 0 here?

vaughnbetz · 2025-11-15T00:06:54Z

vtr_flow/arch/multi_die/simple_arch/3d_k4_N4_90nm.xml

-          <loc side="top" layer_offset="1">io.outpad io.inpad io.clock</loc>
-          <loc side="right" layer_offset="1">io.outpad io.inpad io.clock</loc>
-          <loc side="bottom" layer_offset="1">io.outpad io.inpad io.clock</loc>
+          <loc side="left">io.outpad io.inpad io.clock</loc>


Add a comment at the top of this arch file explaining what it is.

vaughnbetz · 2025-11-15T00:08:12Z

vtr_flow/arch/multi_die/stratixiv_3d/3d_full_OPIN_inter_die_stratixiv_arch.timing.xml

+    <sg_pattern name="name" type="unidir">
+      <gather>
+        <!-- Gather 30 connections from the end of L4 wires at all four sides of a switchblock location -->
+        <wireconn num_conns="0" from_type="L4" from_switchpoint="3" side="rltb"/>


Should this be num_conns = 30?

vaughnbetz · 2025-11-15T00:08:35Z

vtr_flow/arch/multi_die/stratixiv_3d/3d_full_OPIN_inter_die_stratixiv_arch.timing.xml

+  <!-- Specify a scatter-gather pattern for 3D connections. -->
+  <scatter_gather_list>
+    <sg_pattern name="name" type="unidir">
+      <gather>
+        <!-- Gather 30 connections from the end of L4 wires at all four sides of a switchblock location -->
+        <wireconn num_conns="0" from_type="L4" from_switchpoint="3" side="rltb"/>
+      </gather>
+      <scatter>
+        <!-- Scatter 30 connections to the start of L4 wires at all four sides of a switchblock location -->
+        <wireconn num_conns="30" to_type="L4" to_switchpoint="0" side="rtl"/>
+      </scatter>
+      <sg_link_list>
+        <!-- Link going up one layer, using the 'seg4_inter_die_driver' multiplexer to gather connections from the bottom layer and using the LZ node/wire to move up one layer -->
+        <sg_link name="L_UP" z_offset="1" x_offset="0" y_offset="0" mux="seg4_inter_die_driver" seg_type="LZ"/>
+        <!-- Link going down one layer, using the 'seg4_inter_die_driver' multiplexer to gather connections from the top layer and using the LZ node/wire to down up one layer -->
+        <sg_link name="L_DOWN" z_offset="-1" mux="seg4_inter_die_driver" seg_type="LZ"/>
+      </sg_link_list>
+      <!-- Instantiate 6 'L_UP' and 6 'L_DOWN' sg_links per switchblock location everywhere on the device -->
+      <sg_location type="EVERYWHERE" num="60" sg_link_name="L_UP"/>
+      <sg_location type="EVERYWHERE" num="60" sg_link_name="L_DOWN"/>
+    </sg_pattern>
+  </scatter_gather_list>


Make sure there is a comment at the top of the arch file explaining the 3D part.

amin1377

Looks good, Soheil! Thanks! I don’t have much to add to what Amir and Vaughn already mentioned. Just one question: have you explained fc_override and how it helps with OPIN connections to other layers in the document? Thanks!

amin1377 · 2025-11-14T20:36:07Z

libs/librrgraph/src/base/rr_graph_utils.cpp

-                    limited_to_opin = false;
-                    break;
-                }
+            e_rr_type to_type = rr_graph.node_type(to_node);


It’s not directly related to this section, but it occurred to me while reading this part. I think we assume that CHANZ is definitely stretched across multiple layers. If that’s correct, do we have a check for this in the RR graph validation to ensure that assumption holds?

…en setting fc

…types

soheilshahrouz added 5 commits October 10, 2025 15:13

clean some comments

c3ea108

clean get_track_to_pins a bit

cd680d7

remove direct pin to other layer's track connectivity

8294d2d

remove arch_inter_die_switch

6857133

Merge remote-tracking branch 'origin/master' into temp_opin_chanz

271ed58

github-actions bot added VPR VPR FPGA Placement & Routing Tool libarchfpga Library for handling FPGA Architecture descriptions lang-cpp C/C++ code labels Oct 28, 2025

soheilshahrouz added 9 commits October 28, 2025 15:53

remove pin layer offset and ipin_cblock_switch_index_between_dice

bd51bd9

remove layer_offset from rr_graph.cpp

2c37b80

Merge remote-tracking branch 'origin/master' into temp_opin_chanz

5e5b118

doxygen comment for process_fc()

c5d509c

Merge branch 'temp_arch_xml_naming_convention' into temp_opin_chanz

a9b490a

doxygen comment for label_wire_muxes()

1f13be8

Fc for OPIN-CHANZ can be 1

4104e23

update the 3d_full_opin arch file

027ab8b

add OPIN--CHANZ edges

9210a3d

github-actions bot added the libvtrutil label Nov 1, 2025

soheilshahrouz added 12 commits November 1, 2025 15:03

move get_parallel_seg_index() to get_parallel_sets.cpp

f747061

move add_edges_opin_chanz to rr_graph_3d

230fead

remove pins_connected() function and rename rr_graph_3d files to rr_g…

0097c56

…raph_sg

move add_and_connect_non_3d_sg_links() to rr_graph_sg

d2af592

noexcept for strongid methods

21dfd5d

use the correct layer for CHANZ nodes when initalizing src_opin_delays

6803792

allow OPIN to be connected to CHANZ in check_adjacent()

1ede4c4

make format

ea14ba1

add find_pin_nodes_at_side() to RRSpatialLookup

78afdc2

add add_edges_opin_chanz_per_side()

0dcb2fa

update inter_layer_connections_limited_to_opin() to check OPIN-CHANZ

72e130c

some cleanups in rr_graph files

3610503

soheilshahrouz added 6 commits November 5, 2025 17:26

add rr_chanz_segment_width to DeviceContext

63cbe06

calculate acc_tile_num_inter_die_conn_ using pre-computed chanz width

5d477c1

Merge remote-tracking branch 'origin/master' into temp_opin_chanz

27cbc2a

update expected_delay_cost before VTR_ASSERT

eadc8fe

add sg and z segment to 3d_k4_n4_90nm

2657db0

remove duplication get_parallel_seg_index() function

55db1c8

soheilshahrouz changed the title ~~[WIP] 3D OPIN-CHANZ connectivity~~ 3D OPIN-CHANZ connectivity Nov 10, 2025

soheilshahrouz requested review from AmirhosseinPoolad, amin1377 and vaughnbetz November 10, 2025 23:00

soheilshahrouz added 2 commits November 10, 2025 18:09

update golden results for vtr_strong/strong_ap/basic_3d_ap/

644fe8a

don't update segment index if not found

e76138f

AmirhosseinPoolad reviewed Nov 12, 2025

View reviewed changes

vaughnbetz requested changes Nov 15, 2025

View reviewed changes

amin1377 requested changes Nov 15, 2025

View reviewed changes

soheilshahrouz added 13 commits November 24, 2025 18:59

Merge remote-tracking branch 'origin/master' into temp_opin_chanz

4f88fed

make format

21b7215

doxygen comment for get_parallel_seg_index

17d5145

doxygen comment for find_pin_nodes_at_side()

088f075

remove layer from indexing comments

b199b7f

prime_factors uses std::vecotr & commenting style

f4b17f0

rename ofs to offset & 4 --> e_side::num_2d_sides

19e337b

add underline to private members vars of ClockRib

221fafe

explain why CHANZ wires are excluded in alloc_and_load_actual_fc() wh…

620ded6

…en setting fc

std::array for sb_loc instead of two vars

00cf6ec

remove layer from indexing comment of pin-track and track-pin lookup …

c731f9a

…types

make format

4794d13

add comments

2d833bf

AmirhosseinPoolad mentioned this pull request Nov 25, 2025

[Spelling] Fixed Simple Spelling Mistakes in Route Dir #3360

Merged

helper functions for add_edges_opin_chanz_per_side()

9a5aa94

3D OPIN-CHANZ connectivity #3323

Are you sure you want to change the base?

3D OPIN-CHANZ connectivity #3323

Conversation

soheilshahrouz commented Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AmirhosseinPoolad left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vaughnbetz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amin1377 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

soheilshahrouz commented Oct 28, 2025 •

edited

Loading