SPACX hybrid Network on Chip (NoC) Simulation

This project simulates the Optical plus Electrical Network on Chip (NoC) architecture SPACX using SystemC TLM 2.0

Prerequisites

• SystemC (version 2.3.3 or later)
• C++ compiler supporting C++17 or later (e.g., GCC 7+ or Clang 5+)
• Make

Setup

1. Install SystemC on your system.

2. Set the SYSTEMC_HOME environment variable to point to your SystemC installation directory:

   export SYSTEMC_HOME=/path/to/your/systemc/installation
   
   

Compilation

./build.sh

Output simple test - 1 Task

./sim --configFolder simple_test

Output - 2 Task

./sim --configFolder simple_2_point_test

Output - to test all type of communication

./sim --configFolder all_type_data_test or ./sim --configFolder all_type_test_1

Output log files

• /out/ni_gb_comm.log It gives the log information of the communication between global buffer and the Ratatoskr PE which is responsible for all the actions of GB. PE of GB generates all the inputs and accepts the output features, which is communicated back and forth between the GB module and PE through the network interface is given in the file ni_gb_comm.log (stands for network Interface of global buffer communication log)
• /out/gb_comm.log Gives the log information of all activity by the global buffer module, such as receiving the data from parent Ratatoskr PE, sending them as single chiplet or cross chiplet transactions to all chiplets, receiving the output features from the PEs on the chiplet and so on.
• /out/interposer_comm.log : communication to and from interposer interface
• /out/chiplet_comm.log : communication to and from chiplet interface
• /out/oec_comm.log : communication to and from Optical to Electrical or Electrical to Optical module
• /out/ni_comm.log
• /out/report.log : general communication like token transfer
• /out/pe_comm.log : communication to and from Ratatoskr PE
• /out/nm_conn.log
• /out/noc_setup.log

Types of communications involved are

• Single chiplet communication
  • Sending input features from GB to every PE on each chiplet
• Cross chiplet communication
  • Sending weight kernels from GB to particular PE on every chiplet
  • Sending data from one PE to another through GB
• Unicast communication
  • Output feature communication from each PE to GB

Project directory structure

SPACX

• src
  • GB
    • Global_Buffer.h
    • Global_Buffer.cpp
  • Interposer
    • Interposer_Interface.h
    • Interposer_Interface.cpp
  • Chiplet
    • Chiplet_Interface.h
    • Chiplet_Interface.cpp
  • Optical_Electrical
    • O_E_converter.h
    • O_E_converter.cpp

  - networkInterface
    - NetworkInfaceGB.h
    - NetworkInfaceGB.cpp
    - NetworkInfacePE.h
    - NetworkInfacePE.cpp
  

  • networkManager
    • NetworkManager.h
    • NetworkManager.cpp
  • noc
    • noc.h
    • noc.cpp
  • ratatoskrUtils
  • Utils
    • configuration.h
    • noc_logger.h
    • noc_logger.cpp
    • token_manager.h
• scripts
  • input_feature.py
  • weight_kernal.py
• out
• obj
• main.cpp
• Makefile
• README.md

SPACX initial implementation

• The Global Buffer is initializing all the transactions
• In the beggining GB generates weights and input features as matrices and convert them to payloads
• The GB is connected to interposer interface to accept the payload through TLM blocking communication
• The address is the wavelegth, which is encoded into the payload and transmitted
• The interposer interface simply diverts the payload to chiplet interface again as TLM blocking communication
• The chiplet interface is connected to all PEs in a chiplet through the optical to electrical converter
• The chiplet interface copies the data to the curresponding OEC based on the address that is the wavelength
• Once the data is in the OEC, they are simply forwarding them to the PEs using the TLM blocking communication
• In PEs the data is stored into 2 fifos, the weight buffer and the buffer for input features
• The PE then multiplies it and gets the Psum stored into the accumulation buffer
• The Psums are accumulated locally to obtain the output features
• once the Output Feature (OF) value is ready, PE weights for the token, and then gives the data to the OEC
• OEC then transfers the data to the chiplet interface, and then to interposer interface in backward
• This way the OFs are back at global buffer through multiple TLM blocking transport

SPACX architecture and its operation

1. SPACX Overview: SPACX leverages photonic interconnects for efficient data communication in chiplet-based architectures, focusing on the needs of DNN inference, such as broadcast communication. It uses a hierarchical design:

• Global waveguide: Connects all chiplets.
• Local waveguide: Exists within each chiplet for intra-chiplet communication.

2. Key Components: Chiplets and Processing Elements (PEs):

• Each chiplet has multiple PEs (e.g., 8 per chiplet in the example).
• PEs perform computations like Multiply-and-Accumulate (MAC) for DNN workloads.
• Each PE has:
  • Two receivers for incoming broadcast data (local and global).
  • One transmitter for sending computed results back to the global buffer (GB). Wavelengths and Waveguides:
• Data transmission uses wavelength-division multiplexing (WDM).
• Wavelength allocation:
  • Global (cross-chiplet) wavelengths: Used for broadcasting the same data to corresponding PEs across all chiplets.
  • Local (single-chiplet) wavelengths: Used for intra-chiplet broadcasts and communication between chiplet PEs and the GB. Optical Splitters and Filters:
• Optical splitters: Split light (data) across multiple destinations by controlling the split ratio. Used for broadcasts.
• Optical filters: Select or forward specific wavelengths for targeted communication.

3. Communication Mechanisms: A. Cross-Chiplet Broadcast (Global Waveguide):

• Example: Data for PE0 across all chiplets is sent using wavelength λ0.
• Process:
  • The GB modulates data onto λ0.
  • Splitters in the global waveguide distribute power to each chiplet's local waveguide.
  • Local waveguides deliver the data to PE0 on each chiplet. B. Single-Chiplet Broadcast (Local Waveguide):
• Example: Data for all PEs within a single chiplet (e.g., Chiplet0) is sent using wavelength λ8.
• Process:
  • The GB modulates data onto λ8.
  • Optical filters direct all power of λ8 from the global waveguide to the local waveguide.
  • Splitters on the local waveguide evenly distribute data to all PEs within Chiplet0. C. PE-to-GB Communication:
• Each chiplet's PEs share a single wavelength for sending computed results back to the GB (e.g. λ8 for Chiplet0).
• Token-based arbitration ensures orderly transmission:
  • A token is passed sequentially among PEs, granting transmission access to one PE at a time.