Configuration Space Shadow BRAM Implementation

This document describes the implementation of the full 4 KB configuration space shadow in BRAM for the PCILeech FPGA firmware generator.

Overview

The configuration space shadow BRAM implementation provides a complete 4 KB PCI Express configuration space in block RAM (BRAM) on the FPGA. This is a critical component for PCIe device emulation, as it allows the PCILeech firmware to accurately respond to configuration space accesses from the host system.

Key features:

  • Full 4 KB configuration space shadow in BRAM
  • Dual-port access for simultaneous read/write operations
  • Overlay RAM for writable fields (Command/Status registers)
  • Initialization from device configuration data or synthetic generation
  • Little-endian format compatible with PCIe specification
  • Automatic overlay mapping for writable registers

Architecture

The implementation consists of the following components:

  1. Configuration Space BRAM: A 4 KB block RAM that stores the entire configuration space of the emulated PCIe device.
  2. Overlay RAM: A smaller RAM that stores writable fields, allowing the host to modify certain configuration registers.
  3. State Machine: Handles PCIe configuration space access requests (reads and writes).
  4. Overlay Mapper: Automatically detects which registers need overlay entries based on PCIe specifications.

SystemVerilog Modules

  • pcileech_tlps128_cfgspace_shadow.sv: The main module implementing the configuration space shadow.
  • pcileech_tlps128_bar_controller.sv: The BAR controller that interfaces with the configuration space shadow.

Configuration Space Generation

The configuration space is generated through multiple methods depending on the available data:

1. VFIO-based Configuration Space Reading

For devices bound to VFIO drivers, the system reads the actual configuration space:

  1. The ConfigSpaceManager class handles VFIO device binding and configuration space access.
  2. Configuration space is read via VFIO region info queries or sysfs fallback.
  3. The raw configuration space data is extracted and validated.
  4. Extended configuration space (up to 4 KB) is supported when available.

2. Synthetic Configuration Space Generation

When VFIO data is not available, the system generates synthetic configuration space:

  1. The ConfigSpaceManager.generate_synthetic_config_space() method creates a complete 4 KB configuration space.
  2. Device identification data is populated from device profiles.
  3. Standard PCI header fields are generated with appropriate defaults.
  4. Capability structures (MSI, MSI-X, PCIe) are added based on device requirements.
  5. BAR configurations are populated based on device analysis.

3. Configuration Space Initialization

The generated configuration space is converted to FPGA-compatible format:

  1. The ConfigSpaceHexFormatter class converts binary data to hex format.
  2. Data is formatted as 32-bit words in little-endian format.
  3. A config_space_init.hex file is generated for SystemVerilog $readmemh initialization.
  4. Comments are added for debugging and register identification.

Overlay RAM for Writable Fields

The overlay RAM provides a mechanism for handling writable fields in the configuration space. The OverlayMapper class automatically detects which registers need overlay entries based on PCIe specifications:

Automatic Overlay Detection

The system automatically identifies writable registers:

  • Standard PCI registers: Command, Status, Cache Line Size, Latency Timer, BIST
  • BAR registers: Detected as special handling type with size-based masks
  • Capability registers: MSI, MSI-X, PCIe capability registers with mixed read/write fields
  • Extended capabilities: AER, Power Management, and other extended capabilities

Overlay Operation

When reading from a register with writable fields:

  1. The base value is read from the main configuration space BRAM
  2. The overlay mask determines which bits come from overlay RAM
  3. The final value combines: (base_value & ~mask) | (overlay_value & mask)

When writing to a register with writable fields:

  1. Only the writable bits (defined by the mask) are updated in overlay RAM
  2. Read-only bits remain unchanged in the base configuration space

Integration with Build Process

The configuration space shadow is integrated into the build process through the PCILeechGenerator and template system:

Template-based Generation

  1. Template Context Building: The PCILeechContextBuilder creates a comprehensive template context containing:
    • Device configuration data from VFIO or synthetic generation
    • Overlay mapping automatically generated by OverlayMapper
    • Extended configuration space pointers
    • MSI-X and capability configurations
  2. SystemVerilog Generation: The AdvancedSVGenerator processes templates to create:
    • Configuration space shadow module (cfg_shadow.sv)
    • Configuration space initialization file (config_space_init.hex)
    • Overlay constants and lookup tables
  3. Hex File Generation: The ConfigSpaceHexFormatter converts configuration space data to:
    • Little-endian 32-bit words
    • Vivado-compatible hex format
    • Debug comments for register identification

Build Integration

  1. The configuration space shadow module is automatically included in the generated TCL script.
  2. The config_space_init.hex file is included in the project for BRAM initialization.
  3. Overlay constants are generated as SystemVerilog parameters for efficient lookup.

SystemVerilog Implementation Details

Configuration Space Shadow Module

The main module (pcileech_tlps128_cfgspace_shadow) implements:

  • Dual-port BRAM: Main configuration space storage with parameterized size
  • Overlay RAM: Separate storage for writable fields with automatic indexing
  • State Machine: Handles PCIe configuration reads/writes with overlay logic
  • Shadow Control: Determines when to use shadow vs. hardware based on address ranges

Key Features

  • Automatic Overlay Mapping: Lookup tables generated from OVERLAY_MAP template data
  • Byte-enable Support: Proper handling of partial register writes
  • Extended Configuration Space: Support for capabilities beyond standard 256 bytes
  • CFGTLP Integration: Proper integration with PCIe configuration TLP handling

Testing

The implementation includes comprehensive testing:

  1. Python Unit Tests:
    • test_config_space_manager.py: Tests for configuration space reading and generation
    • test_overlay_mapper.py: Tests for overlay detection and mapping
    • test_hex_formatter.py: Tests for hex file generation and formatting
  2. Integration Tests:
    • VFIO binding and configuration space access
    • Synthetic configuration space generation
    • Template rendering and context building

Usage

The configuration space shadow is automatically included in the build process. The system:

  1. Detects Device Configuration: Automatically reads from VFIO or generates synthetic data
  2. Maps Overlay Requirements: Automatically detects which registers need overlay entries
  3. Generates SystemVerilog: Creates the shadow module with proper initialization
  4. Integrates with Build: Includes all necessary files in the FPGA project

Configuration Options

The system supports various configuration options:

  • CONFIG_SPACE_SIZE: Parameterized size (256 bytes to 4 KB)
  • OVERLAY_ENTRIES: Number of overlay RAM entries (automatically calculated)
  • EXT_CFG_CAP_PTR: Extended capability pointer offset
  • DUAL_PORT: Optional dual-port BRAM configuration

Limitations

  • The current implementation supports a single PCIe function (function 0).
  • Overlay RAM has a practical limit based on FPGA resources.
  • Extended capabilities are limited to commonly used types.

Future Enhancements

Possible future enhancements include:

  • Support for multiple PCIe functions
  • Dynamic reconfiguration of the configuration space
  • Enhanced error handling and reporting
  • Support for device-specific extended capabilities