Multi-Agent Deep Reinforcement Learning for Beam Codebook Design in RIS-aided Systems

Asmaa Abdallah, Abdulkadir Celik, Mohammad M. Mansour, Ahmed M. Eltawil

Research output: Contribution to journalArticlepeer-review


Reconfigurable intelligent surfaces (RISs) play a vital role in future wireless systems with the capability of enhancing propagation environments by intelligently reflecting the signals toward the target receivers. However, optimal tuning of the phase shifters at the RIS is challenging due to the passive nature of reflective elements and the high complexity of acquiring channel state information (CSI). Furthermore, the joint active beamforming and RIS reflection beam design is a tedious task due to the high computational complexity and the dynamic nature of the wireless environment. Today’s cellular networks establish data transmission by relying on pre-defined generic beamforming codebooks, which are neither site-specific nor adaptive to the changes in the wireless environment. Moreover, identifying the best beam is typically performed using an exhaustive search approach that prohibits the use of large codebook sizes due to the resulting high beam training overhead. Depending merely on the binary received signal strength, this work develops a multi-agent deep reinforcement learning (MA-DRL) framework that jointly designs the active and the passive reflection beam codebooks for the BS and the RIS, reflectively. To accelerate learning convergence and reduce the search space, the proposed model divides the RIS into multiple partitions and associates beam patterns to the surrounding environments with low computational complexity. Moreover, a hierarchical beam training solution is proposed to further reduce the beam training overhead of the single-beam training approach. Simulation results show that the proposed MA-DRL approach can provide a 97% beam training overhead reduction over the discrete Fourier transform (DFT) codebook.

Original languageEnglish (US)
Pages (from-to)1
Number of pages1
JournalIEEE Transactions on Wireless Communications
StateAccepted/In press - 2024


  • Array signal processing
  • Channel estimation
  • Discrete Fourier transforms
  • Millimeter wave communication
  • Reflection
  • Training
  • Wireless communication

ASJC Scopus subject areas

  • Computer Science Applications
  • Electrical and Electronic Engineering
  • Applied Mathematics


Dive into the research topics of 'Multi-Agent Deep Reinforcement Learning for Beam Codebook Design in RIS-aided Systems'. Together they form a unique fingerprint.

Cite this