Ms. Xinran Li

PhD student

The Hong Kong University of Science and Technology

My research in multi-agent reinforcement learning (MARL) focuses on developing scalable and robust algorithms to address real-world challenges. I tackle key issues in MARL, including efficient coordination and communication, effective exploration strategies, handling heterogeneous agent capabilities, and integrating large language models (LLMs) for enhanced planning and decision-making. MARL is essential for solving problems involving multiple interacting entities, such as autonomous vehicles navigating traffic or robotic teams collaborating in manufacturing. By optimizing collective behaviors through agent interaction and feedback, MARL provides a framework to handle complex systems that traditional single-agent approaches cannot address. Its ability to model emergent behaviors makes it vital for intelligent systems operating in dynamic, unpredictable environments. To improve communication efficiency and scalability, I developed several novel protocols. ExpoComm employs exponential graph topologies for rapid information dissemination with near-linear communication costs. CACOM uses context-aware message exchange with attention and quantization to address bandwidth limitations, while AC2C introduces adaptive two-hop communication to extend agent range in bandwidth-constrained scenarios. For effective exploration in cooperative MARL with sparse rewards, I created ICES, which uses Bayesian surprise for intrinsic motivation, guiding exploration with privileged global information while decoupling it from exploitation. To handle agent heterogeneity, I proposed Kaleidoscope, an adaptive parameter-sharing scheme with learnable masks that balances specialization and knowledge sharing, fostering both individual and collective intelligence. Lastly, I introduced LIET, a framework integrating LLMs for embodied multi-agent tasks. By introducing individual utility functions and evolving communication, it bridges the gap between individual action and global objective, enabling sophisticated coordination. My work continues to push the boundaries of MARL, aiming to develop practical, efficient algorithms for diverse real-world applications.