Matteo Bettini

Matteo Bettini

PhD Candidate

Prorok Lab

University of Cambridge

Hello! đź‘‹

I am Matteo, a PhD student in the Prorok Lab at the University of Cambridge.

With my supervisor, Amanda Prorok, I study resilience and heterogeneity in multi-agent and multi-robot systems. For my research, I employ techniques from the fields of Multi-Agent Reinforcement Learning and Graph Neural Networks.

Prior to my PhD, I investigated the problem of transport network design for multi-agent routing.

Download my CV.

Interests
  • Multi-Robot Systems
  • Heterogeneous Multi-Agent Learning and Coordination
  • Reinforcement Learning
  • Graph Neural Networks
Education

  • PhD in Computer Science, Present

    University of Cambridge

  • MPhil in Advanced Computer Science, 2021

    University of Cambridge

  • BEng in Computer Engineering, 2020

    Politecnico di Milano

Publications

To find relevant content, try searching publications or filtering using the buttons below.

*
All Heterogeneity Multi-Agent Reinforcement Learning Software library
When Is Diversity Rewarded in Cooperative Multi-Agent Learning?
Focusing on multi-agent task allocation problems, our goal is to study the question: What kinds of objectives are best suited for heterogeneous teams? We first consider an instantaneous, non-spatial setting where the global reward is built by two generalized aggregation operators: an inner operator that maps the N agents’ effort allocations on individual tasks to a task score, and an outer operator that merges the M task scores into the global team reward. We prove that the curvature of these operators determines whether heterogeneity can increase reward, and that for broad reward families this collapses to a simple convexity test. Next, we study heterogeneity in multi-agent reinforcement learning (MARL) and introduce Heterogeneous Environment Design (HED), a gradient-based algorithm that optimizes the parameter space of underspecified MARL environments to find scenarios where heterogeneity is advantageous.
2025 In Preprint
PDF Cite arXiv
When Is Diversity Rewarded in Cooperative Multi-Agent Learning?
The impact of behavioral diversity in multi-agent reinforcement learning
In this work, we employ diversity measurement and control paradigms to study the impact of behavioral heterogeneity in several facets of multi-agent reinforcement learning. Through experiments in team play and other cooperative tasks, we show the emergence of unbiased behavioral roles that improve team outcomes; how behavioral diversity synergizes with morphological diversity; how diverse agents are more effective at finding cooperative solutions in sparse reward settings; and how behaviorally heterogeneous teams learn and retain latent skills to overcome repeated disruptions.
2025 In Preprint
Cite Video Soccer trailer arXiv Code Ocean
The impact of behavioral diversity in multi-agent reinforcement learning
Heterogeneous Teams
The aim of this chapter is to provide an overview of heterogeneous teams, from a robotics perspective.
2024 In Encyclopedia of Robotics
Cite Springer link
Heterogeneous Teams
Controlling Behavioral Diversity in Multi-Agent Reinforcement Learning
We introduce Diversity Control (DiCo), a method able to control diversity to an exact value of a given metric by representing policies as the sum of a parameter-shared component and dynamically scaled per-agent components. By applying constraints directly to the policy architecture, DiCo leaves the learning objective unchanged, enabling its applicability to any actor-critic MARL algorithm. We theoretically prove that DiCo achieves the desired diversity, and we provide several experiments, both in cooperative and competitive tasks, that show how DiCo can be employed as a novel paradigm to increase performance and sample efficiency in MARL.
2024 In International Conference on Machine Learning (ICML)
PDF Cite Code Website OpenReview arXiv Poster Talk
Controlling Behavioral Diversity in Multi-Agent Reinforcement Learning
The Cambridge RoboMaster: An Agile Multi-Robot Research Platform
This article introduces a tightly integrated hardware, control, and simulation software stack on a fleet of holonomic ground robot platforms. Our robots, a fleet of customised DJI Robomaster S1 vehicles, offer a balance between small robots that do not possess sufficient compute or actuation capabilities and larger robots that are unsuitable for indoor multi-robot tests.
2024 In Distributed Autonomous Robotic Systems (DARS)
PDF Cite Code Video Docs arXiv
The Cambridge RoboMaster: An Agile Multi-Robot Research Platform
BenchMARL: Benchmarking Multi-Agent Reinforcement Learning
BenchMARL is a library for benchmarking Multi-Agent Reinforcement Learning (MARL) using TorchRL. BenchMARL allows to quickly compare different MARL algorithms, tasks, and models while being systematically grounded in its two core tenets: reproducibility and standardization.
2024 In Journal of Machine Learning Research (JMLR)
PDF Cite Code Docs Wandb Benchmarks Talk Lecture arXiv Poster Proceedings NeurIPS
BenchMARL: Benchmarking Multi-Agent Reinforcement Learning
TorchRL: A data-driven decision-making library for PyTorch
We propose TorchRL, a generalistic control library for PyTorch that provides well-integrated, yet standalone components. With a versatile and robust primitive design, TorchRL facilitates streamlined algorithm development across the many branches of Reinforcement Learning (RL) and control. We introduce a new PyTorch primitive, TensorDict, as a flexible data carrier that empowers the integration of the library’s components while preserving their modularity. TorchRL fosters long-term support and is publicly available on GitHub for greater reproducibility and collaboration within the research community.
2024 In International Conference on Learning Representations (ICLR) - Spotlight (top 5%)
PDF Cite Code Documentation arXiv OpenReview Poster
TorchRL: A data-driven decision-making library for PyTorch
System Neural Diversity: Measuring Behavioral Heterogeneity in Multi-Agent Learning
In this paper, we introduce System Neural Diversity (SND): a measure of behavioral heterogeneity for multi-agent systems where agents have stochastic policies. We discuss and prove its theoretical properties, and compare it with alternate, state-of-the-art behavioral diversity metrics used in cross-disciplinary domains. Through simulations of a variety of multi-agent tasks, we show how our metric constitutes an important diagnostic tool to analyze latent properties of behavioral heterogeneity.
2023 In Preprint
PDF Cite Code arXiv
System Neural Diversity: Measuring Behavioral Heterogeneity in Multi-Agent Learning
Heterogeneous Multi-Robot Reinforcement Learning
In this paper, we crystallize the role of heterogeneity in MARL policies. We introduce Heterogeneous Graph Neural Network Proximal Policy Optimization (HetGPPO), a paradigm for training heterogeneous MARL policies that leverages a Graph Neural Network for differentiable inter-agent communication. HetGPPO allows communicating agents to learn heterogeneous behaviors while enabling fully decentralized training in partially observable environments. Through simulations and real-world experiments, we show that: (i) when homogeneous methods fail due to strong heterogeneous requirements, HetGPPO succeeds, and, (ii) when homogeneous methods are able to learn apparently heterogeneous behaviors, HetGPPO achieves higher resilience to both training and deployment noise.
2023 In Autonomous Agents and Multiagent Systems (AAMAS)
PDF Cite Code Video arXiv Talk Poster
Heterogeneous Multi-Robot Reinforcement Learning
POPGym: Benchmarking Partially Observable Reinforcement Learning
We introduce Partially Observable Process Gym (POPGym), a two-part library containing (1) a diverse collection of 14 partially observable environments, each with multiple difficulties and (2) implementations of 13 memory model baselines – the most in a single RL library. Using POPGym, we execute the largest comparison across RL memory models to date.
2023 In International Conference on Learning Representations (ICLR)
PDF Cite Code arXiv OpenReview Poster
POPGym: Benchmarking Partially Observable Reinforcement Learning
VMAS: A Vectorized Multi-Agent Simulator for Collective Robot Learning
In this paper, we introduce the Vectorized Multi-Agent Simulator (VMAS). VMAS is an open-source framework designed for efficient MARL benchmarking. It comprises a vectorized 2D physics engine written in PyTorch and a set of twelve challenging multi-robot scenarios. Additional scenarios can be implemented through a simple and modular interface. We demonstrate how vectorization enables parallel simulation on accelerated hardware without added complexity.
2022 In Distributed Autonomous Robotic Systems (DARS)
PDF Cite Code Video Talk Lecture arXiv
VMAS: A Vectorized Multi-Agent Simulator for Collective Robot Learning
On the Properties of Path Additions for Traffic Routing
In this paper, we investigate the impact of path additions to transport networks with optimised traffic routing. In particular, we study the behaviour of total travel time, and consider both self-interested routing paradigms, such as User Equilibrium (UE) routing, as well as cooperative paradigms, such as classic Multi-Commodity (MC) network flow and System Optimal (SO) routing. This work aims to provide an analysis and categorization of the properties of objective functions for transport network design, with the purpose of informing algorithm (and also network) designers. Among our results, we prove, via counterexample, that total travel time, under both cooperative and self-interested routing, is not supermodular with respect to path additions.
2022 In IEEE International Conference on Intelligent Transportation Systems (ITSC) Workshop on Co-Design and Coordination of Future Mobility Systems
PDF Cite Code arXiv
On the Properties of Path Additions for Traffic Routing
Transport network design for vehicle routing: results on path addition and capacity reduction
In this thesis, we investigate the problem of optimising the transportation network for AVs both from a theoretical and from a practical perspective. In the first part, we investigate the properties of adding paths to a network and we prove that path additions to transport networks, where AVs are routed, are not supermodular in travel time, extending the seminal result of Braess’ paradox. In the second part, we formulate two network design problems for self-interested AVs. We present the problem of optimising transport networks via path additions and a novel problem design where self-interested users are guided towards optimal paths through the reduction of road capacities. Through capacity reductions, we achieve significant total travel time improvements on six real-world transport networks: Anaheim (USA), Barcelona (Spain), Chicago (USA), Eastern Massachusetts (USA), Sioux Falls (USA), and Winnipeg (Canada). For instance, we improve the Chicago network by up to 7%, saving more than 487 hours of total travel time per traffic hour.
2021 MPhil Thesis
PDF Cite Code Video
Transport network design for vehicle routing: results on path addition and capacity reduction

Contact