Reproducing FACT Papers

Authors: Ana Lucic, Maurits Bleeker. In January 2020, the University of Amsterdam ran its first iteration of a new course on Fairness, Accountability, Confidentiality and Transparency in Artificial Intelligence (FACT-AI). Students worked in groups to reproduce the results from a recent paper at a top AI conference from one of the four topics. By focusing on the reproducibility of papers devoted to FACT algorithms, our aim was to (1) expose students to the general area of FACT, (2) expose them to recent research in this field, specifically algorithmic contributions, and (3) convey the importance of solid, reproducible research.

Reproducing the results from a paper involved implementing the algorithm(s) described in the paper as well as a writeup detailing the level to which the results were reproducible. Approximately 120 students took part and they attempted to reproduce 10 papers, in 32 teams of 2-4 people each. 

As a contribution to the community, we are releasing the best implementation for each paper as open source: We hope this increases the accessibility of working with FACT algorithms. We will continue adding to this repository annually with each iteration of the course.

The following papers are implemented in the repository:

We want to thank the following students from the 2020 edition of the course for their contributions to the course and the repository:

  • Ivan Bardarov, Mathieu Bartels, Laurence Bont, Vanessa Botha, David Cerny, Frederic Robert Chamot, Emil Dudev, Luisa Ebner, Omar Elbaghdadi, Kylian van Geijtenbeek, Julio Joaquín López González, Angelo Groot, Albert Harkema, Christoph Hoenes, Nithin Holla, Aman Hussain, Iulia Ionescu, Eui Yeon Jang, Reitze Jansen, Tom Kersten, Maximilian Knaller, David Knigge, Sietze Kuilman, Anna Langedijk, Daan Le, Nils Lehmann, Hannah Min, Oliviero Nardi, Azamat Omuraliev, Liselore Borel Rinkes, Hinrik Snær, Leila Talha, Martine Toering, Marcel Velez, Thom Visser, Christiaan van der Vlist, David Vos, David Wessels, Thomas van Zwol.

We also want to thank our TAs of the 2020 edition of the course for helping to run the course and guide the students:

  • Morris Frank, Marco Heuvelman, Leon Lang, Phillip Lippe, Andreas Panteli, Simon Passenheim.

Want to read more, and in a bit more detail? One of the groups wrote an excellent blog-post about their self-explaining networks project: