Authors: Ana Lucic, Maurits Bleeker. In January 2020, the University of Amsterdam ran its first iteration of a new course on Fairness, Accountability, Confidentiality and Transparency in Artificial Intelligence (FACT-AI). Students worked in groups to reproduce the results from a recent paper at a top AI conference from one of the four topics. By focusing on the reproducibility of papers devoted to FACT algorithms, our aim was to (1) expose students to the general area of FACT, (2) expose them to recent research in this field, specifically algorithmic contributions, and (3) convey the importance of solid, reproducible research.
Reproducing the results from a paper involved implementing the algorithm(s) described in the paper as well as a writeup detailing the level to which the results were reproducible. Approximately 120 students took part and they attempted to reproduce 10 papers, in 32 teams of 2-4 people each.
As a contribution to the community, we are releasing the best implementation for each paper as open source: https://github.com/uva-fact-ai-course/uva-fact-ai-course. We hope this increases the accessibility of working with FACT algorithms. We will continue adding to this repository annually with each iteration of the course.
The following papers are implemented in the repository:
- Mitigating Unwanted Biases with Adversarial Learning (Zhang et al., AAAI 2018)
- Attention is not Explanation (Jain and Wallace, NAACL 2019)
- Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negatives (Dhurandhar et al., NeurIPS 2018)
- Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictions (Li et al., AAAI 2018)
- Uncovering and Mitigating Algorithmic Bias through Learned Latent Structure. (Amini et al., AIES 2019)
- Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings (Bolukbasi et al, NeurIPS 2016)
- Equity of Attention: Amortizing Individual Fairness in Rankings (Biega et al., SIGIR 2018)
- Full-Gradient Representation for Neural Network Visualization (Srinivas et al., NeurIPS 2019)
- Identifying and Reducing Gender Bias in Word-level Language Models (Bordia and Bowman, NAACL 2019)
- Towards Robust Interpretability with Self-Explaining Neural Networks (Alvarez-Melis and Jaakkola, NeurIPS 2018)
We want to thank the following students from the 2020 edition of the course for their contributions to the course and the repository:
- Ivan Bardarov, Mathieu Bartels, Laurence Bont, Vanessa Botha, David Cerny, Frederic Robert Chamot, Emil Dudev, Luisa Ebner, Omar Elbaghdadi, Kylian van Geijtenbeek, Julio Joaquín López González, Angelo Groot, Albert Harkema, Christoph Hoenes, Nithin Holla, Aman Hussain, Iulia Ionescu, Eui Yeon Jang, Reitze Jansen, Tom Kersten, Maximilian Knaller, David Knigge, Sietze Kuilman, Anna Langedijk, Daan Le, Nils Lehmann, Hannah Min, Oliviero Nardi, Azamat Omuraliev, Liselore Borel Rinkes, Hinrik Snær, Leila Talha, Martine Toering, Marcel Velez, Thom Visser, Christiaan van der Vlist, David Vos, David Wessels, Thomas van Zwol.
We also want to thank our TAs of the 2020 edition of the course for helping to run the course and guide the students:
- Morris Frank, Marco Heuvelman, Leon Lang, Phillip Lippe, Andreas Panteli, Simon Passenheim.