Faired: A Systematic Fairness Analysis Approach Applied In A Higher Educational Context

Higher education institutions increasingly rely on machine learning models. However, a growing body of evidence shows that these algorithms may not serve underprivileged communities well and at times discriminate against them. This is all the more concerning in education as negative outcomes have long-term implications. We propose a systematic process for framing, detecting, documenting, and reporting unfairness risks. The systematic approach's outcomes are merged into a framework named FairEd, which would help decision-makers to understand unfairness risks along the environmental and analytical fairness dimension. The tool allows to decide (i) whether the dataset contains risks of unfairness; (ii) how the models could perform along many fairness dimensions; (iii) whether potentially unfair outcomes can be mitigated without degrading performance. The systematic approach is applied to a Chilean University case study, where a predicting student dropout model is aimed to build. First, we capture the nuances of the Chilean context where unfairness emerges along income lines and demographic groups. Second, we highlight the benefit of reporting unfairness risks along a diverse set of metrics to shed light on potential discrimination. Third, we find that measuring the cost of fairness is an important quantity to report on when doing the model selection.