Responsible Design through Experimentation: Learnings from LinkedIn

As technology advances, there is increasing concern about individuals being left behind. Businesses are striving to adopt responsible design practices and avoid any unintended consequences of their products. We propose a novel approach to fairness and inclusiveness based on experimentation. We use experimentation in order to assess not only the intrinsic properties of products and algorithms but also their impact on people. We do this by introducing an inequality approach to A/B testing. We show how to perform causal inference over this inequality measure. We provide real examples from LinkedIn, as well as an open-source, highly scalable implementation of the computation of the Atkinson index and its variance in Spark/Scala. We also provide over a year's worth of learnings - gathered by scaling our method and analyzing thousands of experiments - on which areas and which kinds of product innovations seem to foster fairness through inclusiveness.