Being Intentional: AB Testing and Data Minimization

Often in analytics and data science we have the 'big table' mental picture of customer data where we are continuously trying to append and link new bits of data back to each customer. It turns out, however, that for many of the statistics needed for tasks like AB Testing, if we are intentional about how the data is collected, then often there is no need to link all of this information back to a central visitor or customer ID. Most of the basic statistical approaches used for inference in AB Testing (t-tests, ANOVA, nested partial f-tests, etc.) can be done on data stored in equivalence classes, or at the task level, rather than at an individual level. The hope is that once armed with options, attendees will be able to consider the trade-offs and make informed decisions on when each approach is most appropriate.