The Data Oath
The Data Oath
Everyone has the expectation of being treated in an equitable manner in their professional and personal lives. If a company or person treated us in a biased way, we would end our relationship as soon as we became aware of it. But what if we were in a situation where we couldn’t end the relationship? What if we had no other choice but to go through a certain individual or company? Think about COMPASS for sentencing and parole recommendations. Or gatekeeper systems that determine our eligibility for jobs, credit, etc. In these scenarios, users cannot opt out of using the system.
Equality is very closely related to Inclusivity. While inclusivity is about including as many people, groups, etc., so that we can create a more representative model of users. Equality is about how the users are treated in comparison to other users, groups, races, genders, etc. It is about how level the playing field is for everyone participating in the system. If the playing field isn’t level, then it would be clear that some users would be benefiting at the expense of others through no fault of their own.
So how do we ensure that systems are treating users equitably? The first step, in our opinion, is to ensure that any system interacting with human users should be Transparent & Explainable. This would allow for the system to be tested regularly for bias, model drifts, etc. Second, we would recommend that teams work hard to ensure that data sets they are using are diverse and representative of the users and communities they are serving. Sure it will be more work, but in the end, the results would lead to more robust and powerful models.
Companies should ensure that their models are designed with the purpose of Improving the Quality of Life and to be Accessible & Contestable by users; the models will have a greater likelihood of being equitable in the treatment of users. After all, we don’t create what we don’t think about. We suggest putting all users’ equality and quality of life at the center of every model design. If we push our thinking, our development teams, our business models, and our standards, there is no limit to what we can create.