In a session yesterday of the NSF CyberAmbassadors leadership training program, my breakout group were tasked with discussing a case study of a potential ethics violation in research data privacy. The Code of Conduct that we were to use to determine if a violation occurred was the Association for Computing Machinery’s (ACM).
The case study involved a research scientist who had made software to analyze three sets of participant data, including DNA records, medical records and social media posts. There was a problem with the program and the scientist wanted to be able to do a crowdsourced code review. They asked their ERB team to review whether they could release the codebase to the public to crowdsource the problem. The ERB approved the request as long as no participant data was also released or could be reidentified. The case expressed a statement that there was a risk of reidentifying data but didn’t say specifically how. Just that the request was approved.
My first impression was that the research scientist was hiding behind item 2.6 in the ACM Code of Conduct, which says to only do work within your area of competence. The way we read it, the researcher relied on the Ethics Review Board (ERB) to make the ethical determination. Since the ERB approved the study, was the researcher in the clear?
Conversation ensued about how a data analytics program that didn’t include test data could be tested, or whether it could be tested with dummy data and a sample of open social media posts/hashtags, etc. but that was actually aside from our real interest, which was the idea that technology developers, including those with less funding, but also those with fewer guardrails, may not be competent to or interested in make ethical decisions.
Someone brought up AI. People working in AI today or really any large, complex model affecting global populations, are often making decisions way outside of their area of competence. They may do well, in one or two disciplines, but understanding and unraveling the externalities of what the thing will do once it’s in the world is of lesser interest since they aren’t ethicists.
In fact, not all companies have ERBs and many big names, you know who, have quietly and unceremoniously disbanded their ethics teams. In a world of move fast and break things, it’s not their area of competence.
Is this the world we want to live in?