Trust and Transparency in AI
In this paper, I consider an important question in the philosophy of AI. Does the fact that we cannot know how an AI reaches its conclusions entail that we cannot reasonably trust it? I argue that it does not.
The relationship between trust and transparency, whether in AI or elsewhere, appears quite puzzling. It seems unreasonable to trust that which is opaque, but too much transparency renders trust superfluous, for trust requires some degree of uncertainty and vulnerability.
So, how is transparency related to trust? To answer this, I appeal to the distinction between type transparency and token transparency. Type transparency is about the purpose of a system and the way it generally operates, whereas token transparency is about its operation in a specific instance. For example, a hospital is being type transparent if it makes clear its standard procedures for treating patients. It is being token transparent if it makes clear what happened in the case of a particular patient. Type transparency, I argue, is the kind that justifies trust, whereas token transparency makes it superfluous. Reasonable trust requires understanding the purpose of a system and broadly how it works. But if we understand all the details of a particular case, then there is no room for trust, at least in that case. The puzzle of transparency arises from conflating the two.
For AI, opacity about how it reaches a specific conclusion is no barrier to trust, for that is merely a lack of token transparency. Insofar as we can understand the broader process and purpose—it is trained on large amounts of data until it can reliably produce accurate results—it has type transparency. There are plausibly other conditions for reasonable trust, but the transparency condition of reasonable trust can be met by an AI.
However, there is a caveat. Some AIs lack even type transparency, for it is unclear not just how they reach their conclusions but what their purpose is. So, while the black box problem does not stop us from trusting all AIs, it does become more problematic for general purpose AI.