Human-AI Interaction & Trust
Using A-nudging to enhance trust in AI
A human-AI collaboration framework is a structured approach to designing and deploying artificial intelligence systems in ways that maximize the benefits of collaboration between humans and AI. This framework emphasizes complementary roles, assigning tasks to AI that leverage its speed, scalability, and data-driven insights while ensuring that human’s abilities, contextual understanding and know-how are leveraged as well. A well-structured framework ensures that AI systems operate as supportive tools that enhance human decision-making and ensure upskilling rather than merely automating tasks or replacing humans (Amershi et al., 2019, Yang et al. 2020).
Unfortunately, current human-AI collaboration frameworks suffer from a ‘trust problem’. As Miller’s recent paper Explainable AI is Dead, Long Live Explainable AI clearly lays out, the current paradigm of recommendation-based AI decision support can lead to either over-reliance or under-reliance on AI systems by human users: users often do not trust AI recommendations and thus ignore them; others follow AI even when it’s clearly unreliable. The paper by Miller discusses how introducing Explainable AI does not really solve the problem.
Miller and others have highlighted how the best way forward to solve these issues is to deeply integrate human and AI decision making. Miller proposes “Evaluative AI” as a hypothesis-driven support, where AI assists by presenting evidence for and against human-generated hypotheses rather than providing direct recommendations. Becks and Weis (2022), among others, point to how using AI nudging may solve some of the issues: the AI would nudge a human to make the best decision rather than offering a decision itself [2].
Unfortunately, these approaches have two main shortcomings. For starters they overlook the fact that rational decision-making is highly contextual and depends on decision agents’ subjective confidence and epistemic priors such as past performance and experience (as in any Bayesian framework). Second, they do not consider that decision making is not just about accuracy and success. As we make decisions in a collaborative environment, we may value speed but also human autonomy.
In contrast, the approach proposed here adopts AI nudging and explainable AI in a way that is context dependent while leveraging the distinction between human fast and slow thinking to secure human trust and model the best collaborative decision-making strategy. In this framework, AI makes informed decisions about how best to engage users but - at the same time - users will also make informed choices about how to integrate AI’s input, thus maintaining their autonomy.
Nudging is a behavioral strategy designed to influence the thoughts and actions of individuals. In their book “Nudge” Thaler and Sunstein focus on the behavioral aspect of human decision making and define a nudge as an element of a choice architecture, “understood as the background against which people make choices”. This element “alters people’s behavior in a predictable way without forbidding any options”, That is, nudges arguably preserve (a certain degree of) freedom of choice: they influence us without being fully manipulative or coercive. Even if nudged, we can choose to opt-out fairly easily and take a different path. This is true even if we rarely fully recognize the influence of nudges on us and we find ourselves adopting a certain course of action without knowing that we have been influenced.
These techniques are pervasive in various aspects of our daily lives, targeting both fast and unconscious human thinking, such as using images to evoke fear, and more deliberate, effortful slow thinking, for example, by presenting information that prompts reflection on our choices.
In the paper Value-based Fast and Slow AI Nudging, my collaborators and I introduced and explored a collaborative AI-human framework where AI systems employ nudging strategies to propose decision recommendations to improve overall decision performance while preserving values such as human autonomy. Our framework, named FAst and Slow Collaborative AI (FASCAI) is unique in stimulating human fast thinking, slow thinking, or meta-cognition, depending on when recommendations are presented and the values that need to be preserved. The conditions under which each of these three nudging mechanisms is chosen depend on the previous performance of the humans and the AI separately, along with the confidence of the AI.
The literature presents two primary types of nudges: System 1 and System 2. System 1 nudges take advantage of our quick, instinctive thinking. An example of this is placing fruits and vegetables on easily accessible shelves in a cafeteria to promote healthier eating habits. On the other hand, System 2 nudges encourage us to engage our slow, reflective thinking. An instance of this technique is the nutritional labels found on packaged food that make us consider our dietary choices. This type of nudge does not push us to make specific choices but to be more reflective and rational in our thinking and decision-making. The paper also introduces a third type of nudging: metacognitive nudges. These are designed to prompt introspection and push us to gauge our confidence level in accomplishing a specific task. For example, a metacognition nudge might ask us to rate our confidence in our ability to complete a math problem before we attempt to solve it. This can help us avoid making mistakes by taking on tasks beyond our capabilities.
The framework proposed is adaptable and can nudge humans differently depending on the situation. The architecture takes into consideration the human and the AI past performance (Past[H], Past[AI]) in similar tasks. It also considers the AI confidence level in its solution (low, medium, and high confidence). This is the decision-making table for the architecture:
Given a problem instance, the AI’s nudging controller decides which interaction modality should be adopted. If the controller decides that the machine or the human can autonomously solve the problem, the problem instance is passed to the machine or the human that will generate the output solution. If instead the controller decides that human-machine interaction is needed, then it first passes the problem instance to the machine, which in this case returns a candidate solution back to the controller. This candidate solution is used by the controller in a (S1/S2/MC) nudge for the human. Indeed, in the context of the decision scenarios we’re discussing, the assumption is that, even when AI is involved, human autonomy needs to be preserved to empower individuals to have substantial control over their decisions and to enhance their capacity for critical thinking, self-reflection, and introspection as they engage with machines in decision-making processes.