When AI Fails, Who Do We Blame? Attributing Responsibility in Human-AI Interactions

Simple item page
dc.contributor.authorSchoenherr, Jordan Richard
dc.contributor.authorThomson, Robert
dc.date.accessioned2024-04-17T15:16:48Z
dc.date.available2024-04-17T15:16:48Z
dc.date.issued2024-01-10
dc.description.abstractWhile previous studies of trust in artificial intelligence have focused on perceived user trust, the paper examines how an external agent (e.g., an auditor) assigns responsibility, perceives trustworthiness, and explains the successes and failures of AI. In two experiments, participants (university students) reviewed scenarios about automation failures and assigned perceived responsibility, trustworthiness, and preferred explanation type. Participants’ cumulative responsibility ratings for three agents (operators, developers, and AI) exceeded 100%, implying that participants were not attributing trust in a wholly rational manner, and that trust in the AI might serve as a proxy for trust in the human software developer. Dissociation between responsibility and trustworthiness suggested that participants used different cues, with the kind of technology and perceived autonomy affecting judgments. Finally, we additionally found that the kind of explanation used to understand a situation differed based on whether the AI succeeded or failed.
dc.description.sponsorshipOffice of Naval Research FY21 Multi-University Research Initiative Award N0001422MP00465 C5ISR Agreement USMA23011
dc.identifier.citationSchoenherr, Jordan Richard, and Robert Thomson. "When AI Fails, Who Do We Blame? Attributing Responsibility in Human-AI Interactions." IEEE Transactions on Technology and Society (2024).
dc.identifier.doihttps://doi.org/10.1109/TTS.2024.3370095
dc.identifier.issn2637-6415
dc.identifier.urihttps://hdl.handle.net/20.500.14216/1476
dc.publisherInstitute of Electrical and Electronics Engineers (IEEE)
dc.relation.ispartofIEEE Transactions on Technology and Society
dc.subjectArtificial intelligence
dc.subjectTask analysis
dc.subjectSoftware
dc.subjectSoftware algorithms
dc.subjectEthics
dc.subjectAutomation
dc.subjectDecision making
dc.titleWhen AI Fails, Who Do We Blame? Attributing Responsibility in Human-AI Interactions
dc.typeJournal articles
local.USMAemailrobert.thomson@westpoint.edu
local.peerReviewedYes

Files

Collections

Army Cyber Institute