Ethical Challenges in AI-Driven Decision-Making: Addressing Bias and Accountability in Business Applications

Munifah Munifah; Eka Satria  Wibawa; Kasih  Purwantini

doi:10.51903/jmi.v3i1.48

Authors

Munifah Munifah Universitas Sains dan Teknologi Komputer
Eka Satria Wibawa Universitas Sains dan Teknologi Komputer
Kasih Purwantini Universitas Sains dan Teknologi Komputer

DOI:

https://doi.org/10.51903/jmi.v3i1.48

Keywords:

Ethical AI , Bias in Artificial Intelligence , Accountability in AI System , Business Decision-Making , AI Governance Frameworks

Abstract

The adoption of artificial intelligence (AI) in business decision-making has revolutionized operations but introduced critical ethical challenges, particularly in bias and accountability. This study investigates the sources of bias in AI-driven systems and evaluates current accountability frameworks in business contexts. A mixed-methods approach is employed, combining a comprehensive literature review with in-depth interviews with business leaders across technology, finance, and healthcare sectors. The findings reveal that algorithmic and data biases are prevalent, arising from imbalanced training datasets and opaque algorithmic processes. Existing accountability mechanisms are often insufficient, with responsibility dispersed among developers, managers, and regulators. Practical strategies, such as third-party audits and algorithmic transparency initiatives, are emerging but require further refinement. This study emphasizes the need for robust ethical frameworks, including guidelines like Fairness Accountability Transparency Ethics (FATE), to mitigate bias and ensure responsible AI usage. Key recommendations include the adoption of transparent AI models, enhanced regulatory oversight, and targeted training for stakeholders on AI ethics. These insights contribute to the ongoing discourse on ethical AI deployment and provide actionable pathways for businesses aiming to navigate the ethical complexities of AI.

Author Biography

Munifah Munifah, Universitas Sains dan Teknologi Komputer

References

Akinrinola, O., Okoye, C. C., Ofodile, O. C., & Ugochukwu, C. E. (2024). Navigating and reviewing ethical dilemmas in AI development: Strategies for transparency, fairness, and accountability. GSC Advanced Research and Reviews, 18(3), 050–058. https://gsconlinepress.com/journals/gscarr/content/navigating-and-reviewing-ethical-dilemmas-ai-development-strategies-transparency-fairness

Aldboush, H. H. H., & Ferdous, M. (2023). Building Trust in Fintech: An Analysis of Ethical and Privacy Considerations in the Intersection of Big Data, AI, and Customer Trust. International Journal of Financial Studies, 11(3). https://doi.org/10.3390/ijfs11030090

Auld, G., Casovan, A., Clarke, A., & Faveri, B. (2022). Governing AI through ethical standards: learning from the experiences of other private governance initiatives. Journal of European Public Policy, 29(11), 1822–1844. https://doi.org/10.1080/13501763.2022.2099449

Banerjee, I., Bhattacharjee, K., Burns, J. L., Trivedi, H., Purkayastha, S., Seyyed-Kalantari, L., Patel, B. N., Shiradkar, R., & Gichoya, J. (2023). “Shortcuts” Causing Bias in Radiology Artificial Intelligence: Causes, Evaluation, and Mitigation. Journal of the American College of Radiology, 20(9), 842–851. https://doi.org/10.1016/j.jacr.2023.06.025

Bleher, H., & Braun, M. (2022). Diffused responsibility: attributions of responsibility in the use of AI-driven clinical decision support systems. AI and Ethics, 2(4), 747–761. https://doi.org/10.1007/s43681-022-00135-x

Cancela-Outeda, C. (2024). The EU’s AI act: A framework for collaborative governance. In Internet of Things (Netherlands) (Vol. 27). https://doi.org/10.1016/j.iot.2024.101291

Chen, Z. (2023). Ethics and discrimination in artificial intelligence-enabled recruitment practices. Humanities and Social Sciences Communications, 10(1). https://doi.org/10.1057/s41599-023-02079-x

Díaz-Rodríguez, N., Del Ser, J., Coeckelbergh, M., López de Prado, M., Herrera-Viedma, E., & Herrera, F. (2023). Connecting the dots in trustworthy Artificial Intelligence: From AI principles, ethics, and key requirements to responsible AI systems and regulation. Information Fusion, 99. https://doi.org/10.1016/j.inffus.2023.101896

Femi Osasona, Olukunle Oladipupo Amoo, Akoh Atadoga, Temitayo Oluwaseun Abrahams, Oluwatoyin Ajoke Farayola, & Benjamin Samson Ayinla. (2024). Reviewing the Ethical Implications of Ai in Decision Making Processes. International Journal of Management & Entrepreneurship Research, 6(2), 322–335. https://doi.org/10.51594/ijmer.v6i2.773

Ghai, B., & Mueller, K. (2023). D-BIAS: A Causality-Based Human-in-the-Loop System for Tackling Algorithmic Bias. IEEE Transactions on Visualization and Computer Graphics, 29(1), 473–482. https://doi.org/10.1109/TVCG.2022.3209484

Goumas, G., Dardavesis, T. I., Syrigos, K., Syrigos, N., & Simou, E. (2024). Chatbots in Cancer Applications, Advantages and Disadvantages: All that Glitters Is Not Gold. Journal of Personalized Medicine, 14(8). https://doi.org/10.3390/jpm14080877

Guan, H., Dong, L., & Zhao, A. (2022). Ethical Risk Factors and Mechanisms in Artificial Intelligence Decision Making. Behavioral Sciences, 12(9). https://doi.org/10.3390/bs12090343

Gupta, N. (2023). Artificial Intelligence Ethics and Fairness: A study to address bias and fairness issues in AI systems, and the ethical implications of AI applications. Revista Review Index Journal of Multidisciplinary, 3(2), 24–35. https://doi.org/10.31305/rrijm2023.v03.n02.004

Huang, C., Zhang, Z., Mao, B., & Yao, X. (2023). An Overview of Artificial Intelligence Ethics. IEEE Transactions on Artificial Intelligence, 4(4), 799–819. https://doi.org/10.1109/TAI.2022.3194503

Karimian, G., Petelos, E., & Evers, S. M. A. A. (2022). The ethical issues of the application of artificial intelligence in healthcare: a systematic scoping review. AI and Ethics, 2(4), 539–551. https://doi.org/10.1007/s43681-021-00131-7

Khatri, M. R. (2023). Integration of Natural Language Processing, Self-Service Platforms, Predictive Maintenance, and Prescriptive Analytics for Cost Reduction, Personalization, and Real-Time Insights Customer Service and Operational Efficiency. International Journal of Information and Cybersecurity, 7(9), 1–30. https://publications.dlpress.org/index.php/ijic/article/view/36

Kiseleva, A., Kotzinos, D., & De Hert, P. (2022). Transparency of AI in Healthcare as a Multilayered System of Accountabilities: Between Legal Requirements and Technical Limitations. In Frontiers in Artificial Intelligence (Vol. 5). https://doi.org/10.3389/frai.2022.879603

Kordzadeh, N., & Ghasemaghaei, M. (2022). Algorithmic bias: review, synthesis, and future research directions. European Journal of Information Systems, 31(3), 388–409. https://doi.org/10.1080/0960085X.2021.1927212

McLennan, S., Fiske, A., Tigard, D., Müller, R., Haddadin, S., & Buyx, A. (2022). Embedded ethics: a proposal for integrating ethics into the development of medical AI. BMC Medical Ethics, 23(1). https://doi.org/10.1186/s12910-022-00746-3

Nadeem, A., Marjanovic, O., & Abedin, B. (2022). Gender bias in AI-based decision-making systems: a systematic literature review. Australasian Journal of Information Systems, 26. https://doi.org/10.3127/AJIS.V26I0.3835

Nazer, L. H., Zatarah, R., Waldrip, S., Ke, J. X. C., Moukheiber, M., Khanna, A. K., Hicklen, R. S., Moukheiber, L., Moukheiber, D., Ma, H., & Mathur, P. (2023). Bias in artificial intelligence algorithms and recommendations for mitigation. PLOS Digital Health, 2(6). https://doi.org/10.1371/journal.pdig.0000278

Novelli, C., Taddeo, M., & Floridi, L. (2022). Accountability in Artificial Intelligence: What It Is and How It Works. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4180366

Oluwafunmilola Oriji, Mutiu Alade Shonibare, Rosita Ebere Daraojimba, Oluwabosoye Abitoye, & Chibuike Daraojimba. (2023). Financial Technology Evolution in Africa: a Comprehensive Review of Legal Frameworks and Implications for Ai-Driven Financial Services. International Journal of Management & Entrepreneurship Research, 5(12), 929–951. https://doi.org/10.51594/ijmer.v5i12.627

Pragati Agarwal, Sanjeev Swami, & Sunita Kumari Malhotra. (2022). Artificial Intelligence Adoption in the Post COVID-19 New-Normal and Role of Smart Technologies in Transforming Business: a Review. Journal of Science and Technology Policy Management.

Rana, S. A., Azizul, Z. H., & Awan, A. A. (2023). A step toward building a unified framework for managing AI bias. In PeerJ Computer Science (Vol. 9). https://doi.org/10.7717/peerj-cs.1630

Ridley, M. (2022). Explainable Artificial Intelligence (XAI): : Adoption and advocacy. Information Technology and Libraries, 41(2), 1–17. https://ital.corejournals.org/index.php/ital/article/view/14683

Stahl, B. C., Antoniou, J., Ryan, M., Macnish, K., & Jiya, T. (2022). Organisational responses to the ethical issues of artificial intelligence. AI and Society, 37(1), 23–37. https://doi.org/10.1007/s00146-021-01148-6

Varsha, P. S. (2023). How can we manage biases in artificial intelligence systems–A systematic literature review. International Journal of Information Management Data Insights, 3(1), 100165.

Zhang, C. (Abigail), Cho, S., & Vasarhelyi, M. (2022). Explainable Artificial Intelligence (XAI) in auditing. International Journal of Accounting Information Systems, 46. https://doi.org/10.1016/j.accinf.2022.100572