Syed Ishtiaque Ahmed

Jiessie Tie, Bingsheng Yao, Tianshi Li et al.

Software engineers are increasingly incorporating AI assistants into their workflows to enhance productivity and alleviate cognitive load. However, experiences with large language models (LLMs) such as ChatGPT vary widely. While some engineers find them useful, others deem them counterproductive due to inaccuracies in their responses. Researchers have also observed that ChatGPT often provides incorrect information. Given these limitations, it is crucial to determine how to effectively integrate LLMs into software engineering (SE) workflow. Analyzing data from 26 participants in a complex web development task, we identified nine failure types categorized into incorrect or incomplete responses, cognitive overload, and context loss. Users attempted to mitigate these issues through scaffolding, prompt clarification, and debugging. However, 17 participants ultimately chose to abandon ChatGPT due to persistent failures. Our quantitative analysis revealed that unhelpful responses increased the likelihood of abandonment by a factor of 11, while each additional prompt reduced abandonment probability by 17%. This study advances the understanding of human-AI interaction in SE tasks and outlines directions for future research and tooling support.

Ramaravind K. Mothilal, Joanna Roy, Syed Ishtiaque Ahmed et al. · utoronto

The discourse around toxicity and LLMs in NLP largely revolves around detection tasks. This work shifts the focus to evaluating LLMs' reasoning about toxicity -- from their explanations that justify a stance -- to enhance their trustworthiness in downstream tasks. Despite extensive research on explainability, it is not straightforward to adopt existing methods to evaluate free-form toxicity explanation due to their over-reliance on input text perturbations, among other challenges. To account for these, we propose a novel, theoretically-grounded multi-dimensional criterion, Human-Aligned Faithfulness (HAF), that measures the extent to which LLMs' free-form toxicity explanations align with those of a rational human under ideal conditions. We develop six metrics, based on uncertainty quantification, to comprehensively evaluate HAF of LLMs' toxicity explanations with no human involvement, and highlight how "non-ideal" the explanations are. We conduct several experiments on three Llama models (of size up to 70B) and an 8B Ministral model on five diverse toxicity datasets. Our results show that while LLMs generate plausible explanations to simple prompts, their reasoning about toxicity breaks down when prompted about the nuanced relations between the complete set of reasons, the individual reasons, and their toxicity stances, resulting in inconsistent and irrelevant responses. We open-source our code at https://github.com/uofthcdslab/HAF and LLM-generated explanations at https://huggingface.co/collections/uofthcdslab/haf.

Md Arid Hasan, Firoj Alam, Md Fahad Hossain et al. · utoronto

Online social media platforms are central to everyday communication and information seeking. While these platforms serve positive purposes, they also provide fertile ground for the spread of hate speech, offensive language, and bullying content targeting individuals, organizations, and communities. Such content undermines safety, participation, and equity online. Reliable detection systems are therefore needed, especially for low-resource languages where moderation tools are limited. In Bangla, prior work has contributed resources and models, but most are single-task (e.g., binary hate/offense) with limited coverage of multi-facet signals (type, severity, target). We address these gaps by introducing the first multi-task Bangla hate-speech dataset, BanglaMultiHate, one of the largest manually annotated corpus to date. Building on this resource, we conduct a comprehensive, controlled comparison spanning classical baselines, monolingual pretrained models, and LLMs under zero-shot prompting and LoRA fine-tuning. Our experiments assess LLM adaptability in a low-resource setting and reveal a consistent trend: although LoRA-tuned LLMs are competitive with BanglaBERT, culturally and linguistically grounded pretraining remains critical for robust performance. Together, our dataset and findings establish a stronger benchmark for developing culturally aligned moderation tools in low-resource contexts. For reproducibility, we will release the dataset and all related scripts.

Ramaravind Kommiya Mothilal, Faisal M. Lalani, Syed Ishtiaque Ahmed et al. · utoronto

The reference to assumptions in how practitioners use or interact with machine learning (ML) systems is ubiquitous in HCI and responsible ML discourse. However, what remains unclear from prior works is the conceptualization of assumptions and how practitioners identify and handle assumptions throughout their workflows. This leads to confusion about what assumptions are and what needs to be done with them. We use the concept of an argument from Informal Logic, a branch of Philosophy, to offer a new perspective to understand and explicate the confusions surrounding assumptions. Through semi-structured interviews with 22 ML practitioners, we find what contributes most to these confusions is how independently assumptions are constructed, how reactively and reflectively they are handled, and how nebulously they are recorded. Our study brings the peripheral discussion of assumptions in ML to the center and presents recommendations for practitioners to better think about and work with assumptions.

Soaad Hossain, Syed Ishtiaque Ahmed

Given the success of ChatGPT, LaMDA and other large language models (LLMs), there has been an increase in development and usage of LLMs within the technology sector and other sectors. While the level in which LLMs has not reached a level where it has surpassed human intelligence, there will be a time when it will. Such LLMs can be referred to as advanced LLMs. Currently, there are limited usage of ethical artificial intelligence (AI) principles and guidelines addressing advanced LLMs due to the fact that we have not reached that point yet. However, this is a problem as once we do reach that point, we will not be adequately prepared to deal with the aftermath of it in an ethical and optimal way, which will lead to undesired and unexpected consequences. This paper addresses this issue by discussing what ethical AI principles and guidelines can be used to address highly advanced LLMs.

Soaad Hossain, James Rasalingam, Arhum Waheed et al.

With the growing interest in using AI and machine learning (ML) in medicine, there is an increasing number of literature covering the application and ethics of using AI and ML in areas of medicine such as clinical psychiatry. The problem is that there is little literature covering the economic aspects associated with using ML in clinical psychiatry. This study addresses this gap by specifically studying the economic implications of using ML in clinical psychiatry. In this paper, we evaluate the economic implications of using ML in clinical psychiatry through using three problem-oriented case studies, literature on economics, socioeconomic and medical AI, and two types of health economic evaluations. In addition, we provide details on fairness, legal, ethics and other considerations for ML in clinical psychiatry.

Nestor Hernandez, Ruben Recabarren, Bogdan Carbunar et al.

Online app search optimization (ASO) platforms that provide bulk installs and fake reviews for paying app developers in order to fraudulently boost their search rank in app stores, were shown to employ diverse and complex strategies that successfully evade state-of-the-art detection methods. In this paper we introduce RacketStore, a platform to collect data from Android devices of participating ASO providers and regular users, on their interactions with apps which they install from the Google Play Store. We present measurements from a study of 943 installs of RacketStore on 803 unique devices controlled by ASO providers and regular users, that consists of 58,362,249 data snapshots collected from these devices, the 12,341 apps installed on them and their 110,511,637 Google Play reviews. We reveal significant differences between ASO providers and regular users in terms of the number and types of user accounts registered on their devices, the number of apps they review, and the intervals between the installation times of apps and their review times. We leverage these insights to introduce features that model the usage of apps and devices, and show that they can train supervised learning algorithms to detect paid app installs and fake reviews with an F1-measure of 99.72% (AUC above 0.99), and detect devices controlled by ASO providers with an F1-measure of 95.29% (AUC = 0.95). We discuss the costs associated with evading detection by our classifiers and also the potential for app stores to use our approach to detect ASO work with privacy.

Ananya Bhattacharjee, S M Taiabul Haque, Abdul Hady et al.

Objective: This study aims to identify the social determinants of mental health among undergraduate students in Bangladesh, a developing nation in South Asia. Our goal is to identify the broader social determinants of mental health among this population, study the manifestation of these determinants in their day-to-day life, and explore the feasibility of self-monitoring tools in helping them identify the specific factors or relationships that impact their mental health. Methods: We conducted a 21-day study with 38 undergraduate students from seven universities in Bangladesh. We conducted two semi-structured interviews: one pre-study and one post-study. During the 21-day study, participants used an Android application to self-report and self-monitor their mood after each phone conversation. The app prompted participants to report their mood after each phone conversation and provided graphs and charts so that participants could independently review their mood and conversation patterns. Results: Our results show that academics, family, job and economic condition, romantic relationships, and religion are the major social determinants of mental health among undergraduate students in Bangladesh. Our app helped the participants pinpoint the specific issues related to these factors as participants could review the pattern of their moods and emotions from past conversation history. Although our app does not provide any explicit recommendation, participants took certain steps on their own to improve their mental health (e.g., reduced the frequency of communication with certain persons). Conclusions: Overall, the findings from this study would provide better insights for the researchers to design better solutions to help the younger population from this part of the world.

Soraia Oueida, Soaad Hossain, Yehia Kotb et al.

This paper presents a new approach to prevent transportation accidents and monitor driver's behavior using a healthcare AI system that incorporates fairness and ethics. Dangerous medical cases and unusual behavior of the driver are detected. Fairness algorithm is approached in order to improve decision-making and address ethical issues such as privacy issues, and to consider challenges that appear in the wild within AI in healthcare and driving. A healthcare professional will be alerted about any unusual activity, and the driver's location when necessary, is provided in order to enable the healthcare professional to immediately help to the unstable driver. Therefore, using the healthcare AI system allows for accidents to be predicted and thus prevented and lives may be saved based on the built-in AI system inside the vehicle which interacts with the ER system.

Soaad Hossain, Syed Ishtiaque Ahmed

With there being many technical and ethical issues with artificial intelligence (AI) that involve marginalized communities, there is a growing interest for design methods used with marginalized people that may be transferable to the design of AI technologies. Participatory design (PD) is a design method that is often used with marginalized communities for the design of social development, policy, IT and other matters and solutions. However, there are issues with the current PD, raising concerns when it is applied to the design of technologies, including AI technologies. This paper argues for the use of PD for the design of AI technologies, and introduces and proposes a new PD, which we call agile participatory design, that not only can could be used for the design of AI and data-driven technologies, but also overcomes issues surrounding current PD and its use in the design of such technologies.

Abdul Kawsar Tushar, Muhammad Ashad Kabir, Syed Ishtiaque Ahmed

Mental health is a global epidemic, affecting close to half a billion people worldwide. Chronic shortage of resources hamper detection and recovery of affected people. Effective sensing technologies can help fight the epidemic through early detection, prediction, and resulting proper treatment. Existing and novel technologies for sensing mental health state could address the aforementioned concerns by activating granular tracking of physiological, behavioral, and social signals pertaining to problems in mental health. Our paper focuses on the available methods of sensing mental health problems through direct and indirect measures. We see how active and passive sensing by technologies as well as reporting from relevant sources can contribute toward these detection methods. We also see available methods of therapeutic treatment available through digital means. We highlight a few key intervention technologies that are being developed by researchers to fight against mental illness issues.

Md Mahfuzul Haque, Mohammad Yousuf, Ahmed Shatil Alam et al.

There has been a growing interest within CSCW community in understanding the characteristics of misinformation propagated through computational media, and the devising techniques to address the associated challenges. However, most work in this area has been concentrated on the cases in the western world leaving a major portion of this problem unaddressed that is situated in the Global South. This paper aims to broaden the scope of this discourse by focusing on this problem in the context of Bangladesh, a country in the Global South. The spread of misinformation on Facebook in Bangladesh, a country with a population over 163 million, has resulted in chaos, hate attacks, and killings. By interviewing journalists, fact-checkers, in addition to surveying the general public, we analyzed the current state of verifying misinformation in Bangladesh. Our findings show that most people in the `news audience' want the news media to verify the authenticity of online information that they see online. However, the newspaper journalists say that fact-checking online information is not a part of their job, and it is also beyond their capacity given the amount of information being published online everyday. We further find that the voluntary fact-checkers in Bangladesh are not equipped with sufficient infrastructural support to fill in this gap. We show how our findings are connected to some of the core concerns of CSCW community around social media, collaboration, infrastructural politics, and information inequality. From our analysis, we also suggest several pathways to increase the impact of fact-checking efforts through collaboration, technology design, and infrastructure development.

Soaad Hossain, Syed Ishtiaque Ahmed

With the shortage of physicians and surgeons and increase in demand worldwide due to situations such as the COVID-19 pandemic, there is a growing interest in finding solutions to help address the problem. A solution to this problem would be to use neurotechnology to provide them augmented cognition, senses and action for optimal diagnosis and treatment. Consequently, doing so can negatively impact them and others. We argue that applying neurotechnology for human enhancement in physicians and surgeons can cause injustices, and harm to them and patients. In this paper, we will first describe the augmentations and neurotechnologies that can be used to achieve the relevant augmentations for physicians and surgeons. We will then review selected ethical concerns discussed within literature, discuss the neuroengineering behind using neurotechnology for augmentation purposes, then conclude with an analysis on outcomes and ethical issues of implementing human augmentation via neurotechnology in medical and surgical practice.

Naeemul Hassan, Amrit Poudel, Jason Hale et al.

Tracking sexual violence is a challenging task. In this paper, we present a supervised learning-based automated sexual violence report tracking model that is more scalable, and reliable than its crowdsource based counterparts. We define the sexual violence report tracking problem by considering victim, perpetrator contexts and the nature of the violence. We find that our model could identify sexual violence reports with a precision and recall of 80.4% and 83.4%, respectively. Moreover, we also applied the model during and after the \#MeToo movement. Several interesting findings are discovered which are not easily identifiable from a shallow analysis.

Leticia Duboc, Curtis McCord, Christoph Becker et al.

The design of software systems inevitably enacts normative boundaries around the site of intervention. These boundaries are, in part, a reflection of the values, ethics, power, and politics of the situation and the process of design itself. This paper argues that Requirements Engineering (RE) require more robust frameworks and techniques to navigate the values implicit in systems design work. To this end, we present the findings from a case of action research where we employed Critical Systems Heuristics (CSH), a framework from Critical Systems Thinking (CST) during requirements gathering for Homesound, a system to safeguard elderly people living alone while protecting their autonomy. We use categories from CSH to inform expert interviews and reflection, showing how CSH can be simply combined with RE techniques (such as the Volere template) to explore and reveal the value-judgements underlying requirements.