AINov 18, 2023
Designing Interpretable ML System to Enhance Trust in Healthcare: A Systematic Review to Proposed Responsible Clinician-AI-Collaboration FrameworkElham Nasarian, Roohallah Alizadehsani, U. Rajendra Acharya et al.
This paper explores the significant impact of AI-based medical devices, including wearables, telemedicine, large language models, and digital twins, on clinical decision support systems. It emphasizes the importance of producing outcomes that are not only accurate but also interpretable and understandable to clinicians, addressing the risk that lack of interpretability poses in terms of mistrust and reluctance to adopt these technologies in healthcare. The paper reviews interpretable AI processes, methods, applications, and the challenges of implementation in healthcare, focusing on quality control to facilitate responsible communication between AI systems and clinicians. It breaks down the interpretability process into data pre-processing, model selection, and post-processing, aiming to foster a comprehensive understanding of the crucial role of a robust interpretability approach in healthcare and to guide future research in this area. with insights for creating responsible clinician-AI tools for healthcare, as well as to offer a deeper understanding of the challenges they might face. Our research questions, eligibility criteria and primary goals were identified using Preferred Reporting Items for Systematic reviews and Meta-Analyses guideline and PICO method; PubMed, Scopus and Web of Science databases were systematically searched using sensitive and specific search strings. In the end, 52 publications were selected for data extraction which included 8 existing reviews and 44 related experimental studies. The paper offers general concepts of interpretable AI in healthcare and discuss three-levels interpretability process. Additionally, it provides a comprehensive discussion of evaluating robust interpretability AI in healthcare. Moreover, this survey introduces a step-by-step roadmap for implementing responsible AI in healthcare.
LGNov 12, 2025
Optimistic Reinforcement Learning with Quantile ObjectivesMohammad Alipour-Vaezi, Huaiyang Zhong, Kwok-Leung Tsui et al.
Reinforcement Learning (RL) has achieved tremendous success in recent years. However, the classical foundations of RL do not account for the risk sensitivity of the objective function, which is critical in various fields, including healthcare and finance. A popular approach to incorporate risk sensitivity is to optimize a specific quantile of the cumulative reward distribution. In this paper, we develop UCB-QRL, an optimistic learning algorithm for the $τ$-quantile objective in finite-horizon Markov decision processes (MDPs). UCB-QRL is an iterative algorithm in which, at each iteration, we first estimate the underlying transition probability and then optimize the quantile value function over a confidence ball around this estimate. We show that UCB-QRL yields a high-probability regret bound $\mathcal O\left((2/κ)^{H+1}H\sqrt{SATH\log(2SATH/δ)}\right)$ in the episodic setting with $S$ states, $A$ actions, $T$ episodes, and $H$ horizons. Here, $κ>0$ is a problem-dependent constant that captures the sensitivity of the underlying MDP's quantile value.
LGApr 11, 2024
Data-Driven Portfolio Management for Motion Pictures Industry: A New Data-Driven Optimization Methodology Using a Large Language Model as the ExpertMohammad Alipour-Vaezi, Kwok-Leung Tsui
Portfolio management is one of the unresponded problems of the Motion Pictures Industry (MPI). To design an optimal portfolio for an MPI distributor, it is essential to predict the box office of each project. Moreover, for an accurate box office prediction, it is critical to consider the effect of the celebrities involved in each MPI project, which was impossible with any precedent expert-based method. Additionally, the asymmetric characteristic of MPI data decreases the performance of any predictive algorithm. In this paper, firstly, the fame score of the celebrities is determined using a large language model. Then, to tackle the asymmetric character of MPI's data, projects are classified. Furthermore, the box office prediction takes place for each class of projects. Finally, using a hybrid multi-attribute decision-making technique, the preferability of each project for the distributor is calculated, and benefiting from a bi-objective optimization model, the optimal portfolio is designed.
69.9CYApr 7
CareGuardAI: Context-Aware Multi-Agent Guardrails for Clinical Safety & Hallucination Mitigation in Patient-Facing LLMsElham Nasarian, Abhilash Neog, Kwok-Leung Tsui et al.
Integrating large language models (LLMs) into patient-facing healthcare systems offers significant potential to improve access to medical information. However, ensuring clinical safety and factual reliability remains a critical challenge. In practice, AI-generated responses may be conditionally correct yet medically inappropriate, as models often fail to interpret patient context and tend to produce agreeable responses rather than challenge unsafe assumptions. Unlike clinicians, who infer risk from incomplete information, LLMs frequently lack contextual awareness. Moreover, real-world patient interactions are open-ended and underspecified, unlike structured benchmark settings. We present CareGuardAI, a risk-aware safety framework for patient-facing medical question answering that addresses two key failure modes: clinical safety risk and hallucination risk. The framework introduces Clinical Safety Risk Assessment (SRA), inspired by ISO 14971, and Hallucination Risk Assessment (HRA) to evaluate medical risk and factual reliability. At inference time, CareGuardAI employs a multi-stage pipeline consisting of a controller agent, safety-constrained generation, and dual risk evaluation, followed by iterative refinement when necessary. Responses are released only when both SRA and HRA are less than or equal to 2, ensuring clinically acceptable outputs with bounded latency. We evaluate CareGuardAI on PatientSafeBench, MedSafetyBench, and MedHallu, covering both safety and hallucination detection. Across these benchmarks, the framework consistently outperforms strong baseline models, including GPT-4o-mini, demonstrating the importance of context-aware, risk-based, inference-time safety mechanisms for reliable deployment in healthcare.
APAug 5, 2021
PSTN: Periodic Spatial-temporal Deep Neural Network for Traffic Condition PredictionTiange Wang, Zijun Zhang, Kwok-Leung Tsui
Accurate forecasting of traffic conditions is critical for improving safety, stability, and efficiency of a city transportation system. In reality, it is challenging to produce accurate traffic forecasts due to the complex and dynamic spatiotemporal correlations. Most existing works only consider partial characteristics and features of traffic data, and result in unsatisfactory performances on modeling and forecasting. In this paper, we propose a periodic spatial-temporal deep neural network (PSTN) with three pivotal modules to improve the forecasting performance of traffic conditions through a novel integration of three types of information. First, the historical traffic information is folded and fed into a module consisting of a graph convolutional network and a temporal convolutional network. Second, the recent traffic information together with the historical output passes through the second module consisting of a graph convolutional network and a gated recurrent unit framework. Finally, a multi-layer perceptron is applied to process the auxiliary road attributes and output the final predictions. Experimental results on two publicly accessible real-world urban traffic data sets show that the proposed PSTN outperforms the state-of-the-art benchmarks by significant margins for short-term traffic conditions forecasting
CVAug 5, 2021
Automatic Rail Component Detection Based on AttnConv-NetTiange Wang, Zijun Zhang, Fangfang Yang et al.
The automatic detection of major rail components using railway images is beneficial to ensure the rail transport safety. In this paper, we propose an attention-powered deep convolutional network (AttnConv-net) to detect multiple rail components including the rail, clips, and bolts. The proposed method consists of a deep convolutional neural network (DCNN) as the backbone, cascading attention blocks (CAB), and two feed forward networks (FFN). Two types of positional embedding are applied to enrich information in latent features extracted from the backbone. Based on processed latent features, the CAB aims to learn the local context of rail components including their categories and component boundaries. Final categories and bounding boxes are generated via two FFN implemented in parallel. To enhance the detection of small components, various data augmentation methods are employed in the training process. The effectiveness of the proposed AttnConv-net is validated with one real dataset and another synthesized dataset. Compared with classic convolutional neural network based methods, our proposed method simplifies the detection pipeline by eliminating the need of prior- and post-processing, which offers a new speed-quality solution to enable faster and more accurate image-based rail component detections
CVAug 5, 2021
Intelligent Railway Foreign Object Detection: A Semi-supervised Convolutional Autoencoder Based MethodTiange Wang, Zijun Zhang, Fangfang Yang et al.
Automated inspection and detection of foreign objects on railways is important for rail transportation safety as it helps prevent potential accidents and trains derailment. Most existing vision-based approaches focus on the detection of frontal intrusion objects with prior labels, such as categories and locations of the objects. In reality, foreign objects with unknown categories can appear anytime on railway tracks. In this paper, we develop a semi-supervised convolutional autoencoder based framework that only requires railway track images without prior knowledge on the foreign objects in the training process. It consists of three different modules, a bottleneck feature generator as encoder, a photographic image generator as decoder, and a reconstruction discriminator developed via adversarial learning. In the proposed framework, the problem of detecting the presence, location, and shape of foreign objects is addressed by comparing the input and reconstructed images as well as setting thresholds based on reconstruction errors. The proposed method is evaluated through comprehensive studies under different performance criteria. The results show that the proposed method outperforms some well-known benchmarking methods. The proposed framework is useful for data analytics via the train Internet-of-Things (IoT) systems
CVJul 29, 2021
Viewpoint-Invariant Exercise Repetition CountingYu Cheng Hsu, Qingpeng Zhang, Efstratios Tsougenis et al.
Counting the repetition of human exercise and physical rehabilitation is a common task in rehabilitation and exercise training. The existing vision-based repetition counting methods less emphasize the concurrent motions in the same video. This work presents a vision-based human motion repetition counting applicable to counting concurrent motions through the skeleton location extracted from various pose estimation methods. The presented method was validated on the University of Idaho Physical Rehabilitation Movements Data Set (UI-PRMD), and MM-fit dataset. The overall mean absolute error (MAE) for mm-fit was 0.06 with off-by-one Accuracy (OBOA) 0.94. Overall MAE for UI-PRMD dataset was 0.06 with OBOA 0.95. We have also tested the performance in a variety of camera locations and concurrent motions with conveniently collected video with overall MAE 0.06 and OBOA 0.88. The proposed method provides a view-angle and motion agnostic concurrent motion counting. This method can potentially use in large-scale remote rehabilitation and exercise training with only one camera.