Md Ahasanuzzaman

2papers

2 Papers

47.0AIMay 30
FALAT: Tracing Failures in LLM Agent Trajectories via Dependency-Guided Search

Md Nakhla Rafi, Md Ahasanuzzaman, Dong Jae Kim et al.

LLM-based agents increasingly solve complex tasks through long trajectories involving reasoning steps, tool calls, and inter-agent communication. However, when these agents fail, it is often unclear which agent caused the failure and which step introduced the decisive error. This attribution problem is challenging because mistakes can propagate across the trajectory: later actions may appear incorrect, but only because they depend on an earlier corrupted state. Therefore, failure attribution cannot be treated as independent step-level classification. We propose FALAT, a diagnostic framework for failure attribution in LLM agent trajectories. FALAT frames attribution as a dependency-guided search problem. It first constructs an expectation of how the task should be solved and uses this expectation to identify suspicious regions in the trajectory. It then traces dependencies among decisions, tool outputs, and agent messages to distinguish error-introducing steps from steps that merely inherit or propagate prior mistakes. Finally, FALAT evaluates whether correcting a candidate step would be sufficient to recover the expected outcome, allowing it to identify both the responsible agent and the decisive failure step. We evaluate FALAT on the Who&When benchmark, which includes both algorithm-generated and hand-crafted multi-agent failure trajectories. The results show that FALAT consistently improves responsible-agent and decisive-step attribution. Its best configurations achieve 46.0% step-level accuracy on algorithm-generated trajectories and 29.1% on the more challenging hand-crafted trajectories, outperforming specialized attribution baselines and direct prompting with standalone LLMs. These findings suggest that dependency-aware reasoning is essential for reliable failure diagnosis in LLM agent systems.

SEApr 1, 2021
Studying Ad Library Integration Strategies of Top Free-to-Download Apps

Md Ahasanuzzaman, Safwat Hassan, Ahmed E. Hassan

In-app advertisements have become a major revenue source for app developers in the mobile app ecosystem. Ad libraries play an integral part in this ecosystem as app developers integrate these libraries into their apps to display ads. In this paper, we study ad library integration practices by analyzing 35,459 updates of 1,837 top free-to-download apps of the Google Play Store. We observe that ad libraries (e.g., Google AdMob) are not always used for serving ads -- 22.5% of the apps that integrate Google AdMob do not display ads. They instead depend on Google AdMob for analytical purposes. Among the apps that display ads, we observe that 57.9% of them integrate multiple ad libraries. We observe that such integration of multiple ad libraries occurs commonly in apps with a large number of downloads and ones in app categories with a high proportion of ad-displaying apps. We manually analyze a sample of apps and derive a set of rules to automatically identify four common strategies for integrating multiple ad libraries. Our analysis of the apps across the identified strategies shows that app developers prefer to manage their own integrations instead of using off-the-shelf features of ad libraries for integrating multiple ad libraries. Our findings are valuable for ad library developers who wish to learn first hand about the challenges of integrating ad libraries.