Synthetic Dataset for Evaluating Complex Compositional Knowledge for Natural Language InferenceSushma Anand Akoju, Robert Vacareanu, Haris Riaz et al.
We introduce a synthetic dataset called Sentences Involving Complex Compositional Knowledge (SICCK) and a novel analysis that investigates the performance of Natural Language Inference (NLI) models to understand compositionality in logic. We produce 1,304 sentence pairs by modifying 15 examples from the SICK dataset (Marelli et al., 2014). To this end, we modify the original texts using a set of phrases - modifiers that correspond to universal quantifiers, existential quantifiers, negation, and other concept modifiers in Natural Logic (NL) (MacCartney, 2009). We use these phrases to modify the subject, verb, and object parts of the premise and hypothesis. Lastly, we annotate these modified texts with the corresponding entailment labels following NL rules. We conduct a preliminary verification of how well the change in the structural and semantic composition is captured by neural NLI models, in both zero-shot and fine-tuned scenarios. We found that the performance of NLI models under the zero-shot setting is poor, especially for modified sentences with negation and existential quantifiers. After fine-tuning this dataset, we observe that models continue to perform poorly over negation, existential and universal modifiers.
2.8SEJun 5, 2019
Adroitness: An Android-based Middleware for Fast Development of High-performance AppsOscar J. Romero, Sushma A. Akoju
As smartphones become increasingly more powerful, a new generation of highly interactive user-centric mobile apps emerge to make user's life simpler and more productive. Mobile phones applications have to sustain limited resource availability on mobile devices such as battery life, network connectivity while also providing better responsiveness, lightweight interactions within the application. Developers end up spending a considerable amount of time dealing with the architecture constraints imposed by the wide variety of platforms, tools, and devices offered by the mobile ecosystem, thereby diverting them from their main goal of building such apps. Therefore, we propose a mobile-based middleware architecture that alleviates the burdensome task of dealing with low-level architectural decisions and fine-grained implementation details. We achieve such a goal by focusing on the separation of concerns and abstracting away the complexity of orchestrating device sensors and effectors, decision-making processes, and connection to remote services, while providing scaffolding for the development of higher-level functional features of interactive high-performance mobile apps. We demonstrate the powerfulness of our approach vs. Android's conventional framework by comparing different software metric