George Candea

Beatriz Borges, Negar Foroutan, Deniz Bayazit et al.

AI assistants are being increasingly used by students enrolled in higher education institutions. While these tools provide opportunities for improved teaching and education, they also pose significant challenges for assessment and learning outcomes. We conceptualize these challenges through the lens of vulnerability, the potential for university assessments and learning outcomes to be impacted by student use of generative AI. We investigate the potential scale of this vulnerability by measuring the degree to which AI assistants can complete assessment questions in standard university-level STEM courses. Specifically, we compile a novel dataset of textual assessment questions from 50 courses at EPFL and evaluate whether two AI assistants, GPT-3.5 and GPT-4 can adequately answer these questions. We use eight prompting strategies to produce responses and find that GPT-4 answers an average of 65.8% of questions correctly, and can even produce the correct answer across at least one prompting strategy for 85.1% of questions. When grouping courses in our dataset by degree program, these systems already pass non-project assessments of large numbers of core courses in various degree programs, posing risks to higher education accreditation that will be amplified as these models improve. Our results call for revising program-level assessment design in higher education in light of advances in generative AI.

Amer Chamseddine, George Candea

Smartphones hold important private information, yet users routinely expose this information to questionable applications written by developers they know nothing about. Users may be tempted to think of smartphones as old-style dumb phones, not as powerful network-connected computers, and this opens a gap between the permissions-based security paradigm (offered by platforms like Android) and what users expect. This makes it easy to fool users into installing applications that steal their information. Not surprisingly, Android is now a more favored target for hackers than Windows. We propose an approach for closing this gap, based on the observation that the current permissions system--rooted in good ol' UNIX-style thinking--is both too coarse and too fine grained, because it uses the wrong axes for defining the permissions space. We argue for replacing the paradigm in which "an app accesses device resources" (which is foreign to most non-geeks) with a paradigm in which "an app accesses user-tangible services." By using a simple piece of middleware, we can wrap this view of application control around today's permission system, and, by doing so, no conceptual refactoring of applications is required.