AIセキュリティポータル K Program
Sketch-based Access Control: A Multimodal Interface for Translating User Preferences into Intent-Aligned Policies
Share
Abstract
Developing simple and expressive access controls -- interfaces to specify policies that define who should have access to resources and under what circumstances -- is a longstanding challenge in usable security. We present Sketch-based Access Control (SBAC), a sketch-based, AI-assisted access control authoring system that combines the expressive power of sketching with the interpretive capabilities of multimodal large language models (MLLMs) to support the interpretation and validation of policy specifications as they are iteratively refined. Through a formative study with 14 participants, we identified three design requirements and developed a human-AI collaborative workflow composed of three stages -- Specify, Analyze, and Test -- enabled by the system's ability to maintain and interpret evolving access control specifications. In a user evaluation with 14 participants grounded in their real-world access control scenarios, we found the system and the workflow helped participants progressively refine initially underspecified preferences into more complete and precise policies -- surfacing gaps they had not anticipated, resolving ambiguities through dialogue, and validating policy behavior through concrete scenarios.
The security blanket of the chat world: An analytic evaluation and a user study of telegram
Ruba Abu-Salma, Kat Krol, Simon Parkin, Victoria Koh, Kevin Kwan, Jazib Mahboob, Zahra Traboulsi, M Angela Sasse
Published: 2017
SketchREAD: a multi-domain sketch recognition engine
Christine Alvarado, Randall Davis
Published: 2007
Real life challenges in access-control management
Lujo Bauer, Lorrie Faith Cranor, Robert W Reeder, Michael K Reiter, Kami Vaniea
Published: 2009
Reflecting on reflexive thematic analysis
Virginia Braun, Victoria Clarke
Published: 2019
Experience prototyping
Marion Buchenau, Jane Fulton Suri
Published: 2000
Sketching user experiences: getting the design right and the right design
Bill Buxton
Published: 2010
Controlling fine-grain sharing in natural language with a virtual assistant
Giovanni Campagna, Silei Xu, Rakesh Ramesh, Michael Fischer, Monica S Lam
Published: 2018
Demystifying hidden privacy settings in mobile apps
Chen Y, Zha M, Zhang N, Xu D, Zhao Q, Feng X, Yuan K, Suya F, Tian Y, Chen K
Published: 2019
TaleBrush: Sketching stories with generative pretrained language models
John Joon Young Chung, Wooseok Kim, Kang Min Yoo, Hwaran Lee, Eytan Adar, Minsuk Chang
Published: 2022
The policy continuum–Policy authoring and conflict analysis
Steven Davy, Brendan Jennings, John Strassner
Published: 2008
How do humans sketch objects?
Mathias Eitz, James Hays, Marc Alexa
Published: 2012
Verification and change-impact analysis of access-control policies
Kathi Fisler, Shriram Krishnamurthi, Leo A Meyerovich, Michael Carl Tschantz
Published: 2005
Ambiguity as a resource for design
William W Gaver, Jacob Beaver, Steve Benford
Published: 2003
Intent tagging: Exploring micro-prompting interactions for supporting granular human-GenAI co-creation workflows
Frederic Gmeiner, Nicolai Marquardt, Michael Bentley, Hugo Romat, Michel Pahud, David Brown, Asta Roseway, Nikolas Martelaro, Kenneth Holstein, Ken Hinckley, et al.
Published: 2025
Rethinking Access Control and Authentication for the Home Internet of Things (IoT)
Weijia He, Maximilian Golla, Roshni Padhi, Jordan Ofek, Markus Dürmuth, Earlence Fernandes, Blase Ur
Published: 2018
Guide to attribute based access control (ABAC) definition and considerations
Vincent Hu, David Ferraiolo, D. Kuhn, A. Schnitzer, Knox Sandlin, R. Miller, Karen Scarfone
Published: 2014
Attribute-based access control
Vincent C Hu, D Richard Kuhn, David F Ferraiolo, Jeffrey Voas
Published: 2015
Verification and test methods for access control policies/models
Vincent C Hu, Rick Kuhn, Dylan Yaga, et al.
Published: 2017
Automated verification of access control policies using a SAT solver
Graham Hughes, Tevfik Bultan
Published: 2008
Technology probes: inspiring design for and with families
Hilary Hutchinson, Wendy Mackay, Bo Westerlund, Benjamin B Bederson, Allison Druin, Catherine Plaisant, Michel Beaudouin-Lafon, Stéphane Conversy, Helen Evans, Heiko Hansen, et al.
Published: 2003
Less is not more: Improving findability and actionability of privacy controls for online behavioral advertising
Jane Im, Ruiyi Wang, Weikun Lyu, Nick Cook, Hana Habib, Lorrie Faith Cranor, Nikola Banovic, Florian Schaub
Published: 2023
Usable policy template authoring for iterative policy refinement
Maritza Johnson, John Karat, Clare-Marie Karat, Keith Grueneberg
Published: 2010
My data just goes Everywhere: user mental models of the internet and implications for privacy and security
Ruogu Kang, Laura Dabbish, Nathaniel Fruchter, Sara Kiesler
Published: 2015
Evaluating interfaces for privacy policy rule authoring
Clare-Marie Karat, John Karat, Carolyn Brodie, Jinjuan Feng
Published: 2006
A study of privacy settings errors in an online social network
Michelle Madejski, Maritza Johnson, Steven M Bellovin
Published: 2012
ImaginationVellum: Generative-AI Ideation Canvas with Spatial Prompts, Generative Strokes, and Ideation History
Nicolai Marquardt, Asta Roseway, Hugo Romat, Payod Panda, Michel Pahud, Gonzalo Ramos, Steven M Drucker, Andrew D Wilson, Ken Hinckley, Nathalie Riche
Published: 2025
Measuring privacy: An empirical test using context to expose confounding variables
K. Martin, H. Nissenbaum
Published: 2016
Access control for home data sharing: Attitudes, needs and practices
Michelle L Mazurek, JP Arsenault, Joanna Bresee, Nitin Gupta, Iulia Ion, Christina Johns, Daniel Lee, Yuan Liang, Jenny Olsen, Brandon Salmon, et al.
Published: 2010
Sketching for real-time control of crowd simulations
Luis Rene Montana Gonzalez
Published: 2021
On building a visualisation tool for access control policies
Charles Morisset, David Sanchez
Published: 2018
Privacy expectations and preferences in an {IoT} world
P. E. Naeini, S. Bhagavatula, H. Habib, M. Degeling, L. Bauer, L. F. Cranor, N. Sadeh
Published: 2017
Privacy as contextual integrity
Helen Nissenbaum
Published: 2004
Turtles, locks, and bathrooms: Understanding mental models of privacy through illustration
Maggie Oates, Yama Ahmadullah, Abigail Marsh, Chelse Swoopes, Shikun Zhang, Rebecca Balebako, Lorrie Faith Cranor
Published: 2018
Expandable grids for visualizing and authoring computer security policies
Robert W Reeder, Lujo Bauer, Lorrie Faith Cranor, Michael K Reiter, Kelli Bacon, Keisha How, Heather Strong
Published: 2008
Effects of access-control policy conflict-resolution methods on policy-authoring usability
Robert W Reeder, Lujo Bauer, Lorrie Faith Cranor, Michael K Reiter, Kami Vaniea
Published: 2009
Usability challenges in security and privacy policy-authoring interfaces
Robert W Reeder, Clare-Marie Karat, John Karat, Carolyn Brodie
Published: 2007
World-driven access control for continuous sensing
Franziska Roesner, David Molnar, Alexander Moshchuk, Tadayoshi Kohno, Helen J Wang
Published: 2014
Role-based access control
Ravi S Sandhu
Published: 1998
A robust sketch interface for natural robot control
Danelle Shah, Joseph Schneider, Mark Campbell
Published: 2010
Using a hand-drawn sketch to control a team of robots
Marjorie Skubic, Derek Anderson, Samuel Blisard, Dennis Perzanowski, Alan Schultz
Published: 2007
Access control policy extraction from unconstrained natural language text
John Slankas, Laurie Williams
Published: 2013
Bridging the gulf of envisioning: Cognitive challenges in prompt based interactions with LLMs
Hari Subramonyam, Roy Pea, Christopher Pondoc, Maneesh Agrawala, Colleen Seifert
Published: 2024
Transparent barriers: natural language access control policies for XR-enhanced everyday objects
Kentaro Taninaka, Rahul Jain, Jingyu Shi, Kazunori Takashio, Karthik Ramani
Published: 2025
Teaching Data Science Students to Sketch Privacy Designs Through Heuristics
Jinhe Wen, Yingxi Zhao, Wenqian Xu, Yaxing Yao, Haojian Jin
Published: 2025
Modeling End-User Affective Discomfort With Mobile App Permissions Across Physical Contexts
Yuxi Wu, Jacob Logas, Devansh Ponda, Julia Haines, Jiaming Li, Jeffrey Nichols, W Keith Edwards, Sauvik Das
Code shaping: Iterative code editing with free-form AI-interpreted sketching
Ryan Yen, Jian Zhao, Daniel Vogel
Published: 2025
Share