Xiaoyi Zhang | ๅผ ้ชๆฟ | xiaoyiz@cs.uw.edu
Senior AI + Accessibility Researcher@Apple Seattle
UW CSE Ph.D., advised by James Fogarty
CV Google Scholar ยป
Xiaoyi Zhang | ๅผ ้ชๆฟ | xiaoyiz@cs.uw.edu
Senior AI + Accessibility Researcher@Apple Seattle
UW CSE Ph.D., advised by James Fogarty
CV Google Scholar ยป
Towards Complete Icon Labeling in Mobile Applications Jieshan Chen, Amanda Swearngin, Jason Wu, Titus Barik, Jeffrey Nichols, Xiaoyi Zhang CHI 2022 • PDF
Understanding Screen Relationships from Screenshots of Smartphone Applications Shirin Feiz, Jason Wu, Xiaoyi Zhang, Amanda Swearngin, Titus Barik, Jeffrey Nichols IUI 2022 • PDF • Official Page
Screen Recognition: Creating Accessibility Metadata for Mobile Applications from Pixels Xiaoyi Zhang, Lilian de Greef, Amanda Swearngin, Samuel White, Kyle Murray, Lisa Yu, Qi Shan, Jeffrey Nichols, Jason Wu, Chris Fleizach, Aaron Everitt, Jeffrey P. Bigham CHI 2021 • PDF • Official Page • Video ๐ Best Paper
When Can Accessibility Help?: An Exploration of Accessibility Feature Recommendation on Mobile Devices Jason Wu, Gabriel Reyes, Sam C. White, Xiaoyi Zhang, Jeffrey P. Bigham W4A 2021 • PDF • Official Page ๐ Best Paper
Screen Parsing: Towards Reverse Engineering of UI Models from Screenshots Jason Wu, Xiaoyi Zhang, Jeffrey Nichols, Jeffrey P. Bigham UIST 2021 • PDF • Official Page • Video
Robust Annotation of Mobile Application Interfaces in Methods for Accessibility Repair and Enhancement Xiaoyi Zhang, Anne Spencer Ross, James Fogarty UIST 2018 • PDF • Official Page • Video
Interactiles: 3D Printed Tactile Interfaces on Phone to Enhance Mobile Accessibility Xiaoyi Zhang, Tracy Tran, Yuqian Sun, Ian Culhane, Shobhit Jain, James Fogarty, Jennifer Mankoff ASSETS 2018 • PDF • Official Page • Video
Examining Image-Based Button Labeling for Accessibility in Apps through Large-Scale Analysis Anne Spencer Ross, Xiaoyi Zhang, James Fogarty, Jacob O. Wobbrock ASSETS 2018 • PDF • Official Page ๐ Best Paper Nominee
APPINITE: A Multi-Modal Interface for Specifying Data Descriptions in Programming by Demonstration Using Natural Language Instructions Toby Jia-Jun Li, Igor Labutov, Xiaohan Li, Xiaoyi Zhang, Wenze Shi, Wanling Ding, Tom M. Mitchell, Brad A. Myers VL/HCC 2018 • PDF • Official Page
Epidemiology as a Framework for Large-Scale Mobile Application Accessibility Assessment Anne Spencer Ross, Xiaoyi Zhang, James Fogarty, Jacob O. Wobbrock ASSETS 2017 • PDF • Official Page ๐ Best Paper Nominee
Interaction Proxies for Runtime Repair and Enhancement of Mobile Application Accessibility Xiaoyi Zhang, Anne Spencer Ross, Anat Caspi, James Fogarty, Jacob O. Wobbrock CHI 2017 • PDF • Official Page • Video
Smartphone-Based Gaze Gesture Communication for People with Motor Disabilities Xiaoyi Zhang, Harish Kulkarni, Meredith Ringel Morris CHI 2017 • PDF • Official Page • Video
Examining Unlock Journaling with Diaries and Reminders for In Situ Self-Report in Health and Wellness Xiaoyi Zhang, Laura R. Pina, James Fogarty CHI 2016 • PDF • Official Page • Video
Leveraging Dual-Observable Input for Fine-Grained Thumb Interaction Using Forearm EMG Donny Huang, Xiaoyi Zhang, T. Scott Saponas, James Fogarty, Shyamnath Gollakota UIST 2015 • PDF • Official Page • Video
See UV on Your Skin: An Ultraviolet Sensing and Visualization System Xiaoyi Zhang, Wenyao Xu, Ming-Chun Huang, Navid Amini, Fengbo Ren BodyNets 2013 • PDF • Official Page
Proper Running Posture Guide: A Wearable Biomechanics Capture System Xiaoyi Zhang, Ming-Chun Huang, Fengbo Ren, Wenyao Xu, Nan Guan, Wang Yi BodyNets 2013 • PDF • Official Page
An Epidemiology-Inspired Large-scale Analysis of Android App Accessibility Anne Spencer Ross, Xiaoyi Zhang, James Fogarty, Jacob O. Wobbrock ACM Transactions on Accessible Computing (TACCESS) 2020 • PDF • Official Page
Beacon: Designing a Portable Device for Self-Administering a Measure of Critical Flicker Frequency Ravi Karkar, Rafal Kocielnik, Xiaoyi Zhang, Jasmine Zia, George N. Ioannou, Sean A. Munson, James Fogarty IMWUT (Ubicomp) 2018 • PDF • Official Page
FridgeNet: A Nutrition and Social Activity Promotion Platform for Aging Population Yuju Lee, Ming-Chun Huang, Xiaoyi Zhang, Wenyao Xu IEEE Journal of Intelligent Systems, 2015 • PDF • Official Page
BreathSens: A Continuous On-Bed Respiratory Monitoring System with Torso Localization using an Unobtrusive Pressure Sensing Array Jason Liu, Ming-Chun Huang, Wenyao Xu, Xiaoyi Zhang, Luke Stevens, Nabil Alshurafa, Majid Sarrafzadeh IEEE Journal of Biomedical and Health Informatics (JBHI) 2015 • PDF • Official Page
Using Pressure Map Sequences for Recognition of On Bed Rehabilitation Exercises Ming-Chun Huang, Jason Liu, Wenyao Xu, Nabil Alshurafa, Xiaoyi Zhang, Majid Sarrafzadeh IEEE Journal of Biomedical and Health Informatics (JBHI) 2014 • PDF • Official Page
ON-THE-FLY CALIBRATION FOR IMPROVED ON-DEVICE EYE TRACKING Jeffrey P. Bigham, Mingzhe Li, Samuel C. White, Xiaoyi Zhang, Qi Shan, Carlos E. Guestrin Google Patent
PIXEL-BASED OPTIMIZATION FOR A USER INTERFACE Jeffrey P. Bigham, Colin Lea, Jason Wu, Xiaoyi Zhang Google Patent
METHODS AND SYSTEMS FOR SELF-ADMINISTERED MEASUREMENT OF CRITICAL FLICKER FREQUENCY (CFF) George Ioannou, James Fogarty, Jasmine Zia, Rafal Kocielnik, Ravi Karkar, Sean Munson, Xiaoyi Zhang Google Patent
AUTOMATED E-TRAN APPLICATION Xiaoyi Zhang, Harish Kulkarni, Meredith Ringel Morris Google Patent
USER INTERFACE CONTROL USING GAZE TRACKING Kai Kohlhoff, Xiaoyi Zhang Google Patent
Academic Conferences Invited Paper Reviewer Reviewed 30+ papers in CHI / UIST / MobileHCI / INTERACT / TEI / SUI / DIS.
MICROSOFT RESEARCH, 2017 Research Intern in Enable Team Open-sourced SwipeSpeak (Github), a phone app that combines human and machine intelligence to enable eye gesture communication. Will soon open-source an eye gesture recognition library (cross-platform).
MICROSOFT RESEARCH, 2016 Research Intern in Enable Team Built a low-cost, robust, portable, easy-to-learn and fast gaze gesture communication phone app for people with ALS.
GOOGLE RESEARCH, 2015 Research Intern in Machine Intelligence group Developed 3 new features in coLaboratory (Collborative iPython Notebook).
MOVNPIC, 2014 Co-founder & Chief Creative Officer Design & Develop an iOS app to create Interactive images.
GOOGLE RESEARCH, 2013 Research Intern in Computer Vision & HCI group Built non-rigid face features detection, pose & gaze tracking for 3 teams.
APPLE, 2012 (Freshman Summer) Software Engineering Intern in iCloud Implemented 1 new feature, improved UX of 3 webapps and fixed 47 bugs.
SUNSHINE LIBRARY, 2011-2013 Webmaster & Developer Charitable rural information education using low-cost Android Tablet like OLPC.
TALKWAYS, 2011-2012 Co-founder & Former CTO Video chat platform to learn language with culture; received $1 million investment.
DEMOHOUR, 2011 Inventor of "Gesture@Mac" Initiate the first project on Demohour, the largest crowd-funding platform in China now.
iPhones Can Now Automatically Recognize and Label Buttons and UI Features for Blind Users
Reported by TechCrunch about our Screen Recognition feature in iOS 14.
Microsoft app helps people with ALS speak using just their eyes
Reported by Newsweek / New Scientist / The Telegraph and 50+ medias.
Gesture@Mac
Reported by Engadget China / 36kr / China Youth Daily
Corin Anderson Endowed Fellowship, 09/2014 CSE Dept, University of Washington $50000 / It allows me to "pursue academic course and research work in complete freedom" (in the first year).
CESASC Scholarship, 04/2014 Chinese-American Engineers&Scientists Association of Southern California $1000 / One of national 10 winners, nominated by CESASC.
New Face of Engineering College Edition, 02/2014 National Society of Professional Engineers $500 / One of national 15 winners, nominated by IEEE and other Engineer Associations.
Student Participation Grant, 09/2013 European Alliance for Innovation $600 / Present two papers at Bodynets conference.
Best Overall, Mobile App&Data Visualization, 04/2013 LA Hacks, 40+ teams $7500 / Project EyeWeight attracted Eytan Elbaz (Google Adsense founder).
Second Place, 04/2013 Facebook Hackathon, 250+ participants $800 / Project AResume was appreciated by Facebook recruiter.
Grand Prize&Tech Difficulty Prize, 10/2012 HackUCLA, 20+ teams $33000 / Project Music++ attracted Howard Marks (Activision founder).
