Teaching: Wearable Cognitive Assistants

Aimed at integrating cutting-edge psychological science into the classroom, Teaching Current Directions in Psychological Science offers advice and how-to guidance about teaching a particular area of research or topic in psychological science that has been the focus of an article in the APS journal Current Directions in Psychological Science.

More teaching resources from the Observer: Parenting by Lying

Klatzky, R. L., & Satyanarayanan, M. (2023). Psychological Science Meets Wearable Cognitive Assistance. Current Directions in Psychological Science, 32(6), 446-453.

’Twas the night before Christmas, when all through the house
The parents were swearing, waking the mouse.¹
There were toys to assemble, and more than a few;
The dollhouse took an hour, and the trampoline took two.
With paper instructions too brief and guidance videos too distant,
What these parents needed was a wearable cognitive assistant.  

Headworn devices such as Google Glass, augmented reality glasses, and virtual reality headsets have been on the market for approximately a decade, with most users enjoying them for taking photos, checking apps, and gaming. Others use wearable devices to create multiple virtual “screens” that augment work efficiency, an approach that was anticipated by the 2002 Steven Spielberg movie Minority Report.  

According to APS James McKeen Cattell Fellow Roberta Klatzky and computer scientist Mahadev Satyanarayanan, wearable devices can act as cognitive assistants, providing real-time guidance with assembling furniture, cooking a new recipe, or operating a defibrillator during a sudden cardiac arrest (Klatzky & Satyanarayanan, 2023). The idea is similar to GPS navigation in which a sensor detects your location and interacts with a computer system (digital maps) to provide step-by-step guidance for you to reach your destination.  

GPS navigation, though, turns out to be a relatively simple example of effective human–computer interaction because it requires minimal sensors (GPS) and relatively minimal computing resources (maps). There can be a time lag between location detection and verbal instructions without the user missing a turn. By contrast, cognitive assistance with furniture assembly requires a computational model for each action in the task, dense processing of the user’s 3D environment, and the wireless exchange of data between the sensor and a nearby small computer called a cloudlet (without the nearby computer, processing would be too slow).  

Despite the many technical challenges, Satyanarayanan, Klatzky, and colleagues have programmed wearable devices such as Google Glass to assist users in building Lego structures, assembling Ikea lamps, and guiding novices to use a defibrillator (Chen et al., 2017; Olguín Muñoz et al., 2021; Zhao et al., 2020).  

Several lessons from their research program will align with your classroom teaching activity (see below): 

1. The speed of the wearable cognitive assistant needs to match or exceed the speed of innate human cognitive processes; if it’s too slow, then users lose motivation because they no longer feel a sense of control over their environment.   

2. Wearable cognitive assistants not only must match the individual user’s cognitive abilities in processing speed and working memory, but they may also need to be tailored to different personalities, demographics, and cultures.   

3. Despite remarkable technological advances, human cognitive processing still holds advantages over computer-based models, which take a bottom-up approach to sensing the environment and are therefore susceptible to errors such as objects being blocked by the user’s hands. Without observable cues, computers simply lack the top–down conceptual understanding that a goal is completed, which is a cornerstone of human cognition.   

Footnotes 

¹ The poem ’Twas the Night Before Christmas was written by Clement Clarke Moore in 1823 under the title Account of a Visit from St. Nicholas. 

Feedback on this article? Email [email protected] or login to comment.

References 

Chen, Z., Hu, W., Wang, J., Zhao, S., Amos, B., Wu, G., Ha, K., Elgazzar, K., Pillai, P., Klatzky, R. L., Siewiorek, D., & Satyanarayanan, M. (2017, October). An empirical study of latency in an emerging class of edge computing applications for wearable cognitive assistance. In Zhang, J. (Chair), Proceedings of the Second ACM/IEEE Symposium on Edge Computing (Article 14, pp. 1–14). https://doi.org/10.1145/3132211.3134458 

Klatzky, R. L., & Satyanarayanan, M. (2023). Psychological science meets wearable cognitive assistance. Current Directions in Psychological Science. https://doi.org/10.1177/09637214231187912 

Olguín Muñoz, M., Klatzky, R., Wang, J., Pillai, P., Satyanarayanan, M., & Gross, J. (2021). Impact of delayed response on wearable cognitive assistance. PLOS ONE, 16(3), Article e0248690. https://doi.org/10.1371/journal.pone.0248690 

Zhao, S., Wang, J., Leng, H., Liu, Y., Liu, H., Siewiorek, D., Satyanarayanan, M., & Klatzky, R. (2020). Edge-based wearable systems for cognitive assistance: Design challenges, solution framework, and application to emergency healthcare [Technical Report CMU-CS-20-102]. School of Computer Science, Carnegie Mellon University. https://www.cs.cmu.edu/~satya/docdir/CMU-CS-20-102.pdf