NeighboAR: Efficient Object Retrieval using Proximity- and Gaze-based Object Grouping with an AR System
This repository contains the supplementary material of the publication:
Aleksandar Slavuljica, Kenan Bektaş, Jannis Strecker, and Simon Mayer. 2024. NeighboAR: Efficient Object Retrieval using Proximity- and Gaze-based Object Grouping with an AR System. Proc. ACM Hum.-Comput. Interact. 8, ETRA, Article 225 (May 2024), 19 pages. https://doi.org/10.1145/3655599
Humans only recognize a few items in a scene at once and memorize three to seven items in the short term. Such limitations can be mitigated using cognitive offloading (e.g., sticky notes, digital reminders). We studied whether a gaze-enabled Augmented Reality (AR) system could facilitate cognitive offloading and improve object retrieval performance. To this end, we developed NeighboAR, which detects objects in a user’s surroundings and generates a graph that stores object proximity relationships and user’s gaze dwell times for each object. In a controlled experiment, we asked N=17 participants to inspect randomly distributed objects and later recall the position of a given target object. Our results show that displaying the target together with the proximity object with the longest user gaze dwell time helps recalling the position of the target. Specifically, NeighboAR significantly reduces the retrieval time by 33%, number of errors by 71%, and perceived workload by 10%.
If you have questions about this research, feel free to contact Aleksandar Slavuljica: aleksandar.slavuljica@student.unisg.ch or Kenan Bektaş: kenan.bektas@unisg.ch
This research has been done by the group of Interaction- and Communication-based Systems (interactions.ics.unisg.ch) at the University of St.Gallen (unisg.ch).
This project was funded by the Swiss Innovation Agency Innosuisse (#48342.1 IP-ICT) and the Basic Research Fund of the University of St.Gallen.
If you use/modify this source code or refer to our paper, please add a reference to our publication:
Aleksandar Slavuljica, Kenan Bektaş, Jannis Strecker, and Simon Mayer. 2024. NeighboAR: Efficient Object Retrieval using Proximity- and Gaze-based Object Grouping with an AR System. Proc. ACM Hum.-Comput. Interact. 8, ETRA, Article 225 (May 2024), 19 pages. https://doi.org/10.1145/3655599
@article{slavuljica2024,
author = {Slavuljica, Aleksandar and Bekta\c{s}, Kenan and Strecker, Jannis and Mayer, Simon},
title = {NeighboAR: Efficient Object Retrieval using Proximity- and Gaze-based Object Grouping with an AR System},
year = {2024},
issue_date = {March 2024},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
volume = {8},
number = {ETRA},
url = {https://doi.org/10.1145/3655599},
doi = {10.1145/3655599},
abstract = {Humans only recognize a few items in a scene at once and memorize three to seven items in the short term. Such limitations can be mitigated using cognitive offloading (e.g., sticky notes, digital reminders). We studied whether a gaze-enabled Augmented Reality (AR) system could facilitate cognitive offloading and improve object retrieval performance. To this end, we developed NeighboAR, which detects objects in a user’s surroundings and generates a graph that stores object proximity relationships and user’s gaze dwell times for each object. In a controlled experiment, we asked N=17 participants to inspect randomly distributed objects and later recall the position of a given target object. Our results show that displaying the target together with the proximity object with the longest user gaze dwell time helps recalling the position of the target. Specifically, NeighboAR significantly reduces the retrieval time by 33\%, number of errors by 71\%, and perceived workload by 10\%.},
month = {may},
articleno = {225},
numpages = {19},
keywords = { augmented reality, cognitive offloading, eye tracking, object detection, human augmentation}
}The code in this repository is licensed under the Apache License 2.0 (see LICENSE) if not stated differently in the individual files and folders.