Indoor Item Locator with ARKit

A survey shows that 77% of smartphone owners regularly use navigation apps. Indeed, with the help of Global Positioning System (GPS), navigation apps can make the directions more in-depth and precise for the users. However, GPS doesn’t work so well indoors. In many games, there are guidance systems (waypoints) that help players find the mission target more easily. With the improvement of mobile phone performance, nowadays smart phone can handle heavy tasks like image processing in 60 fps. We want to use state-of-the-art technology like augmented reality (AR) to make it easier to find items in the store.

This project is motivated by Google Map AR extention in Google I/O 18 and Shared experiences with ARKit 2 in WWDC 2018. The goal is to create a indoor guidance system to help users locate items in a grocery store.

---

What have been done

• Tag and retrieve real world items with Augemented Reality in our app.

• Object recognition with Inception V3 model using CoreML.

• Share ARWorldMap with one another using MultipeerConnectivity.

Challenges

• How large ARWorldMap can be shared?
• Inception V3 is not tailored for items in a grocery store, is there any better model, or we have to train it ourselves.

Demo

Functionalities

Host

• use "+" button to tag item
• "single tap" on screen to show item list
• "double taps" to close item list
• use "save" button to save current worldMap
• use "retrieve" button to retrieve saved worldMap
• when user is nearby, use "share" button to share the worldMap

User after receiving worldMap

• "single tap" on screen to show item list
• "double taps" to close item list
• select item to show waypoint

References

• Building Your First AR Experience
• Creating a Multiuser AR Experience
• Bluetoothed ARKit 2.0 with ARWorldMap!
• Using Vision in Real Time with ARKit