C#, Unity, HoloLens 2, AR, MRTK2, HoloLens 2 Emulator
Disclaimer: Not yet tested on a HoloLens 2 (and needs changes. Postmortem coming soon.)
1 week sprint project for HoloLens 2. Draw lines in Augmented Reality on the Hololens 2 with MRTK2 and a variety of features.
^ custom AI created promotional artwork for Draw Note
The controls in Draw Note are the regular style of Hololens 2 controls.
While drawing: The User holds out arm in front of their body in a comfortable position. In Normal Mode the drawing will happen just beyond this. The User can move their hand, arm, head, body and look around while continue to draw. Read Drawing Modes later for more details.
There is 3 ways to interact with buttons:
- The User's Gaze is directed at the Button from a distance, and then they make the Air Tap gesture (focus gaze then right click in emulator)
- The User says the Voice Command associated with the button/action ex. "RED", "DRAW", "STOP"
- The User does a nearby hand interaction, by actually pressing the button with their finger
There are two Menus that are context dependant:
- Small Note Drawing Menu - Is smaller and is active while the User is drawing. It simply directs the user to say "STOP" to stop, and it has a Stop Button
- Drawing Details Menu - Is larger and is active while the User is not drawing. It contains buttons for input options listed below, except Stop
| Voice Command | Description |
|---|---|
| DRAW | Start drawing |
| STOP | Stop drawing |
| MODE | Switch drawing mode |
| UNDO | Undo last drawing |
| CLEAR | Clear all drawings |
| WHITE | Set draw color to white |
| RED | Set draw color to red |
| YELLOW | Set draw color to yellow |
| GREEN | Set draw color to green |
It's important for the User to understand and switch between the different Drawing Modes to get the expected behavior.
| Drawing Modes | Description |
|---|---|
| Normal | Draw only on draw plane with hand (Ex. you want to write clearly like on a piece of paper) |
| Mesh | Draw only on meshes with hand (Ex. you want to write on a wall, or floor) |
| Finger | Draw from your finger (Ex. you want to draw a line exactly from your finger position) |
- User Inputs are interpreted by MRTK2 for the correct environment
- Some custom configurations are applied directly to MRTK profile (Ex. custom voice inputs)
- Drawings are stored in an Object with a Unity ParticleTrail component (Each saved node of the instance Drawing requires very little storage)
- Create new drawing Object when starting to draw, or changing colors for easy Undo layers
- Menus follow the User and stay within the camera frustum as per Hololens 2 style. They can also be repositioned
- Unit Tests in Unity Test Runner
Draw Plane description (only relevant while using Normal Mode): Default target drawing distance is 64 centimeters away (approx average human arm length) from the User. DrawPlane GameObject's center is set to 1 meter away with a depth/thickness of 72 centimeters (to more safely catch raycast). It's position and rotation is always set to be that distance and facing the User. So if the User draws directly in the middle of where their looking, that ray would hit the collider at 64 centimeters away.