Sprint 3
Prepared by: Liam Mazure, Chase Mulder, Mohammad Saleh, and Lucas Seeterlin
Team HoloLens’s third sprints intended progress was to get the educational application demo running on the Hololens headset and to start working on the Unity C# scriptable objects and full body models. The group split the work on the full body models up by person. Chase does skeleton, Lucas does skin, Liam is on organs, and Mohammad is on the vascular system. Individually, the group will work on implementing each of their full body models into the final Unity scene as well as updating the corresponding user interface. When they upload, they will push to separate branches on github with specific commit messages to keep track of their progress.
The group has now met twice with the previous developers of the project in order to get any questions answered as well as learn about the structure of how they created the C# scripts and associated scriptable objects. During these meetings, the group discusses ideas and potential features to be added into the project with the previous developers. These meetings have helped the group clarify some issues that have been encountered in the past such as a recurring bug that was preventing the group from creating a project build.
Liam’s plan for the third sprint was to create additional reference points for the full body anatomical model. The additional points include left and right temple, nose, chin, elbow, wrist, knee, and ankle. These points allow for the 3D model to be rendered in proportion to the user's specifications. Liam also continued to research the possible methods for implementing a magnification tool into the Unity project to allow the user to select a magnifying glass and select different parts of the model to zoom in on. Allowing the user to to view selected parts in more detail. Liam’s plan also included increasing the size of the model's lattice and setting particular reference points to particular points on the lattice.
Chase set up a One Note to post code snippets and pictures to layout the Unity C# code and project’s assets at a high level. This planning helped find what the scriptable assets were named like the SkinnedMeshRenderer and where to add buttons and sliders to the User Interface Manager. Currently, there are scripts for the UI, app, and interactions. The UI is user interface, app is the application for the hololens that runs the educational app, and interactions is the hololens native code in C# for controlling the hands and ar points in augmented reality space. The project sponsor wants all these scripts folders worked on with priority on the application and user interface. Changing things like making the UI curved in 3d space, or collaborating with others will come later.
Lucas dedicated the majority of his time during the first half of the sprint digging into the Unity project files to see what the project already contained, and to see what the group will need to add (specifically regarding the User Interface scripts and files) to include the full body model provided to them. For the second half of the sprint, Lucas contacted the client to see what changes they would like to see in the current UI for the project. This includes new icons for the buttons the group plans on adding as well as color and font changes, which Lucas began editing towards the end of the sprint. At this point in time, Lucas has created over a dozen new buttons to match the previous buttons, such as buttons to display the brain, spinal cord, gastrointestinal system, and much more. This required some image manipulation using the Paint3D software in which Lucas spent some time towards the end of the sprint researching.
Mohammad's goal was to isolate the veins from the rest of the body. So going through and writing and scripts for the function. It was a lot of time dispatched towards troubleshooting unity software as well as building issues. As the team spent time meeting we came up with solutions, and spending time answering questions we had really speed up the process.
Liam senses a great deal of excitement from the group at this point in the semester. As a whole there was a sizable amount of progress made on all areas in the projects. Which has only inspired the team to continue to work hard throughout the last half of the semester. Individually Liam was able to complete much of what he set out to finish. Even with a set back regarding a dependency issue while trying to push changes to his branch on the teams repository. Liam however was unable to extend the lattice for the full body model as he found implementation difficult due to the need to bring together all aspects of the project from different members.
Chase spent a moderate amount of time on getting the Unity project to work on his home computer. It took using the Unity Hub app to get the specific version the project was made in and downloading the 11Gb update. A little time and 20$ was spent on buying a TurboSquid asset store model and testing it in our project. The group’s original idea was to purchase 1200$ male and female anatomy models from TurboSquid, but it was found that the previous group’s models were adequate. A majority of the time spent this sprint was on deep diving into the Unity project and understanding what was going on and how to develop further.
Lucas feels a sense of accomplishment at the end of this sprint, as the group successfully overcame issues encountered in the first and second sprints such as not being able to demo the project on the Hololens headset and not having access to the correct Unity files. Now that the group has everything they need, changes are already being made to the project per client request. Continuing to meet with the client and previous developer team makes Lucas feel confident in the progress the group is making, with every meeting leading to more ideas to be implemented. Lucas believes that the next sprint will be somewhat heavy with the work that is expected to be completed, but with various resources available he believes that the work will be completed in a timely manner.
Mohammad believes he made progress isolating the veins from the anatomy of the male and female torso. As well found a solution to stabilizing unity on a linux distribution system. Found that unity runs much smoother on linux distro than on windows 10. Also there was an issue with Unity and HoloLens files being memory heavy. So it would take half an hour to load each time, especially when it would crash. Now it has no problem with rendering without crashing. With a few final tweeks it will be ready to connect to a button. So more time will be spent integrating the the isolation of the veins
Liam struggled with a dependency issue that he encountered while trying to push changes to the teams repository. This prevented Liam from completing work other than handling the issue for a few days. However he was able to fix the issue with the help of the previous team who worked on this project. Liam also found through the research he conducted on how to implement the magnification tool that the addition of another camera into the unity scene would be difficult for the HoloLens to handle and could cause potential problems while running the program. With further discussion with the team Liam was able to come to the conclusion that another method of implementation running the magnification through a connected computer would be the ideal solution.
Chase ran into the problem of deploying the educational application demo on the Hololens headset. The project sponsor sent over a .bundle file and just said open it on the headset. It took some debugging at first because the group thought the app you ran on the Hololens was HoloViewer and the same on the PC, but that’s for the medical app. Then the team ran into certificate issues when trying to run the .bundle file on the PC because the Microsoft store wouldn’t let them run an unapproved app. Finally, the team was able to connect the Hololens to the PC via usb-c and turn on developer mode on the headset and administrative permissions on the PC and send the .bundle file to the headset to run.
Lucas encountered a couple issues throughout the last sprint, the first being failure to match the color of the new icons to the already created ones. After talking with the previous developers, they suggested that Lucas create new icons for everything that already exists in the project since the color and “glow” around the icons were customly created by a graphics artist at the company. The second issue that Lucas ran into was figuring out how to attach the new parts of the model that the group is adding into the project to Unity’s scriptable objects function, which the group asked the previous developers about in order to fix the issue. After a brief discussion and demonstration, the group now feels confident in solving any issues that have been encountered so far.
Mohammad struggled with unity being so taxing on his computer. Unity would crash his computer each time it would freeze. And he would restart the work when it didn't save. So he uploaded unity and the project on a removable drive, but that led to the system lagging. So he decided to run it on a linux distribution system and it turned out to run really smoothly. Since he was able to find some stability it made exploring the hololens file much easier. Mohammad also found it difficult to find resources to understand problems or creative ideas. So when there is a problem you need to take a lot of time to think about it.
Liam feels that the team's progress this past sprint has left everyone in high spirits. Liam is planning on making the magnification of the 3D models with a magnifying glass possible by implementing it so the computer can run the process and render it on the HoloLens. Liam also intends to increase the size of the lattice once he is able to view the full model to allow for the reference points he created to be lined up properly to each specified body structure.
Chase has made progress on the skeletal system overlay on his home PC. The icon for the skeletal system UI was found at https://thenounproject.com/ . The noun project has made free icons for everything! The full body skeletal model was pulled into the main Unity scene and progress is being made on tying the C# scripts to the full body skeletal model. The UI was moved around to add the skeletal icon and when clicked on it will bring up the many different bones and their names.
Lucas feels that the new icons and scriptable objects that have been added thus far is just a sample of what the group can do in the future while working on this project. During this sprint, Lucas educated himself on image manipulation using Paint3D, and used and will continue to use this software for icon creation. He also learned more about how scriptable objects work since the previous group used them very heavily throughout the project, and with this understanding he can now make additions to the project with the full body model such as new systems and organs that were not expected to be added in the beginning couple sprints.
Mohammad has made significant progress isolating the veins and soon linking a button to the isolation of the main feature. Progress was made and quite a bit of time writing out some script and connecting them to the asset folder where the veins components are connected. Also got unity stabilized, it ran smoothly without any crashing.
The third sprint as a whole went as planned for the majority of the time. The educational application demo was deployed on the Hololens headset, the Unity full body model scriptable objects and C# scripts are being made individually on home PCs, and the group has been meeting constantly with the client and previous developer group. Another thing to mention, the project sponsor sent the group his Hololens headset in the mail, and the group can use this headset outside of the Atomic Object showcase room. This means the group can now create builds at home and run tests through the headset rather than having to go into the showcase room while it is available. This burndown chart is the group’s most accurate one yet, showing progress being made throughout the two week period. The reason that only 50ish percent of the issues were completed is the group created some large tickets that are being completed over multiple sprints, so these issues are still a work in progress.
Issues moved into Sprint 4: -Continuing to create new icons and scriptable objects -Magnifying glass
Issues created for Sprint 4: -Redesigning old UI buttons to include updated images -Flash cards and other educational features
Liam’s experience at this point in the semester makes him feel confident in the team's progress. He sees the work ethic everyone has brought to the project and is excited to continue checking off different tasks that present themselves. Everyone comes to each meeting wanting to work through issues and regularly communicate what needs to be done in order for the team to stay on track for the current sprint. Liam knows that the work the team will showcase in the coming sprints will make him feel proud of the work each member has been able to accomplish.
Chase feels like this project is really going to get rolling now that everyone has all the Unity files running on their own PCs. By the end of the semester, our group hopes to have a demo in use for two Hololens headsets. This project is going to challenge Chase in the areas of Unity and C# coding and he hopes to get better because of it. This project is also challenging because working with coding things for the Hololens is so new, so there’s not much information online on how to do things. There’s also a lot of assets to manage in this project and it’s hard to keep track of where everything is stored. By the end of it all, Chase hopes to know a lot more about Unity and the process in creating an application.
Lucas feels confident in the work that the group has done so far including the new additions to the UI and grouping together various parts of the full body model in order to add more functionality into the project. Now that the group has gotten into a development rhythm, it is easier to predict what can be completed in time. Lucas expects that the next sprint will be busy with a lot of new features being added in such as the ability to magnify parts of the body and more educational features such as tooltips and flashcards.
Mohammad is looking forward to Completing the isolation of the veins and starting on mirroring multiple hololenses. Now since there aren't very many resources to refer to, it is a lot of team collaboration and asking for help from the previous development team, which has been extremely resourceful. The next sprint will be combining and making everything functional.
As a whole, the team feels optimistic about the future of the project. They now have all the tools that are needed to implement a great deal of changes to the initial code, including the ability to contact the previous group for questions as well as understanding the structure of the project. As time with the project increases, the group's understanding continues to improve and new ideas such as the ability to hold a magnifying glass and zoom into parts of the body arises, giving the group more features to add. The changes and additions to the UI as well as bringing in the full 3D model has been a vital part to the teams first three sprints, but they know there is a long way to go and lots of more features to add before the project meets the high standards they set for themselves.