12/11/2023 - 14/12/2023 - (Week 10 - Week 15)
Wong Khye Qing / 0361000 Experiential Design/ Bachelor of Design (Hons) in Creative Media
Final Project
Instructions
Final Project
This is the conclusion of the project for the Experiential Design course, compiling and completing the prototype to make it into a mobile application which features Augmented Reality (AR) feature. This final project would be to pinpoint on the features that was yet to be completed in the previous prototype to refine the features into a functional mobile application.
The overall progress of the final project would be documented as below:
The overall progress of the final project would be documented as below:
- Sketches and ideas
- Recap on previous progress
- Unity and scripts
- Live Walkthrough
- App Flow
- Limitations and Alternatives
- Potential improvements
- Learning outcomes
- Reflections and conclusion
1. Sketches and ideas
Fig 1.0 Previous sketch, Latest digital sketch and app icon sketch
The comparison of the of the previous and current digital sketch do have their differences due to some of the features were being unrealistic and wasn't fit for the app. One of the features was not included was finding parking in the campus feature, which would have allowed students to find the parking zones in the campus. It was ultimately dropped due to the some time constraints. In contrast, there was an additional feature, which showed the instructions or tips upon the first time of opening the app to guide the users on some features in the app.
On the other hand the app icon were also sketched for a few times. The first one featured only the AR word which might looked to boring. Whereas the second and third sketch have altered the Taylor's University middle section element to a location icon, I thought that it might not be a good idea to alter the brand identity of an organization or establishment. Which it had led to me of just putting the location icon beside the university's logo to signify that this app would be able to help in finding locations.
2. Recap on previous progress
Fig 2.0 Previous progress of prototype
Previously the 3D scanning were done an imported into the Unity Editor as Area Targets. The line renderer was connected with cubes in order to spawn more accurately in the AR space. Additional venues were scanned in order to be able to have more options and could served as backup. The Line was still unstable as of now as it might not update fully by truly follow the user progress and reduce the line's length when the user is walking.
Next in line actions from previous prototype was that I might opt into baking NavMesh, learning how to automating the line update process by using scripts and scan additional venues as backups.
3. Unity and scripts
NavMesh
Fig 3.0 Inputting the NavMesh into Area Target, Panels added, Unity Editor
Before the baking of NavMesh, I imported the NavMesh as the child of the scanned Area Target. This was the first step before baking the NavMesh onto the Area Target. Some Panels were also added for the baking as the scan for the Area Target was not that complete, so the Panels were created in order to smoothen out the lines for a better quality in baking the NavMesh.
Fig 3.1 NavMesh Baking Configurations, Unity Editor
Some configurations and settings for the NavMesh was also adjusted. Firstly the Mesh Renderer was set to Navigation Static in order to bake the NavMesh. Next, the Agent Radius was set to a smaller number, 0.03, to have a better and accurate NavMesh for the baking process, the smaller the number to longer the time for it to be rendered and baked. After that, the NavMesh could be baked by pressing on the Bake button.
Fig 3.2 Baked NavMesh, Top view of Baked NavMesh, Unity Editor
With that the NavMesh was baked to the Area Target with the Blue mesh within the scanned Area Target. The White NavMesh could be turned off in order to not overtake the appearance of the Area Target.
Exploration of Script Automation
Fig 3.3 Exploring on the Line Renderer, YouTube
Fig 3.4 Line Renderer updates on the player's location, YouTube
I have also searched some alternatives in order to explore on how to automate the script and enable the Line Renderer to update and follow the users direction same as how the video have shown. It was able to calculate and generate the lines with the help of the NavMesh, it was able to determine on the shortest route to the reward.
This video was demonstrating for mobile game navigation, but I thought it might be a great idea to implement this into the indoor navigation, since it was able to generate and update the line. With the integration of the AR camera as the user, a Line Renderer could be generated and guide the user to their destination with accurate line direction updated on the screen as the user moves.
Fig 3.5 Collectable Spawner, C# Script
Then the script was modified from the video in order to suit the indoor navigation usage with the help of my friend. This script that has some alterations that enable an object to spawn which was suppose to be your destination. The baked NavMesh would be used to generate the Line Renderer, as it would be used as a base to determine on the walkable path of the users. Then, the position of the user would be calculated and a shortest path between the user and the destination would be calculated and created to guide the user. The generated line would constantly be updated in order to project the accurate path to the destination with the position of the user.
Fig 3.6 Script functions, C# Script
This part of the script would calculate on the position of the user (AR camera) with the destination's position and also to determine the number of corners that the it would need to reach the destination. Then one of the Prefab's would be spawn on the position for the destination with some height offset of spawn above the destinations position. The calculation will kept on updated with unless should any error occur a debug log would be shown to pinpoint on the error.
Fig 3.7 Configuration of the Collectable Spawner
With the script completed, I was able to determine on some configurations with the script. The Prefab would be spawning the Prefab which would be located at the destination. The Player would be the the AR Camera which represented the user that would be moving when using for navigation. The Path would be the Line Renderer which would be spawn according to the NavMesh. Meanwhile the Path Height Offset would be determining on the Line Renderers spawning height at the destination. Besides that the Actual Goal would representing the destination by allowing user to move its position to the destination's position.
Fig 3.8 Line Renderer followed the direction of the user, Unity Simulation
With the script settled, the simulation of the line has also shown that the Line Renderer was also able to updated according to the position of the user while the player is moving left to right. Furthermore, it was able to spawn a line and guide the user to the destination. It was great seeing it able to follow me whenever I was maneuvering around.
Canvas and Scenes
Fig 3.9 Horizontal Scroll Tutorial, YouTube
In order to learn on how to integrate Horizontal Scroll into the UI of the app, I found a video on YouTube that was able to allow me to swipe the instruction cards left to right and vice versa. It was also able to record on which card I was viewing by the dots below the cards.
i) Instruction page
Fig 3.10 Outcome of the Instruction Card and Horizontal scrolling Configuration, Unity Editor
The function of this Instruction Page was to show the instruction and tips of some features in the app. There were instruction cards that the user can view by swiping left or right, they can also press on the dots below the cards to view different cards. The highlighted red dot indicated the current viewing card. These features were enable by integrating components and scripts into it, the Horizontal Layout Group component and Swipe Script.
The Horizontal Layout Group component allows the user to align the contents to the middle center as well as other positions, meanwhile also allowing users to customize on the paddings for the content box and the Spacing between each cards.
This script allow the user to customize on the color of the dots before and after pressing of the button or moving of the instruction cards by the user. It was also able to determine on the Scrollbar and Image content in order to allow the current dot to be enlarge when the user is viewing the current card meanwhile other cards remain at its original size.
ii) Login page
Fig 3.12 Text Input Box and its Configurations, Unity Editor
Text Input box were also added for the Login page by integrating the InputField (TMP) component into the canvas. I could also change the Content Type of the input box to limit on what characters that can be inputted, for instance the Decimal Numbers for the Student ID.
iii) Main page
Fig 3.13 Main Page and Horizontal Scrolling, Unity Editor
The Main page consisted of a taskbar below and a scroll feature same as the Instruction page with the option to allow the users to go to different locations in the campus. The horizontal scroll feature was altered a little by removing the dots below allowing a cleaner horizontal scrolling experience in the Main Page.
iv) Lecture Theatre 04 & Lecture Theatre 17
Fig 3.14 Canvas and the Area Target of the LT04 and LT17, Unity Editor
Canvas were inputted into both of these scenes to create a UI for the page while in the AR view for navigation. The navigation details were put below the page in order to show the destination as well as the distance left for between the user and the destination. Besides the NavMesh for both of the Area Targets have been baked in order for the Collectable Spawner Script to be able to run and calculate the distance of the user and generate the Line Renderer.
Fig 3.15 NavMesh baking trial and error, Unity Editor
The initial baking of the NavMesh for the first half scanned Area Target were not completed on the floor. Through some trial and errors I was able to include panels on the floor in order to bake the NavMesh again to even out the flaws on the floor that might affect the navigation experience and generation of the Line Renderer
Fig 3.16 Line Renderer color configuration, Unity Editor
There some errors in changing the colors of the Line Renderer at first as seen in the first image, the color still remained as pink even though I have changed the color in the configuration. Fortunately, I was able to solve this issue by importing Materials element, which is a sprite, into the the Line Renderer. After that, I was able to change the color of the Line Renderer with no hassle.
Fig 3.17 Indoor Navigation simulation of LT04 and LT17, Unity Simulation
Simulations were ran through the for both of the Lecture Theatre, the images above have shown the interface of the indoor navigation and the perspective of the player in the Area Target. The Line Renderer was able to spawned successfully according to the baked NavMesh and was able to follow the position of the user and update the Line Renderer. There were also no error for the scripts and the calculation for the line worked well and fine.
App Icon
Fig 3.18 Adding the App Icon, Unity Editor
I also went to the Player Settings in the Build Settings in order to change the App Icon for the app. It could be done by just adding the app icon into the Default Icon section from the Player Settings. Screenshot image of the app icon was also shown above.
Fig 4.0 Live Walkthrough of launching the app, Taylor's Finder App
Before launching the app, users would need to tap on the app icon for the Taylor's Finder App. Then, they would be guided to the instruction page with instruction or tips cards that they can swipe left or right to view them. After that, users would need to log in to the app using their Student ID and passwords to access the app.
Upon logging in, users would be guided to the Main Page which would be shown with taskbar and locations that they could access for indoor navigation. Then Users could swipe left or right for the locations button in order to have a better view of Lecture Theatre 17 button.
Fig 4.1 Live Walkthrough to LT04 and LT17, Taylor's Finder App
After pressing on the button for respective locations, the user would be transported to the AR view for the indoor navigation where there would be a line that guide them to their destination. Information of the destination and the distance left would be displayed below the interface as well as having direction icons to direct on where the user should move forward. The Line Renderer would be updated by calculating on the user's position as well as the destination and update on the length of the line based on the distance that the user have taken.
Fig 4.2 Errors and Misalignments occurred, Taylor's Finder App
Although the Line Renderer was able to be rendered into the AR environment for the app, there were some errors and misalignments that occurred in the app. This would meant that the line might not be spawning according to the position it should be on the area target guiding the users to the destination. It was unable to detect the Area Target and spawned the line randomly, which might be make the lines look weirdly rendered on the AR environment as well as misguiding the user.
Fig 5.0 Overall App Flow, Taylor's Finder App
The overall app flow was simple, users would be given instruction and tips cards for guidance. They would also required to login to the app to be directed to the Main Page. That page would have location buttons for users to choose from. The current app only has two locations Lecture Theatre 04 and 17. Then, they would be directed to the AR indoor navigation space which they would be a line generated to guide them to their destination.
6. Limitations and Alternatives
Limitations
Fig 6.0 Incomplete baked NavMesh from scanning of Area Target
The limitations for this assignment was that the Area Target scanning might sometimes be incomplete making some paths to be generated weirdly as well as making the generation of the line to avoid any incomplete baked NavMesh path.
Besides that, the script for the calculation of spawning the Line Renderer might be limited to the fact that it was only able to calculate on the path given from the scanned Area Target, in comparison with some that used the GPS or location tracking for more accurate positioning.
The calculation of distance between the user and the destination was hard to be displayed for the user to see the distance left to the destination. Some sophisticated script might needed to be implemented in order to be able to display the calculated distance.
Alternatives
Fig 6.1 Alternatives besides using scripts, Taylor's Finder App
One of the alternatives was an idea from the previous prototype, in which the line was connected using the 3D cubes that were positioned on the AR Target. The Line Renderer would be then connected to the 3D cubes and one of the cube was added as the child of AR Camera in order to follow the user movements. This idea was a simple one, considering that the lines were connected with a simple script.
Unfortunately, the downside of it was that the lines were too fixed on the Area Target that even with the movements of the user being followed, it was unable to update well. For example the corners of the line would be remained on the same spot even though the user walked over it. This caused the Line Renderer not updated well and accurately, which might ruin the user experience.
Alternative Idea Google Drive File Submission Link:
7. Potential improvements
Fig 7.0 Improvements that could be done, Calculated distance and Parking feature
The mobile application might be a little lack of features, some refining works needed to be done in order to polish the features for the indoor navigation. The script could be amended to include more features into it, such as including the distance left for the user to travel to the destination and updating the icons for the directions the users should take. A more accurate positioning of the Line Renderer could also be done by having better scanned Area Targets as well as implementing some GPS location tracking to have a more accurate positioning for the indoor navigation feature.
The proposed parking feature could be added as an additional feature app that could help in pinpointing car park zones for the user. Besides that, the integration of pop ups could be done in order to confirm some user's actions as well as alerting the user for any completion of tasks.
8. Learning outcomes
Fig 8.0 The processes of this assignment
- Area Target scanning
- UI layout mechanics in Unity
- C# Scripting
- Line Renderer and Area Target
Through this course I have a few learning outcomes that thought me to understand more about the AR. Firstly, the Area Target has tested my patience in testing multiple alternatives for a good Area Target, it has also allow me to have hands on experience in scanning on-site venues.
Next, the UI layout mechanics in Unity has enable to connect between pages and able to allow me to input the course of actions upon pressing the button. I was also able to learn about the horizontal scrolling which enabled me to enhance the user interface and experience. The UI design in Unity is also important to express the interface look of the app and have a better browsing experience for the user.
The C# scripting has allowed me to automate some process while using some features, it was one of the fundamental part of the assignment in which would be able to allow more customization. The C# script has also allowed more trial and errors in fixing some errors during simulation or during build and run in the app.
Then, the Line Renderer and Area Target were one of the few outcomes that were worth the learning, it was able to guide the user with the line. The Area Target was able to allow me to understand about the scanning an area in order to allow some 3D elements to recognize the area and spawn at the exact position in the AR environment.
Final Project App Walkthrough
Fig 9.0 Recorded Final Project App Live Walkthrough, YouTube
The whole video consisted of a few parts, which included:
- Unity Walkthrough
- Mechanics and scripts
- Live mobile application Walkthrough
- Flaws and disadvantages
- Alternatives and ideas
- Learning outcomes and reflections
Google Drive File Submission Link:
https://drive.google.com/drive/folders/1PTBnLwK3RFrYwVFFIQsg5mZtf69VFrue
https://drive.google.com/drive/folders/1PTBnLwK3RFrYwVFFIQsg5mZtf69VFrue
Alternative Idea Google Drive File Submission Link:
Reflections
This course have thought me a lot about trial and errors. I was able to learn from some of the mistakes that were done in this assignment and able to amend the errors as well as making it function better. There was this sense of achievement and satisfactory when the Line Renderer and Area Target was able to work as intended by spawning and able to track the position of the user. I have experienced the process of starting the whole assignment from scratch by exploring 3D scanning for the Area Target, it was a new thing for me but I was very happy to touch on the technical and practical side of the assignment. I also did not regret sticking to the indoor navigation idea instead of switching ideas, it was also thanks to Mr Razif for encouraging me and providing feedbacks towards my progress. I was glad that I was able to do something other than the ordinary and challenged myself into making indoor navigation that made me interested into AR. I have also found a lot of alternatives for scanning an area by using multiple software or hardware, multiple attempts have been done in order to find the right software to capture and export for Area Target, it has been a great experience exploring them and noting them down in this blog. Finally, it has been a great pleasure to complete the indoor navigation app, it has been a ride and a fun one, thank you for the ones that have helped me along the way.