01/09/2023 - 12/11/2023 (Week 01 - Week 10)
Wong Khye Qing / 0361000 Experiential Design/ Bachelor of Design (Hons) in Creative Media
Project Prototype
Instructions
Project Prototype
The project that was undergoing was about the Mobile application Augmented Reality (AR) for the Indoor Navigation of Taylor's University Lakeside campus. The targeted audiences shall be the freshmen and Taylor's students. Before proceeding into executing and having further design, the prototyping stage was done in order to determine the doability for the project by having multiple trials and errors, research on the mechanics of the functions to problem solve any issues ahead.
The overall progress that was done for the prototype were as below:
- Location scouting
- 3D scanning
- Unity and Vuforia
- Trial and errors
- Problem faced
- Future actions
1. Location Scouting
Fig 1.0 One of the scanning venues, Lecture Theatre 4
A targeted location would need to be chosen as 3D scanning of an area would be needed in order to create a indoor navigation app for students and freshmen. I have chosen the path from the Student Life Center (SLC) to Lecture Theatre 12 (LT.12). Alternate option would be to scan from BizPod to the Lecture Theatre 4 (LT.04), which was both of the venues were near to each others.
2. 3D Scanning
As for the scanning, I have tried using the Matterport from the app, after scanning, I found that the exports that I have got was not compatible with the Unity or Vuforia. I would need to scan an area using devices with Light Detection and Ranging (LiDAR), so that the scanning would be accurate for navigation.
Fig 1.1 Scanning Results and Export Format, Scaniverse
Devices such as Apple Pro products or Matterport Pro 2 scanner that contained LiDAR would be able to scan the areas. At first I would like to loan the Instax360 One 2 from the Vortex Lab, but it was later that I only realized that the Matterport site would required Matterpak for exports that is not free. Then, I enquired about the loan of iPad Pro device for scanning, which I got hold of it and did some scanning of the areas from SLC to LT.12 and BizPod to LT.04. Later on, I exported the files in .FBX and .OBJ, in hopes that the file formats would be compatible with the Unity, but turns out it was not compatible, so I needed to rescan some areas again with alternate tools.
Fig 1.2 Scanning Venue and Scanned Area Targets, Vuforia Studio
After failing to import scans from Scaniverse to the Vuforia Area Target in Unity, I switched to using the Vuforia Studio that was downloaded in iPad Pro to scan the area. The scanning process was a bit tedious for the this app due to it kept on requiring me to relocate the camera to previous location for localization, which lengthen the scanning process. Multiple scans were attempted in order to obtain a better or clearer scan for the Area Target. *Each Area Target generation consumes one credit from the Vuforia account (total 20 credits)
This time around I have took the advice of Mr Razif in scanning in smaller scale and for short distance in order to minimize the scanning time. I have scanned a few venues including the ones to test on my own just so in case I was not in the campus, I would be able to test it within where I was staying. After scanning and generating the scans as Area Target. I shared the .ZIP file from the scans to Google Drive in order to download it from the iPad to my laptop.
Problem faced
- Matterport scanning only provided paid exports services, using MatterPak
- Scannning using Scaniverse was easy and efficient, but file formats can’t be imported into as Area Target
- Scanning area was too big for limited scanning space of Vuforia
Trials and errors
- Test on Matterport mobile as the scanning is user friendly, failed attempt to export files
- Scan using Scaniverse, very convenient and unlimited scanning duration, failed to import into Area Target
- Vuforia Studio was used for scanning and area target file was generated and compatible with the Unity and Vuforia engine
3. Unity and Vuforia
Fig 1.3 Failed attempt to file conversion and Model Target, Vuforia Model Target Generator
Unity was used for this assignment with the integration of the Vuforia engine. I have set up the Unity file by opening a new file as usual, importing the Vuforia package into Unity and adding the Key code into the AR camera. Upon finishing the 3D scan, there was an attempt to import those scan files into the Unity. Turns out export files from Scaniverse do not have the correct file format to be imported into Unity. I have also tried with Vuforia Model Target Generator to convert the .FBX and .OBJ scan file from Scaniverse into Unity package file, but it only worked as a Model Target upon importing into the working file.
Fig 1.4 Limited file formats available for uploads, Vuforia Area Target Generator
Same goes to the Vuforia Model Target Generator, it required Vurforia's native file format, .3DT, in order to generate an Area Target. I needed to scan using the Vuforia Studio app in order to get a .3DT file. This problem problem was solved by generating the scans from Vuforia Studio app as Area Target to get the Unity file upon exporting.
Fig 1.5 3D Object Spawned, AR Camera
With the export using the Vuforia Area Target Generator, a Unity file was generated and could be imported into the Area Target in Unity for indoor navigation. Before the starting of creating any lines for the navigation, I have tried to test out was the AR Camera able to recognise the scanning and display the 3D Object that I have placed into the area. After several attempts, the 3D Object was able to be spawned when I used the AR Camera in Simulation Mode as well as exporting to my phone.
Fig 1.6 Line Renderer linkage with cubes, YouTube
Fig 1.7 Line Renderer connected to cubes, C# Line script
After that, I have tried to use the Line Renderer feature, I have tried to adjust the Position for it, unfortunately, it was not very tricky to make the line turn to directions that I desired. I was having a hard time adjusting it and I sorted to find another way to make the line bend to the directions that I wanted. After that, with some searching, I came across a video about connecting the Line Renderer with cubes that secured its position, I was able to stabilize the line with some script and linking the line with the cubes.
Fig 1.7 Cubes as the child of AR Camera, failed attempt
Fig 1.8 Cubes as the child of AR Target, failed attempt
Fig 1.9 First cube as the child of AR Camera, successful attempt
It was until then I tested out with the AR Camera in simulation that I found out that the line was not following me when I move the camera left and right. I have attempt to put the Line Renderer component into the AR Camera, but it still did nothing. After that, I put the first cube as the child of the AR Camera, as I thought that the all the cubes were connected to each other, the first cube would be able to guide the line I and follow the movement of the AR Camera. Turns out it worked and the Line Renderer was moving with the camera as well.
Fig 1.10 Line appeared without target and enabling features
Unfortunately, despite no AR Targets being detected, the whole Line Renderer appeared on the AR Camera. This problem was solved with the help of Mr Razif, by enabling the disabled Line Renderer and Line Script connecting the cubes and line.
Despite solving the issue above, the Line Renderer including the cubes appeared in front of the Area Target which might make the whole experience of navigation unpleasant. I was trying to figure out on how to make the lines that should be blocked by the wall to not appear in front of the wall. Mr Razif has suggested that I can create multiple Line Renderers and cubes, with box collider to detect any collision. the idea was to make a few points and let each turning lines to appear if the user moved to the next and activate the box collider. Unfortunately, this idea was later scrapped as it has seemed to be inefficient.
Problem faced
- Files converted from the Scaniverse (.FBX and .OBJ) converted into Model Target in Vuforia Model Target Generator.
- 3D Object not showing up in AR Camera
- Line is not following the user movement and stuck in its original position
- Line Renderer shown even when no area target was detected
Trials and errors
- Used Vuforia Area Target Generator to convert .3DT files
- Key in the License Key into AR Camera and 3D Object appeared
- Turns out scans from Vuforia Studio can be generated as Area Target and save the hassle to generate area target in Vuforia Area Target Generator
- Created line script to connect Cubes with Line Renderer and set the first cube as the child of AR Camera to follow user movements.
- Activated the "On Target Found" feature by only enabling the line script and the line renderer upon detecting the area target and spawn the line for navigation
4. Future actions
I found that the line renderer and cube option is not that efficient, it required a lot of manual configuration and consumed time. I might opt for the using Nav Mesh to bake a path from the area target. This is in order to automate the process of generating a line and allowing the line to follow the user movements. As of now, I might needed to tap into coding to automate more process.
As for 3D scanning, I don't have much of an issue with it. It was just that I might require to scan a few more venues in the campus in case that I needed to change some area targets. Besides, I needed to complete the overall user interface design in order to provide better user experience.
Video Walkthrough
Fig 2.0 Recorded Video Walkthrough of Prototype, YouTube
The whole video consisted of a few parts, which included:
- Prototype Walkthrough in Unity
- Explanations of some problem faced
- Prototype Walkthrough in using phone
Feedbacks
Week 10
Overall the prototype is good enough to prove that the AR mechanism for the indoor navigation works. Proceed in improving the overall navigation of the app and the interface.
Week 9
If the whole line still appeared, you can create multiple line renderers with cubes, use box colliders to activate and spawn the line upon user colliding with the hitbox of the line renderer.
Week 8
If the line renderer appeared upon opening the app, just use the On Target Found in the area target to enable the line renderer and line script when the area target was found.
Week 7
Try to scan a smaller venue, it does not have to be a venue that is far from each other