04.04.2023 - 12.06.2023 / Week 1 - Week 12
Yung Siew Weng / 0339847
Advanced Interactive Design /
Bachelor of Design (Hons) in Creative Media
Project 1 / Interactive AR Application
Advanced Interactive Design - Project 1 : Interactive AR Application
INSTRUCTION
Students are required to develop an AR application for an online store. For example, an AR application for glasses, hats or cosmetic shops. Customers of the shop will use the AR application to try different options before making their purchase. Students are welcome to propose new AR application ideas as well, but must obtain the approval from the lecturer.
EXPLORATION
➜ Week 1 : Brainstorm at least 3 ideas for your own filters
Since the project requirement was to develop an AR application for an online store, we are used to brainstorming and designing filters that may be applicable for the store. So, it can be stated that the filters should be something that is a product. And, this filter can be used by the customers to try different options before making their purchase in the online store.
Instead of creating something that is fun and entertaining like the sharing filter under Lectures and Exercises #Week 1, I felt that something more "formal" was needed for an online store. I have lots of ideas popping out of my little brain: hair accessories like hats or hair clips; jewellery or watches; clothing; shoes like Poizon; etc. But since the final outcome was to create a site for the product, I wish that I could make the site full and not just halfway. For example, if I design an AR try-on filter for an online closet store, it seems to not be complete as normally a closet has lots of appearances, including clothes, dresses, pants, etc. So, I came to the conclusion that I had the idea of creating an AR filter for glasses, contact lenses, or lipstick.
#Ideation 1 : A try-on filter for eyeglasses
Fig 1.1 Ideation 1, #draft 1 (week 1/ 09.04.2023)
The idea of a try-on filter for eyeglasses stems from the desire to provide customers with a convenient and immersive shopping experience. By using augmented reality technology, users can virtually try on various eyeglass frames, helping them make more informed decisions about their purchase without the need to physically visit a store. This idea aims to enhance customer satisfaction, increase engagement, and simplify the process of selecting the perfect pair of eyeglasses.
#Ideation 2 : Lipstick Try-on filter
Fig 1.2 Ideation 2, #draft 1 (week 1/ 09.04.2023)
The idea of a lipstick try-on filter stems from the desire to provide users with a convenient and interactive way to explore different lipstick shades. Trying on lipstick virtually allows users to see how different colours would look on their own complexion, eliminating the need for physical product sampling. This idea aims to enhance the shopping experience, empower users to make confident choices, and encourage exploration of various lipstick shades in a fun and engaging way.
#Ideation 3 : Contact Lens Try-on filter
Fig 1.3 Ideation 3, #draft 1 (week 1/ 09.04.2023)
The idea of a contact lens try-on filter inspired from the desire to provide a practical and informative tool for individuals who wear or are considering contact lenses. Virtual try-on experiences can help users visualize how different contact lens styles and colors would look on their eyes, allowing them to make a more confident decision before purchasing. This idea aims to enhance user engagement, simplify the process of selecting contact lenses, and offer a unique AR experience in the realm of beauty and fashion.
Feedback _ Week 2 / 11.04.2023
Good contents. But you can develop things beyond the module information instructions. For this semester, we are not required to build an online store, so it could be more freedom for you to create anything that is beyond the brief. For the contact lens ideation, it seems not suitable to create a filter as the outcome will be very small and focus on the parts of the eyes. For a filter, it will be difficult to see what the style and the affordances are that you have created.
Due to a delayed workshop, the development of Project 1 had to be temporarily halted, due to the fact that we did not a specific theme to explore and further develop. As a result, we are required to switched our "channel" to project 2 in order to not snow-balling the future assignments.
After a brief introduction to the TikTok Effect House during the TikTok Workshop, we have now reached Week 5 and are "excited" to resume our project. With a refreshed understanding of the Effect House and its capabilities, we are eager to dive back into the development process and explore new possibilities for our TikTok effects. In the meanwhile, we were required to reproducing our 3 ideation proposal based on the criteria given.
For this particular project, students/ we are required to participate in the “Effect House Campaign” which categorised the theme in 4 categories :
➜ Creative Randomiser
➜ Engaging Games
➜ Fashion & Beauty
➜ 3D world AR
In fact, after attending the TikTok workshop, I realised that creating filters using the built-in features in TikTok has become incredibly easy due to technological advancements. It seems like anyone can create a filter within minutes, which made my initial ideas of glasses, lenses, or lipstick filters feel basic and unexciting. I began to question the uniqueness and value of my proposed ideas, feeling like they were too simple and lacking interactivity.
However, I don't want to settle for something generic or amusing just for the sake of it. I want to create something engaging in my own style, something that stands out and is not perceived as silly or trivial. Before diving into brainstorming, I decided to conduct some research on the four categories to gain inspiration and find a fresh perspective. And, I had compile the research, project criteria, etc under the PDF below as an overview :
Fig 1.4 Compilation, Research Ideation (week 5/ 07.05.2023)
Visual Slides (week 5/ 07.05.2023)
Feedback _ Week 6 / 09.05.2023
For Idea 1: Masquerade Mask, you can consider incorporating a cultural motif such as the Sarawak culture motif, which can add a unique and intriguing element to the filter. For Idea 3: AR Butterfly Wing Filter You can create integrating frames with the iconic identity of Tinker Bell in the foreground as well as some magical effects that immerse users in a whimsical world and bring a touch of enchantment to the filter.
After the feedback and discussion, I choose the #ideation 3 : Augmented Reality (AR) butterfly wing as my final proposal idea with the ideation of create a "Tinker Bell" identity frames as well as some magical effects. ["And, if possible, make it to 3D" IF].
To create a 3D model, I downloaded a 3D application, Nomad on iPad. I invested a significant amount of time during these weeks to explore and work on creating a 3D model using the application.
Fig 1.6 Screenshot, Proof of "exploration" (week 6/ 12.05.2023)
However, as I had taken too much times on exploring the features of the application, I was unable to showcase my progress during the feedback session in week 7.
Feedback _ Week 7 / 13.05.2023
You need to make a decision regarding the 2D or 3D model, or any necessary plan changes, by the end of this week due to time limitations.
During Week 8, which was the Independent Learning Week, I took full advantage of the time to catch up on my progress. I actively sought out various resources and tutorials to enhance my understanding of the tools and functionalities within the Nomad application. One tutorial that caught my attention was on creating a jelly cake, which provided me with a practical way to explore the basic features of Nomad. While creating a jelly cake may seem unrelated to my project, it served as a valuable exercise in familiarising myself with the essential functions and controls of Nomad as well provided me with a foundation to further develop 3D models.
Fig 1.7 Screenshot, “Jelly Cake" exploration (week 7/ 28.05.2023)
Fig 1.8 Outcome, “Jelly Cake" exploration (week 7/ 28.05.2023)
After following the tutorial, I was able to experiment with different tools, manipulate shapes, and gain a better grasp of how to navigate and utilise the features of the application. So, I started playing around with the shapes and create my "wing".
Fig 1.9 Development #1 (week 7/ 28.05.2023)
While I managed to grasp the foundational knowledge of creating 3D models, I found that the process was more challenging than anticipated. However, despite the difficulties, I thoroughly enjoyed the overall experience and found it to be interesting and fun as it provided me with a valuable opportunity to develop in 3D modeling. Considering that the wing will have a similar design on both the right and left sides, I decided to focus on creating one side of the wing and duplicate the side after.
Fig 1.10 Development #2 (week 7/ 29.05.2023)
I felt there is "shadows" on the wing, but I recognised that further development was necessary. And, I thinking of what the possibility of this wing will be transformed.
Fig 1.11 Development #2 Conceptual Thinking (week 7/ 29.05.2023)
I had initially planned to use the side as a reference to create a ribbon-like wing [Fig 1.11]. However, due to my unfamiliarity with the application, I did not save the progress, and the file was lost. This setback made me question the feasibility of the idea and whether it would be suitable to pursue further. I continued my search for tutorials and diligently followed them, experimenting with various methods. Along the way, I encountered both successes and failures. After persistent trial and error, I successfully created a wing that appeared smoother and more refined compared to my previous attempts.
Fig 1.12 -13 Development #3, Transparency Wing (week 8/ 30.05.2023)
At this moment, I explore the features "matcap", and added some texture and colour :
Fig 1.14 -15 Development #3 with texture (week 8/ 30.05.2023)
While there is still room for improvement, I am satisfied with the progress made thus far. I having the intention to try this outcome in TikTok effect house but the software had shown a pop-up screen that "my file had triangle count had exceeds 50k limit".
Fig 1.16 Issue in TikTok Effect House (Week 8/ 30.05.2023)
As I was not able to solve this issue for current, I recall back my ideation. Since initial plan to create a slightly larger, full-body wing, I delved into further research and experimentation. I remain committed to refining my skills and techniques in order to create an even better and more polished final product. I embarked on the development process once again. Through continuous trial and error, I managed to create a new version of the wing that met my desired specifications.
Fig 1.17 Development #4, Shell-liked wing (week 8/ 30.05.2023)
Feedback _ Week 9 / 30.05.2023
It seems like a shell but overall great, continue your development.
After receiving feedback on the shell-like appearance of my previous version, I made the decision to switch to an ocean theme with a more defined wing shape. I continued my development and arrived at the new outcome, which I felt that it brings along a sense of coral outline.
Fig 1.18 Development #5 (week 8/ 31.05.2023)
Furthermore, I received feedbacks suggesting that the wings should not have a thick component. Taking this into consideration, I revisited the features and refined my design to create a new version.
Fig 1.19 Development #6, One side (week 9/ 03.06.2023)
After completing the creation of one side of the wing, I was pleased with the outcome as it closely resembled a butterfly wing and decide to flip the another side.
Fig 1.20 - 21 Development #6, Full Outcome (week 9/ 03.06.2023)
While it may not be deemed as perfect, I am proud of the progress I have made from starting with zero knowledge to achieving the current outcome through self-exploration and learning. I decide to use this version as my final outcome for the AR wing filter.
Fig 1.22 - 23 Development #6 (week 9/ 03.06.2023)
After finalising the 3D wing model, I continued the development into TikTok effect house and finally I am able to import the file into the software. //TikTok effect house only accepted 3D file that < 5MB// However, the new issue came, which the textures and materials was not able to import into.
Fig 1.24 TikTok Effect House, Development (week 10/ 05.06.2023)
I had figured and explored the issue for a few days, and just figured that the texture from Nomad should export separated from the obj file, but the issue is there are no option for me to export the texture //predict: maybe the texture having copyright from Nomad, yet they are not allowed the user to export with the texture//. To solve this issue, I took a screenshot of the texture and adjust the colour code in Adobe Illustrator, and import the "texture".
1. Create a MatCap [Asset > Material > MatCap]
2. Import the texture [Assets > Import > From Computer]
3. Drag the texture file under the MatCap
4. Go to the 3D model and change the texture to "MatCap"
Fig 1.25 Add Texture (week 10/ 05.06.2023)
After encountering some difficulties with the texture, I sought advice from my friend, Pei Yun. She shared her own experience and suggested using external textures and plugging them into the TikTok Effect House. And, I used her proposed solution and finally I'm able to insert textures into the software.
Fig 1.26 Gradient MatCap Texture (week 10/ 06.06.2023)
Continued to make the "beauty" of the filter, I used the features of the TikTok Effect House to create some make up filter on it, such as light lipstick, eyelashes, etc in order to able the user to be more enjoyable when they used the filter.
Fig 1.27 Add Makeup filter (week 10/ 06.06.2023)
Then, I decided to try the filter in TikTok for a preview. During the try-ons for the wing, I noticed an issue where the wing didn't move with the body. And, I felt that it would be more interactive if the wing could be animated to flip. I searched for a solution on the TikTok Discord channel but couldn't find a satisfactory answer. In light of this, I reached out to Mr. Razif for assistance with the issue. At the same time, I shared my progress with him and received some feedbacks regarding the current progress :
Feedback _ Week 10 / 08.06.2023
This wings look nice. The most 3D animation need to be done in 3D software like blender and not sure whether we can direct animate in TikTok Effect House or not. And, you might add on buttons and more options of wing for user to choose in order to hit the outcome of "interactive design", maybe the hand gestures.
Despite the uncertainty mentioned by Mr. Razif, he requested my file and stated he will assist in exploring the features related to the issue. However, considering the time limitations and the long queue Mr. Razif had, I also decided to continue exploring the possibilities to directly animate the wing in TikTok Effect House, but ultimately gave up on that approach.
Instead, I downloaded and started exploring Blender, a 3D software. I found tutorials on how to create "bones" and animate objects in order to make my wing animated.
Fig 1.28 Blender exploration (week 10/ 10.06.2023)
Fig 1.29 Outcome, Animated Wing in Blender (week 10/ 10.06.2023)
WHO KNOWS! I am really surprised and happy that I was able to manage with Blender as I never use this software before. Nevertheless, thank for all the powerful tutorial video that I had found online. Although a 5 min video tutorial make me exploring for more than 4 hours, rebuild the bones for > 5 times, the end result was well worth the effort.
Then, I continued my progress on TikTok Effect House, I re-import the new 3D model and showing the flipping outcome.
Fig 1.30 Animated 3D Wing in TikTok Effect House (week 10/ 10.06.2023)
Then, I added textures for the wing. Since Mr. Razif had mentioned on suggesting to put button in order to create a more interactive design of filter, I followed the tutorial given by him that shared in the group on "Buttons to Change Filter effects/ visual" and my new outcome of interactive version of wings was released.
Fig 1.31 Interactive Button, Followed Tutorial (week 10/ 10.06.2023)
I tried the filter on TikTok and realised that the buttons I initially implemented in TikTok appeared unattractive on the screen and were not efficient for immediate wing changes, as the filter was designed for full-body effects. In order to address this, I decided to explore the suggested idea of using hand gestures instead. However, I faced difficulties in figuring out how to make the hand gesture repeatable. Sudden, I recalled that Mr. Razif had shared a tutorial on "cycling through effects/filters on screen tap." I followed the tutorial and made adjustments by replacing the "screen tap" with "gesture detection" to achieve the desired control using hand gestures.
Fig 1.32 Visual Scripting on Hand Detection (week 10/ 10.06.2023)
Feeling I can add on more texture for a more interactive filter, I add 2 more texture for the filter and the outcome was captured :
Fig 1.33 Filter Testing (week 10/ 10.06.2023)
However, the issue where the wing does not properly align with the user's body position are still occurred [if full body], which affects the overall interactivity of the filter. To solve this issue, I continued exploring different methods and techniques to ensure that the wing aligns correctly with the user's body. However, Mr. Razif had helped on explore this and figured about the 3D features, body avatar drive but unfortunately, the current outcome was only can affected the ideation of body movement but not the body positioning. So, I continued back to my head tracking and decide to did some minor adjustments towards the position of the wings.
Fig 1.34 Amendment, Filter Testing (week 11/ 12.06.2023)
And, I shared my preview QR code to some friends and invited them to be my "white mouse" for this current outcome.
Fig 1.35 "User Usability Testing" (week 11/ 12.06.2023)
On the week 11 [18 Jun 2023], we had informed to submit our latest outcome to TikTok. [since mine was still in exploration, I was instructed to submit my latest outcome first instead and can amend if we found the alternate ways to solve the issue.] Since we need a Thumbnail before we submitted the filter, I created one using Adobe Illustrator.
Fig 1.36 Screenshot, Development of Thumbnail (week 11/ 18.06.2023)
Since there are series of Thumbnail suggestion given from TikTok Effect House, we were advised to incorporate a gradient background and a drop shadow to improve the contrast of the object. Utilizing the 3D model I had previously built, I repurposed it to create a suitable background. Although this particular type of gradient is not typically my preference, I recognised the importance of enhancing the overall user experience and decided to implement it accordingly.
Fig 1.37 Final Thumbnail (week 11/ 18.06.2023)
After having the final thumbnail, I filled the "filter details" and jump into the waiting stage as it required 1-3 working day to review and approve my submission.
Fig 1.38 Submission Site (week 11/ 18.06.2023)
During the filter approval process, Mr. Razif shared a solution with me regarding to my issue. He informed me that we could download the "Body Rig Template" from the TikTok Effect House website, which allows the filter to follow the body movements. Once we have the Body Rig FBX file, all we need to do is attach the wing to the body rig components. In my case, the most suitable position for the wing attachment was the "Spine 2" as it encompasses the neck and shoulder of the body rig. So, I followed the tutorial video given by him and discovered the "success" of the filter.
In the meanwhile, Mr. Razif had given a feedback regarding to the current outcome :
Feedback _ Week 12 / 19.06.2023
If possible can put some effects into it as it seems to be a bit empty, but not too much.
After receiving feedback from Mr. Razif, I began exploring different effects that could be used in my filter. However, due to file size limitations, I was unable to incorporate external GIFs or 3D models to create the desired effects. Instead, I decided to utilize the "Bling effect" available in TikTok Effect House as my primary tool for creating effects in my filter.
Fig 1.39 Filter Testing, Outcome (week 12/ 19.06.2023)
After, I proceeded with the development of my filter and attempted to resubmit it. However, even though my file size did not exceed 5.0 MB, the TikTok Effect House site still prompted me to resubmit the filter due to size limitations. Throughout this process, I had to repeatedly review, refine, and resubmit the filter, but ultimately, I could only submit it without the intended filter effects. This was because if I included the filter effects, I would have to remove the body rig template, which was not feasible. So, my final outcome was the updated version of AuraWing AR without the filter effect.
Fig 1.40 Proof of TikTok Filter Submission (week 12/ 22.06.2023)
Final Outcome :
Fig 1.41 Filter Outcome, Demo (week 12/ 22.06.2023)
Moreover, I had also share my filter to some of my friends and let them to try out the effects. Below are the compilation of some video clips that I collected :
Fig 1.42 User Testing Compilation (week 12/ 22.06.2023)
FEEDBACK
Week 2 / 11.04.2023 @ Ideation Draft #1
Good contents. But you can develop things beyond the module information instructions. For this semester, we are not required to build an online store, so it could be more freedom for you to create anything that is beyond the brief. For the contact lens ideation, it seems not suitable to create a filter as the outcome will be very small and focus on the parts of the eyes. For a filter, it will be difficult to see what the style and the affordances are that you have created.
Week 6 / 09.05.2023 @ Ideation Draft #2
For Idea 1: Masquerade Mask, you can consider incorporating a cultural motif such as the Sarawak culture motif, which can add a unique and intriguing element to the filter. For Idea 3: AR Butterfly Wing Filter, you can consider to create integrating frames with the iconic identity of Tinker Bell in the foreground as well as some magical effects that immerse users in a whimsical world and bring a touch of enchantment to the filter.
Week 7 / 13.05.2023 @ Progression Check
You need to make a decision regarding the 2D or 3D model, or any necessary plan changes, by the end of this week due to time limitations.
Week 9 / 30.05.2023 @ Progression Check
It seems like a shell but overall great, continue your development.
Week 10 / 08.06.2023 @ Development #6 Outcome
This wings look nice. The most 3D animation need to be done in 3D software like blender and not sure whether we can direct animate in TikTok Effect House or not. And, you might add on buttons and more options of wing for user to choose in order to hit the outcome of "interactive design", maybe the hand gestures.
Week 12 / 19.06.2023 @ Outcome
If possible can put some effects into it as it seems to be a bit empty, but not too much.
REFLECTION
The experience of creating a TikTok filter was both exciting and challenging. Initially, I was drawn to the potential of creating TikTok effect and looked forward to the opportunity to enrol the module. However, the journey proved to be more stressful than anticipated, starting with the delayed workshop and continuing with limitations in terms of GB capabilities for creating the final outcome. These limitations hindered my ability to fully explore and showcase the desired filter effect.
As the limitations, I believe that if there are more capabilities and resources, the outcome could have been more impressive. Although I initially had high expectations and excitement for the outcome, the current status of the filter may not be as engaging or satisfying as my previous portfolio. However, I am proud to say that I created the filter entirely from scratch, without relying on downloaded assets from free 3D websites. This process allowed me to gain valuable experience and indirectly learn 3D software tools such as Nomad and Blender. Thus, I recognise that this experience has provided me with a deeper understanding of the challenges and possibilities of 3D engagement.
Overall, despite the limitations and less than satisfactory outcome, I can confidently say that I put in my best effort and tried my hardest to produce a meaningful result. This experience has set as a fundamental for me to further exploring in the world of effects as well as 3D.
Comments
Post a Comment