[Case Study] How TecAce Used Deep Learning to Create an AR Earbud Simulator

How TecAce Used Deep Learning to Create an AR Earbud Simulator

In this blog post, we will share how we, TecAce, a company specializing in providing professional store display solutions, used deep learning to create an AR earbud simulator that allows customers to try on different designs and colors of earbuds using their mobile phone cameras.

This project was a challenge for us, as we had to develop a machine learning algorithm that can detect ears and ear canals accurately from any angle and augment earbuds on them realistically.

The Problem: How to Display Earbuds in Stores?

Earbuds are becoming more popular as wearable fashion products, as they offer convenience, comfort, and style. However, unlike larger devices like smartphones or TVs, it is difficult to display earbuds in stores due to their size and wearing characteristics. Customers cannot easily experience how earbuds look and feel on their ears without wearing them.

However, wearing earbuds that have been used by many other customers is not hygienic or appealing. Therefore, we wanted to provide a unique customer experience for earbuds in-store that would allow customers to see how different designs and colors of earbuds would look on their ears without physically wearing them.

The Solution: An AR Earbud Simulator Using Deep Learning

To create an AR earbud simulator, we needed to solve two main problems: how to detect ears and ear canals from any angle, and how to augment earbuds on them realistically. To solve the first problem, we developed a machine learning algorithm that recognizes ears using deep learning.

We used a convolutional neural network (CNN) model that can learn features from images and classify them into different categories. We trained the model with various photo data of ears from different angles, lighting conditions, and backgrounds. We also implemented a face detection algorithm that can locate the face region and crop it out from the image. This way, they could reduce the noise and focus on the ear region.

However, detecting ears was not enough. We also needed to detect the ear canal, which is the part where the earbud is inserted. This was a more challenging task, as the ear canal is not always visible or clear in images.

To solve this problem, we developed another algorithm that can estimate the position and orientation of the ear canal based on the shape and contour of the ear. We used geometric calculations and image processing techniques to find the center and direction of the ear canal. We also used a feedback mechanism that can adjust the position and orientation of the ear canal based on the user's input.

To solve the second problem, we obtained 3D models of different designs and colors of earbuds from their clients. We used computer graphics techniques to render these models on the images of ears. We also used image-warping techniques to adjust the size and perspective of the earbuds according to the distance and angle of the camera. We added shadows and reflections to make the earbuds look more realistic and natural.

The Result: A New Customer Experience for Earbuds

We successfully completed this project, providing a new customer experience for earbuds in-store. We developed a solution that enables users to position earbuds on their ears, change the design and color, and experience them using a mobile phone camera. The solution works smoothly and accurately, as it can detect ears and ear canals from any angle and augment earbuds on them realistically. The solution also works fast, as it can process images in real time without noticeable delay.

We demonstrated this solution at various events and exhibitions, receiving positive feedback from customers and clients. We also deployed their solution in several stores across Korea, attracting more customers and increasing sales. We plan to expand our solution to other regions and markets in the future.

TecAce is a company specializing in providing professional store display solutions using cutting-edge technologies such as AR, VR, AI, IoT, etc. We have been working with various clients from different industries such as Samsung Electronics, LG Electronics, SK Telecom, etc., delivering high-quality products and services. If you are interested in learning more about us and our projects, please visit our website at https://www.tecace.com/