Why Businesses are Focusing on On-Device AI and its Future Prospects

This article was translated from Korean into English using AssistAce.

What is On-Device AI?

As artificial intelligence (AI) technology advances at a rapid pace, it is bringing many changes to our daily lives. Among these changes, 'On-Device AI', a key innovation to be significantly spotlighted in 2024, performs AI computations directly within the user's device, providing benefits such as personal information protection, reduced latency, and decreased dependency on internet connections. Compared to cloud-based AI, On-Device AI has the advantage of processing data in real-time within the device. Thanks to these strengths, this technology is expected to be widely used in various fields such as smartphones, wearable devices, and smart home devices, enriching user experience and increasing the autonomy of the devices.

In particular, the development of On-Device AI holds great significance in today's digital environment where personal information protection and data security are critical. As user data is processed only within the device, it has considerable advantages in terms of security and privacy. Also, it can function even without an internet connection, enhancing accessibility and convenience. Thanks to these features, On-Device AI is expected to be more widely applied in various industrial fields in the future.

In this article, we will explore the pros and cons of On-Device AI and understand why many companies are focusing on this technology. This will help us understand the changes and impacts that On-Device AI will bring to our lives and society in the future.

On-Device AI vs Cloud Server AI

On-Device AI enhances data privacy and security and does not require an internet connection. It allows for lower latency and energy-efficient operation, provides a personalized experience, and reduces operating costs. However, it has limited processing capabilities and storage space, and the complexity of model updates, high-level optimization, and development and maintenance can act as drawbacks.

Cloud Server AI, on the other hand, offers the advantage of strong processing capabilities, allowing for large-scale data storage and management, and centralized updates and maintenance. It facilitates integration with various services and enables advanced features and complex computations. However, the necessity of an internet connection, potential for high latency, energy-inefficient usage, and increased operating costs are major drawbacks.

Adoption Status of Big Tech Companies

As of 2024, big tech companies are focusing on the adoption and utilization of On-Device AI. They are exploring various ways to enhance the features of their products and services using On-Device AI, starting from the widely distributed smartphone market worldwide.

Samsung Electronics was the first company to start. Recently, Samsung announced the beginning of a new smartphone era equipped with On-Device AI technology by unveiling the Galaxy S24 series at the Galaxy S24 Unpacked event. The event showcased several AI-specialized features, including real-time phone interpreting, emphasizing the enhancement of security by processing users' personal information within the device. The Galaxy S24 series also offers various AI features, including the 'Note Assist' function to increase the productivity of the 'Samsung Note' app and the Speech-To-Text (STT) function that converts voice recording into text.

This announcement reflects Samsung's strategy to enhance AI technology and increase competitiveness in the premium smartphone market through collaboration with global big tech companies. According to market forecasts, the premium smartphone market is expected to be reorganized around On-Device AI, and the Galaxy S24 series is likely to be one of the leading competitive products in this field.

The collaborations with other companies are also dazzling. Samsung collaborated with Qualcomm and Google for the Galaxy S24 series. They developed the 'Snapdragon 8 3rd Generation for Galaxy' chipset with Qualcomm, focusing on more efficient processing of AI computations. This chipset is installed in the Galaxy S24 Ultra model, and the Plus model and standard model are equipped with 'Exynos 2400' developed by Samsung itself, enhancing AI computation performance.

Through collaboration with Google, they aimed to enhance the AI experience. Gemini AI has been applied to Samsung Note, Voice Recorder, Keyboard app, and Message app. Based on Gemini AI, summarizing note contents and textifying voice recording can be done. The image creation AI model 'Imagin2' was also used on the Galaxy. Using the 'Circle to Search' feature, users can obtain the information they want directly from the screen without having to move to the search app. Also, through Galaxy AI, users can use features such as summarizing text messages while driving and notifying the estimated arrival time. According to Google, Gemini AI has signed a long-term contract as a generative AI model supplier for Samsung phones.

"Now is the era of AI phones" Samsung Electronics aims at the premium market with Galaxy S24 https://news.tf.co.kr/read/economy/2069362.htm

In fact, Google announced the AI model 'Gemini AI', which corresponds to OpenAI's GPT-4, on December 6th last year. Gemini AI was released in three versions optimized for various sizes: Ultra, Pro, and Nano, and the lightest version, 'Gemini AI Nano', was designed to be used in On-Device AI. Unlike the super-large model, Gemini AI Nano 1 uses 1.8 billion parameters, and Nano 2 is a lightweight model using 3.25 billion parameters. Google plans to apply these lightweight models to mid-low-priced smartphones in the future and allow developers to use Gemini AI Nano through the AI core system installed in Android 14. Although 'Gemini AI Nano' was installed in the latest smartphone 'Pixel 8 Pro', there is a high possibility of expanding the distribution of Gemini AI through Samsung Electronics' Galaxy24.

Google's 'GeminAI Nano' installed on AI phone 'Galaxy S24' series https://www.techm.kr/news/articleView.html?idxno=119118

Introducing Gemini: our largest and most capable AI model https://blog.google/technology/ai/google-gemini-ai/

Apple is also not sparing investment in AI-related technologies. In October 2022, Apple introduced the AI chip 'M3'. This chip is offered in three versions: M3, M3 Pro, and M3 Max, with M3 Max supporting up to 128GB of integrated memory, enabling AI developers to work on large transformer models with billions of parameters on their laptops. Based on the M3 chip series, the Neural Engine is planned to accelerate machine learning models more effectively. Apple is also developing its own generative AI model called 'Apple GPT'. This model has 200 billion parameters and shows great advancements in language understanding and generation. As part of a broader strategy to deeply integrate AI into the entire ecosystem, this model is expected to be used to enhance AI features in various Apple devices, including Siri. Apple plans to start AI functions on iPhone and iPad at the end of 2024, which is expected to coincide with the release of iOS 18. These movements of Apple can be seen as focusing on On-Device AI and the development of their own large-scale language model.

Apple unveils M3, M3 Pro, and M3 Max, the most advanced chips for a personal computer https://www.apple.com/newsroom/2023/10/apple-unveils-m3-m3-pro-and-m3-max-the-most-advanced-chips-for-a-personal-computer/

Apple GPT: What We Know About Apple's Work on Generative AI https://www.macrumors.com/guide/apple-gpt/

Apple’s iPhone design chief headed to LoveFrom to work with Jony Ive on AI devices https://9to5mac.com/2023/12/26/iphone-design-chief-lovefrom-ai-devices/

OpenAI's CEO Sam Altman is recruiting former Apple's design chief, Tang Tan, to participate in a new artificial intelligence hardware project. Tang Tan, formerly vice president of Apple's product design division, is expected to lead hardware engineering at LoveFrom. LoveFrom is a design company founded by former Apple design chief Jony Ive. The joining of former Apple designers to LoveFrom can be seen as a significant move symbolizing the fusion of Apple's design and AI technology. This collaboration aims to combine OpenAI's technology and LoveFrom's design capabilities to develop a new form of AI hardware, particularly focusing on providing a user-friendly experience.

OpenAI calls in Apple’s iPhone design boss as CEO Sam Altman eyes launch of AI devices https://fortune.com/2023/12/27/openai-apple-iphone-tan-altman-artificial-intelligence-device/

Why On-Device AI?

There are various reasons why companies are focusing on the development of on-device AI. One of the main reasons is cost savings. The ease of use of AI tools contributes to cost savings and automation of key processes. According to IBM's "Global AI Adoption Index 2023" report, many companies cite cost savings and process automation as the main motivations for AI adoption. On-device AI allows data processing and analysis to take place within the device, reducing dependence on cloud servers or data centers and significantly reducing data transmission and storage costs. Cloud-based AI systems, especially large-scale AI models like GPT, have high operational and maintenance costs. The operating cost of OpenAI's ChatGPT is estimated to be about 1 billion won per day. Such models require complex computations and consume a lot of funds. The operating cost of ChatGPT is estimated to be 100 to 200 times higher than that of a regular search. There are also cloud storage costs. Cloud service providers like AWS charge data transfer fees. On-device AI is emerging as an important alternative for cost savings. By performing data processing within the device, it reduces data transmission costs and overall operation and maintenance costs.

The enhancement of data privacy and security through on-device AI is also important. Data related to the user's personal information is processed and stored within the device, minimizing external exposure. For example, if voice recognition data is processed directly on smartphones or smart speakers, it is not transmitted to the cloud or external servers. This reduces the risk of data leakage and strengthens personal information protection. On-device AI provides security features such as data encryption to further protect user data. For example, fingerprint recognition data on smartphones is encrypted and stored within the device and is not exposed during external communication. This is effective in protecting the user's sensitive data. Therefore, one of the main reasons companies consider on-device AI important is that it significantly enhances data privacy and security by processing and storing data locally, eliminating the need for data transmission through external servers or the cloud. This is especially important at a time when concerns about the protection of personal information are increasing.

Providing a personalized experience is also a major goal of on-device AI. This technology provides a highly personalized experience by processing and analyzing user data within the device. The device learns the user's behavior patterns, preferences, and specific requirements to tailor services or features. For example, on-device AI on smartphones learns the user's usage patterns and can provide app recommendations, battery saving, and customized user interface settings. Smart home devices recognize user's lifestyle habits and automatically adjust lighting, temperature, etc. In medical devices, it analyzes patient health data and provides real-time feedback, and in financial transactions, it automates tasks such as customer credit inquiries, account opening, and loan applications. These personalized experiences make the user's life more convenient and efficient.

In this way, companies are focusing on the development of on-device AI for reasons such as cost savings, data security, and providing personalized experiences. This can play an important role in meeting customer needs and strengthening competitiveness.

IBM's "Global AI Adoption Index 2023" report https://newsroom.ibm.com/2024-01-10-Data-Suggests-Growth-in-Enterprise-Adoption-of-AI-is-Due-to-Widespread-Deployment-by-Early-Adopters

"ChatGPT cost, 200 times that of normal search... cost reduction is the issue” https://www.hankyung.com/article/2023021215731

ChatGPT, astronomical operating costs... DB center GPU price alone is 100 million dollars https://www.g-enews.com/article/Global-Biz/2023/03/2023031413410113239a1f309431_1

What is the future of On-Device AI?

The future of on-device AI seems bright with the continuous development of technology. This technology's application will expand in various industries such as smartphones, PCs, home appliances, autonomous vehicles, robots, security, finance, and healthcare, and it will play an important role in providing a personalized user experience by more accurately understanding user preferences and behavior patterns. The development of sophisticated AI models and lightweight algorithms will enable devices to operate quickly and efficiently, and provide personalized services to users.

In terms of security and privacy, on-device AI will also make significant progress. Since user data is processed and stored only within the device, the risk of data leakage decreases, and personal information protection is strengthened. This is especially important in services related to finance, healthcare, and personal information.

However, there are several challenges to this technological progress. First, current hardware limitations can limit the complexity and functionality of AI models. More powerful and efficient hardware is needed to support more advanced AI algorithms. Second, there are regulatory challenges related to personal information protection. It is linked with legal and ethical considerations to guarantee the security and privacy of user data while promoting technological innovation. Third, the cost of development and maintenance can be a big problem. Especially in small businesses or emerging markets, access to such advanced technology may be limited.

In the long run, on-device AI will enable more intelligent and personalized interactions in various aspects of daily life, which could innovatively transform the transportation, logistics, and service industries. On-device AI will play a key role in enhancing user experience, intelligent operation of devices, and creating a safer and more efficient digital environment. Companies will develop new business models and services in line with these changes, which are expected to bring about changes in the overall industrial structure. In addition, the technology will be utilized to focus on developing personalized services and functions based on user data, thereby increasing customer satisfaction.

In this way, while on-device AI is walking an interesting and valuable path of innovation, a balanced approach to the challenges that emerge with technological advancement is needed. As AI technology continues to advance, on-device AI will play an increasingly important role in leading innovative changes across industries. Through this, we will experience a more convenient and safe everyday life.