For Nio owners, the NOMI robot located in the center front of their vehicle is undoubtedly one of their favorite decorations. The intelligent virtual assistant has also brought a number of awards to the Chinese electric car company.

So, what exactly is NOMI?

In 2017, Nio launched the ES8, its first flagship SUV, and with the launch of the vehicle, its in-car voice assistant, NOMI, became the world's first mass-produced in-car artificial intelligence system.

NOMI is derived from Know Me, representing Nio's pursuit of intelligent interaction experience in the car. If you have seen the science fiction movie "Her", NOMI is Nio's version of "Her".

After the delivery of Nio ES8 and ES6, NOMI has won the hearts of a large number of car owners and Nio fans with its intelligent and humanized interaction experience.

Many people think that NOMI is the voice assistant in the car, but in fact, Nio expects more than that from NOMI.

Many car companies' smart cabin design is centered on the screen, for example, Byton wants to create an interactive experience like sitting in a living room through a large screen. Nio has left the most central position in front of the cockpit to NOMI.

In addition to the NOMI, the Nio ES8 is equipped with a 9.8-inch digital instrument screen, an 11.3-inch center screen, and a 10-inch HUD screen.

Together with the area below the center console that supports wireless charging of cell phones and the NOMI's own screen, these five screens are the foundation of the Nio digital cockpit experience.

Nio believes that where NOMI is located should best reflect the soul and core of a car, while the other screens do not pursue the largest size, but rather the right and good use, as Nio 0001 employee and assistant vice president for product management Li Tianshu said last year, adding that the other screens are to some extent making way for NOMI.

The first thing the Nio team wanted to get NOMI to do at the time was not to complete complex natural language interactions and multiple rounds of conversations, but rather that NOMI could treat the car as if it were its own muscle the moment the owner opened the door, Li said.

When it senses that someone has entered the car, it will turn its face in the direction of someone opening the door and say hello to the user. according to Li, "NOMI complements the car and is the only face and soul of the vehicle to communicate with the user."

As an AI entity, NOMI is able to demonstrate subtle interactions through voice, expressions, and movements that virtual assistants cannot accomplish.

For example, when the user reaches out to touch the screen, NOMI can lower its head a little to see what the user wants to do on top of the screen. When the user takes his hand back, NOMI's head will be raised again.

Nio's development of NOMI began in 2015. As an AI entity that no other car company has ever mass-produced in the past, the development of NOMI encountered multiple challenges.

The first issue was the AI's Valley of Terror effect, which means that humans are attracted to AI when it resembles a human in appearance or movement.

But as the similarity increases, people notice the subtle differences between AI and humans and become fearful.

The second issue is the challenge of vehicle regulations and industrial design. How to perfectly integrate a consumer electronics product like NOMI, which costs a few thousand RMB, with a luxury car that costs 400,000 - 500,000 RMB is a problem that needs to be solved.

At the same time, such an AI entity should meet the requirements of vehicle regulations and not increase the safety risk of users in crash tests.

The third problem is that when NOMI has expression, voice, and action, how to synchronize these actions precisely, which is 1 - 2 steps more difficult than synchronizing voice and TTS (text on screen).

To solve the first problem, Nio developed NOMI from 2015 to 2018 with a long period of time requiring the product not to make sound, but only to communicate with the user with expressions and some simple sounds.

The NOMI team designed a whole set of emotion engines that would allow it to communicate with the user, connecting up the vehicle's underlying gateway, the vehicle's machine, NOMI's emotion library, and the motor drivers of the NOMI screen, thus solving the problem of emotional interaction as well as the synchronization of subtle movements.

NOMI's industrial design finally opted for a minimalist approach, resulting in a product image that is difficult to obsolete.

The development team chose a purely circular shape for the exterior and used minimalist symbols for the emoji.

The screen specification chosen for NOMI is also quite high, a megapixel screen.

One of Nio's principles in developing NOMI was not to display functional prompts.

At that time, Nio was designing a function for fatigue detection and considering that in the past, a small teacup was often displayed in the dashboard position in other cars, it became a question for Nio to weigh whether a teacup icon also needed to be displayed on the screen of NOMI.

In the end, Nio believed that the image of NOMI is similar to a person's face, and its integrity cannot be broken.

The image of NOMI is without a mouth. This is related to the aforementioned Valley of Terror effect of AI.

This is because the addition of the mouth part will cause the problem of matching the voice to the mouth shape during conversation to become more complicated.

After experimenting with NOMI's design without a mouth, Nio found that it was instead a more harmonious and reasonable form of the product.

Nio has put a lot of effort into the selection of this voice and continues to optimize the quality of the voice to this day.

Today's voice technology is much more advanced than when Nio first started developing NOMI 4 or 5 years ago.

In the future, NOMI will make deeper use of multimodal cross-interaction, combining a variety of sensors in the car, using vision, hearing, and touch to perceive and interact with the user, according to Li.

In 2018, Nio released NOMI's second product, NOMI Halo (the first product was the very familiar circular NOMI Mate).

NOMI Halo utilizes a simpler format, using a halo and sound to interact with the user by voice.

It also features a car-grade, round AMOLED screen that will display more information, allowing NOMI to show more of a human side.

As of June 2020, NOMI has spoken to users more than 70 million times, with more than 200,000 conversations per day, and this amount of data is essential to NOMI's growth.

Nio can use NOMI to record user feedback about the vehicle, whether it's a question for improvement or sudden inspiration.

NOMI is a convenient real-time feedback channel for vehicle owners, and Nio has collected tens of thousands of pieces of feedback from users.

For example, the user is using the auto-assisted driving function, but found a place that may not be an as good an experience; or suddenly have an inspiration, think Nio can do so in the future. In all these scenarios, feedback can be given to Nio at any time.

The amount of feedback through NOMI is said to account for more than 40% of all feedback channels from Nio.