Year 2021 is the year of metaverse. Metaverse-related issues are attracting the attentions of many companies. These companies are racking their brains to think of ways to make their products connected with metaverse. Nowadays, the applications of metaverse have brought dramatic changes to the fields of games, education, media industry, etc. The author is also curious about what could happen when the metaverse is combined with the automotive industry. Through patent search and analysis, this article will provide a glimpse of the current answers to this question. In the previous article, the author has introduced hardware technology described in metaverse x future car related patents. The user scenarios of metaverse x future car will be described in this paper.  

User Scenarios

  In addition to the aforementioned hardware, the patents related to the metaverse x future cars emphasize more on various user scenarios, that is, how to integrate metaverse technology into the different use manners of vehicles, in order to increase driving safety and enhance the user experience of the vehicle. Fig. 1 shows the classification of the user scenarios involved in the patents of metaverse x future cars, mainly involving driving assistance and other scenarios such as in-car entertainment.    

Fig. 1 User scenarios of metaverse x future car related patents

  1. Driving Assistant – Display Blind Spot Information   The main purpose of using metaverse technology in vehicles is to enhance the driving safety, such as displaying blind spot information. For example, GM's patent US20170161949, as referred to under Fig. 2, emphasizes the installation of a waveguide HUD 16 on the side window 30 of the vehicle. By using an imaging device to capture the image of the blind spot on the side of the vehicle in real time, the waveguide HUD 16 generates a virtual image of objects in the blind spot to be seen at a virtual plan 38, so that the driver can be aware of the situation of the blind spot from the virtual plan 38.  

Fig.2 Drawing of GM’s patent US20170161949

 

2. Driving Assistant – Display Navigation Information

  Traditional navigation methods only present map information stored in the cloud system and cannot be combined with real-time road images. By using metaverse technology, real-time road images can be combined with navigation information through augmented reality technology, allowing drivers to be aware of driving routes more directly. Taking Hyundai’s patent US11376963 as an example, referred to in Fig. 3, the patent emphasizes the use of augmented reality technology to display navigation information, so that the display area and size of the virtual image can be adjusted according to real-world changes. For example, as shown in Fig. 3, when the vehicle changes lanes, the virtual lane line D-3 and turning information T-3 will also be adjusted to change their display position (change to D-4 and T-4) to avoid overlapping with the real-world lane line.

Fig.3 Drawing of Hyundai’s patent US11376963

 

3. Driving Assistant – Other Vehicle Information Sharing

  Conventionally, drivers can only visually observe and try to predict the behavior of other drivers. If the drivers can understand the behavior of other drivers in advance, they can adopt a safer driving strategy. Using the concept of network virtual environment of metaverse technology, drivers can recognize the behavioral information of other drivers around them while driving. For example, taking Microsoft’s patent US10334199 as an example, referred to in Fig.4, the patent emphasizes the use of augmented reality technology to add crowd-sourced indicators 412 to the real-world image to show the driving behavior of other drivers. The bar chart displayed in crowd-sourced indicators 412 represents the results of other driver's ratings as opposed to the driver of the vehicle, allowing the drivers to understand the behaviors of surrounding drivers, such as whether the driver is a novice driver or a reckless one.

Fig. 4 Drawing of Microsoft’s patent US10334199

 

4. In-Car Entertainment

  As autonomous technology becomes more and more sophisticated, people will not have to focus on driving, but will be able to participate in other activities during transportation. The use of metaverse technology can turn transportation into leisure time or entertainment. Taking LYFT’s patent US10991159 as an example, referred to in Fig. 5, the patent describes a virtual reality riding experience. The user can wear a VR device, and the VR device provides a corresponding image according to real-world inertia of the car while it is moving. For example, Fig. 5 shows a virtual boat ride seen by the user wearing the VR device. According to the inertia force of turning or going downhill when the car is driving, the VR screen will also show the angle of view of the boat turning or going down.

Fig. 5 Drawing of LYFT’s patent US10991159

5. Driving Training

  Traditional driving training requires a training car and a training field, which causes a lot of inconvenience. The use of metaverse technology can simulate the angle of view while driving training and can provide timely driving hints. For example, Toyota's patent US11302217 provides an augmented reality system for vehicle dealers who sell new cars. Using augmented reality technology, vehicle dealers can explain the driving features of new vehicles for the benefit of proposed customers before the production and shipment of new cars, allowing new vehicles to be sold before their delivery. As shown in Figure 6, the dealer can use a tablet device 100 to take a picture of real-world environment 118, and identify the virtual car environment to be presented. The tablet device 100 displays an augmented image 122 of the interior of the corresponding car identified by the real-world environment 118. The view angle of the augmented image 122 can change according to the distance and angle between the real-world environment 118 and the tablet device 100. Therefore, customers can experience the interior and driving features of the new car.

Fig. 6 Drawing of Toyota’s patent US11302217

 

6. Shopping Assistant

  As previously mentioned, as self-driving technology becomes more sophisticated, people will not need to focus on driving. Metaverse technology allows users to do other activities such as entertainment or shopping while driving. Taking PayPal's patent US11074644 for an example, referred to in Fig. 7, the patent emphasizes adding a transparent display device on the windshield. When driving near a store, the transparent display device can display information about the products being sold, including pictures or prices, so that the driver can directly click on a desired product to purchase it, making it easy to obtain the product nearby.

Fig. 7 Drawing of PayPal’s patent US11074644

 

8. MaaS (Mobility as a Service)

  In recent years, the emergence of MaaS (Mobility as a Service) platforms provide convenient ride hailing services and make the information of drivers and passengers more transparent through a scoring mechanism. Taking UBER's patent US10977497 as an example, referred to in Fig. 8, which shows the real-time images of passengers and drivers using their cell phones when calling for driving services, augmented images will show information of both parties. For example, the passengers can use their cell phones to see the name and driving score of the driver in the augmented image, to confirm whether to call the car. Likewise, the driver can use the information displayed in the augmented image to identify the passenger who called the car.

Fig. 8 Drawing of UBER’s patent US10977497

  From the above-described user scenarios, it is obvious that metaverse-related technologies combined with automobiles can be used in a variety of ways, not only to improve driving safety, but also to make the daily life of users more convenient in relation to vehicles.  

Summary

  Compared with traditional cars, the concept of future cars includes technologies such as Communication, Autonomy, Electrification, and Sharing, among which the technologies of Communication, Autonomy, and Sharing are all related to metaverse technologies. In this article, we introduced the changes brought by the application of metaverse to future cars through the perspective of patent analysis. The automotive technology will become more and more diversified with the development of technologies. Companies interested in automotive technology may try to use patent search and analysis to figure out the possible directions of technology roadmap, which can help companies to find opportunities for their research and development.     Author: Jean Chou Taiwan Patent Attorney / Certified Valuation Analyst / Co-Chair of Business IP Management Committee, TWPAA / Vice President of WISPRO Technology Consulting Corporation Email: jeanchou@wispro.com