SIGGRAPH 2022

Technology 01 Enhancing games with cutting-edge AI to unlock new possibilities for game developers and players.

We are evolving Game-AI beyond rule-based systems by using deep reinforcement learning to train robust and challenging AI agents in gaming ecosystems. This technology enables game developers to design and deliver richer experiences for players. The recent demonstration of Gran Turismo Sophy™, a trained AI that beat world champions in the PlayStation™ game Gran Turismo™ SPORT, embodies the excitement and possibilities that emerge when modern AI is deployed in a rich gaming environment. As AI technology continues to evolve and mature, we believe it will help spark the imagination and creativity of game designers and players alike.

Can an AI outrace the best human Gran Turismo drivers in the world? Meet Gran Turismo Sophy and find out how the teams at Sony AI, Polyphony Digital Inc., and Sony Interactive Entertainment worked together to create this breakthrough technology. Gran Turismo Sophy is a groundbreaking achievement for AI, but there's more: it demonstrates the power of AI to deliver new gaming and entertainment experiences.

• Sony AI
• Unveiling Gran Turismo Sophy™ : An AI Breakthrough
• Gran Turismo Sophy

Technology 02 Intelligent Vision Sensor

Intelligent Vision Sensor is the image sensor which is equipped with AI processing functionality, and enables high-speed edge AI processing and extracting only necessary data, which, when using cloud services, reduces data transmission latency, considers privacy, and reduces power consumption and communication costs.

• Sony Semiconductor Solutions Group

Technology 03 ToF(Time of Flight) AR

AR technology expresses extended reality enhanced with virtual visual information of onscreen landscapes by utilizing depth information of objects and buildings. It is mainly used in entertainment applications such as for smartphones. "ToF AR" developed by Sony Semiconductor Solutions Corporation (SSS) can smoothly depict the movements of hands and fingers by using the proprietary AI processing technology jointly developed with R&D Center of Sony Group Corporation. This makes it possible to develop applications that realize the following functionality on Unity: (1) Hand-gesture function, (2) Function to recognize and smoothly depict body movements, (3) Modeling function to create 3D data,(4) Meshing function to build surrounding environments based on depth information, etc.

• "ToF AR" at Developer World
• Back-illuminated, Time of Flight (ToF) 3D image sensor

Technology 04 Game Simulator for AI Technologies in Game QA Automation

In recent years, the scope of video games has become broader and more complex. The cost burden for game QA, as well as development costs, have increased. In light of these trends, there is a need for technologies to automate game QA. Recent advances in AI technology have led to active research and development to promote automation of game QA using AI technologies.

In order to automate game QA, it is necessary to automate gameplay and detect game anomalies, both of which pose very challenging problems. To accelerate research on game QA automation, our team has developed a game application called "Game Simulator" that has all the features necessary for game QA automation research. Beyond game QA automation, we are also using the Game Simulator to research development of game AI technologies for "AI Player," for autonomous gameplay, and for "AI Checker," for anomaly detection.

Technology 05 On-the-fly 6DoF object pose estimation

For applications involving interaction with the real world , such as AR, 3 dimensional(6DoF) pose estimation is a key technology. However, modern ML methods require a large amount of labeling effort, which prevents users to improvise and register their favorites. Our technology solves this issue while achieving a high level of accuracy.

In the right part of the clip above, a 3D model of the object is rendered based on the pose estimation results.

Pick up any object you like and turn it around in front of the camera. This is all it takes to register it.

• Attribute(PDF)

Technology 06 Super-Resolution Technology

We are developing a wide range of image quality enhancing technologies, including super-resolution, noise reduction, gradation and color conversion, and motion blur removal. In the fields of image processing, such as imaging and display as well as compression and transmission, we are pursuing the highest level of performance while developing cutting edge image quality enhancing technology that utilizes machine learning (AI). Our aim is to apply it to movie production and streaming, as well as the visualization of 3D data.

• Super-Resolution Technology That Combines Production Efficiency and High Image Quality

Technology 07 Spatial Reality Display ELF-SR1

Sony's Spatial Reality Display leverages industry-leading high-speed vision sensor technology to enable a glasses-free, high-resolution 3D modelling, and visualization experience*. Using the Unity or Unreal Engine SDK, creators can develop VR, AR, and other types of content and bring this content to life using the display**.

*Computer required with a recommended CPU of Intel Core i7-9700K @3.60GHz or faster; and a graphics card such as NVIDIA GeForce RTX 2070 SUPER or faster.
**Recommend use of "high resolution, quality images" created using Unity or Unreal software.

• ELF-SR1 Spatial Reality Display