Reintroduced AI video modeling technology Every flat and every house dares to “compete” with Google

At the press conference for the launch of the “Digital+” product release and the “Looking for Good Design” design challenge (North China region) between Yuanda Decoration and Everywhere Designers, Shi Fan, the head of Alibaba Everywhere Designers platform business and CEO of Everywhere Designers, officially released the AI video modeling technology and its application product Object Drawer, which was developed by Everywhere Designers in cooperation with Amoy. Object Drawer is the industry’s first 3D modeling product based on neural rendering, with an inference speed 10,000 times higher than that of nerf algorithm.

Based on this technology, consumers do not need to rely on specific equipment, only cell phones around the target product video shooting, you can complete the automatic reconstruction, the resulting three-dimensional modeling effect to achieve a high degree of precision model restoration, but also to be able to put the target product into their own home in the real house to display.

And where does this home black technology come from? How powerful is the tool behind the support? Let’s find out.

VR / AR track rise home home furnishing 3D modeling demand soaring

Meta-universe originated in 1992 Neal Stephenson’s “Snow Crash”, the book describes a virtual world parallel to the real world, Metaverse, with the development of VR/AR applications, in the real world home furnishing home field, is a lot of use of VR, 3D, virtual reality and other technologies, cloud-based 3D shooting material library, scene library has become the digital infrastructure of the business Important link, operators hope to be relieved of the trouble of shooting planning, 3D creation, cloud material can be at hand, at any time to shoot. On the other hand, the demand for 3D modeling of home furnishing products in the market continues to increase, and the demand for home furnishing products modeling may reach more than 100 times of the current demand.

At present, 3D modeling of goods on the market mainly relies on manual modeling, modeling time takes several hours or even days, the cost is often hundreds to thousands of dollars, not only the modeling environment, filming equipment has put forward many requirements, in order to enhance the target commodity shape, texture, material display details, but also requires a lot of manual work for high-precision restoration and detail repair.

Key core technology breakthroughs, 10,000 times faster inference

In recent years, neural rendering technology is developing rapidly, Google’s initial NeRF and its derivatives use neural implicit expression to store the physical geometry of the scene and material texture information, end-to-end optimization of the rendering effect, but in practice there are still many problems, including: first, the inference speed, training speed is slow, 1 frame of high-definition map inference time of more than 50 seconds, an object modeling time up to 2 days or more. Second, the detailed texture can not be restored. Third, the rendering effect of some views is not ideal. Fourth, can not be directly imported into the graphics tools, does not support the explicit use, such as CAD scene with the design; in addition, because only restore the shooting scene lighting also makes NeRF model can not support the application of environmental lighting changes in the scene.

The AI video modeling technology and its bearer product, Object Drawer, are jointly developed with Amoy Technology, which optimizes the network structure, explores the geometric a priori, and pre-stores some information, so that the reasoning speed can reach 200 frames per second, and 30 frames per second on cell phones, while ensuring the rendering quality, thus realizing a real-time HD interactive 3D model, and compared with NeRF algorithm The inference speed is 10,000 times faster than that of NeRF. The training time of the model is compressed from 2 days to only 4 hours, and the size of the model is only 20M, at the same time, for any perspective, the 3D model of the goods are up to the display effect of the real photo, to achieve the automatic high-precision restoration of 3D modeling has been pursued. The automatic reconstruction of artificial intelligence, the generated 3D modeling effect to achieve high precision model restoration. At present, this technology has been open for hand Tao, Tmall part of the home furnishing business.

In order to reflect the light effect of physical rendering on 3D coarse model, such as reflection, shadow, etc., Object Drawer migrates the realistic light effect of physical rendering on the object to the object perspective map generated by neural rendering, and then obtains the neural rendering result with light effect by fusion, and further replaces the coarse model projection in the physical rendering result with the HD composite map with light effect. The resulting high-definition composite image with lighting effects is further used to replace the coarse-mode projection in the physical rendering result to achieve realistic scene rendering. The results of the experimental data show that Object Drawer is able to adapt to various complex light conditions and the migration of detailed shadow effects, and the visual effect is much better than other alternatives.

Object Drawer optimizes the model representation, significantly accelerating the model training time while achieving, for the first time, high-definition fine texture restoration, such as fabric lines and other details are clearly visible.

Every flat and every house and the Tao system technology together to launch the AI video modeling technology based application product Object Drawer on the neural radiation field technology has been improved and upgraded comprehensively, to achieve the automated high-precision modeling process of home furnishing goods, the future, every flat and every house as the home furnishing track of Alibaba’s innovative business, will continue to increase investment in scientific research technology, from “commodity model – house type – design” and other steps, from the “model of goods – house type – design” and other steps, to the “model of the home furnishing track”. Design” and other steps, to help businesses, designers and consumers to reduce the threshold of 3D scene creation, to inject technology into each ring of the whole chain of home decoration and living, to optimize everyone’s experience of living space, physical and aesthetic with technology, and to build a new living model of digital intelligence.