[ad_1]
OpenAI is new – and a first! – Video-generating model, Sora, can pull off some truly impressive cinematographic feats. but the model is even More At least from a technical point of view, OpenAI is capable of paper Published this evening.
The paper, titled “Video generation models as world simulators”, co-authored by several OpenAI researchers, lifts the curtain on key aspects of Sora’s architecture – for example, revealing that Sora can support an arbitrary resolution and aspect ratio. Can generate 1080p video (up to 1080p). According to the paper, Sora is capable of performing a variety of image and video editing tasks, from creating looping videos to extending a video forward or backward in time to changing the background in an existing video.
But most interesting to this author is Sora’s ability to “simulate the digital world,” as the OpenAI co-authors put it. In one experiment, OpenAI set Sora loose on Minecraft and let it render the world – and its dynamics, including physics – with the player controlling it.
Sora is controlling a player in Minecraft – and rendering the video game world just as he does. Note that graininess was introduced by the Video-to-GIF Converter tool, not by Sora. Image Credit: OpenAI
So how is Sora able to do this? as well as saw By Jim Fan, Senior Nvidia Researcher (through quartz), Sora is more of a “data-driven physics engine” than a creative one. This is not simply producing a photo or video, but determining the physics of each object in the environment – and rendering a photo or video (or interactive 3D world, as the case may be) based on these calculations.
“These capabilities suggest that continued scaling of video models is a promising path toward the development of highly capable simulators of the physical and digital worlds and the objects, animals, and people living within them,” the co-authors write.
Now, Sora’s general limitations apply to the video game domain. The model cannot accurately predict the physics of fundamental interactions such as glass breakage. And even with conversation can do The model, Sora, is often inconsistent – for example showing a person eating a burger but failing to show bite marks.
Still, if I’m reading the paper correctly, it seems like Sora could pave the way for more realistic – perhaps photorealistic – procedurally generated games. This is equal parts exciting and frightening (consider the deepfake implications, for one) – which is probably why OpenAI has chosen to leave Sora behind. Very Limited access program for now.
Here’s hoping we learn more sooner rather than later.
[ad_2]
Thanks For Reading