Google Introduces Auto Frame: Neural Network Changes Photo Perspective Retroactively

Google Research, in collaboration with Google DeepMind, announced a new image editing method now available in the Auto frame feature in Google Photos. Users can change the angle of a photo after it's taken—for example, rotating the camera to reveal a different part of the face or correcting wide-angle lens distortion. Unlike traditional cropping or zooming, which do not change perspective, this new approach interprets the photo as a three-dimensional scene and automatically adjusts the position of a virtual camera within it. The method consists of two stages: first, a neural network estimates a 3D point map and the original camera parameters (including focal length); then, using classic 3D rendering, it creates an image with new parameters, and a generative diffusion model fills in the areas not visible in the original. For full automation, machine learning models determine the position and orientation of faces, which is especially useful for portraits. The feature is available in Google Photos as part of Auto frame for photos with people—users receive a second version of the frame with an automatically corrected perspective.