Google Introduces Auto Frame: Neural Network Changes Photo Perspective Retroactively

Google Research and Google DeepMind have introduced Auto Frame, a feature now live in Google Photos that edits photos by changing the camera perspective after capture. Unlike traditional cropping, it interprets a photo as a 3D scene, estimates a 3D point map per pixel along with original focal length, then uses classical 3D rendering to simulate a new camera position and intrinsics. The rendered image inevitably has holes where previously unseen content would appear; these are filled by a specially trained latent diffusion model that learns to reconstruct the target view from a re-rendered source view. For portraits, ML models detect faces and their 3D orientation to automatically suggest optimal framing and correct wide-angle distortion by adjusting virtual camera intrinsics. The fully automatic solution is available as the second rendition option within Auto frame candidates in Google Photos.