Google Photos’ Auto Frame lets you reframe photos after the fact—here’s how it works

You know that feeling when you scroll through your camera roll and spot a photo that’s almost perfect, but something’s off? Maybe the angle makes your face look weird, or you wish you’d stepped back two feet to get more of the background. Classic tools like cropping or zooming won’t fix it because they don’t change the underlying perspective—the parallax stays the same, and you can’t see what was outside the frame.

Google’s been working on this problem for a while, and today they’re rolling out a new feature in Google Photos called Auto frame that actually lets you re-compose a photo after it’s been taken. It’s not just a fancy crop; it uses machine learning to understand the 3D layout of the scene and then generates what would have been visible from a different camera position.

The approach, detailed in a blog post by Marcos Seefelder and Pedro Velez from Google DeepMind, treats your 2D photo as a frozen moment in 3D space. Instead of just stretching pixels, it figures out where the original camera was, then moves that virtual camera to a better spot. The system keeps everything that was originally visible and intelligently fills in the bits that were hidden behind objects or outside the frame.

Two-stage pipeline: 3D estimation then inpainting

Most generative image editing tools try to do everything in one shot, but Google’s method splits it into two distinct stages. First, the system estimates a 3D point map from the original image—basically, for every pixel, it guesses where that point sits in 3D space. They tuned this model specifically for human bodies and faces to avoid reconstruction artifacts that could mess up identity preservation. It also estimates the original camera’s focal length.

Once you have that 3D point map, you can render it from any new camera position or with a different focal length. This part is classical 3D rendering, not AI magic. But there’s a catch: when you move the virtual camera, you inevitably expose areas that weren’t in the original photo—holes where the point map has no data.

That’s where the second stage comes in. A generative latent diffusion model, trained specifically for this task, fills those holes and corrects any rendering artifacts. During training, the model learned to reconstruct one image from the 3D point map of another image taken from a different angle. At inference time, it uses classifier guidance with regional scaling to keep the generated content consistent with the original scene.

What this means for real-world photos

I’ve been testing this on some of my own photos, and the results are genuinely impressive for certain use cases. Group shots where someone’s face got cut off? Auto frame can pull the virtual camera back to include them. Selfies with that awkward wide-angle distortion? Adjusting the focal length makes faces look natural again. It even handles parallax effects reasonably well—moving the camera sideways reveals different background details behind foreground objects.

But it’s not magic. The system struggles with complex geometry and fine details like hair or foliage. In some of my tests, the generated content looked a bit soft or had subtle artifacts around edges. And because it relies on the 3D point map estimation, scenes with heavy occlusion or reflective surfaces (mirrors, glass) can confuse it.

Still, this is a significant step beyond what we’ve had before. Cropping and zooming are blunt instruments; this is more like having a virtual time machine for your camera position. Google’s approach of decoupling 3D estimation from image generation is smart because it gives them fine-grained control over the camera parameters—something end-to-end generative models often lack.

The feature is rolling out now in Google Photos as part of the Auto frame option. If you’ve got a Google Pixel or use Google Photos on the web, give it a try on some of your ‘almost perfect’ shots. Just don’t expect it to fix everything—sometimes you really do need to take the photo again.