现在每个人都是建设者。每个 All Access：https://every.to/builder-pack，都会为你提供完整会员资格及 Builder Pack——价值超过 9,000 美元的我们用来制作工具的积分。
登录即可免费阅读并获取你的 AI 评估提示。
Cyrus Duff is the cofounder of Afterimage, a creative studio that works with tech companies.
Josh Lee is the cofounder of Afterimage, a creative studio that works with tech companies.
A professional creative studio puts GPT-6 to the test
At our Think Week in Panama：https://every.to/context-window/give-yourself-a-promotion in January, creative studio Afterimage filmed a series of six short films about Every’s work with AI. We’d seen what cofounders Cyrus Duff and Josh Lee could do with a camera, so we were curious when they began testing Astra on footage from real shoots. Could it handle the visual effects they’d normally make themselves? They put it to work on everything from replacing a dish in a moving pan to adding a building to a Brooklyn street to making a giant parade float appear outside a window in handheld iPhone footage. Some shots worked. Others didn’t. By the end, they were dreaming up effects they’d never have attempted before.— Kate Lee：https://every.to/@kate_1767
We run Afterimage：https://afterimage.id/ , a creative studio that works with tech companies. This year, we’ve done projects with Cursor, Reve, Pierre Computer Co, Turbopuffer, and several others. We previously led video at the Browser Company of New York. Every was our first client：https://www.youtube.com/watch?v=VDressz8fJ0 .
When GPT-6 Astra：https://every.to/vibe-check/gpt-6-astra-vibe-check launched, our timelines filled with impressive demos of 3D environments Astra created in Blender, a tool for making digital models and scenes—including a full-scale render of New York City：https://x.com/mattshumer_/status/2095609734845927525 . Astra could build 3D models in Blender and refine them when they looked wrong. We wanted to see whether those skills carried over to footage we’d shot. Could it make a new object look as though it belonged in a real scene? Could it do visual effects?
Visual effects (VFX) involve separating subjects from backgrounds, following and altering objects across frames, and matching new elements to existing photography. We’d tried that work with earlier models. Opus 4.6：https://every.to/vibe-check/opus-4-6 , for example, could write deterministic Python scripts for basic blemish and object removal but struggled with more complicated changes. Generative tools like Gemini Omni：https://gemini.google/overview/video-generation/ could make impressive changes to footage but didn’t allow for the granular control most of our workflows require.
We hoped that Astra might give us the best of both worlds, producing convincing effects while letting us control and revise specific parts of a shot. If it could reliably do tasks that usually require hiring a VFX specialist, we believed it would be a breakthrough for independent filmmakers and small studios like ours.
The first step in altering live-action footage is identifying and isolating the relevant parts. This requires masking, which separates the areas we want to change from those we need to preserve, and tracking, which follows the camera or a subject across frames, so an effect stays in the right place.
In the following tests, we compared Astra and Sol on these two skills. Our informal experiments used the same footage, prompt, and time allowance for each model. Each could choose tools, inspect its output, and revise its work but received no follow-up creative direction from us.
Here’s a sample shot from an unreleased ad for Dia Browser.
Imagine we didn’t like the background in the shot and wanted to put the actor somewhere else. His fine hair, heavy grain from the 16mm film stock, shadows from a flashbulb, and movements all complicate this seemingly straightforward request. We prompted Sol and Astra to remove the boxer from the background.
Astra wins here. Sol leaves a patch of the old background above the actor’s hair and cuts into his face near the end. Astra keeps a more coherent outline, including through the bulb flashes. It doesn’t get every strand of hair right, but it separates the boxer well enough to give us workable footage.
The models took different approaches. Sol used a foreground separation method called GrabCut：https://en.wikipedia.org/wiki/GrabCut , starting with a rough outline of the boxer’s head and body, then combining information from nearby frames to keep the cutout’s edges from jittering. Astra used Robust Video Matting：https://github.com/PeterL1n/RobustVideoMatting , an open-source model that separates people from backgrounds using information carried across frames. Astra wrote Python code to run it locally, exclude the boxer’s shadow on the wall, and remove traces of the gray background around soft edges.
To test tracking, we used a cooking shot from the launch video we made for Dia Browser.
Imagine we decided the zucchini looked bad and wanted to substitute a different dish. The replacement would need to follow the pan as it moved and stay behind its rim, the actor’s hands, and his sleeves as they crossed the shot. We prompted Astra and Sol to track and mask the pan’s contents.
Both models track the broad movement. Astra follows the foreground boundaries more closely, while Sol leaves some of the original food visible and puts a piece of the replacement food outside the pan. Neither is perfect, but with follow-up prompts Astra produced a usable mask—Sol did not.
Once we isolated the boxer and the zucchini, we used Astra to create more elaborate effects in both shots, combining code, generated imagery, and 3D geometry.
In the shot of the boxer, we used OpenAI’s latest image model：https://openai.com/index/introducing-chatgpt-images-2-5/ to create a boxing gym background in the same photographic style. Astra then wrote code to match the movement, add film grain, and bring the photographed flash shadow into the new room. A punching bag in the background sways slightly, and a soft foreground rope moves past the camera—both made using pure code.
Getting here took a lot of back-and-forth with Astra. In one version, it added flickering lights to give the gym some life, but the glow was excessive and the dimmed wall looked unnatural. We pointed this out and suggested a gently swaying punching bag instead. Astra removed the flicker and animated the bag already in the background image, keeping the rest of the room still.
For the meal shot, we went beyond masking and asked Astra to replace the pan’s contents.
We pointed out inconsistencies using annotated screenshots and clarified which changes to preserve. Earlier attempts made the pan’s bottom and sides seem to move independently. Later versions treated the pan as one 3D object, with the replacement imagery moving with it.
Astra assembled the final composite in Nuke：https://www.foundry.com/products/nuke-family/nuke , professional software for assembling and adjusting the different elements of a shot. We never touched Nuke ourselves. Astra controlled Nuke through its Python scripting API, using Nuke to keep the new food behind the actor’s arms and sleeves when they crossed the pan. We directed the revisions while Astra carried them out in the software.
The best results we’ve gotten out of Astra combine the visual effects techniques described earlier with its remarkable 3D modeling and rendering capabilities.
Because buildings are rigid and geometric, they’re good candidates for 3D modeling.
Using footage from a recent commercial shoot for Reve, we tried creating and replacing buildings on a Brooklyn street.
We started by tracking the small camera movements caused by heavy wind on the 13th floor of the William Vale in Williamsburg. We then modeled and rendered a 3D building, added generated details to its surfaces and windows, and matched its softness and texture to the photographed buildings to settle it into the scene.
In this example, we added some 3D objects to two tables in a minimal studio setup. This shot is from upcoming commercials we made with Angles：https://angles.app/ .
While it’s less dramatic than a building, the wineglass example might best illustrate how Astra could democratize VFX. Even a small addition like this usually requires hiring a VFX specialist—a luxury reserved for big-budget filmmakers ( David Fincher is famous for using subtle VFX throughout his work).
Astra rendered the wineglass’s transparency believably. Its surface catches reflections, while the lamp’s slim metal pole and the actor’s passing hand appear distorted through the glass.
Imagine you were hanging out at home and saw a giant Snoopy float outside your window.
The casual iPhone shot helps sell the improbable sight. Astra rendered a Snoopy model in Blender and masked the window and curtains in front of it. Astra used Meta’s Segment Anything Model (SAM) 3.1 for the initial mask, refined it with its own code, and used Runway’s video model, Aleph 2.0, for an extra surface and texture pass. We matched the exposure and edges and softened the float so it looked as though you were seeing it through the glass. A small foreground bird helps bring the shot together.
We tried the same approach with an imaginary Afterimage object. In another iPhone shot, the gold sphere flies in, lands on a rug, and opens to reveal a red core with our wordmark recessed into its surface.
This solid geometric object is the kind of thing Astra excels at building in Blender. We could control the sphere’s shape, the opening petals, the lettering, and its arrival path. Astra fitted a virtual camera to the handheld footage and added lighting, reflections, and a shadow to connect the object to the rug. Bright reflections obscured the recessed wordmark at first, so we adjusted the lighting to move them off the letters.
Astra had trouble in two areas: making new elements interact convincingly with the photographed scene and creating organic forms that look alive.
We were curious whether Astra could achieve “body horror” effects, so we took a shot of a hand and tried to make eyes grow out of the fingertips. Astra keeps the eyes aligned with the moving fingers, but they still look tacked onto the fingers’ surfaces. To sell the effect, the eyelids need to merge into the surrounding skin, and the skin needs to stretch and crease as the eyes blink. Those small relationships would improve the effect’s realism, but even after extensive iteration we couldn’t get Astra to make them convincing.
We ran into a similar problem when adding cows to a field in this drone shot from our recent video for Turbopuffer. Astra placed the new cows in the grass and maintained their positions as the camera moved, but they don’t look fully integrated into the scene. Blades of grass should overlap their hooves and respond to their steps. When a cow lowers its head to graze, its mouth doesn’t touch or pull at the grass.
Astra also struggled with the animals themselves. The models looked unconvincing even when still. In motion, their bodies didn’t deform naturally and their steps lacked weight. Repeated revisions didn’t fix either their appearance or movement.
Astra feels like a breakthrough for a small studio like ours. It can take on substantial VFX work using our existing tools and turn specific feedback into editable changes. It still needs close direction, and some shots remain beyond what it can do convincingly. But we can now prototype and execute some of our VFX work without substantial planning or budgeting up front, widening what we can offer clients and letting us stretch their dollars in exciting ways.
But the most interesting consequences for us are aesthetic rather than practical. In a pre-Astra world, we never would have thought to add VFX to clips like the iPhone Snoopy and orb examples above. The resulting combination of casual footage with visual maximalism feels genuinely new and quite bizarre. What excites us most about Astra is its potential to unlock types of images that didn’t previously exist.
Everyone’s a builder now. Every All Access：https://every.to/builder-pack gets you the full membership plus the Builder Pack—$9,000+ in credits for the tools we build with.
Sign in to read for free and get your AI assessment prompt.
扫码直达文章详情