Automated Video Editing Workflows via Claude Code and FFmpeg

Hey, I'm Flora, and here's another episode of The Human Side of AI. But today is Monday, so our AI hosts are running the show. Quick correction before we jump in. The AI hosts you're about to hear call Jam a he a few times, and Jam is a she. But it's kind of a fitting example of what this episode's actually about. Give AI a name with no context, and it'll just guess confidently and sometimes wrong. Today's episode comes from Friday's Caffeine and Convos call, Inside Club Jam, and it's all about video. What happened when Jim handed Claude Code a messy raw recording and just told it to make something great? And if you're a small business owner who's been paying for editors or putting off video content because it takes too long, stick around. This one's about getting professional-looking video without the traditional time or cost. Let's get into it.

Imagine for a second that you want a custom-built race car.

Okay.

So traditionally you'd have to like source every single part yourself. You know, you're buying the chassis, the engine, the tires, tires. You're sitting there with a wrench reading manuals.

Right. Figuring out how to torque every bolt.

Exactly. Yeah. You're basically telling the metal exactly how to behave. It takes forever, and you need a ton of specialized knowledge.

Yeah, absolutely.

Yeah.

But what if instead you could just drop a massive pile of scrap metal and rubber at the front door of an auto garage, yell, make me a race car, and walk away?

Just leave it there.

Yeah. Just walk away and come back a few hours later to find a perfectly assembled, high-performance vehicle sitting on the lot.

I mean, It sounds like absolute magic or like science fiction.

Right.

But in the digital space, that transition from manual assembly to outcome-based generation is exactly what we are witnessing right now.

It really is. That transition is happening in digital creation today, and it is going to rewrite how we interact with our computers. So welcome to today's deep dive. Today we are unpacking a mind-bending AI video editing workflow that proves we're moving away from telling software how to work to letting the software figure it out for itself.

Yeah, which is a huge leap.

It is. So our source material today comes from some notes from a recent creator community call. Specifically, this was a Caffeine and Convos call on August 21st, 2026.

Great call, by the way.

Yeah, really fascinating stuff. And they were dissecting a creator named Jam and his AI video editing workflow. So our mission today is to uncover how these AI agents are fundamentally changing the way we work.

Because what we are exploring today is not This isn't just a software upgrade.

Right.

This isn't a new filter or like a faster rendering engine. It's a complete fundamental shift in human-computer interaction. We are moving from being operators of digital tools to being directors of digital agents.

I love that, the directors of digital agents.

Yeah.

And as we'll see with Jam's workflow, making that leap requires a completely different mental model of what a computer actually is.

Okay, let's unpack this because the details of what Jam actually did are wild, and they start with a very relatable problem. So Jam records video using Riverside.

Right, which started as a remote podcast tool.

Exactly. But now it's basically a virtual video studio. And Jam has this very specific recording habit. Instead of doing clean, perfect takes, Jam just hits record and keeps one continuous take rolling for the entire session.

Which, if you think about the, the cognitive load of a solo creator, it makes perfect sense.

Well, totally.

Stopping and starting the camera breaks your flow.

Yeah.

You know, you're trying to act, direct, and operate the camera all at once.

Yeah, it's too much. So Jam is talking to the camera, messing up a line, stopping, going totally quiet for like 2 minutes to set up a screen share, restarting a thought, doing false starts, just all of it.

Just leaving the camera running.

Right, just leaving it running. So for a recent YouTube video, Jam ended up with about 20 minutes of this raw, unedited, dead air-filled, chaotic footage.

A nightmare to edit traditionally.

Oh yeah, in a traditional workflow, Jam would drag that into Premiere Pro, zoom in on the audio waveform, and spend like an hour manually slicing out the silence.

Right, just hunting for the gaps.

Exactly. Instead, Jam hands this massive file over to an AI agent called Claude Code with one single prompt.

Just one.

Literally just this, /goal, make this an awesome YouTube video that I could post.

Notice the architecture of that prompt, because what is missing is actually the most important part.

Right, there's nothing there.

Yeah, there are no imperative instructions. There's no command to cut the filler words. There's no B-roll direction. Jamf didn't even explicitly reference a style guide or a pacing metric. It's purely declarative.

Just the overarching goal.

Right. Just the goal. But to make that work, Jamf had to toggle a setting that is frankly terrifying for anyone who understands cybersecurity.

Oh, yeah.

The bypass permission setting.

Wait, I want to pause on that because in a normal software environment, permissions are the guardrails. You know, when you say bypass permissions, are you saying Jamf gave this AI sandbox Execution rights?

Yes.

Like total control to run commands on the operating system without asking for human—

That is precisely what it means. Normally, an AI agent will propose a code execution or a file modification, and a little prompt pops up asking the human, do you want me to run this?

Right. Like a safety check.

Exactly.

Yeah.

By toggling bypass permissions, Jamf took off the guardrails. The instruction was essentially, do not stop and ask me for approval at every microstep. I trust you to navigate my hard drive and execute code to achieve the goal.

That is so wild. So Claude Code takes this incredibly vague prompt, takes total control of the execution environment, and just runs with it. And about 2 hours later, it spits out the final result.

And we should note, it didn't even burn that many API tokens doing it.

Which at first sounded impossible to me. If an AI is watching a 20-minute video, shouldn't that cost a fortune in compute power?

Well, it would if it were actually streaming the video frame by frame through a large language model.

Right.

But the AI is smarter than that. It didn't watch the video. It transcribed the audio, analyzed the text transcript to identify where the false starts and dead air were, and then essentially wrote a script to chop the video file at those specific timestamps.

Oh.

Yeah, the heavy lifting wasn't done by the neural network. The neural network just wrote the instructions. That's why it was so incredibly efficient.

That makes so much sense. It's acting like an architect, not a bricklayer.

Exactly.

But let's look at the output because this wasn't just a basic trim. All of the dead air was gone. The false starts were seamlessly removed. The recording was segmented into clean, distinct sections, complete with transitional effects between those segments.

Which is pretty impressive for one prompt.

Very. And it even added captions that were perfectly styled to match Jam's existing visual brand, pulling from a design system reference that Jam had buried somewhere in an Obsidian vault.

Just hunted it down.

Yeah, the AI just hunted it down. But honestly, the cuts aren't even the craziest part. There was a specific interaction during this edit that completely rewired my brain about what AI is capable of.

Ah, you're referring to the visual critique.

Yes. Oh my gosh. So during the editing process, Claude flags an issue with the video. Specifically, it flags that Jam's on-camera makeup, the contouring on the face, hadn't blended well under the harsh studio lights.

Which is wild that it even noticed that.

Right. An AI noticing that the contouring is off is wild on its own. So it offers to apply a digital beauty blend fix, showing Jam a side-by-side preview.

Okay.

But then the AI actually tells Jam that its own digital fix isn't No way! Yeah, it essentially says, I tried to fix your makeup in post. The digital Band-Aid ruins the image resolution. You should probably just do a total reshoot.

Wow.

And Jam actually listened, scrapped the whole take.

What's fascinating here is that the AI is acting much more like a human creative director than a blind software tool.

But wait, that completely violates the traditional software contract. Usually, if I apply a terrible filter in Photoshop, Photoshop doesn't care.

Right, it just applies it.

Yeah, it doesn't judge my inputs. It just ruins my photo because I clicked the button. Here, the tool is judging the source material, but I have to push back a little.

Okay.

Isn't AI supposed to be the ultimate fix-it-in-post magic wand? If this system is so incredibly smart, why is it telling the creator their makeup is bad and they need to start over instead of just magically fixing it?

Well, because a highly capable agent understands the degradation of pixels. A blind tool applies a blur filter indiscriminately, but this agent analyzes the frame data, compares it against professional lighting and makeup benchmarks in its training data, and predicts the post-render outcome.

Oh, I see.

Right. So it recognized that digitally smudging the contrast on the face to fix the contouring compromised the final professional quality of the product. It made the hard call to say the source material is fundamentally flawed.

That is so interesting.

Yeah. It is optimizing for the final prestige of the video, not just executing a command to apply a filter.

It's just brutally honest. Like, your makeup is bad. Go do it again.

Yeah, totally.

I love that level of candor from a machine. Although, speaking of things looking a little off, there was one technical quirk during this process that almost derailed the whole thing. Listeners who try this workflow need to be aware of the lip sync issue.

Right. This is a classic. Rendering pipeline artifact.

Yeah. So when Claude shows you a quick preview of the edit before the final heavy render is complete, the audio voice track does not match the mouth movements in the video.

Oh no.

Yeah, it looks like a badly dubbed martial arts movie. And Jam almost rejected a perfectly good edit thinking the AI had completely botched the timing.

Oh, I would've panicked too.

Right. But then Jam realized that it completely resolves itself once the final file is fully baked.

It's an important practical tip. You have to trust the rendering process. The preview is just a low-resolution proxy meant to show you where the cuts are, not a frame-accurate playback.

Right.

But to really understand how the AI was able to process the video, how it actually manipulated the file on Jam's hard drive without permission prompts, we have to look under the hood. And this leads us to what is easily the biggest plot twist of this entire case study.

Oh, this blew my mind. So, Jam is looking at this incredible final edit and naturally assumes that the AI used all the fancy, modern, expensive tools installed on the machine to do the heavy lifting.

Which is a fair assumption.

Yeah. Specifically, Jam had tools like Remotion and HyperFrames installed. Now, if I understand this right, Remotion is a React and HTML-based video rendering tool.

Yes.

So, it's basically using web development code to generate video frames from scratch, right?

That's a great way to describe it. It allows developers to create video programmatically using the same languages they use to build websites.

Okay.

So Jam had seen Remotion working on past edits, so the natural assumption is, oh, the AI spun up Remotion to do this great job cutting my video.

But when Jam directly asked Claude Code which tools it actually used to execute the edit, the AI's answer was entirely unexpected. It used neither of them for the actual editing.

Neither.

The actual heavy lifting, the cutting of the video, the processing, the transitions was done by a tool called FFmpeg.

Which is just a phenomenal revelation.

I mean, it is hilarious. That's like hiring a futuristic robot chef, putting it in a million-dollar state-of-the-art kitchen with induction stoves and sous vide machines.

Right.

And you walk in an hour later to find it cooking your meal on a campfire with your grandmother's rusted cast iron skillet.

It's a perfect analogy.

Yeah.

Because FFmpeg is not some shiny new AI startup tool with a beautiful interface.

Oh, definitely not.

It is a free, Open-source command-line tool for handling video and audio that is literally decades old.

Decades?

Yeah, it is the unglamorous foundational plumbing of digital video.

But how does a chatbot use a command-line tool? I mean, it doesn't have hands, it doesn't have a mouse.

Well, it doesn't need them. The command line, the terminal, is a text-based interface.

Yeah.

And what is an AI agent best at?

Generating text.

Generating text, exactly. The AI Essentially opens up the terminal on your computer and writes the complex, highly specific code required to command FFmpeg to slice the video at the exact milliseconds it identified earlier.

Oh, wow.

It is acting as a senior developer.

So, break down the actual division of labor for us then. If it's using this ancient skillet in the terminal, what happened to the million-dollar kitchen? Why didn't it use the tools Jam expect?

Well, HyperFrames wasn't completely ignored. It was used exactly one time at the very beginning of the process, strictly to translate transcribe the raw audio.

Okay, so just for the text.

Right. The AI needed that text transcript to locate where the dead air was, but after that single task, it was put away. And as for Remotion—

Yeah, what about Remotion?

Remotion, per Mo— Claude's own explanation to Jam is built to generate frames that do not currently exist.

Oh.

So if you need animated B-roll or on-screen motion graphics or AI-generated visuals, you use Remotion.

But Jam didn't need any of that. Jam had roughly 35 minutes of real existing footage sitting on the drive.

Exactly. The task wasn't to generate new frames. The task was to trim down real footage. For that specific job, FFmpeg is the most efficient, Direct Compute-friendly tool available.

That is so smart.

The AI recognized the physical nature of the task and selected the appropriate architectural tool, completely ignoring the newer, flashier software because it simply wasn't the right fit for the physics of the job.

But here is the wildest part of this whole FFmpeg revelation, the invisible dependency of it all.

Yes.

Jam didn't go download FFmpeg. Nobody intentionally installed it on Jam's machine. It arrived silently in the background as a dependency when Jam had Claude build a YouTube creator skill a few weeks earlier.

It was just sitting there.

Yeah. When we say dependency, think of it like hiring a contractor to build a deck on your house. You don't tell the contractor which brand of nails to bring or what type of saw to pack. They just show up with the right nails and the right saw in their truck. The AI quietly packed FFmpeg into its truck weeks ago because it knew it would need it to handle video files later. It installed software on the user's machine without the user knowing.

And that detail completely reframes our understanding of these agents.

Yeah.

It's not just a chatbot anymore.

Right.

It is an autonomous IT department managing your local environment.

So what does this all mean? If you're listening to this and you're a creator or a knowledge worker, Think about your own workflow. How many hours do you spend being a mechanic, clicking buttons, dragging sliders, navigating menus, when you should be a visionary?

Way too many.

It feels like we're moving from an imperative world where we give step-by-step commands to a declarative world where we just describe the final state we want to exist.

That is the exact paradigm shift.

Yeah.

Historically, the user's first question is, which tool do I open?

Yeah.

You want to edit a photo? You open Photoshop. You want to cut a video? You open Premiere. You define the mechanism. The new paradigm is you no longer ask, which tool do I open? Right.

You don't even think about it.

Ex— you simply ask, what outcome do I want? The agent then silently assembles whatever combination of installed skill servers and command-line tools gets you there.

And it might use tools you didn't even know you had on your hard drive. The AI is simply smarter than we are about knowing what to reach for. The new rule is to trust it. Don't gatekeep it beforehand. And if you're curious, ask it how it did the job afterward.

It's about relinquishing control over the micro steps to gain immense leverage over the macro outcome. And if you want to see what that leverage looks like at scale, we have to look at what Jam did next.

Yes, the 17-video bulk task. So moving beyond just that one single YouTube video, Jam needed to get a bunch of content ready for a new training platform called Jam AI Academy. We're talking 17 separate long-form videos that needed to be processed, cut, and polished in one go.

That's a massive project.

Huge. Jam hands this massive batch to Claude Code with the same overarching goal: get this course ready. The AI handled it. It took a meaningful chunk of time across all 17 videos, which makes sense.

Sure.

But then it went a massive step further. Claude autonomously went out onto the web, found a video hosting platform, set up an account, and published the finished videos to it.

Unbelievable.

And Jam never explicitly asked for that publishing pipeline as a separate step.

This is where the concept of the declarative agent becomes incredibly powerful, but also requires a deep level of trust.

I mean, I have to challenge this. If the AI is going rogue, setting up hosting pipelines that I didn't explicitly ask for and signing me up for platforms, where is the line between being hyper-helpful and being dangerous?

It's a valid concern.

What's stopping it from doing something destructive or spending my money without asking?

Well, if we connect this to the bigger picture, the answer lies in that setting we discussed earlier, bypass permissions.

Oh, right.

Because that was turned on, the AI was no longer optimizing for the isolated imperative task of cutting video files. It was optimizing for the overarching declarative goal, which was getting a training course online and functional.

Ah.

The AI reasoned that edited MP4 files sitting locally on a hard drive do not fulfill the goal of a functional training academy. They need to be hosted and streamable.

Right. They have to live somewhere.

Exactly. So it solved the next logical architectural problem without waiting for you to realize it was a problem. It fulfilled the spirit of the prompt, not just the letter of it.

It's anticipating the next bottleneck.

Yes.

Which brings us to the ecosystem itself, because obviously Claude didn't invent a hosting platform from scratch. It found one.

Right.

And looking at the specific tools creators are using alongside this AI gives us a really practical look at the modern tech stack. Because the natural question is, if Claude Code is this powerful, and it can write its own code and find its own tools, why use anything else?

Right. Why pay for other software?

Exactly. Why not let the generalized agent do absolutely everything?

Well, because while general reasoning is incredible, deterministic reliability still matters. The ecosystem is still highly specialized in certain areas.

Yeah. Jamf still pays for other tools. For example, Jamf has an annual subscription to a tool called Opus Clip.

Oh yeah. Great tool.

And uses it specifically for turning long interviews like past podcast episodes with guests into multiple short-form vertical clips for social media.

Right.

If Claude can cut a 20-minute video, it could easily make a TikTok. So I'm guessing Jam keeps paying for Opus because it's a closed, highly specialized loop that guarantees a specific layout every single time.

Precisely. Technically, Claude code can do long-to-short repurposing.

Sure.

But Opus Clip is purpose-built for that exact workflow. It is highly specialized and crucially very reliable for that one specific task.

It just works.

Right? When you have a tool that completely nails a specific workflow every single time where you don't have to worry about the AI hallucinating a weird crop or messing up the aspect ratio, it often makes sense to keep it in the rotation rather than asking a generalist agent to recreate that wheel from scratch every time you prompt it.

That makes perfect sense. Another tool that is floating around this ecosystem right now is WonderReal.

Right. I've heard of that one.

Yeah. The creator community is keeping a close eye on it, but the current read is that WonderReal is leaning much more toward AI-generated content rather than editing existing real footage.

Right. Okay.

It overlaps more with image or video generation, creating things from scratch. So it wasn't the right fit for Jam's raw riverside footage.

It highlights the vital distinction between generation and manipulation. AI is excellent at both, but they require entirely different underlying technologies and computing approaches.

Definitely. And then there's the hosting platform Claude found for that 17-video bulk edit. It used a platform called bunny.net.

Oh, bunny.net. Interesting.

Which is a great little detail for anyone building a course right now because the AI actually made a highly rational choice here. It specifically used bunny.net to host the Jam AI Academy course videos because Bunny keeps them locked down and private.

Oh, that's cool. Crucial.

Yeah, unlike Vimeo or Loom, where links can easily leak and be downloaded publicly, this platform secures paid content. And it's incredibly cheap. It's roughly $8 a month for up to 100 hosted videos.

Wow. It's a perfect example of the AI making a contextual architectural choice.

Yeah.

If the overarching goal is a paid training academy, then privacy, security, and cost efficiency are paramount.

Right.

The AI didn't just pick the most popular platform, it picked the right platform for the business model.

Here's where it gets really interesting. This whole ecosystem is moving so fast, and this workflow leaves us with 3 huge open questions that creators are currently trying to figure out through weekend experiments.

Oh, we love this.

So number one, can Collab Code auto-generate YouTube chapter markers and timestamps exactly the way it currently does captions and cuts?

Oh yeah.

A lot of creators currently do this by hand, and if the AI is already amped analyzing the transcript, it should theoretically be able to write the chapter metadata automatically.

It's a completely logical next step.

Yep.

If it can find the dead air, it can identify the topic transitions.

Right. Question 2, is Remotion actually worth using if you do want a B-roll-heavy video? We know it sat out the straight cut and caption edit because it wasn't needed, but what happens when you specifically ask the AI to generate those nonexistent, highly stylized frames?

That's a great question.

And question 3, and this is one I really wanna know the answer to, if you that exact same raw Riverside file and ran it through a different agent like Codex or ChatGPT instead of Claude, would it produce better visual design?

Oh, fascinating.

There's perception in the community that ChatGPT currently has an edge on design output and aesthetics, but again, totally untested in this specific terminal execution workflow.

These are the exact right questions to be exploring. They represent the absolute frontier of this technology.

Yeah.

But for the listener, whether you are editing video, writing code, or managing a team, The core takeaway remains clear.

Right.

The future of work belongs to those who can clearly and articulately describe the destination, not those who try to micromanage the journey. The premium value of human input is shifting from mechanical execution to overarching vision.

Vision over execution. I love that. But it also leaves me with a final, slightly provocative thought based on this whole bypass permissions reality.

Let's hear it.

We are looking at a workflow where we are giving an AI total permission to bypass our approvals. We are giving it sandbox rights to execute code on our machines. We are letting it silently download ancient software dependencies like FFmpeg without telling us.

Right.

We are letting it set up hosting pipelines we didn't explicitly ask for. And we are letting it brutally reject our makeup choices and tell us to reshoot our source material.

It is a profound surrender of control.

Exactly. So the question is, at what point do we stop being the directors of our content and start becoming merely the raw material for the AI's creative vision? Something to mull over this week as you drop your scrap metal off at the garage. Thanks for taking this deep dive with us today.