Halladin Labs
← All work

Autonomous action · In progress

The agent isn't the bottleneck

On autonomous ad-creative systems, and why the hard constraint turns out to be permission rather than capability.

Specification designTrust modelsIncrementality
Role
[Your title]
Scope
[e.g. Authored the technical brief and feasibility assessment]
Status
In progress — publishing the reasoning, not the results
Stack
Platform ad APIs · attribution modelling · scheduled pipeline + LLM decision layer

This piece describes work in progress. I'm publishing the reasoning rather than the results, because the reasoning is the part that transfers.


The pitch writes itself. An autonomous agent swarm that generates ad creatives, deploys them, measures them, retires the losers, and promotes the winners — turning ad spend into a self-optimizing loop. Point it at a budget. Walk away.

It is a good pitch. I have spent a while working out how much of it is actually buildable, and the answer is genuinely interesting: on some surfaces, nearly all of it. On others, almost none of it — and not for any reason a better agent would fix.

That asymmetry is the real finding, and it generalizes well beyond advertising.


The loop, stated plainly

Strip away the framing and a creative flywheel is six stages:

Generate → Store → Deploy → Measure → Decide → Retire/Promote → (repeat)

Each stage has a different automation profile, and it is worth being precise about which stages are actually hard.

  • Generate is solved, or close to it. Image and video generation is good enough to produce creative variants at volume.
  • Store is a versioned asset bank. Object storage and a schema. Not hard.
  • Deploy is where the whole thing lives or dies. Discussed at length below.
  • Measure is where most implementations quietly fail. Discussed below.
  • Decide is a rules engine with an LLM layer for the ambiguous cases. Tractable.
  • Retire/Promote is just Deploy again, in reverse.

Notice that four of six stages are essentially engineering. The two that determine whether the system works at all — deploy and measure — are both about your relationship with external parties, not about your architecture.

That is the thesis.


"Agent swarm" is an ambition, not an architecture

The original framing called for an autonomous agent swarm. I want to push back on that phrasing, not because multi-agent designs are wrong, but because the phrase smuggles in an architectural commitment before anyone has established that the problem needs one.

The real requirement is a self-reinforcing loop: insight → decision → deployed creative → new signal → back to insight. Whether that is one agent, several coordinated agents, or a scheduled pipeline with an LLM decision layer is a design choice, and it should be made on the merits.

My strong prior, from having built both shapes: for a loop that runs on a daily cadence against a fixed set of platforms, a scheduled pipeline with an LLM decision layer is the right answer, and a multi-agent swarm is a liability.

The reasoning is the same one that shaped the experiment agent: batch problems permit architectures that interactive problems do not. A daily loop does not need agents negotiating with each other. It needs a DAG, a decision step where judgment is genuinely required, and a very good audit trail. Multi-agent coordination introduces non-determinism into a system that is about to spend real money, in exchange for flexibility the cadence does not require.

Reach for the swarm when the problem is genuinely open-ended and interactive. Deploying ad creatives on a schedule is neither.

Match complexity to the requirement. The most common failure in this category of project is building infrastructure to support an ambition rather than a workload.


The platform asymmetry

This is the heart of it, and it is the part most plans hand-wave.

The flywheel behaves completely differently depending on which surface it is pointed at, and the difference has nothing to do with how good your agent is.

Off-platform social: mature APIs, real autonomy

The major social ad platforms expose genuine programmatic advertising APIs. Creative upload, ad and ad-set creation, deployment, pausing, retirement, automated rules — all available and all documented.

On these surfaces the full loop is buildable today. This is where the scaling opportunity lives, and it is also the higher-risk surface, because real money moves fast and an automated system with a bug spends it faster than a human can.

A few practical constraints worth researching for your own case rather than assuming: rate limits and asset limits at the volume you actually need, the distinction between natively-uploaded and creator-sourced ad formats (which affects both what you can deploy and what you can measure), and each platform's stated policy position on automated creative generation at scale. That last one is not a technical constraint but it is a real one, and it is the kind of thing that changes without warning.

On-platform connected TV: partner-gated, largely manual

Connected-TV and device-level ad surfaces are a different world. Depending on the partner, you may find:

  • Manual placement processes with multi-day turnarounds and no self-serve API at all
  • Faster turnaround but limited data transparency, so your measurement signal is weak even when deployment is possible
  • Roadmapped ad platforms where backend improvements land well before any self-serve tooling

For these surfaces, autonomous deployment is not achievable, and no amount of agent sophistication changes that. There is no API to call. The constraint is commercial and organizational, not technical.

The honest response is not to pretend otherwise. It is to pivot the goal on those surfaces from autonomy to faster human-in-the-loop: tooling that compresses the manual workflow, prepares the assets, drafts the submission, tracks the turnaround, and surfaces the results — so the human step takes ten minutes instead of two hours.

That is a less exciting deliverable. It is frequently the higher-ROI one, and saying so is the main thing that separates a useful plan from a deck.

Why this matters strategically

There is a structural trap here worth naming. If your largest channels are the partner-gated ones, then the surfaces where you spend the most are the surfaces you can automate the least — and the surfaces you can automate are the ones where you have the least existing volume.

That turns a technical project into a portfolio question: the flywheel is not just an efficiency play, it is an argument for shifting spend toward surfaces where compounding automation is possible at all. Worth being explicit about that, because it changes who needs to approve it.


Measurement gates everything

Here is the failure mode I would most expect a team to walk into.

The flywheel's decision quality is bounded entirely by the quality of the performance signal feeding it. An autonomous system making creative decisions on biased data does not make neutral mistakes — it compounds them, because every cycle reinforces whatever the bias favored.

Three specific traps:

Survivorship bias in the measurement set. If you can only measure the creatives that already succeeded — because the measurement pathway is only available for a subset of ad formats, say — then your system learns exclusively from winners. It will have no model whatsoever of why things fail. It will confidently generate variants of past successes and have no ability to recognize a novel failure.

This is the single most dangerous input problem, and it is invisible from inside the system. Every metric looks fine.

Attribution that doesn't match the decision. If you are moving to a source-level attribution model, and your measurement vendor does not support that model natively, then the numbers driving your automated decisions come from a custom modeling layer you built. That layer needs the same scrutiny as the statistics in any experimentation system — and it usually gets far less, because it is plumbing rather than science.

Correlation standing in for incrementality. Before trusting a system to substantially increase spend, you need incrementality measurement — geo tests, holdouts, auction experiments. A creative that correlates with conversions is not the same as a creative that causes them, and a loop optimizing on last-click will happily spend more and more money buying users who were going to convert anyway.

The sequencing implication: the measurement work is not a dependency of the flywheel, it is a prerequisite. Building the loop first and fixing the signal later means running an automated optimizer on a biased objective function, at increasing spend, for however long the fix takes.


The trust model

Assuming the signal is sound, the question becomes how much autonomy to grant and when.

The framing I have found useful is that autonomy is not a setting, it is something earned per action type. A single global "autonomous mode" toggle is the wrong shape.

Stage 1 — Recommend. The system produces recommendations and a human executes them. Every recommendation is logged with its reasoning and its inputs. The metric you are watching is not accuracy in the abstract; it is human override rate, broken out by action type.

Stage 2 — Act with confirmation. The system prepares the action fully — the creative is uploaded, the ad is built, everything is staged — and a human approves with one click. This removes the effort of execution while retaining the judgment.

The distinction between stages 1 and 2 matters more than it looks. Most of the human cost is in doing the thing, not in deciding it. Stage 2 captures most of the value at a fraction of the risk.

Stage 3 — Act autonomously within bounds, by action class. Widen per action type, in the order of how reversible the action is:

  • Pausing an underperformer is cheap to get wrong and easy to undo. Automate first.
  • Promoting a winner — increasing spend — is expensive to get wrong. Automate later, under a budget cap.
  • Generating and deploying novel creative carries brand risk that is not reversible by pausing. Automate last, if at all.

Reversibility is the ordering principle, not confidence. A confidence threshold ("act automatically above 70%") is a reasonable secondary gate, but it should never be the primary one, because it treats an easily-undone pause and an irreversible brand-safety incident as the same class of decision.

Non-negotiables at every stage: a complete audit trail of what acted, on what input, with what reasoning; a kill switch that a non-engineer can hit; and hard spend caps enforced outside the agent's control, so a malfunctioning loop cannot raise its own ceiling.

That last one is worth stating clearly. The budget limit must not be a parameter the system can modify. Put it somewhere the agent cannot reach.


A note on the artifact itself

There is a second thing worth surfacing about this project, separate from the subject matter.

The deliverable I produced was not the plan. It was a brief that instructed another agent to produce the plan — a specification detailed enough that a capable model could research the platform landscape, reason about the architecture, and write the document, with the honesty constraints and structural requirements pinned down in advance.

That turns out to be a genuinely different skill from either writing the plan yourself or prompting casually. A few things I found mattered:

Separate what is known from what must be researched. The brief explicitly marked company context as given ("use it, don't re-derive it") and platform capabilities as requiring verification. Without that split, a model will confidently restate your own context back to you as though it were a finding.

Specify the honesty requirement structurally, not tonally. Asking for a "balanced" or "realistic" assessment produces hedging. Requiring every capability claim to be sorted into possible now / not possible now / open question — with the third being a mandatory, prominent section — produces something you can act on. Give the model a place to put uncertainty and it will use it; give it none and it will resolve uncertainty into false confidence.

Name the anti-goals. "Treat 'agent swarm' as a description of the ambition, not a required architecture" prevented an entire category of over-engineered output. Constraints on what not to assume are more valuable than constraints on what to produce.

Demand the least-bad alternative wherever the answer is no. A plan that says "this isn't possible" is half-finished. A plan that says "this isn't possible, and here is the workflow tooling that captures most of the value" is usable.

I think specification-writing of this kind is becoming a core skill, and it is being under-taught relative to prompt-writing. The difference is that a prompt gets you an output and a specification gets you an output you can evaluate — because the criteria were fixed before you saw it.


The open questions

Some of the most useful output of this exercise was not answers but a list of the specific things worth asking partners, vendors, and internal teams. Framing them as answerable questions rather than general risks is what makes them actionable:

  • What does each ad platform's policy actually say about automated creative generation and deployment at our volume — and who at the platform can confirm that in writing?
  • When do partner-gated CTV surfaces expose any creative-management API, and what is the realistic timeline versus the roadmap timeline?
  • Does our measurement contract permit the raw-export volume a custom attribution model requires?
  • What is the smallest incrementality test that would give us enough confidence to raise spend, and how long does it take?
  • What is our actual rollback procedure if an automated creative causes a brand-safety incident, and has anyone rehearsed it?

A plan whose open-questions section is specific and answerable is worth more than one that appears complete. The completeness is almost always false; the questions are what move the project.


What generalizes

The bottleneck in autonomous systems migrates. It starts as model capability, then becomes verification, then becomes permission. Most teams are still solving for the first while blocked on the third.

Automation potential is a property of your integrations, not your architecture. Audit what you are actually allowed to do programmatically before designing anything.

Order autonomy by reversibility, not by confidence.

An automated optimizer amplifies whatever bias is in its objective. Fix measurement first or you are scaling an error.

Saying "this surface can't be automated" is the most valuable sentence in the plan. It redirects effort to where the effort pays.


Building something like this? The measurement and trust-model questions are the ones I'd want to talk through first — they're where these projects usually go wrong, and they're much cheaper to get right at the design stage.

Building something like this?

The measurement and trust-model questions are the ones worth talking through first — they're where these projects usually go wrong, and they're much cheaper to get right at the design stage.