Engineering brief
The $43K SaaS replacement that shifts the hard part to maintenance
At a glance
Worth watching
- Relevance
- Practical value
- Radar
- Time horizon
- Now
- Warnings
- None
An events company's contest replaced a $43K/yr tool with AI agents for roughly $11K, yet the winner ranked 19th in the automated evaluation. Agent-built software works, but the cost that remains is ownership, maintenance, and human judgment.
Build-versus-buy math changes when agents let a small team build mid-tier software, but ownership costs and human review decide whether it is worth it.
Watch if you are weighing a bounded pilot to replace a mid-tier SaaS and want to see real user eval failures and agent workflow patterns.
Summary
The contest. The organizer of the AI Engineer conferences published a 'Kill my SaaS' challenge after SessionBoard quoted $43,000 a year for four events. 72 teams built replacements from a written brief, 42 screenshots, and a walkthrough video. Five apps cleared the bar for real adoption, and the winning entry from Valentin shipped multiple portals and a schedule the events team wanted to use. The estimated build cost: roughly $1,000 in tokens plus about $10,000 in AI engineering time.
The workflow. None of the finalists typed a prompt and walked away. They researched before coding: one crawled SessionBoard's help portal with Firecrawl, one fed the walkthrough video and API docs into Claude, and one mapped every feature flow before launching parallel agents. Planning was most of the work. The prompt came last. That suggests agents are useful for execution, but the quality depends on the human's knowledge of the product and domain.
The evals. Real users disagree with automated scoring. The team built an AI judge that had Claude log in and click through 20 scenarios; it was useful for bootstrapping, but its top 10 included six apps that human reviewers rated weak. The eventual winner ranked 19th on the AI leaderboard. Humans caught what agents missed: multiple portal types, drag-and-drop schedule state, and adding a talk without going through a CFP form.
The pattern. The builder workflows are the most transferable signal. Claude Code and Codex were roughly tied, and most teams used both. Three patterns emerged: a stronger model plans then a cheaper model writes; one model builds and another checks; no model approves its own work. One team ran Claude and Codex in separate workspaces and required one to verify the other before merging. Separate verification is an operational discipline that matters more than any model feature.
The trade-off. The hard part starts after launch. Someone must fix bugs at 2 a.m., handle integrations, and keep the system running, a real shift from vendor support. The transcript includes a sponsor with a ticket bug and a support chain ending in 'refresh your cache'; the company decided it wanted software it could actually own. But owning means staffing an internal software team, even if some of it is designers and operations people directing agents.
The verdict. This is not proof that AI killed SaaS. It is a credible, transparent experiment showing that a mid-tier tool can be replaced when a team has real user requirements, agent orchestration skills, and an appetite for maintenance. The $43K savings is real, but the long-term cost of ownership is easy to underestimate. Engineering leaders should treat this as a bounded pilot option, not a mandate to stop buying software.
My take
I think this is one of the more useful agent experiments I have seen, because it was judged by people who simply needed the job done. The $1,000 in tokens and $10,000 in expertise is an honest starting estimate, but I would be careful treating it as the total cost. Maintenance, bug fixes, and evolving business rules come after launch, and the video's own 2 a.m. example shows that burden is real. What I would take into an organization is the workflow pattern: separate planning, execution, and verification. The rule that no model approves its own work is more management discipline than technical detail. I would use this as a reason to build a small internal tools team around people who understand the domain, not as a reason to cancel SaaS contracts.
What to do with this
- Interview your operations team about the top three jobs-to-be-done a mid-tier SaaS covers, then estimate the build.
- Run a bounded internal contest with a brief, screenshots, walkthrough video, token budget, and real-user evaluation.
- Create a maintenance and on-call plan before starting any agent-built replacement of a purchased tool.
Watch the video
This video is blocked due to your privacy settings. To watch this video, please accept YouTube marketing cookies.
Related breakdowns
Small Finance Model Beats 235B for Under $500: The Data Wins
A 4B finance agent beats a 235B model for $500 in training. The tradeoff: narrow benchmark, vendor-backed evidence.
Personal AI subs offer 40x more tokens – but at what cost?
Personal AI subs give you 40x the tokens of API for coding. But the tradeoff: account bans, proxy setup, and constant token management.
Fable 5.1: The merge manager generation has arrived in AI coding
Fable 5.1 shifts AI coding from generation to autonomous merge management—89 PRs in 24 hours. But verbose output means one-shot tasks may cost more than…
Get TL;DW
Too Long; Didn't Watch.
A concise breakdowns of the AI and devtools videos that actually matter for engineering leaders.
Free. Weekly. No hype.
Video and thumbnails remain the property of their respective creators. tldw.news provides editorial analysis, commentary, and discovery links to original content.