Frontend is HARDER for AI than backend (here's how to fix it)

00:00There's a weird problem with AI coding agents at the moment, which is that they are way better at writing backend code than they are at writing front-end code. I'm not talking about producing like front-end slop like basic marketing pages and just, you know, the demos that you see people oneshotting.

00:13I'm talking about complicated user interactions, multi-step forms, annoying animation things. If you ever tried using an LM for this, then you will find it is not very good at handling that stuff. The reason for this is that the feedback loops that the LLM is using are way more precise on the back end than they are on the front end.

00:31On the back end, you can write some code, right? And then that code is often tested by more code. The outputs of those tests are written in text and the entire loop, the entire environment is encoded in text. Even the information about the documentation that you're using is written in text.

00:46And the LLMs are really good at writing and understanding text. The difficulty comes when you start adding visual elements to this. A feedback loop in a front end. If you've ever been a front-end developer, you know this is to write some code, then look at the UI to see what changed, go back, write a bit more code, look at the UI again.

01:03In other words, as humans, we're not just looking at text. We're also taking in visual information. We're able to use our innate design sense and get a feel for what is supposed to be where and try out user interactions ourselves. And in fact, we're not just processing images, we're processing video as well.

01:19We're trying to feel how things scroll, how an animation feels when we hover over it. In other words, so much of this feedback loop is visual, not textual. And this is a problem for a lot of AI coding agents. With a back-end feedback loop, you can add unit tests, you can add linting, you can add type checking.

01:35You can add these in a pre-commit hook to make sure when the coding agent commits its code, it's going to see the errors come up if there are any. We can hook up these same things here to the feedback loop, but they're all textual, right? We still need a visual element to make the AI coding agent understand what it's done and whether what it's done actually works.

01:54I've been working with AI to rebuild my blog recently and it kind of looks like this. Pretty basic, but I quite like the minimalist design. I'm going to get claw code to do some QA for me. I'm going to ask it to double check if light mode and dark mode are working okay on the homepage and check if all the content is rendered acceptably on both.

02:10I'm then just going to pass it my local server which is localhost 5175. What it's going to do here is it's going to check the homepage both light and dark mode using Chrome DevTools. Its first tool uses to spin up a new page and I'm going to say yes and don't ask again.

02:23It's now going to take a screenshot of that with a full page true and it took a screenshot of the full current page. It spotted I don't see a visible toggle. The page likely uses system preference. Let me emulate dark mode using a strict script to check the prefers color scheme behavior.

02:37And it continues in this vein doing more screenshotting, checking all the pages. Adding the dark class didn't change anything. The site is using media based query dark mode. Very good. It finally figures out that we're using media prefers color scheme dark and then it manages to emulate that.

02:51Takes a screenshot. Both modes look good and cleans up and provides a summary. The way I've got this set up is with an MCP.json inside my project which is running the Chrome DevTools MCP server pointing it at a running browser URL. Alongside this, I also have a skill inside here for Chrome debugging.

03:08If you don't know, a skill is kind of like something you would put in your agents.md file, except the agent can pull it in on demand when it needs to. This one, for instance, says, "Before using Chrome DevTools MCP, you must launch Chrome in headless mode with remote debugging enabled."

03:21And that's it. That's the entire setup. So, alongside the textbased feedback loops like testing, linting, typeing, we've now got an actual browser that the LLM can go and use. And having the browser there means the LLM is much more like a human coder.

03:35They get access to all of the feedback loops that you have, too. And bear in mind, the LLM is not actually writing tests for this stuff or writing end-to-end tests you need to run on every commit. It's doing the same kind of testing that human devs do, which is ad hoc testing the feature that they built, actually works.

03:51Now, I posted this on X and I got some really great responses. Basically saying that the Playright MCP, which was what I was using at the time, is not very good. And there are several fighting in the same space that are maybe a little bit better.

04:04The one that Damian pointed out here, dev browser, is pretty good, but I have not yet tried it. And the other one that he pointed out, which was this Playriter, I think, yeah, this thing here, another MCP, is also supposed to be really good, too.

04:16What people say about the Playright MCP server and the Chrome DevTools server, too, is that they're pretty context hoggy. They take up a lot of tokens in order to pump their screenshots back into the LLM. and their MCP servers are configured in a way that makes them really context heavy like really detailed tool descriptions, lots of different tools, that sort of thing.

04:36You can also use Claude code directly with Chrome using the Claude Chrome extension. I can't use it at the moment because I'm on WSL and it's not supported on WSL. The last thing to say here is that this front-end feedback loop makes AFK AI coding a lot more powerful.

04:50If you're looping an AI, for instance, in the Ralph Wigum setup, then plugging a browser into your front end or full stack work will be such a massive improvement because without it, your LLM is essentially flying blind. It can't see the execution environment in which the changes are being made.

05:05So, I've noticed an enormous improvement when I plug in a visual feedback loop into this setup. If you dig this, then you will dig all the stuff I'm putting out on AI Hero. There are a ton of free tutorials and free guides that help you start building with AI and especially putting AI into applications which is where things start getting really gnarly and really fun.

05:25But thank you so much for watching and I will see you in the next