Why Your AI Generates Crap Code - And How to Fix It

Introduction

00:00In this video, I'm going to show you how to configure your AI so it doesn't suck. If you're anything like me, you spent countless hours tweaking your agents MD file, downloading different skills, tweaking your documentation, and no matter what you do, it seems like your AI is still making the same mistakes over and over again.

00:13That's why in this video, I'm going to show you how to properly configure an agent's MD file, the different things you need to think about when you're writing that, how skills play a part in this, how other documentation plays a part, what the system prompt does, and all things around configuring how your AI actually writes and handles code inside your application.

00:30But that's not all because I'm also going to show you how this works in a real world project so you can see step by step exactly what I would do. Welcome back to WebDev Simplified. My name is Kyle and my job is to simplify the web for you. And before we start diving through how some of the biggest companies out there such as OpenAI and

What files to configure

00:51Microsoft are making massive mistakes with their agents MD files, we first need to understand what all the different files for configuration are. So I have a really simple example here kind of showcasing the main files you're going to run into.

00:58These are going to be your system prompt, your agents MD, also referred to as cloud MD if you're working with cloud code, any skill files, documentation, your codebase, and then finally your actual prompt. Every single time that you send a prompt to your AI, it doesn't just get the information inside this prompt.

01:14It instead gets a bunch of information fed into it based on what all these other files inside your codebase say. So, the first thing that you need to consider is the system prompt. And this is something that you usually cannot configure, so you don't need to worry about it too much.

01:29Some agents such as PI let you configure this, but for the most part, this is something that like Claude Code or Pi is going to write themselves, and this is just explaining everything related to how the AI should work. For the most part, you don't need to worry about this.

01:37So, we're just going to delete that from our example. But it does get fed to our agent every time we send it a prompt. Next, we have the agents MD file. And this is probably the file that you're most familiar with. Most code bases have some file named agents MD with a bunch of information inside of it.

01:54And this is stored at the root of their entire application. And every single time that you make a prompt, not only is it going to send along whatever your prompt is, but it's also going to take the code inside that agent's MD and put that inside of every single request that you send to your AI.

02:13So if you have a very large agents MD, it gets sent along with every single request. Now, the reason I have Claude MD on here as well is because Claude is special and they don't cohhere to the standard of agents MD and they force you to create a file called Claude MD to work with Claude code.

02:21So, just know if you see claude.md or agents.m MD, they're both exactly the same. You may even see things like gemini.m MD. Essentially, all of them are the same. They are your agents MD file and they get it sent along with every request. Now, another thing that you may have thought about when you're trying to configure your agent to do specific things are skills.

02:40Skills are essentially a set of markdown files that you download and the agent can read those files when it needs to perform a specific task. So let's say that you have a skill for TypeScript code whenever it needs to perform TypeScript stuff in your application.

02:56It'll load up all of the different markdown inside that skill file, read it, and put it inside that prompt as well. So again, every single time you send a prompt, it takes your prompt, it takes that system file, takes your agents file, and any skills that it loads, and it sends all that information along with your original prompt.

03:11Also, if you have various different documentation inside your application or even just your code itself, those can also get sent along with your prompt to actually solve what the problem you're asking is. In our case, we're going to be focusing on how you can configure your agents MD or cloud MD, your different skill files, as well as different documentation to make it so that every single time you send a prompt, it does the thing you expect it to do while writing high quality code.

03:36I'm not going to be focusing on the actual code you write in your codebase because this is all about how do you configure your agents and AI to work properly using these different types of files. And the most important one to configure is going to be your agents MD cuz like I said, it gets sent along with every single request in its totality.

03:44So if you have a bad agents MD, it's going to poison every single request you make to your AI, no matter what you put inside your original prompt. And these examples I have right here that come

Bad AGENTS.md examples

04:00from Codeex, Airflow, and from VS Code, these are all, I think, terrible examples of an agent's MD file. And the main reason that they're so bad is because they are massive. If we just take a look at this one for Codeex, for example, you can see it has a ton of information here.

04:08It has some information on, okay, here is what the thing is supposed to do. And then it just gives us a random list essentially of various different things. My main guess on how this was actually created is someone had a very small agents MD and then they ran into a problem noticing that hey create names weren't properly prefixed so they added that in here and every single time they got a problem it just got added in as a new bullet point in this list and at first that seems like a great idea but as you

04:39run into more and more problems now you're going to have a bulleted list like this that is just absolutely massive and this gets sent along with every single request. So, if you're not doing something related to one of these bullet points, it's still sending that information along, wasting tokens and wasting your actual compute of your AI.

04:54A great example, if we look through this, you can see they have something on the codeex core crate. Next, we have something on code review. I guarantee you not every single request you make to your AI is going to be doing code review. So, you don't need this in your agents MD.

05:02It also has stuff on like API breaking changes. We go further along, you can see that we have a section for the terminal user interface style conventions. Then we have more code conventions. Then again we have styling repeated a second time inside here.

05:10So this is duplicating the exact same thing in two different places. Then we have stuff on testing. You can see things on snapshot testing. A bunch of different information. And as you know not all of this needs to be sent along with every request.

05:26Sometimes you're going to be doing testing, sometimes you're not. Sometimes you're going to be styling things, sometimes you're not. So it's important that we don't send along all this information and instead we make our agents MD as small as possible.

05:31One thing that Codeex is actually doing really well in one particular scenario is up where they have that terminal UI style convention. They're actually linking out to a location where they can find more information. So let's say for example, I want to be able to tell my agent how to style things properly for my application.

05:54I don't want to include that in the agents MD because I don't always do styling for every request. So instead, I'll say here is where our style conventions are and they live inside this external documentation file. That way, my agent only gets sent this small amount of code in my context, which keeps my context small, but also if they need more information on how to style properly, they can just open up and download this file and read it and add in all of that extra context.

06:13So, the important thing is keep your agents MD small and have it link out to other sources. I'll be showing you how to do this in just a second. I just want to show you how these bad examples are doing it so then we can realize what a good example looks like.

06:22Another example here is this Airflow one. You can see this is 537 lines. I mean, this is just a massive list of information. It's a huge overflow of information. I actually ran it through a tokenizer, and you can see that this is actually 8,600 tokens just for the agents MD.

06:38And yes, agents nowadays have like million token context window. But we all know working with agents, as soon as you get to like a 100 or maybe even 200,000 tokens, the quality of your output becomes drastically smaller. So, you're taking up if you only have 100,000 tokens, almost 10% of your entire token context just in this agent's MD file.

06:56that is really hurting the agent more than it's actually helping it. VS Code, it looks at first like it's a little better because hey, this is a really small file, but really all this does is just link to one other file that's completely massive and overbloated with tons of information and no external links.

07:13So, it's the same problem as all the other ones. They just link to a separate file for it, which again makes it quite confusing to work with and overbloat the information inside of here. So, now let me actually show you how to fix this in a essentially mock application.

07:21You can

How to write good AGENTS.md files

07:28see here I have an agents MD file inside this application. This application contains all my code for like API requests, web requests. It's like a full stack application with a backend and a front end. And as you can see, this agents MD, while not as large as some of the other ones, is still quite overwhelming in all the information it contains.

07:44And it contains everything from how to do testing, various different web related conventions, API conventions, CSS, React, TypeScript, and so much more. It includes all this information, but I guarantee you not every request I make to my AI is going to deal with TypeScript or is going to deal with testing or CSS or even deal with the backend at all.

07:59So, what I want to do is I want to split this out into smaller pieces that essentially it can reference if it needs that full information. So, the easiest way to do this is when you start to notice, hey, I'm creating a section in my agents MD for specific work cases.

08:08What you do is just go into your documentation. Let's say that this is going to be for CSS. So, we can just create a file called CSS conventions. Call that a markdown file. And now what we can do is we can take all that code inside of our agents MD for CSS conventions.

08:22We can essentially remove all of that code. Come into here. We can just call this CSS conventions and paste down that code. Obviously you can add more, do more formatting, whatever it is you want to do. But now I have that file here. And in my agents MD, I can just say reference and then I can paste a link to this whatever it is.

08:46I'll just copy the path for this. There we go. for detailed styling conventions. There we go. That's all I need to do. And I would do that for every single section. So my API conventions, I would do the same thing to I would do the same thing for React, TypeScript, repository layout, and so on.

08:58All this information I would essentially condense down just to references for the places that it needs to go to find that information. That alone will save me a ton of space in my agents MD because now it'll only reference the things it actually needs.

09:14And my overall agents MD is quite small. Another thing I noticed in almost every single agent's MD is they usually have a section like getting started or setting up. And this essentially tells it how to actually start the project and run the project and get set it up for the first time like installing dependencies and so on.

09:23This is usually something you don't really need inside of your agents MD. For the most part, you're not setting up a project for a new developer. That's not what the agents MD is for. It's essentially telling a developer that already has the project set up how they would write code and where the different places need to be.

09:41So, instead of having a getting started section like this, instead I would really just keep my agents ID super simple. I would have a one or two sentence description explaining what the project is. This helps the agent know what it's trying to do.

09:49Then maybe if you have some non-standard things like if you're using PNPM instead of npm, just say PNPM is the package manager, something like that so that it knows what the package manager is. And then if you have different weird scripts inside of here that are non-standard, for example, these are all relatively standard scripts.

10:12But if I had a non-standard script, I would throw that into this section as well, saying like the scripts that are available for various different things. I would maybe put that in here if they're like globally relevant to every single thing that you're doing.

10:25Other than that, this doesn't really need to include a lot of information beyond just, hey, here's what my project is. Here's my few things that are very specific to my project. And now I'm going to give you references to all the more specific things I need to do inside my project.

10:34So again, this would just be a reference instead of a long list like this. Now another thing that is really common inside of these different files is going to be how you actually commit and deliver and various different things. So for example here, delivery update documentation summarize implementation tests and any follow-up work.

10:49This is kind of telling the AI, hey, when you implement a feature, I want you to make sure you run linting. I want you to make sure you run prettier. I want you to make sure you do this and this and this. all these different like checks and things that you want to have happen.

10:58Again, that should be something you put inside of its own documentation file. Make it whatever you want it to be. Like for example, creating PRs or submitting PRs. And then in your main agents MD, you would create a section that says, for example, when submitting a PR, reference this documentation.

11:14That way, if you're not doing a PR or whatever, you don't need to reference that documentation. It keeps this so much slimmer and easier to work with. So, let me just do that real quick for this file, and I'll show you what it looks like. I just finished up doing that.

11:23And as you can see, my agents MD is much simpler. I have my brief description. I tell it what package manager I'm using. I have a small section on the general layout of my entire application. And then all of these are just links to files telling it, hey, here's where you go to do something for the API, the web, testing, delivery, and so on.

11:40As you can see, in each one of these, it's just a list of information. And you can even link between these. For example, in my TypeScript one, I'm actually linking to my testing documentation. So, you can have different nesting and linking between these various different sections of your application.

11:48That's perfectly okay. But now my agents empty is super small and concise. And whenever the agent needs to do React code, it can just look at the React specific stuff. That way it's not getting poisoning in its context. For example, if I had my React code inside here and I had Angular things inside here and spelt and so on cuz my application had all of those, it would read them all at once.

12:10But now in this case, it's only going to read the actual framework it's specifically using instead of reading everything all at once. Another thing that I want to caution you with is when you put the layout of your code inside of your agents MD.

12:18This is fine if you're doing general layout like this where it's very high level, but as soon as you start getting really granular such as pointing to individual files and so on, this can become a very slippery slope because file names and locations change very frequently and that can mean your agent's MD is out of date, which can confuse your agent and give you really bad output.

12:36So, if you're planning on putting like a layout or folder structure in here, I would recommend keeping it very high level such as this and not going down to a super granular level cuz that again can become very easily out of date. Also, another incredibly important thing when you're coming up with your agents MD, don't just tell the agent, hey, read my repository and create an agent's MD.

12:59That is a terrible idea. It's going to give you really verose and difficult to use actual agent MD files. Instead, I highly recommend starting with something simple, just a couple sentence description. Maybe tell it what package manager you're using.

13:06maybe a repository layout. And if you've already started to create some conventions, put those inside of here as well. But if you don't have any conventions or anything, you're just starting from scratch, I would recommend just doing that. Just the very simple couple sentences and that's it.

13:22And then as soon as you notice your agent doing something you don't like, that's when you want to start adding conventions for that. For example, if they're doing weird TypeScript things, create a TypeScript conventions file and start listing those things inside of this file based on your findings from the actual AI output.

13:29I would recommend mostly only changing your agents MD when you notice the agent doing something wrong as opposed to trying to preemptively fix them from doing something. You don't know if they're going to make a mistake or not. This is great for essentially you notice a problem, make note of what that problem is in the proper location and now in the future they can hopefully reference that and not make that same mistake.

13:51Now, I briefly mentioned at the beginning how for example in Claude you need a Claude MD and agents MD for everything else and maybe a Gemini MD. So let's say for

Dealing with CLAUDE.md

14:00example I want to have a Claude MD in here for someone that's working with Claude code. A lot of people when they do this, they just essentially say reference and then they put a link to whatever this file is. So we'll just copy the path to this real quick.

14:09And they'll just say reference whatever that file name is. Just like that. And that's all they're going to put inside of here. Now this is fine, I guess, for the most part. But this is really not ideal. and instead you should have your cloud MD file be identical to your agents MD file because it's very common especially if you're using agents to add things to your agents MD or cloud MD that it'll start adding things into the cloud MD or it'll add things just into the agents MD or just in the Gemini MD.

14:34So now you have a difference where in your cloud you have this additional code that's not in the agents MD. This is where I would recommend setting up a symbolic link between these two files and that'll essentially solve this for you. So to do this on Linux or Mac is really simple.

14:50You just say ln- s. Then you put the name of the file you want to have as the original. So in our case, agents.md. And then the name of the file you want to have that link to. And that'll create a symbolic link. Now I'm on Windows, so this is not going to quite work as I expect it to.

14:58So instead, what I want to do on Windows, I'm going to open up a brand new Windows terminal here. So we're just going to open up a command prompt terminal. And inside of this command prompt terminal, I need to type in mklink-d. And then I need to reverse the order.

15:18So, first I'm going to put claude.m MD and then agents.md. And that's going to essentially do the same thing, but instead of dashd, this needs to be slashd. Just like that. It looks like I don't have permission cuz this is not a administrator panel.

15:26So, I'm just going to open it up and as administrator run the exact same command. And now it's going to create that symbolic link for me. And now this file, I can't actually open it in here because it's just directly linking to whatever my agents MD file is.

15:34So, anytime my cla MD file or my agents MD file, sorry, changes, it's going to be referenced in my Claude MD file. So depending on your operating system, you'll need to create that symbolic link for yourself. Now, this

Nested AGENTS.md files

15:51isn't even all that you can do with the agents MD, though. One thing that's actually really common, especially in larger scale applications, is you're going to have almost a mono repo. In our case, we have this apps web section and this apps API.

15:59And if I open up our apps, you can see we have our API, which is our back end, and we have our web, which is our front end. It's very common to have different packages or different repositories all inside of one mono repo. And you will have one root agents MD to help you manage a lot of those things.

16:16for example, where do different pieces of code go? But it doesn't really make sense to have that same agents MD have code for both the web as well as the API. Especially if like the web is TypeScript only and the back end is Python only. You don't want to have all of that information in one agents MD.

16:31Instead, it's actually best to split it between multiple agents MDs. And this is really great because I can go into any folder that's nested in my project and I can create a brand new agents.m MD file. And now these two files are both going to be loaded at the same time.

16:42It's going to first load this outer level agents MD and then it's going to load the inner level agents MD for that specific package or folder that I'm inside of. So this is a great way to add specific instructions for different parts of your application especially in a monor repo.

16:57For example, in here I may want all my web related code. So all this documentation for web conventions, this is actually maybe something I want to put inside my agents MD directly in the web folder cuz that's the only place it makes sense. And I don't even need this convention file here.

17:05So we can get rid of that. And in my agents MD, all that web conventions I can completely remove. I can also do the exact same thing for my API conventions as well. So for example, I'll take all these API conventions and I'll put them in their own agents.md file just like this.

17:22And of course, I would maybe want to put more information like a one-s sentence description of what's going on inside this particular project as well. But this is just a general idea showing you how that works. And again, that cleans up my code drastically because now my main agents MD only cares about the general stuff for my entire project.

17:37And now the more specific things for like what do I do in a web scenario or what do I do in an API scenario are a little bit more nested inside of those specific folders. So it'll only load that when you're working inside of those specific mono repos.

17:53Now the important thing to understand is that if you open up your agent for example in our root folder like we are right now. If I were to open up an agent right here it's only going to load my root agents MD because it's based on what folder you open up the agent inside of.

18:08While if I were to go ahead and I CD into that appi folder, make sure it's apps API. Now that I'm in that API folder, it's going to load both agents MD for the API as well as the one for my root application because it's always looking from what folder you run the AI inside of and not necessarily where it's making changes.

18:24So make sure you keep that in mind. This is really only most useful if you're working in like a monor repo structure. We have lots of projects inside of one single repository. Now, if we take a look at the original example, you can see we've covered agents MD and cloud MD.

18:33And we've kind of covered docs to a degree because docs is just a location in your application. Doesn't have to be anything special where you just put markdown files describing your documentation.

Where Skills fit in

18:49Essentially, it's documentation that's used mostly by your AI, but can also be read by humans as well. So, that's essentially what your docs is. We obviously can understand what a prompt is. I mentioned I'm not covering the codebase. So really the next main thing I want to cover is skills because skills are the next unlock for configuring your agent to make it work like you want because you may notice that as you're building things you use lots of different stuff.

19:06You have authentication, you have databases, you have TypeScript code, you have React code, you have all these different pieces going on inside your application and if you want to fit all of that inside of your agents MD and your documentation, it can get quite bloated very quickly.

19:22And again, it's a little bit difficult for the AI to know exactly what it needs because it just has so much information, especially if you're trying to configure it to do very specific things. This is where skills come in. Skills are essentially documentation you can inject into your AI when it tries to do specific things in your application.

19:37I actually have a full massive crash course covering every single aspect of skills that you need to know. I'll link it in the cards and description for you if you want to deep dive into this subject. But I first want to just cover the basics of what they are and how they work.

19:44So, let's install a skill. The way you install skills generally is going to be running npx skills ad followed by the link to whatever the skill you want is. And if you want to be able to find skills, you can find them by going to the skills.sh website.

20:01This is going to have a bunch of skills that you can search for. This is the easiest way to find the various different skills that you want to use. Whenever we add a skill into our application, it's going to prompt us on a few different things.

20:06This part is not really that important. Let's just make sure it goes through and installs the package. It does all that. It'll say that it's cloning out that information. And here you can specify what agents you want it to work with. In our case, we're just going to hit enter.

20:17So, it's the default. And we say that we want to install this in our project. That's the most important part. That's going to put the code directly in your project. That's all that we need to do. We can proceed with installation. And now, we actually have that skill inside of our application.

20:27And I believe since I was in that API folder, it's actually going to show up in that API folder. Here we go. We can see we have that skill. And inside of every skill, there's a skill.md file. And this essentially tells the agent what this skill is.

20:35For example, it tells us the name of the skill and a description for the skill. And every single time you make a request to your AI, I mentioned how it sends along the system prompt, your agents.md file, it sends along your prompt, and it also sends along all the information for the skills at the top here.

20:54So the name and the description of every single skill. So if you have 100 skills downloaded, it's going to be polluting your context with all of this information cuz the AI needs to see all that to know which one to choose. Then as your code is working, your AI, it can read the descriptions and name of these and say, "Hey, you know what?

21:11I need to create algorithmic art and this seems to be the perfect skill for that." So, it'll load all of the markdown inside of here, read it, and use this to do very specific tasks. Now, this is, in my opinion, not the best example of a skill because it's massive.

21:20It should be broken out and made a little bit easier to read. And again, if you want to deep dive into how to write your own performance skills and make them really useful. I have a full video covering that in the cards and description down below.

21:28But for this particular use case, what you need to understand is skills are good for oneoff tasks. If you have something that's specific you want your application to do in a similar way every single time, create a skill telling it how that thing works and how to do it and it'll load that skill only when it's needed.

21:44So agents MD is great for more general information that applies to almost every single thing that your application is going to do. For example, how do you write TypeScript code in our convention? What is the overall folder structure of our application?

21:52But maybe things that are a little bit more specific and one-off. For example, how do you handle specific authentication in our application? That's a good candidate for a skill because something like authentication, you don't often touch that.

22:08So creating a skill explaining how that works and how to use it is great because now when the AI needs to do authentication, it loads that specific skill, but it leaves your agents MD completely empty because there's nothing about off inside of here at all.

22:16So what you want to do is you want to think, is it general? If so, put it inside your agents MD or more likely inside of your documentation which is referenced in your agents MD. And if it's specific to a single use case, create a skill for that that the agent can then load when it wants to do that specific thing inside of your application.

22:33By combining together this agents MD that's broken into documentation, as well as skills that are only used in specific scenarios where they're needed, you're able to fine-tune your AI by giving it a very small amount of information, but it can use that information to access a vast web of essentially information through your documentation and skills.

22:50Another great thing about skills is you can actually manually invoke a skill. So, for example, if you want to use a skill in most different types of editors. For example, if I just open up PI here, you can just type in slash skill and that'll give you access to all the different skills that you want.

23:07For example, I could access that algorithmic art skill and I can type along a prompt to go with it. So, I can manually invoke these skills or let the agent automatically invoke them itself. That's yet another reason why skills are really useful for an agent for specific tasks because you can essentially tell the agent use this skill while doing this off related task and that skill explains all the off information inside of it.

23:32Now, most people know how to download skills, but very few people know how to write their own skills, which is where the superpower of skills comes in. So, I highly recommend checking out this video right over here. That's going to cover everything you need to know about how to not only download skills other people created, but also how to create your own skills for your specific use cases so you can superpower your AI.

23:47With that said, thank you very much for watching and have a good