How to Get Your Org to Adopt Coding Agents (Without Shipping Garbage) — Eyal Blum, Figma

The three acts of AI adoption

00:12Good afternoon. My name is Ayal Blam. I am a software engineer at Figma. And in my talk today, we're going to talk about how we've adopted or are adopting soft agent into our workflow at Figma while maintaining a high quality for our code base.

00:32So as you may know, Figma is the browser based editor where design and engineering and now AI agent collaborate together to ship code. H this uh Figma has pivoted very strongly from being a traditional tool to an AI first tool. But in this talk, I'm not going to talk about our product.

00:52I'm going to talk more about our internal organization and how our engineering or has been adopting AI agents. And what we've found internally is both organizations, companies and individual there's kind of a three-act process of AI adoption.

01:13You start with picking up something whether it was a lot of the people in this room who have been do using it are very appeal and have been using AI for a while and they picked up something and got some simple things to work very well 10x faster.

01:32Then you start applying those same practices to bigger problems and AI fails pretty badly at that. gives you bad stuff, lots of bugs, and the trust that you build breaks down. And then from that point, you start building the real skill, which is learning how to use AI correctly and put the right guard rails and the right prompting and the right contact and all the stuff that we've been talking all day about here in all the talks in order to actually build a real scale.

02:04And one thing that um is happening internally is we we adopted whether teams or individuals the adoption is uneven. We have teams that are very AI forward and have already transformed their entire workflows and we have teams that are um still experimenting in the earlier act and or have lost confidence and they all need to work together in order to ship our product.

02:26Um so they need to coexist um in the organization and we need to find

Uneven adoption, and teams that must coexist

02:35way to support them and while bringing on everybody along for the journey and getting everybody to the third act of the story. Um aside from that main friction point we have also noticed other friction points that happened um as we adopt AI.

02:52Um, one thing that we've heard a lot from developers and managers have been noticing is a reduced developer agency causes um, engineers to lose some of the job satisfaction. So if a lot of people used to take a lot of pride and enjoyment in writing code and getting into the flow and a lot of people feel like that's been lost or they're losing a lot of that element and getting into more of a prompt cycle where they just wait on output from AI and then speak to the like not as much fun as they used to have and they get burned out.

03:25Um we've noticed another interesting thing is actually our best engineer the one that hold all their context in their brain and they end up getting out of the burden and what ends up happening is they they know where all the pitfalls are. They are like holding together with with like their mental duct tape all the

Why the best engineers adopt last

03:42places that agents are not working well and they're preventing all the really bad stuff from coming in or all they they have all the institutional contact that have never written down in their head and they get so much burden and and become bottlenecks and gets really frustrated.

03:58So they actually end up being slowest to adopt because they see all the problem firsthand. That's another big big issue that we've seen. And and this one I'm sure everybody can resonate or in sorry I'm sure everybody here will resonate um that all of a sudden all the design docs and all the Slack messages and all the emails have gotten three or four times as long and we've gotten two or three times as many emails and they say basically as much as they did before.

04:24So the communication has gotten quite inefficient and some of the markers of like what is high quality and important things versus not so much high quality um has become challenging to navigate. Um so I'm going to spend the next few minutes talking about some of the lessons that we've learned and how we've been trying to apply this.

04:42This is a

Investing in verification first

04:51journey. We have not come out to the other end but we've seen some really interesting progress along a lot of these lines. Um I think this a lot of the speakers here have touched upon this but investing in verification is probably the highest value thing we can do in our codebase.

05:07Um anytime that we can lift left shift anything in our workflow from a human needing to do it to an agent being able to verify it. So for example when playright MCP came out instead of having humans navigate the code now the agent can explore the code that was a big win unlock for productivity in a lot of our teams that's really that always a a big win for us.

05:36The other thing is um even better if when you find something that the agent has found to be useful take the time to take that and encode into a deterministic flow and deterministic flow that can be easily repeated. It's saved on tok and save on time for the and then it also you also know that you're using the the LLM when

Encoding what works into deterministic flows

06:01it needs to reason. But when you have something that is already known and basically can be encoded into a test, spending that time always always pays dividend. And another tip, if you tell your skills or your agent to write the code that you're writing and like at the red to green to red to green at the TDD style, it almost always gives you better results because you set a goal, then you tell the agent to strive toward that goal.

06:28It will almost always give you better result than writing the code and then writing the test afterwards because then it will fit the test to the code rather than fit the code to pass the verification criteria. And this is the testing pyramid kind the the classic testing pyramid from the previous just when you think about the testing themselves which you had the end to end test when the integration test and the unit test.

06:50This is very similar. Move as much as you can down to the deterministic analysis where that's linting the compiler and the unit test

A testing pyramid for agent review

07:04themselves whatever that can be covered easily. You can have engine agent do reviews on it based on on criteria. So um architectural standard that have been easily encoded into the codebase you can move into the agent and then only at the very top you need to have some sort of human review which is usually around the functionality and this is the right thing to build that like only leave the human to do what the humans need to actually be involved in.

07:35And another really important thing is the planning versus prompting. This is really tied into the giving agency back to developers and finding a replacement to the craft of writing code. Spending a lot of time writing the plan and then sending it off to the edges basically as a as an implementation that can be done automatically is something that we find to really kind of reintroduce the joy of of building back into the process.

08:06And so it's not uncommon to spend a week writing a very detailed plan, making all the decision,

Planning instead of prompting

08:14flushing it out, iterating, sending it out to teammates to review, and then only when it's ready and you've flushed out all the decision, you can send it to the agent. The agent will um send it back to you when it's implemented. That that has been really successful also in accelerating and also really restoring some of the joy into the development process.

08:38H. So what makes a good plan? Um really important to start with the why at the top. It really helps preventing agent drift. If you have like a bold big section of kind of like when you write a design doc, you want to have the executive summary.

08:47Put that in there for the agent. Otherwise, they'll start drifting over time. And make sure that the agent don't go back and change that because they feel like it. And so we start with the why. Make sure that your plan can be broken down into small parts that can each be verified independently.

09:08And my personal way of knowing what is a good size is would I want to review that the PR that will correspond to that part. If it's going to be too big for me to want to review in one sitting kind like the test is I'm going to need to get a cup of coffee before I read this.

What makes a plan an agent can follow

09:23That means it's too big and I'm going to want to have it broken down into pieces. And then I make sure that each part can be validated independently because what I don't want to have is have five stages and then the first one is written but not validated and then everything else is is built on top of all the assumption.

09:36So having kind of a validation gate or an exception criteria for each phase really helps um make the plan resilient to drift and and and there's all kind of technique of how to manage the context and doing a software factory on top of that.

09:53But once you have the plan, you can use whatever loop you want or whatever workflow you want in order to implement it. And this is a screenshot that I randomly picked of a plan. But this is what I usually look for the executive summary at the top.

10:07The phases break it down and then each one of them I will go into lots of details so they can just fit it into a sub agent and the sub aent can independently work on that and not have to worry about it and that that there are other workflows that would work or other structures to the plan.

10:23We find that part of the things that great about AI workflows is that everybody can set up the thing that works best for them. Oh, no thank you. Everybody can very easily set up the workflow that work exactly for them for them. So there's the return is trying to centralize everybody on one thing.

10:43But as long as it works for their flow and other people can iterate with them, I find that it generally works very well. And this is just an example kind of a brag of like this is could be a result from a plan and there are probably 20 PRs here.

11:07Some of them would be maybe 10 lines and some of them would be 100 lines or probably nothing bigger than that. And that allows us to this is in the prei world. This plan I probably worked on it for a week. I aligned with with three other teams for another week on that and then I just sent it to an agent to implement overnight and it came back.

11:24This is probably from two plants, not one. But it's it's basically six weeks of of coding work just that only took one week. So that's where I get a 5x speed up if I include the review cycle at the end that you always have to remember.

Handing skeptics the roadmap

11:45Um moving on from planning and back to the issue that we had with the skeptics and the people who are burdened with the most work. Make sure that the you bring them in and take their feedback really seriously. They're specic because they're seeing the the where you're lacking validation where your tools fail.

12:01So their feedback is basically the road map of how to improve your agent interacting with the code base. So just make sure that to bring them in and rather than trying to figure out how to make them use AI just let's have them be in charge of the road map to make AI safe in your organization and they will come along once they see that that the improvement that they're making are actually making their life better.

12:33Um and as you can see they'll not be shy about telling you what you need to fix. This is less than an hour sitting with a bunch of people and this is the result of brainstorms. Um, another thing that's been really helpful with my team specifically and we're working to adopt it in the broader organization as well is to make sure that you have an attention aware communication in the age of AI.

12:52Human

Attention aware communication

12:59attention is a scarce resource. I think I've heard it for multiple talks and a lot of people have noted have come to the same conclusion. you can get more human attention. So where you spend your time and what you're reading is really becomes really important.

13:13And so since it's such a scarce resource, marking what was generated by AI versus what was written by a human is really helpful to know how much time you need to spend reading this and how much slop can you expect in this part of the communication and that kind of building a new culture around that cell of communication.

13:29And it really helps. Um so for example um the team team that I work with we've decided we always every PR description will start with something like that something that I wrote by hand could be very short that describe what this is what this is doing and then the AI description is going to come after that which is I will probably read it.

13:52I will probably edit it to remove some wrong things but I didn't write every line here. So they should be more suspicious and they should pay more attention to what I wrote in the top and that should override it. Things like that in Slack, in email, just like leaning into the fact that everybody knows that you're using AI to to craft your communication, but just don't be shy about it.

14:16Tell them what they should read and what you they should pay less attention to. And I remember early on maybe like earlier in this year I tried to I had some senior engineers in our org that had kind of were very much AI skeptics and I tried to reach out to them to see what was the problem, what was going on and say I tried to run an analysis on some of the PR comments that you've run and obviously I used AI to do that and then I didn't distinguish very clearly what I wrote versus what they what AI generated and they got very upset.

14:47They're like, "Why are you sending I did not expect somebody um that I respect this much to send me SL something that's clearly this sloppy." And then like I I like I apologize. I really should mark it clearly and mark my intention like this is what I wrote.

15:02This is what the wrote and I need your feedback on that because I don't have the context to know if it is sloppy or not. And that's what I'm asking you for. So lessons like that and

Sending unmarked AI output, and what it cost

15:18change the culture is just as important as the some of the engineering challenges that we've been facing. Um, another thing that's really helpful around the adoption is, um, as you progress to adoption, there's a lot of very fancy tools and a lot of very fancy workflow that we've we've been implementing.

15:36But one of the really effective thing is just letting people use AI where they're at. So, it help it really helps normalizes the use of the eyes for everyday task and it helps reduce the friction. And really one of the most powerful thing is being able to tag an agent in the Slack message with somebody and say, "Can you just do this for me and have the agents close the loop in the thread and that that's kind of thing is really powerful."

16:04And then you can go on top of that and have all this thing automated and be all kind of fancy things. But if you have a conversation with somebody who's not fully bought in and then you can tag it in a non like nonp passive aggressive way, you can tag it and say let's try it with see if the agent can get it this time and they close the loop and if it's a good experience that really helps people try it out on their own in other cases

Meeting people where they already work

16:36and our journey continues. We're still learning even though we're shipping externally. Our AI adoption and we're experimenting with with so many things all the time. Our automation story is not fully there yet. We're still find trying to figure out when we should use how we can use cloud agent effectively given all the dependencies that we have for some of our build systems.

16:58And so we are continuing to learn. It's a culture shift. It's an engineering shift. And I don't know about you, but for I've been the I've been working in the valley for the last 15 years and this is the biggest change by orders of magnitude of everything that I've seen in term culture and technology.

17:15So we're all here together and we're all figuring it out and that's that's what I want to talk to you today. Thank you.