How do you diffuse AI into the real world? — Varun Shenoy, Long Lake

Everyone has seen the demo, nothing has changed

00:12Hi everyone, I'm Verun. I'm one of the co-founders at Long Lake and I'm excited to share a little bit about what we've been up to for the last two years. It all comes back to a question all of us have asked time and time again. The models are getting better, but the real question is how do you actually deploy the AI into the real world?

00:27How do you get the models to complete economically relevant tasks? Let me start by saying everyone has seen the demo. Think of the agent automatically booking a flight, the agent automatically completing a ticket in some kind of customer service portal.

00:55Think of an agent completing a block of code ready to commit and go. The reality is we've all seen this and it feels like magic. Two years ago, any of this would have been complete science fiction. The capabilities are real. Real people, real properties, real dollars, real customers all across the US.

01:24You would expect AI to show up by now, but the reality is nothing has changed at all.

Electricity, Ford, and how long diffusion takes

01:34Here's the thing. This is totally normal. And maybe in fact, I'd argue this is what we should expect. This is true for every general purpose technology. You know, take electricity for example. Electricity was invented in the 1880s and it was first demoed at Edison's Pearl Street Station Dynamo Room over in Manhattan.

01:57This was the magic demo of its time. The reality is it took a long time for electricity to be fully adopted. Consider a Ford factory. It's not enough to just have electricity. You have to rip out the existing motors and equipment. You have to bring in the new equipment.

02:15You have to go and train everybody to use that very same equipment. Here's a picture of a Ford electrified moving assembly in 1924. These things take time. Diffusion of any technology takes a generation. And since everyone here in this room today is talking about AI, I would argue

Who Long Lake is, and why they buy the businesses

02:37AI diffusion is perhaps the single most important problem for the next 20 years. The models are going to keep getting better. The big question is how do we actually get these models to be in the real world complete real tasks uh and make people more efficient happier and provide better service.

02:57So taking a quick step step back who are we? Uh we are Long Lake. Over the last two years we've raised over $3 billion from Elad Gil General Catalyst and Alpha Wave since our founding. Here's the strange part. We we don't sell software. We actually go out and acquire and partner with real services businesses in the world.

03:10Uh we've acquired 35 businesses across HOA and property management, architecture, HR services and a lot more. To give you a little bit more flavor, we have roughly a 40 person team right now split between technology, finance, and operations.

03:26More than half our team is part of the technology team focused on uh building products, data, and deploying the core products into the field. Uh we're an eclectic group of folks. A bunch of ex-founders who've worked in the services before, ex-military folks from Palunteer, Ramp, Glean, uh and from the finance side, Blackstone, HIG, etc.

03:50Uh we we are not selling them to these companies above from the outside. We're actually deploying into these companies and figuring out how to get the technology to work. And just to show you the scale we're playing at, we announced recently our $6.3 billion take private of American Express Global Business Travel, the world's largest corporate travel platform.

04:17We own these businesses. So when the AI doesn't work, it's not their problem. We're not the vendor. It's our problem. Concretely, again, we are not the vendor. We are the operator owners. And we work very closely with our teams within the businesses to drive real outcomes.

04:37Now I want to step back and get to the concrete about the how. What are the lessons we've learned over the last two and a half years and what we've learned from deploying AI into companies we've owned. Three quick lessons. One, how we move agents from co-pilots to co-workers.

The ladder from copilot to coworker

04:57two, how we leverage realw world data within these businesses. Remember, we're seeing all of the work that's being done in these real services businesses. There's a lot of interesting problems and solutions embedded within that. And then finally, uh perhaps the most interesting and exciting is how do you actually get all of this technology to compound over time by learning loops in the enterprise?

05:15And we'll get to that at the end over here. So starting off from co-pilots to co-workers, there's a spectrum of how much autonomy you can give an agent. On the left here, you see a co-pilot. This is, you know, your simple rag chatbot from two years ago.

05:31It's very quick. You can ask a question. Maybe it's integrated with some systems. It can give you information back very, very quickly. The second step is a synchronous agent. Consider something like cloud code, codeex, cloud co-work. It's real time.

05:50There's this two-way interaction. It's a bit more sophisticated than a co-pilot. You can go let it run off for 1 to five minutes. Uh it'll call tools, maybe uses skills. Uh it's still synchronous. You still need to step in and ask a query.

Earning the right to more autonomy

06:02So the next obvious rung of the ladder is the asynchronous agent. You can come in here, still ask a query. The agent will go off into the background, do some work, and then come back. Uh and what's really interesting about asynchronous agents is that the user does not have to be the one that triggers them.

06:17You can have external triggers as well. Maybe someone completes a certain task and there is an async job queue uh that allows the async agent to pull off from and proactively offer advice to the end user. Then I'd argue the next step is a long running agent.

06:32How do you get these agents to work for hours, days, weeks, months, etc. I think this is currently a very core problem that a lot of the labs are focused on as are we. And then finally at the end, the holy grail, an AI co-orker. This is where most people start off.

06:53You want a proactive partner that gets work done just alongside you. This is what everyone wants to sell you. But what we've learned from owning the outcomes in this business is you have to earn the right to do more. It's it's not enough to jump to the coworker immediately, right?

07:10For for a bunch of reasons. one for certain tasks the models not might not quite be there yet and two you actually have to work with these companies in the field interact and iterate very very closely so that they understand that this is the beginning of AI and you can work up the rungs over time I think a really unique lens to look at this problem through is that of the jagged frontier we all know that agents are incredibly good at writing code so what does the for example synchronous agent for code generation look like?

07:47This is super simple. This is just your coding agent. Maybe it's codec cloud code just running on your desktop. It has access to a file system. You collaborate with in real time. You get instant feedback and you iterate. The next step is you know if you look at code code generation what is the async agent?

07:59This is also fairly straightforward and largely solved. You take the exact same coding agent you wrap it in a sandbox and you just let it go run. It can build, it can test, and once it's done with its work, it can provide the code in the form of a PR.

Why engineers parallelize and nobody else does

08:16One thing that's really unique about engineers is folks are incredibly good at already paralyzing their work. It's very common place to launch 10 jobs and be comfortable with the fact that job seven might finish before job three. So, engineers are incredibly good at using these async agents.

08:35Now, when we come to services, the equivalent of a synchronous agent is what we talked about a little bit earlier. It's a co-working agent. It's an agent that has deep context about your enterprise. It interacts potentially with MCPs, custom tools, custom integrations, uh, and you can chat with it synchronously just like any of these other products.

08:53I think this is the frontier here in the bottom right. What does it mean to build an asynchronous agent for the services? What does it mean to paralyze work in industries where work is traditionally done in a very very serial manner? This is where we spend a lot of time and this is what I wake up every morning really excited thinking about.

09:08You know, we've we've figured out what the async and forking mechanism for code is. You just spin up a bunch of sandboxes and do work. What does that look like for the rest of the world?

Representing knowledge work as code

09:24So, here's a couple questions we think about pretty seriously. One, you know, the models are trained on code. They want to write code. They're incredibly good at writing code. How do we leverage these coding agents for actual knowledge work?

09:33You know, rather than wait for the models to catch up on doing services knowledge work, what if we just use that code knowledge and represent knowledge work as code? Two, as I mentioned, engineers are used to paralyzing work. How do you paralyze work that's traditionally serial?

09:47You know, people clean out their inbox of one email by one email, not 10 emails at once. And finally, how do you move up the ladder here both in terms of product and user enablement? What are the right form factors? And I'd argue this varies dramatically from industry to industry.

10:05Just because you have one way of launching an async agent for code doesn't mean that same way is going to work for architecture or property management. The second point I want to cover today is leveraging real world data. We all know this. Frontier models have learned from everything humanity has written down.

10:21But the most valuable tasks are not on the internet. How do you actually close the box when you're missing receipts? How do you scope a building for construction in a blueprint, potentially collaboratively?

The valuable tasks that are not on the internet

10:35How do you coordinate vendors for fixing a broken roof? All of this knowledge lives in people's heads in 20-year-old software uh in the way that one senior person on one of these teams just knows how to do it. How do you make this information explicit and create tests that you can actually learn from?

10:48So, we've constructed a little bit of a flywheel. We get our agents to collaborate with our employees to do real work. And this allows us to generate rich traces of data and information. Tool calls, the hiccups, the paper cuts, everything that goes wrong with doing real work.

11:04This in turn allows us to build real world evals. There is a ground truth here. In the case of the roofing example, the question is did the roof get repaired? Did the books get closed? And this allows us to hill climb and build better agents which leads to more and more impact.

11:19And what's really exciting is it ratchets up. Every week our hill climbing benchmarks become a regression test. So our agents get better and better over time. Just to drive a little bit deeper here on the traces, there's three upshots of being able to collect these rich traces.

Traces, ground truth, and evals that mean something

11:44One, we get to generate amazing evals that are built and scored automatically. uh and we were able to gather both implicit and explicit feedback. Explicit feedback in the sense of thumbs ups and thumbs down. Maybe people provide a note telling us whether this response was good or not.

11:57Uh and also implicit feedback, right? Again, we have the ground truth. Maybe there's some data that the AI generated and there's a real diff between that data that the AI generated and what was ultimately submitted. That's rich information that almost no one else has.

12:14Two, we've started post training models internally on all of the data that these businesses operate on and produce. Generally speaking, this is all data that is completely out of distribution for most frontier labs. Think of the task I showed at the beginning.

12:21A lot of the models, a lot of the frontier models today just can't do these tasks yet. And we're trying to post train our own models internally to be able to do that on the rich source of data that we own. And then finally, the actual agents themselves.

12:40The real world is incredibly hairy and messy. And you want customization per company. Every company does things very differently. Customization per user. The way each user does their work is very unique. And customization per client. The way you work with every client is different.

12:59It's a services business. And you want to uphold those standards. I love this picture because it's the whole thing in a single image. Um the the way we usually talk about LLM tasks is the top panel, right? You just it's it's a slope. You got a bike, but but there's clear sight to success.

13:12The reality is most work is not like that. And and you and I both know that. Uh there are hills and ravines. Uh there's death by a thousand paper cuts. But but that's what real work looks like. That's the entire job. The exceptions are the job.

13:37That's that's the demo. That's the actual job. Now, on to the final thing I want to chat with you guys today is learning loops within the enterprise. I'd argue there's two hot trends everyone's talking about in 2026. One, it's continual learning.

13:52How do you make an agent better over time with feedback? I think there are plenty of sessions uh

Continual learning and enablement as one loop

14:00this week on how you can use continual learning, whether it's in the prompt or in the weights. and two enablement. How do you get in these enterprises and actually get them to adopt and use AI? Traditionally speaking, these two initiatives are owned by two separate teams, right?

14:13The continual learning is owned by your research team, your platform engineering team. Enablements owned by growth or deployment or customer experience. Uh usually pretty siloed, not much interaction between the two. We think these are part of the exact same loop.

14:29The agent only improves if people actually use it. And people only use the agent if it's worth adopting. So here's a little graphic of a snowball. More usage drives continual learning, which drives a better agent, which drives more usage. Again, all this to say, there's still a really big elephant in the room.

14:53How do you get the initial usage? I think a lot of people, you know, will use cloud code or or give it to their whole enterprise. expect folks to just start using it. Everyone assumes the usage just shows up. But as we all know,

The elephant: getting the first usage at all

15:09that's simply not the case. It never does, right? Getting a hundred-year-old firm to change his processes is hard. You could have the best AI co-orker on the internet on Earth and if the people if the person who's closed the books for the last 20 years continues to do things the same way, nothing changes.

15:26Nothing happens. So what can you actually do about it? What you know this this seems like incredibly hard. What what's the upshot? How do you actually get this stuff to work? Well, I think a lot about Jensen and how he dominated the market in his words with extreme hardware software codees.

15:46Designing the chips and the software together as one system. We look at this through the lens of extreme software service codees. How do you co-design our products with the people and the processes at our businesses? And I'd argue this is only possible from being within under the same roof.

16:08We need to meet the people within these companies both metaphorically, for example, bringing products to their systems so that the energy required for enablement is kept low and also physically. Get on a plane, show up, say hi, learn what people actually do.

Codesigning in person, not over Zoom

16:25You know, maybe you build a product that's natively embedded into Excel or into their ERP system, maybe their 3D design software or or maybe even their Microsoft products like Outlook, Gmail, etc. Or you show up in person, you do a lunch and learn with a bunch of folks at one of the companies.

16:34You go to their conferences and you create cotton candy and run a stand for them. You go mountain biking and ask them about all the difficulties that they have with their actual day-to-day jobs. or you show up in person, one-on-one or sometimes even two-on-one in this case, and just show them how to use the tools and learn from the feedback because this is what the rest of the world really looks like.

17:00It's not like the folks in this room or in San Francisco. It's a lot more like this. You cannot co-design software with the services business over Zoom uh or over a support ticket. You you have to be there. You have to be in person. And I'd argue this is the part that actually makes it work.

17:15In order to get AI diffusion to work, you have to touch some grass. Thank you so much. Uh, and I'll be around for the rest of the day. If there's anything I could help with, uh, my email is up there. Uh, and yeah, thank you.