How to Stay Great When AI Is Good Enough | Matt Beane, UC Santa BarBara

Intro

00:002025 was the year of try everything with AI, get results with AI, do your very best, experiment, learn, licenses for everyone. 2026 is the year that the board [music] of directors is coming to the CTO and saying, "Where is our return on our investment?"

00:16Jensen Hang from Nvidia is very famously on interviews saying, "If I'm paying $500,000 for an engineer, they need to be burning at least $250,000 worth of tokens." You can burn many many many tokens very inefficiently. All AI can do most of the time is create B+ content.

00:34It will just lots of free B+ content and you will forget what an A+ looks like. A wise [music] leader in an organization will reward people with cash or promotion or visible recognition for stopping a B+ idea. I'm Matt Bean. I'm an associate professor at UC Santa Barbara in the technology management department and also CEO and co-founder of Skillbench, a startup in the AI enablement space.

Don't Let AI Dumb You Down

01:17I wrote an article about this called don't let AI dumb you down and I put that on my Substack. That was 2 and a half years ago. This immediately I saw this technology I went this is a major risk. I can produce many many lots of lines of code.

01:29I can produce documents for free or cheap. Should I have done

The B+ Trap

01:33that? Some experts uh say that the most valuable function of expertise is to say we will not do that thing. That is a B+ idea. That is not an A+ idea. We will skip the B+ ideas. unhealthy organizations, unhealthy uh you know people who create problems for themselves will see lots of ideas and go do them and it turns out they produce lots of B+ ideas.

01:55That's not too good. You have to have restraint. You have to only do the maximum quality thing. Um AI will make that harder. So then even when you're in a task, you produce some content and some output. In general, it could be good, but there are quality problems inside.

02:12If you haven't made yourself learn to think, write, and do whatever this expert process is you're trying to do very well, you will miss these quality problems. [music] And then you slowly and subtly prevent yourself from learning and you might even lose

The Trillion-Dollar Deskilling

02:31skill. I said in the book that this is a trillion dollar problem and I think it's really true. It is actually true that if we don't take active steps to protect ourselves from this, we will deskkill ourselves, other people, the next generation, [music] that will harm the economy, not right now, but a few years from now.

02:48The

Fighters vs. Coasters

02:52difference between uh those who care and are trying to protect their learning and those who are just pushing through and using AI, one is to your disposition. Are you just driven to make sure that you can do your own independent work and get your own independent results?

03:07This is some sometimes just something about a given individual that makes them fight and protect their learning. But a lot of it also has to do with incentives and the social structure you're embedded in. If the nature of my job is to produce more lines of code as a software engineer, I'm focused a lot on software engineering today.

03:27Or burn lots of tokens. This is another one that is very

Burning Tokens, Running in Circles

03:31common now. Jensen Huang from Nvidia is very famously on interviews saying, "If I'm paying $500,000 for an engineer, they need to be burning at least $250,000 worth of tokens." You can [music] burn many, many, many tokens very inefficiently.

03:45It's like saying, "I burned a lot of uh calories today." That this could be okay, but did you just run in circles? The immediate first question is, is that a B+ idea or an A minus idea? You have to set some threshold. A

Reward the B+ Killers

04:01wise leader in an organization will reward people with cash or promotion or visible recognition for stopping a B+ idea from making it forward. You have to do more to create incentives for people to insist on superb ideas only. [music] And then someone who does pick an A+ idea will win.

04:25and you'll go, "Wait, I didn't know you were allowed to do this." And [music] it's like, "Yeah, this is how innovation works. You have to be very patient. You have to be persistent. You have to focus on, no, that's not good enough."

What Shadow Learners Reveal About How We Build Skill

04:40In 2018, I published the first paper out of my dissertation, which was on robotic surgery, uh, which generated this idea of the shadow learning process. Shadow learning is building skill through nonapproved means, inappropriate means. Why would people do this?

04:55The reason is that in current circumstances in surgery, but I have since checked this in over 35 occupations, when a new technology arrives that disrupts the way that you do work, it also disrupts the way that you're supposed to learn on the job.

05:10What most of our skill comes from doing the work, and most of that comes with as we interact with experts on the job. The new technology allows the expert to do more work independently, which means that there's less room for the novice to participate.

05:23So you show up to the work, you hope to learn and build skill just by doing, and the expert is on Zoom, never comes to work. I don't get to interact with them. What am I learning after a year, 2 years? [music] Not as much. Most people will continue to try to learn the old-fashioned way.

05:42you know I will try to find ways to uh get involved in projects or something like this [music] um and they will struggle but a very rare few uh set of individuals will [music] instead of trying to participate normally they will find norm and rulebending culturally inappropriate ways to build skill.

06:03So for example in robotic surgery medical residents who were effective at building skill given this barrier to participation they uh found ways to operate on patients without the senior surgeon in the room. It's not illegal but that is not appropriate for a junior resident to be operating on a patient alone.

06:23But this is what the residents who were very good at building skill quickly did. They also for instance spent a lot of time on YouTube watching recorded video of surgery like a hundred times as much as a normal resident. So shadow learning is I can't build skill the normal way.

06:40So now I have to invent deviant or rulebreaking bending ways to build my skill. I've found this in every other occupation I've looked at. These are not practices we should copy. Obviously it's not appropriate. However, these people are fighting to protect some things in their work experience that will allow them to build skill.

The Skill Code - Three C's

07:06Figuring out what that is, what are they fighting to protect was the purpose of writing my book. The first third of that book, I looked across all the shadow learners that I found across many contexts to figure out what do they have in common.

07:19In fact, none of them explicitly identified these characteristics like I have a plan. I'm going to protect these things in my work. But if you look at their behavior, you can see what they were fighting to protect. And that's the first third of my book.

07:33Challenge, complexity, and connection. The three C's in the skill code. So the point is the shadow learner is a critical source to go to to figure out what are they fighting to protect. What can we do to keep working conditions [music] healthy for productivity but also for skill development?

07:52Not copy them but they are a diagnostic in they they give you critical diagnostic information. So

Challenge - Learn at the Edge

08:00challenge is the one that is perhaps most intuitive for folks which is you need to be close to but not at the edge of your capability to learn to build skill while you're working. So it must be very difficult require a lot of focus. You're not happy and relaxed.

08:17It's very intense experience somewhat stressful and you're not performing at your best a little bit less than your best because you're straining. [music] It's it's a challenge. You must have a healthy amount of this challenge in order to really build skill wealth.

08:30Another though is that an expert is there to help you manage the frustration associated [music] with that challenge. It's natural when you try to do something that is at the edge of your capability that you will fail in small ways. This is required.

08:41When you fail, you get frustrated. If an expert is present and doing a good job, they can help you put this failure into context [music] to say, "Yeah, you tried to do this and you didn't quite succeed, but you're now capable of attempting this task.

08:58Last week, you couldn't even attempt." So, they helped you sort of understand that your frustration is natural and that in fact, you have already progressed quite a bit. The expert [music] has an important role to play in helping you process the frustration of challenge enough so that you can continue to push yourself

Complexity - See the Whole System

09:16further. The next of those is complexity, which is about how you digest and engage with the broader work experience that you're embedded in, not just the focal task. So the focal task for a surgeon might be making a suture well and then tying a knot.

09:30These are very focal skills that I have to get very good at. But a good surgeon that's learning well is also engaging with the nurse, the scrub tech, the supply in the hospital, the finances for the hospital, the computer and the IT system.

09:44They're trying to understand the entire system of work that they're embedded in. And they give themselves time to reflect on the whole system so that they can process and understand it. Engaging with complexity but makes you better able to handle surprise and discover new ideas because I'm more attuned to the system.

10:05The individual, you must always preserve space and time to reflect on the circumstance and look left, look right, try to understand. You yourself can do a great deal to be curious and try to engage with the broader system. At the same time, performance pressures in jobs are very high and [music] it's very hard for individuals to devote the time and effort to do this kind of engagement and leaders [music] in organizations can do a lot to enable this kind of success.

10:30So for instance, you can have job rotation program. I've seen two warehouses where in warehouse 1, [music] people just go in, get on the line, and do their job. And they get paid the same wage as someone over here who um will learn more in uh warehouse B because they come in and for 2 to 3 days they work on the line, they put their item in the bag and then they are rotated to a different part of the line.

10:55And [music] these rotations give you an exposure to different parts of the experience. same wage, same job, same job title, but that person is going to be more adaptive. [music] Interestingly, the leaders in those organizations are not doing it to try to develop employees.

11:11When I ask them, it's very fascinating. They're doing it to create resiliency in the line. What matters to them is that the building is an effective processing unit. That means it needs to be able to handle surprise. It needs to be able to notice problems and catch quality issues.

11:28And then the

Connection - Trust Is a Skill

11:30third C which is connection which is a bond of trust between human beings. trust and respect actually I should say we don't often think of bonds of trust and respect as critical for learning except actually if you reflect on your own work experience anybody and you think about when did I learn the most and why often it's a who there is a person the person who trusted you who gave you an opportunity who gave you some healthy feedback that you wanted to earn their trust and respect by getting better I can speak for myself this was part of my career for sure this kind of

12:04bond of trust and respect for the novice. It provides them with motivation to do better because you it's just important to you intrinsically. This is how humans are wired. We want to earn the trust and respect of mentors, senior people. And it goes for the other direction as well.

12:17Senior people, it's meaningful to [music] realize that you can help a junior person develop and give them challenges to help them do this. You want to earn their trust and respect in response. But for the junior person, it also means that that senior person, if you do this, will give you your next opportunity.

12:33They'll help you. So it's a functional thing as well. It's not just relational. Challenge, complexity, and connection were consistently evident across all of the shadow learning that I looked at. All these people who were breaking and bending rules, they were finding ways to protect challenge, complexity, and connection.

12:52Unconventional ways, inappropriate ways, but they were grabbing for those three things.

What Leaders Must Do Now

13:02What's the leader's role in protecting healthy good quality target selection?

Learn in Public

13:10Individual leadership action [music] is and has always been important. It will remain important. You have to lead by example and get primary data, personal experience with how people are doing the job in the organization. [music] most effective leaders now that I see spend personal significant time with the most advanced AI building things and showing how quickly [music] they can do that in a way that's valuable to the organization and including displaying failure and waste when they use reporting out to the organization as the leader or a leader of a function and

13:44saying you know I tried to do this thing with AI and it was very bad I decided it's a bad idea but still revealing this to people shows no one knows how to use this technology. Now, so modeling this is is very important. The

Hire Juniors, Learn Both Ways - Inverted Apprenticeship

13:58general trend uh that seems to have shown up in the economy in the last year or year and a half is that firms have slowed down hiring at the junior level and tried to retain and grow talent at the more senior expertise level. It is very shortsighted.

14:13You need to be much more assertive about hiring junior people who are AI native and can do astounding things with AI even if they're professionally less experienced or have no experience and set up learning dynamics that are birectional so that the senior person can learn from the junior person and the junior person can learn from the senior person.

14:33I called this in my book and in my research inverted apprenticeship especially when new technologies arise. You [music] need a birectional learning. New employees have fresh perspective on the organization and can help improve things. So [music] healthy organizations are willing to take a shortrun uh hit to the productivity in order to ensure that they're ready for tomorrow.

When AI Beats Us at Everything

14:58It's very rare in human history that we get a general purpose technology [music] that really changes the way we do things. It is going to be messy in the beginning. No one will get it right. We must all learn from each other. And so it's important to have have high standards, but also be forgiving with yourself and others and understand that everybody's going to be wasteful and make mistakes.

15:18I think it's important to be open to the possibility that AI in between 3 and [music] 40 years will be better at everything than humans are. Literally everything, all tasks, including empathy, including judgment, including creativity. This is not a crazy view to hold anymore.

15:38It is not obvious to me that our governmental, educational, institutional systems will adapt fast enough, even given a 30-year timeline, even given 50. And that means change will be more disruptive and difficult. We can either make AI part of the solution and [music] make for a better future and a better path to the future that is less painful, more beneficial for more people or change will come in more [music] painful ways.

16:06I want to live in the future where we are grateful that AI has arrived. That [music] we can look around and see that everybody is better off, not just a few, because of AI. And I think we must act immediately and together to create that kind of future.