How AI Changed Software Engineering in 12 Months: An Inside Look

Good morning. I'm Gergely and thank you for the introduction. I now spend most of my time writing the pragmatic

engineer hosting the podcast which is software engineering a big tech from the inside

and today I want to bring you into a journey show show you some stories some

information that might not be as readily available inside AI labs big tech and startups. In the last few months I went

inside the HQ of open AI and Tropic cursor ramp. I also talked with folks at Uber extensively at linear and other

startups. Here's some of the photos where I went especially ramp with the office guy.

And today I want to cover first what changed then what did not change

things that have broken across the industry and what come next. Let's start with what's changed and a

lot of things have changed. First thing nobody writes by code by hand anymore except for a a few of you

folks but in in general it's it's just more more and more rare to see. Of course we we still edit some code by

hand but it's crazy how much we are writing with AI and this was kind of obvious to see in January like we we all

woke up in January after the winter break and we just saw that these models starting from Opus 4.5 and GPT 5.2 were

just really really good with the harnesses and from there on people switched and in most startups

what they're tracking these numbers it's going to close to 100% AI generated code that of course engineers prompt.

Another thing that's changed working with several agents it's increasingly common.

In March I talked with Boris Cherny who told me about how he has five terminal tabs five cloth codes on his local

machine plus five to ten cloths on the web. Now this is Boris Cherny this is an tropic obviously they will be you know

cutting edge and and they will just honestly use AI sometimes maybe even for the sake of AI

but I talked with Dima Zats who is an an old colleague of mine at an Uber. He now works at Linear. And he put it really

nicely. He said, "It used to be a simple world, one mouse, one keyboard, one screen. You physically couldn't work on

more things than one. But no, we no longer have that limitation." So, he's doing five to 10 work trees. He's doing

five to 10 agents at the same time. And just the day before yesterday, I talked with Peter Mattis, co-founder and

CEO of Cockroach Labs. He's the person who built Gmail's first back end storage and a bunch of really cool storage stuff

inside of Google. And he told me the same thing. Often times he's doing things in parallel. My

cognitive overhead of five to 10 agent sessions concurrently. And he just often does that as well. So, again, so many

people, so many kind of hardcore engineers, if you will, are have kind of moved to this mode of just parallel

agents. It's pretty common. IDEs are just fading away. This was really unexpected for me to see cuz I I

love my IDE, but the trend of this heavyweight IDE usage is trending down. People are using them just not as much.

Antigravity released in November, the Windsurf kind of rebrand, if you will, was the last major VS Code fork that was

forked. And I talked with the OpenAI team just in June and they told me that in January they were considering forking

VS Code for the next version of Codex, but they were kind of torn in between, should we do it, should we not? And for

a little bit they thought it was a mistake to build a

Codex without an IDE, but they quickly realized actually they they did it at the right time and and it was a win.

JetBrains, one of the most beloved IDE brands, they're pivoting to Air, which is also more of an agentic harness, if

you will. You can check out it looks similar to Codex. Cursor, the biggest VS Code fork, has

pivoted in April to just an agentic interface. Again, looks very similar to to Codex. So, all of these tools and

when I talk with the Cursor team inside of Cursor, they told me specifically that the IDE is now just a legacy

product. They need it for enterprise, but it's not growing and it's shrinking. And I was talking with Kent Beck about

this yesterday and he texted me this. He said like, "Hey, on the on the topic of IDE, it's not that we don't need more

perspective and context for human-based decisions, it's that the context has changed. So, maybe that's why it's going

away." Everyone and then some are building their own harness and agent platform.

And by everyone I'm I'm I'm talking the startups, the big tech, the the mid-size companies.

I I put this kind of tweet half-jokingly saying, "If you're not built your AI coding harness, are you even a serious

tech company?" And here are companies that built it. Ramp, Stripe, Uber, Block, Shopify, Google, Meta, Amazon,

Dropbox, Shopify, DoorDash, Grab, WorkOS, Monzo, HubSpot, Sierra, Harvey, Bowery, so many more. Usually, they have

cool internal hacks. By the way, it's really easy to do it as well and they often do it so they can plug in their

their data sources that they might not want to plug into Codex or or whatnot. And some of them are I I are building on

top of Codex, Claude Code, Open Code, and so on. Developer work now often starts in

Slack. Uh so, inside OpenAI, they just tag Codex implement this. Codex is plugged into

all of the OpenAI systems. Anthropic, same thing with Claude, right? They launched it as a product as well.

Startups increasingly build their own Slack agent and AI bot integrations. Linear does the same with their their

Linear bot and so on. The agentic software factory. This used to be a buzzword that I used

to be like, "Agentic software what? Like, you know, like like cars? Sure, one day." But no, it's actually uh

happening. And one of the best examples is OpenAI's agentic software factory, which

a deep dive is going out in a few hours in the pragmatic engineer so you'll be able to see but

in this diagram that describes how their agentic software factory works those dash lines those dotted lines are where

agents act on typically agent input and they go back to code x to generate a pull request. A human in the end often

reviews a pull request at for critical ones for large vast radius but open AI has this really cool concept

of a perf factory run by agents which monitor production and raises PRs to improve production. And they're just

keep iterating on it. It's not fully automated and they're they'll have to go but it's happening and this is a big

change. Migration is no longer take years if you were paying attention to where it's not

as surprising but some examples Anthropic migrated famously Zig to Rust with one engineer

who wrote most of Bun by the way so this was really fast and it it probably would have taken

years and it probably just wouldn't have been done just because how when you migrate you need to maintain another

thing. Open AI as I'll describe in a deep dive I I this is new information that I I got from

them. They are migrating their API that hits all of their interfaces from Python to Rust.

They are about 90% done which was kind of what everyone says by the way so but they're saying that it took about

like four to five months to get here under production load nothing seen from the outside and they're expecting to to

finish in a couple more months and usually we know these things would have taken years so but this is a great

example cuz I I don't just want to show the kind of days which is a little bit like fairy tale stories.

Airbnb moved they discontinued their enzyme um uh UI testing library and and and went

to React testing library. They migrated in six weeks and did a really nice write up about how they did it AI assisted.

Asana did the same thing in about 2 weeks. There There's some nuance there because they they they

spent a few years before doing it. And again, these are these are practically full rewrites of

test suites. It's It's not a replacing a library. It's There's some interesting details there. Uber moved from JUnit 4

to 5 in 4 months, uh which was millions of lines of code. And again, it it it literally would have taken them just

years clog-slogging away beforehand. So, these things are just cheaper and better and easier.

Agentic generated issues are on the the rise. Fresh data from Linear shows that on

their platform, the white line is human-created issues tickets, and the blue line is agentic or MCB-created

tickets. And for the first time, more agentic-created tickets are happening on their platform than human-created ones.

That's really And you can see the sharp rise in the last few months. Um

we're al- also seeing This is about uh agentic pull request agent only pull requests are on the

rise. So, just in July and August, agentic pull requests are are spiking. Uh

Sorry, agent-only reviewed pull requests with no human reviews, which is also very interesting trend.

Uh This is data source from GitHub on how agent-generated PRs are just exploding.

In January, there were 7.7 million. The The number is not as important than the the trend. Nearly 10x increase by

August. Agent only So, this is the agentic software factory at work. And skills usage is also going more

mainstream. More people are doing it. This is data from Factory AI, uh where they're seeing about

80-something percent of their users use AI skills. So, it's just again going kind of mainstream from compared to

February when it was 30-something percent. Cost is is a big concern for a lot of

you. Like you you will know this. Uh but back in back in May, uh there was already a widespread trend about

companies worried about cost. This was around when it was made it news that Uber blew blew through its its annual

budget for AI and Claude code was was uh becoming more popular. AI was getting

more expensive. Now by September, a lot of the large companies have managed to take control

of uh their cost curve and they're cutting back cost by took per per token cost about 50% keeping cost flat after

investing in open models, smarter routing, and so on. Another interesting trend is this the

disappearance of engineering specializations. Uh

inside OpenAI talk with with with Solomon uh Chachere, a ChatGPT's head of engineering, and he said that although

specializations in engineering would he suspected it would go away, he still surprised how quickly it has happened.

Last years they were still looking for iOS and Android specialized engineers, this year they're not because

with Codex in in in their case or LLMs, they feel that anyone can do the work at

a good enough level, which is a very very interesting one. And and I've seen this a lot for front-end, mobile, I mean

most engineers have been familiar with back-end, but the same thing. A front-end engineer can do more back-end

now. Projects are increasingly being done by one or two engineers. So,

inside Anthropic I talked with Catherine Less, head of engineering at for Claude platform, and she she just

put it out so simple. She said, "On an individual project, you cannot often have more than one or two engineers

working on it. This is because these engineers are already running several agents. And so as an engineer, you're

already fighting against your agents, which those agents are stepping on each other's toes. So, there's only, you

know, maximum one other engineer you can do you can take who will also bring their agents cuz now you're looking at

you know, a lot of individuals doing it and one person projects are becoming so much more common in inside of startups

who I'm talking with by the way. Some other other other other changes uh teams are getting smaller. We're seeing

this everywhere. This kind of comes logically from the the points before. There's less

junior hiring happening across the industry. It's it's falling not dramatically, but it just keeps

going slower. We're seeing data up inside of pragmatic engineering elsewhere about intern hiring just

keeping going and lower and lower and agentic infra is finally its own discipline.

So, this was a lot of changes. I I mean we went through 16 different things. I I was just throwing things at you. I

I was trying to keep it concise, but I just so many different things are all changing and these changes I want to

stress out. These are the last 12 months. Before none of these were really true

maybe except for the junior hiring one. So, what did not change because you know, the industry is is not like we're

we're not pulling a meta where we're we're trying to disrupt completely ourselves, but it is happening. Some

things have not changed though. Number one is the teams is remained as the unit of work. Inside of Entropic, I

was pretty pleased to hear that teams are a big thing and again Caitlin told me this. She said one thing I've heard

from some people is oh, we have two humans and a bunch of agents and then she says like this is not where at. I

still have teams whose job is to own a piece of software, own on call and so on and while each of these humans is

supercharged by AI, the size and the shape is still similar. They still have two pizza teams.

Like before. Same thing by the way with OpenAI. Same thing and when you look at any successful agentic product, it's

it's with a team, right? Like people go on vacation. They they create energy with each other. They inspire each

other. They hold a high bar. They keep each other accountable. This is not going anywhere. Don't let anyone tell

you that the you know two people and a bunch of agents, that's not going to be a team. That will be a one one off

project maybe. Planning has remained for complex work. Uh

inside of Anthropic for example, they Kathleen told me that for their more complex infrastructure related products,

they planning hasn't changed much from for example like you know years back when she worked at Stripe and so on.

And building the wrong thing is still a waste even when building is faster. Planning is especially important for

infra complexity aligning teams communicating with customers and I think this is not changing.

Testing validation remains important. Uh Jared Summer on the the

we know this but this is for example Jared Summer on on the bun team who rewrote bun inside of Anthropic. He said

that the AI writes pretty much all code but we have all write the AI write pretty much all tests.

Before AI for production ready software, we spent about the same time writing tests as we did on writing code and this

has not changed. The the ratio has not changed. And he is stressing that you need to have a way to trust your code

and tests are he says are probably the best way to do so. There are of course different layers of testing kind of

formal verification is also starting to become more popular but but we need to validate this this code that we're

producing. This is an interesting one. I keep hearing about how non-engineers are now

able to ship production code. And I've been asking around the past week everyone if this is happening. No, it's

it's not. Uh they're not shipping to prod. They are building websites. They are tagging

in Slack saying oh this is a bug can you look into it and sometimes that Slack bot might create a pull request but

those people typically don't even know what a pull request is and they are not shipping production code. They are

alerting the engineering team to bugs but engineers every company every startup from Linear to Cursor to OpenAI

I've seen, you don't have non-engineers push production ready code in any shape or form. So, this is still not happening

and I'm I'm not sure if it will, by the way. Uh

and then this is interesting. Old patterns are being rediscovered again. I talked with Matt

Pocock, uh the creator of the GrilMe scale uh and a well-known educator, and he told me that after he discovered the

term tracer bullets from the Pragmatic Programmer book, he started to dive into different books. He was looking for

keywords for leading words where the agent does better work, and he started to read the old books. He came across

the philosophy of software design where he got into like deep modules and shallow modules, and he figured that

when he says these words to the AI, it writes better code. Who would have thought, like

on one end it was trained on it, and on the other hand those methods, I guess, work even with

guiding the AI. Fascinating. So,

we went over what changed, what did not change. I'll talk about what broke. Code

code quantity and and our assumptions about it have just gone out the the wall. We see the

GitHub outages, and they are publishing to their credit data on on what's happening.

This speed of how many pull requests are are are pushed per month, the commits, the new repos, in our 3-year timeline,

they're up about 5x, but in just in the last like 6 months, they're they're up by so much more.

Uh data that GitHub gave me actually showed that

they're Yeah, so they're seeing again that 10x increase in agentic-only pull requests,

but but in last 2 months, pull requests have gone up 2x, that includes everything as well. It's it's just

speeding up really quickly. Code reviews are are just done for. I I talked with an engineer who asked not to

put his name on on here, but he said what a lot of people are saying. Everyone is playing the theater

of doing reviews, but with the volume of changes get thrown your way, he observed that people just find the path of least

resistance. Some teammates just give up on this too. Looks good to me on all of them.

Inside this sort of there gradually starting to phase out code reviews themselves because it just doesn't work.

Like I'm I'm start I just call these zombie code reviews. This is happening everywhere. If if your team is still

doing code reviews, you're probably not doing code reviews. If you're unless you've you've cut cut

out on you're not doing code reviews on 80% of them or you're not mandating them. Inside of Ramp, uh they have

started to say like, "Look, we're not going to we're going to trust you to not do code reviews for certain non-critical

parts of code." And I think more companies will do that. Quality and reliability is you can see

this on every software across. It's just breaking. Everything is broken. Some some people

can read about talk about it. You you look at different sites. I see it all the time in just the sites I use uh from

from Spotify to Substack to a lot of a lot lot lot lot of paper cuts. Clearly people are are shipping more with agents

or agents are shipping more and it's it's you can see it everywhere right now.

This is interesting. Infra capacity. We know there's a GPU shortage across the industry, but unless you're an AI

lab or an inference company, you're not going to worry too much about it. Or an AI startup that wants to get GPUs, then

you will be worried. There's a memory shortage, which means memory is more expensive, but again, most of us will

not see too much of that. What I was interested to it's it very surprised to hear there's CPU

shortage as well for cloud companies that are running just you know, normal workloads. It's starting to get harder

and harder to secure your CPUs. I I covered this in the Pragmatic Engineer just very recently with a bunch of

examples including from Turbo Buffer from Anthropic. The

ordering a CPU or a server used to take 1 to 2 weeks of of back ordering time. It's now up to 6 months. Inside of cloud

providers, you cannot if you're a midsize customer, you cannot reserve CPUs in certain regions or in some

regions then no more orders can be taken at all. And some companies are paying for CPU

capacity that is not arrived is not will not arrive until December and they're paying for it cuz that's the only way

they can get it. It's it's manic. Another thing that broke is our focus and our productivity.

I just took one quote from Dima also at Linear. Um and I had a really good conversation with he he said like look

Dima is an amazing engineer from from before AI as well. And he told me there's just a lot of context switching.

There's always another agent that waits your response. And also it's a lot more work. He was really honest. He said like

look when AI was ramping up I felt more productive than my peers cuz I did it in an hour or two what it took them a day

and it was awesome. However, now this has become the baseline expectation that everyone does parallel things. And so he

ends up feeling as having more work compared to pre-AI. It's worth reflecting if if you're feeling like

that cuz a lot of people are feeling like this. Finally, engineering leadership.

Yeah. Um there's something going on with with with engineering leadership. A lot of

you are here so I'm I'm glad that you're you've not walked out the door but a lot of people have walked out the door. I

did a deep dive on this. I talked with about like 20 20 25 engineering leaders who are either taking a career break or

will be taking a career break. And there's a lot of reasons for this but the the main one I'll just list it

briefly. The job of an engineering leader just got a lot worse a lot harder. Some of the some of the because

of founder mode, expectations with AI. Some founders are pushing you to fire half the team. The other are pushing you

to show revenue going 2x when AI is going to go 2x and so on. It's unrealistic expectations. It's just

harder to get job. Startups many startups are losing and leaders see that they're losing.

Your equity is getting worthless or the startup is on path to bankruptcy so you just you try to turn around you cannot

you check out. A lot of startups are not adopting AI workflows fast enough. And if you are in

this room you will know that career stability is also important. You're working at a company and you might work

here in two years or three years or four years time, but there's a good chance you won't. What you want to make sure is

if this doesn't work out or it's it's one of the reasons above, you can go and work at another place. Carry on being an

engineer leader. To do that, your current company should adopt AI practice. You should be changing how the

world is changing and when it's not, that's what I see a lot of engineer leaders quit. This is slow-moving

organizations where you are stuck working in the same way as in 2022 for the next two or three years,

you will go in the career kind of hole. Another one, smaller teams. With smaller teams, you need fewer leaders. A lot of

smaller teams come with middle management being cut as well. Sometimes there are just a lot less need for this.

Fractional CTO work is becoming surprisingly popular, especially in metro hubs like New York, San Francisco,

maybe even Seattle. So, some of these engineer leaders who quit voluntarily, they just don't want to go back right

now cuz it's too messy, but they will do fractional CTO work. And all these fractional CTOs want to be hired

full-time, but they don't want to go back full-time. It's a very interesting trend.

Also, I mean, elephant in the room, at the AI startups will pay way more total compensation, sometimes even cash

compensation, than non-AI startups play play execs. You see how Anthropic is getting the CTOs of every single major

publicly traded company to join as an IC or someone with a small team. They're also paying more. That's also one of the

open secrets. A lot of leaders like you are realize your hands-on. If you want to do a

startup, now is a perfect time. You can probably also reach VC funding if you if you have the background and you have the

pedigree. And finally, some people are just burnt out.

Feeling it's a good time to take a break, see things settle, and then come back a bit later.

So, what comes next after all of this? Here are a couple of things that I'm pretty pretty certain that are happening

and will happen. First one is just cloud agents and harnesses.

Everyone with some engine resources are doing it for many reasons, but with ramp, ramp is doing one of the most

advanced right now, one of the most advanced agentic harnesses that works really well for them, way better than

Codex or Cloud Code or any of them. And I I talked with them in detail on why they're doing it and you they do it for

three reasons. One, local machines are a lot more limited than the cloud. Two, they can actually just they they're huge

on front-end. They build better front-end tooling in the cloud than they have on local machines. And three, they

wanted to have remote developer environments anyway and this just kind of like it kills two birds with one

stone. The only problem with this approach is when you have a CPU shortage, it might be a little bit

trickier to secure, but again, it's a small detail. So many other companies are doing it and I so many will follow.

Uh this is again Dima from Linear. I asked Dima like, "Hey Dima, do you what do you think about cloud versus local?"

And he said like, "He was such a big fan of a local development and the industry is still lagging behind on on dev pods,

that was something we did at Uber, and tooling for cloud development." However, he changed his mind about the future

because he now sees cloud as the future cuz whenever he kicks off stuff running in the cloud, it doesn't hog his

machine. It's faster, easier, etc. This is this is definitely future. We are

we will stop reading the code. This doesn't just come from me, it comes from Charity Majors who is such a big

skeptic and I love her. But we we on the podcast we're talking about like, "What would it take for you

to be comfortable shipping code without reading and understanding it?" Because this is what ops have been doing. This

is what QA have been doing and this is it's not an it's not an if, it's a when. It's coming.

We're all building we will be building out and are building out new type of internal infra. This

means eval's will be part of CICD if it's not already. We will have these agentic factories that we keep iterating

on. Agents will be part of deployments, observability, incident management, and

so on. And again, it's it's it's already happening and will just keep happening even more.

We will have a refactoring wave of migrations, rewriting. I mean, look, when it's

when it's so easy to or cheaper to to refactor, why not do it? Plus, it's great exercise to like

use risk-free some some of these agentic skills.

AI fluency and positivity will be a baseline for hiring and to be hired. I talked with a

engineering director at Series A startup who said that they have started to screen for AI positivity in their in in

their hiring process. He said, "We just don't want people to join. We we want people to join who wants us help to push

the limits of what we can build with AI." As a leader, just you know, just know this is the case and that this is

true when you're hiring and also it's true when you are interviewing. And getting to AI maturity is something

that so many companies will be doing. A startup might think they're already

there. A larger company will be pushing for it. This is Lara Hogan, who's at the Pragmatic Summit in February. She's

saying that orgs who are moving faster and maturing do something different. They begin with a business outcome and

work backwards from there, and they build an agentic system that reduces the handoffs, increases learning, and

removes friction. Now, all of this is a little bit high-level, but it is the

the right direction. And she's saying that the wrong way to do things is one of the

easy way to do is is do an individual-level simple automations, but what's hard to do but what all us

leaders should be doing is doing team-level agentic systems or even company-level agentic systems. That's

why you'll see all of that that AI infra happening, cuz that's how you do it. Finally, engineers with deep domain

knowledge will be so much more in demand. Titus Winters, yesterday, author of Software Engineering at Google, told

me something such interesting. I I I love Titus. He said, "What you need to succeed anywhere? Intelligence, wisdom,

and charisma. You need Intelligence meaning you know how to do it. Wisdom means you know what to do and charisma

means you know you convince others to do it. Now when intelligence becomes commonplace or you know with AI, the how

to do it becomes just kind of very cheap, so widespread, wisdom and charisma become much more important.

As leaders especially remember this one. And I'm not saying you know don't focus on intelligence at all, but he has a

point here. So in this new world, how do you thrive as an engineering leader?

My advice is future-proof your career. Get hands-on if you're not already, but most of you are. Stay hands-on building

software shipping to production. You will hear stories. Will Larson in the other room, he's chief product and

technology officer at a 50-person engineering organization. He has shipped more

uh code into production the past 15 months than in the past like 5 or 10 years combined. And if he's doing it,

you'll you can do that as well. Build out AI infra systems that help your team and your company succeed. And you'll

also be hands-on with this. Use AI to remove friction from your own work, from your team's work, and for your company's

work. And don't outsource your learning. Use AI to get better. Don't for for the important stuff, don't kind of hand it

off and you know like you're you're not going to get ahead and and

but your peers will that do it. And finally, accept that you will probably do just do less people management for

now. This is not really the time. ICs don't expect it as much. We don't even know what that career looks like. We

will figure it out. We will get back to it. But also this is not your main expectation from from above

uh and for from founders and so on. Finally, just lean in, learn more before, and you'll be fine.

With Peter Mattis, I had such an inspiring conversation. Again, this person built so much incredible

software. He is a CTO of a I think a of 600-ish person organization uh 100 and something engineers if if I

recall correctly. At some point he went on to being a hands-off engineering leader and now he's back to hands-on and

he told me you just got to get in there. You need to be using these AI tools all the time. You need to push yourself to

learn. He says that I feel I've learned more in the past year than in the previous 5 years combined which is weird

cuz he's a wicked experienced software engineer and he's built some amazing software.

And finally, just ask yourself, why did I get into tech when you started? What why did you?

Peter told me his reason. He said, I got into software engineer because I like building stuff and now I can build

faster and without some of the compromises I had before. It's a little bit exhausting but it's also very very

exciting. So with this, just keep on keeping on and check out the Pragmatic Engineer and

I'll see you at the Entisys booth. Thank you.