Staying in charge

The situation

I generate so much code, with so many Claudes, in so many projects, that I don't read every line. If I did we'd never get anywhere. A human can't keep up with the AI when it comes to checking its output, and I don't have a fully formed answer to that yet.

What I did

My answer so far has been to black-box the code and look at the things around the outside. I read parts of it, especially around the edges: the inputs, the outputs, the shape of a Docker container, where the secrets are. I like to put code inside a Docker container because then I can see which ports go in and out, and control it on the border. I've looked at the shape of the code and some of the architecture, and I ask a lot of questions about what it's doing.

I've put rules in place to stop the AIs doing things I wouldn't want to see from an intern or another developer on a project. No god classes, no huge single files with everything in them, code that is reasonably structured. Quality now checks that for me, so when I do read code I'm reading for judgement and direction rather than for typos.

Adding a test stops the AI deleting a feature. But I also saw AIs decide a test was no longer valid, because that code hadn't come up in their session, and try to delete it because it was blocking their commit. So I had to put things in place to stop that, at several levels. If the number of tests drops for any reason, the push is blocked and the drop gets investigated.

Where I draw the line

When I've had staff on projects, there were peer reviews from other engineers, and sometimes I'd look at the code myself, depending on what was on my plate. When I use a higher-level language I don't go and look at the assembly it produces. If there's a bug in a library I use, I haven't reviewed that library either. We're all drawing a line somewhere. With AI it has moved, and so has who's doing the work.

The result

I look less and less into the code, and more of that checking is done by tools. My attention goes on the test cases, the interfaces, the shape of the outside of the code and the user experience. I think that's more and more what this work will be.

What a team can take from it

I still think you need a software developer, and probably always will to some extent. You need somebody there who knows how to deal with the code coming out of the AI, even if they're not reading every line of it.