Mindless Rambling Nonsense

How I use LLMs? - six

Sat, 25 Jul 2026 08:00:00 +0000

Let's ~~growth hack blog visitor numbers~~ and keep on recording how I use agents/LLMs today as a little reflection on where I think they work and where they don't. Something I can revisit as the tech (and my skill with it) improves

This is update number five, or entry number six, depending on how you want to count it.

someone at work named the episodes for me, i love it

so now… part 6 - i think maybe everything changed

in today's edition of AI hot takes

saying "slop" considered harmful
ship value, not craft
don't read code
manage loops and context - fuck prompts
if you make me copy & paste a prompt you've lost me

First, a necessary but brief diversion into how I use tools generally

I was largely self-taught as a developer and (after vbscript 😱) I started with DotNet. I found JetBrains Resharper early on and never looked back. The assistance in the IDE to write, refactor, and improve code was game-changing. I don't begrudge anybody any tool they want to use to level-up. Wanna use a graphical git client, or emacs, or anything else… go for it, go make cool things

Because of that entry to the industry I'm also not super keen on VSCode (you love it? great, see above, you do you, go make cool things). I'm too used to clever interventions helping me and VSCode is too barebones (and so much slower than SublimeText)

So, I like having tools directly involved in my workflow. I learned LINQ more quickly and more thoroughly because I had resharper prompting me inline, at write-time: "hey, why not like this?"

saying "slop" considered harmful

I've noticed a habit for folk reporting a bug with a system to say things like: "ugh, the clink expander doesn't work. why even ship this slop?!"

my friends, if you think there were no bugs prior to December 2025 then you were not paying attention.

through most of 2025 if you were just shipping AI generated code without thorough checking then you were going to be in for a bad time… but the world changed in Dec 2025 and several times again through 2026.

the code written by LLMs is now at least as good as that written by the worst junior developers i've worked with over ~20 years. and sometimes much better.

those junior developers all managed to ship value despite being inexperienced.

and the goal is shipping value

ship value, not craft

i have long considered myself a software crafts person. attended user group meetups; run brownbags at work; practised deliberately and consistently; learned about XP, agile, tdd, and more.

why… because i believe that software gets harder to change and extend over time otherwise

and that's important because i have always been focused on shipping value to people

a problem has a solution in a context

as agents have improved the context has changed gigantically

the problem of needing to ship value remains but the solution of software craft doesn't fit any more

software needs to be easy to change and extend by agents way more than by humans

we would slow down when software files became complex because they broke our meat brains… but our meat doesn't need to parse it any more, our agents will

if software is for agents not humans, then one change (whether you like or not) is…

don't read code

to quote a (frankly fantastic) colleague:

I'm sure when the first FORTRAN compiler was released, for a time, engineers still read the machine code output that it produced to make sure it was up to scratch.

Who does that still? Compiler developers and nobody else.

It will be like that for generative LLMs producing code also.

Imagine someone making the argument that if you don't read, understand and verify all the assembly that GCC outputs then they're not a real engineer or they're somehow not doing their job.

I think about this like I am working with a team of developers. Not in the team. with the team.

If I hire them to make an API for me. I'll write tests directly against their API maybe. Against my use of it for sure. I'm very, very unlikely to look at their code. When I'm using agents. I'm working with them, not in them. So, the code they write is simply unimportant if it works.

I have always asked three questions:

how do you know the software doesn't do it today
how will you know the software does do it in this change
how will you know the software still does it the day after the deployment

the answer has always been "writing tests, gathering metrics, and talking to users"

this becomes so much more important in this new world

deciding what to build and whether it was good idea once you've finished is where you should spend your time. reading code used to support that, but not any more

if you make me copy & paste a prompt you've lost me

i noticed i now find tools that expect me to copy their output to the next LLM annoying

the future is a world of agents interacting. not people passing text between agents.

different people will want to grab output from the loop of genies at different points. your org might require you use a particular system or an air-gapped network or whatevs and you'll need to break the loop and move data yourself in some way.

but lots of systems will be a set of agents triggered by timers or external signals passing data and prompts between themselves… and it's gonna be super cool.

How I use LLMs? - five

Thu, 04 Jun 2026 08:00:00 +0000

I have lost roughly 60% of my cynicism about the current high-water mark for LLMs and augmented coding… I am certain this last 9 months has been a significant generational change in how we can approach work.

Oh no, I'm AI pilled

Let's ~~growth hack blog visitor numbers~~ record how I use them today as a little reflection on where I think they work and where they don't. Something I can revisit as the tech (and my skill with it) improves

This is update number four, or entry number five, depending on how you want to count it.

someone at work named the episodes for me, i love it

so now… part 5 - yee haw, paul is AI pilled

in today's edition of AI hot takes

the agent is my to-do list
no longer focused on the code, 5% focus down from 95%
i don't use these tools, i hire them into my team
genuine frustration when they lack autonomy
stop making me tell you when to run
skills skills skill skills

First, a necessary but brief diversion into how I use tools generally

So, I like having tools directly involved in my workflow. I learned LINQ more quickly and more thoroughly because I had resharper prompting me inline, at write-time: "hey, why not like this?"

the agent is my to-do list

why write something down in a to-do list when i can open a new conversation and write exactly the same text as a prompt?! i immediately switch away and carry on with my day, like with a to-do list.

but my agent is connected to our code and to all of our analytics, so almost every time when I switch back it has context and challenge. or a PR

to-do list on steroids

no longer focused on the code, 5% focus down from 95%

Last time I said:

Recent improvements to agents and their UX has meant I only have to actively write code when I choose to. It would be possible to completely work via agents.

I can literally never write a line of code manually anymore and it is so ok.

Because the cost of understanding the code is basically zero now. The importance of the code being understandable is dropping.

It still has to be correct and fast but if I can't quickly grok it. I can just ask an LLM to explain it, or write some tests to exercise it. What might have taken hours in the past, I've worked on code that took me days to understand, will now take seconds.

Writing code is now far from the highest leverage thing I could do with my time

"Oh but Paul some code is very important!"

Yep, and cool, the literal code might still be the highest leverage thing you can do. But it isn't the highest leverage thing I could do any more. And I'd argue for many people that's true too.

And that is truly magical because now I can care about and achieve a bunch of stuff all at the same time and that would have been impossible for 2025 Paul.

i don't use these tools, i hire them into my team

If I am nailing two pieces of wood together it is up to me to figure out how to do it. To choose the hammer. To choose the nail. To pick the place and the angle that the nail goes in. To make sure I don't hit my thumb.

If I hire a carpenter then I make them a brew and tell them I'm here if they need me.

So, the mindset shift I'm suggesting is that I am hiring a junior engineer or product manager or whatever. And I am responsible for giving them context and guard rails. And then I let them do their work. I'm still responsible for if the work is valuable, but I'm not literally hitting the nail any more.

It changes so much about how I interact with the thing. And it lets me completely change how I work. I've moved into an executive role and I can still sling lots of code cos I don't hold the hammer, I hire the carpenter.

genuine frustration when they lack autonomy

We run a bunch of AI review tools on our PRs. And they have varying quality. They get better over time bit by bit. And they are earning their keep (just about - although they're pretty expensive tbh)

But over the last months since part 4 I've noticed a new expectation from me… they post their comment and I think "ugh, now i have to read and act on that… such bullshit"

By comparison, I have a "PR Shepherd" skill. I've had Claude do research about XP on the C2 wiki. I've had it interview me about software engineering to see what I care about in how work is done. And I've had it review the public history of my PR reviews to see what I care about in reviewing.

And when I run that skill it does a bunch of reviews in parallel, and anything that has a single solution, it just does. You can see an example here - I have it run and publicly comment on all of my PRs. It now feels rude to have a human review before I've had the robot find things. In the same way that for the longest time it's been rude to make a human lint your code when a tool could do it.

So, increasingly, the word machine can no longer earn its keep by only turning one set of words into another. It needs to turn that set of words into action. Just like if I hire a junior engineer and they asked me "should i fix this obvious bug" i would be frustrated.

stop making me tell you when to run

this has a side effect… i don't want to have to remember to ask an LLM for a whole bunch of stuff. same way i don't want to have to ask my colleagues about a bunch of stuff.

i want to say "hey, we should check the clink expander every two hours and route fanglers appropriately" and that just happens

i think the next great frontier is long-running and trigger-driven agents. and i am so excited for it

skills skills skill skills

I mention "PR shepherd" above. It does all the mechanical work of getting a PR through review. It checks if a PR is open, opens it if not. marks it ready for review, runs a swarm of parallel reviewers and responds to them, checks external reviews and responds to them. It defers to me when it has to, otherwise it checks CI until it's green. It took me about an hour in total so far prompting to create it. And has saved me multiples of that.

Soooo much time back.

I have "Paul pair". Every time the robot finishes work or is about to ask me a question it has an instruction to run the "Paul pair" skill. Which reminds it what I care about and to only interrupt me with important things. It's mecha paul. Saving meat paul from repeating the thing i was obviously going to repeat.

I have an executive coach (admittedly not a skill). It checks granola and slack and github and the org chart etc etc and once a week tells me if I'm working on what I've said is important. Tells me what I'm missing, what patterns it sees, what the top 3 things I should do to get back on track. It's not always right, but it doesn't have to be - it's on me to manage my work and attention regardless of what advice i receive.

If you have never done it then stop what you're doing. Open a new session with your favourite robot and type "look at our conversations and tell me what repeated interactions we could automate"

Then sit back and prepare to have your mind blown.

Here are all my public skills

paul, you sound irredeemably AI pilled

yep, I am. sorry, it's happened. there's no going back. i've been sucked into the bubble. this is the future.

"but LLMs gets things wrong sometimes!"

yep, and this might shock you. so do human colleagues. even incredible colleagues can do silly things. yes, reader, even I have once or twice shipped a bug

it's not about the LLM being perfect. it doesn't have to be perfect. it's about the LLM being useful. And it really f-in is.

Where have I been?

Fri, 22 May 2026 10:00:00 +0000

At a recent company offsite a bunch of people asked me how many of the off-sites I'd been on. I didn't remember so I made this list since if I don't remember now I have no chance of remembering in 5 or 10 years.

2021

Place	Type	Note	Hackathon
London	my onboarding	Jams said "just do the right thing" which was a good signal this company was cool
Milan	working, not a company trip	i just had to go to Italy for my soul after covid

2022

Place	Type	Note	Hackathon
Barcelona	all engineering offsite	got hard feedback, realised I had to up my game
Iceland	all company offsite	my first posthog hackathon, realised just how talented everyone was	writing session replay ingestion to s3. it was a bad implementation.
Cambridge	onboarding	the one where i felt i could talk about posthog well
Montesilvano	working, not a company trip	I just needed time alone
Rome	small team offsite	this was the turning point for me being successful and feeling at home at PostHog. I got amazing, direct, constructive feedback. life-changing, no cap	in-app notifications on top of the activity log
Cambridge	onboarding	the one where we got locked in one of the Cambridge colleges after maybe breaking in
Lisbon	work-blast	the one where Ben and I wanted to prove network logging was quick to build - it was. Kaarl told me my schema design was good, I was so in awe of his clickhouse brain i thought i'd faint	network request logging in replay

2023

Place	Type	Note	Hackathon
Aruba	all company offsite	the one where we celebrated turning into a juggernaut	bug reporting tool in PostHog
Cambridge	onboarding	the one where i made pizza with bread flour, no bueno
Glossop, UK	onboarding	the one where i couldn't travel and so the team braved Glossop, i made some pretty good pizza
Bologna	small team offsite	the one where posthog 3000 was born, we presented each other's ideas. It was silly, laughed so much	Sampling for session replay

2024

Place	Type	Note	Hackathon
Trapani	work blast	Ben and I wanted to prove Heatmaps was quick to ship, it was	Heatmaps
Vienna	onboarding	the best beetroot i've had to date, my favourite part of italy
Cambridge	onboarding	manoel did amazing bbq steak
Valencia	small team offsite	the one where i realised i could push people more	AI summarisation of survey responses

2025

Place	Type	Note	Hackathon
Cambridge	onboarding	very cold, we saw a fox, Ben improv'd the architecture diagram
Budapest	onboarding	great coffee, so many irish bars
Tulum, Mexico	all company offsite 2025	the one where i realised just how much we'd grown	CSP violation reporting… that has started to make bank now
Munich	onboarding	my first ever visit to germany, how?!
Palermo	small team offsite	the one where we experimented with combining small team offsites (and it changed things a lot)	letting people add screenshots to action and event definitions
London	xmas hackathon and onboarding	the one where Tue got very excited about a smash burger	tooling to make session replay support easier

2026

Place	Type	Note	Hackathon
Lisbon	analytics platform small team offsite	the one where we figured out how to make a query performance team	adding AI summary to insight subscriptions
Barbados	all company offsite	my first all team offsite as an exec. lots of great feedback and pina coladas	creating an AI user researcher

It turns out I've been away from home for roughly 26 weeks in the time I've been at PostHog. Or roughly 10% of the weeks I've worked here. Wouldn't change it for the world.

Let's plan that

Fri, 13 Mar 2026 10:00:00 +0000

My favourite quote is "The plans are useless, the planning is essential"

Since the plans are useless, why do we run planning sessions at all?

It's planning season at work. We are high trust, high agency, and minimise collaboration. It is pretty common for folk to change their plans and they do so without needing to seek approval.

Without shared context and an understanding of the broader picture, people would change plans but be pulling in different directions. Having the big picture means you can rely more on instinct and curiosity rather than rigid plans.

Torture a metaphor? If you insist…

The landscape you're building software in probably doesn't look like this:

We're generally operating under imperfect conditions. Trying to figure out where we are is more like being in the fog:

A friend was for a while a member of mountain rescue (who are incidentally incredible - you should give them money). They once described to me how they navigate when they have very low visibility.

In pairs:

use the map to figure out where you are
use that information to figure out what direction to go
using a compass one of you slowly walks in that direction
the other stays still and calls out when the walker is about to disappear into the fog
then that person catches up with the walker
repeat

Looking at the context of where they are against what they know about the world. Working together to understand what that means, right then. Watching each other and relying on communication. Chopping the journey into many safer parts.

The short loop of frequent interactions with your colleagues, your users, your analytics are the small steps in the fog. But you have to plan to have the destination and route. This navigation completely falls apart without the map. It requires the wider context.

But surely you can't keep going with this metaphor?

Once we were hiking on Maiden moor and within a half an hour we went from clear skies to low visibility in fog. My wife who is a competent navigator asked me: "why is your thumb there (on the map)?". I explained it's where we were. And she pointed out distant landmarks to triangulate how wrong I am.

My error made it into a much longer and more challenging walk than we had planned.

Her navigation is better because she is looking at the bigger picture. Whereas I'm counting field boundaries.

Even once we have the map and the route, we still need to navigate by looking at the landscape and landmarks to stay on course.

Your customers and your competitors don't stand still so you have to keep looking up to see if the world changed under your plan.

OK, we're done now with the hiking metaphor now, surely?

Well, why was I counting field boundaries? Once we have the map and the route we have our plan. But when you try to follow it, you discover all the things that make it difficult. A copse of trees was cleared. A farmer moved a stile. A path is impassable because of thorns or mud.

The route very quickly becomes a series of small decisions and adjustments as you navigate around obstacles. You can't make those adjustments with a static plan. You can make them because you have practised making the route. You know why it is the route and so you can adapt it to changing conditions.

The actual plan is the starting point. The thing we thought would meet our goal before reality hit us over the head. What you learn on the way will often change your plan. Ignoring that is as silly as not having any goal to head towards.

Please no more metaphor

In all of these scenarios the planning doesn't get you anything without execution - no point planning a hike and not walking. But the planning has to happen to make the route any good.

That's what good teams do. They have a series of nested loops and they check themselves at each point. Short term loops like daily standups that focus on the next few hours. And long term loops like quarterly planning that focus on the next few months.

You have to plan but be receptive to changing plan. If you stuck to a plan despite what you learn you'll get stuck in a bog, but if you never plan you wouldn't avoid any.

How I use LLMs? - four

Tue, 10 Mar 2026 08:00:00 +0000

I remain very cynical about the current high-water mark for LLMs and augmented coding… but at the same time I use them every day and I don't think they're finished improving.

This is update number three, or entry number four, depending on how you want to count it.

in today's edition of AI hot takes

basically never braincode
it's still about your brain
hot take

First, a necessary but brief diversion into how I use tools generally

So, I like having tools directly involved in my workflow. I learned LINQ more quickly and more thoroughly because I had resharper prompting me inline, at write-time: "hey, why not like this?"

basically never braincode

Recent improvements to agents and their UX has meant I only have to actively write code when I choose to. It would be possible to completely work via agents.

It really is only necessary to "braincode" when it is easier for me to sketch a solution than to prompt an LLM to write it. Even then I would quickly fall back to "hey agent, here's what i mean. finish this off"

Having the agent in slack is chef's-kiss-dot-reacji. Someone says "hey, should the foo be wider" and you reply "@agent, make the foo wider" Like having a super eager and precocious intern sat waiting for the next task.

it's still about your brain

You'll see a lot of people saying variations on "it was never about the code" or "it was always about what you chose to make not how you made it". And tbh I 90% agree with them. The code was a hard part but it wasn't the only hard part.

We've automated a big chunk of code understanding and generation but we still need to choose the direction. Reaching through the system of systems you can affect to try and deliver value.

Astronaut-with-pistol-dot-meme

hot take

Code is no longer for humans.

We are seeing the end of the era of code needing to be in a format that humans can directly parse. Languages are already abstractions on top of what is really being executed. The format being determined by the available technology to let humans create code. And by the needs of the humans at understanding the code.

In a world where we can say "hey robot, how does the Foo component manage the ClinkExpander interactions". It no longer matters what the underlying code looks like.

My expectation is that there are already folk working on a storage format that is accessible to agents but loses the trade-offs it had to make for human readability. Maybe a binary DAG format since we don't really need the translation from an abstract syntax tree in memory into text anymore.

If the agents keep getting better, my expectation is that I need to be able to ask for representations of the code that help me and them interact with it way more than I need a flat text file.

I shipped 1000 PRs in a year, AMA

Wed, 11 Feb 2026 08:00:00 +0000

I shipped 1000 PRs in the last 12 months at PostHog. And someone asked me how I do it… let's see if i know.

preamble

yes, yes, I know you don't think I should count PRs
when a measure becomes a target it stops being a useful measure
embrace yourself

advice

it is running not cycling
small steps ftw
practice like a jazz musician
love making things
one more iteration

yes, yes, I know you don't think I should count PRs

So, first something important…

Counting PRs is a very poor metric for measuring progress. Someone who ships 10 PRs has not provided a guaranteed ten times the value of someone who ships one PR.

I'm not telling you to count PRs. Or to worship the github tile graph.

But….

Yes, weighing yourself is a bad way of checking if your last meal was healthy. Yes, merging a PR is a bad way of checking if you just provided any value.

But not weighing yourself is a great way to put on weight without noticing.

And not measuring what you ship is a great way to accidentally ship less value.

However………..

"when a measure becomes a target it stops being a useful measure": Charles Goodhart in preparation for my birth

Taking a look at how many PRs or lines of code people ship is a measure of what people are doing. It's vague and lossy but it's a starting point.

And, yes, of course, someone could ship lots of PRs but they're no good or only ship one thing a week but it's super impactful.

So even if you use it as a measure it should not be your only measure of progress

But, importantly, if you tell people "number of PRs is related to promotion review" then they will damn well make sure they ship more PRs without that being related to actual value to users.

It is no longer a useful measure of progress because it became a target.

embrace yourself

didn't that feel nice?

no, not like that

Throughout my career I've been told I spin too many plates and to try and reduce my work in progress. I tried really hard to lower personal WIP and it was sooooo much effort.

My colleague Marius at PostHog said (roughly): "you always apologise for spinning plates, but you're good at it. why don't you try to get better?"

So, I've done that. I've accepted that I am easily distracted and that I will start many things. And I've concentrated on getting better at making sure the plates I'm spinning are moving forward. And throwing away the ones that are just distraction.

I've started more than 1000 PRs but recognised the ones that aren't going anywhere.

For you this will look different. You might have to not listen to music with lyrics. You might need to make sure you go for a walk in the morning. Or plan blocks of the day with zero distractions.

My advice would be - stop listening to advice that makes you fight your nature. Just because someone else wants to work on one thing at a time doesn't mean you should. And just because I can have 23 open PRs right now and be on top of my work doesn't mean you should.

Ok, now that I've told you to not listen to other people's advice.

I will dress in a robe, walk down from the holy mountain of the github tile graph, and hand down my advice of what I believe contributes to my productivity:

it is running not cycling

I used to cycle a lot and having struggled up a big hill it was super cool to stop pedalling and coast down the other side. Then I started running, and having struggled up the big hill I discovered I still had to put in effort to run downhill. Even if gravity was helping me if I stopped moving my legs I stopped moving.

Delivering valuable software to users is running, not cycling. If you stop shipping your momentum is quickly lost.

It is up to you to move your legs. To bring the appropriate intensity and effort.

Want to ship a lot? Just do the work…

small steps ftw

I have seen time and time again that small, concrete steps that you can explain, complete, and measure are the best way to make progress.

See "But why regularly measure" in one of my earlier weblogs

If you feel stuck with the coding, break it into a smaller piece. Not sure how to do the work? Figure out what the simple pieces are and start with those. If the remaining pieces are "too big" or "too hard" figure out how to change the system so the solution changes and is smaller or easier.

Often, I will make one big PR that covers the whole change to prove it works. Then break that into smaller pieces that refer back to the whole. To make it easier to review or deploy.

practice like a jazz musician

When I was younger I was labouring under the misapprehension that I was going to change the world of music. And I was lucky to spend some time with some people who were amazing musicians.

They would practice frequently and deliberately. They were purposefully aiming to get better at the little things that made them better at the whole.

Later I spent time with folks talking about the craft of software engineering. Practising skills like TDD and refactoring.

The little things that make you better at the whole.

Ah, but Paul this new world of agentic software engineering makes skills like that outdated and unnecessary.

I love this quote from Andrej Karpathy

And I've found the same… If the prompt is important then knowing what to prompt is important.

And when the prompt isn't enough to get the job done being able to sketch enough code to give a better start to the agent is important.

In both cases you actually have to know what you want. And practicing what good software, or good architecture looks like is still a differentiator.

I'd add practicing writing good prompts and managing context well as new skills that it is worth practicing.

Be purposeful about improving your ability as a person that provides value through creating and changing software. Watch what you do, see what makes it better, double down on those things.

Avoid through practice and diligence ever doing the wrong thing harder.

love making things

I make things all the time. I am sad when I don't get to make things. This is a vocation for me.

Nobody points at the musician that plays music for the sheer joy of it and says: "ha, look at that loser. playing music even though they're not getting paid for it"

So, in the evening when I've finished getting the kids to bed. I sit and fiddle with software while I watch TV or listen to music.

I am incredibly lucky to have found something that gives me joy and makes me money.

Not a vocation for you - ok, cool. Live your life as you see fit. But one of the ways that I am productive is that this is a joyful vocation. Also, if software is not your vocation, why the heck are you reading about how to ship 1000 PRs in twelve months?! Go, ride horses or fix gates or whatever the thing is that does bring you joy 💖

one more iteration

It feels like it goes without saying that iteration is good… but…

Iteration is the process of repeating a set of steps or instructions multiple times, often to reach a specific goal, solve a problem, or improve a result.

"One more iteration" is a phrase that Tim at work uses to push us to think about the quality of what we build…

OK, you did something. Stop and ask: "am I really finished?"

How do I know if I'm really finished?

are people using it?
do they like it?
did it move the needle the way I expected?
would I know if it was broken?
now it is functional does it need to also be delightful?
does it help me see what should come next?

I think this sits really wonderfully with "small steps ftw". Iteration of large blocks of work is inherently slower than iteration of small steps.

The benefit of small steps is you can be nimble in response to what you learn.

Worse… Something that takes a month to put together will build up a lot of weight in your subconscious. It's hard to throw it away or change it because of the time you have sunk into it. Not only are you less nimble, you're less likely to correct your course.

A concrete example of how i do small steps and iteration…

A customer reported they were unhappy with filtering in our tool. It was multiple clicks to get to the point where they had type-ahead search. And that was frustrating for them.

Frustration is no bueno, so I started spinning a new plate. I mocked up a solution for them very quickly and shared a video.

They loved it.

BUT I OPTIMISE FOR ITERATION AND SO I DID NOT SIMPLY SHIP IT

I wrapped it in a flag and released it just for our team and that specific user that I was talking to

I got fast feedback that they did not like it. They liked the description, and they liked the demo. But what people think they will like and use is not always what they will actually like and use.

In practice by optimising for one journey through the filtering UX I had made other journeys harder. I wouldn't have known that if I didn't think about how to release this in a way that meant I could check if it was any good.

In the end, this took four attempts to get right.

And now I have rolled out the flag to all users. After an experiment over those multiple iterations showed folk came back and searched more using the new versions of this change.

But rolling out the flag isn't the end of the work. I'm now working on tidying up the code that was there so that I could dual run and measure the change. I've got to finish updating the documentation. I need to check that users behaviour really does change over coming weeks and react to that. So there are still more PRs to come from this work.

An inspirational sign-off?

The value here isn't that I shipped lots of PRs. The value is that I work in small steps, and I check direction in between them. That I can loop between what I learn and what I can do next to learn more.

How you do that might be different, perfect! But my advice: do that.

How I do user interviews

Wed, 11 Feb 2026 08:00:00 +0000

I've been lucky to watch some excellent engineers, product managers, and user researchers carry out user interviews.

And had the opportunity at PostHog to practice that skill. I've explained all this a couple of times to different people, so thought I'd write it down.

set expectations
write down quotes
show > tell
compound interest
sometimes stop
be a Labrador
this is the redesign that will kill the the Facebook

set expectations

Whenever I'm starting a user interview I try to set very clear expectations for the person. I will say something like:

Thanks for your time today. And importantly it's your time. So, while I'm mostly interested in $feature_X if you want to talk about other parts of the product, that's fine - we can cover the things you're interested in. I'd really love you to share your screen so I can see what you're doing as well as hear you talk about it. To start out can you show me how you use $feature_X?

You might want to constrain what they talk about to a specific feature or topic. So your intro would be different. But I find it's generally better to hear what they're thinking and feeling even if it doesn't relate to what I'm focused on.

write down quotes

I had the great privilege of working with user interviewers with a GDS background and was always impressed by their skill.

One thing I learned from the wonderful Simon Hurst was to write down quotes when you're taking notes. And to editorialise them later.

You're aiming to capture the user's statement not what you thought about it at the time.

The benefit of having the quote is that you don't have to remember what they said later when you're grouping.

This is also a great way to introduce colleagues to the process. They can shadow your user interviews and capture quotes as they listen.

Of course, if you're doing these remotely it's trivial to record or transcribe the interview. But I still think there's value in going through and pulling out the quotes to capture them. I love to talk to a team and find that they know their users by name and can refer back to specific quotes.

You might worry about acting on anecdotal evidence, and you should test and measure your changes, but that kind of human understanding will guide you to better decisions.

show > tell

It is super powerful to get users to show you what they mean.

If you're interviewing someone and they say "I wish your software let me do Y" (sometimes the software already does do that). The amazing Neil Kakkar would ask "can you show me where you'd look in the UI to do that?"

Whether the feature exists or not already, you now know what the UI for discovering it might look like.

Or a user will say something like "competitor X does this better." the important response is to ask "can you show me how to do it in competitor X?". Sometimes you'll see that it really is better, sometimes that it is just more familiar to them in the other UI.

The insight isn't that you should copy what the competitor does. You should try to understand why they prefer it, across multiple users this will help you understand how to position that feature or introduce that user flow.

compound interest

Setting up and running user interviews can take a lot of time. And so people often skip them. But 1 interview is better than none. 2 is better than 1. And so on.

And this compounds over time. Before you know it you've met tens or hundreds of users.

The knowledge you accumulate about your users (individually and organisationally) will give you instincts on what to build and how to build it.

Decide how many interviews you want to run each month. And stick to it… there's always something more to learn.

sometimes stop

Sometimes you think it's going to be a user interview. But it turns out this is a user that's unhappy with the product. Or doesn't understand the platform and just needs help. I used to struggle through a bunch of open questions trying to "stay in interview mode".

Then I observed Annika Schmid on one call recognise it for what it was and shift into training/explaining mode. It already wasn't going to be a great user interview. So, may as well make it a good experience for the user.

be a Labrador

Importantly, you are there to learn. So, you need to be as interested in their opinions as a Labrador is in cheese.

"I'd love to know more about…" "Can you show me…" "Oh wow, that's not how we expected folk to use this! Can you dive into…"

One thing I've picked up from the fabulous Eric Duong is the phrase "out of interest…"

Not just in user interviews but in every customer interaction. You can drop it in… "Ah, it doesn't work like that… you can workaround by x, y, and z… out of interest what does it unlock for you if we support this?"

I love that framing. "Out of interest what does it unlock for you if we support this?"

That user might not commit to 30 minutes for a user interview. But I bet they'll spend 30 seconds replying to a slack message. And since these things compound over time, you'll learn a lot from accruing all these tiny insights.

this is the redesign that will kill the Facebook

Related to "competitor X does it better" i remember at least three occasions where Facebook changed their UI and everyone was wild in the comments talking about how it would kill Facebook.

Twitter changing the star to a heart made the national news.

With enough users you'll discover that you will get negative feedback no matter what you do. And if you chase every user, you'll end up with a product that no one wants.

Even though, if you don't listen to the users, you'll end up with a product that no one wants.

Gathering all this feedback isn't so that you can just make all the changes people tell you they want. It's to get an instinct for how and why they use your product. You will be asked for conflicting things and without the gut-feeling for what users want, you'll end up with Homer's car.

How I use LLMs? - three

Fri, 30 Jan 2026 08:00:00 +0000

I remain very cynical about the current high-water mark for LLMs and augmented coding… but at the same time I use them every day and I don't think they're finished improving.

This is update number two, or entry number three, depending on how you want to count it.

the first entry is here the second entry is here

XP pedant agent
primarily using LLMs
throw away way more work
be very strict with what i do type
i approach understanding code differently

First, a necessary but brief diversion into how I use tools generally

So, I like having tools directly involved in my workflow. I learned LINQ more quickly and more thoroughly because I had resharper prompting me inline, at write-time: "hey, why not like this?"

XP pedant agent

I love the XP simplicity rules for guiding decision making…

Passes all the tests.
Expresses every idea that we need to express.
Says everything OnceAndOnlyOnce.
Has no superfluous parts.

I like that they're in tension with each other.

I've added them to my AGENTS.md and if it forgets I can prompt it "hey, apply the simplicity rules".

Often the change it makes is the one I would have manually made.

Primarily using LLMs

Over the last three months particularly the capability of LLMs has transformed. I can now go whole days mostly prompting an LLM and writing only a fraction of the code myself. This is… fine.

I've been working on super complex, tricky to find memory leak problems. I would not have had the patience to notice some of the things that the robot can when i provide it with context from my investigations.

And while it's working on the next step I can be improving my investigation, or catching up on Slack, or taking my kids to school.

I don't feel like I've lost anything by not (always) typing out the code directly. And since I taught the robot some extreme programming techniques, i find i don't need to re-work much at all.

I genuinely didn't think I'd switch this much… and like all sticky tools I didn't have to think about it. The tool hooked me.

That said… i don't like vibe coding… I'm not asleep at the wheel and do care about the diff that's being generated. So I feel slightly vindicated by this research from Anthropic https://www.anthropic.com/research/AI-assistance-coding-skills

In a randomized controlled trial, we examined 1) how quickly software developers picked up a new skill (in this case, a Python library) with and without AI assistance; and 2) whether using AI made them less likely to understand the code they’d just written. We found that using AI assistance led to a statistically significant decrease in mastery. … Importantly, using AI assistance didn’t guarantee a lower score. How someone used AI influenced how much information they retained.

throw away way more work

Since I didn't type it, I don't feel invested. I dump a prompt in. Come back, see what came out. And sometimes I just throw it away. It doesn't help as much as I expected, or a UI change doesn't look as good as I hoped.

Without feeling invested in the creation I am a little more rational about whether to throw it away.

It's cheap to do many things at once and keep the useful things.

be very strict with what I do type

This is an addition to "manage context".

Sometimes I find myself writing a huge prompt. I have low tolerance for this. Or I find the task is spinning and churning and going nowhere. So, instead, when I recognise that I stop, I clear the prompt, and I spike the work. Then when I have a good enough sketch I point the robot at the changes and say "hey, like this". I believe we do understand some problems better through trying to solve them.

The robot is still more stupid than it is clever. So, giving it code is a good way to clever it up.

It is still valuable to be a decent software engineer. But it might be more valuable to be a decent software engineer who dumps a sketch into an LLM. (if it isn't now, and they can keep improving, then it will be soon)

I approach understanding code differently

This one is difficult for me to accept, but it's snuck up on me and over the last few months changed how I work.

The cost of writing code is falling and the cost of understanding code is falling.

When the cost of a thing fundamentally changes, you have to change your approach.

"Hey robot, write a script to validate the blah does thingy (or not) when the wotsit is twanged"

"Hey robot, why does the clinkexpander accept a boolean?"

"Hey robot, how does the clinkexpander work?"

"Hey robot, what does the clinkexpander actually do?"

Tasks that would have taken me hours or days or weeks in the past are genuinely done in minutes now.

I've spent a big chunk of time recently feeding heap snapshots to the robot. I can only give huge thanks I've not had to manually grok this myself.

Is the robot perfect? No. not even.

Is the robot a good software engineer? No. not even.

Is it silly to ignore the fact that the robot does not need to be perfect or good for it to be a fantastic tool?

yes, veryeven.

the game has changed. pandora's box is open. the robot is here to stay.

It reminds me of once… I used to work with someone who had worked mostly with signal processing baked into hardware. Very painful to discover a bug after release. We were sat together and I changed a field in a JSON response. He asked in a mid-panic: "but how many bytes will that add to the payload?". Genuinely confused, I said: "I've barely ever had to care… I don't think it matters"

We're on the verge of that kind of change. We're not there ffs but some point down the road someone is going to ask "but isn't that AI slop" and their colleague will answer, genuinely confused: "yes, why?"

Will some of that software be bad? yes always-has-been-dot-meme

Things that a different me would have been tipped into psychosis by

Wed, 03 Dec 2025 08:00:00 +0000

There are a few memes (in the original sense) that have got stuck in my head. They feel like the beginning of madness. I'm going to track them, to see how many I accumulate

water isn't real
old people are time travellers

Each of them feels like something that would tip a different me over the edge into madness.

Water isn't real

I could be easily convinced that water isn't real.

Sit in a boat, look over the edge at the water.

It's obviously CGI. Can't possibly be real.

Old people are time travellers

Whenever I've had a baby (a common thing I've done) I'll be walking with the baby in the pram, and very sleep deprived.

An old person passing by will say something like: "Oh, what a lovely baby". Some similar nice thing to that.

In my sleep deprived state it once occurred to me that a very lucrative business - if time travel existed - would be to let you travel back and see your grandparent as a baby.

Can't get that out of my head now. Whenever an old person is nice to me, i'm a little odd with them… because at the back of my mind there's a voice saying "that's exactly what a time traveller would say".

How I use LLMs - two?

Fri, 24 Oct 2025 08:00:00 +0000

I remain very cynical about the current high-water mark for LLMs and augmented coding… but at the same time I use them every day and I don't think they're finished improving.

This is update number one, or entry number two, depending on how you want to count it. So, here are the changes in how I'm using LLMs

(we had a brownbag on this at work yesterday, so it's been on my mind)

the first entry is here

manage context
track longer tasks in a structured file
smallest prompt that describes how you want to work
be sparing with auto-accept edits

First, a necessary but brief diversion into how I use tools generally

REMINDER: I don't begruge anybody any tool they want to use to level-up. Wanna use a graphical git client, or emacs, or anything else… go for it, go make cool things.

You don't use LLMs, or not like this, cool… go make cool things.

So, I like having tools directly involved in my workflow. I learned LINQ more quickly and more thoroughly because I had resharper prompting me inline, at write-time: "hey, why not like this?"

Anything removed from entry one?

Being your army of interns

I don't believe in having a personal WIP of one item in progress. So, I like that I can have a higher WIP without exhausting my own kreplets

But… I find the background work is best as tasks like

"upgrade Jest to the next major version"

"why is this component rendering too much"

very, very constrained yak shavey tasks.

otherwise I find that the LLM needs a lot of attention to complete the task well, and so my kreplets are being used up - and because I'm not doing the work I'm learning more slowly.

Manage context

I'm pretty vicious about managing context. If I start a new task I'll clear the context completely. If I get even the slightest inkling that the LLM is going off on a tangent, I'll clear the context completely.

It isn't thinking despite all the cognitive biases that make us think it is. It's just churning out tokens based on the tokens in its context. So, if the context gets screwey so does the output.

Longer tasks, track them in a file

If we're doing a big change or exploring something complex. I tell it to make a markdown file with a to-do list and what we're working on.

For example, i asked it to track down and edit every event listener in our front end. There are ~350 files it would need to touch. It regularly would partially edit around 30 of them and then we'd hit a "you're absolutely right" loop.

So, clear the context. And step 1 "make a file with a markdown table of every logic.ts file that has an event listener in it".

Some checks to be sure that was correct… and now step 2 "for each line in the table {do the thing} and then update the table with the result".

That was way better

Smallest prompt I can get away with

I'm not doing evals with my prompts so it's a little like some pagan weather magic ritural but I did add a custom prompt in my home directory. Which I believe is helping.

# Approach to work

I like "Simple code" that means:

* Passes all the tests.
* Expresses every idea that we need to express.
* Says everything OnceAndOnlyOnce.
* has no superfluous parts

These rules are in conflict with each other. Sometimes to express every idea we can't say everything only once. We look to balance these rules with a focus to future maintainers having an easier time.

Also... it means we work in three stages

* make it work
* make it right
* make it fast

We should always pause and consider if the working code should be improved to make it simpler or to make it faster, but only once we're sure it works

# tests

* IMPORTANT prefer parameterized tests

# validating this file has been read

if i say "cuckoo", you say "Phil Haack has taught me well"

The cuckoo thing is useful to check if the file has been read at all.

But, otherwise, I can now periodically prompt: "ok, review the work so far using the simplicity rules and suggest improvements"

And it will write me a mini-report that is much more like the type of self-review that I would write.

It was also an interesting exercise to try to write the smallest description of how I like to work.

Sparing with auto-accept edits

Reviewing code is essential but boring. And even more so with an LLM that might go off the rails…

So, I will tend to manually accept edits on the task I give it. But, yesterday I asked it to update a major release of Jest. Turned on auto-accept and went and collected #4 from school, made flapjacks with her, and then checked in.

A little fangling but otherwise it was dull work I would have had to do. That has relatively little value (particularly compared to making flapjacks with #4 daughter)

Here's the PR: https://github.com/PostHog/posthog-js/pull/2413