The calculation for when to automate is easy! We do this times per week, it takes time to complete, and automating it takes time, so the payoff time is weeks, right? There is even XKCD #1205 about it:
Is It Worth the Time?
This is the point where I bring out my inner 3-year-old, and say, “No.”
And yes, this means I am saying the venerable Randall Munroe is wrong. *
Why? Because this doesn’t take into account speed, nor supply and demand: by automating the process you make it both incredibly fast and incredibly cheap, so people will end up using it much more often than before. There is a long tail of value in most manual processes that isn’t captured because the delay and cost of them are relatively high. If you reduce the cost and turnaround time, suddenly that extra tail becomes economically viable and demand rapidly increases.
The ‘simple’ formula above means you should only automate if your implementation effort is less than the green block. I’m saying automate if it is less than the green plus orange block ***. Yeah, big difference.
Let’s take an example: manual QA vs. automating much of the same work into an automated test suite. Say a full-depth QA run takes a week, you run it a few times a year before a big release, and you decided to automate it**. If you then only run the test suite a few times a year, it probably wasn’t worth automating. Keep your manual processes, and follow XKCD’s guidelines above for the decision.
But if every engineer can now run the full test suite on every small change, the value is huge, because they get almost instant feedback on everything that they broke, and can adapt their design and continue with their work instead of waiting on QA. It would be cost-prohibitive to do full manual QA on every tiny code change. It would also result in engineers spending most of their time waiting on QA results. This entire workflow and all the value and improved delivery time is only unlocked by automation.
If you can then automate out a lot of the coding gruntwork with an agent, the ability to automate the validation side becomes even more useful.
And in this example we haven’t even touched the value of being confident that your emergency 3am bugfix didn’t break everything else.
The XKCD math is still useful, but only as a lower bound of how much value you can unlock by automating. Unless there’s really zero reason to do the task more frequently, or faster, the value will be higher than that, which means automating (potentially much, much) earlier.
P.S. This also applies to improvements of existing automated processes.
P.P.S. It can even go as far as running into Jevons Paradox: as something gets cheaper, you actually spend more money on it in total, because you use it so much more. In our case here, that could mean that the cost of running the automations might exceed the cost of the old manual processes due to the massively increased volume. You may also spend more money maintaining the automation than you spent doing the work manually before; if that unlocks sufficient value it can also be worth it.
* Based on his tone in the hover-text I give it a 50% chance he’s being sarcastic, so if you really push me I’ll settle for saying he’s 50% wrong. But even the careful wording of the text above the chart makes the assumption that you won’t do it more often once it is automated.
** QA has more important things to do than manually check the same stupid things repeatedly. Like trying to break stuff before your customers do.
*** Organizations generally get less productive per person as they get larger. So take that into account if you’re choosing a giant thing to automate and the payoff might only be in the distant future. But that applies to the simpler math as well.