Bedtime Routines – Types of Development Work
By Tim Donlan
Having four kids, one of the most challenging parts of my day is bedtime. I have to wrangle four excited, rampaging, sweet talking tyrants into their own rooms, PJs, beds and finally dreaming their own dreams. I’m not alone in this daily task, which often feels as insurmountable and exhausting as summiting Everest.
But some techniques have come in handy, honed over many years of trial and error. And the interesting thing is how many of these techniques can be applied to the world of work: developing solutions for customers, shipping a product or building a team.
Bath Time: The Batch Process
It’s well known that kids are gross. Unlike adults who generally need a shower in the mornings to wash away the fog of sleep and become presentable. With kids it’s the opposite. It’s necessary to wash away the grime of the day before tucking them into their freshly laundered Minecraft sheets. Baths also have a constraint: the limited supply of hot water. Pity the poor wretch last in line who has to wash shampoo out with cold water!
The solution: the batch process! Throw them all in the bath at once. They can share the hot water, the bubble bath, the rubber duckies and plastic submarines. I can sit calmly, scrolling through Slack messages, making sure they don’t splash too hard and overflow the sides.
Batch processes are pretty ancient when it comes to software engineering. Back in the day, most jobs had to be carefully grouped together and handed off to the mainframe to run overnight. Interestingly enough, with the rise of queue-based systems and distributed processing, the old school batch process has gone out of style. From a speed perspective, some jobs just make sense to properly group up in a data structure and crunch through in a single process. Kill n birds with one stone.
Of course, beware overflowing the tub: too much data for a single CPU. It’s good to finish up in under 30 minutes, so a human can keep an eye on it. Patientco runs some of our core business processes in a single batch job: grouping daily payments to build an ACH file. One of the additional benefits is the batch process can be wrapped in a database transaction, allowing for rollback if the final state isn’t properly saved and validated.
What kind of teamwork falls under the batch process analogy? Weekly Ticket Grooming Sessions and Retros are a great way to take a look at the entire backlog and get a holistic view of the work to be done. Daily standups can also be a bit like a daily bath, to clear the air and share dirty truths. Just be sure to have good rinse afterwards!
Brushing Teeth: The Assembly Line
Onto the next one: Brushing Teeth! The optimal way to brush all four kid’s teeth is to stand at the sink, the brushes arrayed in a colorful fan, the minty blue Avengers Crest uncapped and at the ready. The first kid steps up, opens wide and I can scrub those pearly whites in sixty seconds. Rinse, spit, wipe, repeat! It’s an assembly line, sequential process.
The key to a good assembly line is for each station to have all supplies ready and every action down to an optimized science. This has been well understood since the days of Henry Ford. However, building software is not assembling Model Ts.
The assembly line motif has been applied to both software processes and organizational processes, and in fact, many times over-applied. Many jobs are broken into individual messages, which flow through a queue and are worked by a consumer. The problem is when the work for a single message isn’t uniform with all the others. It errors out, takes longer due to database or resource contention, or perhaps the message is malformed. A well designed system will throw the bad message onto a dead letter queue and alert an engineer. But what if it hangs and all the consumers are tied up? It’s like when my two year old decides to bite down on his Mickey Mouse toothbrush and wont listen to reason. Everyone else is left waiting.
At Patientco, we have monitoring on our queues to alert for dead letters, and also check throughput. If queue volume gets too high, or messages are sitting on the queue for too long, it’s a sign that consumers are getting clogged up.
What about ticket tracking systems, like Jira? Tickets are written, dragged onto the main board, and start down the assembly line of development. However, tickets on a Jira board are not one size fits all. Some may be quite simple: fix a typo, add a bit of config for a new client. Others amount to building an MVP in a few days. Some teams get around this with ticket ranking: bid on the difficulty of a ticket. A 3 or 5 point ticket may take days, but you can crank through .25 point tickets all afternoon without breaking a sweat. One solution: if a developer is feeling productive but less than brilliant – grab a few of the low pointers and crank through them. But is that fair to the others who are now stuck with the 10-point Refactor the Business Layer Dependency Injector ticket? Something to keep in mind when shoehorning processes into assembly lines.
Bedtime Story: TLC
After they’re bathed, teeth brushed, dressed in PJs and the chaos of their rooms temporarily tamed, it’s time for a bedtime story. Sometimes, a batch process will do here: Reading Harry Potter to the older kids, or a special evening of rare family camaraderie (‘Twas the Night before Christmas). But usually every kid wants their own story. The older boys crave the adventure of wizards and warriors. My daughter Princess or Paris themed. And the two year old just wants to point at dinosaurs and make roaring noises. So each child gets some individual attention as I escort them closer to the land of nod.
Some problems just need TLC. I’m thinking client ‘Go-Lives’ and implementations. Sometimes you just need to get on the call and crunch through it with a befuddled IT person on the other line. Same thing with software engineering. Not all jobs or processes fit into the standard pattern and framework. Sometimes it is actually faster and more efficient to write a custom process tailored for one job or even client. (Sacrilege!)
Perhaps if I had two dozen kids (or a thousand) I would think differently. The various nuances of the bedtime routine would then need to be analyzed by committee and normalized into a Uniform Nighttime Process where each kid was read to by the digitized reading assistant the requisite 12.5 minutes of age appropriate literature. Asides about death, the universe and farts are strictly prohibited.
The essence of the MVP is all about TLC: focusing directly on the problem at hand and doing what needs to be done, no more and no less. Patientco Engineering schedules quarterly Hack Weeks, where a group of engineers tackle a novel problem. Sometimes solutions utilize existing tech stacks (message queues or batch patterns), others are completely new. The unifying factor is focused attention to solve a single problem. A little TLC goes a long way.
So that’s it. The kids are in bed. Hopefully they stay there. Time for a well earned drink, cozy up on the couch, maybe check out Netflix or a new book on Kindle.
And then PagerDuty blares. My angriest and often most immature child of all is demanding attention. The duties of a parent never cease…
To hear from us from time to time, subscribe below.