Practical Principles for Engineering Teams

Through­out my soft­ware engin­eer­ing career, I’ve had the good for­tune to be part of some great teams from which I’ve learnt loads. It’s been great to see what char­ac­ter­ist­ics work and don’t work, what’s hard and what’s easy.

The key fea­ture of high-pro­­duct­iv­ity teams that I’ve seen is a good push-and-pull dynam­ic between prag­mat­ism and ideal­ism. Full-throttle prag­mat­ism res­ults in a con­tinu­al pro­cess of dirty, tac­tic­al hacks that slows any tech organ­isa­tion right down even­tu­ally. On the oth­er hand, over-ideal­ist­ic teams who strive for The Per­fect Code won’t deliv­er much value and will just exist to serve themselves.

So, here’s a few tips that I believe will help lay the found­a­tions for keep­ing engin­eer­ing teams productive.

Be busi­ness build­ers, not coders #

Engin­eers receive many titles (sev­er­al being self-appoin­ted): ​“Tech­ies”, ​“Code mon­keys”, ​“Devs”, ​“Nerds”. It’s an iden­tity most of us are used to and one some of us even quite like. But the issue is that it can eas­ily put engin­eers in a box, that their respons­ib­il­ity is solely to stay in their lane. Write code and let the busi­ness folk do the busi­ness work.

It’s a setup that many embrace, for many reas­ons. A love of pro­gram­ming and a dis­in­terest in how com­pan­ies work can make the pro­spect of cod­ing, and just cod­ing, all day long very attractive.

But if you’re just writ­ing code for the sake of it, then the oppor­tun­ity for adding value is slim. You may feel pro­duct­ive, but may not be pro­du­cing value. You may be writ­ing loads of tests, get­ting your infra­struc­ture just right, and yet not be productive.

Many of us for­get the reas­on employ­ers hire us isn’t to write code, but to grow their busi­ness using tech­no­logy. To do that, you have to under­stand how your employ­er makes money, what costs are incurred, what’s import­ant to users, etc. Your pri­or­ity is to build busi­nesses, not your code­base, so take a look at the chal­lenges you face as a com­pany and use your tech­nic­al skill­set to address them.

Build strong team val­ues #

“Teams are great­er than the sum of their parts”

A team which is built to col­lab­or­ate closely, sup­port each oth­er and level-up it’s mem­bers is much more pro­duct­ive than were those indi­vidu­als act­ing in isol­a­tion. But good teams aren’t cre­ated by simply sit­ting people next to each oth­er and let­ting the magic hap­pen, it requires care­ful nurturing.

Build the right cul­ture #

Build­ing teams and fos­ter­ing cul­ture is tricky. To make a team, you need to win over mem­bers to the idea that everyone’s there to help them and vice versa. Instilling the right val­ues and atti­tudes to get that buy-in is the first step.

• Trust — Build­ing trust and cohe­sion between team mem­bers is key. If someone on your team was unre­li­able and nev­er upheld their respons­ib­il­it­ies, then you likely wouldn’t trust them. You’d be less inclined to col­lab­or­ate with them and wouldn’t put as much weight in their opin­ions. If you sac­ri­ficed time to help them out, but they were con­sist­ently unwill­ing to recip­roc­ate, then trust is dam­aged and the team suf­fers as a result.
• Empathy — Being human is some­thing every­one has to deal with (for bet­ter or worse!). When things go wrong and people need help, it’s import­ant to empath­ise and sup­port them where you can. 
• Pos­it­iv­ity — Make enthu­si­asm, praise and grat­it­ude corner­stones of your cul­ture. Per­petu­al neg­at­iv­ity in a team has the power to kill it, it’s mem­bers sent scram­bling for a more pleas­ant envir­on­ment, in anoth­er team or even anoth­er company.

Suc­ceed and fail as a team #

When teams win, they should win togeth­er. Reward­ing spe­cif­ic people, des­pite efforts from every­one in the team can cause ten­sions and rifts.

Exactly the same can be said for when teams screw up, attrib­ut­ing blame and pun­ish­ing spe­cif­ic people for what was a team fail­ure also cre­ates dis­har­mony. But is that fair for team-mem­bers who were only tan­gen­tially involved in a failed pro­ject? Who did their part flaw­lessly and were let down by the actions of others?

Yes.

A team should act as one, and if it isn’t, it’s fail­ing. It’s everyone’s respons­ib­il­ity to steer a pro­ject back on course should they see it going off a cliff. To act as one, it’s import­ant to allow input from all mem­bers on all aspects of a fea­ture and to action the input where appro­pri­ate. Fail to do this and no will both­er giv­ing input and tak­ing respons­ib­il­ity because they know they’re going to be ignored. Everyone’s focus will return to their sole respons­ib­il­it­ies and col­lab­or­a­tion will suffer.

When pro­jects become team respons­ib­il­it­ies, the dia­logue shifts from​“I”-centric to​“we”-centric. It goes from​“What do I need to do?” to​“what do we need to do?”. It builds col­lab­or­a­tion, camarader­ie and the need to sup­port every­one on the team, par­tic­u­larly those who may be struggling.

Always be shar­ing know­ledge #

When teams get to a cer­tain size, it’s hard for every­one to know everything, inform­a­tion stops flow­ing​“by osmos­is”. Besides, know­ing everything isn’t usu­ally neces­sary or desir­able, know­ing enough is the tar­get. When teams don’t know enough mis­takes start to hap­pen and effort starts being wasted.

For engin­eer­ing teams, I believe there’s sev­er­al use­ful prac­tices that when adop­ted can go a long way to keep­ing teams well-informed:

• Stan­dups — Short daily catch-ups (10−15 minutes max) are a good way to share high-level updates with team mem­bers. If someone decides a par­tic­u­lar update is dir­ectly rel­ev­ant to some­thing they’re work­ing on and they require more detail, they can have a more inde­pth con­ver­sa­tion out­side of the standup.
• Pair-pro­gram­ming — Pair pro­gram­ming is a fairly con­tro­ver­sial prac­tice whereby two developers work on a single task togeth­er on the same machine. It’s oth­er mer­its and faults aside, it’s a great tool for shar­ing code­base know­ledge across engin­eers. When employed on crit­ic­al fea­tures it really shines. It guar­an­tees at least two engin­eers have inde­pth know­ledge of how a fea­ture func­tions, so should one author be off-sick and sud­denly a pro­duc­tion issue erupts, there’s anoth­er on hand to resolve it.
• Code reviews — These are great oppor­tun­it­ies for engin­eers to get an eyes-on look at code that their peers are writ­ing. Should in the future, they be required to update it, they’re already famil­i­ar with it. It’s also a prime oppor­tun­ity for juni­or engin­eers and new-join­ers to get famil­i­ar with team devel­op­ment prac­tices. Code-reviews aren’t simply a gate­keep­ing prac­tice, they’re an effect­ive meth­od to share know­ledge around teams.

Make it easy to do the right thing #

This is one of my favour­ite principles.

Expan­ded on a little, it means that in the code we write, the sys­tems we archi­tect, the chan­nels of com­mu­nic­a­tion we build, it should be obvi­ous and require less effort for people with lim­ited con­text to behave in a way that’s desir­able. Often team mem­bers go off-piste and start caus­ing chaos because they didn’t prop­erly under­stand the rules that had been agreed by oth­ers beforehand.

Take MVC applic­a­tion frame­works, for example. By estab­lish­ing con­ven­tions and being con­sist­ent, it’s easy for a new engin­eer who has lim­ited know­ledge of a Ruby on Rails code­base, but exper­i­ence with MVC, to start writ­ing Rails code quickly because they know the rules of the game. In oth­er words, using a com­­monly-under­­stood archi­tec­ture makes it easy for the engin­eer to do the right thing. It’s easy to write code in the right way, in the right place.

Doing things this way avoids, in a very nat­ur­al way, the need for doc­u­ment­a­tion and act­ive edu­ca­tion. People just get it and get on with their job.

Adopt Con­tinu­ous Deliv­ery #

Get­ting your team setup with a Con­tinu­ous Deliv­ery (CD) pipeline is an import­ant tool in main­tain­ing pro­ductiv­ity for sev­er­al reasons.

High through­put, high momentum #

Momentum is about cap­it­al­ising on the enthu­si­asm and buzz gen­er­ated by deliv­er­ing value and chan­nel­ing it into deliv­er­ing more value.

A good deliv­ery pipeline is one that enables a team to keep up momentum by facil­it­at­ing mul­tiple releases every day. When release through­put is monthly, fort­nightly or even weekly, momentum can quickly die. We settle in, we say​“we’ve not shipped any­thing for two weeks, what’s a week more?”. RIP productivity.💀

There are some val­id chal­lenges to keep­ing up momentum. For example, slow manu­al pro­cesses imposed by Apple and Google cre­ates prob­lems for the​“release often” man­tra for mobile app engin­eers. Des­pite these bar­ri­ers, every effort should be taken to ship as often as is feas­ible with­in the imposed constraints.

Release early #

By com­mit­ting to ship­ping early, it forces us to think about fea­tures in their simplest forms so that we can deliv­er value as soon as pos­sible. It make us answer import­ant ques­tions early on, such as:

• “What hoops need jump­ing through before this fea­ture can be shipped?”
• “What’s the min­im­um level of func­tion­al­ity we’re com­fort­able ship­ping?” Front-load­ing these ques­tions avoids them being asked at the last-minute, caus­ing pre­vent­able delays.

Release often #

By ship­ping often, it forces us into chunking fea­tures into mul­tiple small releases, which reduces the com­plex­ity and risk of each one.

It reduces ser­vice out­ages caused by bad releases and when pro­duc­tion issues do arise, they are easi­er to dia­gnose and resolve because of the smal­ler sur­face area involved in the release.

Lower release com­plex­ity reduces the like­li­hood of dif­fi­cult roll­backs that can dam­age team momentum and cause fea­tures to lan­guish longer than they should.

Pipeline dash­board #

Talk­ing more tac­tic­ally for a moment, hav­ing a dash­board vis­ible to engin­eers which visu­al­ises the cur­rent state of your deploy­ment pipeline is anoth­er per­fect example of mak­ing it easy to do the right thing.

• It gives every engin­eer an up-to-date per­spect­ive on what is cur­rently being shipped, so they can make bet­ter decisions on how to sched­ule their release and who they need to coordin­ate with. Less time is wasted on communication.
• It gives a clear pic­ture of the cur­rent state of each envir­on­ment (i.e. sta­ging, pro­duc­tion). Every­one is clear about what code is where.
• Should an issue arise while an update is mak­ing its way through the pipeline (e.g. a test fails), it’s soon appar­ent. The respons­ible engin­eer can quickly address it and keep things flowing.

Test (pro­por­tion­ally) #

When in product-based engin­eer­ing teams, auto­mated test­ing is unques­tion­ably a good idea. It requires up-front effort, but it’s my belief that long-term, it speeds devel­op­ment up by identi­fy­ing regres­sions straight away. By integ­rat­ing into deliv­ery pipelines, they improve ser­vice reli­ab­il­ity by block­ing bad releases from reach­ing pro­duc­tion. For a siz­able product, not hav­ing a basic test suite which runs through crit­ic­al user paths is a dan­ger­ous and fright­en­ing prospect.

But taken too far in the oppos­ite dir­ec­tion is equally as dan­ger­ous. Over-test­ing can waste devel­op­ment time on niche edge-cases and cause head­aches in the future when minor refact­ors and fea­tures cause hun­dreds of tests to fail. Days can be sunk into cor­rect­ing overly-cau­­tious tests.

A judge­ment call needs to be made when decid­ing how much to test. The key things to con­sider are:

• Import­ance — Is this fea­ture a crit­ic­al flow? If it were to break, how many users will be affected and what will be the impact to the busi­ness? Could import­ant data be lost forever?
• Dif­fi­culty — How hard is this fea­ture to test? Does the effort to write tests out­weigh the dam­age if it breaks?

Descope, descope, descope #

“Per­fec­tion is achieved, not when there is noth­ing more to add, but when there is noth­ing left to take away.” — Ant­oine de Saint Exupéry.

The per­fect fea­ture is one which deliv­ers max­im­um value with min­im­um effort. There­fore, giv­en a scope of work, if we can reduce the func­tion­al­ity required to deliv­er the desired value then we’re sav­ing time and effort. We’re being far more productive.

If your team takes a long time to deliv­er giant fea­tures and is miss­ing dead­lines, the solu­tion isn’t to work long hours and grind yourselves down. Try descop­ing your spec down to the abso­lute min­im­um required to achieve your tar­gets. This may require com­prom­ises, but to be prag­mat­ic is to be com­fort­able with com­prom­ises. Be care­ful not to throw the baby out with the bathwa­ter though, always keep one eye on the goal lest you com­prom­ise away all the value you set out to deliver.

Scope creep rab­bit holes #

Often engin­eers find them­selves down scope creep rab­bit holes. A rel­at­ively simple fea­ture turns ugly because the engin­eer decides this is required and that isn’t as it should be, etc, etc. Before you know it, this simple fea­ture has grown to twice the size and taken three times as long as it should have. I’ve done it plenty of times, most of us have.

The tricky thing is lift­ing your­self out of the hole once you’re down it, which is hard. Often the cruel sunk cost fal­lacy pushes us into dig­ging deep­er and deep­er. We need to recog­nise when it’s hap­pen­ing by ask­ing ourselves​“is this abso­lutely required?”. Cor­rect your path to one that deliv­ers the value you set out to deliv­er and noth­ing more. Descope, descope, descope!

Find a bal­ance with tech­nic­al debt #

When we make com­prom­ises with code, we incur tech­nic­al debt that needs to be paid off at a later date. I love the​“debt” ana­logy because it work so well.

People take out debt so they can make a big pur­chase (e.g. a car you need for your new job). They then gradu­ally, over time pay back that debt, with a bit of interest. If you’re not mak­ing those pay­ments, the interest is going to spir­al and you’re going to be in a state where every penny you earn is going towards pay­ing off that debt.

The exact same is true for tech­nic­al debt. You make a bodgey hack that goes against the estab­lished archi­tec­ture to quickly ship a big fea­ture. Tak­ing on the debt may have been the right call in the cir­cum­stances, get­ting the fea­ture shipped on time was the pri­or­ity. Six months later, the bodgey hack is still there, only now it’s been used as a tem­plate for sim­il­ar fea­tures by anoth­er engin­eer! Things can’t go on like this much longer, the time has finally come for the debt to be paid, but now with interest. Half the team downs tools to unpick the bodgey hack.

This scen­ario aside, nev­er incur­ring tech­nic­al debt can be just as bad. Over-pol­ish­ing things and mak­ing every fea­ture per­fect, regard­less of import­ance, is a total waste of time. Spend­ing hours and hours pol­ish­ing an exper­i­ment­al pro­to­type that is likely going to be binned isn’t worth the effort.

We just have to be sure that when we accu­mu­late tech­nic­al debt, we make a record of it and even­tu­ally pri­or­it­ise a fix. Not doing so will grind pro­ductiv­ity to a halt. Set up a pro­cess to peri­od­ic­ally sort the things that need address­ing most urgently, or bet­ter yet, fix them in future fea­tures that touch code con­tain­ing the tech­nic­al debt.

Sum­mary #

• Be busi­ness build­ers, not coders
• Build strong team values
• Make it easy to do the right thing
• Adopt Con­tinu­ous Delivery
• Test (pro­por­tion­ally)
• Descope, descope, descope
• Find a bal­ance with tech­nic­al debt