Every time I’m close to ship a new product, I do a retrospective on myself and the top 10 things I’ve learn as a technical leader.
For now, the summary of things I learned are:
- Dependencies is where shit happens
- Do not wait to go full continuous deployment
- Building a new service is not free
- Always be customer obsessed
- Have a vision and share it to your team
- Do not increase too much the complexity of your data-flow because of ownership
- Go in beta AS SOON as possible
- Create a “sense of urgency”
- As a lead, do not drive by the date but by the scope
- Keep in mind that the last meters (to ship) can be long.
- Organization structure is truly important
- If you are manually deploying, do it often
- Invest in testing and deployments processes before starting the project
- Don’t increase the technical debt, even if you think it’s not worth it
- Work on logs, metrics, alarms and system recovery as you go
- The perfect scrum team size is 4
- PoC (Proof of Concept) beats long design documents
- Have clear expectation for your “oncall” operator
I’m adding 10 new learnings on top of this list today:
- Organization’s structure should follow architectural structure
I’ve learn in the past that ownership matter, but only if it’s an ownership that make sense.
“Conway’s Law is an adage named after computer programmer Melvin Conway, who introduced the idea in 1968. It concerns the structure of organizations and the corresponding structure of systems (particularly computer software) designed by those organizations. ”
Consider a large system S. We hire a company X to build system S.
Say company X has three engineering groups, E1, E2, and E3 that participate in the project.
Conway’s law suggests that it is likely that the resultant system will consist of 3 major subsystems (S1, S2, S3), each built by one of the engineering groups. More importantly, the resultant interfaces between the subsystems (S1-S2, S1-S3, etc) will reflect the quality and nature of the real-world interpersonal communications between the respective engineering groups (E1-E2, E1-E3, etc). more info.
- Create a set of user stories at the beginning is mandatory
I was doing a “like-to-like” migration where I didn’t have a PM to help me define the requirements. I thoughts it wasn’t a problem because we were just doing what the existing software was doing, so I did a spec at the macro-level.
I was wrong and missed a lot of features. Some of them needed to be deprecated or were hidden for a simple user.
- Scrum doesn’t help in all situations
I LOVE Scrum, I did Scrum the past 5 years of my life. But I have to say: Scrum doesn’t answer to all problems. A methodology depends on a lot of things, especially the kind of leaders and the kind of projects you have. It exists others methodologies and you need to revisit how you do software really often.
- You need strong senior IC to back you up
Having a team of junior people is not a big problem. Especially if the “core work” you are doing doesn’t require a 15y+ super senior Software Engineer.
At least, that’s what I thought. As a tech lead, you will do mistakes, and you will be happy that another technical leader catches your mistakes and avoid you to lower your own bar. Having at least a second pair of eyes is worth every penny.
- Being territorial is wrong
Being in a big company allow you to not have to worry about short term funding. Not having the knife under my chin allowed me to not be too scare if my product takes a delay because one of my partner is late.
But at the end, I realized we are on the same boat and the customer doesn’t care if my team completed the work on-time. What the customer care about if having a new product in hands.
- Take care of performance and security as you go
I learned in the past that exception recovery and operation readiness needs to be done as we develop the new features. That way it is more granular and easy to follow the progress of the project.
This is also true for two things: Security and Performance.
At the beginning of the project you need to set the expected SLA for everything you develop and see how the performance degrades over time. Performance should be seen like a budget you can consume, spend and keep track of.
Security should be part of the team culture; the engineers needs to ask themselves the right question. A penetration test will not be enough to catch all the flaws.
- Full deployment has a high cost
I know full CD is the right thing to do, but I understood the high cost of it. It is not a magical solution (but I strongly advice it).
Before, manual deployment where done thought CM (Changement Management Process) with an operator doing a set of sanity check before deployment.
Being full CD moves this workload during the Code Review process (or just after). As I explained in this article, every single move from an engineer means a huge part of operation readiness associated with it. And ultimately the overall sentiment engineers shared is that we are doing mostly operation work instead of writing new code, even if we were building a brand new V1 project.
- Use your own product more often
It is a mistake I keep doing again and again. I focus on the code, the graphs/metrics/alarms, the technical debt but I don’t pay enough attention to the product until it’s late. As a tech lead, I need to daily use my product and see it changing day after day to have a sense of the advancement, the overall fit ‘n’ finish quality, and the stability of it.
- Stay strong when someone ask you to refactor
You know this feeling when a developer asks you a small refactoring and you think it is not the time/worth the investment? And even if you say no, he/she keeps insisting on doing it and you finally let go? DO NOT LET GO!!! If you keep your bar high in each feature you develop, the only huge piece of refactoring should be in legacy code. If this is true, you will be able to pay the technical debt back when you will have to touch this part of the code. You will reduce the debt little by little. Don’t create a task where the goal is to refactor a piece of software. Always tight it to a business needs with a business achievement behind it.
- Rebuilding a software will cost you as much time as the first time
I’ve read enough book and had enough experience in rebuilding software to say two things: 1/ it is rarely a good idea to do so, and 2/ no matter how much you learn from your mistakes the first time, you will need as much time to rebuild it. New stack, new code is equal to new set of problems and thinking it will be shorter is lying to yourself.