The Siren of Scalability

By Dan, January 8, 2010 3:41 am

Every CTO knows the old story – one moment you are in your office calmly reviewing the product backlog to prepare for yet another meeting and the next moment the schedule, the team and seemingly the world have been shattered and you are left feeling like Newton if his meditations had been interrupted by a concrete pylon rather than by an apple.  What has happened is that your star developer – your top producer – dropped by for a chat and told you that he’s getting a bit bored in his current role and would like to try something new – he’s thinking about becoming more of an architect.

It may seem innocent enough, but I call this problem the Siren of Scalability because, if wrongly handled, it will pull the organization, the developer and the product through the doldrums of non-productivity and onto the rocks of non-competitiveness.

This may seem like a gross overreaction, but think about it for a bit.  Everyone wants to grow, increase her skills, responsibilities and earning potential.  And it can usually be done in one of two ways – either becoming deeper in your chosen craft or by becoming a manager.  So each person thinks about what she could do to accomplish that.  In a startup there often isn’t a clear upward trajectory on the management track so that means your ambitious developer concentrates on perceived skills growth – the “hard” problems for which people get paid big bucks – scaling and architecture.

There are a few problems with this.  Only the very largest organizations really need more than one architect.  Imagine what your house would look like if was designed by a committee of architects.  Software can have the same problem.  It’s true that “architect” is actually a misnomer – what software architects due more resembled the tasks of a structural engineer that a true architect, but “software structural engineer” doesn’t have the same ring.  And you still don’t need more than one.

And if you believe the buzz about cloud computing – which in the long term is pretty hard to ignore – you actually may not need an architect at all.  If you build your system on a commercial or open source framework in the cloud, only the top few hundred websites are likely to hit scaling issues requiring this set of specialty skills.

What will always be useful, however, are top producers.  Alternative skills to hone are product and business-based since no one knows the product better than its developers and virtually every good developer is also a business analyst.  But that’s hard to explain when they come in with stars in their eyes and the Siren in their ears.

2 Responses to “The Siren of Scalability”

  1. Jeff Meyer says:

    I stumbled upon this blog by way of a series of random clicking events and found myself reliving a small set of questions during an interview with you a few years ago. Even while I answered them I knew they would be the last questions you would ask me during the interview… but I had to answer honestly. Essentially, you asked if I wanted to hone my skills and become a superstar developer or if I would eventually go the executive route.

    I can feel your pain from this post, but as I’ve personally grown as an entrepreneur I’ve found that helping others to achieve their goals generally leads to unforeseen success.

    Best of fortune in your decision on how to handle the situation. I look forward to your future posts.

  2. Dan Ratner says:

    I don’t disagree, Jeff. I think that helping others to achieve their goals is critical. What makes me sad in this case is that it seems clear to me that this particular set of goals is wrong and will ultimately make them less rather than more employable. The challenge is how to coach them in a direction that will be fulfilling for them and support the company’s needs as well. Scalability ain’t it.

    Thanks for posting and best of fortune to you as well.

Leave a Reply

Panorama Theme by Themocracy