A typical story in software development is the application that grows out of control. A client approaches the development team with a seemingly simple request. However, over time, the process of fulfilling that request gets more complicated. New considerations and questions of implementation mean the scope of the project begins to expand. From a client’s perspective, a project with the best intentions takes on an all-consuming life of its own.
It’s tempting to think that the complexity comes from hiring the wrong developer or poor project management. However, the reality is the complexity was there all along. At the start of the project, the client (and the development team) did a poor job of investigating the project’s depth. What seemed like a simple idea at the start actually involved more moving parts than the team first realized.
Peeling back those layers and magnifying those finer details at the beginning of a development project is the job of a good senior developer. I’ve seen it time and again when helping clients understand the scope of the applications they’re asking for. Let’s take a look at why projects are often more complicated than they seem.
Exploding Complexity
Kannan Chandrasegaran gives an excellent example of what he calls "exploding complexity" in developing web applications. As an example, he proposes a new application that matches users based on books they own and books they want to read. Users can trade, buy, or sell books from other users in a big library. Pretty simple, right?
Well, no. As Kannan explains, implementing something like this isn’t as simple as creating a forum for people to post books they have and books they’re looking for. We’ll probably want them to pay for the books in the app so we can take a cut, so we’ll need to integrate a payment processor. We’ll also want to group users geographically so we’ll need to support location services and mapping. Book titles should be standardized to make matching easier, so we’ll need a database of book titles. We’ll also want to implement a reputation system for users so we can weed out scammers and bad actors.
Kannan brings up other issues like how the books get delivered and whether you’ll want identity authentication. By now, what seemed like a simple application to match people looking for books with people who own books seems much more complicated. It didn’t become more complicated. It always was this complex, but we’re only realizing it by mapping the whole thing out.
Unknown Unknowns
Exploding complexity means the more you zoom in on the project, the more you realize you’ll need to do. Few projects are as simple to implement as they first sound. However, by scoping a project early, you can avoid the feeling that it’s running away from you. Scoping a project early usually also allows for cutting 30-50% of features from the get-go, before you ever build them. It also helps identify the core features and functionalities that need to go into a minimum viable product at the start of the process.
It’s key to realize, though, that projects will always have exploding complexity. Additionally, it’s often impossible to plan for this complexity. We don’t know what challenges we’ll encounter as we dig deeper. In general, on an agile project, a good 20% of the release plan’s budget should be held in reserve for contingency. Unknown unknowns: we don’t yet know what we don’t know about the project.
Scope Creep
Exploding complexity is different from scope creep. Scope creep is when a client asks for new features outside of the original project agreement. Exploding complexity deals with necessary components of the original plan that you didn’t realize you needed. It’s frustrating, but it’s not as infuriating for a dev team as a client who falls prey to scope creep.
Of course, agile methodology helps with scope creep to an extent. If we’re able to pivot based on market conditions without throwing away previous work, then changing requirements is less of an issue. The problem comes when requirements change, and we have to scrap work we’ve already done. Scope creep is also problematic when clients swap apples for oranges, asking for new features that create more or different work for our team than the original plan. In those cases, we have to renegotiate the project to make sure everyone is happy.
Conclusion
Proper planning and understanding that there will be contingencies can go a long way toward estimating project scope from the outset. As a result, clients and the development team feel like the project is on track, instead of feeling like the ground is shifting beneath them from week to week.