Software development process models

As a software architect, it is important for you to understand some of the common development processes that are currently used in most enterprises. A software development process defines how people in a team produce and deliver software. In general, this process relates to a software engineering theory called the software development process model. Ever since software development was first defined as an engineering process, many process models for developing software have been proposed. Let us review the traditional software models, and then look at the agile ones that are currently common.

Reviewing traditional software development process models

Some of the models introduced in software engineering theory are already considered traditional and obsolete. This book does not aim to cover all of them, but here, we will give a brief explanation of the ones that are still used in some companies – the waterfall and incremental models.

Understanding the waterfall model principles

This topic may appear strange in a software architecture book from 2023, but yes, you may still find companies where the most traditional software process model remains the guideline for software development. This process executes all fundamental tasks in sequence. Any software development project consists of the following steps:

• Requirements: where a product requirements document is created, and it is the basis for the software development process
• Design: where the software architecture is developed according to the requirements
• Implementation: where the software is programmed
• Verification: where tests are performed in the application
• Maintenance: where the cycle starts again after delivery

Let us look at a diagrammatic representation of this:

Figure 1.3: The waterfall development cycle (https://en.wikipedia.org/wiki/Waterfall_model)

Often, the use of waterfall models causes problems such as delays in the delivery of a functional version of the software and user dissatisfaction due to the distance between expectations and the final product delivered. Besides, in my experience, having application tests start only after the completion of development always feels terribly stressful.

Analyzing the incremental model

Incremental development is an approach that tries to overcome the biggest problem of the waterfall model: the user can test the solution only at the end of the project. The idea of a model following this approach is to give the users opportunities to interact with the solution as early as possible so that they can give useful feedback, which will help during the development of the software.

Figure 1.4: The incremental development cycle (https://en.wikipedia.org/wiki/Incremental_build_model)

The incremental model presented in the preceding picture was introduced as an alternative to the waterfall approach. The idea of the model is to run for each increment a set of practices related to software development (communication, planning, modeling, construction, and deployment). Although it mitigated problems related to the lack of communication with the customer, fewer increments were still a problem for big projects because the increments were still too long.

When the incremental approach was used on a large scale – mainly at the end of the last century – many problems related to project bureaucracy were reported, due to the large amount of documentation required. This clunky scenario caused the rise of a very important movement in the software development industry – agile.

Understanding agile software development process models

At the beginning of this century, developing software was considered one of the most chaotic activities in engineering. The percentage of software projects that failed was incredibly high, and this fact proved the need for a different approach to deal with the flexibility required by software development projects.

In 2001, the Agile Manifesto was introduced to the world, and from that time forward various agile process models were proposed. Some of them have survived up until now and are still very common.

One of the biggest differences between agile models and traditional models is the way developers interact with the customer. The message behind all agile models is that the faster you deliver software to the user, the better. This idea is sometimes confusing for software developers who understand this as – Let’s try coding, and that’s all, folks!

However, there is an important observation of the Agile Manifesto that many people do not read when they start working with agile:

Figure 1.5: Manifesto for agile software development

A software architect always needs to remember this. Agile processes do not mean a lack of discipline. Moreover, when you use the agile process, you will quickly understand that there is no way to develop good software without discipline. On the other hand, as a software architect, you need to understand that soft means flexibility. A software project that refuses to be flexible tends to ruin itself over time.

The 12 principles behind agile are foundational to this flexible approach:

1. Continuously delivering valuable software to satisfy the customer must be the highest priority of any developer.
2. Changing requirements needs to be understood as an opportunity to make the customer more competitive.
3. Use a weekly timescale to deliver software.
4. A software team must be composed of businesspeople and developers.
5. A software team needs to be trusted and should have the correct environment to get the project done.
6. The best way to communicate with a software team is face to face.
7. You can see the greatest software team achievement when the software is really working in production.
8. Agile works properly when it delivers sustainable development.
9. The more you invest in techniques and good design, the more agile you are.
10. Simplicity is essential.
11. The more self-organized the teams are, the better-quality delivery you will have.
12. Software teams tend to improve their behavior from time to time, analyzing and adjusting their process.

Even 20 years after the launch of the Agile Manifesto, its importance and connection to the current needs of software teams remain intact. Certainly, there are many companies where this approach is not fully accepted, but as a software architect, you should understand this as an opportunity to transform practices and evolve the team you are working with.

There are many techniques and models that were presented to the software community with the agile approach. The next subsections will discuss lean software development, extreme programming, and Scrum, so that you can decide as a software architect which ones you might use to improve your software delivery.

Lean software development

After the Agile Manifesto, the approach of lean software development was introduced to the community as an adaptation of a well-known movement in automobile engineering, Toyota’s model for building cars. The lean manufacturing method delivers a high level of quality even with few resources.

Mary and Tom Poppendieck mapped seven lean principles for software development, really connected to agile and the approach of many companies of this century, which are listed here:

• Eliminate waste: You may consider waste to be anything that will interfere with the delivery of the real need of the customer.
• Build quality in: An organization that wants to guarantee quality needs to promote it in processes from the very beginning, instead of only considering it when code is being tested.
• Create knowledge: All companies that have achieved excellence have a common pattern of generating new knowledge by disciplined experimentation, documenting that knowledge, and guaranteeing that it is spread all over the organization.
• Defer commitment: Plan decisions at the latest possible moment without causing damage to the project.
• Deliver fast: The faster you deliver software, the more elimination of waste you have. Companies that compete using time frequency have significant advantages over their competitors.
• Respect people: Giving reasonable objectives to the team, together with plans that will guide them to self-organize their routine, is a matter of respecting the people that you work with.
• Optimize the whole: A lean company improves the cycle of value; from the moment it receives a new requirement to the point at which it delivers the software.

Following the lean principles helps a team or company to improve the quality of the features that are delivered to the customer. It also creates a reduction in time spent on features that will not be used by the customer. In lean, deciding the features that are important to the customer guides the team in delivering software that matters, and this is exactly what the Agile Manifesto intends to promote in software teams.

Extreme programming

Just before the release of the Agile Manifesto, some of the participants who designed the document, especially Kent Beck, presented to the world the extreme programming (XP) methodology for developing software.

XP is based on the values of simplicity, communication, feedback, respect, and courage. It was considered later as a social change in programming, according to Beck in his second book about the topic. It certainly promotes a huge change in the flow of development.

XP states that every team should simply do only what it was asked to do, communicating face to face daily, demonstrating the software early to get feedback, respecting the expertise of each member of the team, and having the courage to tell the truth about progress and estimates, considering the team’s work as a whole.

XP also delivers a set of rules. These rules may be changed by the team if they detect something is not working properly, but it is important to always maintain the values of the methodology.

These rules are divided into planning, managing, designing, coding, and testing. Don Wells has mapped XP at http://www.extremeprogramming.org/. Although some of the ideas of the methodology were criticized strongly by many companies and specialists, there are many good practices that are still used nowadays:

• Writing software requirements using user stories: User stories are considered an agile approach to describing user needs, together with acceptance tests, which are used to guarantee the correct implementation.
• Divide software into iterations and deliver small releases: The practice of iterating in software development is implemented by all methodologies aside from waterfall. The fact of delivering faster versions decreases the risk of not meeting the customer’s expectations.
• Avoid working overtime and guarantee a sustainable velocity: Although this must be one of the hardest tasks a software architect may deal with, overtime indicates that something is not working properly in the process.
• Keep things simple: While developing solutions, it is quite common to try to anticipate features that the customer would like to have. This approach increases the complexity of the development and the time to market the solution. A different approach will cause high costs, and probably a low level of features that are used in the system you are developing.
• Refactoring: The approach of refactoring the code continuously is good because it enables the evolution of your software and guarantees the design improvement that will truly be necessary due to the normal technical changes of the platforms you use to develop.
• Keep the customer always available: If you follow XP, you should have an expert customer inside your team. This is certainly something that is hard to do, but the main idea of this approach is to guarantee that the customer is involved in all parts of development. As another bonus, having the customer close to your team means they understand the difficulties and expertise the team has, enabling an increase in trust between the parties.
• Continuous integration: This practice is one of the bases of the current DevOps approach. The less difference you have between your personal code repository and the main code repository, the better.
• Code the unit test first: A unit test is an approach where you program specific code for testing a single unit (class/method) of your project. This is discussed in a current development methodology called Test-Driven Development (TDD). The main goal here is to guarantee that every business rule has its own unit test case.
• Code must be written to agreed standards: The need to determine standards for coding is connected to the idea that no matter which developer you have working on a specific part of the project, the code must be written so that any of them will understand it.
• Pair programming: Pair programming is another difficult approach to achieve in every single minute of a software project, but the technique itself – one programmer coding and the other actively observing and offering comments, criticism, and advice – is useful in critical scenarios.
• Acceptance tests: The adoption of acceptance tests to meet user stories is a good way to guarantee that newly released versions of the software do not cause damage to its current needs. An even better option is to have these acceptance tests automated.

It is worth mentioning that many of these rules are today considered vital practices in different software development methodologies, including DevOps and Scrum. We will discuss DevOps later in this book, in Chapter 8, Understanding DevOps Principles and CI/CD. Let us get into the Scrum model right now.

Getting into the Scrum model

Scrum is an agile model for the management of software development projects. The model comes from lean principles and is one of the more widely used approaches for developing software nowadays.

As you can see in the following figure, the basis of Scrum is that you have a flexible backlog of user requirements (Product Backlog) that needs to be discussed in each agile cycle, called a Sprint. The Sprint goal (Sprint Backlog) is determined by the Scrum Team, composed of the Product Owner, the Scrum Master, and the Development Team. The Product Owner is responsible for prioritizing what will be delivered in that Sprint. During the Sprint, this person will help the team to develop the required features. The person who leads the team in the Scrum process is called the Scrum Master. All the meetings and processes are conducted by this person.

Figure 1.6: The Scrum process

It is common to apply Scrum together with another agile technique, called Kanban, also developed by Toyota for manufacturing cars and commonly used for software maintenance. The main purpose of Kanban is to enable a visual system to make sure everybody understands what is going on in the product that is being developed. The famous Kanban board is an incredible way to do so, where you define what the team must do, what they are doing, and the things that are already done.

It is important to note that the Scrum process does not discuss how the software needs to be implemented, nor which activities will be done. Again, you must remember the basis of software development, discussed at the beginning of this chapter; Scrum needs to be implemented together with a process model. DevOps is one of the approaches that may help you use a software development process model together with Scrum. Check out Chapter 8, Understanding DevOps Principles and CI/CD, to understand it better.

Scaling agile throughout a company

Today it is quite common to find companies where agility is being practiced and evolving in a good way, considering the results of the techniques presented in the previous sections. The mixture of Scrum, Kanban, and XP, together with the evolution of the maturity of the software development process, has brought good results for companies and we have a world where software development is one of the key strategies for the success of a business.

Some companies naturally need to scale up the number of teams, but the important question in this process is how to evolve without missing agility. And you can be sure that this question may be addressed to you, as a software architect. You may find in SAFe® – Scaled Agile Framework a good answer to this question:

SAFe® for LeanEnterprises is a knowledge base of proven, integrated principles, practices, and competencies for achieving business agility using Lean, Agile, and DevOps.”

– Dean Leffingwell, creator.

© Scaled Agile, Inc.

Based on the core values of alignment, built-in quality, transparency, and program execution, the framework provides a detailed path for delivering products with the agility needed in companies where you have one or more value streams. Its principles enable agility and incremental delivery, system thinking, fast and economic decisions, and mainly, organization around value.

As a software architect, you may find opportunities for growth, considering you can work as a software architect in a system team, a system architect in an agile release train, or even an enterprise architect in the company. For sure, this will require a lot of studying and dedication, but this structure is what you will find in big companies.

As with every framework, technique, or model that you will find in this book, the purpose of presenting SAFe to you is not to cover every single detail of the content. You will find excellent material and training on their website. But as a software architect, understanding how to scale up a company may be good knowledge to have in your toolbelt! Now that you know it, let us go back to the stages of designing software with high quality, discussing how to gather the right information to design it.