Software Engineering - Process Models
Foundations of Software Engineering
The 4 Challenges of Software Development
- Complexity
- Requirements concerning functionality and efficiency are constantly growing
- Quality
- Software malfunction may cause injuries and economic losses
- A small change in a program can have huge impact on its behaviour (Discontinuity property of software)
- Human factors
- Managing software teams effectively is challenging
- Programmer productivity varies vastly from person to person
- Maintenance & Legacy Systems
- Long lifetime of software systems
- Software must be continuously adapted to new and changing requirements
Programming vs Software Engineering
Programming | Software Engineering | |
---|---|---|
Scale | Small | Huge |
Updates | Sequential | Parallel |
Life-span | Short-lived | Long-lived |
Error/Bugs | Small consequences | Large consequences |
Software Engineering mainly includes
- determining what to build
- organizing teams to cooperatively build systems
- lifecycle and process management
- analysis and testing
- software architecture
- system engineering
Software engineers deal with a very broad range of projects, kinds of systems, organizational settings, scale, criticality, domain expertise, etc.
Degree of Formality
Large projects with globally distributed teams require sufficient level of formality to communicate requirements, designs, and other aspects of development.
- Degree of formality in software process depends on project scale and domain
- Smaller projects with small teams (ten or less) at a single location don’t need the same degree of formality
Life Cycle and Process Models
Life Cycle: Life Cycle of software system.
- phases in the use of software system
Process Models: How do we perform the activities.
- activities (what it produces - work products; who performs - roles; when - dependencies; how - tools and methods used)
Life Cycle is the subset of Process Models
The orderings are:
- Requirements elicitation
- Specification
- Design
- Implementation
- Integration
- Maintenance phase
- Retirement
- …
Build and Fix model
Basically:
- No planning/up-front requirements at all (Chaotic Development)
Advantage:
- Appropriate for small programs written by one person
Disadvantage:
-
No requirements engineering happening, poor understanding of customer need
-
Lack of proper architecture and design
-
Lack of repeatable (automated) test
- Wrong system maybe built
- messy (no design) and buggy (no systemic V&V)
-
Understandability and maintainability decrease rapidly with increasing program size
Waterfall model
Basically:
- Sequential steps/phases with feedback loops between 2 phases
- Documentation-driven
- Assume each activity is carried to completion before moving to the next
Advantage:
- Lightweight, Easier maintenance (Follow logical flow)
- Documentation
Disadvantage:
- Customer involvement in the first phase only
- Product becomes available very late in the process (Risk of building wrong system due to no user feedback, Operational quality portion of product in the end)
- Only work best when there is a complete and frozen specification document
Key takeaway:
- High risk for new systems because of specification and design problems
- Low risk for well-understood developments using familiar technology
Rapid prototyping model
Basically:
- Waterfall model but no feedback loops
- frequent change, then discard (A Throwaway prototype)
Advantage:
- Customer involvement is strong
- Address problem of waterfall model: Risk of building wrong system due to no user feedback
- Deduce the risk of wrong system built
Disadvantage:
- Higher development effort
- Danger that due to schedule slip, the prototype becomes part of the product
Key takeaway:
- Low risk for new applications because specification and program stay in step
- High risk because of lack of process visibility
Incremental model
Basically:
- Each release adds more functionality
- Better than Waterfall model
Advantage:
- Better Customer involvement (Operational quality portion of product within weeks)
- Frequent release and feedback
Disadvantage:
-
Not suitable for large and critical system
- You can’t deliver just a small prototype of an engine control for a rocket and then just try out if the rocket would fly and fix afterwards. But you can use the approach of incremental releases for a mobile app.
-
Bad complete design upfront
-
If up-front requirements are done, during building of the system and releasing increments to the customer you discover that some of the parts of the specified functionality were not what the user actually wants/needs, then some of the up-front requirements engineering effort would be wasted.
- This just acknowledges the fact that given working releases to a customer can help you refine requirements more effectively than doing everything just on paper up front. There are cases where doing up front is needed, like in the case of large and complex safety critical systems, like an airplane.
Evolutionary model (for Agile development)
Basically:
- New versions implement new and evolving requirements
- Agile development
- don’t have up-front requirements and up-front design four the whole system done before you code
- do requirements and design for each increment
- Middle ground between waterfall and prototyping
Advantage:
-
Constant customer involvement and validation
- Address the need for customer feedback and reduce the risk of building the wrong system
-
Allows for good risk management
Disadvantage:
- Build-and-fix danger
- The potential risk is that if you keep requirements and design in evolutionary lean, some lazy developers might say, hey lets just eliminate requirements and design (including unit testing) and do just coding (and debugging) as needed, which would end up being just code and fix.
- Without requirements and design then it is just code and fix
V-model
Basically:
- waterfall model, but represented as a V to emphasize verification and validation using different type of testing
- widely used in embedded systems industry
Advantage:
- Easier maintenance (Follow logical flow)
- Emphases on verification and validation
- Makes clear which activities in the left branch feed reference information for which activities in the right branch
Disadvantage:
- Same as waterfall model, not enough feedback
- Product becomes available very late in the process (Risk of building wrong system due to no user feedback, Operational quality portion of product in the end)
- Only work best when there is a complete and frozen specification document
Spiral model
Basically:
- Waterfall model + risk analysis preceding each phase + evaluation following each phase
- Prototyping for high-risk specifications
- Middle ground between waterfall and prototyping
- If all risks cannot be resolved, the project is immediately terminated
- Appropriate only for big projects (high management overhead)
- Originally developed for big defense contracts
Hybrid process models
- Large systems are usually made up of several subsystems.
- The same process model need not be used for all subsystems, e.g.
- Prototyping for high-risk specifications
- Waterfall model for well-understood developments
Taylor the process to a problem.
Agile Methods
Uncovering better ways of developing software by doing it
Agile (Evolutionary) vs Waterfall
- Indivduals and interactions > processes and tools
- Working software > comprehensive documentation
- Customer collaboration > contract negotiation
- Responding to change > following a plan
The 3 prominent agile methods:
- XP
- Scrum
- Lean
Extreme Programming (XP)
Planning Meeting:
- Brainstorm list of features
- Customers: proposes fixes, features and ranks them based on value/priority
- Developers: design improvements and rank feature by technical difficulty, estimate effort and plan iterations
Write tests
Pair Programming
- Reduce bug / code conflict
- 1 guy focus on the function while another guy focus on the architecture
- Learning each other
Test
Integration:
- Pull request to integrate new function
- Evolutionary development
- Collection of 12 “Best Practices”
- Focus on working code that implements customer needs (rather than documents)
- Testing is a crucial element of the process
- Focus on flexibility and efficiency of the process
- Designed for small teams (<10)
XP Practices
Small releases
- Start with the smallest useful feature set
- Release early and often (e.g., every two weeks), adding a few features each time
The planning game
- Stakeholder meeting to plan the next iteration
- Business people decide on business value of features
- Developers on the technical risk of features and predicted effort per feature
- Each system features typically captured as a user story
Simple Design
- Always use the simplest possible design that gets the job done (runs the tests and states intentions of the programmer)
Testing
- Unit tests test individual system modules
- Programmers write unit tests and customers write acceptance tests
- Test-first: write test, then implement it
Refactoring
- Refactoring = Improving design without changing functionality
- Refactoring is done continuously; the code is always kept clean
Pair programming
- All production code written by two programmers
- One programmer is thinking about implementing the current method, the other is thinking strategically about the whole system
- Pairs are put together dynamically
Collective code ownership
- Any programmer that sees an opportunity to add value to any portion of the code is required to do so at any time
Continuous Integration (CI)
- Use of version and configuration management (e.g., CVS)
- All changes are integrated into the code-base at least daily
- The tests have to run 100% before and after the integration
On-site customer
- Development team has continuous access to a real life customer/user
Coding standards
- Everyone codes to the same standards
- Ideally, you should not be able to tell by looking at it who has written a specific piece of code
40-h week
- Programmers go home on time
- Overtime is a symptom of a serious problem
- No errors by tired developers; better motivated developers
Additional Agile Practices:
Estimation
- Keep track of feature set to be implemented in each iteration
- Estimate the effort per feature at the beginning of the iteration
- Keep track and review the actual effort at the end of the iteration
- Use the experience to improve estimation in the next iteration
Stand-Up / Scrum Meetings
- Short (< 15 min) daily morning meetings
- Each team member asked
- What have you done since last meeting?
- What has impeded your work?
- What will you do next?
- Try to resolve impediments quickly, or schedule smaller meetings immediately following
Advantages of XP
- Low management overhead
- no complicated procedures to follow, no documentation to maintain, direct communication, pair programming
- Continuous risk management
- early feedback from the customer
- Continuous effort estimation
- Emphasis on testing; tests help in evolution and maintenance
Disadvantages of XP
- Appropriate for small teams (up to 10 developers) only (does not scale)
- Large development groups may require more structures and documents
- If maintainers are not the people that developed the code, good documentation is necessary
- Generic design maybe necessary to enable expected future development
Software Development Scrum
Scrum (https://www.scrumguides.org/scrum-guide.html) is an agile method
Key takeaway:
- Achieve quality through transparency and monitoring, rather than processes
- Daily meetings (‘daily scrum’)
- Timeboxes with deliverables (‘sprints’; like XP iterations)
- Small, self-managed, cross-functional teams of 3-9 people
- Developers
- do the actual analysis, design, implementation, testing, etc.
- Product owner
- represents the stakeholders and the business; keeps track of the backlog and what’s due for a sprint
- Scrum master
- maintains the processes (typically in lieu of a project manager)
- Developers
Stand-Up / Scrum Meetings
- Short (< 15 min) daily morning meetings
- Each team member asked
- What have you done since last meeting?
- What has impeded your work?
- What will you do next?
- Try to resolve impediments quickly, or schedule smaller meetings immediately following
Scrum Process
Scrum Sprint
Sprint means run at full speed over a short distance.
- A short development interval (time-boxed)
- e.g. 2 weeks, but definitely no more than a month
- Well-defined goal: what’s going to be implemented, what’s going to be demoed at the end of the spring
- Keep track of Backlog
- Product Backlog: Prioritized list of work to be done (features, stories, requirements)
- Rough estimate of ideal number of days required to implement each item
- Sprint Backlog: Before each sprint, a planning session is held to decide what items from the backlog will be addressed by the sprint
- May use a simple text file, Wiki, a spreadsheet, or a specialized app (e.g., Jira for Confluence, Asana), or built-in functionality in gitlab or github
End of sprint:
- Demo
- Collect feedback
- Discuss performance, process, accuracy of estimates
- Plan next sprint
Lean Software Development
Lean software development is an adaptation of lean manufacturing principles from the Toyota Production System and practices to the software development domain.
Lean manufacturing is about making value flows explicit and optimizing it to eliminate waste
- 7 principles, with #1 being: Eliminate waste
- Everything that does not add value to the customer is waste
- E.g., code or documentation that doesn’t get used
- Defines additional software practices, e.g., Kanban board
Kanban Board
- basically a task board, visually depicts the status and flow of tasks within a project in a tabular way
- Typical columns in a task board include
- Backlog
- In progress
- Done
- Blocked
To implement Kanban Board
- A simple text file, Wiki, Notion cards, a spreadsheet, or a specialized app (e.g., Jira for Confluence, Asana), or built-in Issue Board functionality in gitlab or github
- You can customize the specific columns to your needs and use additional advice from online resources
- E.g., organizing tasks using multiple dimensions in Issue Board in gitlab
Summary
- Some practices are key
- Short iterations with frequent deliveries
- Unit and integration tests
- Effective communication (e.g., through like daily scrum)
- Understanding the value flow in your project (Kanban board)
- Addressing technical debt through refactoring
- In practice, a few you can experiment with practices from different methods and mix and match if you find them to work well for your team and project
Exercise Questions
Revisit the concepts of Process Models
- Waterfall model
- Risk of building wrong system due to no user feedback, Operational quality portion of product in the end
- Only work best when there is a complete and frozen specification document
- V-model
- Basically waterfall model, but represented as a V-shape to emphasize verification and validation using different type of testing
- Still have the same risk of Waterfall model.
What is the biggest drawback of the waterfall model?
Risk of building wrong system/delivering incorrect product due to not enough feedback
How can you address the main risk of the waterfall model?
Obtain more frequent feedback from customer/users; e.g., building a prototype up front, or using an iterative model
What is the main idea behind the V process?
Linking unit, integration, and system test stages with, respectively, unit design, system design, and system specification.
What is the main difference between the waterfall model and the V-model?
V-model is basically the waterfall model emphasise Verification and Validation using different type of testing
V-model is like waterfall, but it adds more detailed stages for the V&V part, including unit, integration, and system testing (and acceptance testing); each of these test stages is fed with specification information from, respectively, unit, design, and requirements development stages.
What problem does early prototyping address?
Reducing the risk of building wrong system because more customer involvement
What lifecycle model is used in agile development?
Evolutionary model
When is the waterfall model applicable?
When the customer requirements are fully clear and frozen (no uncertainties)