Goal(s) with this Lecture

Purpose of Architecture Evaluations Overview of some Architecture Evaluation methods

Architecture Evaluation

Mikael Svahnberg Mikael.Svahnberg@bth.se

Purpose
Do we meet the quality requirements on the system? Increase understanding of system Are all requirements accounted for? Done Early Costly to realise that quality requirements are not met once system is developed Costly to change a system post-facto Leads to patchy systems to change post-facto Identify weak spots in architecture Identify paths to improvements

Different Methods
Early Architecture Evaluation Will we satisfy quality requirements (potential of architecture) Late Architecture Evaluation Hard metrics How did we do? What needs to be improved for the next release?

Early Architecture Evaluation Methods

Scenario-based SAAM ATAM Global Analysis (Siemens 4 views) Based on Experiences and Logical Reasoning Often Ad-hoc Amalgamation with scenarios Simulation-based Build parts of architecture Build model of architecture Architecture Description Languages

Inspections
Formal process to review documents (or code) Inspection group identify major and minor faults Example: No compiled code until inspected student project. Cf. Cleanroom software development.

Based on Mathematical Models Often domain-specic (E.g. High-performance computing, real-time systems) ABAS (ATAM) Other Software Inspections

Simulation-based
Build model of architecture Build parts of architecture Simulate system Measure

SAAM
Involves all stakeholders of software architecture Are all interests accounted for in architecture?

Use Architecture Description Languages (ADL) to (formally) describe architecture (build model): Often domain specic Examples: Aesop, ArTek, C2, Darwin, LILEANNA, MetaH, Rapide, SADL, UniCon, Weaves, Wright, ACME.

1. 2. 3. 4.

Develop scenarios Describe Candidate Architecture Classify Scenarios (direct & indirect) Perform Scenario Evaluations (needed changes to support scenario, costs of changes) 5. Reveal Scenario Interaction (scenarios requiring changes in same component of system (may indicate fat components)) 6. Overall Evaluation (Weighting of scenarios)

ATAM
1. 2. 3. 4. 5. 6. 7. 8. 9. Based on SAAM Focus on Quality Attributes Identify trade-offs between quality attributes ??? Present ATAM to stakeholders Present business drivers (context & need of system) Present architecture Identify Architectural Approaches Generate Quality Attribute Utility Tree (develop scenarios) Analyse Architectural Approaches Brainstorm and prioritize scenarios (general scenarios) Analyse architectural approaches (wrt scenarios from step 7) Present Results

The BTH Way

3-day design workshop (Danaher) 4-hour evaluation (Student Projects Research Comparative Architecture Evaluation

BTH @ Danaher (Architecture Design)

Day 1 Inspiration Requirements Formulation Basic, hard-to-realise, future Dene Scenarios Basic Conguration Growth Maintenance Visionary Day 2 Sculpture Architecture Design. Several Proposals, Iterated. Day 3 Vernisage Present to Marketing Evaluate Workshop

BTH @ Student Projects (Architecture Evaluation)

Large Team Software Engineering Project 20 credits (20 weeks) 10-15 persons / project Real Industry Customer Simulate a real software development project Architecture Design Architecture Design Evaluation (4 hours) Shorter variant of SAAM/ATAM You will use this later in course!

BTH @ Student Projects Schedule

1. Introduction to the project (project) (15 min) 2. List Quality Attributes (project) (20 min) 3. List Scenarios (45 min) 4. Present Architecture (architect)(20 min) 5. Scenario/Architecture Analysis (90 min) 6. Summary and Conclusions (15 min) Somewhere: Break (20 min)

BTH @ Student Projects What & Who

Documents collected before assessment Architecture design Requirements Specication (Quality attributes are especially important) Any reports from prototypes, tests, measurements List of scenarios, if available Participants from project Architect Test architect Someone from the design team Someone who have participated in customer contact In addition, there should be someone that can be easily and quickly reached via ICQ/e-mail and phone that can send additional documents if necessary.

BTH @ Student Projects Introduction

Introduction Present assessment team & project team State Purpose of evaluation Not out to get anyone Documentation from Evaluation Written on-the-y Continuously presented on an OH-Beamer Project checks that they understand what is written and that it is accurate Present Document Distribution List Team Leader Head of Department

BTH @ Student Projects Conduct Assessment

Conduct assessment according to schedule Optimal with 3 assessors Listing quality attributes often takes more time than scheduled Use categorisation (e.g. ISO 9126) as a start Provoke measurable values List 2 scenarios each for 3 most important quality attributes Assessment Walkthrough of scenarios in architecture List weak spots Calculate/estimate e.g. Transfer times Summary Most important issues

BTH Research - Blend of Quality Requirements

Research @ BTH Not all Quality Requirements are equally important Ordinary evaluation methods focus on achieving every quality attrbute to its highest possible level May spend time and money on improving an unimportant quality attribute. Must support mix of quality requirements Moreover: Hard to create absolute measures People have implicit assumptions People prefer familiar structures

Blend of Quality Requirements

Identify relevant quality attributes Prioritise quality attributes Develop / Identify several architecture candidates For each quality attribute, which software architecture has the best potential for achieving it? For each architecture candidate, which quality attribute does it have the most potential for fullling?

Introducing Method

Discuss!

Step 1: Create Individual Frameworks

Step 4: Evaluate Framework against Literature

Step 3: Create Unified Framework

Step 2: Discuss Individual Frameworks

Step 5: Analyse Framework

(4)

Discuss Individual Framework

What to do With Evaluation

Go or no-go decision Depends on goal with Investigate Further Architecture Evaluation Improve Architecture Remove bottlenecks Transform Architecture. Bosch suggests: Impose Architectural Style Complete restructuring of system E.g. Pipes and Filters, Object-Orientation, Implicit Invocation Impose Architectural Pattern E.g. Concurrency, Persistence, Distribution Apply Design Pattern Apply on a smaller part of the architecture Convert Quality Requirement to Functionality

Differences between Individual Views Different Backgrounds Different Experiences Different Interpretations Better Joint Understanding Avoid differing interpretations later Where they would cause problems

Conclusions
Further Understanding through: Structured way of Expressing Knowledge Focussed Discussions Avoid Problems Later Dene Goal of Architecture Evaluation!