A detailed and thorough reference on the discipline andpractice of systems engineering The objective of the International Council on SystemsEngineering (INCOSE) Systems Engineering Handbook is todescribe key process activities performed by systems engineers andother engineering professionals throughout the life cycle of asystem. The book covers a wide range of fundamental system conceptsthat broaden the thinking of the systems engineering practitioner,such as system thinking, system science, life cycle management,specialty engineering, system of systems, and agile and iterativemethods. This book also defines the discipline and practice ofsystems engineering for students and practicing professionalsalike, providing an authoritative reference that is acknowledgedworldwide. The latest edition of the INCOSE Systems EngineeringHandbook: * Is consistent with ISO/IEC/IEEE 15288:2015 Systems and softwareengineering--System life cycle processes and the Guide to theSystems Engineering Body of Knowledge (SEBoK) * Has been updated to include the latest concepts of the INCOSEworking groups * Is the body of knowledge for the INCOSE CertificationProcess This book is ideal for any engineering professional who has aninterest in or needs to apply systems engineering practices. Thisincludes the experienced systems engineer who needs a convenientreference, a product engineer or engineer in another discipline whoneeds to perform systems engineering, a new systems engineer, oranyone interested in learning more about systems engineering.
Ebooka przeczytasz w aplikacjach Legimi na:
Liczba stron: 729
HISTORY OF CHANGES
APPROVED FOR SEH V4:
LIST OF FIGURES
LIST OF TABLES
1 SYSTEMS ENGINEERING HANDBOOK SCOPE
1.5 DEFINITIONS OF FREQUENTLY USED TERMS
2 SYSTEMS ENGINEERING OVERVIEW
2.2 DEFINITIONS AND CONCEPTS OF A SYSTEM
2.3 THE HIERARCHY
2.4 DEFINITION OF SYSTEMS OF SYSTEMS
2.5 ENABLING SYSTEMS
2.6 DEFINITION OF SYSTEMS ENGINEERING
2.7 ORIGINS AND EVOLUTION OF SYSTEMS ENGINEERING
2.8 USE AND VALUE OF SYSTEMS ENGINEERING
2.9 SYSTEMS SCIENCE AND SYSTEMS THINKING
2.10 SYSTEMS ENGINEERING LEADERSHIP
2.11 SYSTEMS ENGINEERING PROFESSIONAL DEVELOPMENT
3 GENERIC LIFE CYCLE STAGES
3.2 LIFE CYCLE CHARACTERISTICS
3.3 LIFE CYCLE STAGES
3.4 LIFE CYCLE APPROACHES
3.5 WHAT IS BEST FOR YOUR ORGANIZATION, PROJECT, OR TEAM?
3.6 INTRODUCTION TO CASE STUDIES
4 TECHNICAL PROCESSES
4.1 BUSINESS OR MISSION ANALYSIS PROCESS
4.2 STAKEHOLDER NEEDS AND REQUIREMENTS DEFINITION PROCESS
4.3 SYSTEM REQUIREMENTS DEFINITION PROCESS
4.4 ARCHITECTURE DEFINITION PROCESS
4.5 DESIGN DEFINITION PROCESS
4.6 SYSTEM ANALYSIS PROCESS
4.7 IMPLEMENTATION PROCESS
4.8 INTEGRATION PROCESS
4.9 VERIFICATION PROCESS
4.10 TRANSITION PROCESS
4.11 VALIDATION PROCESS
4.12 OPERATION PROCESS
4.13 MAINTENANCE PROCESS
4.14 DISPOSAL PROCESS
5 TECHNICAL MANAGEMENT PROCESSES
5.1 PROJECT PLANNING PROCESS
5.2 PROJECT ASSESSMENT AND CONTROL PROCESS
5.3 DECISION MANAGEMENT PROCESS
5.4 RISK MANAGEMENT PROCESS
5.5 CONFIGURATION MANAGEMENT PROCESS
5.6 INFORMATION MANAGEMENT PROCESS
5.7 MEASUREMENT PROCESS
5.8 QUALITY ASSURANCE PROCESS
6 AGREEMENT PROCESSES
6.1 ACQUISITION PROCESS
6.2 SUPPLY PROCESS
7 ORGANIZATIONAL PROJECT-ENABLING PROCESSES
7.1 LIFE CYCLE MODEL MANAGEMENT PROCESS
7.2 INFRASTRUCTURE MANAGEMENT PROCESS
7.3 PORTFOLIO MANAGEMENT PROCESS
7.4 HUMAN RESOURCE MANAGEMENT PROCESS
7.5 QUALITY MANAGEMENT PROCESS
7.6 KNOWLEDGE MANAGEMENT PROCESS
8 TAILORING PROCESS AND APPLICATION OF SYSTEMS ENGINEERING
8.1 TAILORING PROCESS
8.2 TAILORING FOR SPECIFIC PRODUCT SECTOR OR DOMAIN APPLICATION
8.3 APPLICATION OF SYSTEMS ENGINEERING FOR PRODUCT LINE MANAGEMENT
8.4 APPLICATION OF SYSTEMS ENGINEERING FOR SERVICES
8.5 APPLICATION OF SYSTEMS ENGINEERING FOR ENTERPRISES
8.6 APPLICATION OF SYSTEMS ENGINEERING FOR VERY SMALL AND MICRO ENTERPRISES
9 CROSS-CUTTING SYSTEMS ENGINEERING METHODS
9.1 MODELING AND SIMULATION
9.2 MODEL-BASED SYSTEMS ENGINEERING
9.3 FUNCTIONS-BASED SYSTEMS ENGINEERING METHOD
9.4 OBJECT-ORIENTED SYSTEMS ENGINEERING METHOD
9.6 INTERFACE MANAGEMENT
9.7 INTEGRATED PRODUCT AND PROCESS DEVELOPMENT
9.8 LEAN SYSTEMS ENGINEERING
9.9 AGILE SYSTEMS ENGINEERING
10 SPECIALTY ENGINEERING ACTIVITIES
10.1 AFFORDABILITY/COST-EFFECTIVENESS/LIFE CYCLE COST ANALYSIS
10.2 ELECTROMAGNETIC COMPATIBILITY
10.3 ENVIRONMENTAL ENGINEERING/IMPACT ANALYSIS
10.4 INTEROPERABILITY ANALYSIS
10.5 LOGISTICS ENGINEERING
10.6 MANUFACTURING AND PRODUCIBILITY ANALYSIS
10.7 MASS PROPERTIES ENGINEERING
10.8 RELIABILITY, AVAILABILITY, AND MAINTAINABILITY
10.9 RESILIENCE ENGINEERING
10.10 SYSTEM SAFETY ENGINEERING
10.11 SYSTEM SECURITY ENGINEERING
10.12 TRAINING NEEDS ANALYSIS
10.13 USABILITY ANALYSIS/HUMAN SYSTEMS INTEGRATION
10.14 VALUE ENGINEERING
APPENDIX A: REFERENCES
APPENDIX B: ACRONYMS
APPENDIX C: TERMS AND DEFINITIONS
APPENDIX D: N
DIAGRAM OF SYSTEMS ENGINEERING PROCESSES
APPENDIX E: INPUT/OUTPUT DESCRIPTIONS
APPENDIX F: ACKNOWLEDGMENTS
SEH V4 CONTRIBUTIONS
APPENDIX G: COMMENT FORM
END USER LICENSE AGREEMENT
Table 2.1 Important dates in the origins of SE as a discipline
Table 2.2 Important dates in the origin of SE standards
Table 2.3 Current significant SE standards and guides
Table 2.4 SE return on investment
Table 3.1 Generic life cycle stages, their purposes, and decision gate options
Table 4.1 Examples of system elements and physical interfaces
Table 5.1 Partial list of decision situations (opportunities) throughout the life cycle
Table 8.1 Standardization-related associations and automotive standards
Table 8.2 Attributes of system entities
Table 9.1 Types of IPDTs and their focus and responsibilities
Table 9.2 Pitfalls of using IPDT
Figure D.1 Input/output relationships between the various SE processes. INCOSE SEH original figure created by David Walden. Usage per the INCOSE Notices page. All other rights reserved.
Figure 1.1 System life cycle processes per ISO/IEC/IEEE 15288.
Figure 1.2 Sample of IPO diagram for SE processes.
Figure 2.1 Hierarchy within a system.
Figure 2.2 Example of the systems and systems of systems within a transport system of systems.
Figure 2.3 System of interest, its operational environment, and its enabling systems.
Figure 2.4 Committed life cycle cost against time.
Figure 2.5 Technology acceleration over the past 140 years.
Figure 2.6 Project performance versus SE capability.
Figure 2.7 Cost (a) and schedule (b) overruns correlated with SE effort.
Figure 2.8 Systems science in context.
Figure 2.9 SE optimization system.
Figure 2.10 Professional development system.
Figure 3.1 Generic business life cycle.
Figure 3.2 Life cycle model with some of the possible progressions.
Figure 3.3 Comparisons of life cycle models.
Figure 3.4 Importance of the concept stage.
Figure 3.5 Iteration and recursion.
Figure 3.6 Vee model.
Figure 3.7 Left side of the Vee model.
Figure 3.8 Right side of the Vee model.
Figure 3.9 IID and evolutionary development.
Figure 3.10 The incremental commitment spiral model (ICSM).
Figure 3.11 Phased view of the generic incremental commitment spiral model process.
Figure 4.1 Transformation of needs into requirements.
Figure 4.2 IPO diagram for business or mission analysis process.
Figure 4.3 Key SE interactions.
Figure 4.4 IPO diagram for stakeholder needs and requirements definition process.
Figure 4.5 IPO diagram for the system requirements definition process.
Figure 4.6 IPO diagram for the architecture definition process.
Figure 4.7 Interface representation.
Figure 4.8 (a) Initial arrangement of aggregates; (b) final arrangement after reorganization.
Figure 4.9 IPO diagram for the design definition process.
Figure 4.10 IPO diagram for the system analysis process.
Figure 4.11 IPO diagram for the implementation process.
Figure 4.12 IPO diagram for the integration process.
Figure 4.13 IPO diagram for the verification process.
Figure 4.14 Definition and usage of a verification action.
Figure 4.15 Verification level per level.
Figure 4.16 IPO diagram for the transition process.
Figure 4.17 IPO diagram for the validation process.
Figure 4.18 Definition and usage of a validation action.
Figure 4.19 Validation level per level. Note: System requirements are validated against the stakeholder requirements.
Figure 4.20 IPO diagram for the operation process.
Figure 4.21 IPO diagram for the maintenance process.
Figure 4.22 IPO diagram for the disposal process.
Figure 5.1 IPO diagram for the project planning process.
Figure 5.2 IPO diagram for the project assessment and control process.
Figure 5.3 IPO diagram for the decision management process.
Figure 5.4 IPO diagram for the risk management process.
Figure 5.5 Level of risk depends upon both likelihood and consequences.
Figure 5.6 Typical relationship among the risk categories.
Figure 5.7 Intelligent management of risks and opportunities.
Figure 5.8 IPO diagram for the configuration management process.
Figure 5.9 Requirements changes are inevitable.
Figure 5.10 IPO diagram for the information management process.
Figure 5.11 IPO diagram for the measurement process.
Figure 5.12 Measurement as a feedback control system.
Figure 5.13 Relationship of technical measures.
Figure 5.14 TPM monitoring.
Figure 5.15 IPO diagram for the quality assurance process.
Figure 6.1 IPO diagram for the acquisition process.
Figure 6.2 IPO diagram for the supply process.
Figure 7.1 IPO diagram for the life cycle model management process.
Figure 7.2 Standard SE process flow.
Figure 7.3 IPO diagram for the infrastructure management process.
Figure 7.4 IPO diagram for the portfolio management process.
Figure 7.5 IPO diagram for the human resource management process.
Figure 7.6 IPO diagram for the quality management process.
Figure 7.7 IPO diagram for the knowledge management process.
Figure 8.1 Tailoring requires balance between risk and process.
Figure 8.2 IPO diagram for the tailoring process.
Figure 8.3 Product line viewpoints.
Figure 8.4 Capitalization and reuse in a product line.
Figure 8.5 Product line return on investment.
Figure 8.6 Service system conceptual framework.
Figure 8.7 Organizations manage resources to create enterprise value.
Figure 8.8 Individual competence leads to organizational, system, and operational capability.
Figure 8.9 Enterprise SE process areas in the context of the entire enterprise.
Figure 9.1 Sample model taxonomy.
Figure 9.2 SysML diagram types. From Friedenthal et al. (2012), Figure 3.2.
Figure 9.3 Functional analysis/allocation process.
Figure 9.4 Alternative functional decomposition evaluation and definition.
Figure 9.5 OOSEM builds on established SE foundations.
Figure 9.6 OOSEM activities in context of the system development process.
Figure 9.7 OOSEM activities and modeling artifacts.
Figure 9.8 Sample FFBD and
Figure 9.9 Examples of complementary integration activities of IPDTs.
Figure 9.10 Lean development principles.
Figure 10.1 Contextual nature of the affordability trade space.
Figure 10.2 System operational effectiveness.
Figure 10.3 Cost versus performance.
Figure 10.4 Affordability cost analysis framework.
Figure 10.5 Life cycle cost elements (not to scale).
Figure 10.6 Process for achieving EMC.
Figure 10.7 Supportability analysis.
Figure 10.8 Reliability Program Plan development.
Figure 10.9 Resilience event model.
Figure 10.10 Sample Function Analysis System Technique (FAST) diagram.
Table of Contents
International Council on Systems Engineering (INCOSE) 7670 Opportunity Rd, Suite 220San Diego, CA, USA 92111-2222
Compiled and Edited by:
DAVID D. WALDEN, ESEPGARRY J. ROEDLER, ESEPKEVIN J. FORSBERG, ESEPR. DOUGLAS HAMELINTHOMAS M. SHORTELL, CSEP
Copyright © 2015 by John Wiley & Sons, Inc. All rights reserved
Published by John Wiley & Sons, Inc., Hoboken, New JerseyPublished simultaneously in Canada
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permissions.
Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.
For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002.
Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products, visit our web site at www.wiley.com.
Library of Congress Cataloging-in-Publication Data:
Systems engineering handbook : a guide for system life cycle processes and activities / prepared by International Council on Systems Engineering (INCOSE) ; compiled and edited by, David D. Walden, ESEP, Garry J. Roedler, ESEP, Kevin J. Forsberg, ESEP, R. Douglas Hamelin, Thomas M. Shortell, CSEP. – 4th edition. pages cm Includes bibliographical references and index.
ISBN 978-1-118-99940-0 (cloth)1. Systems engineering–Handbooks, manuals, etc. 2. Product life cycle–Handbooks, manuals, etc. I. Walden, David D., editor. II. Roedler, Garry J., editor. III. Forsberg, Kevin, editor. IV. Hamelin, R. Douglas, editor. V. Shortell, Thomas M., editor. VI. International Council on Systems Engineering. TA168.S8724 2015 620.001′1–dc23 2014039630
This International Council on Systems Engineering (INCOSE) Technical Product was prepared by the INCOSE Knowledge Management working group. It is approved by INCOSE Technical Operations Leadership for release as an INCOSE Technical Product.
Copyright ©2015 by INCOSE, subject to the following restrictions:
Author Use: Authors have full rights to use their contributions unfettered, with credit to the INCOSE technical source, except as noted in the following text. Abstraction is permitted with credit to the source.
INCOSE Use: Permission to reproduce and use this document or parts thereof by members of INCOSE and to prepare derivative works from this document for INCOSE use is granted, with attribution to INCOSE and the original author(s) where practical, provided this copyright notice is included with all reproductions and derivative works. Content from ISO/IEC/IEEE 15288 and ISO/IEC TR 24748-1 is used by permission, and is not to be reproduced other than as part of this total document.
External Use: This document may not be shared or distributed to any non-INCOSE third party. Requests for permission to reproduce this document in whole or in part, or to prepare derivative works of this document for external and/or commercial use, will be denied unless covered by other formal agreements with INCOSE. Copying, scanning, retyping, or any other form of reproduction or use of the content of whole pages or source documents are prohibited, except as approved by the INCOSE Administrative Office, 7670 Opportunity Road, Suite 220, San Diego, CA 92111-2222, USA.
Electronic Version Use : All electronic versions (e.g., eBook, PDF) of this document are to be used for personal professional use only and are not to be placed on non-INCOSE sponsored servers for general use. Any additional use of these materials must have written approval from the INCOSE Administrative Office.
General Citation Guidelines : References to this handbook should be formatted as follows, with appropriate adjustments for formally recognized styles:
INCOSE (2015). Systems Engineering Handbook:
A Guide for System Life Cycle Process and Activities (4th ed.). D. D. Walden, G. J. Roedler, K. J. Forsberg, R. D. Hamelin, and, T. M. Shortell (Eds.). San Diego, CA: International Council on Systems Engineering. Published by John Wiley & Sons, Inc.
Change description and rationale
Systems Engineering Handbook
(SEH) created by INCOSE members from several defense/aerospace companies—including Lockheed, TRW, Northrop Grumman, Ford Aerospace, and the Center for Systems Management—for INCOSE review
Initial SEH release approved to update and broaden coverage of SE process. Included broad participation of INCOSE members as authors. Based on Interim Standards EIA 632 and IEEE 1220
Expanded coverage on several topics, such as functional analysis. This version was the basis for the development of the Certified Systems Engineering Professional (CSEP) exam
Reduced page count of SEH v2 by 25% and reduced the US DoD-centric material wherever possible. This version was the basis for the first publically offered CSEP exam
Significant revision based on ISO/IEC 15288:2002. The intent was to create a country- and domain-neutral handbook. Significantly reduced the page count, with elaboration to be provided in appendices posted online in the INCOSE Product Asset Library (IPAL)
Added detail that was not included in SEH v3, mainly in new appendices. This version was the basis for the updated CSEP exam
Updated version based on ISO/IEC/IEEE 15288:2008. Significant restructuring of the handbook to consolidate related topics
Clarified definition material, architectural frameworks, concept of operations references, risk references, and editorial corrections based on ISO/IEC review
Correction of errata introduced by revision 3.2.1
Significant revision based on ISO/IEC/IEEE 15288:2015, inputs from the relevant INCOSE working groups (WGs), and to be consistent with the Guide to the Systems Engineering Body of Knowledge (SEBoK)
The objective of the International Council on Systems Engineering (INCOSE) Systems Engineering Handbook (SEH) is to describe key process activities performed by systems engineers. The intended audience is the systems engineering (SE) professional. When the term systems engineer is used in this handbook, it includes the new systems engineer, a product engineer or an engineer in another discipline who needs to perform SE, or an experienced systems engineer who needs a convenient reference.
The descriptions in this handbook show what each SE process activity entails, in the context of designing for required performance and life cycle considerations. On some projects, a given activity may be performed very informally; on other projects, it may be performed very formally, with interim products under formal configuration control. This document is not intended to advocate any level of formality as necessary or appropriate in all situations. The appropriate degree of formality in the execution of any SE process activity is determined by the following:
The need for communication of what is being done (across members of a project team, across organizations, or over time to support future activities)
The level of uncertainty
The degree of complexity
The consequences to human welfare
On smaller projects, where the span of required communications is small (few people and short project life cycle) and the cost of rework is low, SE activities can be conducted very informally and thus at low cost. On larger projects, where the span of required communications is large (many teams that may span multiple geographic locations and organizations and long project life cycle) and the cost of failure or rework is high, increased formality can significantly help in achieving project opportunities and in mitigating project risk.
In a project environment, work necessary to accomplish project objectives is considered “in scope”; all other work is considered “out of scope.” On every project, “thinking” is always “in scope.” Thoughtful tailoring and intelligent application of the SE processes described in this handbook are essential to achieve the proper balance between the risk of missing project technical and business objectives on the one hand and process paralysis on the other hand. Chapter 8 provides tailoring guidelines to help achieve that balance.
Kevin Forsberg, ESEP, Chair, INCOSE Knowledge Management Working Group
Garry Roedler, ESEP, Co-Chair, INCOSE Knowledge Management Working Group
William Miller, INCOSE Technical Director (2013–2014)
Paul Schreinemakers, INCOSE Technical Director (2015–2016)
Quoc Do, INCOSE Associate Director for Technical Review
Kenneth Zemrowski, ESEP, INCOSE Assistant Director for Technical Information
System life cycle processes per ISO/IEC/IEEE 15288
Sample of IPO diagram for SE processes
Hierarchy within a system
Example of the systems and systems of systems within a transport system of systems
System of interest, its operational environment, and its enabling systems
Committed life cycle cost against time
Technology acceleration over the past 140 years
Project performance versus SE capability
Cost and schedule overruns correlated with SE effort
Systems science in context
SE optimization system
Professional development system
Generic business life cycle
Life cycle model with some of the possible progressions
Comparisons of life cycle models
Importance of the concept stage
Iteration and recursion
Left side of the Vee model
Right side of the Vee model
IID and evolutionary development
The incremental commitment spiral model (ICSM)
Phased view of the generic incremental commitment spiral model process
Transformation of needs into requirements
IPO diagram for business or mission analysis process
Key SE interactions
IPO diagram for stakeholder needs and requirements definition process
IPO diagram for the system requirements definition process
IPO diagram for the architecture definition process
(a) Initial arrangement of aggregates; (b) final arrangement after reorganization
IPO diagram for the design definition process
IPO diagram for the system analysis process
IPO diagram for the implementation process
IPO diagram for the integration process
IPO diagram for the verification process
Definition and usage of a verification action
Verification level per level
IPO diagram for the transition process
IPO diagram for the validation process
Definition and usage of a validation action
Validation level per level
IPO diagram for the operation process
IPO diagram for the maintenance process
IPO diagram for the disposal process
IPO diagram for the project planning process
IPO diagram for the project assessment and control process
IPO diagram for the decision management process
IPO diagram for the risk management process
Level of risk depends on both likelihood and consequences
Typical relationship among the risk categories
Intelligent management of risks and opportunities
IPO diagram for the configuration management process
Requirements changes are inevitable
IPO diagram for the information management process
IPO diagram for the measurement process
Measurement as a feedback control system
Relationship of technical measures
IPO diagram for the quality assurance process
IPO diagram for the acquisition process
IPO diagram for the supply process
IPO diagram for the life cycle model management process
Standard SE process flow
IPO diagram for the infrastructure management process
IPO diagram for the portfolio management
IPO diagram for the human resource management process
IPO diagram for the quality management process
IPO diagram for the knowledge management process
Tailoring requires balance between risk and process
IPO diagram for the tailoring process
Product line viewpoints
Capitalization and reuse in a product line
Product line return on investment
Service system conceptual framework
Organizations manage resources to create enterprise value
Individual competence leads to organizational, system and operational capability
Enterprise SE process areas in the context of the entire enterprise
Sample model taxonomy
SysML diagram types
Functional analysis/allocation process
Alternative functional decomposition evaluation and definition
OOSEM builds on established SE foundations
OOSEM activities in context of the system development process
OOSEM activities and modeling artifacts
Sample FFBD and N
Examples of complementary integration activities of IPDTs
Lean development principles
Contextual nature of the affordability trade space
Systems operational effectiveness
Cost versus performance
Affordability cost analysis framework
Life cycle cost elements (not to scale)
Process for achieving EMC
Reliability program plan development
Resilience event model
Sample Function Analysis System Technique (FAST) diagram
Important dates in the origins of SE as a discipline
Important dates in the origin of SE standards
Current significant SE standards and guides
SE return on investment
Generic life cycle stages, their purposes, and decision gate options
Examples of systems elements and physical interfaces
Partial list of decision situations (opportunities) throughout the life cycle
Standardization-related associations and automotive standards
Attributes of system entities
Types of IPDTs and their focus and responsibilities
Pitfalls of using IPDT
This handbook defines the discipline and practice of systems engineering (SE) for students and practicing professionals alike and provides an authoritative reference to understand the SE discipline in terms of content and practice.
This handbook is consistent with ISO/IEC/IEEE 15288:2015, Systems and software engineering—System life cycle processes (hereafter referred to as ISO/IEC/IEEE 15288), to ensure its usefulness across a wide range of application domains—man-made systems and products, as well as business and services.
ISO/IEC/IEEE 15288 is an international standard that provides generic top-level process descriptions and requirements, whereas this handbook further elaborates on the practices and activities necessary to execute the processes. Before applying this handbook in a given organization or project, it is recommended that the tailoring guidelines in Chapter 8 be used to remove conflicts with existing policies, procedures, and standards already in use within an organization. Processes and activities in this handbook do not supersede any international, national, or local laws or regulations.
This handbook is also consistent with the Guide to the Systems Engineering Body of Knowledge (SEBoK, 2014) (hereafter referred to as the SEBoK) to the extent practicable. In many places, this handbook points readers to the SEBoK for more detailed coverage of the related topics, including a current and vetted set of references.
For organizations that do not follow the principles of ISO/IEC/IEEE 15288 or the SEBoK to specify their life cycle processes (including much of commercial industry), this handbook can serve as a reference to practices and methods that have proven beneficial to the SE community at large and that can add significant value in new domains, if appropriately selected and applied. Section 8.2 provides top-level guidance on the application of SE in selected product sectors and domains.
This chapter defines the purpose and scope of this handbook. Chapter 2 provides an overview of the goals and value of using SE throughout the system life cycle. Chapter 3 describes an informative life cycle model with six stages: concept, development, production, utilization, support, and retirement.
ISO/IEC/IEEE 15288 identifies four process groups to support SE. Each of these process groups is the subject of an individual chapter. A graphical overview of these processes is given in Figure 1.1:
) include business or mission analysis, stakeholder needs and requirements definition, system requirements definition, architecture definition, design definition, system analysis, implementation, integration, verification, transition, validation, operation, maintenance, and disposal.
Technical management processes
) include project planning, project assessment and control, decision management, risk management, configuration management, information management, measurement, and quality assurance.
) include acquisition and supply.
Organizational project-enabling processes
) include life cycle model management, infrastructure management, portfolio management, human resource management, quality management, and knowledge management.
Figure 1.1 System life cycle processes per ISO/IEC/IEEE 15288.
This figure is excerpted from ISO/IEC/IEEE 15288:2015, Figure 4 on page 17, with permission from the ANSI on behalf of the ISO. © ISO 2015. All rights reserved.
This handbook provides additional chapters beyond the process groups listed in Figure 1.1:
Tailoring processes and application of systems engineering
) include information on how to adapt and scale the SE processes and how to apply those processes in various applications. Not every process will apply universally. Careful selection from the material is recommended. Reliance on process over progress will not deliver a system.
Crosscutting systems engineering methods
) provide insights into methods that can apply across all processes, reflecting various aspects of the iterative and recursive nature of SE.
Specialty engineering activities
) include practical information so systems engineers can understand and appreciate the importance of various specialty engineering topics.
Appendix A contains a list of references used in this handbook. Appendices B and C provide a list of acronyms and a glossary of SE terms and definitions, respectively. Appendix D provides an N2 diagram of the SE processes showing where dependencies exist in the form of shared inputs or outputs. Appendix E provides a master list of all inputs/outputs identified for each SE process. Appendix F acknowledges the various contributors to this handbook. Errors, omissions, and other suggestions for this handbook can be submitted to the INCOSE using the comment form contained in Appendix G.
A common format has been applied in Chapters 4 through 7 to describe the system life cycle processes found in ISO/IEC/IEEE 15288. Each process is illustrated by an input–process–output (IPO) diagram showing key inputs, process activities, and resulting outputs. A sample is shown in Figure 1.2. Note that the IPO diagrams throughout this handbook represent “a” way that the SE processes can be performed, but not necessarily “the” way that they must be performed. The issue is that SE processes produce “results” that are often captured in “documents” rather than producing “documents” simply because they are identified as outputs. To understand a given process, readers are encouraged to study the complete information provided in the combination of diagrams and text and not rely solely on the diagrams.
Figure 1.2 Sample of IPO diagram for SE processes.
INCOSE SEH original figure created by Shortell and Walden. Usage per the INCOSE Notices page. All other rights reserved.
The following heading structure provides consistency in the discussion of these processes:
To ensure consistency with ISO/IEC/IEEE 15288, the purpose statements from the standard are included verbatim for each process described herein. Inputs and outputs are listed by name within the respective IPO diagrams with which they are associated. A complete list of all inputs and outputs with their respective descriptions appears in Appendix E.
The titles of the process activities listed in each section are also consistent with ISO/IEC/IEEE 15288. In some cases, additional items have been included to provide summary-level information regarding industry best practices and evolutions in the application of SE processes.
The controls and enablers shown in Figure 1.2 govern all processes described herein and, as such, are not repeated in the IPO diagrams or in the list of inputs associated with each process description. Typically, IPO diagrams do not include controls and enablers, but since they are not repeated in the IPO diagrams throughout the rest of the handbook, we have chosen to label them IPO diagrams. Descriptions of each control and enabler are provided in Appendix E.
One of the systems engineer’s first and most important responsibilities on a project is to establish nomenclature and terminology that support clear, unambiguous communication and definition of the system and its elements, functions, operations, and associated processes. Further, to promote the advancement of the field of SE throughout the world, it is essential that common definitions and understandings be established regarding general methods and terminology that in turn support common processes. As more systems engineers accept and use common terminology, SE will experience improvements in communications, understanding, and, ultimately, productivity.
The glossary of terms used throughout this book (see Appendix C) is based on the definitions found in ISO/IEC/IEEE 15288; ISO/IEC/IEEE 24765, Systems and Software Engineering—Vocabulary (2010); and SE VOCAB (2013).
This chapter offers a brief overview of the systems engineering (SE) discipline, beginning with a few key definitions, an abbreviated survey of the origins of the discipline, and discussions on the value of applying SE. Other concepts, such as systems science, systems thinking, SE leadership, SE ethics, and professional development, are also introduced.
While the concepts of a system can generally be traced back to early Western philosophy and later to science, the concept most familiar to systems engineers is often traced to Ludwig von Bertalanffy (1950, 1968) in which a system is regarded as a “whole” consisting of interacting “parts.” The ISO/IEC/IEEE definitions provided in this handbook draw from this concept.
The systems considered in ISO/IEC/IEEE 15288 and in this handbook
[5.2.1] … are man-made, created and utilized to provide products or services in defined environments for the benefit of users and other stakeholders.
The definitions cited here and in Appendix C refer to systems in the real world. A system concept should be regarded as a shared “mental representation” of the actual system. The systems engineer must continually distinguish between systems in the real world and system representations. The INCOSE and ISO/IEC/IEEE definitions draw from this view of a system:
… an integrated set of elements, subsystems, or assemblies that accomplish a defined objective. These elements include products (hardware, software, firmware), processes, people, information, techniques, facilities, services, and other support elements. (INCOSE)
[4.1.46] … combination of interacting elements organized to achieve one or more stated purposes. (ISO/IEC/IEEE 15288)
Thus, the usage of terminology throughout this handbook is clearly an elaboration of the fundamental idea that a system is a purposeful whole that consists of interacting parts.
An external view of a system must introduce elements that specifically do not belong to the system but do interact with the system. This collection of elements is called the operating environment or context and can include the users (or operators) of the system.
The internal and external views of a system give rise to the concept of a system boundary. In practice, the system boundary is a “line of demarcation” between the system itself and its greater context (to include the operating environment). It defines what belongs to the system and what does not. The system boundary is not to be confused with the subset of elements that interact with the environment.
The functionality of a system is typically expressed in terms of the interactions of the system with its operating environment, especially the users. When a system is considered as an integrated combination of interacting elements, the functionality of the system derives not just from the interactions of individual elements with the environmental elements but also from how these interactions are influenced by the organization (interrelations) of the system elements. This leads to the concept of system architecture, which ISO/IEC/IEEE 42010 (2011) defines as
the fundamental concepts or properties of a system in its environment embodied in its elements, relationships, and in the principles of its design and evolution.
This definition speaks to both the internal and external views of the system and shares the concepts from the definitions of a system.
In general, engineering can be regarded as the practice of creating and sustaining services, systems, devices, machines, structures, processes, and products to improve the quality of life—getting things done effectively and efficiently. The repeatability of experiments demanded by science is critical for delivering practical engineering solutions that have commercial value. Engineering in general and SE in particular draw heavily from the terminology and concepts of science.
An attribute of a system (or system element) is an observable characteristic or property of the system (or system element). For example, among the various attributes of an aircraft is its air speed. Attributes are represented symbolically by variables. Specifically, a variable is a symbol or name that identifies an attribute. Every variable has a domain, which could be but is not necessarily measurable. A measurement is the outcome of a process in which the system of interest (SOI) interacts with an observation system under specified conditions. The outcome of a measurement is the assignment of a value to a variable. A system is in a state when the values assigned to its attributes remain constant or steady for a meaningful period of time (Kaposi and Myers, 2001). In SE and software engineering, the system elements (e.g., software objects) have processes (e.g., operations) in addition to attributes. These have the binary logical values of being either idle or executing. A complete description of a system state therefore requires values to be assigned to both attributes and processes. Dynamic behavior of a system is the time evolution of the system state. Emergent behavior is a behavior of the system that cannot be understood exclusively in terms of the behavior of the individual system elements.
The key concept used for problem solving is the black box/white box system representation. The black box representation is based on an external view of the system (attributes). The white box representation is based on an internal view of the system (attributes and structure of the elements). There must also be an understanding of the relationship between the two. A system, then, is represented by the (external) attributes of the system, its internal attributes and structure, and the interrelationships between these that are governed by the laws of science.
Early pioneers of SE and software engineering, such as Yourdon (1989) and Wymore (1993), long sought to bring discipline and precision to the understanding and management of the dynamic behavior of a system by seeking relations between the external and internal representations of the system. Simply stated, they believed that if the flow of dynamic behavior (the system state evolution) could be mapped coherently into the flow of states of the constituent elements of the system, then emergent behaviors could be better understood and managed.
Klir (1991) complemented the concepts of a system in engineering and science with a general systems methodology. He regarded problem solving in general to rest upon a principle of alternatively using abstraction and interpretation to solve a problem. He considered that his methodology could be used both for system inquiry (i.e., the representation of an aspect of reality) and for system definition (i.e., the representation of purposeful man-made objects).
In the ISO/IEC/IEEE usage of terminology, the system elements can be atomic (i.e., not further decomposed), or they can be systems on their own merit (i.e., decomposed into further subordinate system elements). The integration of the system elements must establish the relationship between the effects that organizing the elements has on their interactions and how these effects enable the system to achieve its purpose.
One of the challenges of system definition is to understand what level of detail is necessary to define each system element and the interrelations between elements. Because the SOIs are in the real world, this means that the response to this challenge will be domain specific. A system element that needs only a black box representation (external view) to capture its requirements and confidently specify its real-world solution definition can be regarded as atomic. Decisions to make, buy, or reuse the element can be made with confidence without further specification of the element. This leads to the concept of hierarchy within a system.
One approach to defining the elements of a system and their interrelations is to identify a complete set of distinct system elements with regard only to their relation to the whole (system) by suppressing details of their interactions and interrelations. This is referred to as a partitioning of the system. Each element can be either atomic or it can be a much higher level that could be viewed as a system itself. At any given level, the elements are grouped into distinct subsets of elements subordinated to a higher level system, as illustrated in Figure 2.1. Thus, hierarchy within a system is an organizational representation of system structure using a partitioning relation.
Figure 2.1 Hierarchy within a system.
This figure is adapted from ISO/IEC/IEEE 15288:2015, Figure 1 on page 11 and Figure 2 on page 12, with permission from the ANSI on behalf of the ISO. © ISO 2015. All rights reserved.
The concept of a system hierarchy described in ISO/IEC/IEEE 15288 is as follows:
[5.2.2] The system life cycle processes … are described in relation to a system that is composed of a set of interacting system elements, each of which can be implemented to fulfill its respective specified requirements.
The art of defining a hierarchy within a system relies on the ability of the systems engineer to strike a balance between clearly and simply defining span of control and resolving the structure of the SOI into a complete set of system elements that can be implemented with confidence. Urwick (1956) suggests that a possible heuristic is for each level in the hierarchy to have no more than 7 ± 2 elements subordinate to it. Others have also found this heuristic to be useful (Miller, 1956). A level of design with too few subordinate elements is unlikely to have a distinct design activity. In this case, both design and verification activities may contain redundancy. In practice, the nomenclature and depth of the hierarchy can and should be adjusted to fit the complexity of the system and the community of interest.
A “system of systems” (SoS) is an SOI whose elements are managerially and/or operationally independent systems. These interoperating and/or integrated collections of constituent systems usually produce results unachievable by the individual systems alone. Because an SoS is itself a system, the systems engineer may choose whether to address it as either a system or as an SoS, depending on which perspective is better suited to a particular problem.
The following characteristics can be useful when deciding if a particular SOI can better be understood as an SoS (Maier, 1998):
Operational independence of constituent systems
Managerial independence of constituent systems
Evolutionary development processes
Figure 2.2 illustrates the concept of an SoS. The air transport system is an SoS comprising multiple aircraft, airports, air traffic control systems, and ticketing systems, which along with other systems such as security and financial systems facilitate passenger transportation. There are equivalent ground and maritime transportation SoS that are all in turn part of the overall transport system (an SoS in the terms of this description).
Figure 2.2 Example of the systems and systems of systems within a transport system of systems.
Reprinted with permission from Judith Dahmann. All other rights reserved.
The SoS usually exhibits complex behaviors, often created by the existence of the aforementioned Maier’s characteristics. “Complexity” is essentially different from “complicated.” In complicated systems, such as an automobile, the interactions between the many parts are governed by fixed relationships. This allows reasonably reliable prediction of technical, time, and cost issues. In complex systems, such as the air transport system, interactions between the parts exhibit self-organization, where local interactions give rise to novel, nonlocal, emergent patterns. Complicated systems can often become complex when the behaviors change, but even systems of very few parts can sometimes exhibit surprising complexity.
The best way to understand a complicated system is to break it down into parts recursively until the parts are so simple that we understand them and then to reassemble the parts to understand the whole. However, this approach does not help us to understand a complex system, because the emergent properties that we really care about disappear when we examine the parts in isolation. A fundamentally different approach is required to understand the whole in context through iterative exploration and adaptation. As a result, SE requires a balance of linear, procedural methods for sorting through complicatedness (“systematic activity”) and holistic, nonlinear, iterative methods for harnessing complexity (“systemic” or systems thinking and analysis—always required when dealing with SoS). The tension between breaking things apart and keeping them in context must be dynamically managed throughout the SE process.
The following challenges all influence the engineering of an SoS (Dahmann, 2014):
—In an SoS, each constituent system has its own local “owner” with its stakeholders, users, business processes, and development approach. As a result, the type of organizational structure assumed for most traditional SE under a single authority responsible for the entire system is absent from most SoS. In an SoS, SE relies on crosscutting analysis and on composition and integration of constituent systems, which in turn depend on an agreed common purpose and motivation for these systems to work together toward collective objectives that may or may not coincide with those of the individual constituent systems.
—Recognizing that the lack of common authorities and funding poses challenges for SoS, a related issue is the challenge of leadership in the multiple organizational environment of an SoS. This question of leadership is experienced where a lack of structured control normally present in SE requires alternatives to provide coherence and direction, such as influence and incentives.
Constituent systems’ perspectives
—SoS are typically composed, at least in part, of in-service systems, which were often developed for other purposes and are now being leveraged to meet a new or different application with new objectives. This is the basis for a major issue facing SoS SE, that is, how to technically address issues that arise from the fact that the systems identified for the SoS may be limited in the degree to which they can support the SoS. These limitations may affect initial efforts at incorporating a system into an SoS, and systems’ commitments to other users may mean that they may not be compatible with the SoS over time. Further, because the systems were developed and operate in different situations, there is a risk that there could be a mismatch in understanding the services or data provided by one system to the SoS if the particular system’s context differs from that of the SoS.
Capabilities and requirements
—Traditionally (and ideally), the SE process begins with a clear, complete set of user requirements and provides a disciplined approach to develop a system to meet these requirements. Typically, SoS are comprised of multiple independent systems with their own requirements, working toward broader capability objectives. In the best case, the SoS capability needs are met by the constituent systems as they meet their own local requirements. However, in many cases, the SoS needs may not be consistent with the requirements for the constituent systems. In these cases, SoS SE needs to identify alternative approaches to meeting those needs either through changes to the constituent systems or through the addition of other systems to the SoS. In effect, this is asking the systems to take on new requirements with the SoS acting as the “user.”
Autonomy, interdependencies, and emergence
—The independence of constituent systems in an SoS is the source of a number of technical issues facing SE of SoS. The fact that a constituent system may continue to change independently of the SoS, along with interdependencies between that constituent system and other constituent systems, adds to the complexity of the SoS and further challenges SE at the SoS level. In particular, these dynamics can lead to unanticipated effects at the SoS level leading to unexpected or unpredictable behavior in an SoS even if the behavior of the constituent systems is well understood.
Testing, validation, and learning
—The fact that SoS are typically composed of constituent systems that are independent of the SoS poses challenges in conducting end-to-end SoS testing, as is typically done with systems. First, unless there is a clear understanding of the SoS-level expectations and measures of those expectations, it can be very difficult to assess the level of performance as the basis for determining areas that need attention or to ensure users of the capabilities and limitations of the SoS. Even when there is a clear understanding of SoS objectives and metrics, testing in a traditional sense can be difficult. Depending on the SoS context, there may not be funding or authority for SoS testing. Often, the development cycles of the constituent systems are tied to the needs of their owners and original ongoing user base. With multiple constituent systems subject to asynchronous development cycles, finding ways to conduct traditional end-to-end testing across the SoS can be difficult if not impossible. In addition, many SoS are large and diverse, making traditional full end-to-end testing with every change in a constituent system prohibitively costly. Often, the only way to get a good measure of SoS performance is from data collected from actual operations or through estimates based on modeling, simulation, and analysis. Nonetheless, the SoS SE team needs to enable continuity of operation and performance of the SoS despite these challenges.
—SoS is a relatively new area, with the result that there has been limited attention given to ways to extend systems thinking to the issues particular to SoS. Work is needed to identify and articulate the crosscutting principles that apply to SoS in general and to develop working examples of the application of these principles. There is a major learning curve for the average systems engineer moving to an SoS environment and a problem with SoS knowledge transfer within or across organizations.
Beyond these general SE challenges, in today’s environment, SoS pose particular issues from a security perspective. This is because constituent system interface relationships are rearranged and augmented asynchronously and often involve commercial off-the-shelf (COTS) elements from a wide variety of sources. Security vulnerabilities may arise as emergent phenomena from the overall SoS configuration even when individual constituent systems are sufficiently secure in isolation.
The SoS challenges cited in this section require SE approaches that combine both the systematic and procedural aspects described in this handbook with holistic, nonlinear, iterative methods.
Enabling systems are systems that facilitate the life cycle activities of the SOI. The enabling systems provide services that are needed by the SOI during one or more life cycle stages, although the enabling systems are not a direct element of the operational environment. Examples of enabling systems include collaboration development systems, production systems, logistics support systems, etc. They enable progress of the SOI in one or more of the life cycle stages. The relationship between the enabling system and the SOI may be one where there is interaction between both systems or one where the SOI simply receives the services it needs when it is needed. Figure 2.3 illustrates the relationship of the SOI, enabling systems, and the other systems in the operational environment.
Figure 2.3 System of interest, its operational environment, and its enabling systems.
This figure is excerpted from ISO/IEC/IEEE 15288:2015, Figure 3 on page 13, with permission from the ANSI on behalf of the ISO. © ISO 2015. All rights reserved.
During the life cycle stages for an SOI, it is necessary to concurrently consider the relevant enabling systems and the SOI. All too often, it is assumed that the enabling systems will be available when needed and are not considered in the SOI development. This can lead to significant issues for the progress of the SOI through its life cycle.
SE is a perspective, a process, and a profession, as illustrated by these three representative definitions:
Systems engineering is an interdisciplinary approach and means to enable the realization of successful systems. It focuses on defining customer needs and required functionality early in the development cycle, documenting requirements, and then proceeding with design synthesis and system validation while considering the complete problem: operations, cost and schedule, performance, training and support, test, manufacturing, and disposal. Systems engineering integrates all the disciplines and specialty groups into a team effort forming a structured development process that proceeds from concept to production to operation. Systems engineering considers both the business and the technical needs of all customers with the goal of providing a quality product that meets the user needs.
Systems engineering is an iterative process of top-down synthesis, development, and operation of a real-world system that satisfies, in a near optimal manner, the full range of requirements for the system.
Systems engineering is a discipline that concentrates on the design and application of the whole (system) as distinct from the parts. It involves looking at a problem in its entirety, taking into account all the facets and all the variables and relating the social to the technical aspect.
Certain keywords emerge from this sampling—interdisciplinary, iterative, sociotechnical, and wholeness.
The SE perspective is based on systems thinking. Systems thinking (see Section 2.9.2) is a unique perspective on reality—a perspective that sharpens our awareness of wholes and how the parts within those wholes interrelate. When a system is considered as a combination of system elements, systems thinking acknowledges the primacy of the whole (system) and the primacy of the relation of the interrelationships of the system elements to the whole. Systems thinking occurs through discovery, learning, diagnosis, and dialogue that lead to sensing, modeling, and talking about the real world to better understand, define, and work with systems. A systems thinker knows how systems fit into the larger context of day-to-day life, how they behave, and how to manage them.
The SE process has an iterative methodology that supports discovery, learning, and continuous improvement. As the process unfolds, systems engineers gain insights into the relationships between the specified requirements for the system and the emergent properties of the system. Insights into the emergent properties of a system can therefore be gained from understanding the interrelationships of the system elements and the relation of these to the whole (system). Due to circular causation, where one system variable can be both the cause and effect of another, even the simplest of systems can have unexpected and unpredictable emergent properties. Complexity, as discussed in Section 2.4, can further exacerbate this problem; hence, one of the objectives of the SE process is to minimize undesirable consequences. This can be accomplished through the inclusion of and contributions from experts across relevant disciplines coordinated by the systems engineer.
SE includes both technical and management processes, and both processes depend upon good decision making. Decisions made early in the life cycle of a system, whose consequences are not clearly understood, can have enormous implications later in the life of a system. It is the task of the systems engineer to explore these issues and make the critical decisions in a timely manner. The roles of the systems engineer are varied, and Sheard’s “Twelve Systems Engineering Roles” (1996) provides one description of these variations.
The modern origins of SE can be traced to the 1930s followed quickly by other programs and supporters (Hughes, 1998). Tables 2.1 and 2.2 (Martin, 1996) offer a thumbnail of some important highlights in the origins and standards of SE. A list of current significant SE standards and guides is provided in Table 2.3.
Table 2.1Important dates in the origins of SE as a discipline
British multidisciplinary team to analyze the air defense system
Bell Labs supported NIKE missile project development
SAGE air defense system defined and managed by Massachusetts Institute of Technology (MIT)
Recommendation by the RAND Corporation to adopt the term “systems engineering”
Invention of systems analysis by RAND Corporation
A Methodology for Systems Engineering
Modeling of urban systems at MIT by Jay Forrester
National Council on Systems Engineering (NCOSE) established
INCOSE emerged from NCOSE to incorporate international view
ISO, IEC, IEEE, INCOSE, PSM, and others fully harmonize SE concepts on ISO/IEC/IEEE 15288:2008
Table 2.2Important dates in the origin of SE standards
Army Field Manual 770-78
Perry Memorandum urges military contractors to adopt commercial practices. EIA 632 IS (interim standard) and IEEE 1220 (trial version) issued instead of Mil-Std 499B
EIA 632 released
IEEE 1220 released
ISO/IEC 15288 released, adopted by IEEE in 2003
Guide to the Systems Engineering Body of Knowledge (SEBoK)
Table 2.3Current significant SE standards and guides
Systems and software engineering—System life cycle processes
Processes for engineering a system
Systems engineering—Application and management of the systems engineering process (replaces IEEE 1220™)
Guide to the systems engineering body of knowledge
Tysiące ebooków i audiobooków
Ich liczba ciągle rośnie, a Ty masz gwarancję niezmiennej ceny.
Napisali o nas:
Nowy sposób na e-księgarnię
Czytelnicy nie wierzą
Legimi idzie na całość
Projekt Legimi wielkim wydarzeniem
Spotify for ebooks