• Air Traffic Control: Immature Software Acquisition Processes Increase FAA System Acquisition Risks The Federal Aviation Administration (FAA) is spending billions of dollars to modernize software-intensive air traffic control systems. The General Accounting Office (GAO) examined the processes used to acquire software, using models developed by Carnegie Mellon University's Software Engineering Institute to define and determine an organization's software process maturity. FAA did not fully meet any of the criteria to achieve a "repeatable" level of maturity, the second of a five-level maturity scale.

• Esprit - Risk Driven Software Process Improvement The aim of DriveSPI is to produce and validate by trial applications a European framework for improving the software process maturity with strong emphasis on risk management.

• CRM Certified Instructors These are instrutors that are certified in Continuous Risk Management (CRM) and recognised by NASA Software Assurance Technology Center (SATC). There several flight and research centers listed with links to thier instructors.

• Continuous Risk Management (SEI Course) This three-day course is based on the Continuous Risk Management Guidebook, and it provides an extended case study (not contained in the guidebook). The case study exercises are used to provide an integrated, consistent view of continuous Risk Management and how it could be implemented in a typical project.

• Information Security and Software Risk Management (Johns Hopkins University Course) This course offers a comprehensive review of current risk management and security methods used in today's global communications. The course describes current industry practices and the latest advances in Software Risk Management. The course will benefit Program Managers, Project Managers, Systems Analysts, Software Developers, and Users engaged in the development of modern software and software-based systems. (Course No. S00-717)

• Introduction to the Software Acquisition Capability Maturity Model (SA-CMM) (SEI Course) This course provides a two-day introduction to the SA-CMM. The course is designed to give participants an overview of the SA-CMM model and its fundamental concepts.

• Motivation for Software Risk Management (SEI Video) This Software Engineering Institute (SEI) video provides a high-level introduction to software Risk Management delivered by one of the world's foremost experts on software Risk Management. Robert N. Charette provides specific examples of companies using Risk Management today to maintain a competitive edge in the marketplace. (25 Minutes)

• Software Engineering for Program Managers (DACS Course) This course is designed for professionals involved in software engineering or program management. Topics covered during the course include; Program Management and Software Engineering; Software Process Maturity (SPM); Life Cycle Management; Software Project Management; and Software Risk Management.

• Software Risk Management Conference (Formerly International Software Assurance Certification Conference or ISACC) Three days of expert speakers, hands-on tutorials, and break out sessions as well as one-on-one consulting will help you increase your knowledge base and rise to the challenge of increasing your software's security, safety, and reliability.

• System Engineering (DACS Course) The seminar provides an understanding of System Engineering as it is practiced in the DoD and associated government agencies. Topics discussed in the seminar include System Engineering Overview; Requirements Engineering; Software Development; System Architecture; Risk Management; Performance Measurement and Evaluation; Life Cycle Acquisition; Life Cycle Costing; Support and Specialty Engineering; System Software Engineering Tools; and Trends and Directions.

• Up the Down Escalator (SEI Video) Using the philosophy that a business standing still is losing ground, Up the Down Escalator's presenter, Robert N. Charette, addresses why businesses must manage risk to survive. Charette presents a dynamic model for thinking about how Risk Management can be used to improve an organization's competitive position in the marketplace. The video presentation puts Risk Management improvement into perspective with other quality improvement activities and integrates business management, process improvement, and risk management into a coherent whole. (43 Minutes)

• Anderson, Peter Peter G. Anderson is contractor who supports the Safety & Mission Assurance organization at Marshall Space Flight Center in the areas of Space Shuttle Integration Product Assurance, Risk Management and shuttle transition.

  Pete spent eight years as a propulsion systems engineer on the Saturn/Apollo program, including being involved in the "go-no go" decisions for leaving earth orbit and heading for the moon. He also has over 25 years in computational fluid dynamics (CFD) dealing mainly with code development and solutions to internal fluid flow problems on the Space Shuttle Main Engines. In addition, he developed real fluid and chemical kinetic models for CFD applications, used method of moments procedures for describing soot formation in hybrid rocket engines and analyzed the effects of nuclear blast waves on fully and partially deployed "pop up" phased radar domes. Peter holds a Bachelor of Science degree from Georgia Tech. Peter Anderson may be contacted at: Peter.Anderson@msfc.nasa.gov.

• Best, Tim Mr. Best is a Project Assurance Manager for the Risk Management Office of the Office of Safety and Assurance Technologies at Glenn Research Center. In this position he is responsible for working closely with the Space Power and Propulsion Division and Space Transportation Division Managers assessing Program/Project needs for Safety and Mission Assurance (SMA). His seventeen-year civil servant career has been with both the US Air Force and NASA. His assignments with NASA have included key materials and processes, and SMA roles for programs and projects in the Aeronautics and Space Directorates.

  Mr. Best holds a Masters of Science Degree in Industrial Engineering from Cleveland State University. He also holds a Bachelors Degree in Metallurgical Engineering from Youngstown State University. Tim Best may be contacted at: Timothy.D.Best@lerc.nasa.gov.

• Botzum, Steve Mr. Botzum is a System Analyst with the Software Assurance Technology Center (SATC) at NASA's Goddard Space Flight Center. Mr. Boztum is a certified Continuous Risk Management (CRM) course trainer responsible for providing CRM training and support at NASA sites throughout the country. He also conducts research into and reviews Risk Management tools and processes for use in updating CRM course materials. Mr. Botzum joined the SATC in August 1999.

  Prior to joining Unisys, Mr. Botzum spent 20 years in the U.S. Naval Intelligence Group, as an Intelligence Analyst providing real-time intelligence support to military commanders while stationed at various locations worldwide. Mr. Botzum is a highly-skilled Project Manager, Intelligence Analyst, Training Manager and Linguist, being knowledgeable in French, German, and Greek. Steve Botzum may be contacted at: sbotzum@pop300.gsfc.nasa.gov.

• Charette, Robert N. Robert N. Charette is a Cutter Consortium Fellow and the Director of the Risk Management Service. He is a frequent contributor to the Agile Project Management and Business-IT Strategies Practices. With more than 25 years' experience in a wide variety of international technology and management positions, Dr. Charette is recognized as an international authority and pioneer regarding information systems, technology, and telecommunications Risk Management.

  Dr. Robert N. Charette is the President of ITABHI Corporation, an international Risk Management consultancy company. Dr. Charette is past Chairperson of both the SEI Risk Advisory Board and NSIA Software Committee, is a founding member of the PMI Risk SIG, and risk management advisor to the PSM project. Dr. Charette has written dozens of papers and several books on Risk Management, including Software Engineering Risk Analysis and Management, Applications Strategies for Risk Analysis and An Introduction to the Management of Risk. (Charette@erols.com)

• Defense Acquisition Deskbook - Ask a Professor The Ask an Expert service accepts questions to experts at the Defense Acquisition Deskbook. Topics covered by the service include acquisition policy, acquisition logistics, business cost estimating, business cost financial management, computer systems acquisitions, Continuous Acquisition and Lifecycle Support (CALS), and security.

• Flippen, Alexis Ms. Alexis Flippen is the staff Risk Analyst at Hernandez Engineering Inc. (HEI) at the NASA Ames Research Center, working in support of the System Safety and Mission Assurance Division. With more than 20 years experience in the DoD and NASA arenas, she leads and directs SS&MA studies and develops risk analysis methodologies and safety policies. During her tenure at Ames, she has received multiple awards in recognition of her work in Risk Management.

  Prior to joining HEI, Ms. Flippen worked at ARC as a Faculty Research Scientist from the University of Southern Colorado. She has also served as the Lead System Safety Engineer at the Lockheed Engineering & Sciences Company and Boeing Aerospace Operations at the center. Ms. Flippen holds a M.S. from the University of the Pacific at Pacific Medical Center in San Francisco and is currently a Ph.D.candidate in industrial engineering at the University of Bradford, UK. Alexis Flippen may be contacted at: aflippen@mail.arc.nasa.gov.

• Gallo, Al Al Gallo - manages the Systems Quality Assurance Department at GSFC, NASA. Mr. Gallo has 15 years of Software Systems Engineering and Quality Assurance experience and has experience in all phases of Systems Development with an emphasis on database design. Mr. Gallo is also one of the lead trainers at the SATC in Continuous Risk Management (CRM), having provided training and consulting throughout the NASA agency. He also serves as a primary reviewer for updates and maintenance to course materials as well as project-developed Risk Management Plans.

  Mr. Gallo holds Bachelors degrees in both Pure Mathematics and Computer Science as well as an M.S. in Technical Management from the Johns Hopkins University, Baltimore MD. Prior to joining Unisys, Mr. Gallo was Senior Systems Analyst at the Actuarial Division of the US Department of Labor in Washington, DC. Al Gallo can be contacted at: al.gallo@gsfc.nasa.gov.

• Galuska, Mike Mike Galuska is an aerospace engineer working in the Safety, Reliability and Quality Assurance Department where he serves as the Systems Safety and Risk Management Lead. In his time at NASA, he has also performed the duties as the Safety and Mission Assurance Lead on the Tethered Satellite System, Multipurpose Logistics Module, and ISS Nodes 2 and 3.

  Prior to joining NASA in 1989, he worked as a design engineer for 13 years with the Tennessee Valley Authority on various hydroelectric and nuclear power plants. Mr. Galuska holds a BSME from the University of Miami and a MS in Engineering Mechanics from the University of Tennessee. Mike Galuska may be contacted at: Mike.Galuska@msfc.nasa.gov.

• Hankinson, Allen Allen L. Hankinson (Al) is a principal systems engineering consultant with the Software Assurance Technology Center (SATC). Mr. Hankinson has more than 33 years of professional software management and engineering experience in Government and Industry. He is an authority on distributed systems standards and software engineering environments. Mr. Hankinson holds a B.S. in Mathematics and a M.S in Computer Science. He is a former member of the Board of Directors of the IEEE Computer Society and the Board of Directors of UNIFORUM International (the international association of open system professionals).

  Prior to joining the SATC, Mr. Hankinson was Senior Software Architect, for the Health Care Technology Group, Science Applications International Corporation (SAIC). As Chief of the Systems and Software Technology Division within the Computer Systems Laboratory of the National Institute of Standards and Technology (NIST), Mr. Hankinson was the chief architect for establishing the U.S. federal government's open systems environment initiative. He has extensive experience with the concepts and technologies that enable open, distributed, heterogeneous computing. Al Hankinson can be contacted at: Allen.L.Hankinson.1@gsfc.nasa.gov.

• Havenhill, Maria Ms. Havenhill joined the Software Product Assurance group in the fall of 1999. She brings 6 years experience as a system safety engineer for various glovebox investigations, space experiments, and the Fluids Integrated Rack of the FCF. Past activities include participating in the development of the Spaceflight Safety Handbook, creating the GRC system safety website, and authoring the NASA safety process chapter during the development of the NASA software safety course. She is considered a software safety expert. Ms. Havenhill holds a BS and MS in mechanical engineering from Case Western Reserve University. Maria Havenhill may be contacted at: Maria.A.Havenhill@lerc.nasa.gov

• Johnson, Paul Paul W. Johnson is an Aerospace Engineer at NASAs Marshall Space Flight Center where he works as a lead safety engineer in the Safety and Mission Assurance Office. His current assignments include assurance activities related to space flight hardware as well as membership on the MSFC Safety Review Panel. Mr. Johnson previously supported numerous Space Shuttle missions at the Neutral Buoyancy Facility as a utility and cameral SCUBA diver.

  Mr. Johnson first came to NASA as a cooperative intern in 1982. He subsequently entered the US Navy where he flew jet aircraft and accumulated over 1500 flight hours. Upon his return to NASA, Mr. Johnson also served in a diplomatic capacity at the NASA Liaison Office, Moscow, before his return to MSFC. Mr. Johnson holds a BS degree in Mechanical Engineering from the University of Louisville in Kentucky. In addition to his regular duties, he also volunteered to be a site instructor of Continuous Risk Management at MSFC and was one of the first individuals to become certified there. Paul Johnson may be contacted at: Paul.Johnson@msfc.nasa.gov

• Kelm, Gary Mr. Kelm is a registered Professional Engineer with a B.S. degree in Mechanical Engineering and a wide range of experience with NASA and private industry. Before assuming his current position, Program Assurance Manager for space, he served as Program Assurance Manager for Microgravity Science, Project Assurance Manager for the Advanced Communications Technology Satellite (ACTS) and Verification Manager for the Space Station Electrical Power System. In private industry, Mr. Kelm worked to develop engine oil and other additives for the automotive industry. His work included overseeing product qualification programs for marketing, directing gear lubricant testing and managing a fuel economy/emissions laboratory and road simulator facility.

  His early career at NASA included research to develop Stirling engines, biomedical and energy projects under the NASA Technology Utilization program, and test operations engineering for wind tunnel and acoustic research to improve V/STOL aircraft. Mr. Kelm has a broad background in Risk Management. He highly recommends the NASA CRM course because it is practical, focused and offers immediate and long-term benefits to the projects and students who participate. Gary Kelm may be contacted at: Gary.G.Kelm@lerc.nasa.gov.

• Kurtz, Tim Mr. Kurtz worked for Defense Contract Management Command (DCMC), for the 13 years prior to joining SAIC. During that time, he was the program manager for the Mk 48 ADCAP torpedo program at DPRO Westinghouse. He also implemented the software quality assurance program and monitored the transfer of software and development of test equipment for the Mk 50 torpedo from Hughes to Westinghouse.

  In 1989 Tim served DCMC Dayton as the Staff SQA Specialist were he was responsible for training and overseeing the SQA activities of nine Software Quality Assurance Specialists who monitored DoD software development contracts and the development and maintenance of all Air Force simulators. Trained in ISO 9000 auditing and Software Development Capability Evaluation Training, Tim developed and implemented the ISO 9000 Qualification Audit system for DCMC Dayton to provide second party ISO certification to defense contractors and provided software certification training for all Software Professional Development Program applicants in DCMC. Tim received Software Professional Development Program level II certification. Tim Kurtz may be contacted at: Timothy.J.Kurtz@lerc.nasa.gov.

• Mendoza, Dr. Donald R. Dr. Mendoza has worked as an aerospace engineer for the United States Air Force at the Flight Tests Center at Edward's AFB in California and as an independent consultant. Dr. Mendoza also held an appointment as a National Research Council Associate conducting independent research in fluid mechanics.

  Dr. Mendoza received his BS degree in aeronautical engineering from the California Polytechnic State University at San Luis Obispo in 1985 and his Ph.D. degree in mechanical engineering from the University of California at Berkeley in 1996. Dr. Donald Mendoza may be contacted at: drmendoza@mail.arc.nasa.gov.

• Motivation for Software Risk Management (SEI Video) This Software Engineering Institute (SEI) video provides a high-level introduction to Software Risk Management delivered by one of the world's foremost experts on Software Risk Management. Robert N. Charette provides specific examples of companies using Risk Management today to maintain a competitive edge in the marketplace. (25 Minutes)

• O'Neill, Don As an independent consultant, Don O€Neill conducts defined programs for managing strategic software improvement. These include directing the National Software Quality Experiment, participating in the National Software Council, and producing and maintaining the section on software inspections in the Software Engineering Institute (SEI) Software Technology Reference Guide. Contact Don at: ONeillDon@aol.com

• Packard, Michael Mr. Packard works for Science Applications International Corporation (SAIC), supporting the work of the Office of Safety and Assurance Technology, Risk Management Office at NASA Glenn Research Center. As a Senior Engineer he performs risk and reliability assessments, design review, and failure analysis, and is developing curriculum and teaching courses in system reliability and Risk Management. He has over 20 years experience in test and measurement engineering, design engineering, reliability engineering, and failure analysis engineering in the aerospace and automotive industry. He also has been responsible for management of prototype research and development facilities and supervision of reliability engineering. He has taught courses in statistics and economics at Ashland University and other technical colleges and authored a number of technical papers on testing methodologies, reliability and Software Quality. Mr. Packard has a degree in Mechanical Engineering from Cleveland State University and an MBA from Ashland University. Michael Packard may be contacted at: Michael.H.Packard@lerc.nasa.gov.

• Remp, Kerry Mr. Remp is the Plum Brook Station Resident for the Risk Management Office of the Office of Safety and Assurance Technologies at Glenn Research Center. In this position he is active in both the long-term as well as day-to-day risk assessment of unique, world-class testing facilities. He is also an active CRM instructor for Glenn Research Center Space and Facilities projects. Since the early1980's, he has been an engineer, manager and risk facilitator involved in safety, quality and Risk Management activities for nuclear, space, and aeronautics programs and facilities. Mr. Remp holds a Bachelor of Science Degree in Marine Engineering from the U.S. Merchant Marine Academy with a minor in Nuclear Engineering. Kerry Remp may be contacted at: Kerry.L.Remp@grc.nasa.gov.

• Rosenberg, Linda Dr. Linda H. Rosenberg is the NASA manager of the Software Assurance Technology Center (SATC) and oversees all SATC work areas. She has implemented a variety of metric programs for NASA that establish a basis for numerical guidelines and standards for software development. She has been instrumental in assisting GSFC managers use metrics to assess project risks and improve management of software development. Dr. Rosenberg has presented numerous tutorials and original technical papers on relationships between software metrics, project management, and product quality improvement. Dr. Rosenberg is also skilled in the areas of hypertext, specification languages, and user interfaces.

  Dr. Rosenberg holds a Ph.D. in Computer Science, a M.E.S. with emphasis in Computer Science, and a B.S. in Mathematics. She is a member of Electrical and Electronic Engineers (IEEE), the IEEE Computer Society, the Association for Computing Machinery (ACM) and Upsilon Pi Epsilon. Prior to managing the SATC, Dr. Rosenberg was an Assistant Professor in the Mathematics/Computer Science Department at Goucher College. Dr. Linda Rosenberg may be contacted at: Linda.Rosenberg@gsfc.nasa.gov.

• Scaglione, Lois Dr. Lois Scaglione works in the Risk Management Office at GRC, where she works closely with Aeronautics Managers assessing Program needs for Safety and Mission Assurance (SMA). Over the past seventeen years, her assignments have included key SMA roles in the management of electronic parts for such programs as Atlas/Shuttle Centaur, Space Station Freedom and Space Experiments. Dr. Scaglione support's the agency's effort to standardize the practice and implementation of Continuous Risk Management at GRC. She has identified programs and projects that could benefit and has established workshop metrics to achieve center-based goals.

  Dr. Scaglione holds a Ph.D. in Engineering from Cleveland State University with a focus on applied mathematical modeling and simulation of electronic Particle Impact Noise Detection testing and neural networks. She also holds a Masters degrees in Engineering as well as Chemistry and has bachelors degrees in both Electrical Engineering and Chemistry. In addition to NASA experience, Dr. Scaglione has extensive teaching experience at the Naval Postgraduate School, the Ohio Aerospace Institute, Cleveland State University, the University of Dayton and Indiana University of Pennsylvania. Dr. Scaglione may be contacted at: Lois.J.Scaglione@lerc.nasa.gov.

• Shivers, Herb Dr. Herb Shivers is the manager of the Safety, Reliability, and Quality Assurance Department in Marshall Space Flight Center's Safety and Mission Assurance Office (S & MA). The department is responsible for System Safety, Reliability and Quality Assurance discipline guidance; the MSFC Payload Safety Readiness Review Board; Shuttle Quantitative Risk Assessments; Risk Management Guidance and Support; Industrial Safety; Test Area Quality and Safety monitoring; Program Critical hardware moves; the center's ISO-9000 compliance, administration and audits; in-house hardware inspections; vendor and supplier audits; Software Quality Assurance; NASA Engineering Quality Audits; and ALERTS tracking. S & MA also maintains a web site for both products and information.

Dr. Shivers has twenty-five years experience in the Health and Safety field. He has worked on industrial safety with the Department of the Army in ammunition plants, with the Tennessee Valley Authority corporate health and safety office, payload systems safety at MSFC and his current assignments. He holds a BIE from Auburn University, a MEIE from Texas A&M, and a Ph.D. in Industrial and Systems Engineering and Engineering Management from the University of Alabama in Huntsville. In addition, he is a part time lecturer in Engineering Economic Analysis at the University of Alabama in Huntsville. Dr. Shivers is a both a Professional Engineer licensed in Alabama as well as a Certified Safety Professional. Dr. Herb Shivers may be contacted at: Charles.H.Shivers@msfc.nasa.gov.

• Smidts, Carol A professor at the University of Maryland, Dr. Smidts' research areas focus on dynamic probabilistic risk assessment, human reliability, software reliability, quantitative risk assessment, and software testing. csmidts@eng.umd.edu

• Smith, Douglas L. Smith is a lead System Safety Engineer in the System Safety and Mission Assurance Division at Ames Research Center, NASA. He has 25 years of system safety, project management and engineering development experience in all phases of commercial production. He is responsible for system safety and mission assurance activities including development of Risk Management plans for a variety of aeronautics and facilities projects at ARC.

Mr. Smith holds a B.S. in Mechanical Engineering from Arizona State University, is a Registered Professional Engineer in the State of California, is president of a local chapter of the System Safety Society, and a member of that organization's national executive council.

Prior to joining NASA Mr. Smith was the Aeronautics Systems lead for Hernandez Engineering in support of NASA operations at Ames Research Center. Doug Smith may be contacted at: dlsmith@mail.arc.nasa.gov.

• Software Engineering Institute (SEI) - Risk Management FAQ - This resource form the SEI answers everything from "What is Risk Management?" to "What does success look like?". This FAQ is online or downloadable in PDF.

Software Program Manager's Network (SPMN) The SPMN's mission is to enable managers of large-scale, software-intensive development or maintenance projects to more effectively manage and succeed by identifying and conveying to them best management practices, lessons learned, and directly useful support.

• Arizona State University (ASU) - Software Risk Management A Risk Management overview, risk identification questionnaire, faculty profile, and expert system are highlighted on this site.

• Defense Acquisition Deskbook - Risk Management The Systems Engineering organization with DTSE&E is responsible for Risk Management in DoD and has, at the direction of the Undersecretary of Defense, Acquisition and Technology (USD (A&T)), examined DoD's approach to managing risk. Systems Engineering formed a Working Group, composed of representatives from the Services and other DoD agencies involved in systems acquisition, to assist in the evaluation of the Department's approach to Risk Management.

  This page provides bibliographic references to DoD and non-DoD Risk Management publications, speeches, policies and procedures as well as Risk Management Related Websites.

• International Software Benchmarking Standards Group, Ltd. (ISBSG) The ISBSG is a not for profit organization. The Group maintains, develops and exploits a repository (over 1,250 projects) of international software project metrics to help software developers with project estimation and benchmarking. ISBSG supports research and the publication of resulting information to support process improvement. This site includes the metrics repository, an estimation tool, and project risk resources.

• Seven Principles of Risk Management by the SEI Seven principles provide a framework to accomplish effective Risk Management: global perspective, forward-looking view, open communication, integrated management, continuous process, shared product vision, and teamwork.

• Software Acquisition Capability Maturity Model (SA-CMM) (an SEI model) - The SA-CMM is a model for benchmarking and improving the software acquisition process. The model follows the same architecture as the Capability Maturity Model for Software (SW-CMM^TM), but with a unique emphasis on acquisition issues and the needs of individuals and groups who are planning and managing software acquisition efforts.

• Software Acquisition Management Improvement (An SEI program) The goal of the software acquisition management improvement effort at the SEI is to lead the community to establish acquisition practices that are self-sustained and continuously improving. Software risk evaluation, Continuous Risk Management, and team Risk Management are studied by the SEI.

• Software Engineering Information Repository (SEIR) Maintained by the Software Engineering Institute, the SEIR is designed to serve the software engineering community in the role of gathering, coordinating, analyzing, and disseminating data and information on the impact of software engineering practices as well as practices and innovations leading to improvement.

• Software Engineering Institute (SEI) Risk Management Overview The SEI Risk Management paradigm illustrates a set of functions that are identified as continuous activities through the life cycle of a project.

• Software Engineering Risk: Understanding and Management (SERUM) Software projects have a high probability of failure so effective software development means dealing with risks adequately. The purpose of this project was principally to identify where and how risks should be addressed in the software development process. It also determined how Risk Management might be supported by tools and involved some prototype development. Particular emphasis was placed on risk associated with the identification and controlled evolution of computing systems within organisations. The project contributed to the development of the BASE methodology and included a consideration of Risk Management in the context of business analysis using the Soft Systems Methodology. Period: January 1995 - December 2000

• Software Program Manager's Network (SPMN) The SPMN's mission is to enable managers of large-scale, software-intensive development or maintenance projects to more effectively manage and succeed by identifying and conveying to them best management practices, lessons learned, and directly useful support.

• C/S Solutions, Inc. (C/SSI) C/SSI produces integrated analytical tools for cost, schedule, and Risk Management. Their tools are specifically designed to engage Integrated Product Development (IPD) team members and/or Cost Account Managers (CAMs) in proactive cost, schedule and Risk Management of complex programs.

• GRafP Technologies Inc. GRafP develops software packages which can be used to identify threats, and to analyze and manage the risks to which an entity (i.e. organization, project, individual, etc.) is exposed. Two such products are X:PRIMER and S:PRIMER. Services offered as part of that mission include risk ratings and assessments, process assessments, remedial action planning, and training.

• KLCI - KLCI helps software development organizations accelerate completion of their projects. Their methodologies include: Software Risk Management, software project management, and critical path management.

• R.S. Pressman & Associates, Inc. - R.S. Pressman provides services and products that help an organization to improve its software engineering practices. The company offers video training products, consulting services, and Software Process Improvement products. Their WWW site provides access to a comprehensive collection of software engineering resources.

• Risk Services & Technology (RST) - RST offers services in the following areas: Project Risk Management, DoD (Directive 5000.2-R, Clinger-Cohen Act), Earned Value Management Systems, and Risk Management Software Products.

• SEI Continuous Risk Management Service (CRM) The CRM Service from the Software Engineering Institute (SEI), incorporates all that the SEI has learned from its research and working with more than 50 clients in the field of Risk Management. This service tailors the SEI Continuous Risk Management processes, methods, and tools to a specific project or organization. The service integrates and adapts the practice of CRM, as defined in the Continuous Risk Management Guidebook, with current program management practices. The cornerstone of this service is the Risk Clinic; an on-site workshop that builds a tailored Risk Management Practice for the project and a plan for implementing the practice.

• Software Risk Evaluation Service (SRE), an SEI Service The SEI Software Risk Evaluation (SRE) Service is a diagnostic and decision making tool that enables the identification, analysis, tracking, mitigation, and communication of risks in software-intensive programs. An SRE is used to identify and categorize specific program risks emanating from product, process, management, resources, and constraints. The program's own personnel participate in the identification, analysis, and mitigation of risks facing their own development effort.

Articles and White Papers

• Assessing Project Risk - DoD Software Tech News Vol. 2, No. 2 by Shari Lawrence Pfleeger, University of Maryland. This article was adapted from her book, Software Engineering: Theory and Practice, with permission from Prentice-Hall. Topics covered include: What is a Risk?; Risk Management Activities; and Risk Exposure Calculation.

• Book Review: Software Engineering: A Practitioner's Approach - DoD Software Tech News Vol. 2, No. 1 - This Fourth Edition printing by Roger S. Pressman provides good coverage of the current thinking on Risk Management. The reader is introduced to Risk Management concepts used by Pressman, Charette, Boehm, the Software Engineering Institute (SEI), the Air Force and others.

• Criteria for Managing Technical Reviews and Audits - DACS Newsletter (Spring 1995) - Lillian Zelinski of the SAIC, discusses how under contract to the Program Executive Office for Cruise Missiles and Unmanned Aerial Vehicles (PEO-CU), the SAIC developed a set of criteria to facilitate the management of system development milestones (technical reviews and audits such as Critical Design Review and Functional Configuration Audit) for a system development effort.

• DoD Software Acquisition Management Overview - CrossTalk (April 1997) The Department of Defense (DoD) 5000 series directives for lifecycle management, coupled with benchmarking, simulation and modeling, and buy-vs.-make practices, provide a litmus test for the software acquisition management process. This test can help respective DoD organizations learn best practices to become more competitive, simulate the future environment before making a major investment, decide on making vs. buying software, and deploy software that improves their business. CrossTalk is a publication of the DoD Software Technology Support Center (STSC).

• Earned Value (EV) and Automated Information Systems (AIS) - DoD Software Tech News (1997; Vol. 1, No. 2) The discipline of a detailed plan and control of changes moves decisions about the software ahead of the development process. This identifies risks and forces evaluation of schedule and cost impact of changes. The DoD Software Tech News is a publication of the DoD Data & Analysis Center for Software (DACS).

• Measuring and Evaluating Maintenance Process using Reliability, Risk, and Test Metrics by Norman F. Schneidewind, Naval Postgraduate School. This article first appeared in IEEE Transactions on Software Engineering, Volume 25, Number 6, November/December 1999, pp. 768-781. Keywords: Measurement, Reliability, Risk, Testing & Metrics

  This link will donwload a PDF document.

• Process Improvement Models Continue to Evolve - DACS Newsletter (Winter 1994) The Software Engineering Institute's (SEI) Software Process Improvement Program is based on the Software Capability Maturity Model (CMM). Several discrete changes being considered for CMM (Version 2) were the addition of several new key process areas such as Risk Management, reuse, and reengineering.

• Risk Management - Crosstalk (April 1997) The theme of the April 1997 (Vol. 10, No. 4) issue of Crosstalk, the Journal of Defense Software Engineering, was Risk Management - Finding and Diffusing Problems before they Explode.

• Risk Management Map - DoD Software Tech News 2-2 by Dr. Elaine Hall, Level 6 Software. The Risk Management Map is a practical guide to understanding the path to increasing your ability to manage risk by transitions through five stages. At each stage, a vision provides the direction for your journey.

• Risk Management by Chester Simmons This white paper provides an introduction and discussion on Risk Management together with recommendations for its implementation. It is quite extensive.

• Risk Management: Coming of Age - DoD Software Tech News 2-2 by Dr. Robert N. Charette, ITABHI Corporation. This article discusses how Active Risk Management has been more of an afterthought than a primary factor in Department of Defense (DoD) decision making.

• Riskit: Increasing Confidence in Risk Management - DoD Software Tech News 2-2 by Jyrki Kontio and Victor R. Basili, this article discusses using systematic Risk Management to avoid a large number of problems by using procedures and techniques early in projects.

• Software Acquisition Risk: A Perspective - DoD Software Tech News 2-2 by Martin L. Shooman, Polytechnic University and Ernest Lofgren, SAIC, this article deals with two specific issues; Software Acquisition Risk(cost and schedule) and Software Reliability.

• Software Engineering Measurement and Analysis Initiative at the SEI - DoD Software Tech News (1997; Vol. 1, No. 2) Software Measurement lies at the heart of the answer to questions about project control, organizational performance, and return on investment. The Software Engineering Measurement and Analysis (SEMA) Initiative at the Software Engineering Institute (SEI) established a data analysis testbed, or laboratory to investigate the application of statistical and other analytical techniques to software engineering problems. The DoD Software Tech News is a publication of the Data & Analysis Center for Software (DACS).

• Software Risk Management: The Practical Approach - DoD Software Tech News 2-2 by George Holt, Mei Technology Corporation, this article discusses a practical approach to Risk Management that will accommodate flexibility and adaptability to diverse software projects by stressing early prototyping, frequent functional builds, and a set of metrics to provide management insight during software development.

• Successful Application of Software Reliability Engineering for the NASA Space Shuttle by Norman F. Schneidewind, Naval Postgraduate School. This article first appeared in International Symposium on Software Reliability Engineering, November 4, 1997, pp. 71-82.

  This link will download a PDF document

• The Cohen Amendment Impacts Government IT Acquisitions - DoD Software Tech News (1997; STN 1-1) New rules concerning Government acquisitions of Information Technology (IT) took effect on 8 August 1996. The new legislation is the Information Technology Management Reform Act (ITMRA), which is also known as the Cohen Act, or Cohen Amendment. Among the management practices specified in the ITMRA are cost-benefit analyses, return on investment, risk assessments and minimization, performance-based and results-based management. The DoD Software Tech News is a publication of the Data & Analysis Center for Software (DACS).

• The Software Program Managers Network - DACS Newsletter (Spring 1996) The SPMN's goal is to help software-intensive projects operate as successfully as possible by identifying and conveying to practitioners and managers what works, and what does not work, in real-world software management. One of the Software Acquisition Best Practices Initiative components includes the Seven Issues Panel. The panels address seven key software management areas: Risk Management, Planning and Baselining, Program Control, Program Visibility, Engineering Practices and Culture, Process Improvement, and Solicitation and Contracting.

• What Do You Mean You Can't Tell Me If My Project Is In Trouble? - DoD Software Tech News 2-2 by Dr. Joseph Kasser, University of Maryland and Victoria Williams, Keane Federal Systems, Inc. This article discusses Risk Management in regards to the Software Development Lifecycle (SDLC) for large systems.

• An Introduction to Team Risk Management (Ver. 1.0) (An SEI report) This report from Software Engineering Institute (SEI) introduces the team Risk Management approach for managing risks within a software dependent development program.

• Software Acquisition Risk Management Key Process Area (KPA) - A Guidebook (Version 1.02) In this guidebook, the author provides sponsors of acquisition improvement programs and their immediate staff with guidelines on how to implement a software acquisition risk management program satisfying the goals of the Acquisition Risk Management (ARM) Key Process Area (KPA) of the Software Acquisition Capability Maturity Model (SA-CMM). Brief overviews of software acquisition and the SA-CMM are included. This version is an editorial update to align with Version 1.02 of the SA-CMM.

• Software Design Methods: A DACS Technical Report This report provides an analysis of the status of software design methods. The topic of software design is an extensive one with a rich history. Views of software design can range from the very focused to those which cover the whole spectrum of software development. This report provides readers with a useful snapshot of software design technology that can be used as a tutorial for the uninitiated, a starting point for detailed research, or a guide for those who will be developing software in the future. The report includes coverage on the nature of design, its evolution, its status, and directions for the future. This report concludes with the authors' perception of the state-of-the-art of software design as indicated by this research. Object-Oriented technology and its influences on software design are covered because this technology promises to have a large impact on future software development.

• Software Prototyping and Requirements Engineering This DACS technical report includes the motivation for using software prototyping in general and specifically in the context of requirements engineering. An overview of software prototyping covers life cycle models, approaches, pitfalls, and opportunities. The section on software requirements and specification establishes a basis for investigating techniques. The summary analyses of software requirements and specification techniques and tools for prototyping address twenty techniques across a variety of language models. Each technique summary analysis includes the history, technique overview, method, supporting tools, language features, and strengths/weaknesses. The description of needed detailed analyses includes a summary of common aspects among the techniques to be developed in a repository. Software technology transfer is addressed in this report from the standpoint of past problems, avenues of opportunity, and actual experience in this area. The report ends with potential areas of future research and a summary.

• Operational Risk Management - Controlling opportunities and threats - ISBN #0957907400 by Simon M. Walker, Published by Connley Walker

  Author's Abstract: The book's main focus is to de-mystify operational Risk Management and make it a useful tool for management. The main difference between this book and others on the topic is that it is not focused solely on financial institutions. As such, it is relevant to any industry. It is structured to take the reader through the basics of Risk Management, various risk modeling techniques and then leads on to chapters that show how the techniques can be applied. It includes chapters on security, fire safety, occupational health and safety, equipment and technology failures, industrial relations and litigation. It then rounds this off with a discussion on developing and implementing Risk Management strategies. An interesting section on common Risk Management standards and guidelines is also provided.

• Assessment and Control of Software Risks by Capers Jones This handbook summarizes more than 50 of the major problems of building and maintaining software projects, and outlines the prevention control "therapies" available. Prentice-Hall ECS Professional; Yourdon Press Computing; February 1994. ISBN: 0137414064

• Managing Risk: Methods for Software Systems Development by Elaine M. Hall, Ph.D. The highlights of this book include: six disciplines for managing product development; the steps to predictable Risk Management process results; how to establish the infrastructure for a risk-aware culture; methods for the implementation of a Risk Management plan; and case studies of people in crisis-and in control. Software Engineering Institute (SEI) Series in Software Engineering; Addison-Wesley Publishing Company; February 1998. ISBN: 0201255928

• Risk Management Processes for Software Engineering Models by Marian Myerson The potential threats associated with software development are identified as the author explains how to establish an effective Risk Management program. The text details the six critical steps involved in applying the process and discusses various software metrics approaches which can be used to measure software quality. Artech House; January 1997; ISBN: 0890066353

• Software Engineering Risk Analysis and Management by Robert N. Charette, Ph. D. This text presents the principles of risk analysis and management. Case studies are included to supplement the narrative. McGraw-Hill; February 1989. ISBN: 0070106614

• Software Engineering Risk Management by Dale Walter Karolak, Ph. D. and N. Karolak This book details strategies for implementing and planning development plans in a cost-effective and timely manner. It discusses vital issues and in particular their costs, schedules, technical performance, and strategies for software development. It also identifies metrics and several models for measuring and predicting risk. IEEE; January 1998. ISBN: 0818671947

• Software Risk Management by Barry W. Boehm, Ph.D. This book identifies the major sources of risk to a software project. It discusses applying practical risk assessment and management techniques to reduce the project's risk. IEEE Computer Society; August 1989. ISBN: 0818689064

• Strategies for Software Engineering: The Management of Risk and Quality by Martyn A. Ould This text offers a decision-making approach to planning and managing all types of software engineering projects. The book establishes a framework for selecting a development strategy, development methods, and support tools with the ultimate goal of minimizing technical risk and increasing product quality. Specific topics include the range of quality attributes (fitness for purpose, fitness for use, and timely delivery), standards for quality management systems, the work breakdown structure, and the use of metrics and indicators. John Wiley & Sons; Software Engineering Institute (SEI) Series; September 1990. ISBN: 0471926280

• Trends in Software Engineering Process Management (TSEPM) Published by Marotz Inc., this E-zine is a FREE (and no-obligation) monthly trade journal that focuses on software process, risk and project management. TSEPM is read by thousands of industry leading CEOs, CIOs, software project managers and engineers from organizations worldwide.

• US Navy - Acquisition and Business Management (ABM) Acquisition and Business Management (ABM) is an online magazine produced by the US Navy. It provides access to Department of the Navy policy, procedures, information, data, and tools, of interest to the Navy Acquisition/Procurement work force.

• George Mason University - Software Risk Management Bibliography This bibliography was developed during a graduate level course project on Software Project Management at George Mason University.

• R.S. Pressman & Associates, Inc. (RSP&A) -- Software Engineering Resources: Risk Analysis This page offers a collection of print and electronic risk analysis resources.

• SEI Risk Management Bibliography A Software Rsik Management Bibliography maintained by the Software Engineering Institute (SEI).

• Software Risk Management Bibliography This site presents a bibliography with a compilation of software Risk Management articles.

• University of Mississippi - Software Engineering Seminar Bibliography This collection of citations was developed in support of a graduate level seminar on Software Risk Management.

• C/S Solutions, Inc. (C/SSI) C/SSI produces integrated analytical tools for cost, schedule, and Risk Management. Their tools are specifically designed to engage Integrated Product Development (IPD) team members and/or Cost Account Managers (CAMs) in proactive cost, schedule and Risk Management of complex programs.

• Defense Acquisition Deskbook - Risk Management Software Tools This portion of the Defense Acquisition Deskbook Catalog provides descriptions of software tools that assist Program Managers in Risk Management activities.

• Galorath Inc. (also known as GA SEER Technologies) provides a comprehensive set of decision-support and production optimization tools. Consulting and support services are available for these tools. The tools help manage product design and manufacturing operations, driving out costs and building in quality. The tools derive cost, schedule, labor and materials estimates by assessing the interaction and impact of product, organizational and even operational variables.

• RISKMAN Risk Management Expert System Riskman is intended for use by software engineers with minimal software project planning experience who are interested in planning a small team software development project. Riskman was written in Quntus Prolog and should be useable on any version of Prolog.

• Risk Radar, an SPMN product Risk Radar is a Risk Management database from the Software Program Managers Network (SPMN). It's prupose is to help project managers dentify, prioritize, and communicate project risks in a flexible and easy-to-use form. Risk Radar provides standard database functions to add and delete risks, as well as specialized functions for prioritizing and retiring project risks. Each risk can have a user-defined Risk Management plan and a log of historical events.

• RiskTrak Home page for Risk Services & Technology and RiskTrak, their software management tool. RiskTrak is Risk Management groupware that allows you to view, analyze, communicate, report and manage risk (cost, schedule and technical) throughout the duration of your projects and programs. RiskTrak is designed to help businesses meet new standards on Risk Management such as: Clinger-Cohen Act (ITMRA), DoD Directive 5000.2-R, CAIV and OMB Circular A-11. RiskTrak supports Best Commercial Practices and is designed to be integrated with any Earned Value Management System (EVMS).

• Software Insight Tool for Internal Risk Mitigation Reviews and CIO Assessments, V 3.2 August 1999 This document presents a comprehensive set of questions designed to assist the Program Manager in evaluating a their program against Statutory and Regulatory requirements, as well as software acquisition Best Practices. Use of this document will aid in the reduction of program risk and help ensure a higher level of quality software.

• Turbo Streamliner Developed by the Navy Acquisition Reform Office, Turbo Streamliner provides the tools and references to assist in reviewing or developing acquisition solicitation packages. This tool describes how to implement acquisition reform policies in preparing Requests for Proposal (RFPs) and other contractual vehicles. Turbo Streamliner covers the following topics: RFP Review Checklist, Reporting Metrics, Lessons Learned, Acquisition Reform Principles, Risk Management, and Post-Award Benchmarking.

• X:PRIMER and S:PRIMER Thess software tool are for isolating process related risks in a project or an organization. They use questionnaires on the Software Engineering Institute's (SEI) Capability Maturity Model (CMM) and SEI Risk Taxonomy.

  X:PRIMER is GRafP Technologies new Web-based product line for predicting what can go wrong, identifying the root causes of potential failures and recommending the best countermeasures.
  S:PRIMER is a Web-based application derived from the X:PRIMER product line to assess the risks associated with developing and deploying Information Technology-based solutions.

• Army Software Insight Questions to Prepare for Milestone Reviews This matrix presents useful guidance for US Army Project Managers (PMs) in software-related areas to help ensure successful projects. It presents a comprehensive set of questions designed to assist the PM in evaluating a program against Statutory and Regulatory requirements, as well as software acquisition "best practices" to reduce program risk and ensure software quality. Prepared by US Army CECOM; 5 September 1997; Version 1.0.

  This is part of the the ARMY SOFTWARE INSIGHT QUESTIONS TO PREPARE FOR MILESTONE REVIEWS document found at: http://www.sec.army.mil/.

• Defense Acquisition Deskbook (Version 3.1) The Defense Acquisition Deskbook is an electronic knowledge presentation system providing the most current acquisition policy for all DoD Services and Agencies. Deskbook's extensive reference material includes information on the various functions, disciplines, activities and processes of the DoD beginning with "User" requirements, flowing through concept development, program establishment, contracting, testing, production, sustainment, and ending with disposal. Is database includes over 1,000 mandatory and discretionary policy documents, DoD and component discretionary practices, software tools and descriptions, front line wisdom and advice, formats and samples.

  The Defense Acquisition Deskbook originated from an Acquisition Reform Initiative to reduce directives while assisting managers to make informed decisions and is sponsored by the Deputy Under Secretary of Defense (Acquisition Reform), and the Office of the Under Secretary of Defense (Acquisition and Technology)/Acquisition Program Integration.

• Defense Acquisition Deskbook - Risk Management The Systems Engineering organization with DTSE&E is responsible for Risk Management in DoD and has, at the direction of the Undersecretary of Defense, Acquisition and Technology (USD (A&T)), examined DoD's approach to managing risk. Systems Engineering formed a Working Group, composed of representatives from the Services and other DoD agencies involved in systems acquisition, to assist in the evaluation of the Departmentıs approach to Risk Management. This page provides bibliographic references, speeches, policies and procedures as well as Risk Management Related Websites.

• DoD Risk Management Policies and Procedures for Acquisition Programs This page lists the DoD policies and procedures that address Risk Management for acquisition programs The four key documents include: DoD Directive (DoDD) 5000.1, Defense Acquisition; DoD Regulation 5000.2-R, Mandatory Procedures for Major Defense Acquisition (MDAPS) and Major Automated Information System (MAIS) Acquisition Programs; DoDD 5000.4, OSD Cost Analysis Improvement Group; and DoD Manual 5000.4-M, Cost Analysis Guidance and Procedures.

• Risk Management Guide for DoD Acquisition This document is a product of a joint effort among the DSTE&E, the Defense Acquisition University, and the Defense Systems Management College. It is based on the materials developed by the DoD Risk Management Working Group, included in the Defense Acquisition Deskbook.

Visit the DACS Home Page (http://iac.dtic.mil/dacs/) for resources on over 30 other Software Engineering topic areas.