How to create Quality, Maintainable Systems-

The Evolution of System Development

To appreciate what LCI has to offer, it is useful to look at the evolution of systems development. At the logical conclusion of this evolution we have arrived at the stage where we can now build quality systems that have zero defects and that do not deteriorate when maintenance takes place or modifications are applied throughout production.

It is only when an organization’s system development techniques have evolved to the point where an engineering discipline is used throughout the Analysis, Design, and Implementation phases of the development life cycle that a zero-defect level of quality can be obtained.

System Development in the 1960s

In the early days of the “Data Processing” profession there was no system development engineering discipline. For these early systems, the requirements were gathered haphazardly using no particular method, by someone called a "programmer" with the requestor of the system, typically known as the "user." Every programmer assigned to develop a system used his or her own "gut feel" methods. Regular text, logic flow charts and the system’s source code were the only tools available to document the system, for quality control, and to ensure that the system conformed to requirements.

The bulk of quality control was conducted "after-the-fact" by testing the finished programs via unit testing, system testing, integration testing, and acceptance testing. As errors were discovered, "patches" were made to the code. This "patching" was often not reflected in any form of documentation and this often led to the inadvertent creation of additional errors. Typically systems were delivered even though the developer knew that they contained a number of undetected bugs. Because of the unreliability, difficulty, and expense of modifying these systems, they usually have short production lives (i.e., little return on investment). Very few systems built in the 1960s to early 1980s are in production today (or at least they shouldn’t be).

Structured Programming was the solution introduced by Edsgar Dijkstra in the 1960s to address the lack of quality in programming. His approach was to clean up poorly constructed, un-readable computer code and to provide a means of logically proving the correctness of the code logic. Structured Programming advocated using only the fundamental logic constructs of Sequence, Selection, and Iteration and the introduction of a readable logic style using indentation to show subordination. Object Oriented Programming was also introduced in the 1960s to modularize the coding activity.

Today, these techniques are so widely used as to be taken for granted. Nonetheless, they represented a fundamental leap forward in the evolution of systems development towards an engineering discipline.

System Development in the 1970s

Structured Programming’s solution to poor programming quality uncovered the upstream problems caused by the lack of an adequate engineering design discipline. In the 1970s the ideas of Structured Design were put forth by pioneers such as Ed Yourdon and Larry Constantine in the attempt to improve the quality of internal system architecture/design. Structured Design injected quality into this development phase by introducing ideas of modularity to break large, complex systems into smaller, more manageable blocks or "modules." Modules could be developed, proven correct and maintained far more easily than could large monolithic programs. Structure Charts, a modeling tool showing hierarchical levels of control, were introduced and used to graphically model the system’s internal architecture. The quality of the design could be measured using metrics such as the levels of cohesion and coupling of the system’s modules.

With the introduction of Structured Design, the quality of the system’s design could be established before beginning the actual coding. Today, the ideas of modularity and of partitioning large, complex programs into manageable units and of creating a design model or Object Oriented Model are accepted practices throughout the quality systems development profession. Their introduction was another major evolutionary step towards a design engineering discipline. Also, in the 1980’s Object Oriented Design was introduced using new tools for database design and data modeling to modularize the design phase.

System Development in the Late 1970s and 1980s

The introduction of quality design concepts in the 1970s uncovered the lack of clarity and rigor in the Analysis phase (Design’s predecessor in the development cycle). The problem was that of ambiguous requirements specifications leading to systems that did not meet the user’s requirements. What was needed was a user’s "Business View" and not a technician’s "System View" in order to verify the system’s business requirements. This caused authors such as Tom DeMarco (for Data Modeling) and Peter Chen for (Relationship Modeling) to introduce some of the ideas of Structured Analysis. In the same era Object Oriented Analysis was introduced to modularize the Analysis phase.

These ideas brought Analysis up to the same levels of quality as were available for Implementation (coding) and Design. Analysis techniques used a set of graphical and semantic tools to convey the user’s requirements. Structured Analysis introduced a modeling tool called the Data Flow Diagram that tracked data on the move and Object Oriented Analysis used objects to model these aspects. These tools allowed developers to pre-establish the quality of the user’s statement of business requirements. At the same time other tools became available to keep track of static data, which then evolved into the discipline of Business Information Analysis. Information Analysis used tools such as the Entity Relationship Diagram to keep track of static data and the Relationships between that data.

It was this third rung on the evolutionary ladder of quality systems development that helped developers deliver systems that conformed to the organization’s business needs. It is ironic (although understandable) that this evolution began with Programming/Implementation (the final phase of the systems life cycle) and then progressed through the previous phases of Design and then Analysis. Nonetheless, for the first time quality had been addressed throughout the system development life cycle.

Even with these advances, many systems were still being built using old development methods and these systems did not have the level of quality required to provide bug free operations, let alone systems that were easy to modify and maintain (as we are used to today). These systems often suffered from hidden defects that caused frequent failures. Developers of these systems expected them to have short life cycles (an assumption that now haunts us in the form of "Year 2000" bugs). In addition, one reason for systems becoming obsolete early in the life cycle is that many of them were built around their implementation technology (i.e., designed to run only on a particular hardware, support software, file structures, etc.). Systems built around a particular platform are, unfortunately, obsolete as new hardware and support software is developed. Systems built around a technology will last only as long as that technology and will require major new development efforts to port or rewrite the system for the new platform. Today’s businesses can no longer be at the mercy of technology and allow technology to determine the characteristics of their business structure.

At this point there was still a serious need for a development methodology that could integrate the necessary set of techniques and their modeling tools.

LCI’s Contributions

In the 1980s Brian Dickinson founded Logical Conclusions, Inc. and published his first methodology that showed the techniques and management tasks and deliverables necessary to create quality systems. LCI’s Methodology provides the abilities and tools needed to build systems in terms of their Business Objectives rather than the needs of their processing devices. Systems developed using LCI’s Methodology are flexible and able to absorb changes easily because they are partitioned along the lines of business they support.

In the 1980s LCI began advocating a change of focus away from the needs of an organization’s internal systems to the needs of the organization’s external customers. Towards this end, LCI developed a comprehensive external Business Event Methodology to describe what an organization does in terms of its Customer Issues as opposed to System Issues. The basic partitioning unit in this view was not what the organization’s systems needed, but rather the partitioning was based along the lines of what the organization’s external customers needed. This customer focused partitioning used the concept of modeling the Stimulus and Response nature of systems. In this view an external Customer Stimulus would trigger a series of related internal processes to generate the organization’s Responses. The basis of partitioning in this view came to be known as an "Event."

After years of building on this concept, we have come to realize that Event Partitioning is the critical element that links all the preceding systems development evolutionary advances into a unified system building methodology. It is only when we understand the central role of customers and their stimulating Events that we know not only what to build, but also how to build it.

Business Development in the 1990s and 2000s

Many new paradigms emerged in the 90s such as Reengineering and Workflow. These concepts became accepted but unfortunately many organizations used them to build faster bad systems by creating new designs based on their existing organizational structure.

In 1996 Brian Dickinson of LCI published the book “Risk-Free Business Re-Engineering”. Building on the concept of the External Event, this book described how to analyze and model all aspects of an organization’s operations, independently of the existing design structure and implementation. Towards this end, LCI refined its Event Methodology to define five types of Events:

Strategic Events

System Events

Business Events

Regulatory Events

Dependent Events

With the understanding of these different Events, we can model the processing and data causes by any type of stimulus entering an organization. This view allows organizations to discover their essential processing and data and helps them identify and remove the processing performed and the data kept solely for the needs of its old systems and other "internal customers." We have found that if an analyst does not recognize the old design characteristics of a system, he or she is likely to carry it, regardless of how restricting it is on our customers, forward into the new design.

True Business Engineering today

With the Internet widely accepted customers now expect much faster response to their requests from companies. Any organization that has built its current infrastructure on the old paradigm will find it increasingly difficult to keep up with pace of change needed today. We are now forced into developing an infrastructure that is focused on the customer and not on internal system boundaries and hierarchical structures.

In 1998 LCI released its latest ideas via the book, Creating Customer Focused Organizations, which built on the ideas in its previous reengineering book. The focus now being on true Business Engineering based on the outside customer’s needs rather than System engineering based on inside “political” needs. This book described the Event Based technical architecture needed to produce the most efficient design and implementation of an organizations systems - be they manual or automated. LCI advocates a different internal structure for systems (in which there are no Accounting Systems, Stock Control Systems, or department/bureau boundaries)

In 2006 LCI released the book Managing Event-Driven Organizations applying the same principles of customer focused structures to the management environment.   The proposed management structure (in which there are no Boss-Subordinate relationships, recommends empowered, Business Event based teams of employees and facilitators). This refinement further allows organizations to focus on the events that are critical to the success of the business (rather than spend resources catering to the needs of its internal systems) and to engineer the total organization, both technical and managerial, around a customer focused partitioning.