In an industry with over 35 years of software engineering experience, it's amazing that we have failed to produce any significant breakthrough that improves the ability of executing successful software projects. The brightest minds have been humbled battling the tide of anarchy that persistently undermines the progress of their complex initiatives.
"Discipline and control", the gurus cry, will help curb our irresponsible creative surges. Envying their manufacturing counterparts, knowledge workers have reached out to embrace disciplines that have reduced the insanity while simultaneously eroding innovation and creativity beneath the weight of bureaucratic process and fragmented work distribution. In this protected environment, the "sigh of relief" eventually evaporates into a new level of panic, as emerging competitors run roughshod across existing markets with disruptive innovations.
Successful assembly and delivery of complex artifacts is ultimately driven by microeconomics. In the software industry, the emerging market value has become increasingly complex to analyze and the capability constructs similarly growing in complexity and interdependence. It is into this intersection of complexity and discipline that requirements engineering has been born, and to some degree helped extend the coping lifespan of adopting organizations.
Systems Architects have long used the practice of abstraction and information hiding to manage the complexity of large-scale implementations. Requirements engineering is an attempt to leverage these disciplines to articulate the market needs or ultimately value opportunities in a manageable, traceable fashion. It is a formal attempt to align opportunity value with construction cost. However, trying to reduce the interaction with the opportunity domain to a clear-cut API in the form of well constructed requirements is failing for several reasons:
- Requirements create a dysfunctional buffer between the value context and the construction capabilities. This cripples the ability of creative problem solvers to contribute except at the highest levels of the process.
- Requirements fragment the value context in an attempt to facilitate a work break-down and scope determination. The risk of value dilution increases often resulting in unnecessarily complex or irrelevant implementations.
- Requirements are underpinned by a contract based process, built on a philosophy of control and protectionism. Companies are crippled through an inflated time to market and the denial of using early corrective value validation, placing too much faith on the ability of the requirements process to communicate value effectively.
The journey out of this dilemma, is both simple and difficult. It requires a new or transformed set of skills and a change of corporate mindset. First let's focus on the activities needed for success:
- Clearly understand and communicate opportunity value to all contributing levels of the organization.
- Effectively envision and validate capability early and often.
- Powerfully push context as deep as possible into the design and construction domains.
- Rapidly and iteratively test design possibilities with the opportunity domain stakeholders.
Tools and practices for connecting design possibilities include: early design prototyping; iterative design checkpoints and feedback; agile construction practices and multi-disciplinary design teams.
Companies like IDEO and Google have demonstrated the success of these approaches. We too need to relinquish our safety nets and grasp the new paradigms of innovation for success in the emerging market. As Gary Hamel, considered by some as “the world’s leading expert on business strategy” states, "Innovation is the only cure for the debilitating hyper-competition that drives margins ever downward."