Recently in Agile Methods Category

Companies that pay attention to emerging trends and adapt quickly are most likely to succeed, while laggards suffer the consequences of latecomers and miss business opportunities. The DevOps movement is one of these emerging trends that companies should be taking seriously despite numerous failures by vendors to jump start this area in the past. However, in our opinion, DevOps appears to already be taking root this time and for several very good reasons.

Why is it different this time?

The impermeable wall between IT development and operations is legendary. Despite numerous past efforts to encourage development and operations organizations to work together, "the wall" still exists in most IT organizations today. With this historical record of failure, is the current DevOps movement also doomed to inevitable failure? As an industry watcher who witnessed the past failed attempts to breach the wall, we believe there is something very different about today's DevOps movement that sets it up for potential success.

Past efforts were driven primarily by vendors who were trying to create a market for selling more software. Development, operations teams and their leaders were less than enthusiastic when vendors presented them with weak value propositions, telling them that development and operations teams "should" collaborate, and it was a good thing to do. It's no wonder that IT organizations chose to invest in other areas with more pressing needs, and left the wall between development and operations still standing.

So what's different about today's DevOps movement? One major difference is a combination of driving forces that reinforce the need for today's DevOps movement, which present strong and compelling reasons for both development and operations to work together. The main driving force comes from business leaders faced with the need to innovate and respond quickly to increasing competitive pressures. This, in turn, puts increased pressure on IT development and operations teams to deliver new services more quickly.

As a result of the changing business needs, development teams are feeling increasing pressure to create new, innovative solutions rapidly and to respond quickly to changing requirements. This is one of the reasons why development organizations are beginning to adopt Agile development methods, which increase the speed and velocity of software development and changes. It also shortens the development cycle. The velocity of software changes that must be deployed into production pose serious challenges for development and operational hand-off processes that are manual and ill-equipped to handle the frequency of new software releases. 

On the other side of "The Wall", businesses are also exerting pressure on IT operations teams to deploy new, innovative solutions and respond quickly to business needs. This is in addition to meeting existing high expectations that applications run smoothly, and are always available. This further challenges the delicate balancing act for Business Service Management, speeding innovation delivery while maintaining stability and minimizing risk to business services.

So operations and development teams are looking to cloud computing as one alternative for fast delivery of infrastructure and application deployment. Many IT operations teams are already in the process of doing the necessary leg work for cloud computing, which requires standardization, processes, and automation. This preparation work will also help move DevOps initiatives forward because manual deployment methods that worked for waterfall development schedules, with well-spaced periodic deployments, will fail to keep up with the constant and rapid arrival of agile software updates.

Telltale Cracks in "The Wall"

Cracks already appear in "The Wall", which may be a harbinger of success for the current DevOps' movement. Several IT organizations have deployed their first cloud initiatives in development test environment deployment. They have chosen the development test environment because it is a low risk, quick return project. But what is most significant about this choice is that it opens the dialog and collaboration between IT operations and development test teams. In my opinion, this demonstrates that "The Wall" is no longer impermeable.

Another indicator of significant cracks appearing in "The Wall" comes from cloud computing early adopters. The development organizations from several early adopter cloud customers saw the pressing need to streamline their service delivery processes in order to speed their time to market and to increase their efficiency.  As vendor integrations between the tools did not exist, they launched internal projects to re-engineer and integrate their service delivery processes. These efforts included integrating selected development and IT operational tools, as well as standardizing and employing automation. Through such initiatives, the companies managed to knock down substantial portions of "The Wall". What is significant about these examples is that development teams were the driving force behind the initiatives. This contrasts markedly with the past, where development teams were typically the ones resisting the change. The resulting payoff for these early adopter companies was faster, more efficient delivery of new business services.

And finally, development tool vendors and operations tool vendors are increasingly delivering capabilities to enable improved collaboration between development and operations teams. In addition, IBM is proposing the adoption of Open Services for Lifecycle Collaboration (OSLC) integration and data exchange standards to enable easier integration across disparate tools. If integration standards are adopted, paving the way for easier integration of disparate development and operations tools, the result would be a significant step forward for the potential success of DevOps' integration.   

The Final Word

There are early indications that DevOps may be taking root this time. Several market forces are at work that place collaboration between development and operations at a critical juncture. Today's DevOps value proposition is much stronger because of the importance to the business, which moves it from a "nice to do" to a "must do".

DevOps is not a power struggle to see who wins between development and IT operations. It should be a collaborative effort between development and operations to deliver what the business needs, as quickly and efficiently as possible, as it enables the company to beat its competition.

In her recent post Getting Ready for Agile 2011, Anne Mullaney gave an outline of my forthcoming sessions at the conference. Specifically, she highlighted the emergence of new forms of Agility:

"Super-Fresh Code" is a term Israel coined (an extension of the "Super-Fresh Web" concept) to describe code that results from seizing upon the opportunities opened by combining recent advances in Agile software methods, cloud computing, mobile applications, and social networking. With the right mix, a company can outgun, outclass and outmaneuver its competition through real-time requirements management and superior business designs. Essentially, super-fresh code becomes the source of competitive advantage. This is a workshop that will make you think about Agile in ways you never have before.

Viewed from this perspective, devops becomes an integral part of Agile methods. One starts an enterprise level Agile roll-out in order to, well, gain Agility. The metaphorical wall between dev and IT ops puts a damper on end-to-end agility. Hence, applying Agile principles to the cusp between dev and ops becomes an essential part of the Agile initiative. In fact, Cutter recommends to its clients to integrate the two all the way down to the backlog stories.

Appropriately enough for the anniversary year of the Agile Manifesto, my strong conviction indeed is that we are just about witnessing Agile going beyond being "just" a software method. Markets are becoming hyper-segmented. There is no way to reach tiny, granular market segments economically without sophisticated software. Moreover, markets are becoming ultra-fluid. It takes a high degree of software-based business agility to penetrate market segments that form and collapse at the speed with which social networking groups emerge (and disappear). Hence, software is becoming a bigger and bigger part of just about any business -- avionics, financial services, healthcare, retail, transportation, telco, and so on. In fact, in many engagements Cutter consultants carry out, the software is the company. Unless Agile methods are used strategically, the ability of a company to generate value for its customers and capture profit for itself might be in jeopardy: the company simply cannot adapt fast enough when dev and IT ops operate as two silos.

I can't wait to discuss these topics with you and other Agile 2011 participants in just about two weeks!

Chris Bruzzo, the CTO of Starbucks, and Narinder Singh, the founder of Appirio, demonstrate Starbucks Pledge 5 application, built on the platform.

They did it in 21 days.  That’s the real value of the cloud.


As an advocate of using "agile" principles to improve the alignment of business and IT, I'd encourage evaluation of the March, 2011 Agile Enterprise Forum 2011 ...which addresses how the CIO can effectively speed the development of business-oriented software. The need to restructure the process of developing new applications and modifying existing applications seems mandatory if IT is going to enable their users to stay ahead of the competitive curve.  Israel Gat's session on "Agile Governance: Tying Delivery to Value" is a business service management (BSM) approach to ensuing tangible value is delivered to IT's business customers by IT development ...cutting through traditional blockages.    

This article was written by Jasmine Noel, one of my business partners. In it, she points out that behind true business agility lies much more than simply use of a hot-button' technology and marketing slogans. Adherence to some fundamental principles in application design, collaboration and management must be followed. She makes her point along excellent examples and detail. Enjoy!
agileWhy would a business executive be interested in Agile software development? 

Why is Agile a topic of interest to the Business-oriented Service Management community? The answer involves strengthening the connection between the developer (...who provides software capabilities for business use) and the business entity (...who uses software technology for critical business functions.)  These two groups are frequently bridged (...successfully or unsuccessfully) by IT operations, adding complexity and increased business frustration to the BSM process of aligning business with IT (...both operations and development or DevOps.)

Read Bill Keyworth's book review >>

Three years ago I had the privilege and honor to recommend to Inovis executives to implement Scrum as their core software method. The “All In!” implementation style was chosen and successfully implemented by colleague and friend Erik Huddleston. To quote Erik:

The results speak for themselves.  In addition to compelling productivity and quality improvements, we also had profound unanticipated benefits.  Within a few sprints, we were using development as a competitive weapon, bringing development to bear to influence the outcome of individual (strategic) sales cycles.  We dramatically increased our innovation through market dialog.  With almost 25000 customers, Inovis struck up quite a market conversation!  Finally, we found that Agile started driving alignment between teams and sites, facilitating tremendous cross product synergy and value.

In a recent post entitled The Agile Flywheel, Inovis’ Ray Riescher describes the effect of the Agile implementation on IT Operations. Here is an excerpt from his post:

Scrum set the flywheel in motion and caused the rest of the IT process life cycle to respond. ITIL’s processes still form the solid core of service support and we’ve improved the processes’ capability to handle intense work velocity. The organization adapted by developing unprecedented speed in the ability to deliver production fixes and solve root cause problems with Agility.

What I think we are witnessing is a manifestation of Agile Business Service Management: a holistic agile methodology running across the IT process spectrum that’s delivering eye popping change and tremendous results.

Since 1994 the Standish Group “Chaos” reports have been regularly mentioned in various posts in software and IT blogs. The following figure from the 2002 study is quite representative of the data provided in the Standish annual surveys of the state of software projects:


The January/February 2010 issue of IEEE Software features an article entitled The Rise and Fall of the Chaos Report Figures. The authors - J. Laurenz Eveleens and Chris Verhoef of the VU University, Amsterdam - give the following summary of their findings:

In 1994, Standish published the Chaos report that showed a shocking 16 percent project success. This and renewed figures by Standish are often used to indicate that project management of application software development is in trouble. However, Standish’s definitions have four major problems. First, they’re misleading because they’re based solely on estimation accuracy of cost, time, and functionality. Second, their estimation accuracy measure is one-sided, leading to unrealistic success rates. Third, steering on their definitions perverts good estimation practice. Fourth, the resulting figures are meaningless because they average numbers with an unknown bias, numbers that are introduced by different underlying estimation processes. The authors of this article applied Standish’s definitions to their own extensive data consisting of 5,457 forecasts of 1,211 real-world projects, totaling hundreds of millions of Euros. The Standish figures didn’t reflect the reality of the case studies at all.

I will leave it to the reader to draw his/her conclusion with respect to the differences between the Standish Group and the authors. I would, however, quote Jim Highsmith’s deep insight on the value system within its context we measure performance. The following excerpt is from Agile Project Management: Creating Innovative Products:

It we are ultimately to gain the full range of benefits of agile methods, if we are ultimately to grow truly agile, innovative organizations, then, as these stories show, we will have to alter our performance management systems…. We have to be as innovative with our measurement systems as we are with our development methodology.

See pp. 335-358 of Jim’s book for details on transforming performance management systems. His bottom line is elusively simple:

The Standish data are NOT a good indicator of poor software development performance. However, they ARE an indicator of systemic failure of our planning and measurement processes.
Jim refers to the standard definition of  project “success” - one time, on budget, all specified features. His focus is usually on software development. I would contend, however, that his good counsel is much broader. IMHO it applies to any IT project.


See the press release for details on the acquisition of Phurnace by BMC. This acquisition is in perfect accord with the vision articulated in BSM Review on the subject Agile Business Service Management.

'The Man in the Dock' Theory

| 3 Comments | 1 TrackBack

Eric Ries has just published a post entitled Continuous deployment for mission-critical applications. In this post he takes a very clear stand on the suitability of continuous deployment to mission-critical applications, as follows:


"I want to directly challenge the belief that continuous deployment leads to lower quality software. I just don't believe it. Continuous deployment offers three significant advantages over large batch development systems. Some of these benefits are shared by agile systems which have continuous integration but large batch releases, but others are unique to continuous deployment.

    1. Faster (and better) feedback... Engineers working in a continuous deployment environment are much more likely to get individually tailored feedback about their work.
    2. More automation... Continuous deployment requires living the mantra: 'have every problem only once.'
    3.  Monitoring of real-world metrics... There are huge classes of bugs that "work as designed" but cause catastrophic changes in customer behavior. My favorite: changing the checkout button in an e-commerce flow to appear white on a white background. No automated test is going to catch that, but it still will drive revenue to zero. Continuous deployment teams will get burned by that class of bug only once. 
    4. Better handling of intermittent bugs... For example, consider a bug that happens only one-time-in-a-million uses. Traditional QA teams are never going to find a reproduction path for that bug. It will never show up in the lab. But for a product with millions of customers, it's happening (and being reported to customer service) multiple times a day! Continuous deployment teams are much better able to find and fix these bugs.
    5. Smaller batches... Continuous deployment tends to drive the batch size of work down to an optimal level, whereas traditional deployment systems tend to drive it up."

I could not agree more - continuous deployment is very effective as a software quality improvement strategy. Whether you do BSM, ERP, transaction management or any other mission-critical application, thoughtful continuous deployment is an excellent way to go. The laws of software engineering apply to any kind of application you might be developing and deploying.

I believe, however, we might have a metrics problem on our hands. What often happens is that continuous deployment flies at the teeth of the 'Man in the Dock' theory. When a major disruption happens, we look for a single point of accountability instead of deciphering the complex pathways to the disruption. Such a theory in use, of course, leads to less frequent deployments which in the long run adversely affect software quality.

A major task for Agile Business Service Management is the development of metrics that take us away from 'The man in the Dock' mindset. These metrics need to satisfy two criteria:

  1. Map software quality to customer value.
  2. Help us realize that service disruptions are systemic. They are a matter of complicated pathways, not of the incompetence of one individual or another.

The Future Behind Us

| 1 Comment | No TrackBacks

In Backing into the Future, Bernard Knox makes a fascinating observation:

"The early Greek imagination envisaged the past and the present as in front of us - we can see them. The future, invisible, is behind us... Paradoxical though it may sound to the modern ear, this image of our journey through time may be truer to reality than the medieval and modern feeling that we face the future as we make way forward into it."

How appropriate this mental model is in view of the recent and no so recent past. As pointed out by Perez, we should view the burst of the "dot-com bubble" in 2000 and the financial collapse in 2008 as a double bubble which is part of the fifth techno-economic cycle. Building on this premise. I made a bullish 2010 prediction for the Cutter Consortium. Here is the botton line from this prediction:

"I expect 2010 to be the first year of a prolonged golden age. Serious as the various problems we all are wrestling with after the 2008-2009 macro-economic crisis are, they should be viewed as systemic to the way a new generation of revolutionary infrastructure gets assimilated in economy and society." 

Likewise, my prediction for The Agile Executive is optimistic with respect to Agile Business Service Management:

"Agile breaks out of Development into IT (and beyond) in the form of Agile Infrastructure and Agile Business Service Management."

Click here for details of my techno-economic prediction for Cutter, here for my Agile-focused predictions for The Agile Executive. 

The premise of Agile Business Service Management is that software methods can be successfully applied to  domains like IT Operations and Service Management. An illustration of the power of applying Agile beyond programming is given in Anne Gentle's post Agile Across the Enterprise. Anne discusses the kind of hyper-productivity that transforms the econonics of various classes of books. The potential disruption to traditional publishing is mind boggling...

Colleagues Rico, Sayani and Sone recently published a book entitled The Business Value of Agile Software Methods: Maximizing ROI with with Just-in-Time Process and Documentation. Based on hundreds of systemic research studies, the authors cite some very impressive numbers. Even their least impressive numbers are awesome. For example:

  • Agile methods ROA 1.6X more than traditional [software] methods
  • Agile methods NPV 2.3X more than traditional [software] methods

We do not have anything like this rigorous study in the nascent field of Agile Business Service Management. I would, however, allow myself to suggest that the gains to be had through Agile  Business Service Management might be as impressive. The three main reasons for my suggesting so are as follows:

  • Deja vu. Much of the gain in applying Agile methods to software engineering comes from breaking the metaphorical wall between development and test. Likewise, the application of Agile principles, technologies and practices to Business Service Management leads to breaking the metaphorical wall that seperates development and test from operations.
  • Life cycle economics. As observed by Boehm, defects fixed during the Operations phase can be two orders of magnitude more expensive to fix than during the Requirements phase, one order of magnitude more expesive to fix than during the Code phase.
  • Data-driven decision making. Agile deployment enables "real time" customer feedback. Hence, the quality and timeliness of business decisions are greatly improved.

I don't really know whether colleague Dr. David F. Rico might have an interest in Agile Business Service Management. I will most certainly try to induce him....

I recommend the following acid test in The Agile Executive post entitled Prior to Sprint Zero: A Note on Jakob Nielsen's "Agile User Experience Projects":

If you consider this "prior to sprint zero" approach a bit heavy-handed, I would offer a simple test to assess its reasonableness. Play with a number of IT Service Management (ITSM) products that you picked in random. Once you did so, compare the numbers of those that clearly have services at their core, to the number of those that integrated services into their user interface as an afterthought.

Granted, the test was originally conceived to make a point with respect to the timing of user interface architecture tasks in the cycle of Agile software development. However, it is actually applicable to ITSM products in general. For example, the results of the test cited above are indicative of how far a system management vendor shifted his mindset, architecture, technology and practices from servers to services.

I would actually go one step further to suggest the user interface architecture test as a reliable "bullshit detector" for vendor hype. Whenever a fundamental application paradigm change takes place, the degree to which a vendor embraced the change can be tested by examining the orientation of the user interface. As a potential customer, it tells you how seriously the vendor walks the talk with respect to embracing paradigm changes and investing in so doing in earnest.

The Joys of Real Hardware

| 4 Comments | No TrackBacks

Colleague and friend Sebatian Hassinger sent me Jeff Dean's presentation Designs, Lessons and Advice from Building Large Distributed Systems. The presentation is fascinating in quite a few ways, not least of which is the (implied) statements it makes about requirements for business service management at large scale. For example, here is an excerpt from the slide entitled The Joys of Real Hardware:

Typical first year for a new cluster:


~0.5 overheating (power down most machines in <5 mins, ~1-2 days to recover)

~1 PDU failure (~500-1000 machines suddenly disappear, ~6 hours to come back)

~1 rack-move (plenty of warning, ~500-1000 machines powered down, ~6 hours)

~1 network rewiring (rolling ~5% of machines down over 2-day span)

~20 rack failures (40-80 machines instantly disappear, 1-6 hours to get back)

~5 racks go wonky (40-80 machines see 50% packetloss)

~8 network maintenances (4 might cause ~30-minute random connectivity losses)

~12 router reloads (takes out DNS and external vips for a couple minutes)

~3 router failures (have to immediately pull traffic for an hour)

~dozens of minor 30-second blips for dns

~1000 individual machine failures

~thousands of hard drive failures, slow disks, bad memory, misconfigured machines, flaky machines, etc.

The bullets listed above resonate with my Agile Business Service Management thinking. They can simply be thought of as the reality underlying BSM at scale. The scale and scope of operating on top of such envirnments necessitate new techniques in BSM. For example, Jeff discusses Protocol Buffers as one such technique used by Google to attain the requisite efficiencies. Likewise, treating infrastructure as code is - as we say in chess - a practically forced variant. In both cases, the traditional wall between development and operations is moot.

UPDATE: we now have a BSM Maturity Model (registration required) >>

One of our unstated goals at is to create a maturity model for Business Service Management and beyond. Of course, this maturity model may differ slightly by industry, but the idea is to create a model which is good enough to create a "common roadmap" for IT and its business partners (yes, we will include cloud services).

To start the discussion, I've brought together some of the traditional thinking from IT 1.0, and some "edge insights" from people like JSB.

To start, let's look at Gartner's IT Management Process Maturity Model from 2005. Looks familiar, doesn't it? What should Level 5 and Level 6 look like? 


For nGenera, a few years ago, Vaughan Merlyn created a different sort of maturity model based on demand and supply:

He asks:

Business demand is also a function of IT supply - low supply maturity will constrain business demand.  For example, an IT infrastructure that is unreliable and hard to use will tend to dampen the business appetite to leverage IT for business innovation and for collaboration with customers and partners.  Typically, if business demand gets too far ahead of IT supply, there will be a change of IT leadership.  On the other hand, if IT supply gets too far ahead of business demand, IT will be seen to be overspending, resulting in a change of IT leadership.  The most common patterns are that at Level 1, business demand leads IT supply; in Level 2, IT supply tends to 'catch up' with and overtake demand, and in Level 3, demand and supply are closely aligned. From the perspective of late 2007, we see the majority of companies at mid-Level 2, some at high Level 2, and a minority at either low Level 3 or high Level 1.  Why are so many at mid-level 2, and seem to be struggling to get to the next level?
Good question. Any ideas?

Then there's Accenture's Service Management Maturity model from their ITILv3 practice - they rightly state that ITILv3's focus is on business results; hence their advocacy for adoption:


At Deloitte, JSB and Tom Winans have built an interesting map for "autonomic computing" which is focused on the direction of IT's evolution. It's part of a series of papers on cloud computing. It's a technology maturity model, if you will:


Finally, I borrowed this SOA Maturity model from Infosys:

Taken together, we have enough food for thought and discussion, don't you think? I have this silly notion that a business service management maturity model must begin and end not with IT but the business.  And cloud computing will certainly play a giant role in this transformation from physical datacenter to cloud service grids.  And of course we'll still have to worry about compliance and security.

Once again, I'll defer to the JSB and Winans vision for the future.  After we get to autonomic computing, then comes the service grid:


If I understand correctly, here's what they're saying: technology platforms will be business platforms.

With that, let's ask once more: what does a Business Service Management Maturity Model look like to you? 


HP has an ITIL-view which is evolutionary:



gives us a look at a maturity model developed by Macehiter Ward-Dutton:


Stay tuned.

As part of today's Rally Agile Success Tour event in London, I had the pleasure of speaking 1-1 with an IT Director in a major F500 company. This IT Director has already made strides into Agile Business Service Management. His algorithm for success is straightforward:

Involve your Service Managers in the software development work from the outset.

The benefits of so doing cut both ways:

  • Service Managers become integral part of the development and test team.
  • Developers and testers are appriased on an on-going basis on the service view. 
Even more important, the team commitment is for the delivery (install, configure, run, evolve) of software as required at any point in time.

Simple and effective.

Sure Footedness

| No Comments | No TrackBacks

Malcolm Fry brings up an excellent question in his ITIL Lite article:

So why are there so many people seeing ITIL v3 more as a methodology than as a framework?

As summarized in The Tool is the Methodthe phenomenon observed under similar circumstances in Agile initiatives is correlation with sure footedness (or lack thereof). The tendency is to follow the 'prescription' as is until one becomes sure footed enough to interpret it in an intelligent manner. 

Agile BSM

When development, deployment and operations evolve in parallel from a business services perspective, we get Agile Business Service Management. That's according to Israel Gat, in his article: The Case for Agile Business Service Management >>

Israel, for those of you who don't know, is a founding member of the Agile Business Service Management movement, which in his own words is "the fusion of modern software development methods with the prevailing preference to run IT from the perspective of the business customer."

In this related interview, Israel talks to the illustrious Jim Highsmith on the same subject >>

Agile Infrastructure

| No Comments | No TrackBacks

Michael Cote, Andrew Shafer and I did a fun podcast on Agile Infrastructure a few months ago. Here is the key excerpt from the podcast:

Israel asks, is ITIL for the data-center like water-fall for development? Both Andrew and I weight in on how much water-fall you can buy into, making analogues to eat-the-whole-pie RUP. This also recalls a conversation I had on another podcast, the IT Management & Cloud podcast with Rob England, aka, The IT Skeptic, on the topic of CMDBs and ITIL.

Click for the full podcast. You would not be disappointed...


You do not need to be an expert in Value Stream Mapping to appreciate the power of speeding up deployment to match the pace of Agile development. By aligning development with deployment, you streamline "production" with "consumption." The rationale for so doing is aptly captured in the first bullet of the Declaration of Interdependence:

We increase return on investment by making continuous flow of value our focus.

Flickr and IMVU seem to be doing an exceptionally fine job streamlining the flow of value: every thirty minutes and every nine minutes respectively. A recent presentation in Velocity 2009 by John Allpsaw and John Hammond adds color how development and operations at Flickr cooperate to accomplish "10+ deploys per day."

What does such fast pace mean to the business? In a nutshell, much of the guess work as to what features are really needed is eliminated when you develop, deploy and collect customer feedback in ultra fast manner. Consequently, the company's business design is likely to be transformed. Click here, here, and here for more detailed discussions how the business design gets transformed.

About this Archive

This page is an archive of recent entries in the Agile Methods category.

Access Management is the previous category.

Applications Management is the next category.

Find recent content on the main index or look in the archives to find all content.