SOA adoption: The human face of governance

Human and organizational factors such as politics, information silos, and change management generally underlie IT success and failure.

During a recent conversation, IT analyst and fellow ZDNet blogger, Dana Gardner, affirmed the extent to which this view applies to service-oriented architecture (SOA) projects.

I asked Dana to elaborate on the importance of organizational dynamics with respect to SOA governance and adoption:

In the many discussions we’ve had with analysts, users and suppliers over the advancement of SOA and governance, we always came back to the human element as a weak or missing link. Successful adoption and efficiency of SOA hinges on acceptance, trust, collaboration and news ways for people to work together. SOA also depends on people and groups that had not in the past worked closely to begin doing so.

When I heard more about Asuret and Pragmatic Enterprise 2.0 [my professional involvements], I saw a unique way to bring the human element of SOA into focus. The process of gathering perceptions across teams and any affected people — with trend lines over time for those — really strikes me as essential to understand how and why SOA is working — or not.

I think that SOA efforts will gain significant tools for how to gauge and assuage the progress of SOA adoption, and the best means of governance over long periods of time once SOA activities gain traction.

Dana also highlighted the gap between technical and human governance issues:

SOA governance goes a great job at tracking artifacts and defining and automating rules and policies for artifacts. The role of IT infrastructure can be managed via SOA governance, and the composition and business process alignment of services are also well served by governance. But how people really feel about how the processes are development, implemented and refined is a bit of a block hole when SOA governance is employed. Without a strong view of the perceptions and their change patterns, SOA and governance are obtuse rather then precise instruments.

Following our discussion, Dana brought together a group of top SOA analysts to examine these issues in depth.

This post reprints, in its entirety, Dana’s article on that analyst session. You can listen to the podcast by clicking the player at top of this post.

=============

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Read a full transcript or  download a copy. Charter Sponsor: Active Endpoints. Also sponsored by TIBCO Software.

Welcome to the latest BriefingsDirect Analyst Insights Edition, Vol. 47. Our topic this week on BriefingsDirect Analyst Insights Edition centers on how to define, track and influence how people actually adapt to and adopt technology.

Any new information technology might be the best thing since sliced bread, but if people don’t understand the value or how to access it properly — or if adoption is spotty, or held up by sub-groups, agendas, or politics — then the value proposition is left in the dust. Perceptions count … a lot.

A crucial element for avoiding and overcoming social and user dissonance with technology adoption is to know what you are up against, in detail. Yet, data and inferences on how people really feel about technology is often missing, incomplete, or inaccurate.

In this discussion, we hear from two partners who are working to solve this issue pragmatically. First, with regard to Enterprise 2.0 technologies and approaches. And, if my hunch is right, it could very well apply to service-oriented architecture (SOA) adoption as well.

I suppose you could think of this as a pragmatic approach to developing business intelligence (BI) values for people’s perceptions and their ongoing habits as they adopt technology in a business context.

So please join Michael Krigsman, president and CEO of Asuret, as well as Dion Hinchcliffe, founder and chief technology officer at Hinchcliffe & Co. to explain how Pragmatic Enterprise 2.0 works. Our panel also includes Joe McKendrick, prolific blogger and analyst;  Miko Matsumura, vice president and chief strategist at Software AG; Ron Schmelzer, managing partner at ZapThink; Tony Baer, senior analyst at Ovum; Sandy Rogers, independent industry analyst, and Jim Kobielus, senior analyst at Forrester Research.

This periodic discussion and dissection of IT infrastructure related news and events, with a panel of industry analysts and guests, comes to you with the help of our charter sponsor, Active Endpoints, maker of the ActiveVOS, visual orchestration system, and through the support of TIBCO Software.

And the discussion is moderated by me, Dana Gardner, principal analyst at Interarbor Solutions.

Here are some excerpts:

Hinchcliffe: … As many of you know, we’ve been spending a lot of time over the last few years talking about how things like Web 2.0 and social software are moving beyond just what’s happening in the consumer space, and are beginning to really impact the way that we run our businesses.

More and more organizations are using social software, whether this is consumer tools or specific enterprise-class tools, to change the way they work. At my organization, we’ve been working with large companies for a number of years trying to help them get there.

This is the classic technology problem. Technology improves and gets better exponentially, but we, as organizations and as people, improve incrementally. So, there is a growing gap between what’s possible and what the technology can do, and what we are ready to do as organizations.

I’ve been helping organizations improve their businesses with things like Enterprise 2.0, which is social collaboration, using these tools, but with an enterprise twist. There are things like security, and other important business issues that are being addressed.

Businesses are about collaboration, team work, and people working together . . .

But, I never had a way of dealing with the whole picture. We find that that folks need a deep introduction to what the implications are when you have globally visible persistent collaboration using these very social models and the implications of the business.

Michael Krigsman, of course, is famous for his work in IT project risk — what it takes for projects to succeed and what causes them not to succeed. I saw this as the last leg of the stool for a complete way of delivering these very new, very foreign models, yet highly relevant models, to the way that organizations run their business.

Businesses are about collaboration, team work, and people working together, but we have used things like email, and models that people trust a lot more than these new tools.

There is usually a lot of confusion and uncertainty about what’s really taking place and what the expectations are. And Michael, with Asuret, brings something to the table. When we package it as a service that essentially brings these new capabilities, these new technologies and approaches, it manages the uncertainty about what the expectations are and what people are doing.

Krigsman: Think about business transformation projects — any type. This can be any major IT project, or any other type of business project as well. What goes wrong? If we are talking about IT, it’s very tempting to say that the technology somehow screws up. If you have a major IT failure, a project is late, the project is over budget, or the project doesn’t meet expectations or plan, it’s extremely easy to point the finger at the software vendor and say, “Well, the software screwed up.”

If we look a little bit deeper, we often find the underlying drivers of the project that is not achieving its results. The underlying drivers tend to be things like mismatched expectations between different groups or organizations.

For example, the IT organization has a particular set of goals, objectives, restrictions, and so forth, through which they view the project. The line of business, on the other hand, has its own set of business objectives. Very often, even the language between these two groups is simply not the same.

As another example, we might say that the customer has a particular set of objectives and the systems integrator has its own objectives for the particular project. The customer wants to get it done as fast and as inexpensively as possible. The systems integrator is often — and I shouldn’t make generalizations, but — interested in maximizing their own revenue.

If we look inside each of these groups, we find that inside the groups you have divisions as well, and these are the expectation mismatches that Dion was referring to.

If we look at IT projects or any type of business transformation project, what’s the common denominator? It’s the human element. The difficulty is how you measure, examine, and pull out of a project these expectations around the table. Different groups have different key performance indicators (KPIs), different measures of success, and so forth, which create these various expectations.

Amplifying weak signals

How do you pull that out, detect it inside the project, and then amplify what we might call these weak signals? The information is there. The information exists among the participants in the project. How do you then amplify that information, package it, and present it in a way that it can be shared among the various decision-makers, so that they have a more systematic set of inputs for making decisions consistently based on data, rather than anecdote? That’s the common thread.

… We’re not selling software. We offer a service, and the service provides certain results. However, we’ve developed software, tools, methods, techniques, and processes that enable us to go through this process behind the scenes very efficiently and very rapidly.

Rogers: What we discovered in our studies is that one of the fundamental needs in running any type of business project — an SOA project or an Enterprise 2.0 IT project — is the ability to share information and expose that visibility to all parties at levels that will resonate with what matters to them the most, but also bring them outside of their own domain to understand where dependencies exist and how one individual or one system can impact another.

One of the keys, however, is understanding that the measurements and the information need to get past system-level elements. If you design your services around what business elements are there and what matters to the business, then you can get past that IT-oriented view in bringing business stakeholders in aligned management and business goals to what transpires in the project.

Any way that you can get out — web-based, easy-access dashboard with information — and measure that regularly, you can allow that to proliferate through the organization. Having that awareness can help build trust, and that’s critical for these projects.

Baer: What Dion and Michael are talking about is an excellent idea in terms of that, in any type of environment where there is a lack of communication and trust, data is essential to really steer things. Data, and also assurances with risk management and protection of IT and all that. But, the fact is that there are some real clear hurdles.

An example is this project that my wife is working on at the moment. She was brought in as a consultant to a consulting firm that’s working for the client, and each of them have very different interests. This is actually in a healthcare-related situation. They’re trying to do some sort of compliance effort, and whoever was the fount of wisdom there postponed the most complex part of this problem to the very end. At the very end, they basically did a Hail Mary pass bringing a few more bodies.

They didn’t look for domain expertise or anything. Essentially it’s like having eight women be pregnant and having them give birth to a baby in a month. That’s essentially the push they are doing.

On top of that, there is also a fear among each tribe of the other coming up with a solution that makes the other tribes look bad. So, I can’t tell exactly the feedback from this, but I do know that my wife came in as a process expert. She had a pretty clear view on how to untie the bottlenecks.

Krigsman: We gather a lot of data. The essential elements have been identified during this conversation. … It’s absolutely accurate to look at this tribally. Tony spoke about tribal divisions and the social tribal challenges.

The fundamental trick is how to convert this kind of trust information. Jim was talking about collaborative project governance. All of this relates to the fact that you’ve got various groups of people. They have their own issues, their own KPIs, and so forth. How do you service issues that could impact trust and then convert that to a form that can then be examined dispassionately. I’d love to use the word “objectively,” but we all know that being objective is a goal and it’s never outcome that you can ultimately reach.

At least you have a way to systematically and consistently have metrics that you can compare. And then … when you want to have a fight, at least you are fighting about KPIs, and you don’t have people sitting in a conference room saying, “Well, my group thinks this. We believe the project ‘blank.’ If somebody says the same, my group thinks that.” Well, let’s have some common data that’s collected across the various information silos and groups that we can then share and look at dispassionately.

Schmelzer: … We think that the whole idea of project management is just an increasing fallacy in IT anyway. There is no such thing now. It’s really a discrete project.

Can you really say that some enterprise software that you maybe buying or building yourself or maybe even sourcing as a service is really completely disconnected from all the other projects that you have going on or the other technology? The answer is, they are not.

The enterprise is a collection of many different IT projects, some of which are ongoing, some of which may have been perceived to be dead or no longer in development, or maybe some are in the future.

So, it’s very hard to do something like discrete project management, where you have defined set of requirements and a defined timeline and defined budget, and make the assumption or the premise, which is false, that you’re not going to be impacting any of the other concurrently running projects.

We think of this like a game of pick-up sticks. The enterprise is a collection of many different IT projects, some of which are ongoing, some of which may have been perceived to be dead or no longer in development, or maybe some are in the future. The idea that you could take any one of those little projects, and manipulate them without impacting the rest of the pile is clearly becoming false.

McKendrick: Michael and Dion, I think you’re on the right track. In fact, it’s all about organization. It’s all about the way IT is organized within the company and, vice-versa, the way the company organizes the IT department. I’ll quote Mike Hammer, the consultant, not the detective, “Automate a mess and you get an automated mess.” That’s what’s been happening with SOA.

Upper management either doesn’t understand SOA or, if they do, it’s bits and pieces — do this, do that. They read Enterprise Magazine. The governance is haphazard, islands across the organization, tribal. Miko talks about this a lot in his talks about the tribal aspect. They have these silos and different interest groups conflicting.

There’s a real issue with the way the whole process is managed. One thing I always say is that the organizations that seem to be getting SOA right, as Michael and Dion probably see with the Enterprise 2.0 world, are usually the companies that are pretty progressive. They have a pretty good management structure and they’re able to push a lot of innovations through the organization.

Matsumura: … This type of an approach really reflects the evolution of the best practice of adoption. Some of the themes that we’ve been talking about today around this sharing of information, communication, and collaboration, are really are essential for success.

I do want to caution just a little bit. People talk about complexity and they create a linkage between complexity and failure. It’s more important to try to look at, first of all, the source of the problem. Complexity itself is not necessarily indicative of a problem. Sure, it’s correlated, but ice-cream consumption is correlated with the murder rate, just as a function of when temperatures get hot, both things happen to increase. So complexity is also a measure of success and scale.

… The issue it comes down to for me is what Sandy said, which is that the word “trust,” which is thrown in at the very end, turns out as extremely expensive. That alignment of organization and trust is actually a really important notion.

What happens with trust is that you can put things behind a service interface. Everything that’s behind a service interface has suddenly gotten a lot less complex, because you’re not looking at all that stuff. So, the reduction of complexity into manageability is completely dependent on this concept of trust and building it.

Kobielus: … A dashboard is so important when you are driving a vehicle, and that’s what a consolidated view of KPIs and metrics provides. They are a dashboard in the BI sense, and that’s what this is, project intelligence dashboard for very complex project or mega programs that are linked projects. In other words, SOA in all of its manifestations.

In organization, you have to steer your enterprise in a different direction. You need obviously to bring together many projects and many teams across many business domains. They all need to have a common view of the company as a whole — its operations, various stakeholders, their needs, and the responsibilities internally on various projects of various people. That’s highly complex. So, it’s critical to have a dashboard that’s not just a one-way conduit of metrics, from the various projects and systems.

In the BI world, which I cover, most of the vendors now are going like crazy to implement more collaboration and work-flow and more social community-style computing capabilities into their environments. It’s not just critical to have everybody on the same page in terms of KPIs, but to have a sideband of communication and coordination to make sure that the organization is continuing to manage collectively according to KPIs and objectives that they all ostensibly agree upon.

Hinchcliffe: … The way the process works is that we come into a client with an end-to-end service. Most organizations — and this is going to be true of Enterprise 2.0 or SOA — are looking at solving a problem. There’s some reason why they think that this is going to help, but they’re often not sure.

There are often a lot of unstated assumptions about how to apply technology to a business problem and what the outcome is going to be.

We start with this strategy piece that looks at the opportunity and tries to identify that for them and helps them correct the business case to understand what the return on investment (ROI) is going to be. To do that, you really have to understand what the needs of the organization are. So, one of the first things we do is bring Michael’s process in, and we try and get ground truths.

There are often a lot of unstated assumptions about how to apply technology to a business problem and what the outcome is going to be. Particularly with SOA, you have so many borders that are typically involved. It’s the whole concept around Conway’s Law that the architecture tends to look back at the structure of the organization, because those are the boundaries in which everything runs.

One of the ways that we can assure that we have ground truth is by applying this dispassionate measurement process upfront to understand what people’s expectations are, what their needs are, and what their concerns are. It’s much more than just a risk-management approach. It’s a way to get strategic project intelligence in a way that hasn’t been possible before. We’re really excited about it.

A lot of uncertainty

My specialty has always been focusing on emerging technology. There is always a lot of uncertainty, because people don’t know necessarily what it is. They don’t know what to expect. They have to have a way of understanding what that is, and you have an array of issues including the fact there are people who aren’t willing to normally admit that they don’t know things.

But, here is a way to safely and succinctly, on a regular basis, surface those issues and deal with them before they begin to have issues in the project. We then continue on through implementation and then regular assessments on the KPIs that can cause potential issues down the road. I think it’s a valuable service. It’s low impact, compared to another traditional interview process. This is something most organizations can afford to do on a regular basis.

Krigsman: … I am so hesitant to use the term psychological, because it has so many connotations associated with it. But, the fact is that we spoke about perception earlier, and there has been a lot of discussion of trust and community and collaboration. All of these issues fundamentally relate to how people work together. These are the drivers of success, and especially the drivers of lack of success on projects of every kind.

It therefore follows that, if we want our projects to be governed well and to succeed, one way or another we have to touch and look at these issues. That’s precisely what we’re doing with Asuret and it’s precisely the application that we have taken with Dion into Pragmatic Enterprise 2.0. You have to deal with these issues.

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Read a full transcript or  download a copy. Charter Sponsor: Active Endpoints. Also sponsored by TIBCO Software.

(Cross-posted @ IT Project Failures)