Understand SOA

2.2. Common Questions About SOA First let's get to the most basic questions about SOA. To help a business executive understand the value of changing the enterprise's IT infrastructure into a service-oriented architecture, the following questions deserve convincing responses: What is SOA? Why do companies need SOA? What benefits will businesses receive if they implement SOA? What opportunities will companies miss if they don't implement SOA? What is different with SOA compared to previous approaches? The following sections attempt to provide answers to these questions. We don't provide a prescriptive method; instead, we attempt to highlight the current perspectives from both the business and IT viewpoints. Further, in this book we look in more detail at the conditions for an SOA project (see Chapter 4, "SOA Project Planning Aspects"), and we consider a more formal approach to the modeling of services and common semantics (Chapter 5, "Aspects of Analysis and Design"). 2.2.1. What Is SOA? Chapter 1, "Introducing SOA," adequately provided a technical as well as a business definition of service-oriented architecture. In summary, from the business viewpoint, SOA is a set of business, process, organizational, governance, and technical methods to enable an agile, business-driven IT environment for greater competitive advantage. It provides the flexibility to treat business processes as well as the underlying IT infrastructure as components that can be reused and recombined to address changing business priorities. Thus, in essence, SOA is the map that guides you down the road to competitive advantage. 2.2.2. Why Do Companies Need SOA? The combination of SOA and Web services is in some ways being marketed as the silver bullet that companies have been looking for to magically solve all business issues of today. There are valid doubts about such claims, which certainly never will come true. Many business issues are not just solved by a specific IT architecture or a certain approach to making business decisions. As long as people are involved, errors are still likely to occur. However, with a foundational architecture like SOA, you can expect to do the following: Realize the long-promised potential of utilizing IT to extensively accelerate or improve business Justify IT expenses and capital outlays Provide nontechnical people with a clear understanding of what IT does, how they do it, and their intrinsic value Projects that study the possibility for using SOA and its related technologies and methods (see Chapter 5) are of great help to do the following: Determine a company's core competencies Identify an outsourcing strategy Define a plan to reduce the complexity (such as cost) of the company's shared IT infrastructure needs with its trading partner community From a technical perspective, it is easy to see how the claims of SOA can be realized. With SOA and Web services, incompatible computer systems can communicate without the previous technical complexity or high cost of maintenance. SOA can help attain business agility through the reuse of a company's current IT assets and integrated processes. Finally, SOA makes it easier to create a more direct, measurable association between business assets and services, and IT assets and services. 2.2.3. What Benefits Will Businesses Receive if They Implement SOA? At a very high level, the answers to the question of what benefits companies receive through an SOA implementation are as follows: It saves money, time, and effort over the long term through reuse of "components" and because of the flexibility of SOA. It eliminates frustrations with IT through flexible solutions and shorter times to deployment. It justifies IT investments more clearly through the closer association of IT to business services. It provides business executives with a clear understanding of what IT does and its value. It allows the creation of and changes to services incrementally rather than leaving a guesstimate of the development costs, thereby eliminating the classic IT 6-6 answer: "The project will take 6 months and cost 6 figures." It provides a business and competitive differentiator, with direct rationalization and relation to how that competitive advantage is implemented in IT. Other technologies have made these same claims. However, many business leaders, analysts, and technical leaders agree that, with SOA, we have a different situation that validates this in reality. Chapter 10, "Case Studies in SOA Deployment," offers lessons learned from real projects that support these claims. 2.2.4. What Opportunities Will Companies Miss if They Don't Implement SOA? Implementing business-driven IT with SOA promises to simplify and accelerate business transactions. Therefore, the agility and flexibility options of an SOA allow organizations to better handle business situations like the following: Department, intracompany, or intercompany mergers Acquisition Divestiture Product or service rollouts Business partner, customer, or supplier changes Geographical expansion Competitive gains in market share 2.2.5. What Is Different with SOA Compared to Previous Approaches? SOA is not a new idea. Solutions have been built according to SOA principles for at least a decade, although they have usually been implemented in proprietary (expensive) ways. Although SOA-like solutions have been possible on a departmental or multidepartmental scale, it became extremely difficult to scale these solutions across large enterprises and across multiple companies and partners. So what's so different now? Previous SOA-based solutions have a history of successful implementation. Even with their high costs, they still provide economic value and functional flexibility in a company. The principles of SOA can be successfully put together for a single organization or even a few select partners through careful coordination. Software vendors have even tried to establish communities of partners built around their application offerings. However, making this possible on a wide scale in the industry requires well-defined and cross-platform standards implemented in the software offerings of many vendors. The reason the term "SOA" has become so prevalent now is due to the rise of new technologies that make SOA-based solutions much more cost efficient and productive to implement. This new standard technology is a result of Web services. Open Web services standards are important because they break down the proprietary barriers between vendors and software programs. This is demonstrated by the commitments of major vendors, such as BEA, IBM, Oracle, SAP, and Microsoft, to standardize their respective hardware and software, allowing the sharing of information and data. 2.2.6. Rethinking Components for Business and Applications As noted previously, the premise of SOA that makes its claims possible is the concept of reusable components. Components are the discrete business processes and services that make up a business. An SOA makes it possible to continually reuse components. Think of it as a mosaic of individual, functional components that can be arranged and rearranged. A company that follows the concept of SOA can build, deploy, integrate, and link applications and heterogeneous systems and platforms together across the organization. SOA has yet to reach its full potential. Most applications today are still integrated with custom-developed code—they're hardwired, if you will—making it slow, difficult, and costly to rearrange the pieces in the mosaic. What everyone has been waiting for is a way to standardize the connections among all those components so that they work the same way everywhere without requiring additional, customized programming, which is costly and can prevent reuse. The common components and information utilized by and derived from business processes can be reused multiple times in an SOA to give the customer varying experiences. For example, a piece of software is now represented as a service, matching any business component you can identify. This is illustrated in Figure 2.3. In this sense, business services can now be more accurately reflected in coded software. To make it possible to rearrange and compose components, your IT systems need to satisfy a number of criteria, mainly an agreed upon set of standards and rules that need to be in place. These include the following: Standards: Web services for implementation and enterprise, and respective industry standards that apply to the business side; this includes, for example, industry services models as shown in Chapter 5. A common roadmap: Acts as the master plan to guide your enterprise to a successfully changed and agile business as outlined in Chapter 4. Common information model and industry semantics: This is necessary to avoid ambiguities and to support understanding between all involved parties (see Chapter 5). Strong governance processes: Prevents your loosely coupled systems from degrading into anarchy (see Chapter 4). Flexible business processes: Based on dynamic business process modeling approaches, backed by appropriate methods and tools (see Chapter 5). Business transformation and openness for change as a base: Changing a business involves changing not just the system but also the people involved, including those on the IT and business ends. 2.2.7. When Not to Implement an SOA In certain cases, it is better not to implement an SOA because it is not a suitable solution. As with all approaches, there is no one-size-fits-all with SOA. ZapThink.com senior analyst Jason Bloomberg explains, "When Not to Use an SOA"[4]: [4] Jason Bloomberg, "When Not to Use an SOA," http://www.zapthink.com/report.html?id=ZAPFLASH-02162004. Jason Bloomberg is a senior analyst at ZapThink LLC in Waltham, Massachusetts. When you have a homogeneous IT environment. Theoretically, you might still need an SOA if you want to consider opening services to an external environment. When true real-time performance is absolutely critical, such as at telephone switches or in other situations that require nanosecond response times. Though as hardware and networks develop, you might find SOA-based solutions here as well. When flexibility is not needed, such as if the processes of the business are already direct and uncomplicated and do not need change. If there already exist optimized commodity processes, then the organization may not benefit from changing to an SOA. When tight coupling is needed, such as within a cohesive component or application. If the organization isn't ready for it. SOA is typically a cross-organization effort. It is rarely likely to deliver immediate business benefits for a single line of business application, such as order entry for standardized products.

< Day Day Up >

SOA

Here is my summarizing definition of SOA:

SOA is an architectural paradigm for dealing with business processes distributed over a

large landscape of existing and new heterogeneous systems that are under the control of

different owners.

• The key technical concepts of SOA are services, interoperability, and loose coupling.

• The key ingredients of SOA are infrastructure, architecture, and processes (including

the metaprocess of establishing SOA, governance).

• The key success factors for SOA are understanding, governance, management support,

and homework.

• SOA is neither a specific technology nor a silver bullet. There are places where SOA is

appropriate and places where it is not.

• Web Services are one possible way of realizing the infrastructure aspects of SOA. Using

Web Services is often recommended because it seems to be becoming established as the

standard technology.

Services

Here is my summarizing definition of “service” in the context of SOA:

A service is the IT realization of some self-contained business functionality.

• By focusing on the business aspects, a service hides technical details and allows business people to deal with it.

• Technically, a service is an interface for (multiple) messages that are exchanged

between provider(s) and consumer(s).

• The complete description of a service from a consumer’s point of view (signature and

semantics) is called a “well-defined interface” or “contract.” A contract is agreed individually

between a certain provider, and a certain consumer, and usually includes

nonfunctional aspects such as SLAs.

• There are several attributes that services may (or, according to some definitions, must)

have. According to my understanding, they are situation-dependent; services will

almost always have different attributes, and should be classified according to those

attributes.

Loose coupling

Loose coupling is a fundamental concept of SOA (and large distributed systems in general)

aimed at reducing dependencies between different systems.

• There are different forms of loose coupling, and you will have to find the mixture of

tight and loose coupling that’s appropriate for your specific context and project.

• Any form of loose coupling has drawbacks. For this reason, loose coupling should

never be an end in itself.

• The need to map data is usually a good property of large systems.

Enterprise service bus (ESB)

An enterprise service bus (ESB) is the infrastructure of SOA.

• Its purpose is to provide interoperability (connectivity, data mapping, and routing)

combined with some additional services such as security, monitoring, and so on.

• An ESB can be heterogeneous (and often is).

• Whether the ESB is protocol-driven or API-driven is a fundamental decision.

• A protocol-driven ESB defines a protocol that providers and consumers have to match,

but it is up to the providers and consumers how to match that protocol. The ESB and

connected systems are decoupled in such a way that they do not share any code, so the

ESB does not have to deploy libraries for the systems. The drawback is that any protocol

changes force the providers and consumers to make corresponding updates.

• An API-driven ESB provides an API for providers and consumers to use to implement

and/or call services. This allows protocol details to be transparent, but requires some way

for the ESB to deploy generated code, and/or libraries to the providers and consumers.

• There are different value-added services that an ESB might provide. The most important

is the ability for distributed debugging.

• Business activity monitoring (BAM) allows you to monitor the ESB, so you can learn

about your business in real time and react accordingly. This might create some business

opportunities and market advantages.

_ Messaging services: Support a wide variety of types of messaging, provide

intelligent content-based routing, and guarantee message delivery.

They can also split and combine messages.

_ Management services: Can monitor their own performance, helping to

enforce service level agreements by recording and responding to message

latency. They can implement message priorities and apply global

business rules across all the applications or components they connect.

_ Interface services: Can validate messages against their schema definitions

(which they hold in their version of a registry).They support Web

services standards and provide application adapters for some non–Web

services types of interfaces.

_ Mediation services: Transform messages between the formats used by

the sending and receiving applications.

_ Metadata services: Related to mediation, these services can also transform

data from one format to another by using metadata definitions held

in their own version of a registry.

_ Security services: Encrypt messages where needed and include a standardized

security model to authorize, authenticate, and audit all ESB

activity.

Service Clasification

Services typically fall into different categories. It is up to you to find the appropriate categories

to deal with the differences services in your context.

• Based on how you categorize your services, you can define corresponding service layers

and stages of SOA expansion.

• The fundamental service classification specifies basic, composed, and process service

categories. The corresponding stages of expansion are fundamental SOA, federated

SOA, and process-enabled SOA.

• Basic services should always belong to only one backend system and are responsible

only for consistency inside this backend.

• Composed and process services are responsible for consistency over multiple backends.

• The usual approach to force consistency across multiple backends is compensation, not

transaction safety or two-phase commit (2PC).

• When designing long-running processes, think carefully about whether to implement

them as stateful process services or to keep the state in a backend, providing basic services

to perform state transformations.

• Other categorizations might be possible and useful. They don’t necessarily have to lead

to clear, distinct service types.

• Your SOA landscape may include technical or infrastructure services. Strictly speaking,

they don’t fulfill the major service requirement of providing self-contained business

functionalities. Consider carefully how to name and deal with these services.

• Services are not a silver bullet for any type of communication between distributed systems.

Their primary purpose is to allow distributed business processes. Synchronizations

between redundant master and slave data, decoupling frontends, and so on are different

tasks.

Business process management (BPM)

Business process management (BPM) and business process modeling provide

approaches to identify services that are parts of distributed business processes.

• In practice, process design is a combined top-down and bottom-up approach, which

involves decomposing processes into smaller chunks (steps and activities, which are

represented as services) while taking existing system landscapes and services into

account.

• Another typical approach for identifying necessary services is portfolio management.

However, with this approach you’ll have to deal with the risk of providing services

nobody ends up needing.

SOA and Organization

SOA is a strategy that impacts a company or organization as a whole.

• Distributed processing requires collaboration, because you need distributed planning,

distributed design, distributed realization, distributed operation, and distributed incident

management.

• SOA leads to a combination of department-oriented structures responsible for data and

core business rules and project-oriented structures responsible for designing and realizing

solutions for business requirements.

• The role of project or solution manager is required to coordinate among departments.

This person is responsible for creating a high-level design, providing support during the

realization of the solution, and assisting in maintenance efforts.

• There are different funding models possible for realizing new processes and services.

Most common are funding pools and models where the first consumer or solution to

require the service pays the development costs.

• Collaboration and top management support are key success factors for SOA.

Service Life Cycle.

Service implementations are software and have lifecycles just like any other software.

• The lifecycle is usually triggered by some process that identifies the need for a new service

(business process modeling with process decomposition, portfolio management,

modification of an existing service, etc.).

SOA versioning

SOA requires a smooth migration strategy for new service versions.

• The best approach is to treat each modification of a service (in production) technically

as a new service.

• If you have typed APIs for services, versioning of service types is also recommended

(although there are alternatives).

• To avoid an explosion of versions, you have to introduce processes to deprecate and

remove old service versions. This is one reason why it’s useful to be able to monitor

service calls.

• To become independent from versioning aspects of called services, service consumers

usually should have a thin layer that maps external data types into internal data types.

• Service modifications should never affect anyone other than the service provider and

consumer(s).

• To be “forward compatible,” you might provide an attribute for upcoming call constraints.

This enables consumers to signal special needs at runtime, without having to

modify service signatures. However, be cautious about making everything generic,

because this might introduce a lot of complexity and hidden dependencies.

There are several valid reasons for introducing stateful services. Note, however, that

there are different ways to deal with state, depending on the view. Dealing with state

can be pretty complicated and confusing.

• Prefer stateless services, but use stateful services if necessary. Note that stateful services

need more resources but usually do not prevent linear scalability.

• The way state is handled should be transparent to service consumers.

• Try to make your services idempotent. If this is not possible, think about a common

way (policy) to deal with services that are not idempotent.

• Support unit tests for services. Note, however, that for non-basic services unit testing

can become very complicated, because of the task of providing consistent distributed

test data.

• Distribution looks great when everything runs smoothly, but when problems occur it can

become a nightmare. It’s important to provide support for debugging and monitoring.

• Use correlation IDs, and use a string data type for these IDs.

• Provide the ability to extend message headers later without breaking the binary compatibility

of interfaces (via key and value lists).

• Be careful when defining the fundamental data types and programming models for

your service infrastructure (ESB). I recommend using common structured data types.

• Consider defining a common exception type that is used by all services. This allows you

to deal with errors in a generic way.

Web Services:

Web Services are (evolving as) the de facto standard for SOA infrastructures.

• Based on the core protocols XML and HTTP, the most important protocols are WSDL,

for the description of Web Services interfaces, and SOAP, as a protocol format for Web

Services communications.

• WSDL files have different layers for the service interface, bindings, and deployment

details. For this reason, you usually need processes to deal with the lifecycle of WSDL

files.

• WSDL is not bound to SOAP. You can use other bindings, so the interface descriptions

can (in principle) be used with any technology.

• WSDL as a standard defines only core properties for interoperability. Additional

attributes for security, SLAs, and so on are not provided or are provided by other

standards.

• A lot of different Web Services standards and versions are specified by different standardization

organizations. For this reason, interoperability can be a problem. The Web

Services Interoperability Organization (WS-I) tries to solve this issue by providing different

profiles for certain sets of standards.

• In practice, to be interoperable you should usually use the WS-I Basic Profile and the

SOAP document/literal wrapped binding.

• Web Services is a protocol-driven ESB standard with inherent point-to-point communications.

For this reason, you might have to introduce interceptors. This also gives you

the opportunity to use a different protocol or technology inside the ESB.

• For better synergy, you might consider implementing an API-driven ESB.

Repository or registry?

As we have seen, both the service repository and the service registry have a place in the overall enterprise SOA implementation. A service repository is the foundation of enterprise SOA governance, supporting centralized management of all of services-related information, including design, implementation, and usage artifacts. A service registry is the foundation of service location virtualization, allowing for centralized control over location and invocation policies for all of the enterprise services. The key differences between service repositories and registries are:

• The repository contains all of the design and development artifacts of services that the design tools may need at design and build time. In contrast, the registry contains a subset of this information that is required at runtime binding.
• The service repository is optimized for store large amounts of assets and to enable a large user population to make ad-hoc queries to find these assets. The main design requirement, in this case, is flexible classification and query support, along with the scalability of the repository itself as well as a user-friendly interface. The service registry, on the other hand, is optimized for runtime lookups of services endpoint addresses. The main design requirement is lookup performance and high availability.
• Access to the repository takes place within the enterprise boundaries. In contrast, the registry often needs to be accessed from within and from the outside of these boundaries.

So, in the end, it's not a choice between service repository and registry -- both are needed as parts of a successful, SOA implementation. Instead, it's a matter of making a determination of where to use which of the two. It is important to keep these concepts distinct, since they serve very different purposes.

Repositories and registries help to manage services, both from an organizational and a

technical point of view.

• Repositories manage services from a business point of view, dealing with the whole

contract but no infrastructure-specific information. Registries manage services from a

technical point of view and contain all the information required to deploy and run

those services in a specific infrastructure.

• There are multiple aspects to take into account when defining the roles and responsibilities

of repositories and registries.

• If you change your infrastructure, you should be able to keep your repository.

• Repositories and registries are not typically needed when you begin establishing a SOA.

It is usually fine to introduce them when they become useful or necessary.

Quick list of top SOA pitfalls

The following list recaps the top obstacles to creating a smooth transition to a true SOA. Click each link to go back to that section in the article.

1. Beware of vendor proprietary service offerings. Do not get locked into SOA vendor offerings that are proprietary in nature; you could lose the interoperability and flexibility benefits of a true SOA.
2. Seek stability in the use of open standards. The latest open standard specification in the industry is not always the most stable; as a result, it may not be mature enough for adoption.
3. Carefully assess your legacy modernization. Take a holistic view of the enterprise when choosing particular legacy systems for modernization. Silo approaches for SOA transition may create redundancy.
4. Avoid "waterfall" development and lack of service versioning. SOA transition should be iterative in nature. A service life-cycle management should possess the capability to maintain multiple versions of a service.
5. Know the technical constraints of your legacy system. Consider all the technology limitations of a legacy system before jumping ahead into a legacy modernization effort.
6. Don't equate SOA with Web services. Acknowledge the difference between SOA (an architectural style) and Web services (a set of standards for SOA implementation).
7. Avoid the silo approach to service creation and ownership. Understand the paradigm shift between traditional application development and an SOA-based development.
8. Steer away from the use of fine-grained services. A service is a higher-level abstraction than fine-grained application program interfaces (APIs). Services should be coarse-grained and business aligned.
9. Avoid point-to-point invocation. Make an SOA ecosystem manageable and loosely coupled. Bring in a mediation layer that handles service discovery, invocation, and neutralizes underlying technical differences between different SOA implementations.
10. Avoid lack of adherence to standards. Adopt stable and proven industry-specific standards. This approach will bring in interoperability benefits for your SOA.
11. Use redundant data stores. Concentrate on a data consolidation strategy. Mask the data redundancy by creating a virtualized data service.
12. Stay away from using a "Big Bang" approach. For complex SOA transitions, forget a Big Bang approach to the finish line. Acknowledge and respect that a smooth SOA transition is best achieved by adopting an iterative approach.
13. Allocate service ownership. Do not orphan a service. Give it a home and make a line of business its owner. This ownership allows someone to be responsible to maintain the nonfunctional qualities of your services.
14. Institute SOA governance. Empower a governance body to manage the entire service life cycle.

What Are the Key Success Factors for SOA?

As with any new strategy, a lot of aspects are important in making SOA a success story.

You must begin with an appropriate understanding, realistic expectations, and the right

people, and you must work to discover and formulate appropriate technologies and processes.

Therefore, my usual answer to this question incorporates the following elements:

Understanding

You must know up front that SOA is far more than just an infrastructure.

Collaboration

You must work to create an organizational culture in which people want to help and

serve.

Management support

Management must be persuaded to give enough support (time and money) and be able

to make appropriate organizational decisions if necessary.

Careful introduction

A time frame of three or more years should be permitted for the introduction of SOA,

and you must build in the ability to adjust things based on first experiences.

Homework

Enough care for excellence must be taken that all teams are keen on making (your) SOA.

Where Is SOA Not Appropriate?

You might have the impression or expectation that SOA, as a solution for distributed processes,

is appropriate for any form of communication between different systems. But of

course, there are limits for SOA. The requirements might be too tough, or the overhead

SOA introduces might lead to more problems than it solves.

Database replication, mass data processing, and local clients are not particularly SOAs

strengths. You might benefit from concepts and solutions based on the SOA approach, but

don’t forget that SOA, like any other concept, technology, or paradigm, is no silver bullet.

Note that the question should never be whether or not something is appropriate for SOA;

rather, the question is whether or not the solution you’re considering is appropriate for

your problem(s) and requirements.

I see the counter-hype to SOA coming in a few years. Finding that the price for distribution

and loose coupling is too high, we might go back to the roots of software silos (of

course, using a new, positive term such as “general-contractor-driven architecture”).

Service Boker

The service broker is the component that actually makes all the connections

between components work. It acts like a needle threading one component to

the next in a business process. It uses information about the components it

finds in the SOA registry and threads the components together for the workflow

engine. The service broker gets things started. After it does its job of

threading all the components of one business process together, it wanders

off looking for another one to start.

The SOA registry stores information describing what each SOA component

does so that business analysts or programmers can select

components and connect components together to create composite

applications. The registry also stores information about how one component

connects to another.

_ The workflow engine connects together a whole business process in an

end-to-end manner, flowing work from one individual or process to

another as the business process is carried out.

_ The service broker actually makes all the connections between components

work. The service broker is a needle that threads together all the

software components of a business process by using information it

gleans from the registry.

XML eXtensible Markup The definition language that can

 _ HTTP is the address.

_ XML is for decoding messages.

_ SOAP is for writing messages.

_ WSDL is for describing interfaces.

_ UDDI is a directory for finding services — just like a telephone directory.