Understand SOA
|
2.2. Common Questions About
SOA
First let's get to the most basic questions about SOA.
To help a business executive understand the value of changing the
enterprise's IT infrastructure into a service-oriented architecture, the
following questions deserve convincing responses:
- What is SOA?
- Why do companies need SOA?
- What benefits will businesses receive if they
implement SOA?
- What opportunities will companies miss if they
don't implement SOA?
- What is different with SOA compared to previous
approaches?
The following sections attempt to provide answers to
these questions. We don't provide a prescriptive method; instead, we attempt
to highlight the current perspectives from both the business and IT viewpoints.
Further, in this book we look in more detail at the conditions for an SOA
project (see Chapter
4, "SOA Project Planning Aspects"), and we consider a more
formal approach to the modeling of services and common semantics (Chapter
5, "Aspects of Analysis and Design").
Chapter
1, "Introducing SOA," adequately provided a technical as well
as a business definition of service-oriented architecture. In summary, from
the business viewpoint, SOA is a set of business, process, organizational,
governance, and technical methods to enable an agile, business-driven IT
environment for greater competitive advantage. It provides the flexibility to
treat business processes as well as the underlying IT infrastructure as
components that can be reused and recombined to address changing business
priorities. Thus, in essence, SOA is the map that guides you down the road to
competitive advantage.
2.2.2. Why Do Companies Need SOA?
The combination of SOA and Web services is in some ways
being marketed as the silver bullet that companies have been looking for to
magically solve all business issues of today. There are valid doubts about
such claims, which certainly never will come true. Many business issues are
not just solved by a specific IT architecture or a certain approach to making
business decisions. As long as people are involved, errors are still likely
to occur. However, with a foundational architecture like SOA, you can expect
to do the following:
- Realize the long-promised potential of utilizing
IT to extensively accelerate or improve business
- Justify IT expenses and capital outlays
- Provide nontechnical people with a clear
understanding of what IT does, how they do it, and their intrinsic value
Projects that study the possibility for using SOA and
its related technologies and methods (see Chapter
5) are of great help to do the following:
- Determine a company's core competencies
- Identify an outsourcing strategy
- Define a plan to reduce the complexity (such as
cost) of the company's shared IT infrastructure needs with its trading
partner community
From a technical perspective, it is easy to see how the
claims of SOA can be realized. With SOA and Web services, incompatible
computer systems can communicate without the previous technical complexity or
high cost of maintenance. SOA can help attain business agility through the
reuse of a company's current IT assets and integrated processes. Finally, SOA
makes it easier to create a more direct, measurable association between
business assets and services, and IT assets and services.
2.2.3. What Benefits Will Businesses Receive if They
Implement SOA?
At a very high level, the answers to the question of
what benefits companies receive through an SOA implementation are as follows:
- It saves money, time, and effort over the long
term through reuse of "components" and because of the
flexibility of SOA.
- It eliminates frustrations with IT through
flexible solutions and shorter times to deployment.
- It justifies IT investments more clearly through
the closer association of IT to business services.
- It provides business executives with a clear
understanding of what IT does and its value.
- It allows the creation of and changes to services
incrementally rather than leaving a guesstimate of the development
costs, thereby eliminating the classic IT 6-6 answer: "The project
will take 6 months and cost 6 figures."
- It provides a business and competitive
differentiator, with direct rationalization and relation to how that
competitive advantage is implemented in IT.
Other technologies have made these same claims.
However, many business leaders, analysts, and technical leaders agree that,
with SOA, we have a different situation that validates this in reality. Chapter
10, "Case Studies in SOA Deployment," offers lessons learned
from real projects that support these claims.
2.2.4. What Opportunities Will Companies Miss if They
Don't Implement SOA?
Implementing business-driven IT with SOA promises to
simplify and accelerate business transactions. Therefore, the agility and
flexibility options of an SOA allow organizations to better handle business
situations like the following:
- Department, intracompany, or intercompany mergers
- Acquisition
- Divestiture
- Product or service rollouts
- Business partner, customer, or supplier changes
- Geographical expansion
- Competitive gains in market share
2.2.5. What Is Different with SOA Compared to Previous
Approaches?
SOA is not a new idea. Solutions have been built
according to SOA principles for at least a decade, although they have usually
been implemented in proprietary (expensive) ways. Although SOA-like solutions
have been possible on a departmental or multidepartmental scale, it became
extremely difficult to scale these solutions across large enterprises and
across multiple companies and partners.
So what's so different now?
Previous SOA-based solutions have a history of
successful implementation. Even with their high costs, they still provide
economic value and functional flexibility in a company. The principles of SOA
can be successfully put together for a single organization or even a few
select partners through careful coordination. Software vendors have even
tried to establish communities of partners built around their application
offerings. However, making this possible on a wide scale in the industry
requires well-defined and cross-platform standards implemented in the
software offerings of many vendors. The reason the term "SOA" has
become so prevalent now is due to the rise of new technologies that make
SOA-based solutions much more cost efficient and productive to implement.
This new
standard technology is a result of Web services. Open Web services standards
are important because they break down the proprietary barriers between
vendors and software programs. This is demonstrated by the commitments of
major vendors, such as BEA, IBM, Oracle, SAP, and Microsoft, to standardize
their respective hardware and software, allowing the sharing of information
and data.
2.2.6. Rethinking Components for Business and
Applications
As noted previously, the premise of SOA that makes its
claims possible is the concept of reusable components. Components are the
discrete business processes and services that make up a business. An SOA
makes it possible to continually reuse components. Think of it as a mosaic of
individual, functional components that can be arranged and rearranged. A
company that follows the concept of SOA can build, deploy, integrate, and
link applications and heterogeneous systems and platforms together across the
organization.
SOA has yet to reach its full potential. Most
applications today are still integrated with custom-developed code—they're
hardwired, if you will—making it slow, difficult, and costly to rearrange the
pieces in the mosaic. What everyone has been waiting for is a way to
standardize the connections among all those components so that they work the
same way everywhere without requiring additional, customized programming,
which is costly and can prevent reuse.
The common components and information utilized by and
derived from business processes can be reused multiple times in an SOA to
give the customer varying experiences. For example, a piece of software is
now represented as a service, matching any business component you can
identify. This is illustrated in Figure
2.3. In this sense, business services can now be more accurately
reflected in coded software.
To make it possible to rearrange and compose
components, your IT systems need to satisfy a number of criteria, mainly an
agreed upon set of standards and rules that need to be in place. These
include the following:
- Standards: Web services for implementation and
enterprise, and respective industry standards that apply to the business
side; this includes, for example, industry services models as shown in Chapter
5.
- A common roadmap: Acts as the master plan to guide
your enterprise to a successfully changed and agile business as outlined
in Chapter
4.
- Common information model and industry semantics:
This is necessary to avoid ambiguities and to support understanding
between all involved parties (see Chapter
5).
- Strong governance processes: Prevents your loosely
coupled systems from degrading into anarchy (see Chapter
4).
- Flexible business processes: Based on dynamic
business process modeling approaches, backed by appropriate methods and
tools (see Chapter
5).
- Business transformation and openness for change as
a base: Changing a business involves changing not just the system but
also the people involved, including those on the IT and business ends.
2.2.7. When Not to Implement an SOA
In certain cases, it is better not to implement an SOA
because it is not a suitable solution. As with all approaches, there is no
one-size-fits-all with SOA. ZapThink.com senior analyst Jason Bloomberg
explains, "When Not to Use an SOA"[4]:
- When you have a homogeneous IT environment.
Theoretically, you might still need an SOA if you want to consider
opening services to an external environment.
- When true real-time performance is absolutely
critical, such as at telephone switches or in other situations that
require nanosecond response times. Though as hardware and networks
develop, you might find SOA-based solutions here as well.
- When flexibility is not needed, such as if the
processes of the business are already direct and uncomplicated and do
not need change. If there already exist optimized commodity processes,
then the organization may not benefit from changing to an SOA.
- When tight coupling is needed, such as within a
cohesive component or application.
- If the organization isn't ready for it. SOA is
typically a cross-organization effort. It is rarely likely to deliver
immediate business benefits for a single line of business application,
such as order entry for standardized products.
|
Here is my summarizing definition of
SOA:
SOA is an architectural paradigm for
dealing with business processes distributed over a
large landscape of existing and new
heterogeneous systems that are under the control of
different owners.
• The key technical concepts of SOA are
services, interoperability, and loose coupling.
• The key ingredients of SOA are
infrastructure, architecture, and processes (including
the metaprocess of establishing SOA,
governance).
• The key success factors for SOA are
understanding, governance, management support,
and homework.
• SOA is neither a specific technology
nor a silver bullet. There are places where SOA is
appropriate and places where it is not.
• Web Services are one possible way of
realizing the infrastructure aspects of SOA. Using
Web Services is often recommended
because it seems to be becoming established as the
standard technology.
Services
Here is my summarizing definition of
“service” in the context of SOA:
A service is the IT realization of some
self-contained business functionality.
• By focusing on the business aspects, a
service hides technical details and allows business people to deal with it.
• Technically, a service is an interface
for (multiple) messages that are exchanged
between provider(s) and consumer(s).
• The complete description of a service
from a consumer’s point of view (signature and
semantics) is called a “well-defined
interface” or “contract.” A contract is agreed individually
between a certain provider, and a
certain consumer, and usually includes
nonfunctional aspects such as SLAs.
• There are several attributes that services
may (or, according to some definitions, must)
have. According to my understanding,
they are situation-dependent; services will
almost always have different attributes,
and should be classified according to those
attributes.
Loose
coupling
Loose coupling is a fundamental concept
of SOA (and large distributed systems in general)
aimed at reducing dependencies between
different systems.
• There are different forms of loose
coupling, and you will have to find the mixture of
tight and loose coupling that’s
appropriate for your specific context and project.
• Any form of loose coupling has
drawbacks. For this reason, loose coupling should
never be an end in itself.
• The need to map data is usually a good
property of large systems.
Enterprise
service bus (ESB)
An enterprise service bus (ESB) is the
infrastructure of SOA.
• Its purpose is to provide
interoperability (connectivity, data mapping, and routing)
combined with some additional services
such as security, monitoring, and so on.
• An ESB can be heterogeneous (and often
is).
• Whether the ESB is protocol-driven or
API-driven is a fundamental decision.
• A protocol-driven ESB defines a
protocol that providers and consumers have to match,
but it is up to the providers and
consumers how to match that protocol. The ESB and
connected systems are decoupled in such
a way that they do not share any code, so the
ESB does not have to deploy libraries
for the systems. The drawback is that any protocol
changes force the providers and
consumers to make corresponding updates.
• An API-driven ESB provides an API for
providers and consumers to use to implement
and/or call services. This allows
protocol details to be transparent, but requires some way
for the ESB to deploy generated code,
and/or libraries to the providers and consumers.
• There are different value-added
services that an ESB might provide. The most important
is the ability for distributed
debugging.
• Business activity monitoring (BAM)
allows you to monitor the ESB, so you can learn
about your business in real time and
react accordingly. This might create some business
opportunities and market advantages.
_ Messaging services: Support a wide
variety of types of messaging, provide
intelligent content-based routing, and
guarantee message delivery.
They can also split and combine
messages.
_ Management services: Can monitor their
own performance, helping to
enforce service level agreements by
recording and responding to message
latency. They can implement message
priorities and apply global
business rules across all the
applications or components they connect.
_ Interface services: Can validate
messages against their schema definitions
(which they hold in their version of a
registry).They support Web
services standards and provide
application adapters for some non–Web
services types of interfaces.
_ Mediation services: Transform messages
between the formats used by
the sending and receiving applications.
_ Metadata services: Related to
mediation, these services can also transform
data from one format to another by using
metadata definitions held
in their own version of a registry.
_ Security services: Encrypt messages
where needed and include a standardized
security model to authorize,
authenticate, and audit all ESB
activity.
Service Clasification
Services typically fall into different
categories. It is up to you to find the appropriate categories
to deal with the differences services in
your context.
• Based on how you categorize your
services, you can define corresponding service layers
and stages of SOA expansion.
• The fundamental service classification
specifies basic, composed, and process service
categories. The corresponding stages of
expansion are fundamental SOA, federated
SOA, and process-enabled SOA.
• Basic services should always belong to
only one backend system and are responsible
only for consistency inside this
backend.
• Composed and process services are
responsible for consistency over multiple backends.
• The usual approach to force
consistency across multiple backends is compensation, not
transaction safety or two-phase commit
(2PC).
• When designing long-running processes,
think carefully about whether to implement
them as stateful process services or to
keep the state in a backend, providing basic services
to perform state transformations.
• Other categorizations might be
possible and useful. They don’t necessarily have to lead
to clear, distinct service types.
• Your SOA landscape may include
technical or infrastructure services. Strictly speaking,
they don’t fulfill the major service requirement
of providing self-contained business
functionalities. Consider carefully how
to name and deal with these services.
• Services are not a silver bullet for
any type of communication between distributed systems.
Their primary purpose is to allow distributed
business processes. Synchronizations
between redundant master and slave data,
decoupling frontends, and so on are different
tasks.
Business
process management (BPM)
Business process management (BPM) and
business process modeling provide
approaches to identify services that are
parts of distributed business processes.
• In practice, process design is a
combined top-down and bottom-up approach, which
involves decomposing processes into
smaller chunks (steps and activities, which are
represented as services) while taking
existing system landscapes and services into
account.
• Another typical approach for
identifying necessary services is portfolio management.
However, with this approach you’ll have
to deal with the risk of providing services
nobody ends up needing.
SOA and Organization
SOA is a strategy that impacts a company
or organization as a whole.
• Distributed processing requires
collaboration, because you need distributed planning,
distributed design, distributed
realization, distributed operation, and distributed incident
management.
• SOA leads to a combination of
department-oriented structures responsible for data and
core business rules and project-oriented
structures responsible for designing and realizing
solutions for business requirements.
• The role of project or solution
manager is required to coordinate among departments.
This person is responsible for creating
a high-level design, providing support during the
realization of the solution, and
assisting in maintenance efforts.
• There are different funding models
possible for realizing new processes and services.
Most common are funding pools and models
where the first consumer or solution to
require the service pays the development
costs.
• Collaboration and top management
support are key success factors for SOA.
Service Life Cycle.
Service implementations are software and
have lifecycles just like any other software.
• The lifecycle is usually triggered by
some process that identifies the need for a new service
(business process modeling with process
decomposition, portfolio management,
modification of an existing service,
etc.).
SOA
versioning
SOA requires a smooth migration strategy
for new service versions.
• The best approach is to treat each
modification of a service (in production) technically
as a new service.
• If you have typed APIs for services,
versioning of service types is also recommended
(although there are alternatives).
• To avoid an explosion of versions, you
have to introduce processes to deprecate and
remove old service versions. This is one
reason why it’s useful to be able to monitor
service calls.
• To become independent from versioning
aspects of called services, service consumers
usually should have a thin layer that
maps external data types into internal data types.
• Service modifications should never
affect anyone other than the service provider and
consumer(s).
• To be “forward compatible,” you might
provide an attribute for upcoming call constraints.
This enables consumers to signal special
needs at runtime, without having to
modify service signatures. However, be
cautious about making everything generic,
because this might introduce a lot of
complexity and hidden dependencies.
There are several valid reasons for
introducing stateful services. Note, however, that
there are different ways to deal with
state, depending on the view. Dealing with state
can be pretty complicated and confusing.
• Prefer stateless services, but use
stateful services if necessary. Note that stateful services
need more resources but usually do not
prevent linear scalability.
• The way state is handled should be
transparent to service consumers.
• Try to make your services idempotent.
If this is not possible, think about a common
way (policy) to deal with services that
are not idempotent.
• Support unit tests for services. Note,
however, that for non-basic services unit testing
can become very complicated, because of
the task of providing consistent distributed
test data.
• Distribution looks great when
everything runs smoothly, but when problems occur it can
become a nightmare. It’s important to
provide support for debugging and monitoring.
• Use correlation IDs, and use a string
data type for these IDs.
• Provide the ability to extend message
headers later without breaking the binary compatibility
of interfaces (via key and value lists).
• Be careful when defining the
fundamental data types and programming models for
your service infrastructure (ESB). I
recommend using common structured data types.
• Consider defining a common exception
type that is used by all services. This allows you
to deal with errors in a generic way.
Web Services:
Web Services are (evolving as) the de
facto standard for SOA infrastructures.
• Based on the core protocols XML and
HTTP, the most important protocols are WSDL,
for the description of Web Services
interfaces, and SOAP, as a protocol format for Web
Services communications.
• WSDL files have different layers for
the service interface, bindings, and deployment
details. For this reason, you usually
need processes to deal with the lifecycle of WSDL
files.
• WSDL is not bound to SOAP. You can use
other bindings, so the interface descriptions
can (in principle) be used with any
technology.
• WSDL as a standard defines only core
properties for interoperability. Additional
attributes for security, SLAs, and so on
are not provided or are provided by other
standards.
• A lot of different Web Services
standards and versions are specified by different standardization
organizations. For this reason,
interoperability can be a problem. The Web
Services Interoperability Organization
(WS-I) tries to solve this issue by providing different
profiles for certain sets of standards.
• In practice, to be interoperable you
should usually use the WS-I Basic Profile and the
SOAP document/literal wrapped binding.
• Web Services is a protocol-driven ESB
standard with inherent point-to-point communications.
For this reason, you might have to
introduce interceptors. This also gives you
the opportunity to use a different
protocol or technology inside the ESB.
• For better synergy, you might consider
implementing an API-driven ESB.
Repository or registry?
As we have seen, both the service repository and the
service registry have a place in the overall enterprise SOA implementation. A
service repository is the foundation of enterprise SOA governance, supporting
centralized management of all of services-related information, including
design, implementation, and usage artifacts. A service registry is the
foundation of service location virtualization, allowing for centralized control
over location and invocation policies for all of the enterprise services. The
key differences between service repositories and registries are:
- The
repository contains all of the design and development artifacts of services
that the design tools may need at design and build time. In contrast, the
registry contains a subset of this information that is required at runtime
binding.
- The
service repository is optimized for store large amounts of assets and to
enable a large user population to make ad-hoc queries to find these
assets. The main design requirement, in this case, is flexible
classification and query support, along with the scalability of the
repository itself as well as a user-friendly interface. The service registry,
on the other hand, is optimized for runtime lookups of services endpoint
addresses. The main design requirement is lookup performance and high
availability.
- Access to
the repository takes place within the enterprise boundaries. In contrast,
the registry often needs to be accessed from within and from the outside
of these boundaries.
So, in the end, it's not a choice between service
repository and registry -- both are needed as parts of a successful, SOA
implementation. Instead, it's a matter of making a determination of where to
use which of the two. It is important to keep these concepts distinct, since
they serve very different purposes.
Repositories
and registries help to manage services, both from an organizational and a
technical
point of view.
•
Repositories manage services from a business point of view, dealing with the
whole
contract
but no infrastructure-specific information. Registries manage services from a
technical
point of view and contain all the information required to deploy and run
those
services in a specific infrastructure.
•
There are multiple aspects to take into account when defining the roles and
responsibilities
of
repositories and registries.
•
If you change your infrastructure, you should be able to keep your repository.
•
Repositories and registries are not typically needed when you begin
establishing a SOA.
It
is usually fine to introduce them when they become useful or necessary.
The following list recaps the top obstacles to creating a
smooth transition to a true SOA. Click each link to go back to that section in
the article.
- Beware of
vendor proprietary service offerings. Do
not get locked into SOA vendor offerings that are proprietary in nature;
you could lose the interoperability and flexibility benefits of a true
SOA.
- Seek
stability in the use of open standards.
The latest open standard specification in the industry is not always the
most stable; as a result, it may not be mature enough for adoption.
- Carefully
assess your legacy modernization. Take a
holistic view of the enterprise when choosing particular legacy systems
for modernization. Silo approaches for SOA transition may create
redundancy.
- Avoid
"waterfall" development and lack of service versioning. SOA transition should be iterative in nature. A service
life-cycle management should possess the capability to maintain multiple
versions of a service.
- Know the
technical constraints of your legacy system. Consider all the technology limitations of a legacy system
before jumping ahead into a legacy modernization effort.
- Don't
equate SOA with Web services. Acknowledge
the difference between SOA (an architectural style) and Web services (a
set of standards for SOA implementation).
- Avoid the
silo approach to service creation and ownership. Understand the paradigm shift between traditional application
development and an SOA-based development.
- Steer
away from the use of fine-grained services. A service is a higher-level abstraction than fine-grained
application program interfaces (APIs). Services should be coarse-grained
and business aligned.
- Avoid
point-to-point invocation. Make an SOA
ecosystem manageable and loosely coupled. Bring in a mediation layer that
handles service discovery, invocation, and neutralizes underlying
technical differences between different SOA implementations.
- Avoid
lack of adherence to standards. Adopt
stable and proven industry-specific standards. This approach will bring in
interoperability benefits for your SOA.
- Use
redundant data stores. Concentrate on a
data consolidation strategy. Mask the data redundancy by creating a
virtualized data service.
- Stay
away from using a "Big Bang" approach. For complex SOA transitions, forget a Big Bang approach to the
finish line. Acknowledge and respect that a smooth SOA transition is best
achieved by adopting an iterative approach.
- Allocate
service ownership. Do not orphan a
service. Give it a home and make a line of business its owner. This
ownership allows someone to be responsible to maintain the nonfunctional
qualities of your services.
- Institute
SOA governance. Empower a governance body
to manage the entire service life cycle.
What
Are the Key Success Factors for SOA?
As
with any new strategy, a lot of aspects are important in making SOA a success
story.
You
must begin with an appropriate understanding, realistic expectations, and the
right
people,
and you must work to discover and formulate appropriate technologies and
processes.
Therefore,
my usual answer to this question incorporates the following elements:
Understanding
You
must know up front that SOA is far more than just an infrastructure.
Collaboration
You
must work to create an organizational culture in which people want to help and
serve.
Management
support
Management
must be persuaded to give enough support (time and money) and be able
to
make appropriate organizational decisions if necessary.
Careful
introduction
A
time frame of three or more years should be permitted for the introduction of SOA,
and
you must build in the ability to adjust things based on first experiences.
Homework
Enough
care for excellence must be taken that all teams are keen on making (your) SOA.
Where Is SOA Not Appropriate?
You
might have the impression or expectation that SOA, as a solution for
distributed processes,
is
appropriate for any form of communication between different systems. But of
course,
there are limits for SOA. The requirements might be too tough, or the overhead
SOA
introduces might lead to more problems than it solves.
Database
replication, mass data processing, and local clients are not particularly SOAs
strengths.
You might benefit from concepts and solutions based on the SOA approach, but
don’t
forget that SOA, like any other concept, technology, or paradigm, is no silver
bullet.
Note
that the question should never be whether or not something is appropriate for
SOA;
rather,
the question is whether or not the solution you’re considering is appropriate
for
your
problem(s) and requirements.
I
see the counter-hype to SOA coming in a few years. Finding that the price for
distribution
and
loose coupling is too high, we might go back to the roots of software silos (of
course,
using a new, positive term such as “general-contractor-driven architecture”).
Service Boker
The
service broker is the component that actually makes all the connections
between
components work. It acts like a needle threading one component to
the
next in a business process. It uses information about the components it
finds
in the SOA registry and threads the components together for the workflow
engine.
The service broker gets things started. After it does its job of
threading
all the components of one business process together, it wanders
off
looking for another one to start.
The
SOA registry stores information describing what each SOA component
does
so that business analysts or programmers can select
components
and connect components together to create composite
applications.
The registry also stores information about how one component
connects
to another.
_
The workflow engine connects together a whole business process in an
end-to-end
manner, flowing work from one individual or process to
another
as the business process is carried out.
_
The service broker actually makes all the connections between components
work.
The service broker is a needle that threads together all the
software
components of a business process by using information it
gleans
from the registry.
XML eXtensible Markup The definition
language that can
_ HTTP is the address.
_ XML is for decoding messages.
_ SOAP is for writing messages.
_ WSDL is for describing interfaces.
_ UDDI is a directory for finding services — just like a telephone directory.