Service-oriented Integration (SOI) – a logical extension of functional integration – is where the applications integrate by using service interactions in an SOA environment. Service are provided by source application (service provider) and consumed by target applications (service consumers). Typically, SOI is implemented as systems that consume and provide XML-based Web services.

SOI addresses the issues of integrating heterogeneous and inflexible systems while overcoming the difficulties of functional Integration in terms of location and technology of the function. For example, functionality in a mainframe can be exposed as a web service implemented using a .Net framework.

In SOI, the service defines a contract – such as the technology, communications protocols, and message definitions – that all service consumers must conform to in order to communicate with the service. SOI enables loose coupling, thereby bringing flexibility and interoperability to newer heights – to the extent that the service provider as well as consumers need not be fixed – and the change in the service provider can be transparent to the service consumers.

In spite of its various benefits, Web Services using XML is not a panacea for all integration scenarios – as it involves considerable effort (for implementation) and overhead (during runtime). For example, usage of XML while providing maximum level of data independence does have a performance overhead due to transformations required.

Let us look at the W3 Definition of Web Services – “Web services architecture is an interoperability architecture that provides a standard means of interoperating between different software applications, running on a variety of platforms and/or frameworks”.

Web Service and XML make the most sense in case of Integration between Unknowns, applications running in heterogeneous environment, and applications that change frequently.

If the organization has application predominantly running on a single platform and hence do not have the complexities associated with integration between disparate platforms – which is chiefly addressed by SOI,  using SOI makes most sense for integration with external world – web portals, other web sites etc. – where the flexibility and interoperability is most required.

If SOI is considered for integration of applications internal to the organization – as they provide the maximum flexibility and interoperability, use of Enterprise Service Bus concept and use of commercial tools is strongly recommended.

For basic information on SOA, Web Services and ESB, refer to earlier blogs http://itknowledgeexchange.techtarget.com/enterprise-IT-tech-trends/essentials-of-soa-web-services-and-esb-in-the-integration-context-part-i/.

Traditional Enterprise application integration (EAI) provided a hub-and-spoke architecture as a better solution compared to direct point-to-point connections. ESB moved further to provide the concept of bus – where the nodes have more intelligence – and offers better flexibility and scalability.

SOA offers great promise but also has great pitfalls. Making SOA work, avoiding the pitfalls and getting ROI, is hard. Usage of commercial tools simplifies SOA implementation enabling the focus to remain on business requirements – and not on implementation platforms and protocols.

Originally, an ESB product had a core asynchronous messaging backbone supplemented with intelligent transformation and routing to ensure messages are passed reliably. In other words, ESB was seen as a shared messaging layer for connecting applications and other services throughout an enterprise.

Today ESB is seen as a collection of architectural patterns based on traditional enterprise application integration (EAI), message-oriented middleware, Web services, .NET and Java interoperability, host system integration, and interoperability with service registries and asset repositories. The commercial tools offers support for various types of integration – using EAI, BPM, SOA, Message driven, event-driven, B2B as well as adapters for communication with different protocols, databases as well as standard products like ERP, CRM etc.

Reasons for using ESB as the integration backbone include the following capabilities:

• Provide location transparency and enable service substitution;
• Support integration in heterogeneous environments ;
• Support SOA principles, separating application code from specific service protocols and implementations;
• Act as a single point of control over service addressing and naming;

In an ESB, there is no direct connection between the consumer and provider. With an ESB, the infrastructure shields the consumer from the details of how to connect to the provider. While the service endpoints can have their own integration techniques, protocols, security models etc., ESB provides a simplified view to the service consumers. Thus an ESB allows the reach of an SOA to extend to non-SOA-enabled service providers. ESB also supports a variety of ways to get on and off the bus.

ESB supports Integration at various levels including:

• Database(s)
• Application adapters
• Connectivity to EAI middleware
• Service mapping
• Protocol transformation
• Data enrichment
• Application server environments (J2EE and .Net)
• Language interfaces for service invocation (Java, C/C++/C#)

Message oriented middleware (MOM), the key behind ESB, offers flexibility in application development. MOM permits time-independent responses because it operates in an asynchronous mode.

According to W3, “Web services architecture is an interoperability architecture that provides a standard means of interoperating between different software applications, running on a variety of platforms and/or frameworks”.

The core technologies used for Web services are:

• XML: generic language that can describe any kind of content in a structured way, separated from its presentation to a specific device.
• SOAP: Platform-neutral protocol that allows a client to call a remote service.
• WSDL: XML-based interface and implementation description language. Using a WSDL document, the service provider specifies the operations that a Web service provides and the parameters and data types of these operations.
• UDDI: Universal Description, Discovery, and Integration (UDDI) is both a client-side API and a SOAP-based server implementation used to store and retrieve information about service providers and Web services.
• WSIL: Web Services Inspection Language (WSIL) is an XML-based specification that locates Web services without using UDDI.

Web services have the following key properties:

• Web services are self-contained. On client side, no additional software required. On server side, an HTTP and SOAP server would suffice;
• Web services are self-describing; Format definition travels with the service – no metadata repository required.
• Web services can be published, located, and invoked across the Web;
• Web services are language-independent and interoperable;
• Web services are loosely coupled;
• Web services are dynamic; With UDDI and WSDL, the discovery is automated and Web services can be deployed without disturbing the clients.
• Web services provide programmatic access; Service consumers have to know the interfaces to Web Services and no knowledge of implementation required.

Web services alone cannot handle the complex requirement of SOA within an enterprise. That is where ESB – the Enterprise Service Bus – seen as the Universal Integration Backbone comes in.

Service-oriented architecture (SOA) is an approach of defining flexible integration architectures based on the concept of a service. SOA brings the benefits of loose coupling and encapsulation to integration at an enterprise level. Using SOA aims at enabling an organization to implement changing business processes quickly and also to make extensive reuse of components.

Services are the building blocks to SOA. Services can be invoked independently by service consumers to process simple functions, or can be a collection of functions to form a process. The key aspects of services are:

• Encapsulate reusable business functions (e.g., Get Customer details, Update customer payments etc.); The importance of reusability cannot be stressed enough in an integration context. Every effort should be made to create functions that are reusable across different consumers (not necessarily service oriented) and avoid building specific point-to-point integrations.
• Are defined by explicit, implementation-independent interfaces. Implementation independent interfaces allow the systems to change their implementation (say the underlying database or even the platform in which the system runs) without affecting other systems. A service contract – is an explicit interface definition – that binds the service producer and the service consumer(s).
• Are invoked through communication protocols that stress location transparency and interoperability. While the service is defined once through a service interface, there could be multiple implementations with different access protocols for the same service. Multiple implementation protocols allow reuse of service from multiple channels as well as heterogeneous systems (running in different platforms).

While SOA is quite useful within the enterprise, the real need for SOA is when it comes to integration with the external world – B2B, B2C etc. And this is where “Web Services” fits in. Web service is one of the key methods of enabling SOA.

Most people agree with Charles Edwards (of Agile EA) comment, that though software development and architecture has been making excellent progress over the past two decades, the same cannot be said of Enterprise Architecture. There are ongoing debates and discussions on topics like Is Enterprise Architecture dying? What is killing Enterprise Architecture? Is there any market for Enterprise Architecture?

The typical drawbacks of Enterprise Architecture (EA) are observed to be:

1. It takes way too long and requires too much effort
2. Not able to prove that significant value is being created
3. Lack of Communication

In an attempt to solve the major drawbacks of Enterprise Architecture, other methodologies (at times referred to as third generation EA) have come up. Still there is no consensus and as practitioners we realize that the hybrid approach is what works best.

From this perspective, the following key points could be directly used by practitioners of EA as a step toward agility:

1.    instead of trying to make a perfect architecture / model – which takes a lot of time and effort and hence quite detailed but not really more usable or correct (as with time the changes would have happened and the model may not reflect the current status), aim at a simple, usable and meaningful model that makes sense to both the Business and IT people and hence really effective (as people really would read and use them!).

2.    Not rely on a specific modeling approach and but use a variety of tools to depict the information and adopt an “iterative and incremental” way to build it. The key is to use the “Right approach – with the right level of detail – for the right context”.

3.    Look at the whole picture – using the various perspectives which are depicted in multiple artifacts.

AgileEA is a free open source EA Operational Process. It is a framework that is designed to either use as is, or to tailor and publish your own Enterprise Architecture Operational Process website. Right now, the site seems to be under maintenance - some time back it was in Beta and asking for suggestions. A whitepaper on Agile EA – Phases, Iterations & Discipline by Charles Edwards is available at http://www.agileea.com/Whitepapers/AEA_Phases_Iterations_Disciplines.pdf.

Enterprise Architecture has been there for over two decades now and still means different things to different people. For some it is a graphical model which excruciating level of details, while in the other extreme it is viewed as a practice that brings in business value.

Zachmann, TOGAF, FEA and Gartner’s EA practice are the four significant EA methodologies that are there in use today and most others are a combination of these. Of various artifacts that dwell on Enterprise Architectures and their comparisons, I found “Comparison of the Top Four Enterprise-Architecture Methodologies” in http://msdn.microsoft.com/en-us/library/bb466232.aspx the effective one is bringing out the differences. In spite of the differences in the understanding and approach of what constitutes EA, the objectives are necessarily the same “Business-IT alignment”, Agility, Cost Reduction and Improved Value.

Enterprise Architecture, though seen as an IT initiative (probably due to the reason that it has been handled by IT personnel), has to do more with Business than IT. In general, the essential reasons for developing an EA have been and remain as:

•Alignment – ensuring the reality of the implemented enterprise is aligned with management’s intent

•Integration – realizing that the business rules are consistent across the organization, that the data and its use are immutable, interfaces and information flow are standardized, and the connectivity and interoperability are managed across the enterprise

•Change – facilitating and managing change to any aspect of the enterprise

•Time-to-market – reducing systems development, applications generation, modernization timeframes, and resource requirements

•Convergence – striving toward a standard IT product portfolio as contained in the Technical Reference

Right now, Enterprise Architecture has reached an interesting phase where it is no longer looked upon as a technology or infrastructure or even application related. With Enterprise Architects today reporting to CxOs, the EA charter today includes enabling Business Strategy and addressing business issues.

Though it sounds like an ideal situation, it definitely remains an uphill task for Enterprise Architects. A survey from Forrester points out that lack of support from business is a common concern and improving perception of EA as a common goal expressed by Enterprise Architects.

Both business and technology are changing at a rapid pace and even bringing paradigm shift. Businesses are expanding, merging, de-merging and time-to-market has been shrinking and shrinking. Similarly technology advancements like Cloud computing, Software-as-a-service, COTS are making Application and Technical architecture of lesser importance than hither-to they have had.

Business Architecture, Information Architecture and Integration Architecture are the areas which are to be focused at by Enterprise Architects. Significant progress in these areas coupled with delivering business benefits, that too quickly, is what is expected from Enterprise Architects today.

In most organizations, Enterprise Architects currently spend most of their time in arriving at detailed models which no one else are able to use effectively, in creating detailed level of standards and guidelines that result in frustration during implementation, in acting as enforcement agents of the EA guidelines created and in trying to meet the immediate need of the business – giving short term the priority at the expense of long-term goals.

To enable the level of detail which Architects where arriving at and also to handle localized requirements, most large organizations had Enterprise Architecture groups at local or national level. And this trend is again changing with the focus of Enterprise Architecture increasingly becoming global.

Global Enterprise Architects should streamline their focus and efforts on high impact, high priority areas that are in alignment with the business strategy. Most of what they are currently doing cannot be ignored altogether – and these activities can be relegated to local and regional Architecture groups.

With the improved EA charter, aligning to any single EA approach may no longer be feasible and the best thing to do is to use the right set of tools, producing artifacts at the right level of detail, and most importantly within the right time frame. Communication – that always was essential for the success of EA, has become crucial today.

Enterprise Architecture, as any other discipline, should adapt itself to the changing demands and trends and agility of the EA is what would ensure that EA provides the benefits it promises.