June, 2008

Maintaining consistent data among systems

There are several ways to ensure information are consistent among systems.

Let’s examine two approaches.

You can store information in a sigle system and other systems query this first one.

Another approach is to replicate data among systems.

An important point to consider is the behaviour of the environment when a new system is added.

In the first approach there is a new system querying the provider of the data.

In the second approach the information must be delivered to one more system.

Both approaches above require care, but the second is more dangerous.

Depending of the number of control messages, the amount of data being transferred will exponentially increase.

The integration architecture can also affect the environment performance. If filters are not used, all the messages will be delivered to the new system increasing the traffic in an unnecessary way.

So, when you are designing your environment think how new systems will be added.

ARTS: Standards for retail market

Who is developing products must pay attention to standards. Standards are essential to ensure interoperability among different vendors.

Organizations like Oasis and W3C have developed many generic standards. These standards are applicable to different markets.

The Association for Retail Technology Standards (ARTS) of the National Retail Federation is a retailer-driven membership organization dedicated to creating an open environment where both retailers and technology vendors work together to create international retail technology standards.

ARTS developed standards for Data Model, Point of Sale (POS), Reuqests for Proposal (RFP) and XML.

ARTS XML committee has developed 15 XML schemas like Customer, Price and Inventory. Furthermore, ARTS released an XML Best Practices document.

ARTS announced at January, 2008, the release of the SOA Blueprint for Retail and associated SOA Best Practices technical reports.

“The Blueprint describes the infrastructure components, tools, models for business services and how ARTS standards can ensure success in companies’ SOA implementation. While SOA is a generic infrastructure strategy that can be implemented by any industry, the differentiator is in the business services (functions). The ARTS Blueprint is specific to retail, defining many SOA services by the retail functions Buy, Sell, Logistics and Administer.”

Retail technology vendors must follow closely the standard development at ARTS.

What is the first step for SOA implementation?

An SOA environment is composed of several elements (tools, policies, procedures and guidelines).

When you start planning your SOA initiative you face issues relating to these elements. Then you must make a choice: will you address all the issues on a single project or will you choose a phased approach?

Addressing all the issues on a single project is a very ambitious approach. You will have to convince stakeholders to support the entire initiative at once.

If you choose a phased approach you can start by selecting just some SOA components. Choose components which surely have a high ROI. The stakeholders must recognize the value added by these components.

Evaluate your environment and identify problems that could be solved with the adoption of some SOA elements.

For example, if you have several web services being developed without any control, selecting a registry/repository is a good choice for starting the SOA implementation.

You will gradually build your SOA environment and showing the gains at each step.

SOA and Enterprise Architecture

Some definitions of Enterprise Architecture (EA).

1. Enterprise Architecture is a complete expression of the enterprise; a master plan which “acts as a collaboration force” between aspects of business planning such as goals, visions, strategies and governance principles; aspects of business operations such as business terms, organization structures, processes and data; aspects of automation such as information systems and databases; and the enabling technological infrastructure of the business such as computers, operating systems and networks.

2. An enterprise architecture (EA) is a conceptual blueprint that defines the structure and operation of an organization.

3. The EA is:
What: The structure of an Enterprise and its blueprint describing.
How: How the Enterprise operates and the processes executed by.
Whom: People.
Which: The technology implementing processes.
Where: Showing the location of people and technology.
Why: To streamline, align, blueprint, strategically plan, and confer agility.
When: According to the Enterprise transformation plan to a target state.

4. Enterprise architecture is an agency-wide framework for incorporating business processes, information flows, applications, and infrastructure to support agency goals.

5. Enterprise architecture is the organizing logic for business processes and IT infrastructure.

EA is about describing business and mapping businees to IT systems. It is about guaranteeing that the systems really implement what users need. EA describes which system is responsible for each information asset.

Having an Enterprise Architecture is a key factor to a successful SOA initiative. Analyzing Enterprise Architecture helps to identify required services for the SOA infrastructure.

Assigning responsibility for the services is part of SOA Governance. EA also helps to do this.

If a company does not have an enterprise architecture, it is very recommended embracing SOA and EA together.

Warning: Excessive status message is costly and counterproductive

Control and monitoring are essential features of an EAI environment.

Status message has the main role in monitoring. When a data message is processed by an EAI component, this component generates an status menssage. Then a tool can use this information to show the component where the message stopped.

However you must be careful with the amount of status message.

A large number of status message can prejudice the EAI platform performance. In some cases, the entire environment must work and data message transport stops because status messages can not be processed.

The architecture must guarantee that data messages will pass through EAI even if there is some problem with status message.

Moreover, it is necessary evaluate if the status message amount makes sense.

SOA and EAI metrics

How do you estimate the size of SOA or EAI projects?

I am not a metric expert, but I think use case point (UCP) and function point (FP) are not suitable for SOA and EAI.

UCP and FP are referring to the users view of system rather than a technical aspect. SOA and mainly EAI are not about the users view of systems. They are more about system and information architecture.

It is very bad not to have a method to measure the project’s product. Without a metric method it is impossible to benchmark the performance of the team.

Maybe an expert can determine how apply FP concepts, for example, in an EAI scenario.

Please, let me know what method you use to measure your EAI or SOA projects.

Agent or proxy: what is your preferred approach?

ESB products are usually composed of several applications. Each application is focused in one specific capability like monitoring, security and service orchestration.

These applications or modules are often related to service consumer and service provider.

This class of ESB modules follow one of these approaches:

a) Proxy: Service consumer calls an ESB component that calls the service provider. The proxy can store data from consumer and provider or it can ensure the consumer has the rights to access the provider.

b) Agent: An ESB component intercepts calls from service consumer. Like in the previous example, the component can monitoring the environment and and can ensure security.

What is the best approach?

The answer depends on the stage of SOA adoption.

If you just start developing services, I think an proxy-based product is more suitable. In this case, you don’t have a legacy to be modified to use the proxy.

Otherwise, if you had already developed a lot of services, you must consider the agent approach. Agent-based products are more expansive, but they don’t require any change in the services.

Will ESB and BPMS from different vendors work well together?

Last week I was talking with a colleague about ESB and BPMS. Vendors are offering BPMS with ESB features, but the company where he works will evaluate BPMS and ESB separately.

I think it is hard to define where ESB’s responsability ends and BPMS’s responsability begins. Besides, what kind of requirement will guarantee that ESB and BPMS from different vendors work well together?

Standards could be used for ensure the interoperability of ESB with BPMS. But how to be sure the vendors really follow the standards? For example, if a requirement states that the ESB registry must be compliant with the UDDI Version 3 standard and another says BPMS must be able to discover services at an UDDI Version 3, when ESB and BPMS are from different vendors do you really believe they will always be compatible?

Choosing BPMS and ESB separately can bring the best of two worlds, but you must be careful to avoid a gap between the platforms.