This article is the second part of a two-part series covering best practices for building Service Oriented Architecture (SOA) applications. The following are the seven key steps for effective SOA adoption:

1. Create a portfolio of services
2. Define connectivity and messaging interfaces
3. Process orchestration, workflow, and rules
4. Rich user interfaces
5. Business activity monitoring
6. Security and management
7. Performance and scalability

Rich User Interfaces
We've seen several generations of UI evolution since the emergence of the Web as an application interface paradigm. Initially, HTTP and HTML provided many benefits for administrators through a thin-client approach, but users were faced with GUIs that were much more primitive than what could be done with thick-client interfaces. With the emergence of rich Asynchronous Java And XML (Ajax)-style interfaces, we are now seeing a truly mature thin-client paradigm.

However, developers often find the complex JavaScript code for user interfaces to be cumbersome, hard-to-debug, and repetitive. In this area, the emergence of Java Server Faces (JSF) frameworks that encapsulate rich dynamic GUI capabilities in reusable components has given developers some new tools to make the development of rich Web GUIs easier.

As Web GUI paradigms evolved, developers were faced with more choices. In our first article, BPEL was discussed as the standard for business process orchestration, and GUI page flows are sometimes considered "orchestrated" interface components. However, BPEL is usually not the right abstraction for page flows. We see JSF and its predecessor, Struts, as being the best way to implement user interface control flow in the Java/J2EE world. BPEL is best for structured flows, but page flows are typically semi-structured or unstructured. Although BPEL is also particularly important when you need to maintain audit trails and when the process strictly controls the order of execution of activities, but GUI flows usually don't require these.

Of course, applications often connect their GUIs to business processes through human worklist interfaces, custom Web interfaces, and portals. BPEL's ability to support Web Services interfaces and transactional interfaces via adapters and WSIF bindings makes it easy to integrate J2EE GUIs and portals with BPEL processes. Standards like WS-Remote portlets and JSR-168 mean that vendors can publish process portlets, such as a worklist editor, in a way that's easy for developers to integrate into a portal of their choice.

Business Activity Monitoring
A common complaint in organizations is that they have lots of data but not enough information. For example, we have a client who described his problem as having "14 terabytes of data but no unified view of our customer." One of the best ways to avoid this problem is to define key performance indicators (KPIs) as early in the SOA design process as possible. KPIs are pieces of information that the organization wants to track, such as the number of business transactions that are processed a day, the number of exceptions that are raised, and the amount of the time it takes to process each step. Because KPIs can change over time, the most effective approach for gathering this information is to instrument processes and IT events with "sensors" that monitor the business transactions. The events can then be fed to business activity monitoring (BAM) dashboards (Figure 1) and custom reporting channels without requiring that process logic be changed.

Once the events are identified, correlated, aggregated, and fed to rich real-time dashboards, an organization achieves what we call the "fusion effect." (Figure 2) This occurs when actionable information informs an organization how to improve its processes, and its agile IT environment lets these changes be implemented efficiently.

Security and Management
Security has become increasingly critical as the perfect storm of information proliferation, regulatory change, and identity theft disclosures have come to pass. Industries such as healthcare and financial services require an unrelenting focus on security as information such as an individual's medical and financial data is passed over a network. Addressing these challenges is particularly complex in a heterogeneous and fast-changing technology environment. Conveniently (and not coincidentally), key standards such as WS-Security have emerged to enable the secure exchange of information between processes and services, even across different technology stacks such as J2EE and Microsoft .NET.

WS-Security specifically provides a standard mechanism for authentication and access control for services, as well as full or partial encryption of message data. WS-Security support is available in Microsoft .NET services, Open Source Web Services frameworks such as Apache Axis, and commercial J2EE toolkits such as Oracle, BEA, and IBM's application servers. It's easy to find information describing how this interoperability works. For example, Microsoft MVP Jesus Rodriguez has code examples on his blog demonstrating WS-Security interoperability between Microsoft WSE 3.0 and Oracle BPEL Process Manager (http://weblogs.asp.net/gsusx/archive/2006/03/22/440881.aspx). Likewise, Security Assertion Markup Language (SAML) provides a standard mechanism for role-based access control and federated identity. Standardizing on WS-Security and SAML (Figure 3) for service interfaces gives an organization much more flexibility in its future technology choices and for secure Web Service interactions with trading partners.

It's also important to extract security requirements out of core services and clients and implement them in a policy-oriented fashion. This results in systems that are dynamic, secure, and auditable. Organizations implementing this approach are able to define external security policies and change them dynamically, without needing to modify services or the clients that call them. This approach is supported by leading Web Services management (WSM) products.

Performance and Scalability
Once security policies are in place, the next step to effective SOA is to focus on the performance and scalability requirements in detail. As ever, the principle of "a stitch in time saves nine" applies. For example, we've seen project teams that used all asynchronous interfaces for their services because the toolkit they were using made that very easy to do. After developing sophisticated processes for handling registration for consumer credit services, the project team did stress tests late in the development lifecycle. They discovered that the overhead of the asynchronous interfaces, which required frequent persistence of the overall process, was such that that their anticipated load could only be supported by an unaffordably large number of CPUs. When such information is discovered so late in the process, the choices are bleak: either increase the budget significantly or re-engineer all the services to use different interfaces - which is a change that will propagate painfully throughout the project.

The best way to avoid this scenario is to do a performance POC early in the development process (even at the design stage) and get some real numbers regarding the size of the systems that are needed to achieve expected loads. By doing this during early prototyping and design stages, potential performance bottlenecks will be uncovered while there's still time to change key design decisions.

Another best practice is to choose carefully among synchronous and asynchronous service interfaces, standards such as WS-Addressing, and custom correlation mechanisms for correlating asynchronous messages. WS-Addressing provides a standard mechanism for correlating asynchronous messages so that system A can send a request to system B, and system B can call back to system A when a response is ready. This kind of asynchronous interface does have a performance cost, but you gain reliability and flexibility because the two systems no longer have to be tightly coupled to each other. Of course, projects have been built on top of asynchronous message-oriented middleware such as IBM MQ Series, TIBCO, and JMS messaging for years. What's new is that the benefits of asynchronous interfaces are now available through standards such as WS-Addressing over protocols such as SOAP over HTTP so that such implementations can cross technology and vendor boundaries more easily.

When considering Web Services as an integration approach, people sometimes worry about XML as a performance bottleneck, and it can be when used inappropriately. However, in general, we don't believe that XML in and of itself presents performance overhead sufficient to rule it out, even for very large load requirements, especially given its many benefits. As when Java emerged to replace C and C++ as a preferred programming language, it takes a little time for design-time and runtime tools to evolve to optimal performance for the latest development approaches. We're now starting to see toolkits for XML processing. These toolkits, such as Oracle XDK, allow operations such as dehydration, XSLT transformations, and BPEL assign activities to be applied while the data remains in an optimized binary format. This avoids the most expensive part of XML processing - serialization and deserialization. For external gateway-style transformations or WS-Security support, hardware devices such as the one created by DataPower (recently acquired by IBM) and software tools such as Forum Vantage XML Accelerator can be useful.

However, there are ways to misuse XML. For example, passing very large documents between services via SOAP requires large amounts of bandwidth, processing time, and memory to serialize and deserialize the documents, even if you can minimize these steps. A preferred approach is to store the documents in a central location (a file system, database, or document management system) and then pass references to the document.