With software architecture evolving toward SOA, many projects in this space have encountered challenges associated with accessing data.

As has been said, "The way an organization thinks about applications and data must evolve - it must stop thinking about data as a second-class citizen that only supports specific applications and begin to recognize data as a standalone asset that has both value and utility."

In today's world, two different types of data usage can be found: traditional architecturally layered applications and SOA framework-based applications. Naturally, each one comes with different technological and behavioral characteristics.

On one hand, there are traditional applications - usually designed and written in one language with clear separation between layers, such as enterprise Java with JSP and EJB, or .NET. Communication between layers happens in memory without any intermediate protocol (such as XML). On the other hand, there are SOA frameworks, such as Business Process Execution Language (BPEL) and the concept of the Enterprise Service Bus (ESB), offering everything that looks like a Web Service (exposed through a WSDL) to be orchestrated and to represent itself as a service.

"Organizations should establish their data environments with 'hubs of specific data families' that expose data services that comply with industry standards and service contracts. The goal is to create a set of services that becomes the authoritative way to access enterprise data. In this target service-oriented environment, applications and data work together as peers. Thus, both an organization's business functionality and data can be leveraged as enterprise assets that are reusable across multiple departments and lines of business."

In most cases, modular applications already exist and therefore data services need not be built entirely from scratch. This article focuses on aspects of migration and on exposing application functionality for later use in a SOA. It also discusses the pros and cons of the technologies being used for accessing data.

Technologies Used To Implement Data Services
Table 1 lists common technologies used by applications to obtain data. The question is which of these different implementations of data services should you use in a particular application? There are no hard and fast rules, but this article provides some guidance. Obviously, they can be differentiated through different data formats and different access methods. This article uses these data access technologies to explore the different strengths and weaknesses of data access with SOA enablement.

In a SOA, these technologies are usually composed together, because not all services are implemented in the same technology. This brings up several challenges involving transactional behavior across boundaries, including performance and mass data behavior.

More challenges? So why would you want to introduce a separate set of services in your architecture versus directly accessing a data store? The reasons to consider a separate data service include:

• Defined interfaces: Using data services forces you to define contracts that are used between the service and its clients. This is the first step towards abstracting the contract (the interface) from the implementation.
• Loose coupling/decoupling: Although a data access layer typically encourages good encapsulation and decoupling of data access functionality, it does not force the issue. Having a separate data service not only forces a well-defined contract but also minimizes implementation details creeping into the client. A client cannot bypass the interface contract because consumer and provider are not necessarily implemented with the same technology.
• Reuse: Using a common data service automatically ensures that all consumers can reuse the same implementation, which leads in the long term to reduced maintenance costs (bug fixes, changes), because code is not duplicated. On the other hand, it requires a well-defined process of change control - because more consumers rely on a functioning piece of code and potential downtime affects more than one user.
• Flexibility: Having the implementation completely abstracted from the consumer through a well-defined contract offers greater flexibility. Over time, the implementation technology can change; additional performance enhancements can be introduced; or data stores can be upgraded, migrated, combined, or divided - all of this without distracting the applications using the data service.

Common Data Challenges in a SOA
In general, the challenges can be divided into four main groups, each defining a different part of an overall application: access, enrich, distribute, and persist. Usually you start by thinking about how to access a certain data store. Should it be done via handwritten Java code that embeds SQL with JDBC calls? Should an object-relational mapping (ORM) approach be taken? Should the Java Connector Architecture (J2CA) be used to connect to a foreign data source? Is the client Java or a .NET application - and therefore can a native protocol be used or not?

After considering the access options, the next step is to validate whether aggregation of data across boundaries is necessary, whether a high data load is expected, and where the data comes from. All of these options are considered in the discussion of Challenge 2 (enrich, cleanse, and aggregate data).

Another major challenge involves which and how many users plan to consume a service. In particular, the importance of interoperability should not be underestimated if data is being pulled from or pushed to a consumer (such as business-activity monitoring [BAM]). This is described in the discussion of Challenge 3 (distribute data).

Last but not least, there is the challenge of writing data back and guaranteeing consistency across multiple calls to a service as well as across boundaries. In SOA, use cases are generally implemented across technological and service boundaries as they are orchestrated into a process. These questions are covered in the discussion of Challenge 4 (persisting data).

Challenge 1: Access Data
Abstraction of data - A domain model is abstracted from the underlying data store. This model should be designed according to the requirements of the client applications. It could mirror the underlying data store or provide a rich abstraction. In either case, the consuming application is decoupled from the physical storage by this model.

In reality this means that a customer object used in the application usually differs from the underlying data stores and their view of data, such as object-oriented versus normalized.

Speed considerations - Allow for efficient retrieval of domain objects based on exposed operations. This is basically the exposure of queries the application needs. It also provides a natural boundary for testing and for the introduction of mock data in test scenarios. Because the next generation of applications and business processes are composites that leverage these services, data access abstraction must be considered a key aspect of a service-oriented testing strategy

Challenge 2: Enrich, Cleanse, Aggregate Data
In modern applications, data usually does not come just from one place and get presented directly to consumers. It often needs to be formatted (e.g., a date), aggregated or enriched, or cleaned.

For example, a big part of an employee record might come from the HR system but the employee's vacation time is stored in some other place requiring a merge of those two sources into one.

There are several ways aggregation or enrichment can be implemented. At the most basic level, it can be done natively in the application no matter what the technology. However, if the underlying data model changes (or just a column name changes), several places need to be updated, which can be time-consuming. This leads to the use of object-relational mapping solutions because the domain model is easily shared and the mapping can be changed just in one place. On the more sophisticated side the data service implementation could access multiple data sources using native results to compose the common domain model. (Table 2)

When multiple data sources return a variety of XML documents that comprise the common domain model then XML aggregation is the answer. Due to the evolution of orchestration and XML as a common data format, BPEL is the perfect tool for aggregating or enriching data that is already in XML and later serving it to consumers. Although for large payloads and results, using an Enterprise Service Bus might be a better choice.

Challenge 3: Distribute Data
When all components and layers reside within one application, access is native and in memory. In a SOA, however, services run in a decentralized manner and in different places, introducing the challenge of different protocols and even greater need for solid error handling. What happens if a service is down or not accessible - how will the application be affected? The use of BPEL offers a great way of composing these services into one process or composite services, which can be leveraged later. Using this approach brings more freedom to the service's consumers but also introduces another layer that needs to be maintained.

One purpose of this is data pulled into several UI technologies, such as a portal or BAM to monitor performance.

For example, a supervisor wants to see the performance of employees in real-time and wants to be able to take corrective actions (such as rerouting a request to other teams). In this case, the data from the business process (and its services) can be pulled out into a BAM instance to create a real-time dashboard. (Table 3)

Note: The technology mapping of this section has already been covered in the Access table.