Integrated Infrastructure Blog

Chapter 5: The Data Integration Dialogues (Part 2) is Now Available

Chapter 5 of The Semantic Dialogues, The Data Integration Dialogues (Part II), is available now. Here's an excerpt: "Sanjiev, Patrice and Jerry are reaching consensus on the best approach to their data integration problem. And Cliff is showing some enthusiasm about using the data subdivisions of the SID as a guideline for internal projects. But it will be in vain if the team can’t find a way to put their theories into practice. Will their agreement last long enough to find a technical solution? "

Click here to read Chapter 5, The Data Integration Dialogues (Part II), today. And if you’re enjoying The Semantic Dialogues, be sure to share it with a colleague.

The Semantic Dialogues tells the story of how National Networks, a fictional telecommunications service provider, pursued and achieved data interoperability within their SOA infrastructure. The story begins with an urgent request from Operations for a fix to a problem that's keeping them from billing for new services. Assured that extensions can be made to the common data model (the SID) for the COTS billing system that's causing the headache, the problem is resolved in short order. The story then goes back a year to a time when the lack of a data interoperability layer made similar problems seem insurmountable and a tiger team is formed to put in place an OSS/BSS architecture that will change the way the CSP does business.

Chapter 4: The Data Integration Dialogues (Part 1) is Now Available

Chapter 4 of The Semantic Dialogues, The Data Integration Dialogues (Part I), is available now. Here's an excerpt: "There's a new kid on the block, and Patrice is shaking things up. She wants to write off National Networks' investment in an enterprise data model she hasn't even seen, and replace it with an industry standard that most of the team is unfamiliar with. Cliff is openly skeptical about the upside to standardization. Naturally he's attached to his previous work, but he's also worried about adopting a complex model that looks a lot different from his. Will Jerry's vote for a do-over prevail? "

Click here to read Chapter 4, The Data Integration Dialogues (Part I), today. And if you’re enjoying The Semantic Dialogues, be sure to share it with a colleague.

The Semantic Dialogues tells the story of how National Networks, a fictional telecommunications service provider, pursued and achieved data interoperability within their SOA infrastructure. The story begins with an urgent request from Operations for a fix to a problem that's keeping them from billing for new services. Assured that extensions can be made to the common data model (the SID) for the COTS billing system that's causing the headache, the problem is resolved in short order. The story then goes back a year to a time when the lack of a data interoperability layer made similar problems seem insurmountable and a tiger team is formed to put in place an OSS/BSS architecture that will change the way the CSP does business.

The Enterprise Architecture Dialogues - Chapter 3 of The Semantic Dialogues

Chapter 3 of The Semantic Dialogues, The Enterprise Architecture Dialogues, is available now. Here's an excerpt: "Sanjay believes that enterprise architecture is the solution, and he takes a hard line on forcing it into the enterprise. In his view, all they need to do is complete the architecture with a data strategy. But Jerry convinces him to see things from the business users' perspective, and proposes some radical changes to the way they use applications. Mick sees an opportunity to make the business happy and fix IT's reputation. Are they all drinking Kool-aid?"

Click here to read Chapter 3, The Enterprise Architecture Dialogues, today. And if you’re enjoying The Semantic Dialogues, be sure to share it with a colleague.

The Semantic Dialogues tells the story of how National Networks, a fictional telecommunications service provider, pursued and achieved data interoperability within their SOA infrastructure. The story begins with an urgent request from Operations for a fix to a problem that's keeping them from billing for new services. Assured that extensions can be made to the common data model (the SID) for the COTS billing system that's causing the headache, the problem is resolved in short order. The story then goes back a year to a time when the lack of a data interoperability layer made similar problems seem insurmountable and a tiger team is formed to put in place an OSS/BSS architecture that will change the way the CSP does business.

Report from Lisbon

We have just wrapped up the latest TM Forum Team Action Week in Lisbon.  This semiannual event is unusual as conferences go – no exhibits, no keynote addresses, no marketing – just a couple of hundred people with a common interest in creating, improving and using standards and frameworks. 

It’s something that everyone in the ICE (information, communications, and entertainment) sector should experience at least once.  You can spend the whole week with a single framework or interface team, or sample the ongoing work of a variety of projects, including the key planning sessions for upcoming Catalyst projects that will be demonstrated at Management World a few months later. And it’s not for geeks only – there are as many debates about business requirements as there are about interface specifications. 

Given the state of the world economy, it was reassuring to see the high overall turnout and the record number of first-time attendees.  Even better, people were engaged and enthusiastic.  Meetings of the teams driving TM Forum business transformation initiatives like Cable, Content Encounter, Customer Experience, Revenue Management, and Service Quality attracted respectable crowds and made visible progress towards their goals.  I found similar energy when I participated in meetings of the Industry Group Advisory Council (IGAC) and the Defense Interest Group. The number of liaisons with other standards organizations under IGAC has increased steadily, and this week saw a full meeting table and more active projects than I can recall over the last eight years. The Defense Interest Group meeting was also packed, and the discussion for its catalyst project spilled over into the break periods.

I spent most of the week representing the Information Framework (SID) in cross-team meetings with the Architecture Harmonization, Application Framework (TAM), Process Framework (eTOM), Service Delivery Framework, and TM Forum Interface Program groups.  The meetings were productive, and reflected the Forum’s emphasis on getting value from the synergy that exists between the Frameworks and across standards organizations. As one example among many, inspired by the recent groundbreaking work by the Process Framework team in partnership with itSMF to map the eTOM to ITIL processes, the Information Framework team has begun a study to harmonize the SID with ITIL’s Configuration Management Database (CMDB). We plan to release a draft technical report by Management World in May, and I’m optimistic that the mapping we develop will find its way into demonstrations as well.

And if you’ve never attended, think about catching the next Team Action Week this summer – it’s already scheduled and well worth the investment.

The Data Architecture Dialogues - Chapter 2 of The Semantic Dialogues

Chapter 2 of The Semantic Dialogues, The Data Architecture Dialogues, is available now. "With too many different data stores, too much bad data, too many incompatible definitions of the data (and forget about managing change) it doesn’t seem like a bad idea to create an Enterprise Data Platform. But some members of the team push back. Cliff Chen, database designer, stands his ground as the debate continues."

Read Chapter 2, The Data Architecture Dialogues, today. And if you’re enjoying The Semantic Dialogues, be sure to share it with a colleague.

The Semantic Dialogues tells the story of how National Networks, a fictional telecommunications service provider, pursued and achieved data interoperability within their SOA infrastructure. The story begins with an urgent request from Operations for a fix to a problem that's keeping them from billing for new services. Assured that extensions can be made to the common data model (the SID) for the COTS billing system that's causing the headache, the problem is resolved in short order. The story then goes back a year to a time when the lack of a data interoperability layer made similar problems seem insurmountable and a tiger team is formed to put in place an OSS/BSS architecture that will change the way the CSP does business.

The Semantic Dialogues Have Begun!

Are you thinking about OSS/BSS Integration?

The Semantic Dialogues tells the story of how National Networks, a fictional telecommunications service provider, pursued and achieved data interoperability within their SOA infrastructure. When the VP of Systems Architecture, Eric Golden, welcomed Mick Dundon, his new director of Data Architecture, he didn't pull any punches. "The top priority for your new team is solving our data problem. National Networks is under pressure from competition and regulators and needs to become more efficient. We can't afford another round of failures in our attempt to control OSS/BSS sprawl." And with those instructions ringing in his ears, Mick built the team that could make it happen.

How does this dialogue evolve? Who's involved and what decisions are being made? Are they the right ones? Does the story sound familiar? Read the prologue and first chapter of The Semantic Dialogues today. Then revisit every two weeks when we post the next chapter. Follow The Semantic Dialogues... they have begun.

One data model, many approaches to integration

The use of a common data model in SOA has been recommended in definitive posts - from the viewpoints of integration and of governance.  The common model architecture facilitates integration by providing a common ground for data transformation, and supports governance by bridging the business and implementation views of enterprise information.  When a common model is properly used, these principles generally apply regardless of the runtime environment.  But a common model supports several quite different styles of integration.  We could look at those integration styles as a kind of spectrum, with extremes at each end and a blend in between.

One end of the integration style spectrum is represented by what I call the common interface style.  OSS/J and MTOSI are examples of this style in the communications sector, and there are comparable examples in other sectors.  The common interface style promotes a universal, standard interface between applications.  It requires that all participating applications use the standard interface, reducing some common integration problems and costs.  The level of adoption of standardized interfaces varies among domains and among application vendors.  A common model can be used to implement adapters over non-compliant applications in order to conform to the standard interface.  Although the common model and its mappings may resemble a hub and spoke model at design time, the runtime view of this style typically requires that the adapters be highly distributed.

In contrast to the common interface style, the other end of the spectrum is what I call the data integration layer style.  Several OSS transformation projects are examples of this style.  It starts with each application as-is, and adapts their existing interfaces to a set of "canonical" interfaces based on a common data model that supports mapping between all participating applications.  It also adapts canonical interfaces back into application-specific ones.  This allows process management infrastructure to intercept and redirect canonical requests, providing access to business process that would otherwise have been hidden in (and locked into) the "wiring" between applications.  For this style, in contrast to the hub and spoke design view, the runtime view depends largely on the location of the mapping services, but it usually inserts process management between requesters and responders.

Other integration projects fall somewhere between the ends of the integration style spectrum, with a mix of adapters, canonical interfaces, and direct translation between applications.  It will be interesting to see how these and other styles evolve, as SOA best practices and SOA-supporting technology improve.

Data Models and Message Specifications

Enterprises of all types see increasing value in standards-based development and integration, and they often look to the TM Forum for deep and consistent models of business process, data, applications and system architecture. Moving from analysis through design to implementation, a portion of a project’s focus typically shifts from process and data models to message specifications.

Like other standards organizations, the TM Forum is well known for both data models and message specifications. These standards often overlap and interact, but they fill distinct roles, so it is interesting to trace the paths of their evolution and likely future. And integrators and enterprises reveal their integration philosophies by the way they employ them.

Data models have evolved from both "top-down" and "bottom-upv perspectives. The principal TM Forum information and data model, the Information Framework (SID), shares the top-down, domain-oriented perspectives of the other TM Forum Solution Frameworks. Like the other frameworks, it has a hierarchical structure that supports decomposition or "drill-down" into successively finer levels of detail. At the same time, it has been developed and enhanced from the bottom up by contributions from many sources at multiple levels of detail, and it embraces a "blade" concept that allows domain-specific additions to be made without disrupting the rest of a domain.

Successful message specifications have generally emphasized top-down development to ensure consistency across messages and message sets. Even large-project message specifications like the TM Forum's MTOSI and OSS/J, which rely on multiple design and implementation teams often working in parallel, use centrally managed design guidelines, message patterns, and common data models developed over time. Lessons learned from their success are reflected in the new generation of interfaces designed and implemented under the TM Forum Interface Program. This program will support a variety of interface styles, including but surpassing the scope of MTOSI and OSS/J, and all implemented on a shared framework based on a common data model.

In my next post, I'll look at the ways in which a single data model can support a variety of application and SOA integration philosophies.

NGOSS Contracts Gain Momentum

One of the most enjoyable aspects of my job at Progress is my daily contact with standards organizations like the TM Forum and OASIS. I work with many other industry-supported contributors to develop and extend the frameworks and guidelines that address the challenges they face in application development and systems integration.

Within the last month the TM Forum has issued for member evaluation new versions of three NGOSS frameworks: the Business Process Framework (eTOM) 7.5, the Applications Map (TAM) 3.0, and the Information Framework (SID) 8.0. These releases continue the evolution of the frameworks in both breadth and depth, and also support the upcoming release of the new NGOSS Contracts guidebook.

NGOSS Contracts extend the reach of the NGOSS frameworks into the system and implementation views. These contracts provide standards for application interaction that complement the existing SOA view of services and operations by defining the bigger picture within which those services run, such as conditions and constraints, message exchange patterns, and service level agreements. NGOSS Contracts also represent a meeting place between top-down service design based on the interplay of process, data and application models, and bottom-up service design based on real-world experience with successful service function and granularity.

Having seen firsthand both the promises and disappointments of client-server, CORBA and now SOA, I believe that contracts like those promoted by the TM Forum will make a critical difference in the long-term success of SOA infrastructure compared to earlier architectures, because contracts can cleanly separate the process implementation from the data model. This opens up the possibility of using a data integration layer, along with a robust business process management layer, to achieve real separation of business process from application interface details. The process layer can then select from alternative applications at a fine-grained level without the need for application-specific process changes, allowing an enterprise to reduce or eliminate expensive customization and to encourage competition between application vendors. Successful large-scale OSS transformation projects have demonstrated the technical and economic validity of this approach.

The importance of the data layer is also emphasized in a new white paper by Dave Hollander, a co-inventor of XML and a veteran of changing data standards. Dave examines the disparity between SOA's success in large-scale service interconnection, and its failure to address operational differences between applications, resulting in disappointment and failed projects. He concludes that the common model architecture is the key to achieving that semantic integration. Dave's paper, Common Models in SOA: Tackling the Data Integration Problem, will be available soon and I'll send a link to it when it's finalized.