Sunday, March 09, 2008

Canonical Data Model

Check out this video evangelizing a canonical data model (CDM) at both the enterprise and industry level.  For me, the number of opportunities for a CDM are often much greater than the actual implementation of one.  To date, we only have CDM defined for Items, Lots, and Shipments.  Ironically, most 'canonical' examples of a canonical data format are for a Customer, or Employee - these entities originate from systems outside our systems, and political, boundaries. 

Canonical Data Model Visualized.

No Vendor ESB

Blame it on a full stomach and a good hefeweizen, but I found this InfoQ video, titled 'Jim Webber on "Guerilla SOA"", had a very interesting/refreshing take on the whole 'ESB to enable your SOA' party line that I've been consuming for a little too long. 

The underlying theme is that you don't need a heavy handed, proprietary vendor solution to SOA enable your business, and that business services should come before technical services.  This resonates very well with a similar message of 'middle out architecture' being evangelized by Nick Malik and Gabriel Morgan (don't remember who coined the term first, sorry guys!) - one that I very much agree with.  Getting the balance and timing right of identifying which technical services are needed to complement, or support, your business services can be something of a dilemma. 

However, I think that there are certain key technical services that are likely a corner stone to any implementation: tracking/logging, repository for endpoint management, and recently service virtualization.  I'd probably also throw in metadata management, but I don't know how many other organizations are as disperse as we are.  On top of that, I'd layer best practices for messaging patterns, exposing data contracts for interoperability and a handful of other topics and a lot of guidance from the 'services' guys, but by all means, let the business developers own their services.

Tracking/Logging - not all services are web services and providing a low-friction way to get data into a common store for tracking/logging purposes not only helps in debugging services in a distributed environment, but helps enable monitoring for SLA.  Once you've got a common store, you've got a common user experience for accessing data for debugging, or operational data.  This is something we are taking a new look at solving an old problem, and you might even say the next evolution of it, at least within our organization.  I'll probably post more on this as I'm sure other organizations run into a similar problem and I'd love to hear how others are keeping it from getting out of hand.

Repository - end-point management is one area that I think that everyone in my organization has finally said 'enough'.  Lets have a solution where we register the available endpoints in a repository and have a common method for getting the address of a service.  We are less concerned with an actual industry standard (such as UDDI) as we are with it being low maintenance.  This has less on the impact during development and a greater impact on the long term maintainability of the services layer. 

Service Virtualization - this part of the end-point management umbrella.  I've recently been introduced to this concept for services, but its one I'm familiar with from a hardware perspective.  In the hardware world, you might provide high availability by clustering to machines by providing a 'virtual ip-address'  (VIP) to address a resource hosted on the cluster.  This VIP is then what the client uses to access any resources hosted on the clustered machines.  The physical machines that participate in the cluster can be removed, for maintenance (e.g.), and not impact the clients ability to access the resource.  Service virtualization works much the same way.  You provide a 'virtual' service address (URI) that is used by the client and the 'virtualization service' then manages what physical service the requests get routed to.   With that additional layer of abstraction now between the client and the physical service, additional policies, transformations, etc. can be applied to the request.

Metadata Management - this might also be part of your configuration system, or it might also be a service for simple transformations to core configuration (currently for us).  For example, we have many development (and even production) 'environments'.  Normally, you think of them as 'Development', 'Staging' and 'Production'.   However, when you've got many environments, they sometimes span business units, likely rarely keep the same environment name, etc. so being able to have a quick way to lookup these mappings helps facilitate integration scenarios (which is why your looking at web services already, right?).  My cost center for my ERP application might map to a completely different value for my Order Entry system. 

 

Anyway, let me try to reign these thoughts back in.   The point of this post was that I agree, that a full blown ESB implementation can seem like overkill, especially when the adoption of these services is in question, whether for practical or political reasons.  Of course, since we do own BizTalk, coupled with WCF, MSMQ, WS-* support, you've got many of the components required to build whatever level of implementation is required to enable the business.

Of course, Jim did bang on BizTalk a little, but only in a good sensible way in that you should try to isolate your consumers from the actual implementation of any services.  A lot of that work can be accomplished at the message layer.

I intentionally left security out of the mix (much to the chagrin of many, I'm sure) mainly because security appears to be more of something that's hardwired into an integration story rather than a service that I might want to plug in.  That's not to say that I don't want to be able to apply policy to my services on how they are secured, just that its usually predestined how we will secure our services and is something that'd enabled (policy, or otherwise) with relatively little pain.  I expect this story to evolve over time.

Contact me with any feedback, or additional thoughts.  I'd appreciate it!