Directory Services – Fear and Loathing in the VAN business….

This article is about the arcane subjects of accessible directory services in EDI inter-network relations. It’s about more than that, really. It’s about what we as an industry put up with when we ignore best practices, architectural issues, and standards.Although this is a rather lengthy diatribe, I am hopeful that colleagues of goodwill read and comment.
I was recently reviewing a VAN ID migration request; these frequently arrive in the ECGrid Network Operations inbox (as emails from other VANs), and this re-booted a thought about directory services, or, the EDI comms sector’s lack of a common architecture for resolving network IDs to their owner and provider. Tolerating the  long-standing lack of basic directory service has been fodder for numerous animated conversations amongst our colleagues at other VANs, service provider, and many Enterprise level End-users.

Though the topic ebbs and flows with the seasons , I feel that directory services should be addressed until a workable solution is adopted. This issue is even more important today, as B2B communications networks are evolving and morphing along the lines of  SAAS, Enterprise 2.0 services, and the B2B Cloud.

Cloud computing and hosted, enhanced communications services are changing the way EDI is implemented.  I am fortunate to have a front row perspective on the impact of innovation in the EDI Communications sector, as I serve next to a fellow whose sole focus is on bringing new ideas to our sector – the first advanced, node based EDI network of networks, the first network management API, and even more to come.

That fellow is Todd Gould, Loren Data Corp’s President; I am fortunate to work with the EDI industry’s preeminent communications systems engineer.

I am so often gratified with the enthusiastic participation Loren Data has garnered in bringing API services to the EDI market. It seems, not so long ago, that these ideas were greeted with a very cold shoulder. I have learned how to identify those who actually need high level embedded communications services; it’s the movers and shakers, the entrepreneurs,  it’s the professionals who have a better idea to build and express.

If I ever expected that the well entrenched portions of our sector would throw a party for the market entry of the ECGridOS API, well, I just did not know this was an unrealistic expectation !

However, 2011 can be safely termed a banner year for ECGridOS API adoption. Our developer partners are bringing outstanding products to the market – many of these are becoming well known and quite successful – so therefore we might briefly take a quick bow, and give major credit to the developers who took on ECGridOS projects, because some are quite new to the EDI business, while others are very well-known, ecommerce leaders. Thanks to your all! We have more innovation coming before the end of 2012.

The ECGridOS API has a natural constituency: New EDI ventures, specialist providers in EDI verticals, and B2B service providers of the third generation – as the ECGridOS Product Evangelist, I calls these “EDI Cowboys  and Bandit Queens” – they are highly skilled EDI professional creating entirely new applications. Many are using ECGridOS as a “scaffold” on which to build the next era of ecommerce.

Such optimism stands in sharp contrast against a backdrop of PE funded B2B consolidations that destroy value and reduce innovation. As if the preceding  litany was not enough to start a conversation, we have, in a year, witnessed cloud computing  have a real effect on the economic model of delivering EDI and B2B services. The mid market’s adoption of vertical B2B services (lateral supply chains, ad hoc industrial and technical ecommerce) seems ready to push towards new models.

If I can believe the analyst’s Ouija Board (who amongst us is skeptical?), there is an estimated $$30B in new revenues percolating on the sidelines if the mid market adoption issues can be ironed out. To make that a reality, some of our long lived EDI behemoths simply must take up the gauntlet and work with the smaller innovators; and I know that Todd is ready to take on new partners.

As a network of networks, ECGrid serves professional EDI service ventures, telecoms, managed B2B, enterprise (ERP, WMS) software, and the B2B Cloud (PAAS/ SAAS).

Back to Directory Services:

QID/GLN ID to name resolution was designed into ECGrid from day one. ECGrid went live with directory services in 1997, while ECGridOS API was born in 2009; among its 150+ functions are directory servies and network ID / look-ups.

Todd was ahead of his time, and ahead of the curve in EDI network services, and he probably shall remain so for quite some time.

Just to be clear, I’m not speaking of end-user maintained directories or EDI industry who’s who portals, which might be useful. I am referring to access methodologies used to resolve EDI QID’s (GLNs) to its authoritative provider, such as a VAN, Service Provider, supplier hub, etc.  Modern directory architectures (LDAP, XDP, DNS) offer advanced services exceeding the basic ‘resolvability issues’ plaguing our industry, but the lack of even the most elementary services (resolving ID to Provider), often results in lost productivity, increased operating costs, and degraded reliability. There are even more troubling issues of culture and policy to ponder, which I will discuss in a latter article.

If the loss of productivity and wasted time are insufficient reason to provoke one’s concern, consider a lack of directory services in today’s critical supply chain communications:

As mentioned in the introduction, EDI networks, VANs, and other communications providers, send status messages to their interconnected peers, informing them of ID changes and / or clients that are migrating to other networks. This manual process worked during the golden age of VANs, in a global market comprising no more than a handful of networks – but today we have at least 129 interconnected EDI communications providers that maintain a direct peer system of interconnects.

I cannot name one contemporary VAN or alternative EDI communications hub that maintains a (published) routing table, with an accompanying system to replicate, advertise, and deploy routing changes to said table. Such tables are used in BGP routing systems, while DNS uses replicating root servers. We all make use of these proven systems to resolve names to network IP addresses.

In the quirky world of EDI Communications, network providers will receive a list of ID’s containing long lists of changes with no client names or any tangible guideline to aid their cooperating collegial networks. Those wishing to execute such authorized change requests (as accurately as possible), often find themselves in a bind when preparing to make such crucial changes to their network’s routing logic.

This practice of circulating blind migration orders is considered madness in the “adults only world” of IP data networking services. When there were just a few VANs, such an anonymous list of ID’s with an accompanying request to ‘move yay to yay, and hay to hay” was acceptable. Errors were corrected by some twenty odd colleagues running operations for that era’s VANs, and most were on a first name basis.

In today’s highly fraught world of EDI Communications, an increasingly diverse and competitive cohort of providers vies to capture market share, behaving as if this was a zero sum game. We now see a reason behind “blind lists” of unattributed ID migration requests is nothing more than a fear of competition. This is so silly.

If a large migration list contains errors, and is subsequently circulated, the penalties are not limited to time and profits, but contain a hidden vulnerability that the EDI sector has failed to recognize. Let me draw a picture for you:

I have not seen the following scenario aired in any EDI industry forum:

A medium sized VAN is scooped up by a competitor, who proceeds an expected ID migration. A list of hundreds, perhaps thousands of ID’s are circulated to the acquired VAN’s peers (interconnects) and these fifty or more VANs and supply chain systems start the process of updating their internal routing logic.

To effect a smooth migration,  all kinds of internal accommodations and bits of relevant information (delimiters, firewall port and boundary rules) is required. Only a Netops engineer could love this stuff.  So begins a migration involving dozen’s of ecommerce providers interconnected to the newly acquired VAN.

Does it end happily? If you guessed yes, you assumed wrong; in this instance, as the disaster unfolds and picks up steam, the migration eventually becomes a VAN equivalent of the apocalypse:

An astute engineer at one of the VANs sees the list, and before he initiates migration numero uno,  calls a meeting of front line personnel – they are in agreement that these migrations requests, denuded of all identifying user metadata, are wrong in their specifics. Who is the home network of the IDs in question?

The migration list is rife with errors; the list was circulated to 59 networks, most of whom simply executed the erroneous changes without checking. The resulting confusion over which network ID is ‘homed’ to which network becomes the ‘EDI scandal de Jure, as only two VANs caught the errors, while the rest erroneously updated their routing. Oh, the pain.

Despite best efforts of the two smarter VANs, some supply chains are disrupted for days. The confusion is exacerbated by certain VANs denying the error, while others are so technically inept and disoriented, that their clients are essentially held off-line for days, and in some cases….weeks.

Finally, a few major clients are forced to self migrate to a competent VAN or alternative service providers.

Some will say, “This could never happen!”, “there must safeguards?!” Well, surprise surprise – its has happened, quite recently, and nearly caused a recent VAN buyout to be termed (within the PE establishment) “a non performing acquired asset”.
The migration was saved, by the skin of its teeth, by a white knight VAN that shall remain, nameless.

The sector’s intransigent refusal to implement directory services is representative of a malaise haunting our best minds, and is symbolic of a techno-cultural roadblock that is dis-empowering and disparaging to end-users. Such a state of affairs cannot endure much longer, especially amidst the upheavals of consolidation in the VAN market.

An EDI industry bereft of directory services is a hobbled market.

The sector’s  ability to reach it’s full potential is compromised, the penalty is one where the EDI market remains a fussy, quirky, and inflexible niche technology. Despite the incredible gains made by ecommerce service providers, web-based EDI, integration as a service, and Cloud B2B, the vision of plug and play data interchange remains a lofty and somewhat distant goal, many use the term, “Pipe Dream”.

Directory Services are a mezzanine technology, enabling the evolution of future advanced network services. All networked industries, (take your pick), support some type of global directory services; such directories are used by network providers, are built into the OS, Servers, and client software – directory services make today’s Internet applications possible.

Until the EDI communications sector confronts the implementation of directory services, other EDI innovations shall remain on hold. I can say with conviction that directory services are a special class of services, they mark an interconnected sector as “mature”, or at least ‘well on the way’, to acceptance by a broad mid market.

If any one technology can be credited with greasing the Internet’s skids and enabling its killer applications (such as the Web and email, IM, and VOIP), it is DNS –  a very simple directory for resolving names to IP addresses. Underlying DNS is BGP, or the Internet’s route propagation system, and so on. These standards-based systems facilitate new ventures – because they can be relied on to perform.

Directory services should figure prominently in the future evolution of ecommerce services. The touchstone issues of identity, resolvability, portability, and route-ability are intimately tied to the rights of delivery and message bailment, and the nature of communications provided to contractually bound  parties (trading partners) – an area of policy actively ignored by at least one prominent Value Added Network. The foregoing does not auger well for the future of EDI, the ubiquity of EDI messaging services, and trading partner rights.

But I am an optimist, and the company Todd founded is the industry’s sole API provider. Loren Data Corp advocates strenuously for the technologies that enhance and extends the reach of end users.

If we can create services that are easy to use, along with evolving data interchange standards, more and varied industries beyond the supply chain will become our clients.

Bringing the EDI Communications industry into the 21st Century, by implementing standardized directory services, we can all create a much larger market that recognizes an axiom dear to me, one that has stood the test of time:

“Respecting and empowering the trading partners creates a larger EDI Communications market”. There is simply no additional analysis required other than this axiom. When standards exist to serve your clients with distinction – such standards may be relied on.

Comments are closed.