Agder University
13-September-2007
Translations
Summary
Discussion of EIF 2.0 might have been expected to be dry and technical. But it becomes contentious and political when a vendor attempts to skew the process to their own benefit. Standards groups are so used to achieving unanimous technical consensus that their processes are challenged to handle split votes and political battles.
Gartner's advice to IDABC is so fundamentally flawed that, if followed, it would break down the interoperability that has been achieved via EIF 1.0 and set back any prospect of achieving improved interoperability in the future. Their findings are unbalanced to weight the desires of an IT vendor over the good of IT customers and the fundamental goal of interoperability. This unbalance is so fundamental that Gartner mis-states the very character of standards and the conditions that provide interoperability, and confuses mere formats with standards.
This comment counters Gartner's report with a simple explanation regarding the conditions necessary to achieve interoperability while being fair to all parties. The recent overpowered push for acceptance of Office Open XML in ISO made it clear that the proper conditions for interoperability must be explained to many nontechnical people, if political pressure is not to overwhelm technical reality. Thus, this comment defines basic concepts and uses terms that nontechnical people can be expected to understand. The more technical are requested to bear with us.
Preface
This is a comment upon Preparation for Update European Interoperability Framework 2.0 - FINAL REPORT produced by the Gartner consulting firm and referred to as the "Gartner report" in this document.
IDABC solicited reactions to the Gartner report through their web announcement at http://ec.europa.eu/idabc/en/document/6227
About the Author
Bruce Perens is a co-founder of the Open Source movement in software. He is author of Open Standards: Principles and Practice the most cited definition of Open Standards. He previously served on the World Wide Web consortium Patent Policy Working Group helping to produce W3C's current policy on patents in web standards. You may read more about him at http://perens.com/about/bio/
An Introduction To Interoperability For Non-Technical People.
The technical topic of interoperability has suddenly become crucial to democracy. We vote based on the information that we hear, see, and read, and increasingly we are getting that information through computer networks. Democracy depends on voters having multiple sources of political discourse so that they can judge between them and vote well. If one party controlled the tools that we use to communicate, the computer programs and the way those programs talk with each other, they'd be able control the information we hear, see, and read too, and that would let them control our votes. They'd also be a monopoly, and of course that's bad for the economy. To keep such bad things from happening, we insist that there be many sources of the tools that we use to communicate. But we need all of those tools to talk with each other in a way that all sides understand, and that's called interoperation.
With computer software, the "languages" used to interoperate are the file formats used to store files on a computer disk and to exchange data over a network, and the intercommunication protocols that computer programs use to talk to each other without the use of a file. File formats and intercommunication protocols are written and read by computer programs. That's how they communicate.
This isn't new. A century ago, nations agreed on the first standard for electronic data communications, the International Morse Code. Even if the telegraph operators did not understand each other's language, they both were taught that a short tap on the telegraph key meant a letter "E", and they knew what part of the message was the address. Thus the telegraph operators worldwide could interoperate.
A basic requirement of interoperability is that the producers of computer programs must have them write information that other programs will be able to read. The formats and protocols that programs read and write become Standards when we define them very precisely in writing, and when all of the interested parties come to an agreement that the definition is correct and usable to work with each other.
Open Standards are the best kind of standards. Their definitions are published and are available for anyone to read, and there are no legal restrictions or monetary requirements that would prevent anyone from using them. Thus, they are democratic: everyone can use them. Their development is also generally democratic: everyone can be involved.
The opposite of Open Standards is Closed Standards. These are encumbered by one or more form of restriction: trade secret, a patent royalty, overly restrictive or discriminatory licensing, a non-disclosure agreement, closed membership on the standards definition committee. They generally work to lock some parties out. Often it's Open Source that is locked out, because the terms are not compatible with the distribution of Open Source software to all parties without charge.
Some things that you might think of as standards aren't standards at all.
One example is the ".doc" file format used by
Microsoft Word,
which is a
prorpietary file format
rather than a standard.
Its definition is private to Microsoft and has not been published,
so of course no formal standards body has been involved with it.
Many software developers have
reverse-engineered
it: they've figured out how it works, more or less, through careful
examination without any help from Microsoft. But there's no document
that says what you can and can't write in this sort of file, and it
changes with every new version of
Microsoft Word,
so the interoperability that others can offer to this file format
is less than perfect.
Open Standards do not mean Open Source. Open Source is a way that teams cooperate to produce computer programs very effectively by sharing the rights to each other's innovation. The opposite of Open Source is proprietary software, which does a lot less sharing. Both Open Source and proprietary software are important today, and EIF 2.0 must be fair to both.
Where Gartner Went Wrong
Gartner, in their report to IDABC, shockingly criticizes EIF 1.0 for its dogmatic focus on open standards and for prescribing detailed technical standards at all. But such is necessary for interoperability. Gartner's proposed alternative is a support of "multiple standards" for the same task, and without the specification of any particular standard.
Gartner's proposal, if we followed it, would create a tower of Babel of non-interoperable programs. Imagine a meeting where everybody speaks two languages, but some folks speak French and English while others speak Spanish and German. They wouldn't be able to "interoperate" with each other.
Everybody can get dictionaries for Spanish, English, French, and German, and nobody charges a tax just for speaking those languages. They're sort of like Open Standards. But imagine further that a large and dominant tribe of people spoke "LanguageSoft", and the owners of LanguageSoft made barriers to keep other folks from speaking it so that those folks would not be able to compete as well. They might keep parts of LanguageSoft secret, they might charge a lot for the right to speak it, or they might just make it deliberately more difficult to speak than it should be by making the only book on LanguageSoft unnecessarily long. Then, LanguageSoft would not work like an Open Standard.
That's the kind of world that Gartner is suggesting we have.
Why does Gartner say we should do this? Their statement is
EIF v2.0 should facilitate the most profitable business model(s) of cost versus public value, under proper recognition of intellectual property rights, if any.Well, everybody's for profit. But whose profit are they talking about? It seems to be the profit of the IT vendor, rather than the user.
A basic tenet of economics is that competition drives prices down and quality up. Insisting on Open Standards is the best structure for encouraging competition, because all computer program makers have the right to use Open Standards. By giving the right to compete to everyone, the use of Open Standards creates an environment with the most possible competitors and thus the most efficient economics. By establishing a particular Open Standard that will work across all software that we buy for a particular function, we further encourage competition by simplifying our requirements and making the software developer's job easier. But economically efficient doesn't seem to be what Gartner means by profitable, since they are specifically asking for a de-emphasis of a requirement for open standards at all and an avoidance of designation of any particular standard as being required.
It's confusing that Gartner uses the word profitable in this context, since one of the main targets of EIF is government, and government's goal is not to make a profit but to deliver services to its citizens in a fair and equitable manner while keeping the tax cost low. Gartner's use of profit is equally confusing if their target is business IT users, the other target of EIF, since IT is a cost-center rather than a profit-center except in those rare businesses that happen to sell IT software. So, it must be the IT vendor's profit they're talking about.
Gartner adds
...under proper recognition of intellectual property rights, if any.In line with their abuse of Open Standards as dogmatic, Gartner wants EIF 2.0 to allow closed standards that will be locked up by intellectual property restrictions. Intellectual property restrictions are a favorite way of locking out competition, in particular Open Source competition.
Do we really need intellectual property restrictions on the standards that we use to interoperate? Some proprietary software vendors say yes, that they must protect every bit of their intellectual property, no matter what it's for. But there is nothing new or innovative about the way that programs read and write files and the way they talk to each other. Very many common ways of communicating like TCP/IP, XML, HTML, and OpenDocument are available as Open Standards for everybody to use in their software without any intellectual property restriction. Nobody has to make very much new intellectual property at all just for two programs to be able to exchange data with each other, whatever kind of data that is.
Thus, it's difficult to believe that a file format or an intercommunication protocol has to be considered to be intellectual property for any purpose but one: to lock other folks out. Why let competition in deliberately? Instead of using Open Standards, vendors who want to lock in their customers use closed and proprietary file formats. Sometimes they make a great show of licensing their specificiation to others, but selectively and with terms that not all software developers can meet. They create "interoperability labs" to work with a few favored parties while keeping others out. If these vendors were sincere about wanting to interoperate, they would simply publish full information defining their file formats online, without restrictions or fees, like most Open Standards today. Then, anyone who wanted to build an interoperable program could use that information to do so, without their help.
Some proprietary software manufacturers say that we should allow them to lock out other software producers and Open Source this way. They say this is the way that vendors maintain a competitive advantage. As the purchasers of software, there are some ways that we should encourage our vendors to compete, and some that we should discourage. Surely we should encourage them to add new features, to make their software faster and smarter and more capable. But locking other folks out to drive their own sales? That hurts us and the whole economy because it reduces competition without adding anything that the customer values - only gratuitous incompatibility.
Gartner tells us that we can
Facilitate evolution and avoid vendor lock-in by supporting multiple standards.But this mis-states the cause of vendor lock-in. Vendor lock-in happens when a vendor uses deliberate incompatibility to force customers to only use their brand of software. Multiple standards do not fight vendor lock-in because they do nothing to assure that every vendor's program can talk with every other vendor's program in a way that all parties understand. Indeed, having two standards for the same purpose would invariably create a situation in which some programs use one and not the other, and thus can not talk with each other.
We can avoid vendor lock-in if we insist on having Open Standards in the software we buy, because all vendors can use Open Standards. We can make sure they all use the same Open Standard, and thus are interoperable, by designating what standard that should be in EIF 2.0. Certainly the vendors and Open Source teams should all be part of the conversation while the selection of what standard to use is made.
What about facilitating evolution? That's an important point. Every software producer should be able to make their program smarter, faster, and give it more features. We should let them do that to every part of their program without restriction, except for the file formats and intercommunication protocols. We need those parts to be Open Standards so that we can have a level playing field for all competitors. But what if innovation can't wait the years necessary for the definition of a new Open Standard? That's OK: all we need is for the vendor to publish their change to the file format so that everybody can read and write it the same way, with no royalties or discriminatory licensing. As long as that sort of public definition is around for other programmers to use to make compatible programs, we will still have a competitive market in which everybody can produce interoperable software. We need to watch out for vendors who abuse this process, however. Some of them will throw in changes just to keep others from catching up and making a compatible program. Changes that we accept must create some real value for the customer.
It's really important for us to remember that the users pay for proprietary software. The vendors make it and sell it to us, but it's our money that makes it possible. We have a right to ask those vendors for certain things that make the market fair, like use of Open Standards, so that the vendors won't take advantage of us with things like lock-in. But EIF 2.0 should allow proprietary vendors to keep secret and proprietary the parts that we don't need to see, everything that's not a file format or an intercommunication protocol. That doesn't hurt interoperability.
There are some situations where source code should be visible to all for the protection of the public. It's important for the public to be able to trust the software in voting machines, and software in devices that can harm life or property if they malfunction. Public disclosure of source-code for such critical systems may be the best way to establish that trust. But this is outside of the scope of EIF 2.0, and belongs in other law.
Is There No Place for Multiple Standards?
There is a great place for the multiple standards that Gartner recommends: reading them. Producers of programs should be welcome to make them read as many different kinds of communication as possible. It's only the files that we write that can be a problem. In our offices, we should have our programs write something that everybody else can read, now and in the future. That means an Open Standard. And we should select one Open Standard for a particular purpose in cooperation with software producers, so that all software producers know that's the one they absolutely need to read and write. Selecting which one that is must be an open process, where everybody can participate, and that is fair to both proprietary software and Open Source.
What Did Gartner Do That Made Their Advice So Wrong?
Gartner interviewed some very large IT vendors who happen to be among the largest customers of Gartner's analysis business. In that business, Gartner has both IT vendors and IT customers as their clients, and writes reports making recommendations to IT customers indicating which vendors should be preferred. An IT vendor who is Gartner's customer is assured that Gartner will write something about that vendor in Gartner's reports to IT customers. Thus, large IT vendors represent a significant profit center for Gartner, and Gartner must listen to them. Gartner gave the desires of these vendors more weight than they should have in producing their conclusion on EIF 2.0. They state the vendors sentiment in their report on the industry representatives workshop:
The issue of open standards is a controversial topic among the industry representatives. Their opinion is that the framework should allow competition among standards, open and non-open.
Think about software from the vendor's perspective: wouldn't every vendor want to have no competition, so that they could have really high profit margins and make tons of money very easily? Wouldn't they rather avoid having to constantly make sure that their program's better than everyone else's? Some large IT vendors had just that situation for a decade or two, and got really used to it, so much so that they now believe it's their right.
Those vendors told Gartner what they'd need to keep oversized margins and reduce competition: admit closed standards to the mix, reducing competition between IT vendors by locking some of them out of the capability to interoperate. Lock customers into closed file formats by removing any requirements that applications support any specific open one. Call this a competition between open and closed standards to cloak the fact that it would reduce competition between vendors.
In its attempt to cope with the vendor desire for closed standards and the reality that Open Standards are right for the job, Gartner contradicts itself repeatedly. They promote Open Standards on one hand:
IT vendors and system integrators should also recognize that open standards are the way to go.and then they attempt to shelve the issue to be handled at some fictional future date:
The support for multiple standards allows a migration towards open standards when appropriate in the long run.The reality is that Open Standards are available to use today, and are the easiest alternative for a vendor to implement unless customer lock-in is the goal. There is no reason for customers to accept second best and then "migrate" later.
In the IT business there's a saying about what vendors want: "Every kid wants a pony." It means that vendors will want more than is reasonable for us, the customer, to give to them. For our own good, we need those vendors to have a piece of the pie rather than the whole pie, with reasonable but not huge profit margins, and lots of competition to keep them on their toes. Open Standards and the interoperability that results from them are our main tool in balancing what our vendors want against what is good for us, the customer.
One particular vendor, Microsoft, has multiple standards as a goal at the moment: they have an office suite software file format, Office Open XML, that they'd like us to accept instead of an existing standard: OpenDocument. I believe that goes far toward explaining the emphasis on multiple standards in Gartner's report.
Office Open XML is promoted as an Open Standard, but the definition of it is so long, at 6000 pages, that a programmer could not learn the whole thing in his or her useful lifetime. But that's OK, Microsoft says, because all of the features in Office Open XML are optional. Of course, if everybody has to pick and choose what features their software will understand, they'll all pick different ones and they won't be able to interoperate with each other. The competing OpenDocument standard has a definition only 600 pages long, still no vacation paperback but practical for one person to learn. It is compact enough that features need not be made optional. In contrast, Office Open XML seems to be Microsoft's bid to retain its monopoly while appearing to be more open. With the outcome of the recent ITU vote, Microsoft should resolve that it has lost this battle and should cooperate in adding the features that it needs to the next version of OpenDocument.
Conclusion: What Should We Do?
First and most important, we must acknowledge that as customers we pay for the creation of software. We, the users, are in the driver's seat. We have a right to ask for our vendors to function in a way that will create a level playing field upon which all software makers can compete, with as much competition as possible so that we can have lower prices and higher quality. Our vendors should make money if their work is good and we buy a lot of it. They don't have a right to demand a guaranteed income of us.
We should insist that Open Standards are used for file formats and intercommunication protocols in the software that we buy. EIF 2.0 should be the framework by which we come to agreement on which Open Standards those will be, for each of the various purposes of the software that we buy. We should encourage that the programs we buy read the one mandatory Open Standard required of them, and write it when configured with their default settings. Those programs can read as many additional formats as the developer wishes to incorporate.
When an addition to the material covered by an Open Standard is necessary, we don't need to require that manufacturers wait for years to have their new feature accepted by a standards body. We need only insist that extensions to file formats and intercommunication protocols be necessary rather than gratuitous, and that the information on these extensions be made available for every software creator to implement, without any royalty or discriminatory licensing. These changes can then be incorporated into formal Open Standards as they evolve.
EIF 3.0 and later versions can specify requirements for new standards as they evolve, and define a migration path from older software to new while maintaining openness and interoperability at all times. This may include a schedule for phasing out of requirements for older standards when necessary. However, the very openness of the standards will assure that they can continue to be used by old-to-new format conversion software and by software producers that wish to maintain additional compatibility in their applications.
We don't need to insist on Open Source or establish a preference for Open Source. We should, however, establish a level playing field upon which Open Source or proprietary software can compete with each other fairly, based on both the technical merits of the software and the merits of non-software features such as the potential for software to be redistributed or modified because of permissible copyright licensing, the availability of multiple service vendors, and the viability of the development community or company producing the software. Open Source should be considered in any software acquisition, and used if it has sufficient merit.
Finally, IDABC should be more careful in the future regarding the vendor it contracts to produce independent and unbiased reports. Gartner's extremely lucrative relationship with IT vendors should have disqualified them.
The Author's Sponsorship
Bruce Perens' work in general is sponsored by the entities listed below. However, he maintains his intellectual independence and integrity: none of these funding entities have the right to influence what he writes in a report like this or even whether he chooses to write a report like this or not.
Agder University
Bruce Perens has a grant to from the Competence Fund of Western Norway that supports him to be a visiting researcher at Agder University in Southwestern Norway for part of the year. Through them he regularly advises several Norwegian government agencies. As is usual for researchers at universities, the Competence Fund and Agder University do not attempt to influence the opinions that Perens would render in documents like this. Perens chose to spend part of his summer at Agder this year working on this document.
Action on Technology Policy
Action on Technology Policy is the home of Bruce Perens' public policy activities, and has had more than 3000 individuals sign on to its initiatives. The topic areas for the organization are stated in the Political Policy Areas section on Perens' web site.
Perens LLC
Perens LLC is Bruce Perens' business.
Image Credits
The cover image is The Confusion of Tongues by Gustave Doré, after the Biblical story of the tower of Babel. That image is in the public domain, as its copyright has expired. This version was downloaded from the Wikimedia Commons at http://commons.wikimedia.org/wiki/Image:Confusion_of_Tongues.png
