Blood & Guts: News From The Arena
Some Odds And Ends From The File Formats Arena
Does anyone else feel like they are in one of the old Roman gladiator arenas? On one side, we fight to destroy monopoly and lack of choice. On the other side, they fight against a single standard that would end the monopoly. Whether we favor a single standard or a single competitor, the battle cry comes from the old Highlander series. There can only be one!
Since I first heard of him, Rick's been kind of a thorn in the side of those who believe in truly vendor-neutral standards. In fact, he seems to take every opportunity to try to prove that no sane person could possibly expect Microsoft's current user base to accept a file format that is meant to be freely usable by any vendor. It is, to read his writings, eminently reasonable to perpetuate the date error long into the future. It is, he says, perfectly acceptable for an XML-based file format (which are supposed to be human-readable) to use bit flags. Anyone who disagrees is merely spouting Groklaw's FUD.
One other aspect to vendor-neutrality needs to be intellectual safe-harbour provisions. You cannot implement an “open” standard if doing so opens you to lawsuits. This was true in the recent RAMBUS fiasco, and there are many who fear Microsoft’s patent portfolio hidden behind “Open” XML. The recent sabre-rattling on the patent front seems to confirm this fear.
William: Is anti-MS FUD really the only possible response in any situation? Sentence 1, sounds good. But then sentence 2, raises fear: LAWSUITS! Sentence 3 raises uncertainty: HIDDEN PATENTS. Sentence 4 raises doubt: SEEMS TO CONFIRM THIS FEAR. A by-the-book play, all without a shred of evidence.
The thing is, any reasonable person who listens to what is spouted in Redmond, that is EXACTLY what you hear. "We have X number of patents that we won't disclose, but you guys are violating them and we want money for them." And in the next sentence, "But we've pledged not to sue under certain circumstances… ." Gee, I wonder why no one is convinced? "… without a shred of evidence", he says. Perhaps he should head over to the Computerworld site (or nearly any other major tech site) and read for himself what the company's leadership has been saying.
Rick does have one thing I agree with:
The issue of ODF versus Open XML will not go away, because they both have different drivers: the issue is not “open standards” but how to have vendor-neutral interchange formats that also allow good enough vendor-specific functionality to be used as native formats without embrace-and-extend, and most critically how to verify that the formats are not just open but also fair, participatory and non-discriminatory.
Those drivers are (one one side) the ability to transparently exchange the software used to create the document and the ability to have access to those files at some undetermined time in the future when the original vendor may be out of business or moved on to a different business. On the other side, the drivers are fear (of losing the monopoly) and greed (if everyone continues to use a single vendor for their office suite software, then it is relatively simple to build hooks into that software to make it work better with the vendor's server software).
Rob points out that Not-so-open XML (OOXML) advocates claim that that format's formula specification is good enough to use in implementing the standard. He then examines some mathematical functions to show that they are not actually specified enough for an independent implementer to use.
Over 300 pages devoted to spreadsheet formulas, and they fail to say what basis (radians or degrees) they use for trigonometric functions? Unbelievable!
Another example. Several of the statistical functions in OOXML are defined incorrectly. Take for example, the ZTEST function (Part 4, Section 18.104.22.1682). The key error is following the formula where it says, “where x is the sample mean.” The problem is that x-bar is the sample mean, not x. Someone who implements according to the text will give their users the wrong answer. A similar error is repeated in 8 other statistical functions. Certainly this is a typographical error, but this error changes the answer. Remember, this is a proposed International Standard not a 4th grade school essay. Spelling counts. Providing the right formula and the right description counts. Copy and paste errors should have been taken care of back during the Ecma review.
As I’ve shown, in the rush to write a 6,000 page standard in less than a year, Ecma dropped the ball. OOXML’s spreadsheet formula is worse than missing. It has incorrect formulas that, if implemented according to this standard, will bring important health, safety and environmental concerns, aside from the obvious financial risks of a spreadsheet that calculates incorrect results. This standard is seriously messed up. Shame on all those who praised and continue to praise the OOXML formula specification without actually reading it.
This article was enough to get Doug Mahugh to respond. Yet Doug does not come clean about the "influence peddling" part. Doug, since you are talking about exercising undue influence on the political process, explain to us how the California Legislature followed Microsoft's wishes while ignoring the wishes of its own citizens? What was promised to the Assembly members that ignored their own constituents?
At least Doug did say he recommends making some changes because of these issues. Because of his employment, he can not say, "You're right. Let's wait until we have a workable and usable specification."
Ben points us to Rob's and Yoon Kit's blog articles.
Ben is an independent vendor that makes products that work with both Microsoft and IBM products. As such, he is probably going to try to implement OOXML functionality in his products along with ODF functionality. When he complains about either format, I pay attention, because he is actually likely to be stuck dealing with the issues raised. Too bad Microsoft and Ecma ignored all of his suggestions for improving OOXML.
YK tells us that specifying formulas for a spreadsheet format is something completely new.
While it is true that ODF v1.0 does not include a “Formula Definition”, it is also true that throughout the entire history of spreadsheet usage, there has been no “Formula Definition.” Although this has hindered interoperability to a certain extent, people got by using whatever information they had available to them.
He shows examples from Excel, OOo Calc, KSpread, and Gnumeric, along with both Not-so-open XML specification and the upcoming OpenFormula addendum to the ODF specification, showing how OOXML is mathematically wrong in its implementation of the ceiling function. This is the kind of thing that will give you a usually-correct-but-occasionally-wrong answer. Just the thing for your timesheet, don't you agree?
So between April 2006 (MSOOXML Draft 1.2) and May 2006 (MSOOXML Draft 1.3), the Ecma TC45 was provided with the Microsoft Office Help Files which was dutifully inserted in the draft standard. It is no wonder then how the technical committee managed to pull off such a feat by defining the “Formula Definition” within 2 months!
He then goes on to sum up why ODF is the interoperability champ.
The OpenFormula document is an important piece of work as it goes out of its way for the sake of interoperability not just between the vendors who chose to be represented in OASIS, but also to the leading de-facto vendor who chose NOT to contribute to ODF. The copious notes on the different behaviours of the different applications will help ensure that developers are aware of implementation pitfalls. It will also ensure that the irregular and legacy implementations are inclusive in the specification, to support the “billions” of documents currently in existence.
This is a significant area which clearly demonstrates that ODF is better built for interoperability for legacy documents yet maintaining mathematical accuracy for FUTURE documents. It also nicely demonstrates the collaborative efforts from multiple vendors to allow OASIS to cherry pick the best features from all contributors to create the best Open Formula yet.
So when it comes to comparing MSOOXML and ODF v1.0 on the basis of the inclusion of “Formula Definitions”, it becomes clear that the anti-ODF folk have not much to shout about. In fact MSOOXML’s “Formula Definition” is deficient and inaccurate. As Rob Weir pointed out, in addition to this CEILING problem, most of the other formulae are inadequately defined and not well tested.
Again, I urge Microsoft and Ecma to sit down with OASIS and specify a fully-open interoperable format based upon ODF with namespaces for vendor-specific extensions. Not that I am eager to have MSFT "embrace and extend" ODF. Instead, I urge MSFT to tell all what is in its secret sauce—this ain't Jack-In-The-Box—for the good of the customer, also called the end user. "spaceLikeWeDidIn1995" is not good enough.