The Open Sourcerer

Today’s the day.

At midnight tonight¹ the voting on the approval or not of Microsoft’s draft document specification DIS29500 will be closed. There has been more written about this process than any other standardisation in living memory.

With that in mind and irrespective of the outcome of this vote – be it pass or fail – I though I’d jot down some of my thoughts on what the repercussions may be.

• The ISO has lost all credibility. The way this particular specification’s route through ISO has been scrutinised has shown up massive flaws in the process itself; and this is the body which, rather ironically, gives us the ISO9000 series of procedural standards don’t forget. Basically it has been shown that anyone with enough money can probably, if DIS29500 gets approved, simply buy a standard.
• Microsoft have shown themselves to be more greedy, arrogant and devious than I and many others ever thought possible. However, the repercussions have only helped to make them a laughing stock and their badly written specification a laughing stock. And this whole fiasco has probably initiated their long decent into irrelevance. By attempting to force OOXML through in the way they have done, they have done yet more damage to their already tattered public perception.
• Microsoft’s puppet and henchman ECMA will feel the heat from this fiasco more than they ever anticipated I expect. If the ISO are to come out of this whole charade with any credibility at all, they will have to distance themselves from ECMA by many light years.
• The European Commission has already got it’s teeth into Microsoft’s ankles. If DIS29500 passes, then I expect they will increase their activities, aim a bit higher, and probably find many more skeletons in Microsoft’s cupboard. Which will no-doubt end in further fines, bad press and – hopefully – “encouragement” to all EU member states to avoid choosing Microsoft wherever possible.
• In the commercial sector, nothing much will change whatever the outcome of today. The tremendous growth of Open Source in the Enterprise will certainly continue and I hope spread further down the food chain into the SME/B sector, increasingly eroding Microsoft’s significance.
• On the downside, I expect Microsoft’s stuffed committees to use their position to stifle and stall development of ODF. By abstaining or voting against developments they will attempt to thwart ODF’s progress. There needs to be a concerted effort in the committees that are affected to either fill the “other-side” with friendly members or to remove Microsoft’s cronies.

That’s it. We’ll probably know the result in a day or two. I really hope that DIS29500 is rejected, but I suspect that it will pass, albeit with a small margin. If it does pass, I guess it will probably end up being worse for Microsoft in the long run. They will be chased even harder by the EU Commission, democratic governments will be scrutinised and tested at every turn to prove conformance of their IT choices with the specification – which even Microsoft can’t achieve – and the rise of free and open software will continue, probably at an enhanced pace due to the large numbers of people who have been “awoken” by this farce. Whatever the outcome just remember:

The Way Out Is Open!

¹That’s assuming they don’t change the rules yet again.

Trussst in Microssssoft

I am a man of many business cards like many small entrepreneurs. I have cards with my name on for The Open Learning Centre, Fondoo.net, Dominux Consulting and my personal favorite card lists me as ‘Chief Story Teller’ for A Story For Bedtime. One thing in common with all of these cards is that you can trust that I actually own or work for all of these ventures.

I have not got any cards which portray me as a Vice President of IASA Malaysia though. Doug Mahugh of Microsoft has:

I wonder how often he visits the office? Maybe he telecommutes?

Kaa was drawn by the talented muralist Dan Hewer.

OOXML: Join in the Bug hunt

Let’s join in the bug hunt [Updated: See below]

Rob Weir has produced another little gem of an analysis: to do a reasonably scientific search for errors and bugs in the OOXML specification. The idea is to see just how many errors were really caught in the original 5 month review process for the 6045 page specification, and how effective the BRM, held in Geneva a couple of weeks ago, was at fixing them.

His initial findings do not make very comforting reading…

… I’m not done with this study yet. I’m finding so many defects that recording them is slowing me down considerably. But since this is topical, I will list what I have found so far, based on the first 25 random pages, or 1/8th completion of my target 200. I’ve found 64 technical flaws. None of the 64 flaws were addressed by the BRM. Among the defects are some rather serious ones such as:

• storage of plain text passwords in database connection strings
• Undefined mappings between CSS and DrawingML
• Errors in XML Schema definitions
• Dependencies of proprietary Microsoft Internet Explorer features
• Spreadsheet functions that break with non-Latin characters
• Dependencies on Microsoft OLE method calls
• Numerous undefined terms and features

… this doesn’t look good, does it? Not only am I finding numerous errors, these errors appear to be new ones, ones not detected by the NB 5-month review, and as such were not addressed in Geneva. Since I have not come across any error that actually was fixed at the BRM, the current estimate of the defect removal effectiveness of the Fast Track process is < 1/64 or 1.5%. That is the upper bounds. Of course, this value will need to be adjusted as my study continues. However, it is starting to look like the Fast Track review was very shallow and detected only a small percentage of the errors in the DIS.

If you fancy helping Rob and the rest of the free world, he lists the page numbers (chosen at random) from part 4 of DIS29500 that should be examined in detail for errors and such like. Here’s just a few of the page numbers (out of 200) to check:

… 1102, 1611, 3016, 2646, 3083, 5105, 747, 1142, 2596, 845, 626, 4047, 1415, 5143, 3997

The fact that in examining just the first few pages he finds numerous NEW errors indicates yet again, that this specification should never have been Fast Tracked in the first place and is just simply not in a fit state to be declared an ISO standard.

It really does make me wonder how certain National Bodies can be even remotely sincere when they decide to approve such a badly written specification.

[Update: In the few scant hours since Rob’s article was originally published, go and check the comments section! Even if you aren’t a software engineer, the errors and inconsistencies being reported are simply mind boggling. How on earth can this specification be approved as an international standard when it is just so bad?]

OOXML is REALLY BAD (But then we already knew that didn’t we?)

I know there has been a great deal of written commentary about why OOXML should or should not become an ISO standard. Just look back though this blog for many of them 😉

Recently however, Rob Weir has documented an incredibly simple, telling and fascinatingly descriptive demonstration of exactly why OOXML is such utter crap and should never become a standard in its current format. He shows how a simple formatting rule looks when saved using OOXML or ODF* based applications. See Rob’s table and comments below:

* ODF is the ISO approved standard for document formats and is used by many platform independent applications such as OpenOffice.org and KOffice.

… let’s take a look at how OOXML and ODF represent a staple of document formats: text color and alignment. I created six documents: word processor, spreadsheet and presentation graphics, in OOXML and ODF formats. In each case I entered one simple string “This is red text”. In each case I made the word “red” red, and right aligned the entire string. The following table shows the representation of this formatting instruction in OOXML and ODF, for each of the three application types:

Format	Text Color	Text Alignment
OOXML Text	<w:color w:val=”FF0000″/>	<w:jc w:val=”right”/>
OOXML Sheet	<color rgb=”FFFF0000″/>	<alignment horizontal=”right”/>
OOXML Presentation	<a:srgbClr val=”FF0000″/>	<a:pPr algn=”r”/>
ODF Text	<style:text-properties fo:color=”#FF0000″/>	<style:paragraph-properties fo:text-align=”end” />
ODF Sheet	<style:text-properties fo:color=”#FF0000″/>	<style:paragraph-properties fo:text-align=”end”/>
ODF Presentation	<style:text-properties fo:color=”#FF0000″/>	<style:paragraph-properties fo:text-align=”end”/>

The results speak for themselves.

What is the engineering justification for this horror? I have no doubt that this accurately reflects the internals of Microsoft Office, and shows how these three applications have been developed by three different, isolated teams. But is this a suitable foundation for an International Standard? Does this represent a reasonable engineering judgment? ODF uses the W3C’s XSL-FO vocabulary for text styling, and uses this vocabulary consistently. OOXML’s representation, on the other hand, appear incompatible with any deliberate design methodology.

I fear that before we can tackle harmonization of ODF and OOXML, we will first need to harmonize OOXML with itself!

I love how convoluted and completely indecipherable the OOXML representations are; and they are all different depending on which application you use! It’s laughable really.

Honestly, how on earth can the US NB, for example, have just announced their decision to vote yes to OOXML unless they have been thoroughly corrupted – as have so many other of the ISO National bodies and sub-committees involved in this whole sorry saga.

To my mind there will be two losers if OOXML becomes IS-29500:

• Us – That’s all of us as consumers and users of electronic documents
• ISO – They have already lost a great deal of respect and credibility. If OOXML passes they will have none left. They will become an irrelevance in technology standards at least.

I can see the IETF (The body responsible for much of what has made the Internet work) becoming a far more important standards setter going forward…

On the OOXML BRM (#2)

There is a new report on the events of last week’s BRM in Geneva from one of the delegates. It is articulate and, I can only assume by the language and tone, highly accurate too.

Rob Weir’s erudite description of the proceedings, and the results of having to process a stupid number of issues about a badly written specification, in 5 working days absolutely beggars belief. Please go and read it in full.

We are in the 21st Century. Surely we have better brains and know-how than this?

… As the meeting progressed into Thursday, the tension mounted. As new issues were identified, they were taken off-line and told they could be brought up “Friday morning”. But no one really believed that. It was clear that there was not enough “Friday morning” to go around.

Thursday 9:20am, a delegation objects that they were told only to review Ecma’s responses to their own comments, and that there was never sufficient time to review all 1,000 Ecma responses since January 14th. ITTF’s response: “Nothing we can do about it in the rules — Nothing we could have done in our judgement”.

2:18pm the Convenor announces “This is zero hour”.

There is clearly not even enough time to fully discuss in the meeting the resolution of items that were taken off-line for further discussion. The US is not allowed to present our multi-part proposals to the meeting. We are told get consensus outside of the meeting first, so it can be brought up for quick approval.

Into Friday the BRM spirals further downwards. The issue is not now that NB’s cannot raise new issues. The problem is now that NB’s who have been diligently working on issues off-line with other delegations, meeting over lunch, or early in the morning or into the evening, may not be able to have their proposals heard and acted on.

There simply is not enough time. The anxiety-driven, frantic delegates push even harder. More resolutions are approved with 2 or 3 delegations trying to raise objections, but without being recognized. Tempers grow short. One highly respected Head of Delegation, of unimpeachable reputation and experience started to voice an objection “I am extremely disgusted by the way procedures have been…” before being called out of order by the Convenor, saying that discussion of procedural issues will not be allowed. Another delegation tries to raise a new issue, as they had for the last two days without luck. “We’re using the public money from NNN to come here to speak on our issue. Can we speak on our issue?” Convenor – “We have run out of time.”

And so the BRM came to an end, with the announcement of the results of the paper ballot. Four delegations gave blanket approval to every Ecma comment (Cote D’Ivoire, Czech Republic, Finland, Norway) and three gave default disapproval positions on all undiscussed Ecma responses (India, Malaysia, United States). Most delegations gave a default abstain position, or registered no position. The net is that, although the discussions on Monday and Tuesday demonstrated that the quality of the Ecma responses was such that almost every one required substantial off-line work to make it acceptable, we gradually lowered our standards, so that by week’s end, we approved 800+ comments without any discussion, even in the presence of clear objections.

I want to make it clear that I in no way wish to criticize the Convenor. I think Alex did a remarkable job in trying to carry out his duties and be fair in this no-win situation. He was given an impossible task and had to find out how to fail in the least offensive way. There is an art to crash-landing an airplane and we must acknowledge that.

Here’s what I think should happen:

• The ISO/IEC should be ashamed. They should apologise and instigate an immediate review of procedures and suspend all further activity until they stop this kind of fiasco from happening again.
• Microsoft should be hung-drawn and quartered. Perhaps the EU will see to that in due course.
• ECMA should be banned from submitting ANYTHING to ISO/IEC ever again.
• ECMA-376 (DIS29500, OOXML) should be thrown out today.
• Microsoft/ECMA should refund all delegations their expenses for this total farce.

On the OOXML BRM

Well, that’s it. The BRM is over.

We are starting to get some reports about what happened and what sort of outcome is to be expected.

Let’s start with Brian Jones of Microsoft. Here’s his take on what when on.

Well, the BRM is over and I can only describe the week as a lot of technical work and a lot of great people I was lucky enough to meet and exchange ideas with. The objective of the BRM was to work with all of the National Body delegations in the room and improve the specification on a technical level — and that we did. There were many technical changes the delegates made to really get consensus on some of the more challenging issues, but all of these passed overwhelmingly once they were updated. The process really worked (it was very cool).

The meeting closed with clapping and cheering, folks were really happy about the improved proposals for the specification and it was a very positive experience for me personally.

Sounds like a good result doesn’t it? And Here’s another Microsoft chappy obviously having a good time in Geneva… Not sure about how much work he’s been doing although his masseuse looks nice!

So, one might assume from the above that all is well in the land of ISO and ECMA and the future is bright for the OOXML specification.

But wait, lets see what some other people thought about it:

Andy Updegrove has collated a number of sources of information together and has a quite confusing analysis of what the results will actually mean. It certainly doesn’t sound like Brian’s comment above:

… but all of these passed overwhelmingly once they were updated. The process really worked (it was very cool).

Here’s Andy’s post and here’s an extract:

There are two ways in which you may hear the results of the BRM summarised by those that issue statements and press releases in the days to come. Perhaps inevitably, they are diametrically opposed, as has so often happened in the ODF – OOXML saga to date. Those results are as follows:

98.4% of the OOXML Proposed Dispositions were approved by a three to two majority at the BRM, validating OOXML

The OOXML Proposed Dispositions OOXML were overwhelmingly rejected by the delegations in attendance at the BRM, indicating the inability of OOXML to be adequately addressed within the “Fast Track” process

Interesting. I suggest you read the whole piece to find out why the second of the two results is closer to the mark.

We also have some comments from members of the Malaysian delegation here where their headline states:

“BRM in Geneva is over: big failure for OOXML”

They go on to say:

As you might have guessed, the five day meeting failed to properly address the huge amount of comments and proposed dispositions, and a rushed vote on Friday tried to lump together all unresolved issues in a package where the ECMA dispositions were to be voted on without any discussion. Needless to say, that failed miserably. Only ten national delegations voted, and only 4 P-members were for approval. 4 P-members disapproved, a whopping 15 abstained, and 2 even refused to register a vote in protest.

Although as one would expect they conclude that:

If you count all voting delegates, including those who are not P members, the vote was 6 approvals, 4 disapprovals, 18 abstentions and 4 refusals to vote. Expect this to be announced by Microsoft as a “3 to 2 majority for OOXML approval” in the next few hours. The reality is of course that this is a huge setback for Microsoft. The tricks they have been trying have backfired, and it is now more clear than ever before that OOXML is an immature specification which was totally inappropriate for the fast track procedure.

Hmmm, this doesn’t sound like the report from Brian Jones at the beginning does it?

I’d like to finish with some words from Tim Bray, who was a member of the Canadian delegation. On his blog post he is clearly not enthralled by the experienced:

The process was complete, utter, unadulterated bullshit. I’m not an ISO expert, but whatever their “Fast Track” process was designed for, it sure wasn’t this. You just can’t revise six thousand pages of deeply complex specification-ware in the time that was provided for the process. That’s true whether you’re talking about the months between the vote and when the Responses were available, the weeks between the Responses’ arrival and the BRM, or the hours in the BRM room.

And in scathing language, Tim goes on:

This was horrible, egregious, process abuse and ISO should hang their heads in shame for allowing it to happen. Their reputation, in my eyes, is in tatters. My opinion of ECMA was already very negative; this hasn’t improved it, and if ISO doesn’t figure out away to detach this toxic leech, this kind of abuse is going to happen again and again.

Toxic Leech. I like that! That’s a great expression.

There is some excellent advice from Tim too about what the NBs might like to do with regards to their final decision:

The national bodies that voted on the first round have thirty days to decide if they want to change their vote. I totally don’t believe that ECMA/Microsoft is going to be able to pull together a revised draft of this Frankenstein’s monster in that timeframe. That seems like a pretty serious process issue to me, too.

In practise this means that the heavy politics starts Monday morning. National bodies that are smart will make their decision between 8:30 and 9:00 AM on March 3rd and immediately go on long vacations in Tasmania or Nunavut.

So, who would you tend to believe? A convicted monopolist that has just been fined $1.4bn for ignoring the EU’s judgement of 2004. A business which has been found to be paying NGO’s in India to write to their Government to support OOXML, and providing them with the form letters too!

Or, the rest of the world? I know where my hat rests. Do you? Think about it next time you need to buy some software. Really, Really, Think About It…