Effective Lobbying For Open Standards

Why?

What Are We Talking About?

While I am an advocate for free software and open source software, this is not what we are discussing here. In the (Tuttle, Oklahoma) City Manager versus CentOs incident, the official stated, (as quoted in the Tuttle Times) “This is just a bunch of freaks out there that don’t have anything better to do,” he said. “When I came in to work Monday morning, I had about 500 e-mails, plus anonymous phone calls from all the geeks out there. [CentOS is] a free operating system that this guy gives away, which tells you how much time he’s got on his hands.”

This highlights the fact that some people consider FLOSS to be subversive or abnormal. Since even those people stand to benefit from implementing open standards in government agencies, we need to advocate the two areas separately.

We are talking about open standards, such as standardized network protocols and file formats. As the group behind Samba has found, if one vendor can monopolize an important protocol, it can hinder its competitors from gaining ground in business and government organizations. Corel’s WordPerfect suite and Novell’s NetWare network operating system, both formerly leaders in their categories, may also have suffered from anti-competitive effects of a single vendor’s formats and protocols becoming de facto standards.

If you ever found software that fit the way you worked, and then found that you could not submit files created by that application for submitting documents or resumes to government agencies, and that any documents released by those agencies are in a foreign proprietary format which your application imperfectly translates, you have experienced proprietary, single-vendor, de facto standards. Such standards take away your freedom as a citizen, selling control of your computer and the software you use on it to some company whose interests may be opposed to your own.

So, then, we should seek to require our government agencies, schools, colleges, and their contractors to utilize standardized formats and protocols, which can be freely and completely implemented by any vendor at any time, are not dependent upon particular software or hardware environments (such as a particular operating system or a particular vendor’s software or libraries), which are not subject to “intellectual property” restrictions, and which are implemented (essentially complete implementations) in multiple products, including some with licenses approved by OSI or FSF.

Why Open Standards?

Why should individuals and organizations, and our government agencies in particular, seek to establish open standards? Why should we specify and require open standards?

A government organization has a responsibility to its citizens, not just citizens that use the leading software products, but to all of the citizens. Since there are hundreds of applications (current and former) just in the office software category, the best way to offer reasonable accommodation of all users’s needs is to utilize open standards. This has another positive effect: it keeps any particular vendor from obtaining a taxpayer-subsidized monopoly. This helps keep vendors humble and responsive.

Requiring open standard formats and protocols also makes it easier for free software and open source software projects to compete for an organization's procurement approval.  This can make it possible to choose OSAF's Chandler, for example, or Mozilla's Thunderbird + Lightning, over Microsoft's Exchange & Outlook, without losing any functionality because of one vendor's products communicating using a secret protocol instead of an open protocol, although it is certainly possible to lose some functionality if the applications themselves do not support it.

Using open standard protocols and formats also helps (indirectly) to constrain employees’ feature usage. Have you ever opened a document where someone had downloaded some funky font which was not available on your system? Your software will often try to guess which font is the best substitute for the one specified in the document. This is where two different computers may choose different fonts. Perhaps the user types a line of text and then formats it (bold, underline, 18pt, centered, with .005 inch of extra space beneath the text) as a section heading, but does not use the application’s “heading” styling tool. This again may not translate correctly for those who are not using the same application and version as the original creator of the file. Or maybe the document uses WordArt and your software implements its own artistic text tool differently. In such cases, the best thing is to avoid using features that do not translate well between different applications and versions, and a sound organizational technology use policy will seek to enforce this.

Private organizations have much less responsibility to those outside of the organization. However, a for-profit organization has a responsibility to its owners and investors. Utilizing open standards frees the organization from being a captive wallet to its vendors. If a particular vendor decides to raise its prices by a large amount, the organization gains the ability to say, “roll it back or we change vendors.” This does not mean that the vendor will always back down, but a truly open standard (I would argue) will have multiple substantially-complete implementations (including at least one with a FLOSS-compatible license, which should generally be the reference implementation), which gives the buyer the ability to select another vendor when that is appropriate and desirable.

Using open standard file formats frees an organization from COTfile disease, which is what happens when someone sends you something and you Cannot Open This file.  It reduces the need for import and export filters, while preserving fidelity.  A large proprietary software vendor recently released an update that disabled access to several legacy file formats for "safety" reasons.  Assuming that they were not just trying to scoot customers into buying the newer version of their product instead of retaining the existing version longer, one may speculate that they directly mapped memory structures to file formats, and are now concerned that the structures they used are vulnerable when certain data is retrieved into them.  A good open standard file format is not optimized for any specific vendor, and therefore is less likely to directly map to an application's in-memory representation.  This gives more opportunity to inspect and clean the data, protecting against flawed data (intentional or otherwise).

Using open standard network protocols means that clients and partners can interact and interoperate with your organization more easily, even where one organization is using a different vendor's products than the other organization is using.  Open standards can reduce or eliminate the need for translation layers

On top of this, there is the gratuitous version change, in which your software vendor decides that its revenue isn’t large enough, so it moves some buttons around in the interface, hides some menu options and displays some previously hidden menu options, and tinkers with the application’s default file format just enough to make sure that users need to upgrade their existing software (which presently meets their needs) or face incompatibility with other users with whom they exchange information.

And of course, the “abandonware” problem. If your organization has been around a few years, somewhere in your organization are probably some files created in 1987 using WordStar for DOS or some other product that has gone to the dustbin of history. Depending upon the application, your present software might not be able to read it (even if you could find a working 5″ floppy drive). If the format was a secret, you might not be able to legally open the file today without finding a way to re-animate the original application if opening could require reverse-engineering. Even if older versions are legally safe to reverse-engineer, versions released since DMCA passed may not be safe. The use of vendor-controlled, proprietary file formats places the organization’s data in jeopardy.

I would encourage you to familiarize yourself with the ideas.  Start at the OASIS OpenDocument Format site, to learn why an open file format matters.  In the light of the push to try to force OOXML through ISO, it is also important to know the difference between an open file format and a proprietary format in sheep's clothing. 

Why Lobby?

Despite the opinion of pro-Microsoft lobbying group ACT’s Morgan Reed, open standards, like open software, have been badly outgunned in the government advocacy department. Last year, a single company defeated open standard file format moves in several states, including California and Florida. Until elected officials feel that their offices are at stake, they will kowtow to proprietary software companies seeking exclusive deals, rather than taking a “good government” stance and seeking vendor-neutral acquisition policies. It starts with writing well-reasoned statements of what is desired and why, such as the one I wrote in support of last year’s California bill AB-1668. If you can’t write well, call or visit the offices of your representatives. They need to know that their constituents are watching to see whether they choose to serve our interests or the interests of a small number of corporations. Or speak to different organizations in your local community, explaining to them why their interests are best served by ensuring that their governments use open standards.

Another reason to lobby and otherwise advocate the use of open standards is that the issue has not yet occurred to most people. People need to know that the reason the World Wide Web is useful is because users do not have to use a single vendor’s software to open most Web sites. Whether users have Internet Explorer, SeaMonkey, Firefox, Opera, Safari, K-Meleon, Galeon, Epiphany, Flock, Songbird, or Konqueror, sites and browsers generally follow a common set of open standards, such as HTML 4.01 and XHTML 1.0.  Once people understand that the Internet is made possible by common, standardized protocols and file formats, it is only a short leap to asking why other software on their computers does not work that way.

Using open standards and protocols also enables the organization to select software vendors and products based on what is best for the organization—which vendors and products best meet the organization's needs, including pricing, access to source code (and ability to distribute changes), availability of patches and updated versions, the responsiveness of the application, the responsiveness of the vendor, availability of knowledgeable local resources to help with any implementation or deployment issues, and the underlying software environment the application expects (e.g., which operating system, which Web browser, which libraries).

Okay, How Should I Lobby?

First let us consider how not to advocate: <productname>rox! u haterz suxorz! might sound great when you visit Slashdot, but in the real world, people will reject even good ideas if the messenger brings them in this manner.  Make sure you are clean and odorless.  Leave your politics at home.  This issue is more important than which politician's friends will be allowed to steal from us for the next four or eight years.

Secondly, focus on the public benefit, not on the activities of any particular enterprise.  Do not make your appeal into an anti-anyone rant.  Not anti-Microsoft, not anti-Oracle, not anti-AutoDesk.  While Lewis Mettler has a theoretical solution to Microsoft's repeated misconduct, that solution is not likely to be approved anytime soon.  Rather than try to tailor something to Microsoft, we want something that will not only deter Microsoft from misusing monopoly power, but something which will also deter whatever company follows Microsoft onto the stage.  We want something that will forever separate data (both as files and as messages sent over a network) from being chained to the application that produced it.

Public benefits are things like vendor independence, government sovereignty, lower cost, and easy accessibility by citizens.  Of course, the archive factor is one of the biggest public benefits.  Many of the records that government agencies generate or collect stay around for decades.  Something that enhances the ability to preserve data and to access and utilize it in the future is definitely a benefit.  An example of this would be moving from a closed proprietary set of file formats to an open standard format such as the ISO-standard format ODF.  Avoid getting railroaded into a discussion of OOXML if you can avoid it—the objective is far more than just keeping our governments from selecting Not-so-open XML—instead specifying that setting the right standards for acceptance is more important than whether any specific format meets those standards.

Discuss privacy and security issues caused by vendors' phone-home software: "Do you know that a certain vendor's operating system and office software products call home and tell the company about the computers they are installed on?  Do you know that the computer's owner cannot turn that off or the software stops working?  We do not know what is getting sent, but these days, just the potential that someone's SSN might be part of this transmission should make us think about how to work without that vendor's software."  Be sure to point out that even the most privacy-sensitive agencies still have this snoopware on their systems, as long as they are trapped into using a single vendor's products, and be sure to mention that moving to open standard file formats and network protocols immediately makes it less likely that a vendor will use such underhanded techniques, and gives the agency the chance to leave any vendor which does use them.

Appeal to local pride and competitiveness: use a list such as this one to calm fears that a particular government agency will be the first or only one adopting open standards.  Give information about which localities are considering adopting open standards, so that decision-makers begin to see that they risk being late adopters.  Find out what the major local industry is, and see what is going on in other places where this same industry is found.  Discuss the economic impact of choosing to support a de facto standard format, especially since the vendor behind that format is changing it again.

Discuss how the choice affects your pocketbook and the pocketbooks of others like you, as well as any employers and organizations in which you are involved.  You can be sure that companies which depend upon control of proprietary formats to keep out competitors (as well as companies that depend on such companies) will turn out to oppose anything that could free their customers to utilize competitive offerings.  If you can, show where choosing a proprietary format or protocol instead of an open standard harms locally-based businesses, resulting in lower tax revenues and higher costs to local governments.

Submit information in ODF and other open standard formats, and offer assistance installing Sun’s ODF plugin if the government agency complains that they are unable to read it.  (Of course, if you need the service they are providing, it you may have to submit .doc/.xls versions as well.)

Finally, recognize that this is not a one-shot deal.  What is necessary is continued and long-term involvement in advocating for this issue and assisting with implementations wherever possible.

2008: The Year of Open Document Formats?

Watching the Web recently, it seems that open standards for document formats are the issue of 2008. I think this is a good thing, but it does present some key challenges for those of us who promote openness.

First of all, there are several different models and ideas about openness, and we need to be careful to distinguish between them.

1. Open standards — standards that are openly developed, openly and thoroughly specified, vendor-independent, and implementable by any and all competitors. Some of the issues with an open standard include whether a recognized standards organization has custody of the standard, whether the standard depends upon a particular (hardware or software) environment, whether there are any "intellectual property", licensing, or legal restrictions that could complicate implementation of the standard, whether the standard is compatible with pre-existing standards, whether competing vendors can actually implement the standard, and whether the leading vendor in that market is willing to cooperate with the standard. Open standards enable multiple vendors to produce products that are more or less compatible, which often results in a larger market (unit sales), lower prices, and more choices for their customers. The controversy surrounding Microsoft / Ecma International’s OOXML (ECMA 376) has helped to highlight many of these points.
2. Free / libre and open source software (FLOSS) — software that is released under any of certain licenses (OSI approved licenses, FSF approved licenses), meeting certain standards (OSI standards, FSF standards) much of which is developed using open processes. In some sense, any FLOSS software application’s file formats are open formats, because it is possible to work out the format by examining the software source code.
3. Free culture, often personified by the Creative Commons movement and Wikimedia. This is based around returning control of cultural and scientific information to society in general, instead of concentrating control in the hands of corporations. To quote CC’s purpose statement, "We use private rights to create public goods: creative works set free for certain uses. Like the free software and open-source movements, our ends are cooperative and community-minded, but our means are voluntary and libertarian. We work to offer creators a best-of-both worlds way to protect their works while encouraging certain uses of them — to declare ‘some rights reserved.’"[1] Likewise, Wikimedia Foundation’s purpose statement includes "The mission of the Wikimedia Foundation is to empower and engage people around the world to collect and develop educational content under a free content license or in the public domain, and to disseminate it effectively and globally."[2] Incidentally, Wikipedia and its sister sites are adding support for printing and word processing using open standard file formats (PDF and ODF), which should open even more of their content to users — now as reusers and not merely consumers of the content.
4. Social freedom movements, which seek to promote civil liberties for individuals, as well as to oppose oppression against individuals and people-groups around the world. These groups tend to oppose the concentration and misuse of power within governments and corporations. Looking at recent history in the United States, I think they have been less than effective here, but I expect that to start changing this year. Often, a local manifestation of social freedom activity is open government or open meetings acts and financial reporting requirements. These come from the idea that government should represent the interests of the people it serves, and that the best way to ensure this is to require that most decisions be made in public, where improper attempts to influence decisions can be observed and stopped.
5. Open file formats — file formats that are openly specified (and reasonably complete) and designed to facilitate interchange between computers that may or may not be using different applications. A good open file format is also an open standard, while a pseudo-open file format can be a tool used to make competitors look inferior in order to grab market share from them. I wrote an essay last year about open file formats legislation (AB1668) that was being considered by the California legislature. The Orange County Register agreed.
6. Open network protocols — these are used for network communication and transfer. An open protocol is (again) openly specified and reasonably complete, and is designed to facility information interchange between different computers which may or may not be using the same software. A good open network protocol is also an open standard (such as an IETF RFC), while a poor one may not be complete or specified enough to enable anyone to implement it. A pseudo-open protocol can be used to make competing operating systems and applications look inferior in order to grab market share from them.

Combining #1 with #5, we get open standard file formats.

One of the biggest stories of the past couple of years has been the way that Microsoft ignored all outsiders to create its Office Open XML (OOXML) file formats full of gooey secret sauce, then dumped the fully-formed and essentially unchangeable formats on Ecma’s doorstep for standardization. This format is essentially a subset of the formats implemented for Microsoft Office 2007, so Ecma was really constrained in what changes it could make. Ecma then flipped the formats over to ISO for fast-track standardization, stirring up a hornet’s nest of opposition. The Not-so-open XML formats fail reasonable standards of openness (including those above), which forces Microsoft to oppose any attempt to use such standards for choosing software.

Next month is the ballot resolution meeting, where Microsoft and Ecma will propose changes to OOXML in order to convince member national bodies to vote to approve OOXML as an ISO standard. Since the kinds of changes that will actually make OOXML an open format would completely change the character of the format, it is not likely that the proposed changes will be enough. I hope that ISO hands Ecma a pink slip over this one.Expect a lot of politicking, including deceitful promotional material, to come out over the next month.

Remember, an open standard file format is not the same thing as open source, although a truly open format should have one or more open source implementations. Likewise, good government requirements for using open standard file formats are not requirements to avoid the use of any particular vendor’s products–an open standard format can be implemented by anyone that has the skill and the time to do so. If your vendor cannot figure out how to implement a format that is openly specified, how can you be sure they can implement anything else?

Remember also that "choice of standards" means confusion for end users (consumers) and prevents the interoperability that they crave. When a state employee asks how to open a file sent by a county employee using a different brand of software,this is choice of standards. Choice of applications, on the other hand, means that the products of multiple vendors understand the same formats, so that users do not have to care what brand of software was used to create the document / spreadsheet / presentation. When you go to a Web site and it opens whether you are using Internet Explorer or Firefox or Opera or Safari, you are seeing open standard file formats (choice of applications) in action.

Thanks to the wildly popular XO computer from the OLPC educational organization, schoolchildren around the world are learning to use open standard file formats. We already know that switching to formats such as OpenDocument Format is a rational choice, even if there is some short-term pain, just because of the long-term benefits they bring. Many users have resisted switching because ODF is unfamiliar. Well, a generation of future engineers, scientists, politicians, and doctors will learn on the XO, and will not have irrational preferences for closed proprietary formats.

As Western computer owners / users replace their existing computers and software, they will find that they face similar short-term pain with OOXML, without the long-term benefits to be gained. As we continue to bring the concept of open standard file formats to the forefront, people will begin to consider this in their purchasing decisions. We need to bring it so forcefully forward that even the pointy-haired bosses begin to request ODF compatibility in their companies’ purchases. We’ll really know we’ve made an impact when the generic software included with low-price computers along with the cheap "we’re compatible" office software in the office superstores promotes its compatibility with ODF and other open standards.

Besides, it seems that a good portion of the office applications software suite is moving to the Web, where such sites as Google Docs and Zoho are more or less ODF-compatible.

The challenge is to avoid being portrayed as some kind of nutzo extremists. Just as they did with the AB1668 hearings, leading vendors will try to say that they represent "the American way" and that openness advocates are anti-social throwbacks to the 1960s and 1970s, burning draft cards and bras, while munching on granola and wearing Birkenstocks.Our neighbors and coworkers need to see that we are normal people, just like them, who wish to smooth out some of the speed bumps that slow down data interchange and hinder long-term data preservation. They need to see that we are seeking our children’s (and their children’s) best interests when we insist that their schools stop teaching how to use a specific brand and version of software, instead teaching the general principles that apply to all brands, and offer to give the schools and their students free CDs of OpenOffice or Symphony. They need to see that we are helping poor and rich alike when we insist that our local governments standardize on ODF and other open standard file formats.

Technorati: odf ooxml pdf open standards open formats

Powered by Bleezer

Opening ODF (.odt, .ods, .odp, etc) Files

You are doing your regular day-to-day work, when someone sends you a file that your office applications suite will not open.  What can you do to open this file?  Well, if the file is in OpenDocument Format (ODF), you have the following alternatives available to you.

What Is OpenDocument Format (ODF)?

ODF is a file format specification created by OASIS, an industry standards group.  Its purpose is to create a set of standardized file formats that is XML-based, archivable, not written specifically for any specific vendor's products, and can be freely implemented by anyone.  This standard has been approved by the Internation Organization for Standards (ISO), and is soon to be required for government documents in some European countries and American states.  The format includes handicap accessibility, so that such tools as screen readers can easily hook into an ODF-using application.  It re-uses proven technologies such as the W3C's XML, MathML (for mathematics formulae), and SVG (graphics) formats within the standard, rather than experimental and unproven technologies.

Because ODF is a truly open standard that does not place the user's data into the hands of any particular vendor's control, it is something that government agencies need to implement for the documents that they hold in trust for their citizens.  Best of all, several years from now, after one or more software upgrades, users will still be allowed to access their data—already, some people and companies have found that their historical documents are not readable by their current software—this open format means that any decent programmer can always implement tools to enable access to stored data.

Reading, Printing, Editing & Writing ODF

The following list consists of items known to me as of 2007-01-08.  I will update this list as I find out more information.

1. File Viewing Software: View and Print The File
   • TextMaker Viewer can view and print ODF word processing documents (.odt files).
   • OpenDocument Fellowship's Viewer can view and print ODF wordprocessing (.odt), spreadsheet (.ods), and presentation (.odp) files.  It is available for Windows, Linux, and Mac OSX.  Thanks to Jean Weber for getting me this information.
     Update: 2008-01-18: It appears that OpenDocument Fellowship's .org domain is now held by a squatter.  Use the .com link above. Thanks, Jesse, for reporting it.
     Update: 2012-02-20: It now appears that OpenDocument Fellowship is gone. Clicking the above link leads nowhere.
2. Online Conversion Sites and Online Office Application Suites
   • 3BView offers the ability to convert to or from ODF formats on their Web site, including a free trial. Apparently, 3BView has been purchased by Microsystems, and now offers DOCX-related services only.
   • Google Docs can import .odt (ODF word processing) files, but it apparently exports .sxw files renamed as .odf.  These files will open in OpenOffice.org, but may not open correctly in other applications that support ODF.
   • Zoho Office supports ODF for both uploading and downloading documents.
   • IBM's Workplace—network-based applications for businesses—is adding support for ODF.  Their Lotus SmartSuite product is not currently doing so.  Is SmartSuite even still being developed and sold?  Will IBM make it easy for someone to find out if their business is a potential user of Workplace?
   • Added 2007-01-13:ZamZar offers an online file conversion service, currently zero-price.
   • WebPDF is a server that gets set up to provide service to a company's network, converting between ODF, Microsoft, and other formats or converting them to PDF.  There are pricing plans to enable application service providers to use the server for their customers.
     Update: 2012-02-20: It currently appears that the software now known as WebPDF converts more than 100 formats, including the ODF formats, into PDF/A.
3. Installable Software Applications
   • OpenOffice.org, available for Windows, Linux, FreeBSD, Mac OSX (X11 only), Solaris
   • KOffice, available for Linux, Mac OSX, FreeBSD, and any other UNIX-like operating system
   • I recommend LibreOffice over OpenOffice.org. I believe The Document Foundation is the project shepherd that Sun should have put into place prior to selling itself to Oracle, and that TDF will better ensure that more than just large corporations’ interests are served by the directions taken by the product. Further, LibreOffice is working to make the product slimmer and faster.
   • AbiWord (on Windows, be sure to install the extra file formats plugins) available for Windows, Linux, and Mac OSX
   • NeoOffice, a Mac-native derivative of OpenOffice.org
   • StarOffice for Windows, Linux, Solaris operating systems. It appears that Oracle, the new owner, has closed their commercially-supported version of OpenOffice as part of the transition to the Apache Foundation.
   • added 2007-10-01: IBM Lotus Symphony for Windows, Linux operating systems with at least 512MB of RAM; this one can also handle many of your old Lotus SmartSuite files.  A Mac version is expected in 2008.
   • TextMaker, PlanMaker (SoftMaker Office) for Windows, Linux, FreeBSD operating systems.
   • WordPerfect. Supports word processing files (.odt – OpenDocument text).  You may want to write to them to ask for broader support if you intend to buy their products.
   • added 2008-01-18: Sun's ODF Plugin for Microsoft Office allows users of Microsoft's product to also open and save ODF (.odt, .ods, and .odp) files in the corresponding Microsoft application.  Available for Windows operating systems. UPDATE: 2010-04-20: Oracle / Sun now charges $90 for the plugin, with a minimum 100 unit order. For most of us, this means that this plugin is no longer a reasonable tool for our uses. Update 2011-04-18: A recent search of Oracle’s site gives "page not found" errors for StarOffice / Oracle Open Office and for the ODF Plugin. Oracle did recently announce that OpenOffice.org is being released to the community’s stewardship. Perhaps the plugin is included in that announcement.
   • added 2009-05-01: Microsoft Office 2007 with Service Pack 2 can now read and write ODF files. Available for Windows operating systems.
   • added 2011-04-18: Microsoft Office 2010 can read and write ODF files. Available for Windows operating systems. Their implementation appears to be substandard. Unless you’re at work and have no choice, use LibreOffice or OpenOffice.org instead. Not only do they handle the format better, but they are less annoying to use.
4. Freely available software that can manipulate data in files stored in ODF file formats
   • If you are writing such an application, please inform me and also Rob Weir.
5. Software for mobile phones and PDAs
   • Update, 2007-10-20: SEPT-Solutions Mobile Office can read .odt (text), .ods (spreadsheet), and .odp (presentation) files, with editing capability expected in a future version of the product.  Thanks to Bob Sutor for catching this announcement.
6. Importers / Exporters for Web Content Systems
   • ODT Indexer – Allows Joomla! 1.0 indexing to index contents of ODF text document (.odt) files.
7. OpenDocument Fellowship maintains a list of applicatons at http://opendocumentfellowship.com/applications.  Be sure to use their .com domain in the link, not the .org that has been taken by a cybersquatter.  Thanks to Jean Weber for bringing this list to my attention.

If you have knowledge of something that I missed, please inform me about it, so I can update this information.