OStatic

by Dan Saint-Andre on Apr. 19, 2010

I have been a software developer since the 1970's. There is always a scarcity of documentation until it becomes an integral, critical path deliverable along with working code. Otherwise, it is an add-on, do-later set of busy work that seldom happens because "... developers need to work on feature XX or release YY instead ..."

Consider the following:

1. Every program needs a man page

2. Every library routine needs a man page

... write these first. The testers write using the man page as spec for the unit tests.

3. Projects create "blue print" or similar documents.

... aren't these a variation on "Theory of Operation" materials that explain the reasons your software exists and why someone might use it?

4. Application suites have "User Guides" (here be dragons and beasties)

... most of these are little more than an inventory of options and buttons and screenshots saying, "... push button XX and YY happens ..." There is precious little that speaks about why one might want YY to happen, when in routine work flow XX might appear, the benefits of YY over something else, etc.

Why might things be this way? Developers don't know what the code is going to do. I'm not slinging indictments. Rather, they are chasing an illusive something that works and so the code and its design evolves in real time.

Developer time is limited so they write more code. There are always more features ... so they write more code. There are always more defects ...

so they chase something that works. Ad nauseum.

A technical writer needs information from a subject matter expert. With open source, so often that is those who wrote the code. See the above

'graph to understand why the writers cannot get time with the developers.

Tech Writers are, um, well, "writers" and prefer to avoid reading code to discover what is going on. Further, they prefer to avoid chasing the code through whatever different happens with each daily build. Often, projects do not have a "tech writer" role with read access to whatever design and plans documents might exist without said writer earning thoroughbred developer cred.

Available Tech Writer,

~~~ 0;-Dan

by NicDumZ on Apr. 20, 2010

I could not agree more with @dan.

Joe, perhaps you could clarify what kind of documentation needs to be written for Open Source content. Because when I think "Open Source software is usually poorly documented", I usually mean that code is poorly commented. Or mean that various wikis and resources explaining developers APIs or code from a higher perspective (design plans, etc...) tend to be outdated.

And this aspect can't, and won't be fixed by hiring a "technical writer". In fact, I would say that efforts encouraging existing Open Source developers to extend their documentation are lacking. And because of this, one coding in his freetime would rather improve the code than spending time on documentation.

The more commercial interest there is in building a strong developer community, the more developer documentation you will find.

But perhaps you mean "documentation" as in "end-user documentation", mostly thinking of big graphical applications? Sure, maybe OpenOffice needs more tutorials, maybe. I do believe, however, that such applications only exist because of a whole free stack that allows them to exist. And most of the time, the behind-the-scenes mechanisms that allow a certain library to work and to be extendable are not too well documented.

Those mechanisms should only be documented by developers. A technical writer would mostly loose his time trying to figure out what this specific module is doing.

In this context, Summer of Code are right: applicants should not be expected to write documentation for others, but they should spend time documenting their own features.

by an anonymous user on Apr. 20, 2010

FOSS is a free market. People don't write FOSS code to give you a free ride, but to get something done. That means they'll write documentation if and only if that helps solving their own problem.

Furthermore, there is plenty of documentation, in the form of books It's one of the ways FOSS developers make money.

Finally, have you looked at commercial software recently? It comes with less and less documentation too. Even library documentation is often only extracted source code comments.

by craig on Apr. 20, 2010

have to agree with that - not only about documentation, but also about art.

there are dozens of open source games that could benefit from having additional artists to work on them - maps, textures, backgrounds, 2d & 3d models, etc.

and, at least as importantly, it would be a Good Thing for the open source meme to take hold in the modding communities based around commercial games (unfortunately, a copyright-maximalist version of the "Intellectual Property" propaganda meme is quite common there - with many people seeing borrowing and re-using as "stealing", even when the original artist has clearly permitted such re-use).

e.g. it would be nice to see an open source FPS game that was an actual game with a plot and a story-line and goals rather than just another multi-player death-match.

by Frank Zanten, van on Apr. 20, 2010

Well, there are document management systems which provide perfect documentation on installation and/or operating. Just to name one example: http://www.o3spaces.com. o3spaces focuses on being user-friendly, so o3spaces is very easy to use, has a good forum and provides good documentation.

by an anonymous user on Apr. 20, 2010

Couldn't agree more :) Still usually the biggest problem with OSS projects is the lack of time, not the lack of will to document. I always try to do my best when documenting my work, but that time is taken away from coding. Professional development is of course a different thing.

by Joe Brockmeier on Apr. 20, 2010

@NicDumZ I'm not referring to developer documentation, I'm talking about usable end-user documentation. That said, better developer docs would *also* be a very good thing.

The larger point goes beyond documentation: The programs that are being run now are simply too developer-centric and aren't bringing in people not already inclined to gravitate towards open source.

by J on Apr. 20, 2010

I disagree...

by Anne Gentle on Apr. 20, 2010

Yep, yes, and uh huh! I love seeing ways for other forms of contribution to get noticed and working in an open source project. These are good ideas.

FLOSS Manuals is a community of writers creating free manuals for free software - and we held a book sprint for GSoC to write the mentor's guide - http://en.flossmanuals.net/GSoCMentoringGuide. We approach project leaders and see if they have a specific "book" or "online help system" in mind that they'd like to write, and then organize a book sprint around it. It has also gone the other way, where an open source project finds out about the FLOSS Manuals toolset and comes to us to get a book set up. We've seen projects get away from the old school "manual" and write user-centered guides.

I think everyone's redefining "documentation," and hopefully ditching that "bad rep" it got in the 70s, 80s, and hey, the 90s too. :) I hope community-based documentation can break the old models and give us some new user experiences with doc.

by Karsten Wade on Apr. 20, 2010

Joe and I discussed this separately, but I thought I'd bring my comments here.

I've been involved in discussions around a 'summer of content' since the first GSoC Mentors Summit four years ago, and I assisted very little with OLPC's Summer of Content three years ago:

http://wiki.laptop.org/go/Summer_of_Content_2007

We also worked with a few classes in 2008 who worked on documentation with the Fedora Docs Project -- we supplied work, tools, and mentors. It was successful, but lacked the completeness and longevity we are looking for in recruiting via student summer programs.

This is good news because it means we have experience already to build from, and clearly there is pent-up demand, experienced and hungry mentors, and untapped students.

For this first run of Fedora Summer Coding, we wanted to try out new ideas, but we have to limit how many new program model changes we can make while we are in the middle of building and running this first instance.

We've been discussing both the content idea and running our program for the Southern Hemisphere. Over the next few weeks we'll begin planning for the fall, and that discussion will include planning for adding content projects to the schedule.

What would content be?

Documentation, yes. Also, wiki work, marketing materials, and so forth.

Design, definitely. UI/UX design, website, graphics, many, many areas here.

Localization, good idea. Not sure how this would work out, maybe a series of much smaller bounties to represent discrete chunks of work or to translate lonely parts of the project.

Zonker, if you are interested in being involved, we could really use organizational help for the coming fall program. It's going to be much bigger, if anything because of the momentum from this summer. But if we add in the complexity of content, it will be quite rewarding.

by mike on Apr. 20, 2010

>"And this aspect can't, and won't be fixed by hiring a "technical writer"."

Part of the problem in the developer community is an attitude that the only people who can understand programming are developers, and that the only possible source of documentation is the developers themselves. Many technical writers (I love how the commenter conveys condescension by putting the term in quotes) are programmers or have been programmers, and are perfectly capable of reading code, writing samples, and so on. (Oh, you've met a "technical writer" who wasn't very technical? Guess what, I've met "developers" who weren't very good programmers. Less-than-stellar competence occurs in all fields.)

There are many reasons that Joe is absolutely correct that any effort to generate yet more code should take a wider view about documentation, etc. But to address this specific idea that developers should just be doing, I'll throw out the following set of reasons off the top of my head as to why developers are often poor documentors of their own code:

* As noted repeatedly, developers would rather be coding.

* Many developers are just bad writers, mechanically -- not just spelling and grammar, but being able to articulate what it is they need to say.

* Developers might not understand what the reader actually needs to know -- after all, it's generally obvious to the developer.

* Developers are often focused on details like syntax or parameter definitions, not on the reasons why a piece of code exists or why the user might need to use it.

* Developers often have a hard time grasping that the reader does not have the same knowledge that the developer does. ("You are not the audience.")

* Many developers are not native speakers of English (assuming the target documentation is supposed to be in English).

by an anonymous user on Apr. 20, 2010

I agree with almost all the points above ~ I suppose I should admit to writing badly documented code since 1965, but I have a different good excuse (?) for each and every case.

One specific point: a more recent problem is the advance of Javadoc and similar systems: people have the odd idea they provide documentation so don't bother with any real documentation. (This is just an automation of the point above: "Developers are often focused on details like syntax or parameter definitions, not on the reasons why a piece of code exists or why the user might need to use it."

by DoctorOwl on Apr. 21, 2010

I completely agree with the author and Dan and others in the same vein. I'm not an official technical writer, but I tend to gather lots of my own documentation for companies I work for - and I even wrote up some tutorials on how to use some open source libraries.

I think documentation was better two decades ago, where everything came with a manual and/or tutorial and it was based on how to actually do things.

The large technical manuals I've seen since are exactly as another poster described, Button X does X, without any actual business processes or reasoning or flow-on effects of the action. It's disgraceful that huge products can get created like that, and the knowledge about why is never written down somewhere accessible.

It is worse since Java and .NET, where developers seem to imagine having a documentation stub for a class counts as documentation! Holy cow, look at Microsoft's own documentation, and the venerable "Books OnLine" that gets referenced all the time. It's crap and rarely addresses any real world situations.

It does take a special kind of person to write documentation, one that can read code, write decent English, format a nice document (that's probably the least important thing in my opinion), understand why people do what they do, and be willing to take the time to explain it without talking down to the audience.

Yeah there is so much code out that there is useless because it's not documented, libraries, applications. Sigh. It's the biggest failing of software everywhere I see, both open source and commercial.