Last week I implemented support for translated man pages, fixed yet more bugs and did some serious code cleanups and refactoring as suggested by Thomas Waldmann and pylint. I got a public test wiki setup from the Fedora Infrastructure project, we spent about a day setting it up with Paulo Santos, also with some help from Mike McGrath, thanks guys 🙂
It took such a long time because the Moin instance runs as the “apache” user and my import script runs as the user you happen to be on the console, so permissions have to be set up so that both apache and the script have enough rights for the wiki data directory. Right now there’s some problem with the wiki installation (again) so I won’t give a link yet, especially because Paulo is on a holiday, we’ll probably get it fixed once he comes back.
Then when testing my script there, I hit a serious doclifter problem. With one man page, libbind-getaddrinfo.3, my script seemed to go crazy and take up all the memory on the server. I then tested it locally, same thing here. At first I thought all of this was caused by my script or by Moin and I spent hours trying to debug them. When I eventually found nothing, I realized it is a doclifter problem / bug. So I had to implement filtering in my script, now you can pass it a filename where the file has the names of the files that doclifter can’t handle and my script will then skip those files. Also I noticed that some packages have even 5-6 meg man files, which took about 2 hours to handle, so I had to set up my script so that it skips any files that are bigger than 1MB.
Wiki storage format and editing
I have done some testing and research on the Moin 1.6 DocBook branch from last year’s GSoC. It seems that I can’t use it as I originally planned, since it has no support for the refentry parts of DocBook. Refentry is a DocBook element for saving unix man pages and that’s what doclifter makes. So I don’t have a direct way of converting the DocBook XML to Moin wiki pages. I could probably try to enhance the XSLT stylesheets that do DocBook to Moin conversions. That way we would have man/info pages in wiki markup. But it would probably be way more difficult to do Moin wiki page -> DocBook XML conversions so that there would be no information loss.
As mentioned, DocBook has elements especially meant for man pages, so man -> DocBook -> man can be done with little if any information loss. But wiki markup has no such elements, so man -> wiki -> man, even with DocBook as an intermediate format, can easily cause a lot of information loss.
With that in mind, here’s my suggestion: We have about three weeks of Summercode Finland left, so I think I should leave DocBook as the storage format and concentrate on exporting changes / patches from the wiki against upstream man/info files. That way we would hopefully have the ability to edit man/info pages in a wiki and send patches from those edits upstream. The edits would have to be made in DocBook XML, which is probably not as user friendly as the wiki markup, but the basic idea / framework would be there. Then if it seems possible and someone with more XSLT/XML/DocBook experience than me is interested to help, DocBook refentry -> wiki -> DocBook refentry conversions could be added later.
Readers, what do you think?
Leave a Reply