Last year in September we decided to get rid of the writerfilter-based DOC tokenizer, and I volunteered to actually do this. As cleanups in general have a low priority, I only progressed with this slowly, though yesterday I completed it, that’s why I’m writing this post. :-)

Some background: the writerfilter module is responsible for RTF and DOCX import in Writer. As the above picture shows, the currently used DOC import is independent from it, and there was also an other DOC import filter, that was in writerfilter which was disabled at runtime. As I don’t like duplication, I examined the state of the two filters, and the linked minutes mail details how we decided that the old filter will stay, and we’ll get rid of the writerfilter one. It’s just a matter of deleting that code, right? :-) That’s what I thought first. But then I had to realize that the architecture of writerfilter is a bit more complex:

It has the following components:

The traffic between the tokenizers and dmapper is called tokens. Naturally it’s not enough that tokenizers send and dmapper receives these tokens, they should be defined somewhere as well. And that’s where I realized this work will take a bit more time: instead of having a single token definition, actually the ooxml tokenizer defined its own grammar, and doctok also defined two additional grammars. And of course dmapper had to handle all of that. ;-) Given that OOXML is a superset of the DOC/RTF format, it makes sense to just use the ooxml grammar, and get rid of the other two.

Especially that — by now you probably found this out — if I wanted to kill doctok, I had to kill the sprm and rtf grammars as well. Otherwise just removing doctok would break the RTF and DOCX import as well, as those also used the rtf/sprm grammars.

So at the end, the cleaned up architecture now looks like this:

And that has multiple advantages:

If you ever tried to use nested comments in Writer (make a selection, Insert → Comment, then make an overlapping selection, and do it again), you may have noticed that only the ODF filter can load and save such a document properly. Recently I have improved this situation a lot. Motivated by seeing this is now supported in the DOCX import filter, I now added support for this also to the DOCX export, RTF import/export and binary DOC import/export filters.

If you want to try this out, core.git has a ODT and DOC samples to play with.

I’ve arrived home yesterday from Brussels where I presented at FOSDEM 2014, in the Open document editors devroom.

We also had a Hackfest, kindly hosted by Betacowork on Monday and Tuesday.

Here are a few talks I enjoyed, not counting the LibreOffice ones:

I was also happy to meet Jacobo, Matus, Tim and Tomaž finally personally. :-)

Quite some other slides are now available on Planet, don’t miss them. I also took some pictures, available here, including photos of all speakers in our devroom.

Although LibreOffice 4.2.0 is not yet released, it was already branched off from master in November last year, and improvements for the next release are already cooking in master. One of these will be a major improvement of shape handling in the DOCX import/export filter.

Some background: when DOCX was initially introduced, it still used VML (which is in short an XML equivalent of the binary shape format), and only Word 2010 started to write shapes using drawingML. Given that Word still understands VML, it wasn’t urgent for us to write shapes using the drawingML markup. As for import, Word still writes an approximate version of the shape in VML as a fallback — that’s what we read till now. Needless to say, newer drawingML features have no VML equivalent so with time it became more and more important for us to finally read and write shapes in DOCX using drawingML, which just happened in Writer.

I’m posting here a few screenshots showing the improvements I’ve implemented. Note that final 4.3 is still far from being released, so this is not a complete list. :-) In each case I’m providing a screenshot showing how it looked (at the end of an import/export/import again roundtrip) before, how it looks now in 4.3 and the reference layout. Click on the images to get a larger image:

OK, this has four pictures: before, now, Word 2007 and Word2010. As you can see now we’re now on par with Word 2010. ;-)

If you want to try these out yourself, get a daily build and play with it! If something goes wrong, report it to us in the Bugzilla, so we can try fix it before 4.3 gets branched off. Last, but not at least, thanks for CloudOn for funding these improvements! :-)

You may remember that I wrote about my little ged2dot project about two months ago. It’s a Python script that can be used from commandline with quite some options. I’m happy to announce that there is a GUI for it now, in the form of a LibreOffice Draw import filter. If you open this GEDCOM file, you’ll see something similar to the above image. You can simply build the .oxt from the Git repo, or just get it from the LibreOffice Extensions site.

Technical details for the interested readers: the import filter just glues together existing pieces. First it runs ged2dot, then dot to generate an SVG. To keep things simple, then the filter "inlineizes" all the included images, finally the already existing SVG import filter does the real work. I find it elegant that each step works from and to the memory, and not ugly temporary files. :-) It works on Linux and Windows, feedback on if it works on Mac is appreciated.

Also, if you ask why this is an extension, not part of LibreOffice core: I guess most users are not interested in building family trees; also being someone who works on the core most of the time, I wanted to try out how implementing an import filter as an extension (filter detection, import options dialog, etc.) works. ;-)

This year’s Software Freedom Day event was held yesterday at Szeged, here at Hungary. I gave a talk about the news of LibreOffice 4.1 / 4.2 (slides), and I was happy to notice there were other LibreOffice-related talks as well:

I would like to thank the organizers all their work, it was a pleasure to attend this event! :-)

It was pointed out that LibreOffice Writer is only able to open, not save .dot (Word binary template) files. From mso-dumper experience, I knew that the difference is quite small between .doc (which we already export) and .dot files. The feature freeze for LibreOffice 4.2 is here next week, so why not adding this feature before that deadline? You can see above the result. :-)

For the ones who love our --convert-to commandline switch, an extra fix was necessary to make --convert-to dot work as well, both commits are now in master. You can try it out yourself using a daily build or just wait for the libreoffice-4-2 branchoff and the first beta from that branch.

Happy templating! ;-)

If you ever had to review a document that had change tracking enabled in LibreOffice Writer, you might have noticed an odd behaviour: if e.g. the inserted text has a spelling error, it’s not easy to reject that change. The reason for this is that in case of spelling errors, right mouse click activates the spellcheck popup menu, which lacked the change tracking operations previously.

As you can see in the above screenshot, this is no longer an issue: LibreOffice 4.2 will have the previous change, next change, accept change and reject change operations in the spellcheck popup menu as well.

In one sentence, with LibreOffice 4.2, Writer allows to reject a spelling error it detected itself. ;-)

This year’s LibreOffice conference was held in Milan, Italy. Links to my slides:

Why so many talks? I couldn’t decide what topic to pick up (git or Writer filters), so I did both, the rest — those weren’t sessions where I spoke for 30+ minutes, so those don’t really count. :-)

Regarding the number of attendees, draw your own conclusions from the group picture, made on 26th — probably more than 100 attendees, counting all days.

Thanks for the organizers for this beautiful event — and also the sponsors, including Google (free food/beer during one hackathon) and CloudOn (fee free/beer during the other hackathon + tablets for the best commits)!

My pictures are available here.

For Hungarian readers, Gábor Kelemen (maintainer of the LibreOffice Hungarian translation) also provided blog posts for the first and second day, also libreoffice.hu had a related article.

LibreOffice Writer had an old limitation we inherited from OpenOffice.org times: if a document started with a table, it wasn’t possible to select the whole document text. There were various bugreports for this problem: one from more than 2 years ago, an other from 2002 (!) — and I’m happy to post screenshots about how I improved this in LibreOffice 4.2 (before and after):

If you want to try this out yourself, get a daily build and play with it. :-)

That’s all for today, thanks for you attention.