Index ¦ Archives ¦ RSS

My hack week at Collabora: btLr text direction in Writer

Estimated read time: 4 minutes

As mentioned in my previous such report, a hack week is when we are allowed to hack on anything we want in LibreOffice for a few days at Collabora. I used this time to implement core support for the btLr text direction in Writer.

Motivation

If you work with tables in Word, it’s very easy to create this writing direction: the context menu in a table cell has a menu item to set the direction of the text, where you can rotate the text by 90 degrees counter-clockwise or clockwise. The counter-clockwise btLr direction is the problematic one. Support for tbRl was fine already, since that is needed typically for Chinese/Japanese scripts as well.

Results so far

Here is how the baseline, the current and the reference rendering of btLr text looks like:

https://lh3.googleusercontent.com/o5t6-JQeeNaZpx0YURMvS6xUJv7L4KbkbKnn6VPQ0yzULxHFI15ufkwaw_m0FVY7B2tv8gOnTw1CEY2Uxq6BFTgXHlcorXS52J8X1-mNjsyHKYDmoNG-MQ9X1LtdUWmrLl_W3b2ifQ=w640
Figure 1. btlr-cell.docx, baseline
https://lh3.googleusercontent.com/Y8jXvq7TFyNoKVjwWd_QvJNJaOySZdKZE_HqqBaTwGoi_rExCee3eDAHx4AS49E7d7bcjG8SEgxnXOmdKFXaJx0MzmadungQ7D0SVqdSqC2trMC9InsHdKUTq1iwu5p8bDwUfIizng=w640
Figure 2. btlr-cell.docx, current
https://lh3.googleusercontent.com/KYSBaOiAMdy_U8shsb4zoS8J5uyZzwkFPZY4qIKTrHKlo-M-pCdiHHBUxZ9OmQ-uv_QkBiktXeprTQD6gANDvzDVi8JWs4-Ng2RL-uoMcQCrQNL6Hk7tjXSJBaaqxc2skfZzmepkqA=w640
Figure 3. btlr-cell.docx, reference

You can see how the second paragraph in the cell was missing from the rendered result and now we basically pixel-by-pixel match the reference.

How is this implemented?

If you would like to know a bit more about how this works, continue reading… :-)

The document model and UNO API were reasonably straightforward to implement, but the layout was much more challenging. Writer already supported 3 writing directions:

  • typically used for Latin (left to right, top to bottom)

  • Chinese/Japanese (top to bottom, right to left)

  • Mongolian (top to bottom, left to right) text.

This new one is also a vertical direction, also left to right, but bottom to top. The initial layout contained code to read the new enumerator from doc model, extend the SwFrame class to handle this new bottom to top mode, some handling of switching between horizontal/vertical mode and at the end mapping from Writer layout’s direction to VCL’s "900" font orientation. There are more things to handle in layout, but this was good enough to look at other areas as well.

The ODF filter required updating, which was a bit challenging as it was necessary to write different attribute names depending on which enumerator is used from an emumeration, and we don’t have good support for this. Once the filter code was in place, I could write some layout-level tests as well.

Since we have .ui files for UI descriptions, adding UI support was really easy.

Time came to step away from coding for a moment and write up paperwork to propose this feature to be part of the next ODF version (thanks to Andras for the help there!).

Finally I went back to layout, and improved things a bit more: after fixing baseline offsets, the positioning of the text was exactly matching what Word does. How do I know? I used this little script:

gs -dNOPROMPT -dBATCH -sDEVICE=jpeg -r75 -dNOPAUSE -sOutputFile=btlr.jpg btlr.pdf
gs -dNOPROMPT -dBATCH -sDEVICE=jpeg -r75 -dNOPAUSE -sOutputFile=btlr-word.jpg btlr-word.pdf
composite -compose difference btlr-word.jpg btlr.jpg out.jpg

Which allows seeing the differences between our and Word’s PDF output. Additional work was needed to handle multiple paragraphs in a table cell. At this stage I was happy enough with the rendering result, so finally pulled the trigger and replaced the old DOCX filter hack (using character-level rotation) with simple DOCX filter mapping from OOXML’s btLr direction to Writer’s btLr direction — i.e. what was already done for the tbRl case.

Future work

The feature works good enough already so that this new core feature can be used by the DOCX filter by default, but there are still a few rough edges:

  • the shell code (cursor travelling, selection painting, etc) only has partial support for this new direction

  • RTF and DOC filters are not yet updated

  • the ODF proposal has a list of contexts other than table cells where the new writing direction could be used, which lack UI/filter support/etc at the moment.

All this is available in master (towards LibreOffice 6.3), so you can grab a daily build and try it out right now. :-)


Improving SmartArt import in Impress FOSDEM talk

Estimated read time: 1 minutes

(via cor4office)

The next step in the recent SmartArt story is my Improving SmartArt import in Impress talk at FOSDEM 2019, in the Open Document Editors devroom. The room was a bit far away from the popular places, but the livestream worked out nicely.

There was also a hackfest before the conference, I looked at RTF export of rotated Writer pictures.


SmartArt improvements in LibreOffice, part 3

Estimated read time: 2 minutes

I recently dived into the SmartArt support of LibreOffice, which is the component responsible for displaying complex diagrams from PPTX. I focus on the case when only the document model and the layout constraints are given, not a pre-rendered result.

First, thanks to our partner SUSE for working with Collabora to make this possible.

Continuous Block Process, Accent Process and Organization Chart

In this post I would like to present the progress done last month regarding the above mentioned diagram types — these are used in many documents.

The improvement (as always) come in small incremental steps:

  • Continuous Block Process now reads space width from constraints.

  • Accent Process now has the missing bullets and fixes an incorrect large paragraph-level indent.

  • Organization Chart now has an initial implementation of the hierRoot and hierChild algorithms.

  • Organization Chart now handles multiple employees for a manager.

With all these fixed, we reach a much better state for the mentioned diagram types.

Results so far

The SmartArt test documents from sd/qa/unit/data/pptx/ is what I used for testing this work.

Here is how the baseline, the current and the reference rendering of Accent Process looks like:

smartart-accent-process.pptx, baseline

smartart-accent-process.pptx, current

smartart-accent-process.pptx, reference

And here is how the baseline, the current and the reference rendering of Organization Chart looks like:

smartart-org-chart.pptx, baseline

smartart-org-chart.pptx, current

smartart-org-chart.pptx, reference

This is not not perfect yet, but it’s clearly a large improvement, all text is now readable from the diagrams and bullets are no longer missing!

All this is available in master (towards LibreOffice 6.3), so you can grab a daily build and try it out right now. :-)


SmartArt improvements in LibreOffice, part 2

Estimated read time: 2 minutes

I recently dived into the SmartArt support of LibreOffice, which is the component responsible for displaying complex diagrams from PPTX. I focused especially on the case when only document model and the layout constraints are given, not a pre-rendered result.

First, thanks to our partner SUSE for working with Collabora to make this possible.

Accent Process

In this post I would like to present the progress regarding the Accent Process preset, available in PowerPoint — which is used in many documents.

This exposed several shortcomings of the current diagram layout we have in LibreOffice:

  • Values are not read from constraints (there was a reason for this, they can be complex, given that depending on the context, the unit is points or millimeters and the unit is always implicit).

  • ZOrder offsets were ignored.

  • Linear algorithm did not take size from constraints when it came to recursing into child algorithms.

  • Data point assumed that all text for it is a single "run" (i.e. either all text is bold or nothing, not half of it).

  • followSib axis was not implemented for forEach, so when you have arrow shapes between objects, we created N arrows, not N - 1 ones.

  • Connectors were created as invisible shapes and had the wrong width/height aspect.

With all these fixed, we reach a much better state for handling accent process.

Results so far

smartart-accent-process.pptx is what I used for testing of this work.

Here is how the baseline, the current and the reference rendering of the test documents look like:

smartart-accent-process.pptx, baseline

smartart-accent-process.pptx, current

smartart-accent-process.pptx, reference

This is not not perfect yet, but it’s clearly a large improvement, all text is now readable from the diagram!

All this is available in master (towards LibreOffice 6.3), so you can grab a daily build and try it out right now. :-)


SmartArt improvements in LibreOffice

Estimated read time: 2 minutes

I recently dived into the SmartArt support of LibreOffice, which is the component responsible for displaying complex diagrams from PPTX, especially in case only document model and the layout constraints are given, not a pre-rendered result.

First, thanks to our partner SUSE for working with Collabora to make this possible.

The problem

There are several ones. :-) If you are just interested in high quality viewing of PPTX files, then your problem started with PowerPoint 2007 not writing a pre-rendered drawingML markup of the diagram to the files, only PowerPoint 2010 started behaving like this. Additionally, if a diagram is not edited, then re-saving with PowerPoint 2010 doesn’t seem to generate the drawingML markup, either. This means that data + constraints cases are quite frequent even today.

Also, one day Impress should be able to actually edit these SmartArts as well, so having the knowledge how to lay out SmartArt (even if it’s import-time-only at the moment) is a good thing.

Results so far

I always write cppunit tests when I work on filter code (in this case OOXML), so far all fixes were visible in just two test files: smartart-vertial-box-list.pptx and vertical-bracket-list.pptx.

Here is how the baseline, the current and the reference rendering of these test documents look like:

smartart-vertial-box-list.pptx, baseline

smartart-vertial-box-list.pptx, current

smartart-vertial-box-list.pptx, reference

vertical-bracket-list.pptx, baseline

vertical-bracket-list.pptx, current

vertical-bracket-list.pptx, reference

In terms of code commits, the fixes are split into several ones:

Clearly the results are not perfect yet, but in both cases nothing was visible, and now all text is readable, so we’re moving in the right direction!

All this is available in master (towards LibreOffice 6.2), so you can grab a daily build and try it out right now. :-)


Text layout performance in LibreOffice conference lightning talk

Estimated read time: 1 minutes

Last Friday I gave a Text layout performance lightning talk at LibreOffice Conference 2018. Click on the image to get the hybrid PDF slides!


ReqIF import/export in LibreOffice Writer conference talk

Estimated read time: 1 minutes

Earlier today I gave an Editing ReqIF-XHTML fragments with Writer talk at LibreOffice Conference 2018. The room was well-crowded — perhaps because the previous talk was about OOXML interoperability. ;-)

I expect quite some other slides will be available on Planet, don’t miss them.


odfsig: an Open Document Format (ODF) digital signatures tool

Estimated read time: 1 minutes

I created a toy project to experiment with a few technologies I wanted to try out (cmake, googletest, xmlsec outside LibreOffice, libzip and AppVeyor). The result is a tool with a similar interface as pdfsig from poppler (for PDF files), just for ODF: a cmdline executable to verify the digital signature(s) in an ODF document.

The source code now has CI on Linux and Windows, so perhaps in a shape that is interesting for others to have a look as well. And if not, then no problem, it was interesting to put together these building blocks and see them working as expected. :-)


mutt/vim offline address book with multiple sources

Estimated read time: 1 minutes

I used to have a hackish setup to trigger address book in the form of code completion from vim, when mutt starts it. I recently migrated the last part of it from python2, so it’s now in a relatively consistent state. On a boring flight trip I finally got around to summarize how it works, perhaps it’s useful to someone else as well. :-)

By multiple sources, I mean Google Contacts for my private account, and LDAP for work-related accounts. I have one script for each to have a local backup:

Then comes mutt/contacts/search, which can read these backups from $HOME/.mutt/contacts/ (you can create symlinks to decide which backup should be included in the search results).

The rest is just a short search.vim that integrates the search script into vim, so when mutt starts it with ft=mail, the search script is invoked when you start typing and you press Ctrl-X Ctrl-O (trigger code completion).

And that’s it, I can start typing mails to friends and customers even without network connectivity, without manually maintaining an own address book just for mutt!


Custom user profiles with unoconv

Estimated read time: 2 minutes

https://farm2.staticflickr.com/1859/43598513125_55bed46184_o.png

Background: I was benchmarking Online vs jodconverter vs unoconv the other day for Collabora’s document conversion page. One problem with measuring unoconv performance was that it provided no ways to run multiple unoconv processes in parallel, while the underlying soffice binary obviously allows this.

So while unoconv is not capable of launching the soffice process in a chroot (improves security), nor is capable of forking an already pre-initialized soffice process (improves performance, both are things Online can do for you), there is no real reason why you should not be able to run multiple unoconv processes in parallel. The previously mentioned benchmarking would be quite unfair if even this kind of multiprocessing would be ignored, but unoconv had no way to specify a custom user profile, which has to be different for each soffice process.

So I filed a GitHub pull request on 1st Jun, and finally it was merged on 10th Aug.

Here is how you can use it for example:

unoconv --user-profile /tmp/tmpf_yreswi -f pdf --port 2002 test.txt
Note
It’s your responsibility to give --port a unique value, but that’s not too hard: if you use a thread pool to launch the unoconv processes, then you can add the thread index to a base port and that will give you a unique port.

So this is available in unoconv master (towards unoconv 0.8.2+1), you can grab the sources from git and try it out right now. :-)

© Miklos Vajna. Built using Pelican. Theme by Giulio Fidente on github.