Index ¦ Archives ¦ RSS

Writer line heights: removing a 16bit limit

Estimated read time: 2 minutes

Line heights in Writer are typically defined in points on the UI (e.g. 12pt), though they are measured in twips internally (1 point is 20 twips). This height was stored in a 16bit unsigned integer, so the maximum allowed height was 65536 twips, around 116 cm.

Now we track line heights with 32 bits ints, so this limitation is practically removed.

First, thanks Vector who made this work by Collabora possible.

Motivation

Once you insert an image to a Writer document, you can customize its anchor type. The as-char anchor type is handy if you don’t want text to flow around the image. This has the side effect that a large image significantly increases the nominal height of a line. The problematic document had an image height of 118.9 cm (46.81 inch), so the unsigned integer used to represent its height wrapped around, leading to an incorrect layout.

Results so far

Now it looks like the way you would expect it:

https://share.vmiklos.hu/blog/sw-line-height/sw-line-height.png
Figure 1. Writer as-char image with height that is larger than 65k twips

How is this implemented?

If you would like to know a bit more about how this works, continue reading… :-)

As usual, the end goal was reached via a set of incremental commits:

Want to start using this?

Collabora intends to continue supporting and contributing to LibreOffice, the code is merged so we expect all of this work will be available in TDF’s next release (7.2).


Bibliography improvements in LibreOffice Writer: refer to a specific page

Estimated read time: 3 minutes

The bibliography feature in Writer allows authors of e.g. scientific papers to track sources: first you can insert bibliography entry fields, then at the end you can generate a bibliography table automatically.

Writer recently gained two improvements in this area, and now there is one more: the ability to refer to a specific page of a (potentially long) source.

First, thanks TUBITAK ULAKBIM who made this work by Collabora possible.

Motivation

The same feature in normal HTML linking / citation is well-known: you can place anchors in your document and then links can refer to not only to your page, but also can set the fragment part of the URL to jump to a specific section directly. E.g. this link will jump to the "Motivation" section of the above referred previous post.

Wouldn’t it be nice if you could also jump to a specific page of a scientific PDF?

Results so far

You can add such references by choosing the Insert → Table of Contents and Index → Bibliography Entry menu item. Set Bibliography Source to Document Content and press the New button. The appearing dialog now has widgets to set a specific page number:

https://share.vmiklos.hu/blog/sw-bibliography-page/page.png
Figure 1. Refer to a specic page of a bibliography source, user interface

This is then stored in the URL by setting its fragment to page=..., and typically PDF readers understand this syntax, so when you click on the URL, the PDF will open exactly on the cited page.

This also works the other way around: URLs with such syntax are presented to the user with the dedicated widget to see, edit or delete the page number of the URL.

Finally, if you refer to different pages of the same source, the bibliography text generator detects this and lists the source in the bibliography only once.

How is this implemented?

If you would like to know a bit more about how this works, continue reading… :-)

As usual, the high-level problem was addressed by a series of incremental development steps:

Want to start using this?

Collabora intends to continue supporting and contributing to LibreOffice, the code is merged so we expect all of this work will be available in TDF’s next release (7.2).


Improving borders of merged table cells in Writer

Estimated read time: 2 minutes

Writer now has better support for Word-compatible border rending when it comes to vertically merged cells in tables.

First, thanks Docmosis who made this work by Collabora possible.

Motivation

Both Word and Writer allow specifying borders for any kind of table cells. When the user creates a vertically merged cell, there is a covering cell and there is one or more covered table cells.

The Writer approach is to render the cell borders according to the properties of the covering cell. This has the benefit that each edge of the table cell has a single border style (e.g. dashed or hairline).

The Word approach is to render the cell borders as if there would be no vertical merge, according to the properties of the covered cell. This has the benefit that merging the content of cells vertically doesn’t change the border rendering, but it also requires more complex code for painting.

Results so far

Writer can now detect that your tables originate from Word formats and render table borders the Word way in that case.

Here is how the new rendering result look like:

https://share.vmiklos.hu/blog/sw-merged-border/new.png
Figure 1. Writer rendering in compatibility mode, new output

And here is how it used to look like:

https://share.vmiklos.hu/blog/sw-merged-border/old.png
Figure 2. Writer rendering in compatibility mode, old output

And finally the reference rendering is:

https://share.vmiklos.hu/blog/sw-merged-border/ref.png
Figure 3. Writer rendering in compatibility mode, reference output

You can see that the B4 and B5 cells are covered, they had some unwanted border on their left side and this is now gone.

How is this implemented?

If you would like to know a bit more about how this works, continue reading… :-)

  • First, some building blocks were introduced: SwCellFrame::GetCoveredCellInRow() can look up a covered cell in a certain row, provided that this cell covers it

  • Building on top of this, SwCellFrame::GetCoveredCells() can provide a list of cell frames which are covered by the current cell, due to vertical merge. This is needed, because previously the layout didn’t have to consider properties of covered cells, so while the document model had this information, it was not visible to the layout in a convenient way

  • Using the above functionality, SwTabFramePainter::Insert() can suppress painting of certain border lines in Word compatibility mode

  • Finally, the code change can be covered with a test by recording the rendering and asserting the vertical positions of border points: we can check that all the positions belong to the 1st, 2nd or 3rd rows, and not to a row below them

Want to start using this?

Collabora intends to continue supporting and contributing to LibreOffice, the code is merged so we expect all of this work will be available in TDF’s next release (7.2).


My hack week at Collabora: improvements to gutter margin in Writer

Estimated read time: 3 minutes

As mentioned in a previous such report, a hack week is when we are allowed to hack on anything we want in LibreOffice / Collabora Office / Collabora Online for a few days at Collabora. I used this time to implement core support for RTL gutter margin in Writer.

Motivation

I posted about Writer gutter margin in general back in February, and two follow-up requests accumulated around this new feature since then.

First, the gutter margin could be on the left or at the top for non-mirrored documents, which initially sounded like a complete solution, but later it turned out that right-to-left (RTL) documents want it on the right.

Second, there was a request from the OASIS TC to to implement the ODF filter differently for gutter margin.

Neither of these is simple to do, so this hackweek was a good opportunity to address these problems.

Results so far

Here is the layout and user interface for RTL gutter looks like:

https://share.vmiklos.hu/blog/hackweek-2021/layout-ui.png
Figure 1. RTL gutter margin in Writer, layout and UI

You can see how the gutter is on the right (not the left) and you can alter this behavior with a dedicated checkbox on the UI.

How is this implemented?

If you would like to know a bit more about how this works, continue reading… :-)

Finally I had a little bit of remaining time, so I addressed a request from the OASIS ODF TC:

Want to start using this?

Collabora intends to continue supporting and contributing to LibreOffice, the code is merged so we expect all of this work will be available in TDF’s next release (7.2).


Bibliography improvements in LibreOffice Writer

Estimated read time: 2 minutes

The bibliography feature in Writer allows authors of e.g. scientific papers to track sources: first you can insert bibliography entry fields, then at the end you can generate a bibliography table automatically.

Writer now has two improvements in this area: more information about these entries in the form of a mouse tooltip and clickable URLs in the table.

First, thanks TUBITAK ULAKBIM who made this work by Collabora possible.

Motivation

Generating a bibliography (table) for a document is a two-step process: first the user inserts bibliography entry fields to refer to sources, then later the bibliography can be auto-generated. It was not great that details of a source was only visible in the Writer edit window after generating the bibliography table. A somewhat related shortcoming was that URLs of sources were handled as plain text, while users expect that references are also clickable, similar to e.g. table of contents.

Results so far

This is how a tooltip looks like, available even if there is no bibliography table yet:

https://share.vmiklos.hu/blog/sw-bibliography/tooltip.png
Figure 1. Tooltip for bibliography entry fields

And URLs in the bibliography table are now clickable:

https://share.vmiklos.hu/blog/sw-bibliography/url.png
Figure 2. Clickable URLs in the bibliography table

How is this implemented?

If you would like to know a bit more about how this works, continue reading… :-)

As usual, the high-level problem was addressed by a series of incremental development steps:

Want to start using this?

Collabora intends to continue supporting and contributing to LibreOffice, the code is merged so we expect all of this work will be available in TDF’s next release (7.2).


Gutter margin in Writer

Estimated read time: 3 minutes

The page gutter defines the amount of extra space added to the specified margin (typically left), above any existing margin values.

Writer now has much better support for gutter margins: not only this margin type can be specified explicitly, it’s also possible to select if the gutter should be on the left or on the top, and it works with mirrored margins as well.

This work is primarily for Collabora Online, but the feature is fully available in desktop Writer as well.

Motivation

Word has a gutter margin feature, and we saw that some UI-level workaround appeared to have something similar based on the LibreOffice technology. We thought it’s much better to impelement this properly, so that the result is interoperable with Word, and also available both in Online and on the desktop.

Results so far

This is how the UI looks like in Online:

https://share.vmiklos.hu/blog/sw-gutter-margin/online.png
Figure 1. New UI in Online

This is how the new render result looks like:

https://share.vmiklos.hu/blog/sw-gutter-margin/new.png
Figure 2. New render result in Writer

Matching the reference rendering:

https://share.vmiklos.hu/blog/sw-gutter-margin/ref.png
Figure 3. Reference render result

While the gutter was just missing previously:

https://share.vmiklos.hu/blog/sw-gutter-margin/old.png
Figure 4. Old render result in Writer

You can see that the gutter margin is now handled properly.

How is this implemented?

If you would like to know a bit more about how this works, continue reading… :-)

As usual, the high-level problem was addressed by a series of incremental development steps:

  • First, we added a new "gutter" margin type (length) and a gutter position (left or top) to the document model & UNO API.

  • This was followed by adding the layout, which is always the most challenging part of a new Writer feature. In this case the basic functionality was implemented by reducing the "print area" of page frames, so that the positioning of header, body and footer frames all respect the gutter margin. Then additional care was taken to ignore the gutter margin for page borders to be compatible with Word. Finally the margin position and mirrored margin support was implemented.

  • Then we looked at import/export: this feature is now supported with the ODP, DOCX, DOC and RTF formats. Note that some best-effort handling was there in the DOC filter already, but the gutter margin was (semantically) lost on export.

  • You already saw the new UI above: this allows changing the amount of the gutter margin, specifying its position, finally the preview widget also takes the gutter margin into account. Note that this dialog is shared code between Writer, Calc and Impress: we made sure to not break those other apps with the extended UI, which is specific to Writer.

  • Brief help has been added for these new UI controls and submitting an ODF extension proposal to OASIS is in progress to standardize the so far LO-specific markup:

<style:page-layout-properties loext:margin-gutter="1.27cm">

With these, it’s now possible to add, modify, render and delete gutter margins for your documents, while preserving them during ODT and DOCX import/export, without introducing any weird fake line shapes.

Want to start using this?

You can get a snapshot / demo of Collabora Office and try it out yourself right now: try unstable snapshot. Collabora intends to continue supporting and contributing to LibreOffice, the code is merged so we expect all of this work will be available in TDF’s next release too (7.2).


Handling PDF digital signatures with PDFium FOSDEM talk

Estimated read time: 2 minutes

Figure 1. Slides of the talk

The next step in the recent PDFium-based signature verification story is my Handling PDF digital signatures in LibreOffice with PDFium talk at FOSDEM 2021, in the LibreOffice devroom (pre-recorded video). The talk gives you an overview of digital signing in general, all the ODF/OOXML/PDF handling, signing/verification, various other related past Collabora projects, and then goes into details regarding how PDFium was improved and is used to do a better PDF signature verification in LibreOffice when opening PDF files in Draw.

The virtual room had around 150 participants and the Matrix based online conference was well-organized. Speakers even got a free t-shirt before the event, I appreciated the "bring your own beer" joke :-)

An other benefit of this unusual setup was to avoid the dreaded room is full problems, where you carefully selected a talk to attend and then failed to hear it.

I expect quite some other slides from other Collaborans and the wider community will be available on Planet, don’t miss them.

Want to start using this?

You can get a snapshot / demo of Collabora Office and try it out yourself right now: try unstable snapshot. Collabora intends to continue supporting and contributing to LibreOffice, the code is merged so we expect all of this work will be available in TDF’s next release too (7.2).


Shadow for tables from PPTX in Impress

Estimated read time: 2 minutes

Impress now has much better support for the shadow of table shapes: not only shape styles can result in table shadows, but it’s also possible to add this as direct formatting. Also the shadow result is PowerPoint-compatible in the direct formatting case.

First, thanks to our partner SUSE for working with Collabora to make this possible.

Motivation

We got a PPTX document, which has a table shape with blurry shadow. The shadow was completely missing in Impress. It was discovered that in case you configure the default shape style to have shadow, then there is some initial table shadow support in Impress, but that was not used in the PPTX case.

The request was to improve the shadow rendering to be PowerPoint-compatible and in general support table shadows as direct formatting as a new feature.

Results so far

The table shadow now looks like this:

https://share.vmiklos.hu/blog/sd-table-shadow/new.png
Figure 1. New render result in Impress

Matching the reference rendering:

https://share.vmiklos.hu/blog/sd-table-shadow/ref.png
Figure 2. Reference render result

While shadow was just missing previously:

https://share.vmiklos.hu/blog/sd-table-shadow/old.png
Figure 3. Old render result in Impress

You can see that not only the shadow is there, but also the cell backgrounds and the blurry shadow is rendered properly.

How is this implemented?

If you would like to know a bit more about how this works, continue reading… :-)

As usual, the high-level problem was addressed by a series of small fixes:

With these, it’s now possible to add, edit, render and delete these table shadows, while preserving them during ODP and PPTX import/export.

Want to start using this?

You can get a snapshot / demo of Collabora Office and try it out yourself right now: try unstable snapshot. Collabora intends to continue supporting and contributing to LibreOffice, the code is merged so we expect all of this work will be available in TDF’s next release too (7.2).


Better PDF signature verification in Draw

Estimated read time: 2 minutes

Draw now has much better support for detecting unsigned incremental updates between signatures at the end of PDF documents. We now also make sure that incremental updates introduced for adding signatures really just add annotations and don’t change the actual content.

Motivation

There has been a recent evaluation of PDF signature verification, which included Draw. While we got a checkmark on their Shadow Hide test, their Shadow Replace test found conditional problems and their Shadow Hide-and-Replace test was not happy, either.

So time to look at what are those corner-cases and how the situation can be improved, so people keep trusting that if Draw says a signature is valid, it’s indeed valid.

Results so far

There were 4 incremental improvements in this area:

These were enough so that talking to the authors of that evaluation now confirmed that these problems are all gone.

How is this implemented?

If you would like to know a bit more about how this works, continue reading… :-)

PDF signature verification works by using a custom PDF tokenizer. You can read about that code in the PDF tokenizer section of this post. The bottom line is that we now have both PDFium and this custom tokenizer, somewhat duplicating the functionality.

After talking to the PDFium developers (see the relevant mailing list thread), there were open regarding adding all the high level API to allow PDF signature verification based on PDFium, and not via our own tokenizer. See this header file for the set of relevant APIs added. A combinations of those allowed to adapt the code on our side and use PDFium for signature verification, not the own tokanizer.

Want to start using this?

You can get a snapshot / demo of Collabora Office and try it out yourself right now: try unstable snapshot. Collabora intends to continue supporting and contributing to LibreOffice, the code is merged so we expect all of this work will be available in TDF’s next release too (7.1).


Effecting code review and backporting for Collabora Online

Estimated read time: 3 minutes

Collabora Online now has a ./g script that tries to bring some of the Gerrit-based review benefits to a workflow based on GitHub.

Motivation

Collabora Online is on GitHub, but core.git is still on Gerrit, so it made sense to spend some time on a small shell script that gives you review and backport experience that is closer to Gerrit than the stock GitHub workflow.

How we use GitHub

Most Online committers push their code for review directly to online.git, to private namespaced branches, like private/kendy/master, then a pull request can be created to get commits from that branch into master after CI, review, etc. This workflow has the benefit that you don’t have to deal with the complexity of multiple repos.

Next to master, there are distro branches like distro/collabora/co-6-4, we may or may not want to backport the contents of a PR to that stable branch.

It’s important that Gerrit used to have a git review command to just submit your changes for review, without asking anything. That explains why the stock GitHub workflow where you need to name the source branch of your PR feels unnecessary complexity. Creating the PR by visiting a webpage is again something we want to avoid. Not to mention open questions like should you delete your source branch after a PR is merged? Should you delete your source repo?

On the other hand, we’re interested in GitHub’s ability to have multiple commits in a PR: Gerrit forces to have one commit per change. The GitHub way encourages developers to split changes into more commits, now that the review and CI cost won’t increase just due to such splits.

Submit for review

The happy path is when you have one or more local commits and you want to submit it for review. In this case now you can do:

./g review

And the script will figure out that you want to push your local branch to a remote branch like private/kendy/master and also create a pull request for you, printing its URL.

In case that branch already exists then you need to specify a name:

./g review myhack

So parallel reviews are possible, but only the first gets an inferred name. Both cases need no clicking in a browser, thanks to gh.

Submit a backport

The easiest case is when you can assume that the master branch and the distro branch is so close to each other that there won’t be conflicts to be resolved. In that case, you can do:

./g backport distro/collabora/co-6-4 790

to pick all commits from PR 790 (which is already merged to master) to a distro branch.

Again, you can have multiple backports in progress, e.g. you can do:

./g backport distro/collabora/co-6-4 790 myhack

If the default name is already used. The backport syntax is a bit longer, so you can always just type ./g backport and you’ll get the usage.

This second command is a bit more complicated, as gh has no trivial way to expose what is the commit range of a PR. But there is gh api graphql which can do arbitrary GraphQL queries, which provide this information. At this stage it may make sense to just rewrite the whole ./g script in e.g. Python, but till that happens, we parse the output of the query using jq.

Finally, if you do have conflicts or you want a local build test / manual test before submitting, you can always check out the distro branch manually, cherry-pick there and use plain ./g review to submit your backport for review.

Want to start using this?

You can go to the Collabora Online community website and see how to build the code. Then you may want to solve an easy hack, finally submit your commit for review either by using the above method or whichever way you prefer contributing to GitHub projects.

© Miklos Vajna. Built using Pelican. Theme by Giulio Fidente on github.