Index ¦ Archives ¦ RSS

Multi-page floating tables in Writer: wrap on all pages

Estimated read time: 3 minutes

This post is part of a series to describe how Writer now gets a feature to handle tables that are both floating and span over multiple pages.

This work is primarily for Collabora Online, but is useful on the desktop as well. See the 8th post for the previous part.

Motivation

Multi-page floating tables always wrapped their anchor text only on the last page, to be compatible with Word's default behavior. There is a special flag in DOCX files to wrap on all pages, though. In this part, you can read about handling of this flag in Writer.

Results so far

Regarding testing of the floating table feature in general, the core.git repository has 84 files now which are focusing on correct handling of floating tables (filenames matching floattable-|floating-table-). This doesn't count cases where the document model is built using C++ code in the memory and then we assert the result of some operation.

Here are some screenshots from the fixes this month:

Old, new and reference rendering of a 3 nested, multi-page floating tables

The first screenshot shows a case where multi-page floating tables are nested. For this document, we not only have an inner and an out table, but we also have a middle one, giving us 3 nesting tables. Some of the inner table frames had a bad position, leading to overlapping text, now fixed.

Old, new and reference rendering of wrapping on all pages

The next screenshot shows the case where the magic allowTextAfterFloatingTableBreak flag is set. We used to wrap content of the anchor only on the last page, unconditionally. Now we either wrap on the last page (default) or on all pages (when this flag is present).

Old, new and reference rendering of overlapping floating tables.

The last screenshot shows a document full of floating tables. These used to be inline ones, and then they could not overlap by definition, but now extra effort was needed to position them in a way that no overlap happens between the tables. Now our render result matches Word.

How is this implemented?

If you would like to know a bit more about how this works, continue reading... :-)

As usual, the high-level problem was addressed by a series of small changes:

Want to start using this?

You can get a snapshot / demo of Collabora Office 23.05 and try it out yourself right now: try the unstable snapshot. Collabora intends to continue supporting and contributing to LibreOffice, the code is merged so we expect all of this work will be available in TDF's next release too (24.2).


Multi-page floating tables in Writer: nested tables

Estimated read time: 4 minutes

This post is part of a series to describe how Writer now gets a feature to handle tables that are both floating and span over multiple pages.

This work is primarily for Collabora Online, but is useful on the desktop as well. See the seventh post for the previous part.

Motivation

Multi-page floating tables and nested tables could not be combined so far. Instead we tried to lay out the outer table on multiple pages and all inner tables were still limited to a single page. In this part, you can read about removing this limitation.

Results so far

Regarding testing, the core.git repository has 73 files now which are focusing on correct handling of floating tables (filenames matching floattable-|floating-table-). This doesn't count cases where the document model is built using C++ code in the memory and then we assert the result of some operation.

Here are some screenshots from the fixes this month:

Old, new and reference rendering of a floating inner table in a floating outer table from DOCX.

The first screenshot shows a case where not only a single floating table is split across pages, but also the table's only cell hosts an inner multi-page floating table. This is more complicated at a layout level, because we can't just move part of the table to a next page, which has no parts of the outer table yet.

Old, new and reference rendering of nested, multi-page floating table at cell start.

The next screenshot shows a case where the inner floating table starts at the cell start of the outer table. The DOCX import case needed addition effort to get working, because as soon as the inner floating table's content is moved to the floating frame from the body text, the reference to the cell start at the outer table level was invalidated and the whole table conversion failed.

Old, new and reference rendering of multiple inner floating tables starting at cell start of an outer floating table.

This screenshot shows a bug where multiple inner floating tables started at the cell start of the outer table. Handling this correctly on our end had to ensure that each (potentially multi-page) floating table gets its own unique anchor point, and this can be combined with nesting, with the "at cell start" special case on top of it. Once all three can be combined, a real-world documents gets its table instead of just having the content the of the table in the body text.

Old, new and reference rendering of moving the table start to a new page.

The last screenshot shows an editing session where we keep inserting new paragraphs at the start of the document, so new items are created at the end of the fly chain as needed, and old items are deleted at the start of the fly chain as needed. We had a bug here so that the content we wanted to move forward was inserted at the end of the document, leading to a weird selection that started at the document end and continued on a previous page. Now this works correctly: once the first page is full of empty paragraphs, the second page hosts the first row and half of the second row. The third page then can host the second half of the second row and the third row.

How is this implemented?

If you would like to know a bit more about how this works, continue reading... :-)

As usual, the high-level problem was addressed by a series of small changes:

Want to start using this?

You can get a snapshot / demo of Collabora Office 23.05 and try it out yourself right now: try the unstable snapshot. Collabora intends to continue supporting and contributing to LibreOffice, the code is merged so we expect all of this work will be available in TDF's next release too (24.2).


Multi-page floating tables in Writer: overlap control, border and footnotes

Estimated read time: 4 minutes

This post is part of a series to describe how Writer now gets a feature to handle tables that are both floating and span over multiple pages.

This work is primarily for Collabora Online, but is useful on the desktop as well. See the sixth post for the previous part.

Motivation

The current post features sub-tasks for the multi-page floating table work that is around an explicit table overlap control that Word has (and Writer lacked so far), compatible border rendering of split tables and having footnotes in floating tables, which was not working previously.

Results so far

Regarding testing, the core.git repository has 60 files now which are focusing on correct handling of floating tables (filename matching floattable-|floating-table-). This doesn't count cases where the document model is built using C++ code in the memory and then we assert the result of some operation.

Here are some screenshots from the fixes this month:

Old, new and reference rendering after expanding an autotext.

The first screenshot shows a case where the anchor paragraph of a floating table had some autotext (e.g. "dt", which stands for dummy text), and pressing the relevant shortcut (F3) expands that autotext with the actual content. This includes changing the anchor position of the floating table, which lead to overlapping text. (A multi-page floating table has multiple anchors, we have to make sure we don't set all of them to the new value as-is.)

Old, new and reference rendering of tables with the overlap=never markup.

The next screenshot shows a case where two tables are positioned in a way that they would overlap. Word has a flag that asks the layout to still re-position the second table so the overlap doesn't happen, and now Writer supports this as well.

Old, new and reference rendering of duplicated anchor text.

This screenshot shows a bug where the anchor text on the first page was also duplicated on the second page. Now we properly start the anchor text on the last page of the floating table, like Word does.

Old, new and reference rendering of a multi-page floating table with borders.

What you can see is a floating table that has 2 pages, but simply a split of the table would result in no bottom border on the first page and no top border for the second, like perhaps you would expect it, matching Word. This is now fixed, the layout infers the border style in those cases correctly.

Old, new and reference rendering of a footnote in a floating table.

The last screenshot shows a mini-feature: it was possible to float tables and to have footnotes in tables, but not both at the same time. The screenshot shows a case where a floating table is needed, so a specific paragraph is above the table. But we couldn't float the table, because it had a footnote and that would be lost as-is. Now you can have a correct position for that paragraph and the footnote is there as well, at the same time.

How is this implemented?

If you would like to know a bit more about how this works, continue reading... :-)

As usual, the high-level problem was addressed by a series of small changes:

Want to start using this?

You can get a snapshot / demo of Collabora Office 23.05 and try it out yourself right now: try the unstable snapshot. Collabora intends to continue supporting and contributing to LibreOffice, the code is merged so we expect all of this work will be available in TDF's next release too (24.2).


Multi-page floating tables in Writer: hidden anchors and more

Estimated read time: 5 minutes

Writer now has continued steps to handle tables that are both floating and span over multiple pages.

This work is primarily for Collabora Online, but is useful on the desktop as well. See the fifth post for background.

Motivation

The previous post finished with an "almost done" state for overlapping tables: this is a cluster of problems where tables are allowed to overlap, but various other formatting make them not overlap in practice in Word, but they do overlap in Writer. In this post, we'll see what was necessary so an initial set of old documents now render perfectly, which started to work during the past month.

Results so far

The feature is enabled by default and the DOCX/DOC/RTF import makes use of it.

On the positive side, core.git repository has has 49 files now which are focusing on correct handling of floating tables (filename matching floattable-|floating-table-). Also, there are additional tests that quickly build a specific multi-page floating table in the memory and do some operation on it, e.g. delete the last row and assert what happens.

Here are some screenshots from the fixes this month (right click, open image in new tab for larger size):

Old, new and reference rendering of a floating table with negative vertical position, relative to an empty anchor.

The first case is about a document where the bottom of the page had a floating table, where the position (relative to the anchor) was a negative vertical offset. In this case Writer used to move the floating table to the next page, but now matches Word: the space in the previous page is used for the floating table. This fixed overlapping text on the next page.

Old, new and reference rendering of a floating table with a large bottom margin in the anchor paragraph. Notice how the old rendering had a larger spacing between footnotes, which resulted in a 2 page document. The new and reference renderings have smaller spacing between footnotes and are of 1 page.

The next case is about floating tables in footnotes, the anchor needs to have no visible margins to provide the correct layout.

Old, new and reference rendering of a floating table with an overlap due to vertical offsets. The old rendering had a big second table, it completely covered the first table. The new and reference renderings shift the second table to a next page to avoid overlap.

This case was about an unwanted overlap of floating tables. Incorrect handling of the vertical offsets meant that the second floating table was rendered on top of the first one, making the text in the first floating table unreadable. Now we shift down the second floating table, so no overlap happens.

New rendering of a floating table with anchor text that starts with a newline.

This document had a layout loop on load. The problem was specific to the case when the anchor text of a floating table started with a newline character, which has its own position in the document model, but doesn't really have a width at a layout level, so needs special handling. This is now fixed, the document loads and renders fine.

Old, new and reference rendering of a document with lots of floating tables.

This document had only 4 pages, but lots of floating tables, carefully positioned to not overlap in Word. You can see how the old Writer rendering result was hard to read and it now looks correct.

Old, new and reference rendering of a floating table with a hidden anchor.

Writer used to hide floating tables anchored inside hidden paragraphs, but Word shows them all the time, now we match this behavior.

Old, new and reference rendering of a floating table with an overlap.

The next document first had all of its tables in the first page, now fixed.

Old, new and reference rendering of a floating table with an overlap between the table and the footer area.

A remaining problem was that the second table and the footer area could still overlap. This is now fixed: Word sometimes does allow such an overlap, but it depends on the baseline of the anchor position, so in the current case we can detect that such an overlap is not wanted.

Old, new and reference rendering of a floating table with a half row on the first page.

The last problem with this document was a poor split of the floating table, the first half row of the second table still went to the first page. Now we correctly detect that such a split is not wanted and simply start the second floating table on the second page, which results in a pretty good rendering of this document.

And that's where we stand. Certainly more work is needed to fix rough edges, but we get there step by step -- the theme is slowly transitioning from overlap problems right after load to unexpected rendering problems during editing.

How is this implemented?

If you would like to know a bit more about how this works, continue reading... :-)

As usual, the high-level problem was addressed by a series of small changes:

Want to start using this?

You can get a snapshot / demo of Collabora Office 23.05 and try it out yourself right now: try the unstable snapshot. Collabora intends to continue supporting and contributing to LibreOffice, the code is merged so we expect all of this work will be available in TDF's next release too (7.6).


Multi-page floating tables in Writer: from section breaks to negative vertical offsets

Estimated read time: 5 minutes

Writer now has continued steps to handle tables that are both floating and span over multiple pages.

This work is primarily for Collabora Online, but is useful on the desktop as well. See the fourth post for background.

Motivation

The previous post finished with a first fix for overlapping tables: this is a cluster of problems where tables are allowed to overlap, but various other formatting make them not overlap in practice in Word, but they do overlap in Writer. In this post, we'll see what started to work during the past month.

Results so far

The feature is enabled by default and the DOCX/DOC/RTF import makes use of it. This allows stress-testing the layout code with complex user documents. The next target is to actively search for documents that rendered reasonably in the past but are now unreadable and fix them.

On the positive side, core.git repository has has 40 files now which are focusing on correct handling of floating tables (filename matching floattable-|floating-table-). Also, there are additional tests that quickly build a specific multi-page floating table in the memory and do some operation on it, e.g. delete the last row and assert what happens.

Here are some screenshots from the fixes this month:

Section break directly between floating tables

The first problem was that in case two floating tables were directly after each other, a section break between them was lost. What you can see above is a successful transfer of the break properties to the first paragraph on the next page, so the tables don't overlap but are laid out on separate pages.

Incorrect split of a multi-page floating table

Looking through internal documents, a sample was found where a table with 2 rows was not split, even if the previous page would have enough space for the first row. This is the old, bad layout.

Correct split of a multi-page floating table

And this is how it looks after the fix.

2 floating tables are lost

The rest of the screenshots are from a complex internal document, which has 4 pages, but in total contains 11 floating tables. First I focused on getting correct rendering of this in Writer for DOCX. All the problems were not visible previously, as non-floating tables can't overlap, so bad formatting problems did not cause problems in practice. The first subset (public reproducer) had 3 tables in it, but Writer only rendered one of them.

Normal table and 2 floating tables are fixed

After fixing the problem, both lost tables are now there.

Unwanted overlap of two floating tables

The next case is about overlapping tables: the table is there, but the anchor positions are so close that the content overlaps and it's hard or impossible to read the text.

Fixed overlap of two floating tables

And this is how it looks after the fix.

Bad handling of nested floating tables

The next trouble is with nested floating tables: in case all inner tables are inline, then the table above the images will require more space then requested and that will lead to overlapping text later.

Better handling of nested floating tables

The first fix is to make these inner tables also float if requested in the file format (but not split for now, to develop incrementally). This helps, but the position of the inner table's anchor is still incorrect.

Good handling of nested floating tables

Finally the anchor position of the table is also correct: no intersecting borders, no overlapping text, no unexpected need for lots of vertical space.

Layout-loop for a floating table

The next problem was a layout loop, which was terminated by the built-in loop control in a way that rendered text outside the table to show up inside the table.

Fixed layout-loop for a floating table

And here is the correct rendering. Note that in case a floating table's anchor is positioned with a negative vertical offset, then it can happen that text before the table will be rendered below the table. But this is intentional and also happens in Word.

Bad handling of negative vertical offsets

The last problem for this blog post is the case when a large floating table is anchored in a single-line paragraph. Now Writer insists that the anchored object and the anchor text is on the same page, so even if there is space for the last table on a previous page, it moves the table to the next page.

Good handling of negative vertical offsets

And here is the correct rendering: all tables fitting the first page. Also note that the last table is moved up, so while the document model order is "after-table-2-second table-3 after-table-3", the rendered / layout order will be "table-3 after-table-2-second after-table-3" -- again, intentionally.

And that's where we stand. Certainly more work is needed to fix some remaining unwanted overlapping of floating tables, but we get there step by step.

How is this implemented?

If you would like to know a bit more about how this works, continue reading... :-)

As usual, the high-level problem was addressed by a series of small changes:

Want to start using this?

You can get a snapshot / demo of Collabora Office 23.05 and try it out yourself right now: try the unstable snapshot. Collabora intends to continue supporting and contributing to LibreOffice, the code is merged so we expect all of this work will be available in TDF's next release too (7.6).


Multi-page floating tables in Writer: from multiple columns to chaining

Estimated read time: 5 minutes

Writer now has continued steps to handle tables that are both floating and span over multiple pages.

This work is primarily for Collabora Online, but is useful on the desktop as well. See the third post for background.

Motivation

The previous post finished with crash testing: the interesting subset of that testing tool is to take hundreds of thousands of documents and in the Writer case import them into a document model and layout them. If any of this crashes, mark that for future investigation. In this post, we'll see what else started to work during the past month.

Results so far

The feature is enabled by default and now the DOCX/DOC/RTF import makes use of it if. This allows stress-testing the layout code with complex user documents, hopefully with the found breakage fixed before it would be released in a stable version.

On the positive side, core.git repository has has 37 files now which are focusing on correct handling of floating tables (abbreviated as "floattables"). Also, there are additional tests that quickly build a specific multi-page floating table in the memory and do some operation on it, e.g. delete the last row and assert what happens.

Here are some screenshots from the effort so far:

Floating table inside a multi-column section

The first case is about multi-column sections: in this case Word doesn't try to split them between pages. What you can see on the screenshot is that Writer lays out content on the previous page so that remaining space is left, but we don't try to split the table between the first and the second page, even if there would be space on the first page and even if this means the table overlaps with the second column, matching what Word does.

UI to disable split of a floating table

UI to enable split of floating tables were added quite early: this is a new checkbox on the frame properties dialog. However, disabling the split of floating tables was broken, the already created layout was not updated to properly move back "follow" fly frames from later pages to the current page, which is now fixed.

Chaining enabled, so no split frames

Writer already had a feature to split content in a frame into multiple frames, but that one required creating those frames in the model explicitly, such chaining is a feature that is useful in other use-cases and is parallel to multi-page floating tables. The UI now ensures that the user can split frames only in case chaining is not used, to avoid confusion.

Split enabled, so no chaining

This is now also true in the other way around: if split of a floating table is allowed, then we disable the frame chaining UI to avoid trouble later.

The latest crashing document

At this point I went back to crashtesting & crash bugreports, and the latest reported crash was for a document that is visible on the above screenshot. This was a bit tricky: it required 3 fixes to make it not crash and also a layout loop fix.

Disabling split of frames at a layout level

Next was a mini-feature: even if floating tables normally split across pages by default, Word has a document-level compatibility switch to turn this split on or off by default, at a layout level. Floating tables from RTF are not split by default, DOC and DOCX split them by default.

What you can see on this screenshot is that a DOCX document may have this flag enabled, and then you allow splitting on the UI / at a document model level, but Writer may still decide to not split them, to provide the correct layout.

Overlapping floating tables

The previous post already mentioned the problem area of overlapping tables. A first step in this direction is to fix this bug document with Arabic text and 2 overlapping tables, making them unreadable.

Fixed overlap of floating tables

And in this case here is the fixed version, where reading the table now depends on your language skills. :-)

In this case, the problem was a lost section break of type next page, when a section started with a floating table, which is a corner-case.

And that's where we stand. Certainly more work is needed to fix more unwanted overlapping of floating tables, but we get there step by step.

How is this implemented?

If you would like to know a bit more about how this works, continue reading... :-)

As usual, the high-level problem was addressed by a series of small changes:

Want to start using this?

You can get a snapshot / demo of Collabora Office 23.05 and try it out yourself right now: try the unstable snapshot. Collabora intends to continue supporting and contributing to LibreOffice, the code is merged so we expect all of this work will be available in TDF's next release too (7.6).


Multi-page floating tables in Writer: from row deletion to page breaks

Estimated read time: 6 minutes

Writer now has continued steps to handle tables that are both floating and span over multiple pages.

This work is primarily for Collabora Online, but is useful on the desktop as well. See the second post for background.

Motivation

The previous post finished with cursor traversal: if a floating table is on both page 1 and page 2, then you expect Writer to be able to move between the rows of the table, even if those are not on the same page. In this post, we'll see what else started to work during the past month.

Results so far

The feature is enabled by default and now the DOCX/DOC/RTF import makes use of it if. This allows stress-testing the layout code with complex user documents, hopefully with the found breakage fixed before it would be released in a stable version.

On the positive side, core.git repository has has 19 files now which are focusing on correct handling of floating tables. Also, there are additional tests that quickly build a specific multi-page floating table in the memory and do some operation on it, e.g. delete the last row and assert what happens.

Here are some screenshots from the effort so far:

Editing of floating tables: delete the last row of a table with 3 rows

The first case is about editing: if a floating table had a first, middle and last page, then deleting the last row of a table lead to incorrect layout, which is now fixed.

Selection & dragging of split floating tables

An odd problem is that the vertical position of tables on non-first pages is generated by the layout, which means that normal drag&move to position them won't work, leading to annoying jumps. This is now fixed by selecting the first (master) fly frame on click, and you can always reposition that table (even vertically.)

Bad binary DOC import

Once DOCX import/export was there, the next step is binary DOC import, which gives us access to a larger corpus of test documents, to stress-test the layout code. This shows how the binary DOC import looked before the work.

Good binary DOC import

And this one shows how it works now.

Good binary DOC export

DOC import is not enough, e.g. Collabora Online will save your documents automatically, so we really want to export everything that is possible to import. Here is how good DOC export looks like in Word.

In-footer floating table

At this point the first crashtest results arrived (we try to import about 280 thousand documents and see what crashes). The first problem was floating tables in footers. Well, we should not try to split such tables (even if they don't fit): adding one more page does not give us more footer space.

Bad RTF import

Similar to the DOC filter, RTF can express floating tables. Here is how we did a bad rendering of an RTF document before.

Good RTF import

And here is how we import it currently. The RTF control words are quite close to the binary DOC markup semantically, just the syntax is different.

Bad RTF export

The RTF export side was also missing, as visible in Word, before the work.

Good RTF export

And this is how the good RTF export result looks like in Word.

Floating table in a section

Another crashtest find was that sometimes we map Word's continuous section breaks to Writer sections, so we can't assume that tables are anchored directly in body frames. This is now fixed.

Correct handling of the TableRowKeep flag in floating tables

A related problem was that non-floating tables have a trick, that we call the TableRowKeep mode. If this is on (which is the default for documents imported form Word), a table row will stick to the next table row (we try to keep them on the same page) if the first cell's first paragraph in that row has the "keep with next" paragraph property specified. It turns out, this should be ignored when the table is floating.

Page break before a floating table

A next problem was that some page breaks simply disappeared. It turns out that we need to transfer the "break before" property from the table to the table anchor (paragraph) to get the desired layout, since page breaks are generally ignored inside text frames.

Handling of 2 times nested tables, middle one is floating

All combinations of nesting with floating tables is not yet handled, but at least we should not crash when the user tries to do that. Here is 3 tables, nested in each other, the second table is marked to be floating.

Handling of a floating table, immediately followed by an other table

The last fixed problem is when a floating table is immediately followed by an other, non-floating table. Given that we try to anchor the floating table in the next paragraph, the layout could not handle this previously, but now we ensure that each floating table is followed by a paragraph.

And that's where we stand. Hope to address all problems reported by crashtesting soon. Once that happens, it may be possible to switch from bugfixing mode to feature mode again, e.g. better handling of overlapping or nested tables could be done.

How is this implemented?

If you would like to know a bit more about how this works, continue reading... :-)

As usual, the high-level problem was addressed by a series of small changes:

Want to start using this?

You can get a snapshot / demo of Collabora Office 23.05 and try it out yourself right now: try the unstable snapshot. Collabora intends to continue supporting and contributing to LibreOffice, the code is merged so we expect all of this work will be available in TDF's next release too (7.6).


Multi-page floating tables in Writer: from split rows to cursor traversal

Estimated read time: 6 minutes

Writer now has the early steps to handle tables that are both floating and span over multiple pages.

This work is primarily for Collabora Online, but is useful on the desktop as well. See the first post for background.

Motivation

The previous post finished with split rows are now in a reasonable shape towards our journey to fix tdf#61594. In this post, we'll see what else is needed to get perfect rendering for that single document.

The plan is to iterate on that later, adding more and more incremental improvements & fixes for this feature.

Results so far

The feature is still enabled by default, but the DOCX import only makes use of it if you set the SW_FORCE_FLY_SPLIT=1 environment variable. This allows playing with the feature even if there are lots of known problems still.

On the positive side, core.git sw/qa/core/layout/data/ has 12 files now which are rendered exactly the way Word does. Also, there are additional tests that quickly build a specific multi-page floating table in the memory and do some operation on it, e.g. delete the last row and assert what happens.

Here are some screenshots from the effort so far:

Split row and an additional one on 2 pages

Here the problem was that a normal row went to a next page after a split row. Now the document is correctly of 2 pages, instead of the previous unwanted 3 pages.

Floating table with multiple columns

Here the additional complexity was to have multiple columns on a table, since previously we always had 1 column and 2 or more rows. Now these are also split correctly across pages.

Incorrect widow control inside split floating tables

This is an incorrect table row split, because widow control is broken.

Fixed widow control inside split floating tables

And here is how it looks when it's working. That little line on page 2 is no longer alone.

Working minimal height

Even better when the minimal height for non-first ("follow") table frames is working, as you can notice that space between the last line and the table bottom border on page 2.

At this point, the bug document from the motivation section worked fine, apart from the workaround that one has to re-save it in non-legacy mode in Word. So what's next? We need to instantly add a legacy mode for the brand new (not even fully enabled) multi-page floating table feature, since otherwise whatever we do, some DOCX files will be handled incorrectly.

Legacy mode: bad margin

As it turns out, the core of the legacy mode is that the floating table is sometimes allowed to flow into the footer / bottom margin area of the page, but not always. It's quite inconsistent, so one can understand why this is no longer the default behavior. The above is the naive rendering, which is logical, but incorrect.

Legacy mode: good margin

And this is the correct result in legacy mode. After a bit of experimenting, it seems one can flow into the bottom margin area if the height of the table frame would fit the body frame, but some vertical offset causes it to be pushed down.

Legacy mode: minimal row height causes no row split

The final trick with legacy mode is to make sure that all tables (first one, middle ones, last one) have the required minimal height, which can result in not splitting the row in case a part of that would be less than the minimal height. E.g. a 3 cm minimal height means that a total height of 4 cm (2cm + 2cm) is not enough for a split row.

With this, we reached the goal to render that given bug document perfectly (when compared to Word), and the next step is to fix up breakage that would be caused by enabling by default.

Tracked changes in floating tables

The first problem was tracked changes support, which needs special care: as the importer converts body text to table cells, we need to keep the tracked insert/delete text ranges correctly. This is now working fine.

Nested tables: the outer is floating

The next problem is around nested tables: a normal inner table inside a floating table was lost on DOCX file open, now fixed.

Nested tables: broken inner floating table

The other version is when a normal table has an inner floating table. This broke badly, the outer table was not imported at all.

Nested tables: better inner floating table

And it's now better. The inner table is still not actually floating, but turns out that was never working for DOCX files, so it's not a regression. Fine to revisit that only later.

Follow table: bad horizontal positioning

So far all the previous tables were aligned to the left. It turns out that the horizontal positioning was bad in every other case for non-first tables, e.g. when you wanted to center them.

Follow table: good horizontal positioning

And it's now fixed.

As a last fix for this post, let's look at traveling with the cursor:

Good cursor traversal

After fixing this, now you can use the up/down arrows to go from the A1 cell to A2 and back. The cursor traversal code wasn't aware that the master/follow table frame was connected.

And that's where we stand. Hope to enable even the DOCX import bit by default soon.

How is this implemented?

If you would like to know a bit more about how this works, continue reading... :-)

As usual, the high-level problem was addressed by a series of small changes:

Want to start using this?

You can get a snapshot / demo of Collabora Office 23.05 and try it out yourself right now: try the unstable snapshot. Collabora intends to continue supporting and contributing to LibreOffice, the code is merged so we expect all of this work will be available in TDF's next release too (7.6).


Start of multi-page floating tables in Writer

Estimated read time: 5 minutes

Writer now has the early steps to handle tables that are both floating and span over multiple pages.

This work is primarily for Collabora Online, but is useful on the desktop as well.

Motivation

As requested in tdf#61594 10 year ago, the use-case is that you can already have floating tables:

Table in a Writer text frame

And multi-page tables:

Multi-page table

And what we want is a combination of them, like this:

Multi-page floating table

This is a quite complicated feature, since both floating objects and tables are complex, and this combines them to create even more complexity.

However, such constructs are used in existing DOCX files and we're expected to correctly display them.

Results so far

The feature is enabled by default, but the DOCX import only makes use of it if you set the SW_FORCE_FLY_SPLIT=1 environment variable. This allows playing with the feature even if there are lots of known problems still.

On the positive side, core.git sw/qa/core/layout/data/ has 4 files now which are rendered exactly the way Word does.

A bit of terminology: once a frame is split, the first element of the chain is called master, the remaining frames are called follows.

Here are some screenshots from the journey so far:

Not splitting Writer text frame

This is a fly frame with enough content that it doesn't fit the body frame. It should split, but fly frames could not be split.

Writer text frame kept inside the body frame

First try, just limit the height of the (master) fly frame, so at least it stays inside the body frame. But now some content is not rendered.

Incorrect split of a text frame

Next try. Now have have 2 flys, but the second has zero height and the content of the second fly leaks into the body of the second page.

Last version with bad anchoring

This one is better, but the position of the follow fly frame is bad, no actual wrapping happens. Also, we assume that there are multiple paragraphs after the table, which will cause problems for floating tables at the end of the document. So I reworked the anchoring code to split the anchor to as many pages as necessary...

Duplicated anchor text

Which sounds good, but now the text around the anchor point is duplicated.

Less duplicated anchor text on the first page

Better, now the anchor text is gone in the master anchor, but still there is a misleading paragraph marker.

Last text frame without a table

And now this looks reasonable. Fine, we have some minimal split flys, let's try it with tables instead of just two paragraphs:

Floating table with duplicated anchor text

With a bit of work, the table's two rows can split, but again the text in the anchor is duplicated.

Bad horizontal position

Next try, now the anchor text is correct, but the horizontal position of the table is still bad, it bleeds out towards the left margin area.

Fixed horizontal position

And with more work, now this looks correct.

Fixed vertical position

Let's add some vertical offset! That should be only applied on the first page, and now the follow fly doesn't have that unwanted offset.

Now we have 2 documents that lay out correctly on 2 pages. Let's try 3 pages:

Wanted 3 pages, have 2 pages

This falls apart, the 2nd and the 3rd row are both on page 2.

Correctly rendered 3 pages

After partitioning the fly frames to 3 categories (master, non-last follows, last follow), more than 2 pages also work.

Row split is not performed at all

This is a sample where the table has a single cell, so we need to split the (only) row, not just split the table's rows. The first is harder. Currently we don't even try to split it.

Row split is performed, but the 2nd page's object has a bad position

Next try, now we split it, but the position of the follow fly is wrong.

Row split with correct object positioning on all pages

Finally split of a single row inside multi-page floating tables also work. That's where we are. Don't try to do anything too custom (like inserting a header or footer), those cases are still known-broken.

How is this implemented?

If you would like to know a bit more about how this works, continue reading... :-)

As usual, the high-level problem was addressed by a series of small changes:

The design of the layout representation is documented in the SwFormatFlySplit constructor.

Want to start using this?

You can get a snapshot / demo of Collabora Office 23.05 and try it out yourself right now: try the unstable snapshot. Collabora intends to continue supporting and contributing to LibreOffice, the code is merged so we expect all of this work will be available in TDF's next release too (7.6).


Citation handling: plumbing in Writer for e.g. Zotero

Estimated read time: 5 minutes

Writer now has a set of new automation commands and APIs that allow clients to build user interface for citation handling that's more advanced than the default in-Writer bibliography support.

This work is primarily for Collabora Online, see the CODE release notes for one possible way to use this.

Motivation

Citations and bibliography in Writer, using fieldmarks

Users frequently using scientific citations are probably familiar with the limits of Writer's built-in bibliography support, and solutions like Zotero appeared (with a LibreOffice extension included) to improve that situation.

This means that instead of storing all your scientific notes and data locally, you can store them on a Zotero server, then work with that from anywhere, once you provide your credentials.

The trouble comes when you want to combine this with collaborative editing, which is provided by Online, but you can't use the extension made for the desktop.

The above CODE release notes explains how an end user can use this feature, this post is meant to document what new UNO commands and LOK APIs I added that serve as a backend for this. Especially the UNO commands are also useful in other contexts, like in macros or other extensions.

Results so far

Zotero can store citations using 3 markups in documents: fields (DOCX only), bookmarks (DOCX and ODT) and finally reference marks / sections (ODT only). The added plumbing allows several operations for all 3 cases, to work with existing documents using any of these markups.

The citation and the bibliography is handled the same way for fields (Writer's fieldmarks) and bookmarks. The last case uses reference marks for citations, but sections for the bibliography.

The following operations are supported:

  • create the citation / bibliography

  • read the object under the cursor

  • read all objects of a given type in the document

  • update the object under the cursor

  • update all objects of a given type in the document

  • delete all objects of a given type in the document

Reading is only available to LOK clients, you need to call the getCommandValues() API. The rest is normal UNO commands that you can invoke from document macros or extensions as well.

The added plumbing is the following:

Operation Fieldmark Bookmark Refmark Section
Create .uno:TextFormField .uno:InsertBookmark .uno:InsertField .uno:InsertSection
Read getCommandValues(".uno:TextFormField") getCommandValues(".uno:Bookmark") getCommandValues(".uno:Field") None
Read all getCommandValues(".uno:TextFormFields") getCommandValues(".uno:Bookmarks") getCommandValues(".uno:Fields") getCommandValues(".uno:Sections")
Update .uno:UpdateTextFormField .uno:UpdateBookmark .uno:UpdateField None
Update all .uno:TextFormFields .uno:UpdateBookmarks .uno:UpdateFields .uno:UpdateSections
Delete all .uno:DeleteTextFormFields .uno:DeleteBookmarks .uno:DeleteFields .uno:DeleteSections

How is this implemented?

If you would like to know a bit more about how this works, continue reading... :-)

As usual, the high-level problem was addressed by a series of small changes:

Want to start using this?

You can get a snapshot / demo of Collabora Office 22.05 and try it out yourself right now: try the unstable snapshot. Collabora intends to continue supporting and contributing to LibreOffice, the code is merged so we expect all of this work will be available in TDF's next release too (7.6).

© Miklos Vajna. Built using Pelican. Theme by Giulio Fidente on github.