GtkSourceView/Buffer crash: Gtk-ERROR: Byte index is off the end of the line

Question

I have been working on text editor for writing stories. I am using Python, GTK+ 3 and GtkSourceView 3. The main point of the editor is folding certain regions. While there is no - not yet? - built-in support for folding in GTK TextView / SourceView, I have been using tags with invisible=True and SourceView's source marks to implement the feature.

Source code available here: https://github.com/mkoskim/mawe

The core editor (SourceBuffer and SourceView) is located here: https://github.com/mkoskim/mawe/blob/master/gui/gtk/SceneView.py https://github.com/mkoskim/mawe/blob/master/gui/gtk/SceneBuffer.py

For testing purposes, you can clone the repo, and run the app with:

mawe$ ./mawe.py test/test.txt

Now, the application keeps frequently and randomly crashing to errors like this:

(mawe.py:10556): Gtk-WARNING **: /build/gtk+3.0-2Ut_nl/gtk+3.0-3.18.9/./gtk/gtktextbtree.c:4034: byte index off the end of the line
(mawe.py:10556): Gtk-ERROR **: Byte index 1362 is off the end of the line
Trace/breakpoint trap

There is no other error or warning logs. I have been googling the error without any success.

Other symptoms seem to be:

• Crashes happen even if the editor is idling

• Today I figured out that strangely I can get the crash very quickly by moving mouse over the hidden section :o

I can't be 100% sure, but I think this is related to invisible regions.

Question: Does anyone happen to know if this is some known bug?

Question: Does anyone have any ideas where I could look for possible solutions? Any ideas what can cause the crash, and what I could explore more deeply?

---

Update: I made some more extensive testing with tags. No other properties seem to react to mouse moves, but when turning invisibility on, mouse moves across the region crash the application. I have been searching for reports mouse events crashing gtktextbtree, but without success so far. Looks like this applies to several v3.x GTK versions.

---

UPDATE: I think I have almost found a workaround for this: filtering out motion-notify events from GtkSource.View seems working, like this:

def filter_event(widget, event, *args):
    # Allow these
    if event.type == Gdk.EventType.KEY_PRESS: return False
    if event.type == Gdk.EventType.KEY_RELEASE: return False

    # Block these
    if event.type == Gdk.EventType.LEAVE_NOTIFY: return True
    if event.type == Gdk.EventType.MOTION_NOTIFY: return True

    # Print & allow the rest
    print(event)
    return False

self.text.connect("event", filter_event)

Application still crashes, if you press mouse button near the hidden lines, but seemingly it does not crash to mouse movements anymore.

---

UPDATE: More investigations. Although blocking mouse events will prevent crashes, it also causes quirks, e.g. it is not possible to use mouse to place cursor, select areas, DnD, ... Also, mouse cursor can disappear because it is not updated correctly every time. I am pretty sure there is a bug in algorithm converting mouse/window coordinates to buffer positions (when there is larger hidden blocks in the text), and thus any mouse event can crash the application.

---

UPDATE: I have been trying to create simple test case for the subject. Good things: hiding seems to be working. Bad things: Can't yet reproduce the problem. The test script can be found here:

https://github.com/mkoskim/mawe/blob/master/gui/gtk/test/hidecrash/hidecrash.py

---

UPDATE: Trying to figure it out - test case works, editor does not work. The difference to test case is at least that editor puts hidden tags in event loop(*). Trying to make a test case for that...

(*) There is definitely many different solutions to implement folding with the current Gtk SourceView/TextView. I chose the approach to use "markup" language and applying the folding while editing, as it works with undo/redo. I have tried also other solutions, like:

1. Cut folded scene & insert a widget to text buffer, containing the text itself. Idea: "text [part chosen for folding] text" -> "text [anchor + widget with cut text] text" - Sadly, it does not work with undo/redo.

2. Cut text, give it an ID, and place a specifially marked part containing the ID in the buffer. Idea "text [part chosen for folding] text" -> "text [ID w/ hidden + protected tags] text" - does not work, because cut'n'paste nor undo/redo do not apply tags, so user can destroy the marks.

3. Plain marks: It is just unbelievably hard to try to keep marks with the folding indicators. You would need something like "[char][mark][char]" with protected tags and such to make sure that you don't loose the mark somewhere.

Anyways, I'll keep investigating.

---

UPDATE: Still can't reproduce the problem in my test script, but found something possibly interesting: folding the last scene does not cause the crash - only when folding scenes that follow another scene (folded or not).

Answer 1

I have a fix to the problem. I just don't understand why it works, and why I can't reproduce the problem with test script. Here is the snippet that performs the folding:

fold_start = at.copy()
fold_start.forward_to_line_end()
fold_start.forward_char() # Comment this line -> crash
fold_end = self.scene_end_iter(end)
self.apply_tag(self.tag_fold_hide, fold_start, fold_end)

at is derived at insert-text or delete-range callback and pointing to the line start. fold_end is TextIter to either next scene mark or end of file. If we look the buffer content, it is something like this:

<mark 1><at>Scene 1 heading<eol>
Line
Line
<mark 2>Scene 2 heading

Applying hide tag from <eol> to <mark 2> causes crash. Applying tag from <eol + 1> to <mark 2> works as intended. If folded to the end of file (<mark 2> == buffer.get_end_iter()), folding works. In certain cases it also works, if there is just line feed to be hidden, but not in every case.

As said, I don't understand why this works and why I can't reproduce the problem with simpler script, but I keep investigating more, although now when having a fix, it is not that urgent.

Answer 2

OK, so yesterday I did so, that both the visible and invisible parts are formatted with similar font descriptions. I can't still reproduce this problem with simple test script, but it seems that if visible and hidden part are too far from themselves, something goes wrong with the offset calculations. So far I know that:

• The error message, like Byte index 1362 is off the end of the line Trace/breakpoint trap is about 5-7 bytes off from the length of the part that was hidden.

• When the visible and invisible parts are "close enough" to each other in terms of font size, padding and weight, the error does not appear. In my working example, I format the invisible part with the exactly same font, weight, size and so on, and the program does not crash anymore.