A bug in Widget’s Window picker

Yesterday, I experienced a segmentation fault in my application. After done a series of debugging that also made me compile the OpenSceneGraph (ver 2.6.1) in Debug mode, I figured out that the segmentation fault was caused from a method call from a NULL object.

The bug was in pickAtXY method of WindowManager class that is declared in WindowManager.cpp. The problematic code was like this.

    (!win || win->getVisibilityMode() == Window::VM_PARTIAL) &&
        !win->isPointerXYWithinVisible(x, y)
) continue;

Spot the problem? What if win is NULL? It should proceed to continue immediately and don’t make any call to its method. But in the above code, a NULL win would still make its isPointerXYWithinVisible() called.

The fix should be trivial. But before I made my own modification, I decided to look at the latest version of OpenSceneGraph (2.8.0 rc3) and, guess what, it is already fixed! So I only had to copy the bug-free version and pasted it to the ver 2.6.1 code that I use. The new code is like the following.

    !win ||
        (win->getVisibilityMode() == Window::VM_PARTIAL && !win->isPointerXYWithinVisible(x, y))
) {

Just swapping some parentheses and the bug is gone!

You know what, this is why I like working with Free or Open Source software. You can see the actual code that ticks you off and make changes if necessary.