windows_programming_notes.nbk: Home | Index | Next Page: Sample code | Previous Page: RGB


 Safe Subclassing in Win32

Abstract

This article describes subclassing in the Win32® environment, how it is done, and the rules that should be followed to make subclassing safe. Both instance and global subclassing are covered. Superclassing is described as an alternative to global subclassing.

Subclassing has not changed dramatically from Win16 to Win32. However, there are several new subclassing rules that an application must follow in Win32. The most important (and most visible) of these is that an application cannot subclass a window or class that belongs to another process. This rule cannot be broken, but there are some workarounds that applications can use.

Subclassing Defined

Subclassing is a technique that allows an application to intercept messages destined for another window. An application can augment, monitor, or modify the default behavior of a window by intercepting messages meant for another window. Subclassing is an effective way to change or extend the behavior of a window without redeveloping the window. Subclassing the default control window classes (button controls, edit controls, list controls, combo box controls, static controls, and scroll bar controls) is a convenient way to obtain the functionality of the control and to modify its behavior. For example, if a multiline edit control is included in a dialog box and the user presses the ENTER key, the dialog box closes. By subclassing the edit control, an application can have the edit control insert a carriage return and line feed into the text without exiting the dialog box. An edit control does not have to be developed specifically for the needs of the application.

The Basics

The first step in creating a window is registering a window class by filling a WNDCLASS structure and calling RegisterClass. One element of the WNDCLASS structure is the address of the window procedure for this window class. When a window is created, the 32-bit versions of the Microsoft Windows™ operating system take the address of the window procedure in the WNDCLASS structure and copy it to the new window's information structure. When a message is sent to the window, Windows calls the window procedure through the address in the window's information structure. To subclass a window, you substitute a new window procedure that receives all the messages meant for the original window by substituting the window procedure address with the new window procedure address.

When an application subclasses a window, it can take three actions with the message: (1) pass the message to the original window procedure; (2) modify the message and pass it to the original window procedure; (3) not pass the message.

The application subclassing a window can decide when to react to the messages it receives. The application can process the message before, after, or both before and after passing the message to the original window procedure.

Types of Subclassing

The two types of subclassing are instance subclassing and global subclassing.

Instance subclassing is subclassing an individual window's information structure. With instance subclassing, only the messages of a particular window instance are sent to the new window procedure.

Global subclassing is replacing the address of the window procedure in the WNDCLASS structure of a window class. All subsequent windows created with this class have the substituted window procedure's address. Global subclassing affects only windows created after the subclass has occurred. At the time of the subclass, if any windows of the window class that is being globally subclassed exist, the existing windows are not affected by the global subclass. If the application needs to affect the behavior of the existing windows, the application must subclass each existing instance of the window class.

Win32 Subclassing Rules

Two subclassing rules apply to instance and global subclassing in Win32.

Subclassing is allowed only within a process. An application cannot subclass a window or class that belongs to another process.

The reason for this rule is simple: Win32 processes have separate address spaces. A window procedure has an address in a particular process. In a different process, that address does not contain the same window procedure. As a result, substituting an address from one process with an address from another process does not provide the desired result, so the 32-bit versions of Windows do not allow this substitution (that is, subclassing from a different process) to take place. The SetWindowLong and SetClassLong functions prevent this type of subclassing. You cannot subclass a window or class that is in another process. End of story.

However, there are ways you can add subclassing functionality to every process. Once you get a function inside the address space of a process, you can subclass anything in that process. There are a few ways to do this. The easiest (and most brutal) approach is to add a dynamic-link library (DLL) name to the following key in the registry:

    HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\ CurrentVersion\Windows\APPINIT_DLLS

This key causes Windows to add your DLL to every process in the system. Your DLL would need some way to wake up after every event that the DLL would want to subclass after. A WH_CBT hook usually does the trick. The DLL can watch for the HCBT_CREATEWND event, then subclass the desired windows. The CTL3D sample application uses the WH_CBT hook to do its subclassing, although it does not contain the registry entry that makes subclassing a part of every process. Applications that want CTL3D can link it into their process.

Another way to add your subclassing code to every process is to use a systemwide hook. When a systemwide hook is called from the context of another process, the system loads the DLL containing the hook into the process. The CTL3D code will work for a systemwide WH_CBT hook in the same way that it works for the current thread's local WH_CBT hook.

The third way to add subclassing code into another process is much more complicated: It involves using the OpenProcess, WriteProcessMemory, and CreateRemoteThread functions to inject code into the other process. I don't recommend this method and won't go into any details on how to do it. For developers who insist on using this method, Jeffrey Richter tells me that he is planning to describe the technique in one of his upcoming Win32 Q&A columns in the Microsoft Systems Journal.

Today, many Windows 3.1 applications subclass other processes to augment the processes and to add some cool features. As Windows moves toward an object-oriented system, object linking and embedding (OLE) provides a better way to achieve this capability. Subclassing other processes may get more difficult in future versions of Windows, while exploiting OLE will probably get easier. I recommend that, if possible, you move your applications toward OLE and away from subclassing other processes.

The subclassing process may not use the original window procedure address directly.

In Win16, an application could use the window procedure address returned from SetWindowLong or SetClassLong to call the procedure directly. After all, the return value is simply a pointer to a function, so why not just call it? In Win32, this is a definitive no-no. The value returned from SetWindowLong and GetClassLong may not be a pointer to the previous window procedure at all. Win32 may return a pointer to a data structure that it can use to call the actual window procedure. This occurs in Windows NT™ when an application subclasses a Unicode™ window with a non-Unicode window procedure, or a non-Unicode window with a Unicode window procedure. In this case, the operating system must perform a translation between Unicode and ANSI for the messages the window receives. If an application uses the pointer to this structure to directly call the window procedure, the application will immediately generate an exception. The only way to use the window procedure address returned from SetWindowLong or SetClassLong is as a parameter to CallWindowProc.

Instance Subclassing

The SetWindowLong function is used to subclass an instance of a window. The application must have the address of the subclass function. The subclass function is the function that receives the messages from Windows and passes the messages to the original window procedure. The subclass function must be exported in the application's or the DLL's module definition file.

The application subclassing the window calls SetWindowLong with the handle to the window the application wants to subclass, the GWL_WNDPROC option (defined in WINDOWS.H), and the address of the new subclass function. SetWindowLong returns a DWORD, which is the address of the original window procedure for the window. The application must save this address to pass the intercepted messages to the original window procedure and to remove the subclass from the window. The application passes the messages to the original window procedure by calling CallWindowProc with the address of the original window procedure and the hWnd, Message, wParam, and lParam parameters used in Windows messaging. Usually, the application simply passes the arguments it receives from Windows to CallWindowProc.

The application also needs the original window procedure address for removing the subclass from the window. The application removes the subclass from the window by calling SetWindowLong again. The application passes the address of the original window procedure with the GWL_WNDPROC option and the handle to the window being subclassed.

The following code subclasses and removes a subclass to an edit control:

    LRESULT FAR PASCAL SubClassFunc(HWND hWnd,UINT Message,WPARAM wParam, LPARAM lParam);
    FARPROC lpfnOldWndProc;
    HWND hEditWnd;
    //
    // Create an edit control and subclass it.
    // The details of this particular edit control are not important.
    //
    hEditWnd = CreateWindow("EDIT", "EDIT Test",
                            WS_CHILD | WS_VISIBLE | WS_BORDER ,
                            0, 0, 50, 50,
                            hWndMain,
                            NULL,
                            hInst,
                            NULL);
    //
    // Now subclass the window that was just created.
    //
    lpfnOldWndProc = (FARPROC)SetWindowLong(hEditWnd,
                     GWL_WNDPROC, (DWORD) SubClassFunc);
    .
    .
    .
    //
    // Remove the subclass for the edit control.
    //
    SetWindowLong(hEditWnd, GWL_WNDPROC, (DWORD) lpfnOldWndProc);
    //
    // Here is a sample subclass function.
    //
    LRESULT FAR PASCAL SubClassFunc(   HWND hWnd,
                   UINT Message,
                   WPARAM wParam,
                   LPARAM lParam) {
        //
        // When the focus is in an edit control inside a dialog box, the
        //  default ENTER key action will not occur.
        //
        if ( Message == WM_GETDLGCODE ) {
           return DLGC_WANTALLKEYS;
        }
        return CallWindowProc(lpfnOldWndProc, hWnd, Message, wParam,
                              lParam);
    }

Potential pitfalls

Instance subclassing is generally safe, but observing the following rules ensures safety.

When subclassing a window, you must know what is responsible for the window's behavior. For example, Windows is responsible for all the controls it supplies, and applications are responsible for all the windows they define. Subclassing can be done on any window in the same process; however, when an application subclasses a window for which it is not responsible, the application must ensure that the subclass function does not break the original behavior of the window. Because the application does not control the window, it should not rely on any information about the window that the responsible component might change in the future. A subclass function should not use the extra window bytes or the class bytes for the window unless it knows exactly what they mean and how the original window procedure uses them. Even if the application knows everything about the extra window bytes or the class bytes, it should not use them unless the application is responsible for the window. If an application uses the extra window bytes of a window for which another component is responsible, and the other component decides to update the window and change some aspect of the extra bytes, the subclass procedure is likely to fail. For this reason, Microsoft suggests that you not subclass the control classes. Windows is responsible for the controls it supplies, and aspects of the controls might change from one version of Windows to the next. If your application must subclass a control supplied by Windows, it may need to be updated when a new version of Windows is released.

Because instance subclassing occurs after a window is created, the application subclassing the window cannot add any extra bytes to the window. Applications that subclass windows should store any data needed for an instance of the subclassed window in the window's property list.

The SetProp function attaches properties to a window. The application calls SetProp with the handle to a window, a string identifying the property, and a handle to the data. The handle to the data is usually obtained with a call to either LocalAlloc or GlobalAlloc. When the application uses the data in a window's property list, the application calls the GetProp function with the handle to the window and the string that identifies the property. GetProp returns the handle to the data that was set with SetProp. When the application is finished with the data or when the window is to be destroyed, the application must remove the property from the property list by calling the RemoveProp function with the handle to the window and the string identifying the property. RemoveProp returns the handle to the data, which the application then uses in a call to either LocalFree or GlobalFree. For information on SetProp, GetProp, and RemoveProp, see the Platform SDK documentation.

When an application subclasses a subclassed window, the subclasses must be removed in reverse of the order in which they were performed.

Global Subclassing

Global subclassing is similar to instance subclassing. The application calls SetClassLong to globally subclass a window class. As it does with instance subclassing, the application needs the address of the subclass function, and the subclass function must be exported in the application's or the DLL's module definition file.

To globally subclass a window class, the application must have a handle to a window of that class. To get a handle to a window in the desired class, most applications create a window of the class to be globally subclassed. When the application removes the subclass, it needs a handle to a window of the type the application wants to subclass, so creating and keeping a window for this purpose is the best technique. If the application creates a window of the type it wants to subclass, the window is usually hidden. After obtaining a handle to a window of the correct type, the application calls SetClassLong with the window handle, the GCL_WNDPROC option (defined in WINDOWS.H), and the address of the new subclass function. SetClassLong returns a DWORD, which is the address of the original window procedure for the class. The original window procedure address is used in global subclassing in the same way it is used in instance subclassing. Messages are passed to the original window procedure in the same way as in instance subclassing, by calling CallWindowProc. The application removes the subclass from the window class by calling SetClassLong again. The application passes the address of the original window procedure with the GCL_WNDPROC option and a handle to the window of the class being subclassed. An application that globally subclasses a control class must remove the subclass when the application finishes.

The following code globally subclasses and removes a subclass to an edit control:

    LRESULT FAR PASCAL SubClassFunc(HWND hWnd,UINT Message,WPARAM wParam,
          LPARAM lParam);
    FARPROC lpfnOldClassWndProc;
    HWND hEditWnd;
    //
    // Create an edit control and subclass it.
    // Notice that the edit control is not visible.
    // Other details of this particular edit control are not important.
    //
    hEditWnd = CreateWindow("EDIT", "EDIT Test",
                            WS_CHILD,
                            0, 0, 50, 50,
                            hWndMain,
                            NULL,
                            hInst,
                            NULL);
    lpfnOldClassWndProc =
       (FARPROC)SetClassLong(hEditWnd, GCL_WNDPROC, (DWORD)
          SubClassFunc);
    .
    .
    .
    //
    // To remove the subclass:
    //
    SetClassLong(hEditWnd, GWL_WNDPROC, (DWORD) lpfnOldClassWndProc);
    DestroyWindow(hEditWnd);

Potential pitfalls

Global subclassing has the same limitations as instance subclassing. The application should not attempt to use the extra bytes for either the class or the window instance unless it knows exactly how the original window procedure is using them. If data must be associated with a window, the window's properties list should be used in the same way as in instance subclassing.

In Win32, global subclassing does not affect the classes of any other processes or the windows created from these classes. This is a significant change from the Win16 environment. Windows keeps window class information separately for each Win32 process on the system. For a detailed look at how Windows does this, see the "Window Classes in Win32" technical article in the MSDN Library. Now that global subclassing does not affect other processes, it becomes a usable technique for developers. In Win16, global subclassing was discouraged because it affected every window of the subclassed class—not only for the application that performed the subclass but for the entire system. This was usually not what the application wanted to do, so applications had to use less convenient and less powerful ways to change the behavior of windows created from the system classes. It is far easier to use global subclassing in Win32.

Superclassing

Subclassing a window class causes messages meant for the window procedure to be sent to the subclass function. The subclass function then passes the message to the original window procedure. Superclassing (also known as class cloning) creates a new window class. The new window class uses the window procedure from an existing class to give the new class the functionality of the existing class. The superclass is based on some other window class, known as the base class. Frequently the base class is a Windows-supplied control class, but it can be any window class.

Note

Do not superclass the scroll bar control class because Windows uses the class name to produce the correct behavior for scroll bars.

The superclass has its own window procedure, the superclass procedure, which can take the same actions a subclass procedure can. The superclass procedure can take three actions with the message: (1) pass the message directly to the original window procedure; (2) modify the message before passing it to the original window procedure; (3) not pass the message. The superclass can react to the message before, after, or both before and after passing the message to the original window procedure.

Unlike a subclass procedure, a superclass procedure receives create (WM_NCCREATE, WM_CREATE, and so on) messages from Windows. The superclass procedure can process these messages, but it must also pass these messages to the original base-class window procedure so that the base-class window procedure can initialize.

The application calls [GetClassInfo] to base a superclass on a base class. GetClassInfo fills a WNDCLASS structure with the values from the base class's WNDCLASS structure. The application that is superclassing the base class then sets the hInstance field of the WNDCLASS structure to the instance handle of the application. The application must also set the WNDCLASS structure's lpszClassName field to the name it wants to give this superclass. If the base class has a menu, the application superclassing the base class must supply a new menu that has the same menu IDs as the base class's menu. If the superclass intends to process the WM_COMMAND message and not pass the message to the base class's window procedure, the menu does not have to have corresponding IDs. [GetClassInfo] does not return the lpszMenuName, lpszClassName, or hInstance field of the WNDCLASS structure.

The last field that must be set in the superclass's WNDCLASS structure is the lpfnWndProc field. [GetClassInfo] fills this field with the original class window procedure. The application must save this address so that it can pass messages to the original window procedure with a call to CallWindowProc. The application must put the address of its subclass function into the WNDCLASS structure. This address is not a procedure-instance address because RegisterClass gets the procedure-instance address. The application can modify any other fields in the WNDCLASS structure to suit the application's needs.

The application can add to both the extra class bytes and the extra window bytes because it is registering a new class. The application must follow two rules when doing this: (1) the original extra bytes for both the class and the window must not be touched by the superclass for the same reasons that an instance subclass or a global subclass should not touch these extra bytes; (2) if the application adds extra bytes to either the class or the window instance for the application's own use, it must always reference these extra bytes relative to the number of extra bytes used by the original base class. Because the number of bytes used by the base class may be different from one version of the base class to the next, the starting offset for the superclass's own extra bytes is also different from one version of the base class to the next.

After the WNDCLASS structure is filled, the application calls RegisterClass to register the new window class. Windows of this class can now be created and used.

Applications often used superclassing in Win16 because global subclassing was discouraged. Now that global subclassing is no longer discouraged in Win32, superclassing has lost some of its appeal. You may still find it useful to create a superclass if your application wants to change the behavior for only a subset of the windows (instead of all windows) created from a system class, which is the effect of global subclassing.

Summing Up

Subclassing is a powerful technique that has not changed significantly in Win32. The only major change is that you can no longer subclass a window or class that belongs to another process. Although there are workarounds for this restriction, I recommend that you move your application to OLE rather than relying on subclassing if you need this capability.


windows_programming_notes.nbk: Home | Index | Next Page: Sample code | Previous Page: RGB


Notebook exported on Monday, 7 July 2008, 18:56:50 PM Eastern Daylight Time