ShellExecute doesnt work if some parameters are just int type

Question

I have the below code which is to launch any .exe (in this example its notepad.exe). But this code is not working. though there are no compilation issues.

    [DllImport("shell32.dll", CharSet = CharSet.Auto)]
    static extern bool ShellExecuteEx(ref SHELLEXECUTEINFO lpExecInfo);

    public static void exev()
    {
        SHELLEXECUTEINFO info = new SHELLEXECUTEINFO();
        info.cbSize = System.Runtime.InteropServices.Marshal.SizeOf(info);
        info.lpVerb = "open";
        info.lpFile = "c:\\windows\\notepad.exe";
        info.nShow = 5;
        info.fMask = 0x440;
        info.hwnd = IntPtr.Zero;
        ShellExecuteEx(ref info);
    }

}

[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Auto)]
public struct SHELLEXECUTEINFO
{
    public int cbSize;
    public uint fMask;
    public IntPtr hwnd;
    [MarshalAs(UnmanagedType.LPTStr)]
    public string lpVerb;
    [MarshalAs(UnmanagedType.LPTStr)]
    public string lpFile;
    [MarshalAs(UnmanagedType.LPTStr)]
    public string lpParameters;
    [MarshalAs(UnmanagedType.LPTStr)]
    public string lpDirectory;
    public int nShow;
    public int hInstApp;
    public int lpIDList;
    [MarshalAs(UnmanagedType.LPTStr)]
    public string lpClass;
    public int hkeyClass;
    public uint dwHotKey;
    public int hIcon;
    public int hProcess;
}

I tried the below code where I changed the SHELLEXECUTEINFO structure and then it started working. The changes I have done are renamed the variables hInstApp,lpIDList,hkeyClass,hIcon and hProcess from int to inptr.

[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Auto)]
public struct SHELLEXECUTEINFO
{
    public int cbSize;
    public uint fMask;
    public IntPtr hwnd;
    [MarshalAs(UnmanagedType.LPTStr)]
    public string lpVerb;
    [MarshalAs(UnmanagedType.LPTStr)]
    public string lpFile;
    [MarshalAs(UnmanagedType.LPTStr)]
    public string lpParameters;
    [MarshalAs(UnmanagedType.LPTStr)]
    public string lpDirectory;
    public int nShow;
    public IntPtr hInstApp;
    public IntPtr lpIDList;
    [MarshalAs(UnmanagedType.LPTStr)]
    public string lpClass;
    public IntPtr hkeyClass;
    public uint dwHotKey;
    public IntPtr hIcon;
    public IntPtr hProcess;
}

I would like to know if we can make it work with int datatype only for those variables. Or Does it work with IntPtr only? How are they different in this scenario apart from datatype size? Because it doesn't give me any syntax error when I use int only for hInstApp,lpIDList,hkeyClass,hIcon and hProcess variables but it doesn't work.

Answer 1

You need to dig into the Windows SDK headers to see how large the type is. For example, for a 64 bit process, sizeof(HWND) is 8 in C++ and sizeof(int) is 4 in C# therefore if you use int to store HWND, you are corrupting memory. Same for HKEY, LPITEMIDLIST, HINSTANCE and HICON. IntPtr is designed for this kind of platform specific data type size.

The compiler won't warn you about runtime errors like memory corruption.