My custom tool window in a Visual Studio Extension package is supposed to evaluate custom expressions during debug sessions for VS2012 and VS2013. The final goal is to build a graphical watch window. Therefore, I must fetch the value of the expression result from the debug engine.
The evaluation works fine via
IDebugStackFrame2 -> IDebugExpressionContext2 -> ParseText
IDebugExpression2 -> EvaluateSync -> IDebugProperty2
The first problem here: On Visual Studio 2013 ParseText() does not correctly report errors. But the IDebugProperty2 retrieved is valid (for valid expressions). On VS2012 I get proper errors for invalid expressions.
This is where the real problem starts. The result Property2 represents a complex custom object. In order to visualize it, a specific property from one of its base classes needs to get retrieved. In my current attempt, I am using IDebugProperty2.EnumChildren() to walk along its members up to the right place in the class hierarchy, similar as one would do in the Locals window in Visual Studio:
[result] -> base -> Raw -> base -> Non-public Members -> [private field] ... a.s.o.
This is somehow working but introduces the following problems:
Problem 1: Get an internal member of some complex object
The members and their order received from EnumChildren() seem to depend on some factors, which are not obvious to me: * Does the object has a Debug Visualizer attached? (VS2013 ignores enum_DEBUGPROP_INFO_FLAGS.DEBUGPROP_INFO_VALUE_RAW in EnumChildren and gives back the beautified version + a "Raw View" node, VS2012 works as expected) * Is a 'Non-public Members' node shown? Sometimes it is, sometimes not - dunno from what it depends on * Setting of 'Just my code' (? I read about it in several posts but couldn't really figure out a reliable rule) * ... there must be other dependencies ?
This turns out to be a really hacky way of aquiring an internal member of some complex expression result object. What could be a better way? Using IDebugProperty3 and some Debugger Visualizer tricks? Any pointer would be great.
Problem 2: VS2012 fails to give memory bytes
Once I have the private member of the result object, I use IDebugProperty2.GetMemoryBytes() in order to 'serialize' its value (a plain System.Array) and transfer it to my Custom Tool Window. While this is working in VS2013, in VS2012 IDebugProperty2.GetMemoryBytes() always returns VS_Constants.S_FALSE. I am really out of ideas here. Since there is a Memory Window in VS2012, there ought to be some way to get this working here also? What am I missing?
The code used to query the memory API:
IDebugMemoryContext2 memCtx = null;
IDebugMemoryBytes2 memBytes = null;
// getting the memory context is working ok
if (tmpProperty2.GetMemoryContext(out memCtx) == VSConstants.S_OK
// VS2012 fails here, always returns S_FALSE, memBytes remains null:
&& (tmpProperty2.GetMemoryBytes(out memBytes) == VSConstants.S_OK)) {
...
Some more context:
OS: Windows 8 and 8.1
Visual Studio: 2012 / 2013
Tool Window built with MPF (C#) in VS2013, (removed all vers. 12 references from the package before deploying to VS2012)
Debug engines: C# and Visual Basic