C# Parallel.ForEach() on SPListItemCollection causes exception (0x80010102)

Question

In my ASP.NET MVC application, I am trying to retrieve all items in a list with the version history, and then cast them to a custom object. To do this, I am using Microsoft.SharePoint.

I was initially doing this the following way:

Util.GetSPItemCollectionWithHistory method:

public static SPListItemCollection GetSPItemCollectionWithHistory(string listName, SPQuery filterQuery)
{
    using (SPSite spSite = new SPSite(sp_URL))
    {
        using (SPWeb spWeb = spSite.OpenWeb())
        {
            SPList itemsList = spWeb.GetList("/Lists/" + listName);
            SPListItemCollection listItems = itemsList.GetItems(filterQuery);

            return listItems;
        }
    }
}

GetSPObjectsWithHistory method:

protected static List<SPObjectWithHistory<T>> GetSPObjectsWithHistory(SPQuery query = null, List<string> filters = null)
{
    List<SPObjectWithHistory<T>> resultsList = new List<SPObjectWithHistory<T>>();

    Type objectType = typeof(T);
    string listName = "";

    query = query ?? Util.DEFAULT_SSOM_QUERY;

    if (objectType == typeof(SPProject)) listName = Util.PROJECTS_LIST_NAME;
    else if (objectType == typeof(SPTask)) listName = Util.TASKS_LIST_NAME;
    else throw new Exception(String.Format("Could not find the list name for {0} objects.", objectType.Name));

    SPListItemCollection results = Util.GetSPItemCollectionWithHistory(listName, query);
    foreach (SPListItem item in results)
    {
        resultsList.Add(new SPObjectWithHistory<T>(item, filters));
    }

    return resultsList;
}

SPObjectWithHistory Class constructor:

public SPObjectWithHistory(SPListItem spItem, List<string> filters = null)
{
    double.TryParse(spItem.Versions[0].VersionLabel, out _currentVersion);
    History = new Dictionary<double, T>();

    if (spItem.Versions.Count > 1)
    {
        for (int i = 1; i < spItem.Versions.Count; i++)
        {
            if (filters == null)
                History.Add(double.Parse(spItem.Versions[i].VersionLabel), SPObject<T>.ConvertSPItemVersionObjectToSPObject(spItem.Versions[i]));
            else
            {
                foreach (string filter in filters)
                {
                    if (i == spItem.Versions.Count - 1 || (string)spItem.Versions[i][filter] != (string)spItem.Versions[i + 1][filter])
                    {
                        History.Add(double.Parse(spItem.Versions[i].VersionLabel), SPObject<T>.ConvertSPItemVersionObjectToSPObject(spItem.Versions[i]));
                        break;
                    }
                }
            }
        }
    }
}

This way the code works, but it is extremely slow on large lists. One of the lists has over 80000 items in it, and creating one SPObjectWithHistory item takes about 0.3 seconds, due to the logic in the constructor.

To speed up the process, I wanted to use Parallel.ForEach instead of a regular foreach.

My GetSPObjectsWithHistory was then updated to this:

protected static List<SPObjectWithHistory<T>> GetSPObjectsWithHistory(SPQuery query = null, List<string> filters = null)
{
    ConcurrentBag<SPObjectWithHistory<T>> resultsList = new ConcurrentBag<SPObjectWithHistory<T>>();

    Type objectType = typeof(T);
    string listName = "";

    query = query ?? Util.DEFAULT_SSOM_QUERY;

    if (objectType == typeof(SPProject)) listName = Util.PROJECTS_LIST_NAME;
    else if (objectType == typeof(SPTask)) listName = Util.TASKS_LIST_NAME;
    else throw new Exception(String.Format("Could not find the list name for {0} objects.", objectType.Name));

    List<SPListItem> results = Util.GetSPItemCollectionWithHistory(listName, query).Cast<SPListItem>().ToList();
    Parallel.ForEach(results, item => resultsList.Add(new SPObjectWithHistory<T>(item, filters)));

    return resultsList.ToList();
}

When I now try to run the application, however, I receive the following exception at the Parallel.ForEach:

Message: One or more errors occurred.

Type: System.AggregateException

StackTrace:

at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)

at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)

at System.Threading.Tasks.Parallel.ForWorker[TLocal](Int32 fromInclusive, Int32 toExclusive, ParallelOptions parallelOptions, Action'1 body, Action'2 bodyWithState, Func'4 bodyWithLocal, Func'1 localInit, Action'1 localFinally)

at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IEnumerable'1 source, ParallelOptions parallelOptions, Action'1 body, Action'2 bodyWithState, Action'3 bodyWithStateAndIndex, Func'4 bodyWithStateAndLocal, Func'5 bodyWithEverything, Func'1 localInit, Action'1 localFinally)

at System.Threading.Tasks.Parallel.ForEach[TSource](IEnumerable'1 source, Action'1 body)

at GetSPObjectsWithHistory(SPQuery query, List`1 filters) in...

InnerException:

Message: Attempted to make calls on more than one thread in single threaded mode. (Exception from HRESULT: 0x80010102 (RPC_E_ATTEMPTED_MULTITHREAD))

Type: Microsoft.SharePoint.SPException

StackTrace:

at Microsoft.SharePoint.SPGlobal.HandleComException(COMException comEx)

at Microsoft.SharePoint.Library.SPRequest.SetVar(String bstrUrl, String bstrName, String bstrValue)

at Microsoft.SharePoint.SPListItemVersionCollection.EnsureVersionsData()

at Microsoft.SharePoint.SPListItemVersionCollection.get_Item(Int32 iIndex)

at line of double.TryParse(spItem.Versions[0].VersionLabel, out _currentVersion); in the SPObjectWithHistory constructor.

InnerException:

Message: Attempted to make calls on more than one thread in single threaded mode. (Exception from HRESULT: 0x80010102 (RPC_E_ATTEMPTED_MULTITHREAD))

Type: System.Runtime.InteropServices.COMException

StackTrace:

at Microsoft.SharePoint.Library.SPRequestInternalClass.SetVar(String bstrUrl, String bstrName, String bstrValue)

at Microsoft.SharePoint.Library.SPRequest.SetVar(String bstrUrl, String bstrName, String bstrValue)

Would there be anyone who knows how I could get my code to work?

Thanks in advance!

Answer 1

Apparently, what I was trying to do is not possible. The Microsoft.SharePoint namespace's SP objects are not thread safe, like @JeroenMostert stated.

COM is single threaded unless the code explicitly indicates otherwise, to avoid all the problems inherent with multithreading. This component does not indicate it's safe for threading, so it's not safe for threading, no matter how much you want it to be. Consider using lazy loading -- is it really necessary to retrieve all 80,000 items of that list item up front, for example? What user will browse that? Even if you want custom objects, you could store the necessary referral data in a custom collection and materialize/retrieve these on demand.

Since lazy loading was not an option for me, I decided to split my logic into batches (using System.Threading.Task), which each execute the code from my original post (with the SPQuery.Query changing for every batch). After that, the results from my GetSPObjectsWithHistory are merged into a single list.