Linq to Entities eqiuvalent to a filtered inline view in T-SQL

Question

I have a query from a database I need to make. I understand how to write the query in T-SQL. The real query is much more complicated, but a simple illustration of the pattern is something like this:

SELECT * FROM [dbo].[A] AS a
    LEFT JOIN dbo.[B] AS b ON a.ID = b.ParentID
    LEFT JOIN dbo.[C] AS c ON y.ID = c.ParentID
    LEFT JOIN 
    (
        SELECT * FROM dbo.[D]
        WHERE OtherID = @otherID
    ) AS d ON c.ID = d.ParentID
    LEFT JOIN
   (
        SELECT * FROM dbo.[E]
        WHERE OtherID = @otherID
    ) AS e ON e.ID = e.ParentID
WHERE A.ID = @Id

I need to write that SQL in C# linq to sql (for entityframework core) such that it generates the equivalent of the filtered inline views above. The goal obviously is to return a result set that always contains the tree A->B->C and contains D or E if and only if those nodes also match the secondary filtering. Note that it is quite easy to do the filtering within the inline view, but very difficult to do it outside the inline view because filtering outside the inline view tends to cause C nodes to disappear when there is no matching D child. That is not the intention.

Thanks

PS: To clarify, you might make a first attempt to write the above as:

query = from a in context.A
join bt in context.B on a.ID equals bt.ParentID into btent
from b in btent.DefaultIfEmpty()
join ct in context.C on b.ID equals ct.ParentID into ctent
from c in ctent.DefaultIfEmpty()
join dt in context.D on c.ID equals dt.ParentID into dtent
from d in dtent.DefaultIfEmpty()
.Include(a => a.B).ThenInclude(b => b.C).ThenInclude(c => c.D)
.Where(a => a.ID = myPrimaryID && d.OtherId = myOtherID)

The trouble is that a where clause on the 'd' entity returns only those rows where D entity exists, so the entire stack will be empty if it isn't. If you try to get cute and say filter where the 'd' entity is null or matches the filter, if you inspect the sql generated by EF in that situation, it is incorrect. The correct filtering has to happen within the 'join', as with the T-SQL above.

PPS: Yes, if you aren't filtering except for the parent object, you can dispense with this entirely and just write the includes and the where clause, but I think on reflection you'll realize that filtering by a term that applies to a great-grand-child but doesn't filter the grand-child is complex. If you can write out the query in either 'form', I'd appreciate it.

Answer 1

Apart from the lack of natural left outer join syntax, select being last, and select * requires anonymous/concrete type projection (but it could contain whole entities), LINQ supports the same constructs as standard SQL, including inline subqueries.

So it's possible to write LINQ query the exact way as sample SQL query:

from a in db.A
join b in db.B on a.ID equals b.ParentID
into a_b from b in a_b.DefaultIfEmpty()
join c in (from c in db.C where c.OtherID == myOtherID select c) on b.ID equals c.ParentID
into b_c from c in b_c.DefaultIfEmpty()
join d in (from d in db.D where d.OtherID == myOtherID2 select d) on c.ID equals d.ParentID
into c_d from d in c_d.DefaultIfEmpty()
select new { a, b, c, d }

which is translated by EF Core to:

SELECT [s].[ID], [s0].[ID], [s0].[ParentID], [t].[ID], [t].[OtherID], [t].[ParentID], [t0].[ID], [t0].[OtherID], [t0].[ParentID]
FROM [SO6_A] AS [s]
LEFT JOIN [SO6_B] AS [s0] ON [s].[ID] = [s0].[ParentID]
LEFT JOIN (
    SELECT [s1].[ID], [s1].[OtherID], [s1].[ParentID]
    FROM [SO6_C] AS [s1]
    WHERE [s1].[OtherID] = @__myOtherID_0
) AS [t] ON [s0].[ID] = [t].[ParentID]
LEFT JOIN (
    SELECT [s2].[ID], [s2].[OtherID], [s2].[ParentID]
    FROM [SO6_D] AS [s2]
    WHERE [s2].[OtherID] = @__myOtherID2_1
) AS [t0] ON [t].[ID] = [t0].[ParentID]

Another standard LINQ way is to push the predicates into join conditions (thus not filtering out the outer join result) by using composite join keys:

from a in db.A
join b in db.B on a.ID equals b.ParentID
into a_b from b in a_b.DefaultIfEmpty()
join c in db.C on new { K1 = b.ID, K2 = myOtherID } equals new { K1 = c.ParentID, K2 = c.OtherID }
into b_c from c in b_c.DefaultIfEmpty()
join d in db.D on new { K1 = c.ID, K2 = myOtherID2 } equals new { K1 = d.ParentID, K2 = d.OtherID }
into c_d from d in c_d.DefaultIfEmpty()
select new { a, b, c, d }

which is translated to:

SELECT [s].[ID], [s0].[ID], [s0].[ParentID], [s1].[ID], [s1].[OtherID], [s1].[ParentID], [s2].[ID], [s2].[OtherID], [s2].[ParentID]
FROM [SO6_A] AS [s]
LEFT JOIN [SO6_B] AS [s0] ON [s].[ID] = [s0].[ParentID]
LEFT JOIN [SO6_C] AS [s1] ON ([s0].[ID] = [s1].[ParentID]) AND (@__myOtherID_0 = [s1].[OtherID])
LEFT JOIN [SO6_D] AS [s2] ON ([s1].[ID] = [s2].[ParentID]) AND (@__myOtherID2_1 = [s2].[OtherID])

More compact LINQ way is to use correlated sub queries instead of joins:

from a in db.A
from b in db.B.Where(b => a.ID == b.ParentID).DefaultIfEmpty()
from c in db.C.Where(c => b.ID == c.ParentID && c.OtherID == myOtherID).DefaultIfEmpty()
from d in db.D.Where(d => c.ID == d.ParentID && d.OtherID == myOtherID2).DefaultIfEmpty()
select new { a, b, c, d }

which is happily translated by EF Core to:

  SELECT [s].[ID], [s0].[ID], [s0].[ParentID], [t].[ID], [t].[OtherID], [t].[ParentID], [t0].[ID], [t0].[OtherID], [t0].[ParentID]
  FROM [SO6_A] AS [s]
  LEFT JOIN [SO6_B] AS [s0] ON [s].[ID] = [s0].[ParentID]
  LEFT JOIN (
      SELECT [s1].[ID], [s1].[OtherID], [s1].[ParentID]
      FROM [SO6_C] AS [s1]
      WHERE [s1].[OtherID] = @__myOtherID_0
  ) AS [t] ON [s0].[ID] = [t].[ParentID]
  LEFT JOIN (
      SELECT [s2].[ID], [s2].[OtherID], [s2].[ParentID]
      FROM [SO6_D] AS [s2]
      WHERE [s2].[OtherID] = @__myOtherID2_1
  ) AS [t0] ON [t].[ID] = [t0].[ParentID]

Finally, the most compact and preferred way in EF Core is to use navigation properties instead of manual joins in LINQ to Entities query:

from a in db.A
from b in a.Bs.DefaultIfEmpty()
from c in b.Cs.Where(c => c.OtherID == myOtherID).DefaultIfEmpty()
from d in c.Ds.Where(d => d.OtherID == myOtherID2).DefaultIfEmpty()
select new { a, b, c, d }

which is also translated by EF Core to:

  SELECT [s].[ID], [s0].[ID], [s0].[ParentID], [t].[ID], [t].[OtherID], [t].[ParentID], [t0].[ID], [t0].[OtherID], [t0].[ParentID]
  FROM [SO6_A] AS [s]
  LEFT JOIN [SO6_B] AS [s0] ON [s].[ID] = [s0].[ParentID]
  LEFT JOIN (
      SELECT [s1].[ID], [s1].[OtherID], [s1].[ParentID]
      FROM [SO6_C] AS [s1]
      WHERE [s1].[OtherID] = @__myOtherID_0
  ) AS [t] ON [s0].[ID] = [t].[ParentID]
  LEFT JOIN (
      SELECT [s2].[ID], [s2].[OtherID], [s2].[ParentID]
      FROM [SO6_D] AS [s2]
      WHERE [s2].[OtherID] = @__myOtherID2_1
  ) AS [t0] ON [t].[ID] = [t0].[ParentID]

Answer 2

Fair enough. 99.9% percent of EF questions about translating LEFT JOIN are a simple failure to use Navigation Properties.

EF Core is adding filtered includes in the next version see Filtering on Include in EF Core.

Or you can project A, along with selected child collections something like this:

var q = from a in db.A
        select new 
        {
          a, 
          Bs = a.Bs,
          Ds = a.Bs.SelectMany( b => b.Ds ).Where(d => d.OtherID = dOtherId)
        };