Unity Layer-Based Collision not working for child objects

Question

I have a project in unity where I have the following setup:

• a base object set in the "Ghost" layer, with a NavMeshAgent component (for collider) and a ridigbody (kinematic)

  

• a child object meant to "detect" players, in the layer "PlayerDetector", with a trigger collider

  

  (this is just one example, this happens for multiple "tracker" type children objects)

I've tested, and as long as the base object has a rigidbody, the child objects collides with all layers instead of just the player one

Any idea why this is or how I can fix it in a somewhat clean manner?

The code for when the detection enters:

private void OnTriggerEnter(Collider other)
    {
        if (other.tag == PLAYER_TAG) //this is the issue -> this should not be requried, because it should only collide with the player layer to begin with
        {
            //Debug.Log("ADDED PLAYER: " + playersInRadius);
            playersInRadius.Add(other.gameObject);
        }
    }