Unity || Photon || Ray cast is not Hitting

Question

I am using Photon to learn some multiplayer . In the game I have used Ray casting to detect if the ray is hitting anything with the tag "Player". if so then Debug the Object name. As it is a multiplayer game I had to build the game to test it. But After building some weird bug arrives. When One of the clint trying to shoot the player it just go through the player and hitting the wall behind the player . Basically it is not detecting player .It is like there is nothing between shoot point and wall.... And the weirdest part is sometimes it is just working perfectly. And some times its break. I don't know why this is happening.. if anyone can help .. I really appreciate it here is the code to shoot

 //ray cast and check hit
        ray = Camera.main.ScreenPointToRay(Input.mousePosition);
        if (Physics.Raycast(ray, out hit, allGuns[selectedGun].range))
        {
            Debug.Log("Hit:"+hit.collider.name);

            if(hit.collider.CompareTag("Player"))
            {
                hit.collider.gameObject.GetComponent<PhotonView>().RPC("DealDamage",RpcTarget.All,allGuns[selectedGun].damage,photonView.Owner.NickName);
                PhotonNetwork.Instantiate(playerHitImpact.name,hit.point,Quaternion.identity);
            }else
                Destroy(Instantiate(bulletImpact,hit.point+(hit.normal*0.002f),Quaternion.LookRotation(hit.normal,Vector3.up)),3f);
        }

and player is assigned with correct tag, and it is in default layer.

firstly I thought it was issue with the colliders . As I was using unity's CharacterController, so I tried assigning capsule collider but still its the same.