From the experiment, I obtained two groups of samples, denoted as $X_1, X_2, \dots, X_n$ and $Y_1, Y_2, \dots, Y_m$. Each sample comes from a different distribution, but the distributions within a group are similar (generated by an unknown stochastic non-linear model). All the correlations between any two samples have been calculated, i.e. we already have the pairwise correlations within group X, Y and pairwise correlations between group X and Y. The task for me is to check whether group X and Y are significantly different. In other words, whether the between-group correlations are significantly smaller than the within-group correlations.
Currently my idea is to calculate the Pearson correlation coefficient
$\rho = Cov(mean(X), mean(Y))/\sqrt{Cov(mean(X), mean(X))Cov(mean(Y),mean(Y))}$
Expanding the "mean" and we can obtain
$\rho = mean(Cov(X_i, Y_j))/\sqrt{mean(Cov(X_i,X_j)) mean(Cov(Y_i,Y_j))}$
Assuming Var(X_i) = constant for all i and Var(Y_i) = constant for all i, we have
$\rho = mean(\rho_{X_i, Y_j}) / \sqrt{mean(\rho_{X_i, X_j}) mean(\rho_{Y_i, Y_j})}$
where $mean(\rho_{X_i, Y_j})$ is the mean value of the pairwise between-group correlations and $mean(\rho_{X_i, X_j})$, $mean(\rho_{Y_i, Y_j})$ are the mean values of the pairwise within-group correlations. Then by evaluating the value of $\rho$, I can conclude whether the two groups are significantly different from each other.
Is this idea reasonable? If possible, could you please provide some suggestions for alternative approaches? Thank you very much!