Java SecurityManager only block reflection invoke and set

Question

I want my SecurityManager to only block invoke for method and set for field reflection. getDeclaredMethods or similar should stay allowed. Is this possible? I want to block change of the security field of the private java.lang.System class, but still allow reflection. Here is my code:

public class SecMan extends SecurityManager {

  @Override
  public void checkPermission(Permission perm) {
    if (perm instanceof ReflectPermission) {
      // called on setAccessible
      // determine whether caller is permitted, but how? 
      // can't get reference name using getClassContext()
    }
  }

  @Override
  public void checkPackageAccess(String pkg) {
    if (pkg.startsWith("sun.misc")) {
      // do not allow Unsafe, as it could disable my SecurityManager too.
      throw new SecurityException();
    }
  }

  @Override
  public void checkMemberAccess(Class<?> clazz, int which) {
    // not called sadly, deprecated
  }
}

In particular, i want to block this code:

      Method getDeclaredFields0M = Class.class.getDeclaredMethod("getDeclaredFields0", boolean.class);
      getDeclaredFields0M.setAccessible(true);
      Field[] fields = (Field[]) getDeclaredFields0M.invoke(System.class, false);
      Field securityField = null;
      for (Field field : fields)
        if (field.getName().equals("security")) {
          securityField = field;
        }
      securityField.setAccessible(true);
      securityField.set(null, null);