How can I load a service provider from a module layer with a custom class loader?

Question

I am trying to load a service provider from my own module layer, using a custom class loader.

I am receiving the error: Exception in thread "main" java.util.ServiceConfigurationError: client.MyInterface: Provider implementation.MyInterfaceImpl not found.

Here is my code:

ClassLoader classLoader = new CustomClassLoader();
String module = "implementation";
Path dir = Path.of("/path/to/lib.jar");
ModuleFinder finder = ModuleFinder.of(dir);
ModuleLayer parent = ModuleLayer.boot();
Configuration conf = parent.configuration().resolve(finder, ModuleFinder.of(), Set.of(module));
ModuleLayer layer = parent.defineModules(conf, (String name) -> classLoader);

//layer.findLoader(module).loadClass("implementation.MyInterfaceImpl");


ServiceLoader<MyInterface> loader = ServiceLoader.load(layer, MyInterface.class);
Iterator<MyInterface> iterator = loader.iterator();
for(MyInterface impl: loader){
    impl.doSomething();
}

I can bypass the error by including the line: layer.findLoader(module).loadClass("implementation.MyInterfaceImpl");, however, I wish to avoid this so that I don't have to define implementations at compile time.

Any ideas why this error might be occurring? Shouldn't the service loader by calling the class loader for the module layer in the same way I do? It seems my classloader is not being used.