2020-11-26 17:47:55 ERROR OctetStreamMetadataHandler:503 - No FileSystem for scheme: file java.io.IOException: No FileSystem for scheme: file at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2421) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2428) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:88) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2467) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2449) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:367) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:166) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:351) at org.apache.hadoop.fs.Path.getFileSystem(Path.java:287) at org.apache.orc.impl.ReaderImpl.(ReaderImpl.java:342) at org.apache.orc.OrcFile.createReader(OrcFile.java:342) at com.alation.azure.data.lake.metadata.OctetStreamMetadataHandler.scanORCMetadata(OctetStreamMetadataHandler.java:484) at com.alation.azure.data.lake.metadata.OctetStreamMetadataHandler.getSchema(OctetStreamMetadataHandler.java:748) at com.alation.azure.data.lake.main.AzureFileIndexer.lambda$processAzureFiles$0(AzureFileIndexer.java:199) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1494) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) at java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291) at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:747) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:283) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1603) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
Problem with dependency maven jar , i am getting below error ,
139 Views Asked by jack At
1
Hadoop will usually run on a server, with full access to a file system.
If you intend to POST data from a browser UI to be indexed, perhaps your Spring Boot web app could write the data to a Rabbit MQ using Spring Stream. You could write a listener to the queue that would write a file to the Hadoop server file system and index the data that way.
The problem is that your web app doesn't have access to a file system. It probably runs in a container, without access to a disk.
Why are you deploying your Spring Boot app as a WAR file? You don't need a Java EE app server. It would be much better to deploy it as a fat, executable JAR file. You only need a Java JVM that way.
Don't bundle a UI in with your Spring Boot controller. Let it be an independent micro service. Keep the UI separate from the service.