loading csv file from S3 in neo4j graphdb

Question

I am seeking some suggestion about loading csv files from s3 bucket to neo4j graphdb. In S3 bucket the files are in csv.gz format. I need to import them into my neo4j graph db which is in ec2 instance.

1. Is there any direct way to load csv.gz into neo4j db without unzipping it ?
2. can I set/add s3 bucket path into neo4j.conf  file at neo4j.dbms.directory  which is by default neo4j/import ?  

kindly help me to suggest some idea to load files from S3 Thank you

Answer 1

You can achieve both of these goals with APOC. The docs give you two approaches:

• Load from the GZ file directly, assuming the file in the bucket has a public URL
• Load the file from S3 directly, with an access token

Here's an example of the first approach - the section after the ! is the filename within the zip file to load, and this should work with .zip, .gz, .tar files etc.

CALL apoc.load.csv("https://pablissimo-so-test.s3-us-west-2.amazonaws.com/mycsv.zip!mycsv.csv")