Issue with DataStax graph loader edge loading with property key

85 Views Asked by At

DSE VERSION 6.7 and DSE graph loader version 6.7

We have two vertex name "x" and "y" and there is connecting edges named "z" between for x to y. That edge has property also. So wee needs to load that also. And properties data types are Txt(string) except vertex partitions key wihich is UUID.

As mentioned in documentation from Datastax to load edges with properties we need gzip the CSV . https://docs.datastax.com/en/dse/6.7/dse-dev/datastax_enterprise/graph/dgl/dglMapScript.html Now we are loading data using graph loader to load edges between and x and y .

here is our groovy script

// CONFIGURATION
// Configures the data loader to create the schema
//config create_schema: false,preparation: true, load_new: true, load_vertex_threads: 0,dryrun: true,schema_output: '/tmp/loader_output.txt'

config create_schema :false,load_new :true

inputfiledir = '/root/'
waInput = File.csv(inputfiledir + "test.csv.gz").gzip().delimiter('|')



//load edge 
load(waInput).asEdges {
  label "z"
   outV "x", {
       label "x"
       key "xId"
    }
  inV "y", {
      label "y"
      key "yId"
    }
}

here is our csv file format

xId|yId|newCase|totalActiveCases|totalRecoveries|totalDeaths|allCasesTotal|totalTestConducted
8c49304d-71e9-4e9d-93b3-47ebfd0ce073|e31f0a23-64c0-44ea-add8-c67eb52c0187|0|0|0|0|0|0
d451b2b2-a4b5-4ed6-bb4e-128945795e57|e31f0a23-64c0-44ea-add8-c67eb52c0187|0|0|0|0|0|0

xId and yId is UUID and except them, all are Text(string) datatype in database

now we run the that groovy script with dse graph loader we get this error

2020-04-21 05:25:14 INFO  DataLoaderImpl:213 - Scheduling [test.csv] for reading
2020-04-21 05:25:14 DEBUG Reporter:69 - Input queue [test.csv] throughput is [111.74281809353118] items/s
2020-04-21 05:25:14 DEBUG Reporter:120 - query times 9: p50 4515.0µs, p80 4515.0µs, p90 4515.0µs, p95 4515.0µs, p99 4515.0µs, p99.9 4515.0µs, p99.99 4515.0µs
2020-04-21 05:25:14 ERROR DataLoaderImpl:720 - Failed when finalizing loader, some items that were in the queue may not have been written
com.datastax.dsegraphloader.exception.LoadingException: com.datastax.driver.core.exceptions.InvalidQueryException: Null not allowed
        at com.datastax.dsegraphloader.impl.loader.driver.DseGraphDriverImpl.executeGraphQuery(DseGraphDriverImpl.java:149)
        at com.datastax.dsegraphloader.impl.loader.driver.DseGraphDriverImpl.getOrCreateVertices(DseGraphDriverImpl.java:366)        at com.datastax.dsegraphloader.impl.loader.driver.SafeGraphDriver.lambda$tryGetOrCreateVerticies$28(SafeGraphDriver.java:137)
        at com.datastax.dsegraphloader.impl.loader.driver.SafeGraphDriver.tryDriverCall(SafeGraphDriver.java:72)
        at com.datastax.dsegraphloader.impl.loader.driver.SafeGraphDriver.tryGetOrCreateVerticies(SafeGraphDriver.java:125)
        at com.datastax.dsegraphloader.impl.loader.vertex.VertexLoader.completeLoad(VertexLoader.java:100)
        at com.datastax.dsegraphloader.impl.loader.DataLoaderImpl$LoaderCallable.call(DataLoaderImpl.java:645)
        at com.datastax.dsegraphloader.impl.loader.DataLoaderImpl$DelegatingLoaderCallable.run(DataLoaderImpl.java:530)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: com.datastax.driver.core.exceptions.InvalidQueryException: Null not allowed

Any solutions for that ?

0

There are 0 best solutions below