I recently had to use AWS Neptune in an authorization service to model a dataset that has intrincate relations between entities, most of them are many to many relations. This is the model I came up with:
My main requirement consists on getting all the roles, functional_abilities, attributes and secured entitites for a user within an specific application, so I created this gremlin query to trasverse the graph:
%%gremlin
g
//.with('Neptune#enableResultCache', true)
.V()
.has('User','ecrid','id_here')
.project('Roles')
.by(
union(
out('ASSIGNED_TO_USER_ROLE')
.has('usage_type',within('1','3','5','7','9','11','13','15'))
.as('UserRole')
.out('ASSIGNED_TO_ROLE'),
outE('BELONGS_TO')
.has('usage_type',within('1','3','5','7','9','11','13','15'))
.has('deny_user_access','0')
.inV()
.out('ASSIGNED_TO_GROUP_ROLE')
.as('GroupRole')
.out('ASSIGNED_TO_ROLE')
)
.has('system_id','id_here')
.dedup()
.project('Role','Functional_Abilities','Attributes')
.by(valueMap('role_name'))
.by(
out('ASSIGNED_TO_ROLE_FUNCTIONAL_ABILITY')
.as('RoleFunctionalAbility')
.out('ASSIGNED_TO_FUNCTIONAL_ABILITY')
.project('Functional_Ability','Secured_Entities','Attributes')
.by(valueMap('functional_ability_name'))
.by(out('HAS_SE').valueMap('entity_name','secured_operations').fold())
.by(
select('RoleFunctionalAbility')
.out('HAS_ROLE_FUNCTIONAL_ABILITY_ATTRIBUTES')
.project('Value', 'Name', 'Type')
.by(coalesce(values('attribute_value'), constant('')))
.by(coalesce(out('HAS_ATTRIBUTE_METADATA').values('attribute_name'), constant('')))
.by(coalesce(out('HAS_ATTRIBUTE_METADATA').values('attribute_type_name'), constant('')))
.dedup()
.fold()
)
.fold()
)
.by(
union(
select('UserRole')
.out('HAS_USER_ROLE_ATTRIBUTE')
.has('usage_type',within('1','3','5','7','9','11','13','15'))
.project('Value', 'Name', 'Type')
.by(coalesce(values('attribute_value'), constant('')))
.by(coalesce(out('HAS_ATTRIBUTE_METADATA').values('attribute_name'), constant('')))
.by(coalesce(out('HAS_ATTRIBUTE_METADATA').values('attribute_type_name'), constant(''))),
select('GroupRole')
.out('HAS_GROUP_ROLE_ATTRIBUTE')
.has('usage_type',within('1','3','5','7','9','11','13','15'))
.project('Value', 'Name', 'Type')
.by(coalesce(values('attribute_value'), constant('')))
.by(coalesce(out('HAS_ATTRIBUTE_METADATA').values('attribute_name'), constant('')))
.by(coalesce(out('HAS_ATTRIBUTE_METADATA').values('attribute_type_name'), constant('')))
)
.dedup()
.fold()
)
.fold()
)
I am able to get the results as I want using this query. However with a user that has 60 roles and each role has about ~50 functional abilities and secured entities assigned, I am getting an average response time of 600 ms which is slow for the requirment so I wanted to ask you guys if you have anny recommendation on the model/query that I could improve in order to make my model more performant. By the way I am using an r5dx8large instance the times with cache are good but I would like to improve the response time discarding cache at the moment Thanks in advance.
I just ran this query through the Neptune Gremlin Explainer [1] so I could see the query plan (Profile output would always be better, as it includes query execution metrics).
The query is falling out of the optimized query path very early due to a similar reason that I provided in this response: Optimizing Gremlin Query Performance for Filtering and Retrieving Data with many properties
You'll want to try rewriting portions of this query so that more of the query will execute using Neptune-specific operators and not falling back to TinkerPop execution.
[1] https://docs.aws.amazon.com/neptune/latest/userguide/gremlin-explain.html