YARN log aggregation on a per job basis

87 Views Asked by At

Can properties such as yarn.log-aggregation-enable and yarn.log-aggregation.retain-seconds applied on a per job basis? I would not like this to be enabled at a cluster-wide scale but only for a few tasks.

1

There are 1 best solutions below

0
On BEST ANSWER

Currently, there is no way to aggregate logs for specific YARN applications. https://issues.apache.org/jira/browse/YARN-85 seems to have attempted to provide this feature but the issue is still unresolved.