understanding of some Luigi issues

Question

Look at class ATask

class ATask(luigi.Task):
    config = luigi.Parameter()

    def requires(self):
         # Some Tasks maybe

    def output(self):
        return luigi.LocalTarget("A.txt")

    def run(self):
        with open("A.txt", "w") as f:
            f.write("Complete")

Now look at class BTask

class BTask(luigi.Task):
    config = luigi.Parameter()

    def requires(self):
         return ATask(config = self.config)

    def output(self):
        return luigi.LocalTarget("B.txt")

    def run(self):
        with open("B.txt", "w") as f:
            f.write("Complete")

Question is there is a chance that while TaskA running and start write "A.txt" taskB will start before taskA finishing writing?

The second is that if I start execution like

luigi.build([BTask(config=some_config)], local_scheduler=True )

And if this pipilene fail inside - Could I somehow to know outside about this like return value of luigi.build or smth else?

Answer 1

1. No, luigi won't start executing TaskB until TaskA has finished (ie, until it has finished writing the target file)

2. If you want to get a detailed response for luigi.build in case of error, you must pass an extra keyword argument: detailed_summary=True to build/run methods and then access the summary_text, this way:

     luigi_run_result = luigi.build(..., detailed_summary=True)
     print(luigi_run_result.summary_text)

For details on that, please read Response of luigi.build()/luigi.run() in Luigi documentation.

Also, you may be interested in this answer about how to access the error / exception: https://stackoverflow.com/a/33396642/3219121