Build #5,052

License check with RAT and Checkstyle

Build: #5052 was successful Changes by albertshau <ashau@google.com>

Code commits

caskdata/cdap

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 02c948c47cbed52b1ca8ea678d2a9a2e089f0457

    Merge pull request #12656 from cdapio/feature/CDAP-17202-multi-commit-parallel
    CDAP-17202 perform MultiOutputCommitter duties in parallel

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 664bc3d6338be4a394149e628cc2b8d6ca84fad7 m

    CDAP-17202 perform MultiOutputCommitter duties in parallel
    setup, abort, recover, and commit jobs and tasks in parallel in
    the MultiOutputCommitter.

    Also added a tiny performance gain and debuggability improvement
    by avoiding a union with an empty RDD in sources. This is mostly
    useful for avoiding confusiong when looking at DAGs in the Spark
    UI, where jobs would always start with a union between an empty
    RDD and the real input RDD.

    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/common/output/MultiOutputCommitter.java (version 664bc3d6338be4a394149e628cc2b8d6ca84fad7)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/batch/SparkBatchSourceFactory.java (version 664bc3d6338be4a394149e628cc2b8d6ca84fad7)