Build #5,091

License check with RAT and Checkstyle

Build result summary

Details

Completed
Duration
4 minutes
Labels
None
Agent
bamboo-agent07
Revision
3e8fd78330e17e3d907dd8222bd04748584b60b5 3e8fd78330e17e3d907dd8222bd04748584b60b5
Successful since
#4616 ()

Code commits

Author Commit Message Commit date
Terence Yim <885032+chtyim@users.noreply.github.com> Terence Yim <885032+chtyim@users.noreply.github.com> 3e8fd78330e17e3d907dd8222bd04748584b60b5 3e8fd78330e17e3d907dd8222bd04748584b60b5 Merge pull request #12782 from cdapio/feature/CDAP-17287-collect-spark-history
(CDAP-17287) Collects Spark execution event logs
Terence Yim Terence Yim 63aa9a75cc72dbe0cc11b61f7a15fc0c12274427 63aa9a75cc72dbe0cc11b61f7a15fc0c12274427 (CDAP-17287) Collection of Spark execution event logs
- Introducing the Spark event logging and upload of logs through the runtime service.
  Based on the cConf settings, the event logging collection will be on/off regardless
  of the Spark job spark.eventLog.* configurations.
- Refactoring of the SparkRuntimeEnv class to simplify the logic by using CompletableFuture.
- Make runtime system services shutdown after the SparkExecutionContext stopped.
  Currently they stop concurrently. It is not a problem before this PR since we
  are not doing any important work at the context stop.
  However, this PR introduces uploading the event logs at the context stop,
  which needs to be completed before the system services shutdown and process termination.

JIRA issues

IssueDescriptionStatus
Unknown Issue TypeCDAP-17287Could not obtain issue details from JIRA