(CDAP-17287) Collection of Spark execution event logs
- Introducing the Spark event logging and upload of logs through the runtime service.
Based on the cConf settings, the event logging collection will be on/off regardless
of the Spark job spark.eventLog.* configurations.
- Refactoring of the SparkRuntimeEnv class to simplify the logic by using CompletableFuture.
- Make runtime system services shutdown after the SparkExecutionContext stopped.
Currently they stop concurrently. It is not a problem before this PR since we
are not doing any important work at the context stop.
However, this PR introduces uploading the event logs at the context stop,
which needs to be completed before the system services shutdown and process termination.
