Cask Data Application Platform - Develop Build and Test

Build: #3421 failed Child of CDAP-DRC-5580

Code commits

caskdata/cdap

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 236e802768b7289b0f4def9b17ba3be4ce5b9acf

    Merge pull request #13671 from cdapio/CDAP-18451-spark-k8s-guice-bindings
    CDAP-18451 setup guice bindings for spark on k8s

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> f30b43aa890a4233571b62d9984b54e3205216fa m

    CDAP-18451 setup guice bindings for spark on k8s
    Added a step in the SparkRuntimeContextProvider to create and
    instantiate a MasterEnvironment to use for service discovery and
    other guice bindings.

    To do this, added to the Spark pod template during submission in
    order to mount the podinfo directory that the KubeMasterEnvironment
    expects. Also added a way to specify the name of the master
    environment SPI to use if it exists.

    • cdap-kubernetes/src/main/java/io/cdap/cdap/master/environment/k8s/KubeMasterEnvironment.java (version f30b43aa890a4233571b62d9984b54e3205216fa)
    • cdap-spark-core-base/src/main/java/io/cdap/cdap/app/runtime/spark/SparkRuntimeContextProvider.java (version f30b43aa890a4233571b62d9984b54e3205216fa)
    • cdap-spark-core-base/src/main/java/io/cdap/cdap/app/runtime/spark/distributed/SparkContainerLauncher.java (version f30b43aa890a4233571b62d9984b54e3205216fa)