Cask Data Application Platform - Develop Build and Test

Build: #3424 failed Child of CDAP-DRC-5584

Code commits

caskdata/cdap

  • Vinisha Shah <CuriousVini@users.noreply.github.com>

    Vinisha Shah <CuriousVini@users.noreply.github.com> 9aae24b902025f0af1f90501fb886c75355f67ae

    Merge pull request #13678 from cdapio/feature/k8s-spark-submit-nocompression
    [CDAP-18504] Add config maps for spark localize resources with compression

  • Vinisha Shah

    Vinisha Shah 8c54d82459f3c3ddbbd1ce3071370a764badebb9

    [CDAP-18504] Add config maps for spark localize resources with compression

    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/batch/SparkPreparer.java (version 8c54d82459f3c3ddbbd1ce3071370a764badebb9)
    • cdap-kubernetes/src/main/java/io/cdap/cdap/master/environment/k8s/KubeMasterEnvironment.java (version 8c54d82459f3c3ddbbd1ce3071370a764badebb9)
    • cdap-master-spi/src/main/java/io/cdap/cdap/master/spi/environment/MasterEnvironment.java (version 8c54d82459f3c3ddbbd1ce3071370a764badebb9)
    • cdap-master-spi/src/main/java/io/cdap/cdap/master/spi/environment/spark/SparkConfig.java (version 8c54d82459f3c3ddbbd1ce3071370a764badebb9)
    • cdap-master-spi/src/main/java/io/cdap/cdap/master/spi/environment/spark/SparkLocalizeResource.java (version 8c54d82459f3c3ddbbd1ce3071370a764badebb9)
    • cdap-master-spi/src/main/java/io/cdap/cdap/master/spi/environment/spark/SparkSubmitContext.java (version 8c54d82459f3c3ddbbd1ce3071370a764badebb9)
    • cdap-spark-core-base/src/main/java/io/cdap/cdap/app/runtime/spark/SparkProgramRunner.java (version 8c54d82459f3c3ddbbd1ce3071370a764badebb9)
    • cdap-spark-core-base/src/main/java/io/cdap/cdap/app/runtime/spark/SparkRuntimeContextProvider.java (version 8c54d82459f3c3ddbbd1ce3071370a764badebb9)
    • cdap-spark-core-base/src/main/java/io/cdap/cdap/app/runtime/spark/SparkRuntimeService.java (version 8c54d82459f3c3ddbbd1ce3071370a764badebb9)
    • cdap-spark-core-base/src/main/java/io/cdap/cdap/app/runtime/spark/distributed/k8s/SparkContainerDriverLauncher.java (version 8c54d82459f3c3ddbbd1ce3071370a764badebb9)
    • cdap-spark-core-base/src/main/java/io/cdap/cdap/app/runtime/spark/distributed/k8s/SparkContainerExecutorLauncher.java (version 8c54d82459f3c3ddbbd1ce3071370a764badebb9)
    • cdap-spark-core-base/src/main/java/io/cdap/cdap/app/runtime/spark/submit/AbstractSparkSubmitter.java (version 8c54d82459f3c3ddbbd1ce3071370a764badebb9)
    • cdap-spark-core-base/src/main/java/io/cdap/cdap/app/runtime/spark/submit/DistributedSparkSubmitter.java (version 8c54d82459f3c3ddbbd1ce3071370a764badebb9)
    • cdap-spark-core-base/src/main/java/io/cdap/cdap/app/runtime/spark/submit/LocalSparkSubmitter.java (version 8c54d82459f3c3ddbbd1ce3071370a764badebb9)
    • cdap-spark-core-base/src/main/java/io/cdap/cdap/app/runtime/spark/submit/MasterEnvironmentSparkSubmitter.java (version 8c54d82459f3c3ddbbd1ce3071370a764badebb9)
    • cdap-spark-core-base/src/main/java/io/cdap/cdap/app/runtime/spark/submit/SparkSubmitter.java (version 8c54d82459f3c3ddbbd1ce3071370a764badebb9)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 81ab67c3d9a049394bbb23b07c4411827db62480

    Merge pull request #13680 from cdapio/CDAP-18504-fix-heartbeat-uri
    CDAP-18504 use ip instead hostname for spark execution service

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> a1723355c7450bf8fd521c2eb5d34daa1acb9581 m

    CDAP-18504 use ip instead hostname for spark execution service
    In kubernetes env, the hostname for the SparkExecutionService
    is not accessible, though the IP is.

    • cdap-spark-core-base/src/main/java/io/cdap/cdap/app/runtime/spark/distributed/SparkExecutionService.java (version a1723355c7450bf8fd521c2eb5d34daa1acb9581)
    • cdap-spark-core-base/src/main/java/io/cdap/cdap/app/runtime/spark/submit/MasterEnvironmentSparkSubmitter.java (version a1723355c7450bf8fd521c2eb5d34daa1acb9581)