Cask Data Application Platform - Develop Build and Test

Build: #3646 failed Child of CDAP-DRC-5894

Code commits

caskdata/cdap

  • greeshmaswaminathan <greeshmaswaminathan@users.noreply.github.com>

    greeshmaswaminathan <greeshmaswaminathan@users.noreply.github.com> 387d33a90450f2d28b44bfdddc74ee6a4f6c07e6

    Merge pull request #14243 from cdapio/feature/CDAP-19146-Make-CDAP-version-available
    Add back the constant temporarily for not breaking dependent repos

  • Greeshma Swaminathan

    Greeshma Swaminathan 5c03b11d796596c8fc6015e9f7cb0e60d460b88c

    Add back the constant temporarily for not breaking dependent repos

    • cdap-watchdog-api/src/main/java/io/cdap/cdap/api/metrics/MetricsWriterContext.java (version 5c03b11d796596c8fc6015e9f7cb0e60d460b88c)
  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 101838c01e42aa0ee9e8481b231aa5507647ebb8

    Merge pull request #14240 from cdapio/bugfix/CDAP-19129-3
    [CDAP-19129] Add guava dependency for GCPRemoteAuthenticator

  • greeshmaswaminathan <greeshmaswaminathan@users.noreply.github.com>

    greeshmaswaminathan <greeshmaswaminathan@users.noreply.github.com> 18ec3893a2c91578ef2b0c4942ffb2c53141ca1d

    Merge pull request #14238 from cdapio/feature/CDAP-19146-Make-CDAP-version-available
    Add method for getting platform version in MetricsWriterContext

  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 4975d69574932cc1d18ac6489fb5d406f73a2ed0

    [CDAP-19129] Add guava dependency

    • cdap-authenticator-ext-gcp/pom.xml (version 4975d69574932cc1d18ac6489fb5d406f73a2ed0)
  • nliu132 <nliu132@users.noreply.github.com>

    nliu132 <nliu132@users.noreply.github.com> 51b57caf626ac7ce58d869de647b3bccf9cc14a7

    Merge pull request #14220 from cdapio/feature/CDAP-19062_tether_diff_namespace
    [CDAP-19062] Changes to run tethered programs on different namespaces
    - Use the same program namespace across tethered instances and only reference the runtime Kubernetes namespace during actual program run. This is to allow logs/metrics to be correctly sent back to the tethering server
    - Update Spark pods to use runtime Kubernetes namespace
    - Remove storing the initiating tethered namespace for artifact fetching use

  • Greeshma Swaminathan

    Greeshma Swaminathan a470a49f3c8deb397baace705b7cad5265e1a4bc m

    Add method for getting platform version

    • cdap-watchdog-api/src/main/java/io/cdap/cdap/api/metrics/MetricsWriterContext.java (version a470a49f3c8deb397baace705b7cad5265e1a4bc)
    • cdap-watchdog/src/main/java/io/cdap/cdap/metrics/process/DefaultMetricsWriterContext.java (version a470a49f3c8deb397baace705b7cad5265e1a4bc)
  • Nancy Liu <nancyjliu@google.com>

    Nancy Liu <nancyjliu@google.com> 6d548b60abd58b9a3bd65affd7cba7275242acc0

    [CDAP-19062] Changes to run tethered programs on different namespaces

    • cdap-app-fabric/src/main/java/io/cdap/cdap/app/guice/DistributedProgramContainerModule.java (version 6d548b60abd58b9a3bd65affd7cba7275242acc0)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/deploy/InMemoryProgramRunDispatcher.java (version 6d548b60abd58b9a3bd65affd7cba7275242acc0)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/ProgramOptionConstants.java (version 6d548b60abd58b9a3bd65affd7cba7275242acc0)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/DistributedProgramRunner.java (version 6d548b60abd58b9a3bd65affd7cba7275242acc0)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/tethering/TetheringAgentService.java (version 6d548b60abd58b9a3bd65affd7cba7275242acc0)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/tethering/proto/v1/TetheringLaunchMessage.java (version 6d548b60abd58b9a3bd65affd7cba7275242acc0)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/tethering/runtime/spi/runtimejob/TetheringRuntimeJobManager.java (version 6d548b60abd58b9a3bd65affd7cba7275242acc0)
    • cdap-kubernetes/src/main/java/io/cdap/cdap/master/environment/k8s/KubeMasterEnvironment.java (version 6d548b60abd58b9a3bd65affd7cba7275242acc0)
  • greeshmaswaminathan <greeshmaswaminathan@users.noreply.github.com>

    greeshmaswaminathan <greeshmaswaminathan@users.noreply.github.com> 5c185ef95c07a6abb07a64afa769fba10b497583

    Merge pull request #14182 from alfonso-vidal/fix/eventsSparkMetricsProvider
    Fix/events spark metrics provider

  • nliu132 <nliu132@users.noreply.github.com>

    nliu132 <nliu132@users.noreply.github.com> 3b717c6c11d971b9f5183455587632bc7a400096

    Merge pull request #14226 from cdapio/bugfix/CDAP-19135_spark_serviceaccount
    Create spark service account and role binding during Kubernetes namespace creation

  • Nancy Liu <nancyjliu@google.com>

    Nancy Liu <nancyjliu@google.com> 0f507363c5a943b4ae41216f87ca069d646a9674 m

    [CDAP-19135] Copy spark service account to new Kube namespace

    • cdap-kubernetes/src/main/java/io/cdap/cdap/master/environment/k8s/KubeMasterEnvironment.java (version 0f507363c5a943b4ae41216f87ca069d646a9674)
  • Alfonso Vidal <alfonso.vidal@vodafone.com>

    Alfonso Vidal <alfonso.vidal@vodafone.com> dd6d4f95d0e94f07c244c32cac3f7a1e9d1da13c m

    Add to ExecutionMetrics shuffle values and stageId Return always an array with the different possible COMPLETE stages from spark fix stages url make retrieve metric asynchronous

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/events/MetricsProvider.java (version dd6d4f95d0e94f07c244c32cac3f7a1e9d1da13c)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/events/ProgramStatusEventPublisher.java (version dd6d4f95d0e94f07c244c32cac3f7a1e9d1da13c)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/events/SparkProgramStatusMetricsProvider.java (version dd6d4f95d0e94f07c244c32cac3f7a1e9d1da13c)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/events/http/SparkApplicationsStagesResponse.java (version dd6d4f95d0e94f07c244c32cac3f7a1e9d1da13c)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/events/SparkProgramStatusMetricsProviderTest.java (version dd6d4f95d0e94f07c244c32cac3f7a1e9d1da13c)
    • cdap-app-fabric/src/test/resources/mocked_spark_stages_response_several_stages.json (version dd6d4f95d0e94f07c244c32cac3f7a1e9d1da13c)
    • cdap-event-writer-spi/src/main/java/io/cdap/cdap/spi/events/ExecutionMetrics.java (version dd6d4f95d0e94f07c244c32cac3f7a1e9d1da13c)
    • cdap-event-writer-spi/src/main/java/io/cdap/cdap/spi/events/ProgramStatusEventDetails.java (version dd6d4f95d0e94f07c244c32cac3f7a1e9d1da13c)