Cask Data Application Platform - Develop Build and Test

Build: #3284 failed Child of CDAP-DRC-5402

Code commits

caskdata/cdap

  • Masoud Saeida Ardekani

    Masoud Saeida Ardekani 3d9af80b6a61cd230c5ab319e4b71e746dc7ae65

    Merge pull request #13362 from cdapio/feature/pod-isolation
    CDAP-17960: Pod isolation infrastructure

  • Masoud Saeida Ardekani

    Masoud Saeida Ardekani 1eee585b6805aaf0106fb66e8cf8310df4ba9f8e

    Address comments

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/worker/RunnableTaskLauncher.java (version 1eee585b6805aaf0106fb66e8cf8310df4ba9f8e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/worker/RunnableTaskModule.java (version 1eee585b6805aaf0106fb66e8cf8310df4ba9f8e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/worker/TaskWorkerHttpHandlerInternal.java (version 1eee585b6805aaf0106fb66e8cf8310df4ba9f8e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/worker/TaskWorkerServiceLauncher.java (version 1eee585b6805aaf0106fb66e8cf8310df4ba9f8e)
    • cdap-common/src/main/java/io/cdap/cdap/common/conf/Constants.java (version 1eee585b6805aaf0106fb66e8cf8310df4ba9f8e)
    • cdap-common/src/main/resources/cdap-default.xml (version 1eee585b6805aaf0106fb66e8cf8310df4ba9f8e)
  • wyzhang <wyzhang@google.com>

    wyzhang <wyzhang@google.com> 5e27e6c476b7fa65e07e77276b515744b9cdda9d

    fix incorrect comment

    • cdap-common/src/main/resources/cdap-default.xml (version 5e27e6c476b7fa65e07e77276b515744b9cdda9d)
  • wyzhang <wyzhang@google.com>

    wyzhang <wyzhang@google.com> 4e65d5e31b76fe7f70755104e965216b7fb6b0ad

    Simplify client->dispatcher->workerpool to client->workerpool

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskDispatcherHttpHandlerInternal.java (version 4e65d5e31b76fe7f70755104e965216b7fb6b0ad)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerServiceLauncher.java (version 4e65d5e31b76fe7f70755104e965216b7fb6b0ad)
    • cdap-common/src/main/java/io/cdap/cdap/common/conf/Constants.java (version 4e65d5e31b76fe7f70755104e965216b7fb6b0ad)
    • cdap-common/src/main/resources/cdap-default.xml (version 4e65d5e31b76fe7f70755104e965216b7fb6b0ad)
    • cdap-master/src/main/java/io/cdap/cdap/master/environment/k8s/AppFabricServiceMain.java (version 4e65d5e31b76fe7f70755104e965216b7fb6b0ad)
    • cdap-master/src/main/java/io/cdap/cdap/master/environment/k8s/TaskDispatcherServiceMain.java (version 4e65d5e31b76fe7f70755104e965216b7fb6b0ad)
    • cdap-master/src/test/java/io/cdap/cdap/master/environment/k8s/MasterServiceMainTestBase.java (version 4e65d5e31b76fe7f70755104e965216b7fb6b0ad)
    • cdap-master/src/test/java/io/cdap/cdap/master/environment/k8s/TaskDispatcherServiceServerMainTest.java (version 4e65d5e31b76fe7f70755104e965216b7fb6b0ad)
  • wyzhang <wyzhang@google.com>

    wyzhang <wyzhang@google.com> c00b8d84901b8952a1fca3aaa3475cbdf681596f

    address comments

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/worker/TaskWorkerHttpHandlerInternal.java (version c00b8d84901b8952a1fca3aaa3475cbdf681596f)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/worker/TaskWorkerServiceLauncher.java (version c00b8d84901b8952a1fca3aaa3475cbdf681596f)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/worker/TaskWorkerServiceTest.java (version c00b8d84901b8952a1fca3aaa3475cbdf681596f)
  • wyzhang <wyzhang@google.com>

    wyzhang <wyzhang@google.com> e2189711ffb2c83073fdd0f65a039237fdafb6d2

    change task worker http port 80->11020

    • cdap-common/src/main/resources/cdap-default.xml (version e2189711ffb2c83073fdd0f65a039237fdafb6d2)
  • Masoud Saeida Ardekani

    Masoud Saeida Ardekani d7c5a471f921df1a29daf79214c33fcd3ae45eb8

    Undo AppFabricServiceRuntimeModule changes

    • cdap-app-fabric/src/main/java/io/cdap/cdap/app/guice/AppFabricServiceRuntimeModule.java (version d7c5a471f921df1a29daf79214c33fcd3ae45eb8)
  • Masoud Saeida Ardekani

    Masoud Saeida Ardekani 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b

    Remove unnecessary code for pod isolation and improve documentation

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/EchoRunnableTask.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/RunnableTask.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/RunnableTaskLauncher.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/RunnableTaskModule.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/RunnableTaskRequest.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerHttpHandlerInternal.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerService.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerServiceLauncher.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerTwillApplication.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerTwillRunnable.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/services/AppFabricServer.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/worker/RunnableTaskLauncher.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/worker/RunnableTaskModule.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/worker/RunnableTaskRequest.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/worker/TaskWorkerHttpHandlerInternal.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/worker/TaskWorkerService.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/worker/TaskWorkerServiceLauncher.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/worker/TaskWorkerTwillApplication.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/worker/TaskWorkerTwillRunnable.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/dispatcher/RunnableTaskLauncherTest.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerServiceTest.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerTwillRunnableTest.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/worker/RunnableTaskLauncherTest.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/worker/TaskWorkerServiceTest.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/worker/TaskWorkerTwillRunnableTest.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-common/src/main/java/io/cdap/cdap/common/conf/Constants.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-common/src/main/java/io/cdap/cdap/common/internal/worker/RunnableTask.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-common/src/main/java/io/cdap/cdap/common/internal/worker/RunnableTaskContext.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-common/src/main/resources/cdap-default.xml (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-master/src/main/java/io/cdap/cdap/master/environment/k8s/AppFabricServiceMain.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
    • cdap-master/src/test/java/io/cdap/cdap/master/environment/k8s/MasterServiceMainTestBase.java (version 1fd21d0565c58d83c4eb33ebf13c6034aac6c86b)
  • Masoud Saeida Ardekani

    Masoud Saeida Ardekani ee913a55c36947642a1abe0b3977eab1a228f17f

    Improve error handling in task worker and add more test cases

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerHttpHandlerInternal.java (version ee913a55c36947642a1abe0b3977eab1a228f17f)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/dispatcher/RunnableTaskLauncherTest.java (version ee913a55c36947642a1abe0b3977eab1a228f17f)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerServiceTest.java (version ee913a55c36947642a1abe0b3977eab1a228f17f)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/dispatcher/TestRunnableTaskLauncher.java (version ee913a55c36947642a1abe0b3977eab1a228f17f)
  • wyzhang <wyzhang@google.com>

    wyzhang <wyzhang@google.com> 32d2b4bae31ba16e9963f15f3dfd1b2da6165222

    Extend TaskWorkerServiceTest to cover all http endpoints

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerHttpHandlerInternal.java (version 32d2b4bae31ba16e9963f15f3dfd1b2da6165222)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerService.java (version 32d2b4bae31ba16e9963f15f3dfd1b2da6165222)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerServiceTest.java (version 32d2b4bae31ba16e9963f15f3dfd1b2da6165222)
  • Masoud Saeida Ardekani

    Masoud Saeida Ardekani bc34306dd3f0051a46b455562a371c523d4ed3a9

    Force TaskWorker pod to (1) only accept one request at a time, and (2) be killed once it handles a request

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerHttpHandlerInternal.java (version bc34306dd3f0051a46b455562a371c523d4ed3a9)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerService.java (version bc34306dd3f0051a46b455562a371c523d4ed3a9)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerServiceTest.java (version bc34306dd3f0051a46b455562a371c523d4ed3a9)
  • wyzhang <wyzhang@google.com>

    wyzhang <wyzhang@google.com> 47443a0243ee7ff26fe6a7d4626d50ef5af8f3d5

    add comments and fix logs msg

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/worker/TaskWorkerServiceLauncher.java (version 47443a0243ee7ff26fe6a7d4626d50ef5af8f3d5)
  • Masoud Saeida Ardekani

    Masoud Saeida Ardekani 365cdc945e4a07b3c3869942cf7836e1fb323fbe

    Fix RunnableTask initialization mechanism

    • cdap-app-fabric/src/main/java/io/cdap/cdap/app/guice/AppFabricServiceRuntimeModule.java (version 365cdc945e4a07b3c3869942cf7836e1fb323fbe)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/EchoRunnableTask.java (version 365cdc945e4a07b3c3869942cf7836e1fb323fbe)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/RunnableTask.java (version 365cdc945e4a07b3c3869942cf7836e1fb323fbe)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/RunnableTaskLauncher.java (version 365cdc945e4a07b3c3869942cf7836e1fb323fbe)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/RunnableTaskModule.java (version 365cdc945e4a07b3c3869942cf7836e1fb323fbe)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerHttpHandlerInternal.java (version 365cdc945e4a07b3c3869942cf7836e1fb323fbe)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/dispatcher/RunnableTaskLauncherTest.java (version 365cdc945e4a07b3c3869942cf7836e1fb323fbe)
  • wyzhang <wyzhang@google.com>

    wyzhang <wyzhang@google.com> 5306e50d23b8684b243f106bafb02a8ae04900ca

    address comments

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/worker/TaskWorkerHttpHandlerInternal.java (version 5306e50d23b8684b243f106bafb02a8ae04900ca)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/worker/TaskWorkerService.java (version 5306e50d23b8684b243f106bafb02a8ae04900ca)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/worker/TaskWorkerServiceLauncher.java (version 5306e50d23b8684b243f106bafb02a8ae04900ca)
  • wyzhang <wyzhang@google.com>

    wyzhang <wyzhang@google.com> 545ece40811abc4af23230da46865cfd136345d0

    Make TaskDispatcher http handler call TaskWorker http handler

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskDispatcherHttpHandlerInternal.java (version 545ece40811abc4af23230da46865cfd136345d0)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerHttpHandlerInternal.java (version 545ece40811abc4af23230da46865cfd136345d0)
  • wyzhang <wyzhang@google.com>

    wyzhang <wyzhang@google.com> 730261b83e2a8713167bb4ddacc87a68059d362f

    Checkpoint: task dispatcher http and worker http test passed

  • wyzhang <wyzhang@google.com>

    wyzhang <wyzhang@google.com> 0ba95977664a87ba269362b83b2a416a8278bb58

    Samll fix for missing default paramter causing null pointer exception

    • cdap-common/src/main/resources/cdap-default.xml (version 0ba95977664a87ba269362b83b2a416a8278bb58)
  • Masoud Saeida Ardekani

    Masoud Saeida Ardekani 41038781cb34e46efb2447cc4e9bf63c93690e06

    Add RunnableTask and removed job dispatcher http endpoint

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/EchoRunnableTask.java (version 41038781cb34e46efb2447cc4e9bf63c93690e06)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/RunnableTask.java (version 41038781cb34e46efb2447cc4e9bf63c93690e06)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/RunnableTaskRequest.java (version 41038781cb34e46efb2447cc4e9bf63c93690e06)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskDispatcherHttpHandlerInternal.java (version 41038781cb34e46efb2447cc4e9bf63c93690e06)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskDispatcherServer.java (version 41038781cb34e46efb2447cc4e9bf63c93690e06)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerHttpHandlerInternal.java (version 41038781cb34e46efb2447cc4e9bf63c93690e06)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerServiceLauncher.java (version 41038781cb34e46efb2447cc4e9bf63c93690e06)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/services/AppFabricServer.java (version 41038781cb34e46efb2447cc4e9bf63c93690e06)
    • cdap-master/src/main/java/io/cdap/cdap/master/environment/k8s/TaskDispatcherServiceMain.java (version 41038781cb34e46efb2447cc4e9bf63c93690e06)
    • cdap-master/src/test/java/io/cdap/cdap/master/environment/k8s/MasterServiceMainTestBase.java (version 41038781cb34e46efb2447cc4e9bf63c93690e06)
  • wyzhang <wyzhang@google.com>

    wyzhang <wyzhang@google.com> a5e1d9b1604876302f6d6d2bf47148bae9438742

    Adding missing TwillRunnerService startup, otherwise twill application launch fails with k8s apiClient null exception

    • cdap-master/src/main/java/io/cdap/cdap/master/environment/k8s/TaskDispatcherServiceMain.java (version a5e1d9b1604876302f6d6d2bf47148bae9438742)
  • Masoud Saeida Ardekani

    Masoud Saeida Ardekani d403e9162d47ea4c21d2748dede4b03efa7630ba

    Fix checkstyle error

    • cdap-common/src/main/java/io/cdap/cdap/common/conf/Constants.java (version d403e9162d47ea4c21d2748dede4b03efa7630ba)
  • wyzhang <wyzhang@google.com>

    wyzhang <wyzhang@google.com> 4c2bb3b5d7c681729fdcf63d403b39724bbec79c

    Make TaskDispatchServer launch worker pool via TaskWorkerTwillApplication

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskDispatcherServer.java (version 4c2bb3b5d7c681729fdcf63d403b39724bbec79c)
    • cdap-common/src/main/java/io/cdap/cdap/common/conf/Constants.java (version 4c2bb3b5d7c681729fdcf63d403b39724bbec79c)
    • cdap-common/src/main/resources/cdap-default.xml (version 4c2bb3b5d7c681729fdcf63d403b39724bbec79c)
    • cdap-master/src/main/java/io/cdap/cdap/master/environment/k8s/TaskDispatcherServiceMain.java (version 4c2bb3b5d7c681729fdcf63d403b39724bbec79c)
  • Masoud Saeida Ardekani

    Masoud Saeida Ardekani 32833cde542e1427ac13100dc69ee8a74ebf43a2

    Fix checkstyle error

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/RunnableTaskRequest.java (version 32833cde542e1427ac13100dc69ee8a74ebf43a2)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerHttpHandlerInternal.java (version 32833cde542e1427ac13100dc69ee8a74ebf43a2)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerServiceLauncher.java (version 32833cde542e1427ac13100dc69ee8a74ebf43a2)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerTwillApplication.java (version 32833cde542e1427ac13100dc69ee8a74ebf43a2)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerTwillRunnable.java (version 32833cde542e1427ac13100dc69ee8a74ebf43a2)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/services/AppFabricServer.java (version 32833cde542e1427ac13100dc69ee8a74ebf43a2)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerServiceTest.java (version 32833cde542e1427ac13100dc69ee8a74ebf43a2)
    • cdap-master/src/main/java/io/cdap/cdap/master/environment/k8s/TaskDispatcherServiceMain.java (version 32833cde542e1427ac13100dc69ee8a74ebf43a2)
  • Masoud Saeida Ardekani

    Masoud Saeida Ardekani f57a910f6b13de4a81c947920e50c6c756061a85

    Add RunnableTaskLauncher to load task runnable class, and run the task

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/RunnableTaskLauncher.java (version f57a910f6b13de4a81c947920e50c6c756061a85)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerHttpHandlerInternal.java (version f57a910f6b13de4a81c947920e50c6c756061a85)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/dispatcher/TestRunnableTaskLauncher.java (version f57a910f6b13de4a81c947920e50c6c756061a85)
  • wyzhang <wyzhang@google.com>

    wyzhang <wyzhang@google.com> c32d58031bda3fc9fd95d5cee202b100d8077271

    Adding http handler for task worker

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerHttpHandlerInternal.java (version c32d58031bda3fc9fd95d5cee202b100d8077271)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerService.java (version c32d58031bda3fc9fd95d5cee202b100d8077271)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerServiceTest.java (version c32d58031bda3fc9fd95d5cee202b100d8077271)
    • cdap-common/src/main/java/io/cdap/cdap/common/conf/Constants.java (version c32d58031bda3fc9fd95d5cee202b100d8077271)
    • cdap-common/src/main/resources/cdap-default.xml (version c32d58031bda3fc9fd95d5cee202b100d8077271)
  • wyzhang <wyzhang@google.com>

    wyzhang <wyzhang@google.com> e5aacc83780cebfe26eafdcc83daa4068c186fc4

    Test more than 1 worker: fix worker http server port and have 2 worker

    • cdap-common/src/main/resources/cdap-default.xml (version e5aacc83780cebfe26eafdcc83daa4068c186fc4)
  • wyzhang <wyzhang@google.com>

    wyzhang <wyzhang@google.com> 545aa2b0ab1c47eacbca2cd84da2cf7f6799ae13

    Checkpoint: task dispatcher and task worker pods & services are up

  • wyzhang <wyzhang@google.com>

    wyzhang <wyzhang@google.com> b3ac24625f20a571ef6ae69d31e383c24ece0177

    Add TaskDispatcherServiceMain

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskDispatcherHttpHandlerInternal.java (version b3ac24625f20a571ef6ae69d31e383c24ece0177)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskDispatcherServer.java (version b3ac24625f20a571ef6ae69d31e383c24ece0177)
    • cdap-common/src/main/java/io/cdap/cdap/common/conf/Constants.java (version b3ac24625f20a571ef6ae69d31e383c24ece0177)
    • cdap-common/src/main/resources/cdap-default.xml (version b3ac24625f20a571ef6ae69d31e383c24ece0177)
    • cdap-master/src/main/java/io/cdap/cdap/master/environment/k8s/TaskDispatcherServiceMain.java (version b3ac24625f20a571ef6ae69d31e383c24ece0177)
    • cdap-master/src/test/java/io/cdap/cdap/master/environment/k8s/MasterServiceMainTestBase.java (version b3ac24625f20a571ef6ae69d31e383c24ece0177)
    • cdap-master/src/test/java/io/cdap/cdap/master/environment/k8s/TaskDispatcherServiceServerMainTest.java (version b3ac24625f20a571ef6ae69d31e383c24ece0177)
  • wyzhang <wyzhang@google.com>

    wyzhang <wyzhang@google.com> e58d2d142afb65191ad0126751c386732d6c1006

    Add TaskWorkerTwillApp -> TaskRunnableTwillRunnable -> TaskWorkerService

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerService.java (version e58d2d142afb65191ad0126751c386732d6c1006)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerTwillApplication.java (version e58d2d142afb65191ad0126751c386732d6c1006)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerTwillRunnable.java (version e58d2d142afb65191ad0126751c386732d6c1006)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerServiceTest.java (version e58d2d142afb65191ad0126751c386732d6c1006)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerTwillRunnableTest.java (version e58d2d142afb65191ad0126751c386732d6c1006)
  • wyzhang <wyzhang@google.com>

    wyzhang <wyzhang@google.com> eddf07c2eb3271da8b553984cf7527979670c388

    Add copyrights

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskDispatcherHttpHandlerInternal.java (version eddf07c2eb3271da8b553984cf7527979670c388)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskDispatcherServer.java (version eddf07c2eb3271da8b553984cf7527979670c388)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerTwillApplication.java (version eddf07c2eb3271da8b553984cf7527979670c388)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/dispatcher/TaskWorkerTwillRunnable.java (version eddf07c2eb3271da8b553984cf7527979670c388)
    • cdap-master/src/main/java/io/cdap/cdap/master/environment/k8s/TaskDispatcherServiceMain.java (version eddf07c2eb3271da8b553984cf7527979670c388)
    • cdap-master/src/test/java/io/cdap/cdap/master/environment/k8s/TaskDispatcherServiceServerMainTest.java (version eddf07c2eb3271da8b553984cf7527979670c388)
  • Terence Yim <885032+chtyim@users.noreply.github.com>

    Terence Yim <885032+chtyim@users.noreply.github.com> 4b846b7b11d709c7e3de8816842eadccd36bee31

    Merge pull request #13371 from cdapio/feature/CDAP-17853-fix-controller-ooom
    (CDAP-17853) Multiple fixes around runtime service

  • Terence Yim

    Terence Yim d6381c5dd46e169e28fde3dfc2696df961838463 m

    (CDAP-17853) Multiple fixes around runtime service
    - Fix the temp directory creation in DistributedProgramRunner
    - Reduce synchronization scope in DistributedProgramRuntimeService
    - Simplify the AbstractProgramRuntimeService logic
    -- Simplify resource cleanup logic
    -- Avoid creating multiple RuntimeInfo
    - Modify the SystemProgramManagementServiceTest so that it doesn't depend
      on race condition to pass

    • cdap-app-fabric/src/main/java/io/cdap/cdap/app/runtime/AbstractProgramRuntimeService.java (version d6381c5dd46e169e28fde3dfc2696df961838463)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/AbstractTwillProgramController.java (version d6381c5dd46e169e28fde3dfc2696df961838463)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/DistributedProgramRunner.java (version d6381c5dd46e169e28fde3dfc2696df961838463)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/DistributedProgramRuntimeService.java (version d6381c5dd46e169e28fde3dfc2696df961838463)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/service/SimpleRuntimeInfo.java (version d6381c5dd46e169e28fde3dfc2696df961838463)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/app/runtime/AbstractProgramRuntimeServiceTest.java (version d6381c5dd46e169e28fde3dfc2696df961838463)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/services/SystemProgramManagementServiceTest.java (version d6381c5dd46e169e28fde3dfc2696df961838463)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/services/SystemProgramManagementTestApp.java (version d6381c5dd46e169e28fde3dfc2696df961838463)