Build #488

Build: #488 failed Scheduled with changes by 3 people

Code commits

caskdata/cdap

  • Terence Yim <885032+chtyim@users.noreply.github.com>

    Terence Yim <885032+chtyim@users.noreply.github.com> 4d7beecb2bb36c08ccfc9d0f3819cf8061d0c579

    Merge pull request #12055 from cdapio/feature/CDAP-16369-remote-runtime
    (CDAP-16369) Remote Execution Runtime Redesign

  • Terence Yim

    Terence Yim 51466d79e1d81ca4035d043626660c472eeeb440

    (CDAP-16369) Improve progress logs in runtime client service
    - Progress log per topic and log total number of messages being processed

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/monitor/RuntimeClientService.java (version 51466d79e1d81ca4035d043626660c472eeeb440)
  • Terence Yim

    Terence Yim fa6a82fa1d654c35574d91bd7409c434be603cf8

    (CDAP-16369) Provisioner config and runtime argument to control monitor type
    - Based on CDAP config for each provisioner
    - Overridable by runtime arguments

    • cdap-app-fabric/src/main/java/io/cdap/cdap/app/guice/DistributedProgramContainerModule.java (version fa6a82fa1d654c35574d91bd7409c434be603cf8)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/common/twill/TwillAppLifecycleEventHandler.java (version fa6a82fa1d654c35574d91bd7409c434be603cf8)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/SystemArguments.java (version fa6a82fa1d654c35574d91bd7409c434be603cf8)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/batch/distributed/DistributedMapReduceTaskContextProvider.java (version fa6a82fa1d654c35574d91bd7409c434be603cf8)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/AbstractProgramTwillRunnable.java (version fa6a82fa1d654c35574d91bd7409c434be603cf8)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/DistributedProgramRunner.java (version fa6a82fa1d654c35574d91bd7409c434be603cf8)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/remote/RemoteExecutionDiscoveryService.java (version fa6a82fa1d654c35574d91bd7409c434be603cf8)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/remote/RemoteExecutionTwillRunnerService.java (version fa6a82fa1d654c35574d91bd7409c434be603cf8)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/remote/RemoteMonitorType.java (version fa6a82fa1d654c35574d91bd7409c434be603cf8)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/runtimejob/DefaultRuntimeJob.java (version fa6a82fa1d654c35574d91bd7409c434be603cf8)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/monitor/RuntimeMonitors.java (version fa6a82fa1d654c35574d91bd7409c434be603cf8)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/provision/ProvisioningService.java (version fa6a82fa1d654c35574d91bd7409c434be603cf8)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/runtime/distributed/remote/RemoteExecutionDiscoveryServiceTest.java (version fa6a82fa1d654c35574d91bd7409c434be603cf8)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/runtime/distributed/runtimejob/DefaultRuntimeJobTest.java (version fa6a82fa1d654c35574d91bd7409c434be603cf8)
    • cdap-common/src/main/java/io/cdap/cdap/common/conf/Constants.java (version fa6a82fa1d654c35574d91bd7409c434be603cf8)
    • cdap-spark-core-base/src/main/java/io/cdap/cdap/app/runtime/spark/SparkRuntimeContextProvider.java (version fa6a82fa1d654c35574d91bd7409c434be603cf8)
  • Terence Yim

    Terence Yim f19805f9864f6ca1b4d77f77640f6c451a3ab22b

    (CDAP-16369) Delete unused class
    - Monitoring proxy classes are not needed

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/monitor/DirectRuntimeRequestValidator.java (version f19805f9864f6ca1b4d77f77640f6c451a3ab22b)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/monitor/MessagesBodyProducer.java (version f19805f9864f6ca1b4d77f77640f6c451a3ab22b)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/monitor/MonitorConsumeRequest.java (version f19805f9864f6ca1b4d77f77640f6c451a3ab22b)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/monitor/MonitorMessage.java (version f19805f9864f6ca1b4d77f77640f6c451a3ab22b)
    • cdap-common/src/main/java/io/cdap/cdap/common/conf/Constants.java (version f19805f9864f6ca1b4d77f77640f6c451a3ab22b)
  • Terence Yim

    Terence Yim 9e04c5149b1b036a7b6fdbc384eb383f9a9741cb

    (CDAP-16369) Fixied a ByteBuf leak in runtime server

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/monitor/RuntimeHandler.java (version 9e04c5149b1b036a7b6fdbc384eb383f9a9741cb)
  • Terence Yim

    Terence Yim 680708a7035b4143de032add9f8d3ad73d7cf71e

    (CDAP-16369) Added RuntimeServer to standalone and Hadoop mode
    - In hadoop mode, runtime server runs inside app-fabric instead
      of separate service in YARN. This is due to hadoop support
      is deprecated since 6.2, so we are not going to make a lot of
      effort to make it scalabile on YARN

    • cdap-app-fabric/src/main/java/io/cdap/cdap/app/guice/RuntimeServerModule.java (version 680708a7035b4143de032add9f8d3ad73d7cf71e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/monitor/RuntimeHandler.java (version 680708a7035b4143de032add9f8d3ad73d7cf71e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/monitor/RuntimeServer.java (version 680708a7035b4143de032add9f8d3ad73d7cf71e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/monitor/RuntimeServiceRoutingHandler.java (version 680708a7035b4143de032add9f8d3ad73d7cf71e)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/runtime/monitor/RuntimeClientServerTest.java (version 680708a7035b4143de032add9f8d3ad73d7cf71e)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/runtime/monitor/RuntimeClientServiceTest.java (version 680708a7035b4143de032add9f8d3ad73d7cf71e)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/runtime/monitor/RuntimeServiceRoutingTest.java (version 680708a7035b4143de032add9f8d3ad73d7cf71e)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/services/http/handlers/ProgramLifecycleHttpHandlerInternalTest.java (version 680708a7035b4143de032add9f8d3ad73d7cf71e)
    • cdap-common/src/main/java/io/cdap/cdap/internal/app/store/RunRecordDetail.java (version 680708a7035b4143de032add9f8d3ad73d7cf71e)
    • cdap-master/src/main/java/io/cdap/cdap/data/runtime/main/MasterServiceMain.java (version 680708a7035b4143de032add9f8d3ad73d7cf71e)
    • cdap-standalone/src/main/java/io/cdap/cdap/StandaloneMain.java (version 680708a7035b4143de032add9f8d3ad73d7cf71e)
    • cdap-watchdog/src/main/java/io/cdap/cdap/logging/guice/LogQueryRuntimeModule.java (version 680708a7035b4143de032add9f8d3ad73d7cf71e)
  • Terence Yim

    Terence Yim 482684a10e9c51560f0f7221c3a9be37573b8864

    (CDAP-16369) Change the RuntimeJobManager interface
    - Remove init and destroy, extends from Closeable to make it easier to use

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/remote/RemoteExecutionJobMain.java (version 482684a10e9c51560f0f7221c3a9be37573b8864)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/remote/RemoteExecutionTwillRunnerService.java (version 482684a10e9c51560f0f7221c3a9be37573b8864)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/remote/RuntimeJobRemoteProcessController.java (version 482684a10e9c51560f0f7221c3a9be37573b8864)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/remote/RuntimeJobTwillPreparer.java (version 482684a10e9c51560f0f7221c3a9be37573b8864)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/runtimejob/DefaultRuntimeJob.java (version 482684a10e9c51560f0f7221c3a9be37573b8864)
    • cdap-runtime-ext-dataproc/src/main/java/io/cdap/cdap/runtime/spi/provisioner/dataproc/DataprocProvisioner.java (version 482684a10e9c51560f0f7221c3a9be37573b8864)
    • cdap-runtime-ext-dataproc/src/main/java/io/cdap/cdap/runtime/spi/runtimejob/DataprocJobMain.java (version 482684a10e9c51560f0f7221c3a9be37573b8864)
    • cdap-runtime-ext-dataproc/src/main/java/io/cdap/cdap/runtime/spi/runtimejob/DataprocRuntimeJobManager.java (version 482684a10e9c51560f0f7221c3a9be37573b8864)
    • cdap-runtime-spi/src/main/java/io/cdap/cdap/runtime/spi/runtimejob/RuntimeJob.java (version 482684a10e9c51560f0f7221c3a9be37573b8864)
    • cdap-runtime-spi/src/main/java/io/cdap/cdap/runtime/spi/runtimejob/RuntimeJobManager.java (version 482684a10e9c51560f0f7221c3a9be37573b8864)
  • Terence Yim

    Terence Yim fc4ece49a295409671f0d9f077ba24fb861e2d0e

    (CDAP-16369) Change job launching and monitoring mechanism
    - Remote runtime actively talking to CDAP
    - Supports either SSH tunnel or CDAP exposes URL
    - Unified SSH and RuntimeJob launch code

    • cdap-app-fabric/src/main/java/io/cdap/cdap/app/guice/RemoteExecutionProgramRunnerModule.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/batch/distributed/DistributedMapReduceTaskContextProvider.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/AbstractProgramTwillRunnable.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/AbstractRuntimeTwillController.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/AbstractRuntimeTwillPreparer.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/remote/AbstractRuntimeTwillPreparer.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/remote/RemoteExecutionDiscoveryService.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/remote/RemoteExecutionJobMain.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/remote/RemoteExecutionService.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/remote/RemoteExecutionTwillController.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/remote/RemoteExecutionTwillPreparer.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/remote/RemoteExecutionTwillRunnerService.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/remote/RemoteProcessController.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/remote/RemoteRuntimeTable.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/remote/RuntimeJobRemoteProcessController.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/remote/RuntimeJobTwillPreparer.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/remote/SSHRemoteExecutionService.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/remote/SSHRemoteProcessController.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/remote/SSHSessionManager.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/remote/TwillControllerFactory.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/remote/package-info.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/runtime/RuntimeTwillRunnerService.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/runtime/TwillControllerFactory.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/runtimejob/DefaultRuntimeInfo.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/runtimejob/DefaultRuntimeJob.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/runtimejob/RuntimeJobTwillController.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/runtimejob/RuntimeJobTwillPreparer.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/runtimejob/RuntimeJobTwillRunnerService.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/monitor/GceRemoteAuthenticator.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/monitor/RuntimeClientService.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/monitor/RuntimeMonitor.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/monitor/RuntimeMonitorClient.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/monitor/RuntimeMonitorServer.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/monitor/RuntimeMonitorServerInfo.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/monitor/RuntimeMonitors.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/monitor/RuntimeServer.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/monitor/proxy/MonitorSocksProxy.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/monitor/proxy/MonitorSocksServerConnectHandler.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/monitor/proxy/MonitorSocksServerHandler.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/monitor/proxy/PortForwardingProvider.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/services/ProgramCompletionNotifier.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/services/ProgramNotificationSubscriberService.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/profile/ProfileMetricService.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/provision/ProvisioningService.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/provision/task/ClusterCreateSubtask.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/app/runtime/monitor/RuntimeMonitorTest.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/runtime/distributed/remote/RemoteExecutionJobMainTest.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/runtime/distributed/remote/RemoteRuntimeTableTest.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/runtime/distributed/remote/SSHSessionManagerTest.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/runtime/distributed/runtimejob/DefaultRuntimeJobTest.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/runtime/monitor/RuntimeClientServerTest.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/runtime/monitor/RuntimeClientServiceTest.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/runtime/monitor/RuntimeServiceRoutingTest.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/runtime/monitor/proxy/MonitorSocksProxyTest.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/profile/ProfileMetricServiceTest.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-common/src/main/java/io/cdap/cdap/common/conf/Constants.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-common/src/main/java/io/cdap/cdap/common/discovery/URIScheme.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-common/src/main/java/io/cdap/cdap/common/internal/remote/RemoteAuthenticator.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-common/src/main/java/io/cdap/cdap/common/internal/remote/RemoteClient.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-common/src/main/java/io/cdap/cdap/common/service/AbstractRetryableScheduledService.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-common/src/main/resources/cdap-default.xml (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-master/src/test/java/io/cdap/cdap/master/environment/k8s/MasterServiceMainTestBase.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-runtime-ext-dataproc/src/main/java/io/cdap/cdap/runtime/spi/runtimejob/DataprocJobMain.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-runtime-ext-dataproc/src/main/java/io/cdap/cdap/runtime/spi/runtimejob/DataprocRuntimeJobManager.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-runtime-spi/src/main/java/io/cdap/cdap/runtime/spi/runtimejob/RuntimeJob.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-spark-core-base/src/main/java/io/cdap/cdap/app/runtime/spark/SparkRuntimeContextProvider.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-watchdog/src/main/java/io/cdap/cdap/logging/appender/AbstractLogPublisher.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-watchdog/src/main/java/io/cdap/cdap/logging/appender/LogAppenderInitializer.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-watchdog/src/main/java/io/cdap/cdap/logging/appender/loader/LogAppenderLoaderService.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
    • cdap-watchdog/src/main/java/io/cdap/cdap/logging/appender/tms/TMSLogAppender.java (version fc4ece49a295409671f0d9f077ba24fb861e2d0e)
  • Terence Yim

    Terence Yim 38f0487cb334ae996d64555de565c549f68369af

    (CDAP-16369) Improve Dataproc job manager stop() implementation
    - Instead of always issuing stop, check the job status first

    • cdap-runtime-ext-dataproc/src/main/java/io/cdap/cdap/runtime/spi/runtimejob/DataprocRuntimeJobManager.java (version 38f0487cb334ae996d64555de565c549f68369af)
  • Terence Yim

    Terence Yim ca3b26509e1dee0f2f5d1c98d62d00cef4da5556

    (CDAP-16369) Added getUserArgs() method to RunRecodDetail
    - Unified calls for getting runtime arguments from RunRecordDetail
    - Also fixed deserialization usage of RunRecordDetail

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/remote/RemoteExecutionTwillRunnerService.java (version ca3b26509e1dee0f2f5d1c98d62d00cef4da5556)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/monitor/DirectRuntimeRequestValidator.java (version ca3b26509e1dee0f2f5d1c98d62d00cef4da5556)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/schedule/trigger/TriggerInfoContext.java (version ca3b26509e1dee0f2f5d1c98d62d00cef4da5556)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/store/DefaultStore.java (version ca3b26509e1dee0f2f5d1c98d62d00cef4da5556)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/provision/task/ClusterCreateSubtask.java (version ca3b26509e1dee0f2f5d1c98d62d00cef4da5556)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/reporting/ProgramHeartbeatTable.java (version ca3b26509e1dee0f2f5d1c98d62d00cef4da5556)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/security/impersonation/DefaultUGIProvider.java (version ca3b26509e1dee0f2f5d1c98d62d00cef4da5556)
    • cdap-common/src/main/java/io/cdap/cdap/common/conf/Constants.java (version ca3b26509e1dee0f2f5d1c98d62d00cef4da5556)
    • cdap-common/src/main/java/io/cdap/cdap/internal/app/store/RunRecordDetail.java (version ca3b26509e1dee0f2f5d1c98d62d00cef4da5556)
    • cdap-watchdog/src/main/java/io/cdap/cdap/logging/gateway/handlers/RemoteProgramRunRecordFetcher.java (version ca3b26509e1dee0f2f5d1c98d62d00cef4da5556)
  • Terence Yim

    Terence Yim a4e1668c1942d00cfe842801ed53c15d6595ad06

    (CDAP-16369) Move TrafficRelayServer and ProxySelector out from RuntimeMonitorServer
    - This is for preparing the removal of RuntimeMonitorServer

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/runtimejob/DefaultRuntimeJob.java (version a4e1668c1942d00cfe842801ed53c15d6595ad06)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/monitor/RuntimeMonitorServer.java (version a4e1668c1942d00cfe842801ed53c15d6595ad06)
  • Terence Yim

    Terence Yim 30537f1ba9aae9219c930c849b5ef31a8d24a925

    (CDAP-16369) Added a cursor based scan active run records to AppMetadataStore
    - This allows scanning active run records efficiently across all namespaces
      without holding up large amount of memory

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/store/AppMetadataStore.java (version 30537f1ba9aae9219c930c849b5ef31a8d24a925)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/store/AppMetadataStoreTest.java (version 30537f1ba9aae9219c930c849b5ef31a8d24a925)
  • Terence Yim

    Terence Yim 631346a191b0296a7d0fac43ef21a35e28a0f07b

    (CDAP-16369) Persist all system arguments during program start

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/store/AppMetadataStore.java (version 631346a191b0296a7d0fac43ef21a35e28a0f07b)
    • cdap-common/src/main/java/io/cdap/cdap/internal/app/store/RunRecordDetail.java (version 631346a191b0296a7d0fac43ef21a35e28a0f07b)
  • Terence Yim <885032+chtyim@users.noreply.github.com>

    Terence Yim <885032+chtyim@users.noreply.github.com> 0f30ccd1df99ed11f0fefa3ee61ed43614bb5dbe

    Merge pull request #12057 from cdapio/feature/CDAP-16521-start-async
    (CDAP-16521) Turn program start completely async

  • Terence Yim

    Terence Yim 9794576be3a2887a56b169de0ff5c5f9e4dec741

    (CDAP-16521) Turn program start completely async
    - Have a separate executor in the ProgramRuntimeService for program launching

    • cdap-app-fabric/src/main/java/io/cdap/cdap/app/runtime/AbstractProgramRuntimeService.java (version 9794576be3a2887a56b169de0ff5c5f9e4dec741)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/app/runtime/DelayedProgramController.java (version 9794576be3a2887a56b169de0ff5c5f9e4dec741)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/app/runtime/ProgramController.java (version 9794576be3a2887a56b169de0ff5c5f9e4dec741)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/AbstractProgramController.java (version 9794576be3a2887a56b169de0ff5c5f9e4dec741)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/ProgramControllerServiceAdapter.java (version 9794576be3a2887a56b169de0ff5c5f9e4dec741)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/DistributedProgramRuntimeService.java (version 9794576be3a2887a56b169de0ff5c5f9e4dec741)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/service/InMemoryProgramRuntimeService.java (version 9794576be3a2887a56b169de0ff5c5f9e4dec741)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/service/ServiceProgramRunner.java (version 9794576be3a2887a56b169de0ff5c5f9e4dec741)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/service/SimpleRuntimeInfo.java (version 9794576be3a2887a56b169de0ff5c5f9e4dec741)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/worker/WorkerProgramRunner.java (version 9794576be3a2887a56b169de0ff5c5f9e4dec741)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/services/ProgramNotificationSubscriberService.java (version 9794576be3a2887a56b169de0ff5c5f9e4dec741)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/app/runtime/AbstractProgramRuntimeServiceTest.java (version 9794576be3a2887a56b169de0ff5c5f9e4dec741)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/app/runtime/DelayedProgramControllerTest.java (version 9794576be3a2887a56b169de0ff5c5f9e4dec741)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/services/RunRecordCorrectorServiceTest.java (version 9794576be3a2887a56b169de0ff5c5f9e4dec741)
    • cdap-unit-test/src/test/java/io/cdap/cdap/test/app/TestFrameworkTestRun.java (version 9794576be3a2887a56b169de0ff5c5f9e4dec741)
  • Terence Yim <885032+chtyim@users.noreply.github.com>

    Terence Yim <885032+chtyim@users.noreply.github.com> d450b14b7e9a7ab146ab4c7bf33369f78045424f

    Merge pull request #12056 from cdapio/feature/CDAP-16538
    (CDAP-16538) Make sure the plugin properties are immutable

  • Ajai <1452845+ajainarayanan@users.noreply.github.com>

    Ajai <1452845+ajainarayanan@users.noreply.github.com> afd0cfc3ee81b7f377058192d110f3f73ff2461b

    Merge pull request #12053 from cdapio/bugfix-ui/CDAP-16567
    [CDAP-16567] Properly fix styling for plugin configuration title

  • Terence Yim

    Terence Yim 1319362f4a764fd8b24da733be4da3018b2a4aed

    (CDAP-16538) Make sure the plugin properties are immutable
    - Even though the builder is building an immutable map, when the properties object is deserialized from JSON, Gson use a modifiable map implementation.
    - This lead to bug PLUGIN-128

    • cdap-api/src/main/java/io/cdap/cdap/api/plugin/PluginProperties.java (version 1319362f4a764fd8b24da733be4da3018b2a4aed)
  • Ajai Narayan

    Ajai Narayan f0b9a1e09a7640fca1bc5a5c77949fd8daa9afba m

    [CDAP-16567] Properly fix styling for plugin configuration title

    • cdap-ui/app/cdap/components/ConfigurationGroup/index.tsx (version f0b9a1e09a7640fca1bc5a5c77949fd8daa9afba)