Build #3,003

Cask Data Application Platform - Develop Build and Test

Build: #3003 failed Child of CDAP-DRC-5003

Code commits

caskdata/cdap

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> a08ce16ade8c60fc1de293f9bca203bb314586a7

    Merge pull request #12494 from cdapio/feature/CDAP-17078-transform-executor-refactor
    CDAP-17078 refactored MapReduceTransformExecutorFactory

  • sagarkapare <sagarkapare@google.com>

    sagarkapare <sagarkapare@google.com> 8a1270099406200bf9a64bf6efc706cd98ea7c64

    Merge pull request #12356 from cdapio/feature/CDAP-16931
    Implementation of preview request queue and polling service for standalone more.

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> f709c1e456a495a37ffef8a4196d5fcd5ecb5c77 m

    CDAP-17078 refactored MapReduceTransformExecutorFactory
    Refactored the transform executor factory used in mapreduce so
    that much of the logic can be re-used in Spark as well.

    • cdap-app-templates/cdap-etl/cdap-etl-batch/src/main/java/io/cdap/cdap/etl/batch/ConnectorSourceEmitter.java (version f709c1e456a495a37ffef8a4196d5fcd5ecb5c77)
    • cdap-app-templates/cdap-etl/cdap-etl-batch/src/main/java/io/cdap/cdap/etl/batch/DirectOutputPipeStage.java (version f709c1e456a495a37ffef8a4196d5fcd5ecb5c77)
    • cdap-app-templates/cdap-etl/cdap-etl-batch/src/main/java/io/cdap/cdap/etl/batch/MultiOutputTransformPipeStage.java (version f709c1e456a495a37ffef8a4196d5fcd5ecb5c77)
    • cdap-app-templates/cdap-etl/cdap-etl-batch/src/main/java/io/cdap/cdap/etl/batch/PipeEmitter.java (version f709c1e456a495a37ffef8a4196d5fcd5ecb5c77)
    • cdap-app-templates/cdap-etl/cdap-etl-batch/src/main/java/io/cdap/cdap/etl/batch/PipeStage.java (version f709c1e456a495a37ffef8a4196d5fcd5ecb5c77)
    • cdap-app-templates/cdap-etl/cdap-etl-batch/src/main/java/io/cdap/cdap/etl/batch/PipeTransformExecutor.java (version f709c1e456a495a37ffef8a4196d5fcd5ecb5c77)
    • cdap-app-templates/cdap-etl/cdap-etl-batch/src/main/java/io/cdap/cdap/etl/batch/StageFailureException.java (version f709c1e456a495a37ffef8a4196d5fcd5ecb5c77)
    • cdap-app-templates/cdap-etl/cdap-etl-batch/src/main/java/io/cdap/cdap/etl/batch/UnwrapPipeStage.java (version f709c1e456a495a37ffef8a4196d5fcd5ecb5c77)
    • cdap-app-templates/cdap-etl/cdap-etl-batch/src/main/java/io/cdap/cdap/etl/batch/mapreduce/ETLMapReduce.java (version f709c1e456a495a37ffef8a4196d5fcd5ecb5c77)
    • cdap-app-templates/cdap-etl/cdap-etl-batch/src/main/java/io/cdap/cdap/etl/batch/mapreduce/MapReduceTransformExecutorFactory.java (version f709c1e456a495a37ffef8a4196d5fcd5ecb5c77)
    • cdap-app-templates/cdap-etl/cdap-etl-batch/src/main/java/io/cdap/cdap/etl/batch/mapreduce/TransformRunner.java (version f709c1e456a495a37ffef8a4196d5fcd5ecb5c77)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/common/NoopStageStatisticsCollector.java (version f709c1e456a495a37ffef8a4196d5fcd5ecb5c77)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/exec/ConnectorSourceEmitter.java (version f709c1e456a495a37ffef8a4196d5fcd5ecb5c77)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/exec/DirectOutputPipeStage.java (version f709c1e456a495a37ffef8a4196d5fcd5ecb5c77)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/exec/MultiOutputTransformPipeStage.java (version f709c1e456a495a37ffef8a4196d5fcd5ecb5c77)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/exec/PipeEmitter.java (version f709c1e456a495a37ffef8a4196d5fcd5ecb5c77)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/exec/PipeStage.java (version f709c1e456a495a37ffef8a4196d5fcd5ecb5c77)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/exec/PipeTransformExecutor.java (version f709c1e456a495a37ffef8a4196d5fcd5ecb5c77)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/exec/StageFailureException.java (version f709c1e456a495a37ffef8a4196d5fcd5ecb5c77)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/exec/TransformExecutorFactory.java (version f709c1e456a495a37ffef8a4196d5fcd5ecb5c77)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/exec/UnwrapPipeStage.java (version f709c1e456a495a37ffef8a4196d5fcd5ecb5c77)
    • cdap-app-templates/cdap-etl/hydrator-spark-core/src/main/java/io/cdap/cdap/etl/spark/SparkTransformExecutorFactory.java (version f709c1e456a495a37ffef8a4196d5fcd5ecb5c77)
  • sagarkapare <sagarkapare@google.com>

    sagarkapare <sagarkapare@google.com> 175b520f1a05069c42fa9c73ae3315dc1e17cf18 m

    [CDAP-16931] Added methods to preview store to persist, remove, and list the preview requests that are in WAITING state. Also added implementation of preview request job queue backed by preview store.

    • cdap-app-fabric/src/main/java/io/cdap/cdap/app/preview/DefaultPreviewRunnerManager.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/app/preview/DefaultPreviewRunnerModule.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/app/preview/PreviewConfigModule.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/app/preview/PreviewHttpModule.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/app/preview/PreviewManager.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/app/preview/PreviewRequest.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/app/preview/PreviewRequestQueue.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/app/preview/PreviewRequestQueueState.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/app/preview/PreviewRunner.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/app/preview/PreviewRunnerManager.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/app/preview/PreviewRunnerManagerModule.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/app/preview/PreviewStatus.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/app/store/preview/PreviewStore.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/gateway/handlers/preview/PreviewHttpHandler.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/preview/DefaultPreviewManager.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/preview/DefaultPreviewRequestQueue.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/preview/DefaultPreviewRunner.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/preview/DirectPreviewRequestFetcherFactory.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/preview/PreviewRequestFetcherFactory.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/preview/PreviewRunnerService.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/preview/PreviewRunnerServiceStopper.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/store/preview/DefaultPreviewStore.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/master/environment/k8s/AbstractServiceMain.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/preview/DefaultPreviewManagerTest.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/preview/DefaultPreviewRequestQueueTest.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/preview/PreviewManagerTest.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/preview/PreviewRunnerServiceTest.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/store/preview/DefaultPreviewStoreTest.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/test/java/io/cdap/cdap/datapipeline/preview/PreviewDataPipelineTest.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-app-templates/cdap-etl/cdap-data-streams/src/test/java/io/cdap/cdap/datastreams/preview/PreviewDataStreamsTest.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-common/src/main/java/io/cdap/cdap/common/conf/Constants.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-common/src/main/resources/cdap-default.xml (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-data-fabric/src/main/java/io/cdap/cdap/data/runtime/preview/PreviewDataModules.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-master/src/main/java/io/cdap/cdap/master/environment/k8s/PreviewServiceMain.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-standalone/src/main/java/io/cdap/cdap/StandaloneMain.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-standalone/src/test/java/io/cdap/cdap/StandaloneMainTest.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)
    • cdap-unit-test/src/main/java/io/cdap/cdap/test/TestBase.java (version 175b520f1a05069c42fa9c73ae3315dc1e17cf18)