Build #809

Build: #809 failed Scheduled with changes by 4 people

Code commits

caskdata/cdap

  • Terence Yim <885032+chtyim@users.noreply.github.com>

    Terence Yim <885032+chtyim@users.noreply.github.com> 7031347218eba6b7e95bfa8d2f8391fbd782c5d3

    Merge pull request #13081 from githubwua/develop
    Resolve "java.lang.NumberFormatException: multiple points"

  • Terence Yim <885032+chtyim@users.noreply.github.com>

    Terence Yim <885032+chtyim@users.noreply.github.com> 2ee9340160df03d6a06bee97b7f1902bf103216d

    Merge pull request #13096 from cdapio/feature/CDAP-17610-oauth-macro
    (CDAP-17610) Adding a new macro function for OAuth integration

  • Vitalii Tymchyshyn

    Vitalii Tymchyshyn 972ec0c64064ca8198e299edf76bed7502e06290

    Merge pull request #13137 from cdapio/feature/CDAP-17706-allow-cache-disable
    [CDAP-17706] Allow to disable function cache

  • Terence Yim

    Terence Yim 9b27db84eaa356c51486e72de1e0e785e66d6add

    Close connection in unit-test

    • cdap-unit-test/src/test/java/io/cdap/cdap/test/app/TestFrameworkTestRun.java (version 9b27db84eaa356c51486e72de1e0e785e66d6add)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 23a1e8c9ff5f45c1d5016b4e5093bbd7641e4c7d

    Merge pull request #13134 from cdapio/feature/CDAP-17606-join-expr-validate
    CDAP-17606 basic validation for join condition expressions

  • Vitalii Tymchyshyn

    Vitalii Tymchyshyn 354da91cef7f61377bd9ca3021b06ce634f19625 m

    [CDAP-17706] Allow to disable function cache

    • cdap-app-templates/cdap-etl/cdap-data-streams/src/main/java/io/cdap/cdap/datastreams/SparkStreamingPipelineDriver.java (version 354da91cef7f61377bd9ca3021b06ce634f19625)
    • cdap-app-templates/cdap-etl/cdap-data-streams/src/main/java/io/cdap/cdap/datastreams/SparkStreamingPipelineRunner.java (version 354da91cef7f61377bd9ca3021b06ce634f19625)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/common/Constants.java (version 354da91cef7f61377bd9ca3021b06ce634f19625)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/SparkPipelineRunner.java (version 354da91cef7f61377bd9ca3021b06ce634f19625)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/batch/BaseRDDCollection.java (version 354da91cef7f61377bd9ca3021b06ce634f19625)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/batch/BatchSparkPipelineDriver.java (version 354da91cef7f61377bd9ca3021b06ce634f19625)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/batch/PairRDDCollection.java (version 354da91cef7f61377bd9ca3021b06ce634f19625)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/function/FunctionCache.java (version 354da91cef7f61377bd9ca3021b06ce634f19625)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/streaming/DStreamCollection.java (version 354da91cef7f61377bd9ca3021b06ce634f19625)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/streaming/PairDStreamCollection.java (version 354da91cef7f61377bd9ca3021b06ce634f19625)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/test/java/io/cdap/cdap/etl/spark/function/FunctionCacheTest.java (version 354da91cef7f61377bd9ca3021b06ce634f19625)
    • cdap-app-templates/cdap-etl/hydrator-spark-core/src/main/java/io/cdap/cdap/etl/spark/batch/RDDCollection.java (version 354da91cef7f61377bd9ca3021b06ce634f19625)
    • cdap-app-templates/cdap-etl/hydrator-spark-core2_2.11/src/main/java/io/cdap/cdap/etl/spark/batch/RDDCollection.java (version 354da91cef7f61377bd9ca3021b06ce634f19625)
  • Vitalii Tymchyshyn

    Vitalii Tymchyshyn 8f1fef252e6e802938ccc2c945bc0bd1107b417d

    Merge pull request #13131 from cdapio/feature/remove-travis-cloud-build
    Remove Travis & Cloud Build.

  • Vitalii Tymchyshyn

    Vitalii Tymchyshyn 7c5f3572bc01ec7a667d1adffc6467633d41b3f5

    Merge pull request #13132 from cdapio/feature/remove-junit-report
    Remove SureFire report as it does not work with PRs from forks

  • Terence Yim

    Terence Yim 25b8c1ab6f68348051f2c953110c12bb7c18f4ef

    (CDAP-17610) Added OAuth macro function
    - Expanded plugin config to support object type fields
    - Added a new OAuth macro function that uses the studio app for getting oauth information

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/artifact/ArtifactInspector.java (version 25b8c1ab6f68348051f2c953110c12bb7c18f4ef)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/plugin/PluginInstantiator.java (version 25b8c1ab6f68348051f2c953110c12bb7c18f4ef)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/plugins/test/TestPlugin.java (version 25b8c1ab6f68348051f2c953110c12bb7c18f4ef)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/runtime/artifact/ArtifactRepositoryTest.java (version 25b8c1ab6f68348051f2c953110c12bb7c18f4ef)
    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/main/java/io/cdap/cdap/datapipeline/service/SecureStoreMacroEvaluator.java (version 25b8c1ab6f68348051f2c953110c12bb7c18f4ef)
    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/main/java/io/cdap/cdap/datapipeline/service/StudioService.java (version 25b8c1ab6f68348051f2c953110c12bb7c18f4ef)
    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/main/java/io/cdap/cdap/datapipeline/service/ValidationHandler.java (version 25b8c1ab6f68348051f2c953110c12bb7c18f4ef)
    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/test/java/io/cdap/cdap/datapipeline/DataPipelineServiceTest.java (version 25b8c1ab6f68348051f2c953110c12bb7c18f4ef)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/common/Constants.java (version 25b8c1ab6f68348051f2c953110c12bb7c18f4ef)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/common/DefaultMacroEvaluator.java (version 25b8c1ab6f68348051f2c953110c12bb7c18f4ef)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/common/OAuthMacroEvaluator.java (version 25b8c1ab6f68348051f2c953110c12bb7c18f4ef)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/common/SecureStoreMacroEvaluator.java (version 25b8c1ab6f68348051f2c953110c12bb7c18f4ef)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/common/macro/LogicalStartTimeMacro.java (version 25b8c1ab6f68348051f2c953110c12bb7c18f4ef)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/common/macro/LogicalStartTimeMacroEvaluator.java (version 25b8c1ab6f68348051f2c953110c12bb7c18f4ef)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/test/java/io/cdap/cdap/etl/common/OAuthMacroEvaluatorTest.java (version 25b8c1ab6f68348051f2c953110c12bb7c18f4ef)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/test/java/io/cdap/cdap/etl/common/macro/LogicalStartTimeMacroEvaluatorTest.java (version 25b8c1ab6f68348051f2c953110c12bb7c18f4ef)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/test/java/io/cdap/cdap/etl/common/macro/LogicalStartTimeMacroTest.java (version 25b8c1ab6f68348051f2c953110c12bb7c18f4ef)
  • Terence Yim

    Terence Yim 92b3eb645b952d44101164e275b2dea6bb9a222b

    (CDAP-17610) Added support for system app service discovery
    - Allows runtime to discovery and communicate with system app services

    • cdap-api/src/main/java/io/cdap/cdap/api/ServiceDiscoverer.java (version 92b3eb645b952d44101164e275b2dea6bb9a222b)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/app/services/AbstractServiceDiscoverer.java (version 92b3eb645b952d44101164e275b2dea6bb9a222b)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/batch/MapReduceLifecycleContext.java (version 92b3eb645b952d44101164e275b2dea6bb9a222b)
    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/app/runtime/distributed/remote/RemoteExecutionDiscoveryService.java (version 92b3eb645b952d44101164e275b2dea6bb9a222b)
    • cdap-app-fabric/src/test/java/io/cdap/cdap/internal/app/runtime/service/http/HttpHandlerGeneratorTest.java (version 92b3eb645b952d44101164e275b2dea6bb9a222b)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/common/AbstractStageContext.java (version 92b3eb645b952d44101164e275b2dea6bb9a222b)
    • cdap-app-templates/cdap-etl/hydrator-test/src/main/java/io/cdap/cdap/etl/mock/action/MockActionContext.java (version 92b3eb645b952d44101164e275b2dea6bb9a222b)
    • cdap-app-templates/cdap-etl/hydrator-test/src/main/java/io/cdap/cdap/etl/mock/transform/MockTransformContext.java (version 92b3eb645b952d44101164e275b2dea6bb9a222b)
    • cdap-common/src/main/java/io/cdap/cdap/common/internal/remote/RemoteClient.java (version 92b3eb645b952d44101164e275b2dea6bb9a222b)
    • cdap-spark-core-base/src/main/java/io/cdap/cdap/app/runtime/spark/BasicSparkClientContext.java (version 92b3eb645b952d44101164e275b2dea6bb9a222b)
    • cdap-spark-core-base/src/main/java/io/cdap/cdap/app/runtime/spark/SparkServiceDiscoverer.java (version 92b3eb645b952d44101164e275b2dea6bb9a222b)
  • Terence Yim

    Terence Yim f0b91addbe9d67b9582fb07fedca397a083e70c7

    (CDAP-17610) Refactor to pass in ServiceDiscoverer for Macro usage

    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/main/java/io/cdap/cdap/datapipeline/ExternalSparkProgram.java (version f0b91addbe9d67b9582fb07fedca397a083e70c7)
    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/main/java/io/cdap/cdap/datapipeline/JavaSparkMainWrapper.java (version f0b91addbe9d67b9582fb07fedca397a083e70c7)
    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/main/java/io/cdap/cdap/datapipeline/SmartWorkflow.java (version f0b91addbe9d67b9582fb07fedca397a083e70c7)
    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/main/scala/io/cdap/cdap/datapipeline/ScalaSparkMainWrapper.scala (version f0b91addbe9d67b9582fb07fedca397a083e70c7)
    • cdap-app-templates/cdap-etl/cdap-data-streams/src/main/java/io/cdap/cdap/datastreams/SparkStreamingPipelineDriver.java (version f0b91addbe9d67b9582fb07fedca397a083e70c7)
    • cdap-app-templates/cdap-etl/cdap-etl-batch/src/main/java/io/cdap/cdap/etl/batch/condition/PipelineCondition.java (version f0b91addbe9d67b9582fb07fedca397a083e70c7)
    • cdap-app-templates/cdap-etl/cdap-etl-batch/src/main/java/io/cdap/cdap/etl/batch/customaction/PipelineAction.java (version f0b91addbe9d67b9582fb07fedca397a083e70c7)
    • cdap-app-templates/cdap-etl/cdap-etl-batch/src/main/java/io/cdap/cdap/etl/batch/mapreduce/ETLMapReduce.java (version f0b91addbe9d67b9582fb07fedca397a083e70c7)
    • cdap-app-templates/cdap-etl/cdap-etl-batch/src/main/java/io/cdap/cdap/etl/batch/mapreduce/MapReduceTransformExecutorFactory.java (version f0b91addbe9d67b9582fb07fedca397a083e70c7)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/common/AbstractStageContext.java (version f0b91addbe9d67b9582fb07fedca397a083e70c7)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/common/DefaultMacroEvaluator.java (version f0b91addbe9d67b9582fb07fedca397a083e70c7)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/SparkPipelineRunner.java (version f0b91addbe9d67b9582fb07fedca397a083e70c7)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/batch/ETLSpark.java (version f0b91addbe9d67b9582fb07fedca397a083e70c7)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/function/MultiSinkFunction.java (version f0b91addbe9d67b9582fb07fedca397a083e70c7)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/function/PluginFunctionContext.java (version f0b91addbe9d67b9582fb07fedca397a083e70c7)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/streaming/function/StreamingAlertPublishFunction.java (version f0b91addbe9d67b9582fb07fedca397a083e70c7)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/streaming/function/StreamingBatchSinkFunction.java (version f0b91addbe9d67b9582fb07fedca397a083e70c7)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/streaming/function/StreamingMultiSinkFunction.java (version f0b91addbe9d67b9582fb07fedca397a083e70c7)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/streaming/function/StreamingSparkSinkFunction.java (version f0b91addbe9d67b9582fb07fedca397a083e70c7)
  • Terence Yim <885032+chtyim@users.noreply.github.com>

    Terence Yim <885032+chtyim@users.noreply.github.com> 0a6e51f4eb66e2b3caaf4ae1f2a8ea5b53f5a75a

    Merge pull request #13093 from ssmith-google/develop
    Add new system APIs to support OAuth in pipelines

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 041d9c956cfbd125c550e96b5230a73f758db392 m

    CDAP-17606 basic validation for join condition expressions
    Added validation to ensure that join condition expressions are
    not allowed in MapReduce pipelines or streaming pipelines.
    Also added validation to ensure that input schemas are always
    known when using expressions.

    • cdap-app-templates/cdap-etl/cdap-data-streams/src/main/java/io/cdap/cdap/datastreams/DataStreamsPipelineSpecGenerator.java (version 041d9c956cfbd125c550e96b5230a73f758db392)
    • cdap-app-templates/cdap-etl/cdap-etl-api/src/main/java/io/cdap/cdap/etl/api/join/JoinCondition.java (version 041d9c956cfbd125c550e96b5230a73f758db392)
    • cdap-app-templates/cdap-etl/cdap-etl-api/src/main/java/io/cdap/cdap/etl/api/join/error/ExpressionConditionError.java (version 041d9c956cfbd125c550e96b5230a73f758db392)
    • cdap-app-templates/cdap-etl/cdap-etl-api/src/test/java/io/cdap/cdap/etl/api/join/JoinDefinitionTest.java (version 041d9c956cfbd125c550e96b5230a73f758db392)
    • cdap-app-templates/cdap-etl/cdap-etl-batch/src/main/java/io/cdap/cdap/etl/batch/mapreduce/MapReduceTransformExecutorFactory.java (version 041d9c956cfbd125c550e96b5230a73f758db392)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/spec/PipelineSpecGenerator.java (version 041d9c956cfbd125c550e96b5230a73f758db392)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/test/java/io/cdap/cdap/etl/spec/PipelineSpecGeneratorTest.java (version 041d9c956cfbd125c550e96b5230a73f758db392)
  • Vitalii Tymchyshyn

    Vitalii Tymchyshyn 5f55ff0e9affbddf8d7c53a9e8fbe13d7036ef77 m

    Remove SureFire report as it does not work with PRs from forks due to https://github.com/ScaCap/action-surefire-report/issues/31

    • .github/workflows/maven.yml (version 5f55ff0e9affbddf8d7c53a9e8fbe13d7036ef77)
  • Vitalii Tymchyshyn

    Vitalii Tymchyshyn 64bc626e2b699773f60a9ce24ec87cc20298a052 m

    Remove Travis & Cloud Build.

    • .travis.yml (version 64bc626e2b699773f60a9ce24ec87cc20298a052)
    • cloudbuild.sh (version 64bc626e2b699773f60a9ce24ec87cc20298a052)
    • cloudbuild.yaml (version 64bc626e2b699773f60a9ce24ec87cc20298a052)
  • Steven Smith <stevely@google.com>

    Steven Smith <stevely@google.com> 785759b8f3a2ec5d9e79dc7073a7ff8915b42b2d m

    Add set of APIs for supporting OAuth authentication in pipelines

    • cdap-app-templates/cdap-etl/cdap-data-pipeline/pom.xml (version 785759b8f3a2ec5d9e79dc7073a7ff8915b42b2d)
    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/main/java/io/cdap/cdap/datapipeline/oauth/CredentialIsValidResponse.java (version 785759b8f3a2ec5d9e79dc7073a7ff8915b42b2d)
    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/main/java/io/cdap/cdap/datapipeline/oauth/GetAccessTokenResponse.java (version 785759b8f3a2ec5d9e79dc7073a7ff8915b42b2d)
    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/main/java/io/cdap/cdap/datapipeline/oauth/OAuthClientCredentials.java (version 785759b8f3a2ec5d9e79dc7073a7ff8915b42b2d)
    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/main/java/io/cdap/cdap/datapipeline/oauth/OAuthProvider.java (version 785759b8f3a2ec5d9e79dc7073a7ff8915b42b2d)
    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/main/java/io/cdap/cdap/datapipeline/oauth/OAuthRefreshToken.java (version 785759b8f3a2ec5d9e79dc7073a7ff8915b42b2d)
    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/main/java/io/cdap/cdap/datapipeline/oauth/OAuthStore.java (version 785759b8f3a2ec5d9e79dc7073a7ff8915b42b2d)
    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/main/java/io/cdap/cdap/datapipeline/oauth/OAuthStoreException.java (version 785759b8f3a2ec5d9e79dc7073a7ff8915b42b2d)
    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/main/java/io/cdap/cdap/datapipeline/oauth/PutOAuthCredentialRequest.java (version 785759b8f3a2ec5d9e79dc7073a7ff8915b42b2d)
    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/main/java/io/cdap/cdap/datapipeline/oauth/PutOAuthProviderRequest.java (version 785759b8f3a2ec5d9e79dc7073a7ff8915b42b2d)
    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/main/java/io/cdap/cdap/datapipeline/oauth/RefreshTokenResponse.java (version 785759b8f3a2ec5d9e79dc7073a7ff8915b42b2d)
    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/main/java/io/cdap/cdap/datapipeline/service/OAuthHandler.java (version 785759b8f3a2ec5d9e79dc7073a7ff8915b42b2d)
    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/main/java/io/cdap/cdap/datapipeline/service/StudioService.java (version 785759b8f3a2ec5d9e79dc7073a7ff8915b42b2d)
    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/test/java/io/cdap/cdap/datapipeline/OAuthServiceTest.java (version 785759b8f3a2ec5d9e79dc7073a7ff8915b42b2d)
    • cdap-app-templates/cdap-etl/cdap-data-pipeline2_2.11/pom.xml (version 785759b8f3a2ec5d9e79dc7073a7ff8915b42b2d)
  • WuA <githubwua@gmail.com>

    WuA <githubwua@gmail.com> 361494fb347b358b69cffe6a5404de79caf1237a m

    Resolve "java.lang.NumberFormatException: multiple points"
    SimpleDateFormat is not threadsafe.  Use new SimpleDateFormat() for every new conversion

    • cdap-runtime-ext-dataproc/src/main/java/io/cdap/cdap/runtime/spi/provisioner/dataproc/DataprocClient.java (version 361494fb347b358b69cffe6a5404de79caf1237a)