Build #1,127

Build: #1127 failed Child of CDAP-DRC-4909

Code commits

caskdata/cdap

  • Terence Yim <885032+chtyim@users.noreply.github.com>

    Terence Yim <885032+chtyim@users.noreply.github.com> 0cb4d416c2675de3a1fc77a240879f47ce7a0f7b

    Merge pull request #12238 from cdapio/feature/merge-6.2
    Merge release 6.2 back to develop

  • Terence Yim

    Terence Yim e3a5ab302eec59b0457fda1bebd455740ff5ad0e

    Merge remote-tracking branch 'origin/release/6.2' into feature/merge-6.2

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 3380f117b54180064f4e6d17441cf8ce07511c1e

    Merge pull request #12234 from cdapio/feature/CDAP-16709-pipeline-performance-cp
    Feature/cdap 16709 pipeline performance cp

  • Terence Yim <885032+chtyim@users.noreply.github.com>

    Terence Yim <885032+chtyim@users.noreply.github.com> 9702b3a75a8bde546cf455a59a7cd1f281f031d3 m

    Merge pull request #12236 from cdapio/feature_release62/CDAP-16841-dataproc-logs
    (CDAP-16841) Includes Dataproc API failure message in log message

  • Terence Yim

    Terence Yim f393b17428e320621a4eb28338276e2509caec03 m

    (CDAP-16841) Includes Dataproc API failure message in log message

    • cdap-app-fabric/src/main/java/io/cdap/cdap/internal/provision/task/ProvisioningTask.java (version f393b17428e320621a4eb28338276e2509caec03)
    • cdap-runtime-ext-dataproc/src/main/java/io/cdap/cdap/runtime/spi/provisioner/dataproc/DataprocClient.java (version f393b17428e320621a4eb28338276e2509caec03)
    • cdap-runtime-ext-dataproc/src/main/java/io/cdap/cdap/runtime/spi/provisioner/dataproc/DataprocRuntimeException.java (version f393b17428e320621a4eb28338276e2509caec03)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 1f47838eac74931c50c7a70d79bd447b0507edd9 m

    CDAP-16709 implemented auto-join for spark streaming
    Implemented auto-join for spark streaming by using the same
    JoinerBridge that is used for MapReduce. This means auto-joins
    in streaming pipelines will have the same characteristics as normal
    joins, meaning they will be executed as shuffle hash joins.

    This is probably ok, as only data within the micro batch is being
    joined, which means it shouldn't be too likely to go OOM assuming
    there is enough executor memory.

    • cdap-app-templates/cdap-etl/cdap-data-streams/src/main/java/io/cdap/cdap/datastreams/SparkStreamingPipelineRunner.java (version 1f47838eac74931c50c7a70d79bd447b0507edd9)
    • cdap-app-templates/cdap-etl/cdap-data-streams/src/test/java/io/cdap/cdap/datastreams/DataStreamsTest.java (version 1f47838eac74931c50c7a70d79bd447b0507edd9)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/common/plugin/JoinerBridge.java (version 1f47838eac74931c50c7a70d79bd447b0507edd9)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/SparkPipelineRunner.java (version 1f47838eac74931c50c7a70d79bd447b0507edd9)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/function/JoinMergeFunction.java (version 1f47838eac74931c50c7a70d79bd447b0507edd9)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/function/JoinOnFunction.java (version 1f47838eac74931c50c7a70d79bd447b0507edd9)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/function/PluginFunctionContext.java (version 1f47838eac74931c50c7a70d79bd447b0507edd9)
    • cdap-app-templates/cdap-etl/hydrator-test/src/main/java/io/cdap/cdap/etl/mock/batch/joiner/MockAutoJoiner.java (version 1f47838eac74931c50c7a70d79bd447b0507edd9)
  • yaojie <yaojie@google.com>

    yaojie <yaojie@google.com> 1e6b61f8b926075bac551bd53df9fa6829ff61c1 m

    CDAP-16855 aggregator spark streaming

    • cdap-app-templates/cdap-etl/cdap-data-streams/src/test/java/io/cdap/cdap/datastreams/DataStreamsTest.java (version 1e6b61f8b926075bac551bd53df9fa6829ff61c1)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/function/AggregatorAggregateFunction.java (version 1e6b61f8b926075bac551bd53df9fa6829ff61c1)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/function/AggregatorGroupByFunction.java (version 1e6b61f8b926075bac551bd53df9fa6829ff61c1)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/streaming/DStreamCollection.java (version 1e6b61f8b926075bac551bd53df9fa6829ff61c1)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/streaming/function/DynamicAggregatorAggregate.java (version 1e6b61f8b926075bac551bd53df9fa6829ff61c1)
    • cdap-app-templates/cdap-etl/hydrator-test/src/main/java/io/cdap/cdap/etl/mock/test/HydratorTestBase.java (version 1e6b61f8b926075bac551bd53df9fa6829ff61c1)
  • yaojie <yaojie@google.com>

    yaojie <yaojie@google.com> dfff293a60748bda00e2c4faadbc61b6fd0090f0 m

    CDAP-16856 mapreduce impl

    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/test/java/io/cdap/cdap/datapipeline/ReducibleAggregatorTest.java (version dfff293a60748bda00e2c4faadbc61b6fd0090f0)
    • cdap-app-templates/cdap-etl/cdap-etl-batch/src/main/java/io/cdap/cdap/etl/batch/mapreduce/MapReduceTransformExecutorFactory.java (version dfff293a60748bda00e2c4faadbc61b6fd0090f0)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/common/plugin/AggregatorBridge.java (version dfff293a60748bda00e2c4faadbc61b6fd0090f0)
  • yaojie <yaojie@google.com>

    yaojie <yaojie@google.com> bbaf10709e3664c31bf57a0678a7afa711f5446d m

    CDAP-16855 aggregator spark implementation

    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/test/java/io/cdap/cdap/datapipeline/ReducibleAggregatorTest.java (version bbaf10709e3664c31bf57a0678a7afa711f5446d)
    • cdap-app-templates/cdap-etl/cdap-etl-api/src/main/java/io/cdap/cdap/etl/api/ReducibleAggregator.java (version bbaf10709e3664c31bf57a0678a7afa711f5446d)
    • cdap-app-templates/cdap-etl/cdap-etl-api/src/main/java/io/cdap/cdap/etl/api/batch/BatchReducibleAggregator.java (version bbaf10709e3664c31bf57a0678a7afa711f5446d)
    • cdap-app-templates/cdap-etl/cdap-etl-batch/src/main/java/io/cdap/cdap/etl/batch/mapreduce/MapReducePreparer.java (version bbaf10709e3664c31bf57a0678a7afa711f5446d)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/common/TypeChecker.java (version bbaf10709e3664c31bf57a0678a7afa711f5446d)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/common/plugin/PipelinePluginContext.java (version bbaf10709e3664c31bf57a0678a7afa711f5446d)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/common/plugin/WrappedReduceAggregator.java (version bbaf10709e3664c31bf57a0678a7afa711f5446d)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/common/submit/PipelinePhasePreparer.java (version bbaf10709e3664c31bf57a0678a7afa711f5446d)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/AbstractSparkPreparer.java (version bbaf10709e3664c31bf57a0678a7afa711f5446d)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/SparkCollection.java (version bbaf10709e3664c31bf57a0678a7afa711f5446d)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/SparkPipelineRunner.java (version bbaf10709e3664c31bf57a0678a7afa711f5446d)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/batch/BaseRDDCollection.java (version bbaf10709e3664c31bf57a0678a7afa711f5446d)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/function/AggregatorFinalizeFunction.java (version bbaf10709e3664c31bf57a0678a7afa711f5446d)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/function/AggregatorInitializeFunction.java (version bbaf10709e3664c31bf57a0678a7afa711f5446d)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/function/AggregatorMergePartitionFunction.java (version bbaf10709e3664c31bf57a0678a7afa711f5446d)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/function/AggregatorMergeValueFunction.java (version bbaf10709e3664c31bf57a0678a7afa711f5446d)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/function/AggregatorReduceGroupByFunction.java (version bbaf10709e3664c31bf57a0678a7afa711f5446d)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/streaming/DStreamCollection.java (version bbaf10709e3664c31bf57a0678a7afa711f5446d)
    • cdap-app-templates/cdap-etl/hydrator-test/src/main/java/io/cdap/cdap/etl/mock/batch/aggregator/DistinctAggregator.java (version bbaf10709e3664c31bf57a0678a7afa711f5446d)
    • cdap-app-templates/cdap-etl/hydrator-test/src/main/java/io/cdap/cdap/etl/mock/batch/aggregator/DistinctReducibleAggregator.java (version bbaf10709e3664c31bf57a0678a7afa711f5446d)
    • cdap-app-templates/cdap-etl/hydrator-test/src/main/java/io/cdap/cdap/etl/mock/batch/aggregator/FieldCountReducibleAggregator.java (version bbaf10709e3664c31bf57a0678a7afa711f5446d)
    • cdap-app-templates/cdap-etl/hydrator-test/src/main/java/io/cdap/cdap/etl/mock/test/HydratorTestBase.java (version bbaf10709e3664c31bf57a0678a7afa711f5446d)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> b3429993bc4c49e1ab722568a1f0f4df7632034a m

    CDAP-16711 implemented null safe equality in mapreduce
    Implemented the nullSafe flag for mapreduce auto join.
    This was done by filtering out records on the map side if they
    come from an optional stage and have a null key or a field in
    the key that is null.

    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/test/java/io/cdap/cdap/datapipeline/AutoJoinerTest.java (version b3429993bc4c49e1ab722568a1f0f4df7632034a)
    • cdap-app-templates/cdap-etl/cdap-etl-api/src/main/java/io/cdap/cdap/etl/api/Joiner.java (version b3429993bc4c49e1ab722568a1f0f4df7632034a)
    • cdap-app-templates/cdap-etl/cdap-etl-batch/src/main/java/io/cdap/cdap/etl/batch/mapreduce/MapReduceTransformExecutorFactory.java (version b3429993bc4c49e1ab722568a1f0f4df7632034a)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/common/plugin/JoinerBridge.java (version b3429993bc4c49e1ab722568a1f0f4df7632034a)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/function/JoinOnFunction.java (version b3429993bc4c49e1ab722568a1f0f4df7632034a)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 61b2f016d9aa7937b400fdad306d0a9cc998786d m

    CDAP-16709 implement manual broadcasts
    Honor the broadcast flag set in the JoinDefinition when joining
    multiple DataFrames. Added a small tweak to the join logic to
    join all non-broadcasted datasets first in order to ensure that
    both sides of the join are not broadcast, and to reduce the amount
    of data that is being shuffled in non-broadcast intermediate joins.

    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/test/java/io/cdap/cdap/datapipeline/AutoJoinerTest.java (version 61b2f016d9aa7937b400fdad306d0a9cc998786d)
    • cdap-app-templates/cdap-etl/cdap-etl-api/src/main/java/io/cdap/cdap/etl/api/join/JoinDefinition.java (version 61b2f016d9aa7937b400fdad306d0a9cc998786d)
    • cdap-app-templates/cdap-etl/cdap-etl-api/src/main/java/io/cdap/cdap/etl/api/join/JoinStage.java (version 61b2f016d9aa7937b400fdad306d0a9cc998786d)
    • cdap-app-templates/cdap-etl/cdap-etl-api/src/test/java/io/cdap/cdap/etl/api/join/JoinDefinitionTest.java (version 61b2f016d9aa7937b400fdad306d0a9cc998786d)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/SparkPipelineRunner.java (version 61b2f016d9aa7937b400fdad306d0a9cc998786d)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/batch/ETLSpark.java (version 61b2f016d9aa7937b400fdad306d0a9cc998786d)
    • cdap-app-templates/cdap-etl/hydrator-spark-core/src/main/java/io/cdap/cdap/etl/spark/batch/RDDCollection.java (version 61b2f016d9aa7937b400fdad306d0a9cc998786d)
    • cdap-app-templates/cdap-etl/hydrator-spark-core2_2.11/src/main/java/io/cdap/cdap/etl/spark/batch/RDDCollection.java (version 61b2f016d9aa7937b400fdad306d0a9cc998786d)
    • cdap-app-templates/cdap-etl/hydrator-test/src/main/java/io/cdap/cdap/etl/mock/batch/joiner/MockAutoJoiner.java (version 61b2f016d9aa7937b400fdad306d0a9cc998786d)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 8965fb6a84a7c7332b3de9cb316d541cea0e0367 m

    CDAP-16711 implement null safe keys as an option
    Changed the 'dropNullKeys' property to 'nullSafe' since it more
    accurately describes what is happening. Null keys are not dropped
    in outer joins, they just do not count as being equal to a null
    key on the other side.

    Implemented by using Spark's null safe equality when configured
    to do so and normal equality otherwise.

    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/test/java/io/cdap/cdap/datapipeline/AutoJoinerTest.java (version 8965fb6a84a7c7332b3de9cb316d541cea0e0367)
    • cdap-app-templates/cdap-etl/cdap-etl-api/src/main/java/io/cdap/cdap/etl/api/join/JoinCondition.java (version 8965fb6a84a7c7332b3de9cb316d541cea0e0367)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/SparkPipelineRunner.java (version 8965fb6a84a7c7332b3de9cb316d541cea0e0367)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/join/JoinRequest.java (version 8965fb6a84a7c7332b3de9cb316d541cea0e0367)
    • cdap-app-templates/cdap-etl/hydrator-spark-core/src/main/java/io/cdap/cdap/etl/spark/batch/RDDCollection.java (version 8965fb6a84a7c7332b3de9cb316d541cea0e0367)
    • cdap-app-templates/cdap-etl/hydrator-spark-core2_2.11/src/main/java/io/cdap/cdap/etl/spark/batch/RDDCollection.java (version 8965fb6a84a7c7332b3de9cb316d541cea0e0367)
    • cdap-app-templates/cdap-etl/hydrator-test/src/main/java/io/cdap/cdap/etl/mock/batch/joiner/MockAutoJoiner.java (version 8965fb6a84a7c7332b3de9cb316d541cea0e0367)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> d871bb112e52953bfd6de296eaa81e5feb85467c m

    CDAP-16709 implement auto join for mapreduce
    Implemented AutoJoiner for mapreduce by creating a bridge that
    implements the old Joiner API using the JoinDefinition from the
    new AutoJoiner API.

    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/test/java/io/cdap/cdap/datapipeline/AutoJoinerTest.java (version d871bb112e52953bfd6de296eaa81e5feb85467c)
    • cdap-app-templates/cdap-etl/cdap-etl-batch/src/main/java/io/cdap/cdap/etl/batch/mapreduce/MapReducePreparer.java (version d871bb112e52953bfd6de296eaa81e5feb85467c)
    • cdap-app-templates/cdap-etl/cdap-etl-batch/src/main/java/io/cdap/cdap/etl/batch/mapreduce/MapReduceTransformExecutorFactory.java (version d871bb112e52953bfd6de296eaa81e5feb85467c)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/common/DefaultAutoJoinerContext.java (version d871bb112e52953bfd6de296eaa81e5feb85467c)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/common/plugin/JoinerBridge.java (version d871bb112e52953bfd6de296eaa81e5feb85467c)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/common/submit/PipelinePhasePreparer.java (version d871bb112e52953bfd6de296eaa81e5feb85467c)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/spec/PipelineSpecGenerator.java (version d871bb112e52953bfd6de296eaa81e5feb85467c)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/SparkPipelineRunner.java (version d871bb112e52953bfd6de296eaa81e5feb85467c)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> bbd43a73cb347b42ac6e0a0930ab8c11f1d53ce0 m

    CDAP-16709 batch spark auto-join implementation
    Implemented auto join for batch spark pipelines.

    Added a join method to SparkCollection that takes in the list of
    other SparkCollections that it should be joined to.
    RDDCollection converts RDDs into Datasets and uses the Dataset
    join method to implement the join. This allows Spark to broadcast
    small datasets automatically, and to use sort merge join instead
    of shuffle hash join, which has better memory characteristics.

    As part of this, added a separate RDDCollection implementation for
    Spark1 and Spark2, since the Spark API for joins is not compatible.

    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/test/java/io/cdap/cdap/datapipeline/AutoJoinerTest.java (version bbd43a73cb347b42ac6e0a0930ab8c11f1d53ce0)
    • cdap-app-templates/cdap-etl/cdap-etl-api/src/main/java/io/cdap/cdap/etl/api/join/JoinCondition.java (version bbd43a73cb347b42ac6e0a0930ab8c11f1d53ce0)
    • cdap-app-templates/cdap-etl/cdap-etl-batch/src/main/java/io/cdap/cdap/etl/batch/mapreduce/MapReducePreparer.java (version bbd43a73cb347b42ac6e0a0930ab8c11f1d53ce0)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/common/DefaultAutoJoinerContext.java (version bbd43a73cb347b42ac6e0a0930ab8c11f1d53ce0)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/common/submit/PipelinePhasePreparer.java (version bbd43a73cb347b42ac6e0a0930ab8c11f1d53ce0)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/spec/PipelineSpecGenerator.java (version bbd43a73cb347b42ac6e0a0930ab8c11f1d53ce0)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/AbstractSparkPreparer.java (version bbd43a73cb347b42ac6e0a0930ab8c11f1d53ce0)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/SparkCollection.java (version bbd43a73cb347b42ac6e0a0930ab8c11f1d53ce0)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/SparkPipelineRunner.java (version bbd43a73cb347b42ac6e0a0930ab8c11f1d53ce0)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/batch/BaseRDDCollection.java (version bbd43a73cb347b42ac6e0a0930ab8c11f1d53ce0)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/batch/BatchSparkPipelineDriver.java (version bbd43a73cb347b42ac6e0a0930ab8c11f1d53ce0)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/batch/PairRDDCollection.java (version bbd43a73cb347b42ac6e0a0930ab8c11f1d53ce0)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/batch/RDDCollection.java (version bbd43a73cb347b42ac6e0a0930ab8c11f1d53ce0)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/join/JoinCollection.java (version bbd43a73cb347b42ac6e0a0930ab8c11f1d53ce0)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/join/JoinRequest.java (version bbd43a73cb347b42ac6e0a0930ab8c11f1d53ce0)
    • cdap-app-templates/cdap-etl/hydrator-spark-core-base/src/main/java/io/cdap/cdap/etl/spark/streaming/DStreamCollection.java (version bbd43a73cb347b42ac6e0a0930ab8c11f1d53ce0)
    • cdap-app-templates/cdap-etl/hydrator-spark-core/pom.xml (version bbd43a73cb347b42ac6e0a0930ab8c11f1d53ce0)
    • cdap-app-templates/cdap-etl/hydrator-spark-core/src/main/java/io/cdap/cdap/etl/spark/batch/RDDCollection.java (version bbd43a73cb347b42ac6e0a0930ab8c11f1d53ce0)
    • cdap-app-templates/cdap-etl/hydrator-spark-core2_2.11/pom.xml (version bbd43a73cb347b42ac6e0a0930ab8c11f1d53ce0)
    • cdap-app-templates/cdap-etl/hydrator-spark-core2_2.11/src/main/java/io/cdap/cdap/etl/spark/batch/RDDCollection.java (version bbd43a73cb347b42ac6e0a0930ab8c11f1d53ce0)
    • cdap-app-templates/cdap-etl/hydrator-test/src/main/java/io/cdap/cdap/etl/mock/batch/joiner/MockAutoJoiner.java (version bbd43a73cb347b42ac6e0a0930ab8c11f1d53ce0)
    • cdap-app-templates/cdap-etl/hydrator-test/src/main/java/io/cdap/cdap/etl/mock/test/HydratorTestBase.java (version bbd43a73cb347b42ac6e0a0930ab8c11f1d53ce0)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 84621709dc5a5f73740aefd544788880a7484c01 m

    CDAP-16708 rename Field to JoinField

    • cdap-app-templates/cdap-etl/cdap-etl-api/src/main/java/io/cdap/cdap/etl/api/join/Field.java (version 84621709dc5a5f73740aefd544788880a7484c01)
    • cdap-app-templates/cdap-etl/cdap-etl-api/src/main/java/io/cdap/cdap/etl/api/join/JoinDefinition.java (version 84621709dc5a5f73740aefd544788880a7484c01)
    • cdap-app-templates/cdap-etl/cdap-etl-api/src/main/java/io/cdap/cdap/etl/api/join/JoinField.java (version 84621709dc5a5f73740aefd544788880a7484c01)
    • cdap-app-templates/cdap-etl/cdap-etl-api/src/test/java/io/cdap/cdap/etl/api/join/JoinDefinitionTest.java (version 84621709dc5a5f73740aefd544788880a7484c01)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/test/java/io/cdap/cdap/etl/spec/PipelineSpecGeneratorTest.java (version 84621709dc5a5f73740aefd544788880a7484c01)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> ce55f535672956ee1597e6732a596438f5763888 m

    CDAP-16708 add API for autojoiner
    Add a new set of classes for AutoJoiner, which can be used instead
    of the current Joiner interface. This new API leaves all of the
    implementation details up to the application, which will allow the
    app to perform the join in better ways. For example, in the Spark
    program, it will allow using broadcast joins.

    Plugin developers are responsible for returning a JoinDefinition
    based on information about incoming stages.

    This change includes the JoinDefinition as well as all the classes
    required to create a definition. It also includes validation logic
    to make sure the plugin cannot create a definition that tries to
    join on a field that doesn't exist, or tries to join on fields
    that have mismatched types, or any other type of error.

    • cdap-app-templates/cdap-etl/cdap-etl-api/src/main/java/io/cdap/cdap/etl/api/batch/BatchAutoJoiner.java (version ce55f535672956ee1597e6732a596438f5763888)
    • cdap-app-templates/cdap-etl/cdap-etl-api/src/main/java/io/cdap/cdap/etl/api/batch/BatchJoiner.java (version ce55f535672956ee1597e6732a596438f5763888)
    • cdap-app-templates/cdap-etl/cdap-etl-api/src/main/java/io/cdap/cdap/etl/api/join/AutoJoiner.java (version ce55f535672956ee1597e6732a596438f5763888)
    • cdap-app-templates/cdap-etl/cdap-etl-api/src/main/java/io/cdap/cdap/etl/api/join/AutoJoinerContext.java (version ce55f535672956ee1597e6732a596438f5763888)
    • cdap-app-templates/cdap-etl/cdap-etl-api/src/main/java/io/cdap/cdap/etl/api/join/Field.java (version ce55f535672956ee1597e6732a596438f5763888)
    • cdap-app-templates/cdap-etl/cdap-etl-api/src/main/java/io/cdap/cdap/etl/api/join/InvalidJoinConditionException.java (version ce55f535672956ee1597e6732a596438f5763888)
    • cdap-app-templates/cdap-etl/cdap-etl-api/src/main/java/io/cdap/cdap/etl/api/join/InvalidJoinException.java (version ce55f535672956ee1597e6732a596438f5763888)
    • cdap-app-templates/cdap-etl/cdap-etl-api/src/main/java/io/cdap/cdap/etl/api/join/JoinCondition.java (version ce55f535672956ee1597e6732a596438f5763888)
    • cdap-app-templates/cdap-etl/cdap-etl-api/src/main/java/io/cdap/cdap/etl/api/join/JoinDefinition.java (version ce55f535672956ee1597e6732a596438f5763888)
    • cdap-app-templates/cdap-etl/cdap-etl-api/src/main/java/io/cdap/cdap/etl/api/join/JoinKey.java (version ce55f535672956ee1597e6732a596438f5763888)
    • cdap-app-templates/cdap-etl/cdap-etl-api/src/main/java/io/cdap/cdap/etl/api/join/JoinStage.java (version ce55f535672956ee1597e6732a596438f5763888)
    • cdap-app-templates/cdap-etl/cdap-etl-api/src/main/java/io/cdap/cdap/etl/api/validation/ValidationException.java (version ce55f535672956ee1597e6732a596438f5763888)
    • cdap-app-templates/cdap-etl/cdap-etl-api/src/test/java/io/cdap/cdap/etl/api/join/JoinDefinitionTest.java (version ce55f535672956ee1597e6732a596438f5763888)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/common/DefaultAutoJoinerContext.java (version ce55f535672956ee1597e6732a596438f5763888)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/common/DefaultStageConfigurer.java (version ce55f535672956ee1597e6732a596438f5763888)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/io/cdap/cdap/etl/spec/PipelineSpecGenerator.java (version ce55f535672956ee1597e6732a596438f5763888)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/test/java/io/cdap/cdap/etl/spec/PipelineSpecGeneratorTest.java (version ce55f535672956ee1597e6732a596438f5763888)
  • Edwin Elia <edwinelia@google.com>

    Edwin Elia <edwinelia@google.com> 4549e37ab41e87f1c00bdaed978675141f78d2d7 m

    Merge pull request #12223 from cdapio/docs-md5-update
    fix md5 for docs

  • Edwin Elia <edwinelia@google.com>

    Edwin Elia <edwinelia@google.com> ab061d41e959046f4c5f7d6d7a13f4290aa80686 m

    fix md5 for docs

    • cdap-docs/admin-manual/build.sh (version ab061d41e959046f4c5f7d6d7a13f4290aa80686)
  • Edwin Elia <edwinelia@google.com>

    Edwin Elia <edwinelia@google.com> af8e9e6295c5f8d54b9a94b94a5b963636bf5cfc m

    Merge pull request #12219 from cdapio/fix-docs-build
    update MD5 hash

  • Edwin Elia <edwinelia@google.com>

    Edwin Elia <edwinelia@google.com> d9a2bf0ea66e58b834b384be11e6f4a3dbdbbe7e m

    update MD5 hash

    • cdap-docs/admin-manual/build.sh (version d9a2bf0ea66e58b834b384be11e6f4a3dbdbbe7e)
    • cdap-docs/developer-manual/source/pipelines/developing-plugins/presentation-plugins.rst (version d9a2bf0ea66e58b834b384be11e6f4a3dbdbbe7e)
    • cdap-docs/integrations/build.sh (version d9a2bf0ea66e58b834b384be11e6f4a3dbdbbe7e)
  • Terence Yim <885032+chtyim@users.noreply.github.com>

    Terence Yim <885032+chtyim@users.noreply.github.com> 10b1831a5044ea9f9fd23814a853443217b11261 m

    Merge pull request #12187 from cdapio/feature_release/CDAP-16673-tms-compress
    (CDAP-16673) TMS compression

  • Edwin Elia <edwinelia@google.com>

    Edwin Elia <edwinelia@google.com> 068e06a9d394c2beea47b3466b84934cc543b4c5 m

    Merge pull request #12212 from cdapio/release-license-dependencies
    Release 6.2.0 License Dependencies and Release Notes

  • Edwin Elia <edwinelia@google.com>

    Edwin Elia <edwinelia@google.com> 8477c34a3c9bad5d5cdf09d30916e92130e34020 m

    Added 6.2.0 release notes

    • cdap-docs/reference-manual/source/release-notes.rst (version 8477c34a3c9bad5d5cdf09d30916e92130e34020)
  • Edwin Elia <edwinelia@google.com>

    Edwin Elia <edwinelia@google.com> 58c0eaf051f5fe64b4d12335d900997bc077e2cf m

    Update license dependencies for 6.2.0 release

    • cdap-docs/reference-manual/licenses-pdf/cdap-enterprise-dependencies.pdf (version 58c0eaf051f5fe64b4d12335d900997bc077e2cf)
    • cdap-docs/reference-manual/licenses-pdf/cdap-level-1-dependencies.pdf (version 58c0eaf051f5fe64b4d12335d900997bc077e2cf)
    • cdap-docs/reference-manual/licenses-pdf/cdap-sandbox-dependencies.pdf (version 58c0eaf051f5fe64b4d12335d900997bc077e2cf)
    • cdap-docs/reference-manual/licenses-pdf/cdap-ui-dependencies.pdf (version 58c0eaf051f5fe64b4d12335d900997bc077e2cf)
    • cdap-docs/reference-manual/source/licenses/cdap-enterprise-dependencies.csv (version 58c0eaf051f5fe64b4d12335d900997bc077e2cf)
    • cdap-docs/reference-manual/source/licenses/cdap-enterprise-dependencies.rst (version 58c0eaf051f5fe64b4d12335d900997bc077e2cf)
    • cdap-docs/reference-manual/source/licenses/cdap-level-1-dependencies.csv (version 58c0eaf051f5fe64b4d12335d900997bc077e2cf)
    • cdap-docs/reference-manual/source/licenses/cdap-level-1-dependencies.rst (version 58c0eaf051f5fe64b4d12335d900997bc077e2cf)
    • cdap-docs/reference-manual/source/licenses/cdap-sandbox-dependencies.csv (version 58c0eaf051f5fe64b4d12335d900997bc077e2cf)
    • cdap-docs/reference-manual/source/licenses/cdap-sandbox-dependencies.rst (version 58c0eaf051f5fe64b4d12335d900997bc077e2cf)
    • cdap-docs/reference-manual/source/licenses/cdap-ui-dependencies.rst (version 58c0eaf051f5fe64b4d12335d900997bc077e2cf)
    • cdap-docs/tools/licenses/cdap-dependencies-master.csv (version 58c0eaf051f5fe64b4d12335d900997bc077e2cf)
    • cdap-docs/tools/licenses/cdap-enterprise-dependencies.rst (version 58c0eaf051f5fe64b4d12335d900997bc077e2cf)
    • cdap-docs/tools/licenses/cdap-level-1-dependencies.rst (version 58c0eaf051f5fe64b4d12335d900997bc077e2cf)
    • cdap-docs/tools/licenses/cdap-sandbox-dependencies.rst (version 58c0eaf051f5fe64b4d12335d900997bc077e2cf)
    • cdap-docs/tools/licenses/cdap-ui-dependencies.rst (version 58c0eaf051f5fe64b4d12335d900997bc077e2cf)
  • Edwin Elia <edwinelia@google.com>

    Edwin Elia <edwinelia@google.com> 1397e6b536754306a09c3beadbf4395d7eb31e78 m

    Merge pull request #12189 from cdapio/release-remove-snapshot
    remove -SNAPSHOT version from release/6.2

  • Edwin Elia <edwinelia@google.com>

    Edwin Elia <edwinelia@google.com> 557a59ab8f0e0c53fb35def59e8097ece9d6767d m

    remove -SNAPSHOT version from release/6.2

    • cdap-api-common/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-api-spark/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-api-spark2_2.11/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-api/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-app-fabric-tests/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-app-fabric/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-app-templates/cdap-etl/cdap-data-pipeline/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-app-templates/cdap-etl/cdap-data-pipeline2_2.11/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-app-templates/cdap-etl/cdap-data-streams/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-app-templates/cdap-etl/cdap-data-streams2_2.11/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-app-templates/cdap-etl/cdap-etl-api-spark/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-app-templates/cdap-etl/cdap-etl-api/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-app-templates/cdap-etl/cdap-etl-archetypes/cdap-data-pipeline-plugins-archetype/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-app-templates/cdap-etl/cdap-etl-archetypes/cdap-data-pipeline-plugins-archetype/src/main/resources/archetype-resources/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-app-templates/cdap-etl/cdap-etl-archetypes/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-app-templates/cdap-etl/cdap-etl-batch/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-app-templates/cdap-etl/cdap-etl-core/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-app-templates/cdap-etl/cdap-etl-proto/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-app-templates/cdap-etl/cdap-etl-tools/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-app-templates/cdap-etl/hydrator-spark-core/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-app-templates/cdap-etl/hydrator-spark-core2_2.11/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-app-templates/cdap-etl/hydrator-test/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-app-templates/cdap-etl/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-app-templates/cdap-program-report/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-app-templates/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-cli-tests/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-cli/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-client-tests/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-client/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-common-unit-test/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-common/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-data-fabric-tests/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-data-fabric/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-distributions/bin/build_docs_bucket.sh (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-distributions/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-docs-gen/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-elastic/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-explore-client/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-explore-jdbc/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-explore/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-formats/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-gateway/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-hbase-compat-1.0-cdh/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-hbase-compat-1.0-cdh5.5.0/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-hbase-compat-1.0/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-hbase-compat-1.1/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-hbase-compat-1.2-cdh5.7.0/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-hbase-compat-base/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-hbase-spi/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-integration-test/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-kafka/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-kms/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-kubernetes/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-master-spi/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-master/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-metadata-spi/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-operational-stats-core/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-proto/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-runtime-ext-dataproc/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-runtime-ext-emr/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-runtime-ext-remote-hadoop/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-runtime-spi/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-securestore-ext-cloudkms/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-securestore-spi/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-security-spi/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-security/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-spark-core/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-spark-core2_2.11/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-spark-python/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-standalone/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-storage-spi/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-system-app-api/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-system-app-unit-test/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-test/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-tms-tests/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-tms/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-ui/app/cdap/components/ConfigurationGroup/KitchenSync.tsx (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-ui/cypress/fixtures/fll_airport_pipeline2.json (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-ui/cypress/fixtures/null_splitter_pipeline-cdap-data-pipeline.json (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-ui/cypress/fixtures/pipeline1.json (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-ui/cypress/fixtures/pipeline_old.json (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-ui/cypress/fixtures/union_condition_splitter_pipeline_v1-cdap-data-pipeline.json (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-ui/package.json (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-ui/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-unit-test-spark2_2.11/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-unit-test/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-watchdog-api/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • cdap-watchdog/pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
    • pom.xml (version 557a59ab8f0e0c53fb35def59e8097ece9d6767d)
  • Terence Yim

    Terence Yim a3f6abc29303f6205ca520410e2feb9b5fdde0de m

    (CDAP-16673) Added TMS payload compression supported
    - Compression is on by default

    • cdap-common/src/main/java/io/cdap/cdap/common/conf/Constants.java (version a3f6abc29303f6205ca520410e2feb9b5fdde0de)
    • cdap-common/src/main/resources/cdap-default.xml (version a3f6abc29303f6205ca520410e2feb9b5fdde0de)
    • cdap-master/src/test/java/io/cdap/cdap/master/environment/k8s/MessagingServiceMainTest.java (version a3f6abc29303f6205ca520410e2feb9b5fdde0de)
    • cdap-master/src/test/java/io/cdap/cdap/master/environment/k8s/UserProgramServiceMainTest.java (version a3f6abc29303f6205ca520410e2feb9b5fdde0de)
    • cdap-tms/src/main/java/io/cdap/cdap/messaging/client/ClientMessagingService.java (version a3f6abc29303f6205ca520410e2feb9b5fdde0de)
    • cdap-tms/src/main/java/io/cdap/cdap/messaging/server/MessagingHttpService.java (version a3f6abc29303f6205ca520410e2feb9b5fdde0de)
    • cdap-tms/src/test/java/io/cdap/cdap/messaging/server/MessagingHttpServiceTest.java (version a3f6abc29303f6205ca520410e2feb9b5fdde0de)
  • Terence Yim

    Terence Yim 87561a9d2f16cae6a1e5b77721d388dde7d7f830 m

    (CDAP-16673) Handle arbitrary large TMS publish request
    - Use the SpillableBodyConsumer to spill large requests to temp file

    • cdap-tms/src/main/java/io/cdap/cdap/messaging/server/StoreHandler.java (version 87561a9d2f16cae6a1e5b77721d388dde7d7f830)
    • cdap-tms/src/test/java/io/cdap/cdap/messaging/server/MessagingHttpServiceTest.java (version 87561a9d2f16cae6a1e5b77721d388dde7d7f830)
  • Terence Yim

    Terence Yim e8c20f2802df2cbe15996897267d8544873bac59 m

    (CDAP-16673) Enable message fetching compression

    • cdap-tms/src/main/java/io/cdap/cdap/messaging/client/ClientMessagingService.java (version e8c20f2802df2cbe15996897267d8544873bac59)
  • Terence Yim

    Terence Yim 84a535f0b141a46c753231dbe71776aa95570ad1 m

    (CDAP-16673) Added a SpillableBodyConsumer
    This allows handling large HTTP requests with a memory bound

    • cdap-common/src/main/java/io/cdap/cdap/common/http/CombineInputStream.java (version 84a535f0b141a46c753231dbe71776aa95570ad1)
    • cdap-common/src/main/java/io/cdap/cdap/common/http/SpillableBodyConsumer.java (version 84a535f0b141a46c753231dbe71776aa95570ad1)
    • cdap-common/src/main/java/io/cdap/cdap/common/io/FileSeekableInputStream.java (version 84a535f0b141a46c753231dbe71776aa95570ad1)
    • cdap-common/src/test/java/io/cdap/cdap/common/http/CombineInputStreamTest.java (version 84a535f0b141a46c753231dbe71776aa95570ad1)
    • cdap-common/src/test/java/io/cdap/cdap/common/http/SpillableBodyConsumerTest.java (version 84a535f0b141a46c753231dbe71776aa95570ad1)