Build #124

Build: #124 was successful Changes by albertshau <albert@cask.co> and Albert Shau <albert@cask.co>

Code commits

Hydrator Plugins

  • albertshau <albert@cask.co>

    albertshau <albert@cask.co> 7bf99c72a2b7f54a8f5a2013700498683ff7598d

    Merge pull request #676 from caskdata/merge-release-1-7
    Merge release 1 7

  • Albert Shau <albert@cask.co>

    Albert Shau <albert@cask.co> eaf5f9b6f32e0575fef4c3fe97e4917cacd7e976

    update gitmodules

    • .gitmodules (version eaf5f9b6f32e0575fef4c3fe97e4917cacd7e976)
  • Albert Shau <albert@cask.co>

    Albert Shau <albert@cask.co> 683e7b7208ae3f0b4d5e26fe7a31a512cbeb4d7b

    Merge remote-tracking branch 'origin/release/1.7' into merge-release-1-7

  • vinishavyasa <CuriousVini@users.noreply.github.com>

    vinishavyasa <CuriousVini@users.noreply.github.com> 0850edb7763988585ae18fb26a45c2e9e4ef56f4 m

    Merge pull request #675 from caskdata/fix/wrangler-ref
    Fix wrangler ref

  • Vinisha Shah

    Vinisha Shah 4843d0f8ae0203692f75e637e15854096f9bc2a8 m

    Fix wrangler ref

    • realtime-stream-source (version 4843d0f8ae0203692f75e637e15854096f9bc2a8)
    • wrangler-transform (version 4843d0f8ae0203692f75e637e15854096f9bc2a8)
  • Sreevatsan Raman

    Sreevatsan Raman b940f16d927b2221398c2a4c1a669130a9fdebe1 m

    Merge pull request #673 from caskdata/bugfix/update-wrangler-submodule-branch
    Update wrangler transform submodule branch

  • Bhooshan Mogal

    Bhooshan Mogal af3379f79d4ba9bdddc99f13faf5aab69a362ca7 m

    Update wrangler transform submodule branch

    • .gitmodules (version af3379f79d4ba9bdddc99f13faf5aab69a362ca7)
  • Sreevatsan Raman

    Sreevatsan Raman b88850f9da4a3b731aa2530ad58b5ec77919db9c m

    Merge pull request #669 from caskdata/fix/remove-snapshot
    Fix/remove snapshot

  • Sreevatsan Raman

    Sreevatsan Raman d02799545fccdf254166fa8f1f8974aa4c307424 m

    Merge pull request #672 from caskdata/feature_release/fix-spark-program
    Use the SparkInterpreter abstraction everywhere.

  • Terence Yim

    Terence Yim 84614dd4ca9c4d789fb0bed87e8d4f9e66497666 m

    Use the SparkInterpreter abstraction everywhere.

    • spark-plugins/src/main/java/co/cask/hydrator/plugin/spark/dynamic/ScalaSparkCompute.java (version 84614dd4ca9c4d789fb0bed87e8d4f9e66497666)
    • spark-plugins/src/main/java/co/cask/hydrator/plugin/spark/dynamic/ScalaSparkProgram.java (version 84614dd4ca9c4d789fb0bed87e8d4f9e66497666)
    • spark-plugins/src/main/java/co/cask/hydrator/plugin/spark/dynamic/SparkCompilers.java (version 84614dd4ca9c4d789fb0bed87e8d4f9e66497666)
  • Terence Yim

    Terence Yim 7c70822577a2af3a82e9d93117b7ebd517fe9421 m

    Merge pull request #670 from caskdata/feature_release/spark-program-plugin
    Added a SparkProgram plugin that runs any spark program

  • Terence Yim

    Terence Yim a1f3b43c5b798f8a4c72b3ad91de7c941a45c540 m

    Added a SparkProgram plugin that runs any spark program

    • spark-plugins/docs/ScalaSparkProgram-sparkprogram.md (version a1f3b43c5b798f8a4c72b3ad91de7c941a45c540)
    • spark-plugins/src/main/java/co/cask/hydrator/plugin/spark/dynamic/ScalaSparkCompute.java (version a1f3b43c5b798f8a4c72b3ad91de7c941a45c540)
    • spark-plugins/src/main/java/co/cask/hydrator/plugin/spark/dynamic/ScalaSparkProgram.java (version a1f3b43c5b798f8a4c72b3ad91de7c941a45c540)
    • spark-plugins/src/main/java/co/cask/hydrator/plugin/spark/dynamic/SparkCompilers.java (version a1f3b43c5b798f8a4c72b3ad91de7c941a45c540)
    • spark-plugins/widgets/ScalaSparkProgram-sparkprogram.json (version a1f3b43c5b798f8a4c72b3ad91de7c941a45c540)
  • Sreevatsan Raman

    Sreevatsan Raman 3422a1e6590535ea7205c7470e539ecdaa2e64bb m

    Merge pull request #671 from caskdata/bugfix_release/CDAP-11843-fix-get-schema-for-filesource
    CDAP-11843 make schema request a concrete class

  • Albert Shau <albert@cask.co>

    Albert Shau <albert@cask.co> 716a05dc2b96015540e9c7ce375659153354ce94 m

    CDAP-11843 make schema request a concrete class
    This fixes a bug where the endpoint throws an error, because CDAP
    is unable to instantiate the request, since the request used to
    be a generic that extends an abstract class.

    • core-plugins/src/main/java/co/cask/hydrator/plugin/batch/source/AbstractFileBatchSource.java (version 716a05dc2b96015540e9c7ce375659153354ce94)
  • Vinisha Shah

    Vinisha Shah 2d4d46726cb1a924a518e203067b5896a556a62a m

    Removing snapshot from cdap versions

    • database-plugins/pom.xml (version 2d4d46726cb1a924a518e203067b5896a556a62a)
    • hbase-plugins/pom.xml (version 2d4d46726cb1a924a518e203067b5896a556a62a)
    • hdfs-plugins/pom.xml (version 2d4d46726cb1a924a518e203067b5896a556a62a)
    • hive-plugins/pom.xml (version 2d4d46726cb1a924a518e203067b5896a556a62a)
    • kafka-plugins/pom.xml (version 2d4d46726cb1a924a518e203067b5896a556a62a)
    • pom.xml (version 2d4d46726cb1a924a518e203067b5896a556a62a)
    • spark-plugins/pom.xml (version 2d4d46726cb1a924a518e203067b5896a556a62a)
  • Vinisha Shah

    Vinisha Shah fcb7e1ffff9a379e6cdcea40c989773e8081c19b m

    Remove snapshot using mvn command

    • cassandra-plugins/pom.xml (version fcb7e1ffff9a379e6cdcea40c989773e8081c19b)
    • core-plugins/pom.xml (version fcb7e1ffff9a379e6cdcea40c989773e8081c19b)
    • database-plugins/pom.xml (version fcb7e1ffff9a379e6cdcea40c989773e8081c19b)
    • elasticsearch-plugins/pom.xml (version fcb7e1ffff9a379e6cdcea40c989773e8081c19b)
    • hbase-plugins/pom.xml (version fcb7e1ffff9a379e6cdcea40c989773e8081c19b)
    • hdfs-plugins/pom.xml (version fcb7e1ffff9a379e6cdcea40c989773e8081c19b)
    • hive-plugins/pom.xml (version fcb7e1ffff9a379e6cdcea40c989773e8081c19b)
    • http-plugins/pom.xml (version fcb7e1ffff9a379e6cdcea40c989773e8081c19b)
    • hydrator-common/pom.xml (version fcb7e1ffff9a379e6cdcea40c989773e8081c19b)
    • kafka-plugins/pom.xml (version fcb7e1ffff9a379e6cdcea40c989773e8081c19b)
    • mongodb-plugins/pom.xml (version fcb7e1ffff9a379e6cdcea40c989773e8081c19b)
    • pom.xml (version fcb7e1ffff9a379e6cdcea40c989773e8081c19b)
    • solrsearch-plugins/pom.xml (version fcb7e1ffff9a379e6cdcea40c989773e8081c19b)
    • spark-plugins/pom.xml (version fcb7e1ffff9a379e6cdcea40c989773e8081c19b)
    • transform-plugins/pom.xml (version fcb7e1ffff9a379e6cdcea40c989773e8081c19b)
  • Terence Yim

    Terence Yim 8210439babaca931fef97dc0c81fa126f2091487 m

    Merge pull request #668 from caskdata/feature_release/update-scala-compiler-hack
    Update method signature due to recent change in CDAP

  • Terence Yim

    Terence Yim 54e5e4d04b340a07d6bcb323e98810d196f72fed m

    Update method signature due to recent change in CDAP

    • spark-plugins/src/main/java/co/cask/hydrator/plugin/spark/dynamic/ScalaSparkCompute.java (version 54e5e4d04b340a07d6bcb323e98810d196f72fed)
  • Terence Yim

    Terence Yim 9daa3b7b8f44d883883b36a63aefa45357c2a539 m

    Merge pull request #667 from caskdata/feature_release/CDAP-11410-scala-spark-compute
    (CDAP-11410) Added a SparkCompute plugin that do arbitrary Scala code execution

  • Terence Yim

    Terence Yim dc1eaad631bc1803295e78ebb7e463eba77278a5 m

    Documents imports available.

    • spark-plugins/docs/ScalaSparkCompute-sparkcompute.md (version dc1eaad631bc1803295e78ebb7e463eba77278a5)
    • spark-plugins/src/main/java/co/cask/hydrator/plugin/spark/dynamic/ScalaSparkCompute.java (version dc1eaad631bc1803295e78ebb7e463eba77278a5)
  • Rohit Sinha

    Rohit Sinha 2b5b8228466fd5766d41f9d692402154b301de04 m

    Merge pull request #660 from caskdata/feature/database-plugin-func-move
    Move transformValue function from DBRecord to DBUtils as public statiā€¦

  • Terence Yim

    Terence Yim 8817146381c14e685d2cdad004647a729006d3e6 m

    (CDAP-11410) Added a SparkCompute plugin that do arbitrary Scala code execution

    • pom.xml (version 8817146381c14e685d2cdad004647a729006d3e6)
    • spark-plugins/docs/ScalaSparkCompute-sparkcompute.md (version 8817146381c14e685d2cdad004647a729006d3e6)
    • spark-plugins/pom.xml (version 8817146381c14e685d2cdad004647a729006d3e6)
    • spark-plugins/src/main/java/co/cask/hydrator/plugin/spark/dynamic/ScalaSparkCompute.java (version 8817146381c14e685d2cdad004647a729006d3e6)
    • spark-plugins/src/test/java/co/cask/hydrator/plugin/spark/test/SparkPluginTest.java (version 8817146381c14e685d2cdad004647a729006d3e6)
    • spark-plugins/widgets/ScalaSparkCompute-sparkcompute.json (version 8817146381c14e685d2cdad004647a729006d3e6)
  • vinishavyasa <CuriousVini@users.noreply.github.com>

    vinishavyasa <CuriousVini@users.noreply.github.com> 89b141faba3ebe4befac040716b6e4b3798331aa m

    Merge pull request #666 from caskdata/fix/compilation-error
    Remove submodules

  • Vinisha Shah

    Vinisha Shah d9c5ce5f5dadd2c42b4933513c8398e69525d33d m

    Remove submodules

    • pom.xml (version d9c5ce5f5dadd2c42b4933513c8398e69525d33d)
  • Sree <sreevatsan@gmail.com>

    Sree <sreevatsan@gmail.com> 54a7c9535463a9b14a783c6ec2909a98b45faa23 m

    Update submodules

    • .gitmodules (version 54a7c9535463a9b14a783c6ec2909a98b45faa23)
    • hashing-tf-feature-generator (version 54a7c9535463a9b14a783c6ec2909a98b45faa23)
    • logistic-regression-analytics (version 54a7c9535463a9b14a783c6ec2909a98b45faa23)
    • naive-bayes-analytics (version 54a7c9535463a9b14a783c6ec2909a98b45faa23)
    • skipgram-analytics (version 54a7c9535463a9b14a783c6ec2909a98b45faa23)
    • tokenizer-analytics (version 54a7c9535463a9b14a783c6ec2909a98b45faa23)
  • Sree <sreevatsan@gmail.com>

    Sree <sreevatsan@gmail.com> 1327cdb2507e9b11417677a1563c7445cf6d1247 m

    Update submodule

    • logistic-regression-analytics (version 1327cdb2507e9b11417677a1563c7445cf6d1247)
    • naive-bayes-analytics (version 1327cdb2507e9b11417677a1563c7445cf6d1247)
    • realtime-stream-source (version 1327cdb2507e9b11417677a1563c7445cf6d1247)
    • skipgram-analytics (version 1327cdb2507e9b11417677a1563c7445cf6d1247)
    • tokenizer-analytics (version 1327cdb2507e9b11417677a1563c7445cf6d1247)
    • wrangler-transform (version 1327cdb2507e9b11417677a1563c7445cf6d1247)
  • Rohit Sinha

    Rohit Sinha d469acb5e9e76c16d2b4ba9c0757473302e28651 m

    Addressed comments

    • database-plugins/src/main/java/co/cask/hydrator/plugin/DBUtils.java (version d469acb5e9e76c16d2b4ba9c0757473302e28651)
  • Shankar <gowrishankar1988@gmail.com>

    Shankar <gowrishankar1988@gmail.com> 84475e74aa92153cbeb30a0c9d7aab267446b65c m

    Merge pull request #665 from caskdata/fix/cdap-11525-log-message-fix
    CDAP-11525 fix incorrect error message in decompression transform

  • Shankar Selvam

    Shankar Selvam 30150998f91fb7621aa5b6ef2d182fbd77be7dce m

    CDAP-11525 fix incorrect error message in decompression transform

    • transform-plugins/src/main/java/co/cask/hydrator/plugin/Decompressor.java (version 30150998f91fb7621aa5b6ef2d182fbd77be7dce)
  • vinishavyasa <CuriousVini@users.noreply.github.com>

    vinishavyasa <CuriousVini@users.noreply.github.com> 35e8c84c5f3e5e6f8d28854a3c868d8a1b6622c6 m

    Merge pull request #663 from caskdata/bugfix_release/remove-s3-plugins
    Moving S3 plugins to separate repo

  • Yaojie Feng

    Yaojie Feng cc61b925437a192c1f6c08e38ba65e363c77969a m

    Merge pull request #664 from caskdata/feature_release/CDAP-11558-fix-multiple-mongo-sinks
    Feature release/cdap 11558 fix multiple mongo sinks

  • Yaojie Feng

    Yaojie Feng 0734a6dce9554dc4637312613d817de704b9690f m

    CDAP-11558 fix multiple mongo sinks

    • mongodb-plugins/src/main/java/co/cask/hydrator/plugin/batch/sink/MongoDBBatchSink.java (version 0734a6dce9554dc4637312613d817de704b9690f)
    • mongodb-plugins/src/test/java/co/cask/hydrator/plugin/test/MongoDBTest.java (version 0734a6dce9554dc4637312613d817de704b9690f)
  • albertshau <albert@cask.co>

    albertshau <albert@cask.co> 88f1897ef728f19eca43ab4909c34513b1cf5d3d m

    Merge pull request #661 from caskdata/feature_release/CDAP-11615-macro-enable-table-schema
    CDAP-11615 macro enable table schema

  • Vinisha Shah

    Vinisha Shah f103d1673fd1b114931300e57eb322b1ff46e12b m

    Moving S3 plugins to separate repo

    • core-plugins/docs/S3-batchsource.md (version f103d1673fd1b114931300e57eb322b1ff46e12b)
    • core-plugins/docs/S3Avro-batchsink.md (version f103d1673fd1b114931300e57eb322b1ff46e12b)
    • core-plugins/docs/S3Parquet-batchsink.md (version f103d1673fd1b114931300e57eb322b1ff46e12b)
    • core-plugins/src/main/java/co/cask/hydrator/plugin/batch/sink/S3AvroBatchSink.java (version f103d1673fd1b114931300e57eb322b1ff46e12b)
    • core-plugins/src/main/java/co/cask/hydrator/plugin/batch/sink/S3BatchSink.java (version f103d1673fd1b114931300e57eb322b1ff46e12b)
    • core-plugins/src/main/java/co/cask/hydrator/plugin/batch/sink/S3ParquetBatchSink.java (version f103d1673fd1b114931300e57eb322b1ff46e12b)
    • core-plugins/src/main/java/co/cask/hydrator/plugin/batch/source/S3BatchSource.java (version f103d1673fd1b114931300e57eb322b1ff46e12b)
    • core-plugins/src/main/java/co/cask/hydrator/plugin/common/Properties.java (version f103d1673fd1b114931300e57eb322b1ff46e12b)
    • core-plugins/src/test/java/co/cask/hydrator/plugin/batch/ETLBatchTestBase.java (version f103d1673fd1b114931300e57eb322b1ff46e12b)
    • core-plugins/src/test/java/co/cask/hydrator/plugin/batch/ETLMapReduceTestRun.java (version f103d1673fd1b114931300e57eb322b1ff46e12b)
    • core-plugins/src/test/java/co/cask/hydrator/plugin/batch/sink/S3BatchSinkTest.java (version f103d1673fd1b114931300e57eb322b1ff46e12b)
    • core-plugins/src/test/java/co/cask/hydrator/plugin/batch/source/S3BatchSourceConfigTest.java (version f103d1673fd1b114931300e57eb322b1ff46e12b)
    • core-plugins/widgets/S3-batchsource.json (version f103d1673fd1b114931300e57eb322b1ff46e12b)
    • core-plugins/widgets/S3Avro-batchsink.json (version f103d1673fd1b114931300e57eb322b1ff46e12b)
    • core-plugins/widgets/S3Parquet-batchsink.json (version f103d1673fd1b114931300e57eb322b1ff46e12b)
  • vinishavyasa <CuriousVini@users.noreply.github.com>

    vinishavyasa <CuriousVini@users.noreply.github.com> f065b98fc11fb6b47004deb955d16f3e5f67215f m

    Merge pull request #662 from caskdata/fix/move-azure-blob-store
    Moving Azure blob store out of hydrator plugins

  • Vinisha Shah

    Vinisha Shah 6f4d11bc6b45f5f123d538ccbd8848fa91cf24c7 m

    Moving Azure blob store out of hydrator plugins

    • core-plugins/docs/AzureBlobStore-batchsource.md (version 6f4d11bc6b45f5f123d538ccbd8848fa91cf24c7)
    • core-plugins/src/main/java/co/cask/hydrator/plugin/batch/source/AzureBatchSource.java (version 6f4d11bc6b45f5f123d538ccbd8848fa91cf24c7)
    • core-plugins/widgets/AzureBlobStore-batchsource.json (version 6f4d11bc6b45f5f123d538ccbd8848fa91cf24c7)
  • Albert Shau <albert@cask.co>

    Albert Shau <albert@cask.co> fea714b56c11b960f496164c6e93a12a7810b54b m

    CDAP-11615 macro enable table schema
    Enabling macros for the schema property of tables.

    • core-plugins/docs/Table-batchsink.md (version fea714b56c11b960f496164c6e93a12a7810b54b)
    • core-plugins/docs/Table-batchsource.md (version fea714b56c11b960f496164c6e93a12a7810b54b)
    • core-plugins/src/main/java/co/cask/hydrator/plugin/batch/sink/BatchWritableSink.java (version fea714b56c11b960f496164c6e93a12a7810b54b)
    • core-plugins/src/main/java/co/cask/hydrator/plugin/batch/sink/TableSink.java (version fea714b56c11b960f496164c6e93a12a7810b54b)
    • core-plugins/src/main/java/co/cask/hydrator/plugin/batch/source/BatchReadableSource.java (version fea714b56c11b960f496164c6e93a12a7810b54b)
    • core-plugins/src/main/java/co/cask/hydrator/plugin/batch/source/TableSource.java (version fea714b56c11b960f496164c6e93a12a7810b54b)
    • core-plugins/src/main/java/co/cask/hydrator/plugin/common/TableSinkConfig.java (version fea714b56c11b960f496164c6e93a12a7810b54b)
    • core-plugins/src/main/java/co/cask/hydrator/plugin/common/TableSourceConfig.java (version fea714b56c11b960f496164c6e93a12a7810b54b)
  • Rohit Sinha

    Rohit Sinha 17f2aa3f8ca66888cf93f426dc384888bca08148 m

    Move transformValue function from DBRecord to DBUtils as public static method

    • database-plugins/src/main/java/co/cask/hydrator/plugin/DBRecord.java (version 17f2aa3f8ca66888cf93f426dc384888bca08148)
    • database-plugins/src/main/java/co/cask/hydrator/plugin/DBUtils.java (version 17f2aa3f8ca66888cf93f426dc384888bca08148)
  • Yaojie Feng

    Yaojie Feng 253a99de9ccbf8b39da6dcfe21609c2f92e93d61 m

    Merge pull request #657 from caskdata/feature_release/cdap-9334-update-KeyValueListParser-error-message
    CDAP-9334 update KeyValueListParser error message

  • Yaojie Feng

    Yaojie Feng eb9ad74680d22b7495f461a873950648450ea2d3 m

    CDAP-9334 update KeyValueListParser error message

    • hydrator-common/src/main/java/co/cask/hydrator/common/KeyValueListParser.java (version eb9ad74680d22b7495f461a873950648450ea2d3)