Build #262

Build: #262 was successful

Job: Default Job was successful

Stages & jobs

  1. Default Stage

Code commits

Hydrator Plugins

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 988215d35aaf4d241ec0d191cf4b03a50a25d5a9

    Merge pull request #978 from cdapio/merge-release
    Merge release

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 4fe64954e64347a7aa81a3f3e59ea2919654f245

    Merge remote-tracking branch 'origin/release/2.3' into merge-release

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 11f47841b0673aee5bfe8ac17dce7073f52d4bc7 m

    Merge pull request #977 from cdapio/bump-version-2-3-0
    bump version to 2.3.0 and update submodule

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 7ea874ae0d005b8288cf3d24e1e1f815d9cb2488 m

    bump version to 2.3.0 and update submodule

    • amazon-s3-plugins (version 7ea874ae0d005b8288cf3d24e1e1f815d9cb2488)
    • cassandra-plugins/pom.xml (version 7ea874ae0d005b8288cf3d24e1e1f815d9cb2488)
    • condition-plugins (version 7ea874ae0d005b8288cf3d24e1e1f815d9cb2488)
    • core-plugins/pom.xml (version 7ea874ae0d005b8288cf3d24e1e1f815d9cb2488)
    • database-plugins/pom.xml (version 7ea874ae0d005b8288cf3d24e1e1f815d9cb2488)
    • format-avro/pom.xml (version 7ea874ae0d005b8288cf3d24e1e1f815d9cb2488)
    • format-blob/pom.xml (version 7ea874ae0d005b8288cf3d24e1e1f815d9cb2488)
    • format-common/pom.xml (version 7ea874ae0d005b8288cf3d24e1e1f815d9cb2488)
    • format-delimited/pom.xml (version 7ea874ae0d005b8288cf3d24e1e1f815d9cb2488)
    • format-json/pom.xml (version 7ea874ae0d005b8288cf3d24e1e1f815d9cb2488)
    • format-orc/pom.xml (version 7ea874ae0d005b8288cf3d24e1e1f815d9cb2488)
    • format-parquet/pom.xml (version 7ea874ae0d005b8288cf3d24e1e1f815d9cb2488)
    • format-text/pom.xml (version 7ea874ae0d005b8288cf3d24e1e1f815d9cb2488)
    • google-cloud (version 7ea874ae0d005b8288cf3d24e1e1f815d9cb2488)
    • hbase-plugins/pom.xml (version 7ea874ae0d005b8288cf3d24e1e1f815d9cb2488)
    • hive-plugins/pom.xml (version 7ea874ae0d005b8288cf3d24e1e1f815d9cb2488)
    • http-plugins/pom.xml (version 7ea874ae0d005b8288cf3d24e1e1f815d9cb2488)
    • hydrator-common/pom.xml (version 7ea874ae0d005b8288cf3d24e1e1f815d9cb2488)
    • kafka-plugins (version 7ea874ae0d005b8288cf3d24e1e1f815d9cb2488)
    • mongodb-plugins/pom.xml (version 7ea874ae0d005b8288cf3d24e1e1f815d9cb2488)
    • pom.xml (version 7ea874ae0d005b8288cf3d24e1e1f815d9cb2488)
    • solrsearch-plugins/pom.xml (version 7ea874ae0d005b8288cf3d24e1e1f815d9cb2488)
    • spark-plugins/pom.xml (version 7ea874ae0d005b8288cf3d24e1e1f815d9cb2488)
    • transform-plugins/pom.xml (version 7ea874ae0d005b8288cf3d24e1e1f815d9cb2488)
    • wrangler-transform (version 7ea874ae0d005b8288cf3d24e1e1f815d9cb2488)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> c218e104f7247e05b5fc0a8076f22d98a14aa043 m

    Merge pull request #956 from cdapio/feature/CDAP-15787-4-Aggregators
    (CDAP-15787) 4 Aggregators

  • Vinisha Shah <CuriousVini@users.noreply.github.com>

    Vinisha Shah <CuriousVini@users.noreply.github.com> acf26fce0aabde4a938bb0b4ac7f190f590a4a69 m

    Merge pull request #970 from cdapio/bugfix_release/CDAP-15787-add-validation-decompressor
     [CDAP-15787] Add failure collector for Decompressor

  • Vinisha Shah <CuriousVini@users.noreply.github.com>

    Vinisha Shah <CuriousVini@users.noreply.github.com> 0b8bfcba78d9aa5f626581872e6d8843bf3a8783 m

    Merge pull request #969 from cdapio/feature_release/CDAP-15787-validation-decoder
    [CDAP-15787] Add failure collector for Decryptor and Decoder

  • Vinisha Shah <CuriousVini@users.noreply.github.com>

    Vinisha Shah <CuriousVini@users.noreply.github.com> 190c2f2e329d4cf65301264438d14678d8ddcdc2 m

    Merge pull request #942 from cdapio/feature/add-failure-collector
    [CDAP-15787] Failure Collector for Database plugins

  • Vinisha Shah

    Vinisha Shah 122a37eb18ce8555af25bec0d4e1960d9a524728 m

    [CDAP-15787] Failure Collector for Database plugins

    • database-plugins/src/main/java/io/cdap/plugin/DBConfig.java (version 122a37eb18ce8555af25bec0d4e1960d9a524728)
    • database-plugins/src/main/java/io/cdap/plugin/DBManager.java (version 122a37eb18ce8555af25bec0d4e1960d9a524728)
    • database-plugins/src/main/java/io/cdap/plugin/db/batch/TransactionIsolationLevel.java (version 122a37eb18ce8555af25bec0d4e1960d9a524728)
    • database-plugins/src/main/java/io/cdap/plugin/db/batch/action/DBAction.java (version 122a37eb18ce8555af25bec0d4e1960d9a524728)
    • database-plugins/src/main/java/io/cdap/plugin/db/batch/action/QueryAction.java (version 122a37eb18ce8555af25bec0d4e1960d9a524728)
    • database-plugins/src/main/java/io/cdap/plugin/db/batch/sink/DBSink.java (version 122a37eb18ce8555af25bec0d4e1960d9a524728)
    • database-plugins/src/main/java/io/cdap/plugin/db/batch/source/DBSource.java (version 122a37eb18ce8555af25bec0d4e1960d9a524728)
    • database-plugins/src/test/java/io/cdap/plugin/db/batch/sink/DBSinkTestRun.java (version 122a37eb18ce8555af25bec0d4e1960d9a524728)
    • hydrator-common/src/main/java/io/cdap/plugin/common/ReferenceBatchSink.java (version 122a37eb18ce8555af25bec0d4e1960d9a524728)
    • hydrator-common/src/main/java/io/cdap/plugin/common/ReferenceBatchSource.java (version 122a37eb18ce8555af25bec0d4e1960d9a524728)
  • Vinisha Shah <CuriousVini@users.noreply.github.com>

    Vinisha Shah <CuriousVini@users.noreply.github.com> 9702308fd8424972ace1e77a3495a6a442ba39f4 m

    Merge pull request #949 from cdapio/feature/CDAP-15787-5-Action-Plugins
    (CDAP-15787) 5 Action Plugins

  • Mo Eseifan

    Mo Eseifan 4d56b67568daa51cbd9219bad007596e0ff98e97 m

    [CDAP-15787] Implemented FailureCollector into EmailAction
    [CDAP-15787] Fixed error in QueryAction caused by changing ConditionConfig

    [CDAP-15787] Implemented FailureCollector into FileDeleteAction

    [CDAP-15787] Implemented FailureCollector into FileMoveAction

    [CDAP-15787] Implemented FailureCollector into SSHAction

    [CDAP-15787] Implemented FailureCollector into WindowsShareCopy

    [CDAP-15787] Style fixes

    [CDAP-15787] Refactored validate method

    Addressed comments on PR

    [CDAP-15787] Fixed indentation

    [CDAP-15787] Style fix

    [CDAP-15787] Fixed formatting

    • core-plugins/src/main/java/io/cdap/plugin/batch/action/EmailAction.java (version 4d56b67568daa51cbd9219bad007596e0ff98e97)
    • core-plugins/src/main/java/io/cdap/plugin/batch/action/FileDeleteAction.java (version 4d56b67568daa51cbd9219bad007596e0ff98e97)
    • core-plugins/src/main/java/io/cdap/plugin/batch/action/FileMoveAction.java (version 4d56b67568daa51cbd9219bad007596e0ff98e97)
    • core-plugins/src/main/java/io/cdap/plugin/batch/action/SSHAction.java (version 4d56b67568daa51cbd9219bad007596e0ff98e97)
    • core-plugins/src/main/java/io/cdap/plugin/batch/action/WindowsShareCopy.java (version 4d56b67568daa51cbd9219bad007596e0ff98e97)
    • database-plugins/src/main/java/io/cdap/plugin/db/batch/action/QueryAction.java (version 4d56b67568daa51cbd9219bad007596e0ff98e97)
    • google-cloud (version 4d56b67568daa51cbd9219bad007596e0ff98e97)
    • hydrator-common/src/main/java/io/cdap/plugin/common/batch/action/ConditionConfig.java (version 4d56b67568daa51cbd9219bad007596e0ff98e97)
  • Vinisha Shah <CuriousVini@users.noreply.github.com>

    Vinisha Shah <CuriousVini@users.noreply.github.com> dc8825450ac9c1486b236d6164c1127a9a8d0726 m

    Merge pull request #967 from cdapio/feature_release/CDAP-15787-validation-csvparser
    [CDAP-15787] Add failure collector for CSVParser

  • Mo Eseifan

    Mo Eseifan f630baea7a3c8d113c0ac88d51c9dbbe4658683d m

    [CDAP-15787] Implemented FailureCollector for aggregators

    • core-plugins/src/main/java/io/cdap/plugin/batch/aggregator/DedupAggregator.java (version f630baea7a3c8d113c0ac88d51c9dbbe4658683d)
    • core-plugins/src/main/java/io/cdap/plugin/batch/aggregator/DistinctAggregator.java (version f630baea7a3c8d113c0ac88d51c9dbbe4658683d)
    • core-plugins/src/main/java/io/cdap/plugin/batch/aggregator/GroupByAggregator.java (version f630baea7a3c8d113c0ac88d51c9dbbe4658683d)
    • core-plugins/src/main/java/io/cdap/plugin/batch/aggregator/RowDenormalizerAggregator.java (version f630baea7a3c8d113c0ac88d51c9dbbe4658683d)
    • core-plugins/src/test/java/io/cdap/plugin/batch/aggregator/RowDenormalizerConfigTest.java (version f630baea7a3c8d113c0ac88d51c9dbbe4658683d)
  • Edwin Elia <edwinelia@google.com>

    Edwin Elia <edwinelia@google.com> 9316831c880cb2e9355ca388e3a08e818223a194 m

    Merge pull request #976 from cdapio/add-database-get-schema
    add database get schema button

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 90579708787ef6eb497c8494b5658214e7ea0eca m

    Merge pull request #948 from cdapio/feature/CDAP-15787-Alert-Plugin
    (CDAp-15787) Alert plugin

  • Mo Eseifan

    Mo Eseifan 1077ea6804c3dc7e74e64e05d91ffebbd3b29abc m

    CDAP-15787 Implemented FailureCollector into Validate method

    • core-plugins/src/main/java/io/cdap/plugin/alert/TMSAlertPublisher.java (version 1077ea6804c3dc7e74e64e05d91ffebbd3b29abc)
  • Vinisha Shah <CuriousVini@users.noreply.github.com>

    Vinisha Shah <CuriousVini@users.noreply.github.com> 59f25a8afe6fbab7a5acb0703cf9b0df26a1266e m

    Merge pull request #966 from cdapio/feature_release/CDAP-15787-validation-csvformatter
    [CDAP-15787] Add failure collector for CSVFormatter

  • Edwin Elia <edwinelia@google.com>

    Edwin Elia <edwinelia@google.com> 5266f3c96c75c28b082cf9ec5954a9360f3bffb3 m

    add database get schema button

    • database-plugins/widgets/Database-batchsource.json (version 5266f3c96c75c28b082cf9ec5954a9360f3bffb3)
  • Vinisha Shah

    Vinisha Shah 8d1a2fe315c56c6b303f6433f5f2c7eecdea88b1 m

    [CDAP-15787] Add failure collector for Decompressor

    • transform-plugins/src/main/java/io/cdap/plugin/Decompressor.java (version 8d1a2fe315c56c6b303f6433f5f2c7eecdea88b1)
    • transform-plugins/src/test/java/io/cdap/plugin/DecompressorTest.java (version 8d1a2fe315c56c6b303f6433f5f2c7eecdea88b1)
  • Vinisha Shah

    Vinisha Shah ac1edd7f48ce674887fe1519290e02cc8fbe7f4f m

    [CDAP-15787] Add failure collector for Decoder

    • transform-plugins/src/main/java/io/cdap/plugin/Decoder.java (version ac1edd7f48ce674887fe1519290e02cc8fbe7f4f)
    • transform-plugins/src/test/java/io/cdap/plugin/DecoderTest.java (version ac1edd7f48ce674887fe1519290e02cc8fbe7f4f)
  • Vinisha Shah

    Vinisha Shah 4b90cb3046767dc79f272915e1578d845384c6fe m

    [CDAP-15787] Add failure collector for Decryptor

    • transform-plugins/src/main/java/io/cdap/plugin/Decryptor.java (version 4b90cb3046767dc79f272915e1578d845384c6fe)
  • Vinisha Shah

    Vinisha Shah 3eac3112872950979d6e7b757117d7e96876c568 m

    [CDAP-15787] Add failure collector for CSVParser

    • transform-plugins/src/main/java/io/cdap/plugin/CSVParser.java (version 3eac3112872950979d6e7b757117d7e96876c568)
    • transform-plugins/src/test/java/io/cdap/plugin/CSVParserTest.java (version 3eac3112872950979d6e7b757117d7e96876c568)
  • Vinisha Shah

    Vinisha Shah 77210859b2c047eebd1b1ad8f2a3548abc55e4d6 m

    [CDAP-15787] Add failure collector for CSVFormatter

    • transform-plugins/src/main/java/io/cdap/plugin/CSVFormatter.java (version 77210859b2c047eebd1b1ad8f2a3548abc55e4d6)
    • transform-plugins/src/test/java/io/cdap/plugin/CSVFormatterTest.java (version 77210859b2c047eebd1b1ad8f2a3548abc55e4d6)
  • Vinisha Shah <CuriousVini@users.noreply.github.com>

    Vinisha Shah <CuriousVini@users.noreply.github.com> b192ecdc3cb10417ebfb3bf1515c03d9ee98d061 m

    Merge pull request #964 from cdapio/feature/add-validation-transform
    [CDAP-15787] Add failure collector for CloneRecord and Compressor

  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 99c7e6c61e6811b27e681187d212b887cc0415d4 m

    Merge pull request #975 from cdapio/feature/CDAP-15787-Decoder
    [CDAP-15787] Fixed decoder unit tests to use new encoder

  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> bfc60aa4bf60a4c0cc4bbd68d670a556ae834a64 m

    [CDAP-15787] Fixed decoder unit tests to use new encoder

    • transform-plugins/src/test/java/io/cdap/plugin/DecoderTest.java (version bfc60aa4bf60a4c0cc4bbd68d670a556ae834a64)
  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> fe04fe97e0767f4efe9127f6c53efda9097e9d7f m

    Merge pull request #960 from cdapio/feature/CDAP-15787-UnionSplitter
    [CDAP-15787] Updated UnionSplitter for new Validation API

  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> d43fccbc1e6572b8f089feecbe1d27671ef25761 m

    [CDAP-15787] Updated UnionSplitter for new Validation API
    [CDAP-15787] Removed some unnecessary nesting, added more descriptive corrective action for invalid types

    [CDAP-15787] Fixed invalid type inside of union to only return one validation error

    [CDAP-15787] Improved validation error message for invalid union types

    [CDAP-15787] Fixed unit test for added validation failure

    • transform-plugins/src/main/java/io/cdap/plugin/UnionSplitter.java (version d43fccbc1e6572b8f089feecbe1d27671ef25761)
    • transform-plugins/src/test/java/io/cdap/plugin/UnionSplitterTest.java (version d43fccbc1e6572b8f089feecbe1d27671ef25761)
  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> eaea9a54a6b0168c828800df08fe0a72415fe2d3 m

    Merge pull request #959 from cdapio/feature/CDAP-15787-NullFieldSplitter
    [CDAP-15787] Updated NullFieldSplitter for new Validation API

  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 523230977b516d3e3c9400ecc5d36a7ea80554b4 m

    [CDAP-15787] Updated NullFieldSplitter for new Validation API
    [CDAP-15787] Removed extra comment, changed message wording

    [CDAP-15787] Refactored getNonNullSchema method to use nullable failure collector

    • transform-plugins/src/main/java/io/cdap/plugin/NullFieldSplitter.java (version 523230977b516d3e3c9400ecc5d36a7ea80554b4)
    • transform-plugins/src/test/java/io/cdap/plugin/NullFieldSplitterTest.java (version 523230977b516d3e3c9400ecc5d36a7ea80554b4)
  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 62c8fbe2fa491d52d0675dc10b4b9178324ea614 m

    Merge pull request #965 from cdapio/feature/CDAP-15787-Normalizer
    [CDAP-15787] Updated Normalizer for new Validation API

  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 5deb597fb819feb0baeca29c396c54301bc6c987 m

    [CDAP-15787] Updated Normalizer for new Validation API
    [CDAP-15787] Removed nesting wherever possible

    • transform-plugins/src/main/java/io/cdap/plugin/Normalize.java (version 5deb597fb819feb0baeca29c396c54301bc6c987)
    • transform-plugins/src/test/java/io/cdap/plugin/NormalizeTest.java (version 5deb597fb819feb0baeca29c396c54301bc6c987)
  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> c781c0147d679e9c0763c7f339a357b5c735a9eb m

    Merge pull request #971 from cdapio/feature/CDAP-15787-JSONParser
    [CDAP-15787] Updated JSONParser for new Validation API

  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> f0442e142fd73fb291db5ed36a762e16a00b27b1 m

    [CDAP-15787] Updated JSONParser for new Validation API

    • transform-plugins/src/main/java/io/cdap/plugin/JSONParser.java (version f0442e142fd73fb291db5ed36a762e16a00b27b1)
    • transform-plugins/src/test/java/io/cdap/plugin/JSONParserTest.java (version f0442e142fd73fb291db5ed36a762e16a00b27b1)
  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 5bd75396a4272808bbcdbedc281e468e4b75bc16 m

    Merge pull request #974 from cdapio/feature/CDAP-15787-LogParserTransform
    [CDAP-15787] Fixed LogParserTransform Merge

  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 37bb2c6916bc6b8c0c6b53dee010e5bf7e849a6f m

    [CDAP-15787] Added validation to LogParserTransform
    [CDAP-15787] Fixed unit tests and checkstyle errors for LogParserTransform

    [CDAP-15787] Improved error messaging, changed tests to explicitly assert validation failure causes, changed FailureCollector parameters to be nonnullable

    [CDAP-15787] Added specific assertions for ValidationFailure causes, fixed logic to not continue to validate if input schema is not of type record

    [CDAP-15787] Fixed validation failure causes to only display the configuration property when an input stage field is missing

    [CDAP-15787] Included validation checks only in configurePipeline, improved the error messages, and added overloading validation function to throw an exception during runtime

    [CDAP-15787] Changed schema to NonNullable if isNullable()

    [CDAP-15787] Fixed indentation formatting, improved error messages, removed unnecessary comments, improved if nesting readability

    • core-plugins/src/main/java/io/cdap/plugin/transform/LogParserTransform.java (version 37bb2c6916bc6b8c0c6b53dee010e5bf7e849a6f)
    • core-plugins/src/test/java/io/cdap/plugin/transform/LogParserTransformTest.java (version 37bb2c6916bc6b8c0c6b53dee010e5bf7e849a6f)
  • yaojiefeng <yaojie@google.com>

    yaojiefeng <yaojie@google.com> 8e5e724e41086c47adba54968c960783a4c24ca4 m

    Merge pull request #973 from cdapio/feature_release/throw-Exception-for-AbstractFileSink
    throw IOException in AbstractFileSInk

  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 6721e5c82967dcd4b9d236799c297f1f870b64fe m

    Merge pull request #962 from cdapio/feature/CDAP-15787-Encoder
    [CDAP-15787] Updated Encoder for new Validation API

  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 645c9764f32aa3f4ac6f20e1847a6fa5e8f559b5 m

    [CDAP-15787] Updated Encoder for new Validation API
    [CDAP-15787] Removed extraneous comment

    • transform-plugins/src/main/java/io/cdap/plugin/Encoder.java (version 645c9764f32aa3f4ac6f20e1847a6fa5e8f559b5)
    • transform-plugins/src/test/java/io/cdap/plugin/EncoderTest.java (version 645c9764f32aa3f4ac6f20e1847a6fa5e8f559b5)
  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 5944dc647f9f18918c4722bcb7142fc5c0552c9a m

    Merge pull request #963 from cdapio/feature/CDAP-15787-JSONFormatter
    [CDAP-15787] Updated JSONFormatter for new Validation API

  • yaojie <yaojie@google.com>

    yaojie <yaojie@google.com> ad48d08556cfadcefb6698f94fd4e8b5aee5277d m

    change to Exception

    • format-common/src/main/java/io/cdap/plugin/format/plugin/AbstractFileSink.java (version ad48d08556cfadcefb6698f94fd4e8b5aee5277d)
  • yaojie <yaojie@google.com>

    yaojie <yaojie@google.com> 0318ece301f2a8d0098df5674abe1d054b96a411 m

    throw IOException in AbstractFileSInk

    • format-common/src/main/java/io/cdap/plugin/format/plugin/AbstractFileSink.java (version 0318ece301f2a8d0098df5674abe1d054b96a411)
  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 77e855d898cccd88c4ffec03b6a4c02878c2cd1f m

    Merge pull request #943 from cdapio/feature/CDAP-15787-ProjectionTransform
    [CDAP-15787] Added validation API support for ProjectionTransform

  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 72e7721dd6d3f9d8066ab897d14744b8c81eaeb4 m

    [CDAP-15787] Added validation API support for ProjectionTransform
    [CDAP-15787] Made FailureCollector nonnullable, added specific validation checks to unit tests, improved validation api messages

    [CDAP-15787] Removed @Nonnull annotations

    [CDAP-15787] Fixed some messages to be more consistent, changed unit tests to not assert exception message, moved throwing the validation exception to outside of the init function call

    [CDAP-15787] Moved constant strings to ProjectionTransformConfig

    [CDAP-15787] Removed extra comments, improved error messages

    • core-plugins/src/main/java/io/cdap/plugin/transform/ProjectionTransform.java (version 72e7721dd6d3f9d8066ab897d14744b8c81eaeb4)
    • core-plugins/src/test/java/io/cdap/plugin/transform/ProjectionTransformTest.java (version 72e7721dd6d3f9d8066ab897d14744b8c81eaeb4)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 57a86fa072f8ada01093876234b07ea845fa241f m

    Merge pull request #972 from cdapio/feature_release/CDAP-make-prepareRun-non-final
    Make prepareRun() non final in AbstractFileSink

  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 763e0cbc8e361e758d2dab2b82e2aa60670e56a9 m

    Merge pull request #946 from cdapio/feature/CDAP-15787-JavaScriptTransform
    [CDAP-15787] Updated JavaScriptTransform for new validation API

  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> c058275144a863379c8f90194ac4466337c898bc m

    [CDAP-15787] Updated JavaScriptTransform for new validation API
    [CDAP-15787] Undo the transform method changes, created final strings for the config parameters, modified the ValidationFailures to be more concise

    [CDAP-15787] Fixed indentation errors, moved constant field names to config

    • core-plugins/src/main/java/io/cdap/plugin/transform/JavaScriptTransform.java (version c058275144a863379c8f90194ac4466337c898bc)
  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 9cb3c7cf2fb74f964fcc59c8fc716bce8eb3e420 m

    Merge pull request #950 from cdapio/feature/CDAP-15787-Joiner
    [CDAP-15787] Update Joiner for New Validation API

  • yaojie <yaojie@google.com>

    yaojie <yaojie@google.com> fa9f1b502a204a9e4f117de67556c0daa0ee7191 m

    Make prepareRun() non final in AbstractFileSink

    • format-common/src/main/java/io/cdap/plugin/format/plugin/AbstractFileSink.java (version fa9f1b502a204a9e4f117de67556c0daa0ee7191)
  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 4f44ac9491616a9de67ec05e8cf7043c3cd326c6 m

    [CDAP-15787] Updated Joiner for new validation API
    [CDAP-15787] Changed FailureCollector parameters to be Nonnullable, updated exception catching in unit tests to catch all exception messages

    [CDAP-15787] Fixed formatting to be more consistent, moved exception throwing outside of the function calls, moved constant strings to config, and removed exception message checking from the unit tests

    [CDAP-15787] Changed field references to reference config class

    [CDAP-15787] Removed extra comments, moved if-else nesting upwards

    • core-plugins/src/main/java/io/cdap/plugin/batch/joiner/Joiner.java (version 4f44ac9491616a9de67ec05e8cf7043c3cd326c6)
    • core-plugins/src/main/java/io/cdap/plugin/batch/joiner/JoinerConfig.java (version 4f44ac9491616a9de67ec05e8cf7043c3cd326c6)
    • core-plugins/src/test/java/io/cdap/plugin/batch/joiner/JoinerConfigTest.java (version 4f44ac9491616a9de67ec05e8cf7043c3cd326c6)
  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 3d0117a7d4146704070d21b652d853fc3c7ac616 m

    Merge pull request #951 from cdapio/feature/CDAP-15787-ErrorCollector
    [CDAP-15787] Updated ErrorCollector for new validation API

  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 9aeefab7cb84a1e643e46caafaad4b949891f677 m

    [CDAP-15787] Updated ErrorCollector for new validation API
    [CDAP-15787] Fixed validation messages and fixed code formatting

    [CDAP-15787] Improved error message

    • core-plugins/src/main/java/io/cdap/plugin/error/ErrorCollector.java (version 9aeefab7cb84a1e643e46caafaad4b949891f677)
  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 20fa64aca8843bbdecdd397f04181625246d6d53 m

    Merge pull request #952 from cdapio/feature/CDAP-15787-ExcelInputReader
    [CDAP-15787] Updated ExcelInputReader for New Validation API

  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> b9c2474ef820e6011c132a404dbbe60ebf49ae21 m

    [CDAP-15787] Update ExcelInputReader for New Validation API
    [CDAP-15787] Fixed string format runtime error

    [CDAP-15787] Hid error collection fields, overloaded createDatasets method, fixed some formatting errors, moved exception throwing outside of submethods

    [CDAP-15787] Added new validation for checking mapping fields

    [CDAP-15787] Removed extra comments, removed extraneous exception catching, removed documentation for hidden deprecated fields

    • core-plugins/src/main/java/io/cdap/plugin/batch/source/ExcelInputReader.java (version b9c2474ef820e6011c132a404dbbe60ebf49ae21)
    • core-plugins/widgets/Excel-batchsource.json (version b9c2474ef820e6011c132a404dbbe60ebf49ae21)
  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 341129cc240a413308262d8153084d04ab5ec7f7 m

    Merge pull request #953 from cdapio/feature/CDAP-15787-XMLReaderBatchSource
    [CDAP-15787] Updated XMLReaderBatchSource for New Validation API

  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> ed64b67743243ea23e374cecf10b8336ce430b10 m

    [CDAP-15787] Updated XMLReaderBatchSource and XMLReaderConfigTest for new validation API
    [CDAP-15787] Moved constant field names to config, fixed unit tests to properly assert validations

    [CDAP-15787] Changed table name and table expiry fields to hidden

    [CDAP-15787] Removed extra comments

    • core-plugins/src/main/java/io/cdap/plugin/batch/source/XMLReaderBatchSource.java (version ed64b67743243ea23e374cecf10b8336ce430b10)
    • core-plugins/src/test/java/io/cdap/plugin/batch/source/XMLReaderConfigTest.java (version ed64b67743243ea23e374cecf10b8336ce430b10)
    • core-plugins/widgets/XMLReader-batchsource.json (version ed64b67743243ea23e374cecf10b8336ce430b10)
  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 854d1fb6970b2c2cc2d46d15f3ff18d7b463ccae m

    Merge pull request #954 from cdapio/feature/CDAP-15787-ValueMapper
    [CDAP-15787] Updated ValueMapper for new Validation API

  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 8543146dd5a423021dfaea230a4149a697f4f9bd m

    [CDAP-15787] Updated ValueMapper for new Validation API and added additional validation
    [CDAP-15787] Fixed formatting, fixed unit tests to not use exceptions, moved constant strings to config

    [CDAP-15787] Removed extra comments, improved error messages

    • transform-plugins/src/main/java/io/cdap/plugin/ValueMapper.java (version 8543146dd5a423021dfaea230a4149a697f4f9bd)
    • transform-plugins/src/test/java/io/cdap/plugin/ValueMapperTest.java (version 8543146dd5a423021dfaea230a4149a697f4f9bd)
  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 6252d963476d9d1346e6979a579696de6d9ac20b m

    Merge pull request #955 from cdapio/feature/CDAP-15787-XMLParser
    [CDAP-15787] Updated XMLParser for new Validation API

  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 7b502f73d43eff79fbf68cf50d5b748fecf9416d m

    [CDAP-15787] Updated XMLParser for new Validation API
    [CDAP-15787] Fixed input field to not validate if using macros

    [CDAP-15787] Fixed formatting, fixed unit tests to not check exceptions

    [CDAP-15787] Fixed minor indentation error

    [CDAP-15787] Removed extra comments, improved error messages

    • transform-plugins/src/main/java/io/cdap/plugin/XMLParser.java (version 7b502f73d43eff79fbf68cf50d5b748fecf9416d)
    • transform-plugins/src/test/java/io/cdap/plugin/XMLParserTest.java (version 7b502f73d43eff79fbf68cf50d5b748fecf9416d)
  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> a46ea983273b1da80d3e9a0f22f2a968613c5314 m

    Merge pull request #957 from cdapio/feature/CDAP-15787-XMLMultiParser
    [CDAP-15787] Updated XMLMultiParser for new Validation API

  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 06f3a72b2235498f42f92cb21ebdbd96c4b1d865 m

    [CDAP-15787] Updated XMLMultiParser for new Validation API
    [CDAP-15787] Fixed minor indentation errors

    [CDAP-15787] Removed extra comment, improved message

    • transform-plugins/src/main/java/io/cdap/plugin/XMLMultiParser.java (version 06f3a72b2235498f42f92cb21ebdbd96c4b1d865)
  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> f54dcd45b158f4b8f80b0820688ccf8769ef88c1 m

    Merge pull request #958 from cdapio/feature/CDAP-15787-XMLToJSON
    [CDAP-15787] Updated XMLToJSON for new Validation API

  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> bd52fafb87e46fc86b00e39c4726e5883497a80c m

    [CDAP-15787] Updated XMLToJSON for new Validation API
    [CDAP-15787] Fixed indentation, refactored conditional to make it more clear

    • transform-plugins/src/main/java/io/cdap/plugin/XMLToJSON.java (version bd52fafb87e46fc86b00e39c4726e5883497a80c)
    • transform-plugins/src/test/java/io/cdap/plugin/XMLToJSONConverterTest.java (version bd52fafb87e46fc86b00e39c4726e5883497a80c)
  • Vinisha Shah <CuriousVini@users.noreply.github.com>

    Vinisha Shah <CuriousVini@users.noreply.github.com> 1bae1e563fc28e8987e499f3ea45d25742f7a77b m

    Merge pull request #968 from cdapio/bugfix_release/fix-test
    Fix ETLMapReduceTestRun testDAG and testTableToTableWithValidations

  • Vinisha Shah

    Vinisha Shah b0dd8d039a7d8b5c623c42d4723e048f33b6318d m

    Fix ETLMapReduceTestRun testDAG and testTableToTableWithValidations

    • core-plugins/src/test/java/io/cdap/plugin/batch/ETLMapReduceTestRun.java (version b0dd8d039a7d8b5c623c42d4723e048f33b6318d)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 48359964539feb242544e215cde810c43390c48e m

    Merge pull request #961 from cdapio/feature/CDAP-15917-remove-validator
    CDAP-15917 remove very outdated validator plugin

  • Vinisha Shah

    Vinisha Shah 502693a236250f58a892e74dc1a6803ce54429c9 m

    [CDAP-15787] Add failure collector for CloneRecord and Compressor

    • format-common/src/main/java/io/cdap/plugin/format/input/PathTrackingInputFormat.java (version 502693a236250f58a892e74dc1a6803ce54429c9)
    • format-common/src/main/java/io/cdap/plugin/format/plugin/AbstractFileSink.java (version 502693a236250f58a892e74dc1a6803ce54429c9)
    • format-common/src/main/java/io/cdap/plugin/format/plugin/AbstractFileSource.java (version 502693a236250f58a892e74dc1a6803ce54429c9)
    • format-common/src/main/java/io/cdap/plugin/format/plugin/FileSinkProperties.java (version 502693a236250f58a892e74dc1a6803ce54429c9)
    • format-common/src/main/java/io/cdap/plugin/format/plugin/FileSourceProperties.java (version 502693a236250f58a892e74dc1a6803ce54429c9)
    • transform-plugins/src/main/java/io/cdap/plugin/CloneRecord.java (version 502693a236250f58a892e74dc1a6803ce54429c9)
    • transform-plugins/src/main/java/io/cdap/plugin/Compressor.java (version 502693a236250f58a892e74dc1a6803ce54429c9)
    • transform-plugins/src/test/java/io/cdap/plugin/CloneRecordTest.java (version 502693a236250f58a892e74dc1a6803ce54429c9)
    • transform-plugins/src/test/java/io/cdap/plugin/CompressorTest.java (version 502693a236250f58a892e74dc1a6803ce54429c9)
  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 82522676fbe2d7515b86295eaa47a3b5307b40ea m

    [CDAP-15787] Updated JSONFormatter for new Validation API

    • transform-plugins/src/main/java/io/cdap/plugin/JSONFormatter.java (version 82522676fbe2d7515b86295eaa47a3b5307b40ea)
    • transform-plugins/src/test/java/io/cdap/plugin/JSONFormatterTest.java (version 82522676fbe2d7515b86295eaa47a3b5307b40ea)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 9b7640246b8c00768465524d96c3b44fed5d5d46 m

    CDAP-15917 remove very outdated validator plugin

    • core-plugins/docs/Validator-transform.md (version 9b7640246b8c00768465524d96c3b44fed5d5d46)
    • core-plugins/src/main/java/io/cdap/plugin/transform/ValidatorScriptContext.java (version 9b7640246b8c00768465524d96c3b44fed5d5d46)
    • core-plugins/src/main/java/io/cdap/plugin/transform/ValidatorTransform.java (version 9b7640246b8c00768465524d96c3b44fed5d5d46)
    • core-plugins/src/main/java/io/cdap/plugin/validator/CoreValidator.java (version 9b7640246b8c00768465524d96c3b44fed5d5d46)
    • core-plugins/src/test/java/io/cdap/plugin/batch/ETLBatchTestBase.java (version 9b7640246b8c00768465524d96c3b44fed5d5d46)
    • core-plugins/src/test/java/io/cdap/plugin/batch/aggregator/RowDenormalizerConfigTest.java (version 9b7640246b8c00768465524d96c3b44fed5d5d46)
    • core-plugins/src/test/java/io/cdap/plugin/transform/JavaScriptTransformTest.java (version 9b7640246b8c00768465524d96c3b44fed5d5d46)
    • core-plugins/src/test/java/io/cdap/plugin/transform/LogParserTransformTest.java (version 9b7640246b8c00768465524d96c3b44fed5d5d46)
    • core-plugins/src/test/java/io/cdap/plugin/transform/ProjectionTransformTest.java (version 9b7640246b8c00768465524d96c3b44fed5d5d46)
    • core-plugins/src/test/java/io/cdap/plugin/transform/ValidatorTransformTest.java (version 9b7640246b8c00768465524d96c3b44fed5d5d46)
    • core-plugins/src/test/java/io/cdap/plugin/validator/CoreValidator.java (version 9b7640246b8c00768465524d96c3b44fed5d5d46)
  • Vinisha Shah <CuriousVini@users.noreply.github.com>

    Vinisha Shah <CuriousVini@users.noreply.github.com> 8653afad62b21cfc992a7990814e8f0a3f735076 m

    Merge pull request #940 from cdapio/feature/add-validation-spark-plugins
    [CDAP-15787] Add plugin validation in spark plugins and http plugin

  • Vinisha Shah

    Vinisha Shah eaa26f9c5783a2b049a92adca2277e327a644517 m

    [CDAP-15787] Add plugin validation in spark plugins and http plugin

    • http-plugins/src/main/java/io/cdap/plugin/batch/HTTPCallbackAction.java (version eaa26f9c5783a2b049a92adca2277e327a644517)
    • hydrator-common/src/main/java/io/cdap/plugin/common/http/HTTPConfig.java (version eaa26f9c5783a2b049a92adca2277e327a644517)
    • hydrator-common/src/main/java/io/cdap/plugin/common/http/HTTPPollConfig.java (version eaa26f9c5783a2b049a92adca2277e327a644517)
    • spark-plugins/src/main/java/io/cdap/plugin/spark/FileStreamingSource.java (version eaa26f9c5783a2b049a92adca2277e327a644517)
    • spark-plugins/src/main/java/io/cdap/plugin/spark/FileStreamingSourceUtil.java (version eaa26f9c5783a2b049a92adca2277e327a644517)
    • spark-plugins/src/main/java/io/cdap/plugin/spark/HTTPPollerSource.java (version eaa26f9c5783a2b049a92adca2277e327a644517)
    • spark-plugins/src/main/java/io/cdap/plugin/spark/HTTPPollerSourceUtil.java (version eaa26f9c5783a2b049a92adca2277e327a644517)
    • spark-plugins/src/main/java/io/cdap/plugin/spark/ReferenceStreamingSource.java (version eaa26f9c5783a2b049a92adca2277e327a644517)
    • spark-plugins/src/main/java/io/cdap/plugin/spark/TwitterStreamingSource.java (version eaa26f9c5783a2b049a92adca2277e327a644517)
    • spark-plugins/src/main/java/io/cdap/plugin/spark/TwitterStreamingSourceUtil.java (version eaa26f9c5783a2b049a92adca2277e327a644517)
    • spark-plugins/src/test/java/io/cdap/plugin/spark/test/SparkPluginTest.java (version eaa26f9c5783a2b049a92adca2277e327a644517)
  • Vinisha Shah <CuriousVini@users.noreply.github.com>

    Vinisha Shah <CuriousVini@users.noreply.github.com> 5ae24222cb627ddbe3bd5d94d1b438ef539cbfc1 m

    Merge pull request #947 from cdapio/feature/add-validation-format
    [CDAP-15787] Adding failure collector api for format plugins

  • Vinisha Shah

    Vinisha Shah c4fce8015ad45d0c830bf2a8b1807936c1ff6e52 m

    [CDAP-15787] Adding failure collector api for format plugins

    • core-plugins/src/main/java/io/cdap/plugin/batch/sink/FileSink.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • core-plugins/src/main/java/io/cdap/plugin/batch/sink/SnapshotFileBatchSink.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • core-plugins/src/main/java/io/cdap/plugin/batch/sink/TimePartitionedFileSetSink.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • core-plugins/src/main/java/io/cdap/plugin/batch/source/FTPBatchSource.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • core-plugins/src/main/java/io/cdap/plugin/batch/source/FileSourceConfig.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • core-plugins/src/main/java/io/cdap/plugin/batch/source/SnapshotFileBatchSource.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • core-plugins/src/main/java/io/cdap/plugin/batch/source/TimePartitionedFileSetSource.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • core-plugins/src/test/java/io/cdap/plugin/batch/ETLMapReduceTestRun.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • core-plugins/src/test/java/io/cdap/plugin/batch/source/FileBatchSourceTest.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-avro/pom.xml (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-avro/src/main/java/io/cdap/plugin/format/avro/input/AvroInputFormatProvider.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-avro/src/main/java/io/cdap/plugin/format/avro/output/AvroOutputFormatProvider.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-blob/pom.xml (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-blob/src/main/java/io/cdap/plugin/format/blob/input/BlobInputFormatProvider.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-common/src/main/java/io/cdap/plugin/format/input/PathTrackingConfig.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-common/src/main/java/io/cdap/plugin/format/input/PathTrackingInputFormatProvider.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-common/src/main/java/io/cdap/plugin/format/output/AbstractOutputFormatProvider.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-common/src/main/java/io/cdap/plugin/format/plugin/AbstractFileSink.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-common/src/main/java/io/cdap/plugin/format/plugin/AbstractFileSinkConfig.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-common/src/main/java/io/cdap/plugin/format/plugin/AbstractFileSource.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-common/src/main/java/io/cdap/plugin/format/plugin/AbstractFileSourceConfig.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-common/src/main/java/io/cdap/plugin/format/plugin/FileSinkProperties.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-common/src/main/java/io/cdap/plugin/format/plugin/FileSourceProperties.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-delimited/pom.xml (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-delimited/src/main/java/io/cdap/plugin/format/delimited/input/CSVInputFormatProvider.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-delimited/src/main/java/io/cdap/plugin/format/delimited/input/DelimitedInputFormatProvider.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-delimited/src/main/java/io/cdap/plugin/format/delimited/input/TSVInputFormatProvider.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-delimited/src/main/java/io/cdap/plugin/format/delimited/output/CSVOutputFormatProvider.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-delimited/src/main/java/io/cdap/plugin/format/delimited/output/DelimitedOutputFormatProvider.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-delimited/src/main/java/io/cdap/plugin/format/delimited/output/TSVOutputFormatProvider.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-json/pom.xml (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-json/src/main/java/io/cdap/plugin/format/json/input/JsonInputFormatProvider.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-json/src/main/java/io/cdap/plugin/format/json/output/JsonOutputFormatProvider.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-orc/pom.xml (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-orc/src/main/java/io/cdap/plugin/format/orc/output/OrcOutputFormatProvider.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-parquet/pom.xml (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-parquet/src/main/java/io/cdap/plugin/format/parquet/input/ParquetInputFormatProvider.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-parquet/src/main/java/io/cdap/plugin/format/parquet/output/ParquetOutputFormatProvider.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-text/pom.xml (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
    • format-text/src/main/java/io/cdap/plugin/format/text/input/TextInputFormatProvider.java (version c4fce8015ad45d0c830bf2a8b1807936c1ff6e52)
  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 0a81ae6c799f4f43f803de19e5aac8d68b8039fe m

    Merge pull request #941 from cdapio/feature/CDAP-15787-LogParserTransform
    [CDAP-15787] Added validation to LogParserTransform

  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 4bf79edd055e50250a22f9253ad8044fa26e59dc m

    [CDAP-15787] Changed schema to NonNullable if isNullable()

    • core-plugins/src/main/java/io/cdap/plugin/transform/LogParserTransform.java (version 4bf79edd055e50250a22f9253ad8044fa26e59dc)
  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> a52d661f4d11b13291c25c5fd932c1ab301f946d m

    [CDAP-15787] Included validation checks only in configurePipeline, improved the error messages, and added overloading validation function to throw an exception during runtime

    • core-plugins/src/main/java/io/cdap/plugin/transform/LogParserTransform.java (version a52d661f4d11b13291c25c5fd932c1ab301f946d)
  • Vinisha Shah <CuriousVini@users.noreply.github.com>

    Vinisha Shah <CuriousVini@users.noreply.github.com> ce7fb02828ceb255860d189a60a509dedd64e660 m

    Merge pull request #939 from cdapio/feature/add-validation-hbase
    [CDAP-15787] Add validation apis to hbase plugins

  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 2d60b159553e8bf7cb2210850d2928b8ffa79e37 m

    [CDAP-15787] Fixed validation failure causes to only display the configuration property when an input stage field is missing

    • core-plugins/src/main/java/io/cdap/plugin/transform/LogParserTransform.java (version 2d60b159553e8bf7cb2210850d2928b8ffa79e37)
    • core-plugins/src/test/java/io/cdap/plugin/transform/LogParserTransformTest.java (version 2d60b159553e8bf7cb2210850d2928b8ffa79e37)
  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 063ac3cb570b20ed357e23c2cf13c7aca5c9ccbd m

    [CDAP-15787] Added specific assertions for ValidationFailure causes, fixed logic to not continue to validate if input schema is not of type record

    • core-plugins/src/main/java/io/cdap/plugin/transform/LogParserTransform.java (version 063ac3cb570b20ed357e23c2cf13c7aca5c9ccbd)
    • core-plugins/src/test/java/io/cdap/plugin/transform/LogParserTransformTest.java (version 063ac3cb570b20ed357e23c2cf13c7aca5c9ccbd)
  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 197d46fb74b25fec3cc600409f3c371471fff503 m

    [CDAP-15787] Improved error messaging, changed tests to explicitly assert validation failure causes, changed FailureCollector parameters to be nonnullable

    • core-plugins/src/main/java/io/cdap/plugin/transform/LogParserTransform.java (version 197d46fb74b25fec3cc600409f3c371471fff503)
    • core-plugins/src/test/java/io/cdap/plugin/transform/LogParserTransformTest.java (version 197d46fb74b25fec3cc600409f3c371471fff503)
  • Vinisha Shah

    Vinisha Shah ff5b357f10acca5ed84c922b91748385f64d2a7b m

    [CDAP-15787] Add validation apis to hbase plugins

    • core-plugins/src/test/java/io/cdap/plugin/batch/sink/TableSinkTest.java (version ff5b357f10acca5ed84c922b91748385f64d2a7b)
    • hbase-plugins/src/main/java/io/cdap/plugin/HBaseConfig.java (version ff5b357f10acca5ed84c922b91748385f64d2a7b)
    • hbase-plugins/src/main/java/io/cdap/plugin/sink/HBaseSink.java (version ff5b357f10acca5ed84c922b91748385f64d2a7b)
    • hbase-plugins/src/main/java/io/cdap/plugin/source/HBaseSource.java (version ff5b357f10acca5ed84c922b91748385f64d2a7b)
    • hbase-plugins/src/test/java/io/cdap/plugin/HBaseSinkTest.java (version ff5b357f10acca5ed84c922b91748385f64d2a7b)
    • hydrator-common/src/main/java/io/cdap/plugin/common/SchemaValidator.java (version ff5b357f10acca5ed84c922b91748385f64d2a7b)
    • pom.xml (version ff5b357f10acca5ed84c922b91748385f64d2a7b)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 23dfd98f98ca5a501c33af269558ddaa9035e65c m

    Merge pull request #944 from cdapio/feature_release/CDAP-15629-fix-get-schema
    CDAP-15928 CDAP-15629 CDAP-15927 fix db source schema

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> a777a0035a8ad77ecfebf3fc1321857db144e6da m

    Merge pull request #945 from cdapio/bugfix_release/CDAP-15428-md5-validation
    CDAP-15428 fixed MD5 hasher plugin to check for field type

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 98891d89394cbf6b99c888c6f6817c5543f1199d m

    CDAP-15928 CDAP-15629 CDAP-15927 fix db source schema
    Removed the 'query' property that was only used in the previous
    get schema implementation, which is no longer used. Instead,
    use the import query to get the schema so that the schema
    is properly fetched during stage validation.

    This also fixes a bug where a pipeline could be deployed with an
    empty import query, which would fail at runtime.

    Also fixed a bug that would cause a null pointer exception when
    no password is given.

    Also fixed a bug in the query cleanup logic that strips the
    $CONDITIONS clause from the import query when fetching the schema.
    The previous logic was upper casing the entire query, which would
    cause errors in case senstive DBs like HyperSQL, which is used in
    unit tests. It also was not handling 'ors' in the where clauses.

    Also removed a bunch of warnings in unit tests and changed a unit
    test that was testing for runtime failures to instead test for
    deployment failure since a bad query now will fail at deployment
    instead of runtime.

    • database-plugins/src/main/java/io/cdap/plugin/ConnectionConfig.java (version 98891d89394cbf6b99c888c6f6817c5543f1199d)
    • database-plugins/src/main/java/io/cdap/plugin/db/batch/source/DBSource.java (version 98891d89394cbf6b99c888c6f6817c5543f1199d)
    • database-plugins/src/test/java/io/cdap/plugin/DatabasePluginTestBase.java (version 98891d89394cbf6b99c888c6f6817c5543f1199d)
    • database-plugins/src/test/java/io/cdap/plugin/DatabasePluginTestSuite.java (version 98891d89394cbf6b99c888c6f6817c5543f1199d)
    • database-plugins/src/test/java/io/cdap/plugin/db/batch/sink/DBSourceTestRun.java (version 98891d89394cbf6b99c888c6f6817c5543f1199d)
    • database-plugins/src/test/java/io/cdap/plugin/db/batch/source/ConditionsRemovalTest.java (version 98891d89394cbf6b99c888c6f6817c5543f1199d)
    • database-plugins/src/test/java/io/cdap/plugin/db/batch/source/DBSourceTestRun.java (version 98891d89394cbf6b99c888c6f6817c5543f1199d)
    • database-plugins/widgets/Database-batchsource.json (version 98891d89394cbf6b99c888c6f6817c5543f1199d)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> ab2ca8a141e7bfabe583d947a221da3f98252d95 m

    throw exception in preparerun

    • transform-plugins/src/main/java/io/cdap/plugin/Hasher.java (version ab2ca8a141e7bfabe583d947a221da3f98252d95)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 5946399fe79e645726acd5f9a015fd6f505bd1bf m

    CDAP-15428 fixed MD5 hasher plugin to check for field type
    Fixed the md5 hasher plugin to validate that the fields specified
    are strings and to fail if they are not.

    • transform-plugins/src/main/java/io/cdap/plugin/Hasher.java (version 5946399fe79e645726acd5f9a015fd6f505bd1bf)
  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> d4d48fb8645427878fece636b70eb0e50b9503ae m

    [CDAP-15787] Fixed unit tests and checkstyle errors for LogParserTransform

    • core-plugins/src/main/java/io/cdap/plugin/transform/LogParserTransform.java (version d4d48fb8645427878fece636b70eb0e50b9503ae)
    • core-plugins/src/test/java/io/cdap/plugin/transform/LogParserTransformTest.java (version d4d48fb8645427878fece636b70eb0e50b9503ae)
  • Dennis Li <23002167+dli357@users.noreply.github.com>

    Dennis Li <23002167+dli357@users.noreply.github.com> 97c677fc6426d65fbaf297d88318c36a22714602 m

    [CDAP-15787] Added validation to LogParserTransform

    • core-plugins/src/main/java/io/cdap/plugin/transform/LogParserTransform.java (version 97c677fc6426d65fbaf297d88318c36a22714602)
  • yaojiefeng <yaojie@google.com>

    yaojiefeng <yaojie@google.com> 94444c81ea215831a40d529b79f8104782497c48 m

    Merge pull request #937 from cdapio/update-submodules
    Update submodules

  • yaojie <yaojie@google.com>

    yaojie <yaojie@google.com> 2973a195e383f87c168de788d64b64ee77c85bc2 m

    git submodule update --init --recursive --remote

    • amazon-s3-plugins (version 2973a195e383f87c168de788d64b64ee77c85bc2)
    • condition-plugins (version 2973a195e383f87c168de788d64b64ee77c85bc2)
    • google-cloud (version 2973a195e383f87c168de788d64b64ee77c85bc2)
    • kafka-plugins (version 2973a195e383f87c168de788d64b64ee77c85bc2)
    • wrangler-transform (version 2973a195e383f87c168de788d64b64ee77c85bc2)
  • yaojie <yaojie@google.com>

    yaojie <yaojie@google.com> e379557314f336c7540357c187492767937381b7 m

    update submodule branches

    • .gitmodules (version e379557314f336c7540357c187492767937381b7)