Mini UI E2E bamboo build plan. This is to be used if there is an already existing coopr cluster

Build: #10 failed Custom build by Anudeep Katragadda with revision b529dcbfd83c93e2d960ff39e6a36ffe70f40a5a

Code commits

caskdata/cdap

  • itsanudeep <itsanudeep@google.com>

    itsanudeep <itsanudeep@google.com> b529dcbfd83c93e2d960ff39e6a36ffe70f40a5a

    new draft for pipeline

    • cdap-ui/cypress/fixtures/draft_for_upgrade.json (version b529dcbfd83c93e2d960ff39e6a36ffe70f40a5a)
    • cdap-ui/cypress/helpers.ts (version b529dcbfd83c93e2d960ff39e6a36ffe70f40a5a)
    • cdap-ui/cypress/integration/pipeline.upgrade.spec.ts (version b529dcbfd83c93e2d960ff39e6a36ffe70f40a5a)

caskdata/cdap-integration-tests

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> f9f9ced42f3367d303a4e6abbbfb34f90e02f2ad

    Merge pull request #1068 from cdapio/bugfix/dlp-test
    CDAP-16604 fix dlp test

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> b6c75e26b94094adc9fac51d9b4002ae61863078

    CDAP-16604 fix dlp test
    Fixed the dlp test, which was incorrectly counting an empty file
    as an output line of an empty string.

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/DLPTest.java (version b6c75e26b94094adc9fac51d9b4002ae61863078)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> df0455bf08744f5e8a0a3fd0af773906b1806ddb

    Merge pull request #1063 from cdapio/fix-integration-tests
    fix integration tests

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 7113f402e6e6dad7db9604dab1036d28f9976125

    fix integration tests
    Fix the google library dependencies to versions that all use
    compatible dependencies, and remove the direct gax-grpc depdendency
    that was causing Spanner to fail.

    Also use macros in the BQ tests to both test macros and to
    workaround an issue when the CDAP instance is running on a GCE
    VM.

    • integration-test-remote/pom.xml (version 7113f402e6e6dad7db9604dab1036d28f9976125)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/DLPTest.java (version 7113f402e6e6dad7db9604dab1036d28f9976125)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/GoogleBigQueryTest.java (version 7113f402e6e6dad7db9604dab1036d28f9976125)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/GoogleCloudSpannerTest.java (version 7113f402e6e6dad7db9604dab1036d28f9976125)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/hub/DLPTest.java (version 7113f402e6e6dad7db9604dab1036d28f9976125)
    • integration-test-remote/src/test/java/io/cdap/cdap/test/suite/GCPSuite.java (version 7113f402e6e6dad7db9604dab1036d28f9976125)
    • pom.xml (version 7113f402e6e6dad7db9604dab1036d28f9976125)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 4537dc463645dca1411f566f0c244bc2e339f0c2

    Merge pull request #1062 from cdapio/increase-cluster-size
    increase dataproc cluster size

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> f8e81e6da79311b5ac54541fda7a64459ee714e2

    increase dataproc cluster size
    This fixes an issue where some pipelines in some environments
    are stuck forever with MapReduce jobs that stay in pending state.

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/DataprocETLTestBase.java (version f8e81e6da79311b5ac54541fda7a64459ee714e2)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 12e20df6a676e58aff38d8f07b13d26552402367

    Merge pull request #1061 from cdapio/try-ordering-gcp-tests
    explicitly list tests in the GCP suite

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> c633d4ab031264f272889584a910f81d8d8b6bcd

    don't wait for status before starting

    • integration-test-remote/src/test/java/io/cdap/cdap/apps/fileset/FileSetTest.java (version c633d4ab031264f272889584a910f81d8d8b6bcd)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> e45bb8fd3dac07813a9466550a9cbca7f119a8e4

    Merge pull request #1060 from cdapio/increase-time-between-polls
    CDAP-16454 increase time between status polls

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 36ad42346cdb0f56bfa62de306c193f14c4d45d2

    explicitly list tests in the GCP suite
    Explicitly listing out tests in the GCP suite to control the
    order and see if some failures can be explained by ordering.

    • integration-test-remote/src/test/java/io/cdap/cdap/test/suite/GCPSuite.java (version 36ad42346cdb0f56bfa62de306c193f14c4d45d2)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 161f05fa633bc2d1277169ebd852864b7d4ffcc6

    CDAP-16454 increase time between status polls
    Increase the time between program status polls, since most programs
    take minutes to run and don't need to be polled every 50 milliseconds.
    This also greatly reduces the amount of logging, making it
    easier to investigate failures.

    • integration-test-core/src/main/java/io/cdap/cdap/test/AudiTestBase.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/KVTableWithProjectionTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/TPFSAvroSinkSourceTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/BatchAggregatorTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/BatchCubeSinkTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/BatchJoinerTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/ETLMapReduceTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/ExcelInputReaderTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/HivePluginTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/NormalizeTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/RowDenormalizerTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/TPFSParquetSinkSourceTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/ValueMapperTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/XMLParserTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/XMLReaderTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/DataprocETLTestBase.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/PubSubTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/realtime/DataStreamsTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/wrangler/WranglerServiceTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/wrangler/WranglerTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/fileset/PermissionTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/mapreduce/readless/ReadlessIncrementTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/restart/HangingWorkerTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/serviceworker/ServiceWorkerTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/apps/ApplicationTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/apps/dataset/DatasetTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/apps/fileset/FileSetTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/apps/fileset/PartitionCorrectorTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/apps/fileset/PartitionedFileSetUpdateTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/apps/metadata/ProgramMetadataStressTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/apps/metadata/ProgramMetadataTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/apps/report/ReportGenerationAppTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/apps/schedule/ScheduleTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/apps/spark/SparkPageRankAppTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/apps/workflow/WorkflowTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
    • integration-test-remote/src/test/java/io/cdap/cdap/security/AppImpersonationTest.java (version 161f05fa633bc2d1277169ebd852864b7d4ffcc6)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 82f085e14e04ac4db8a4afb3cdc4f73022f2ab30

    Merge pull request #1058 from cdapio/increase-dataproc-resources
    make dataproc worker resources configurable

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 7e18cc5d8ac7ddf118cb58b94cefb957bd68517a

    make dataproc worker resources configurable
    Add a way to modify the dataproc worker cpus and memory through
    passing in system properties. Also set the default to a higher
    value, since some pipelines seem to be stuck waiting for
    resources.

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/DataprocETLTestBase.java (version 7e18cc5d8ac7ddf118cb58b94cefb957bd68517a)
  • Venudhar Ravishankar <venu.ravishankar@gmail.com>

    Venudhar Ravishankar <venu.ravishankar@gmail.com> 878122bb62f29bfb707f45fc724a408545860256

    Merge pull request #1057 from cdapio/feature/CDAP-16439-dlp-integ-tests
    [CDAP-16439] Added cases for DLP plugins

  • Venudhar Ravishankar

    Venudhar Ravishankar 3895f38c4b2230c2ac8befdafff58303d39e5def

    [CDAP-16439] Added cases for DLP plugins

    • integration-test-remote/pom.xml (version 3895f38c4b2230c2ac8befdafff58303d39e5def)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/hub/DLPTest.java (version 3895f38c4b2230c2ac8befdafff58303d39e5def)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 7e19abdd205b4a52461c1e10822afff44b877b75

    Merge pull request #1056 from cdapio/merge-release-6-1
    Merge release 6 1

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> fec799373e5d2fb71cd7f6a42f1f777a0f409011

    Merge branch 'release/6.1' into merge-release-6-1

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 06894ffb553d2cf7ab4db0c3c9cc721e104af1c4

    Merge pull request #1055 from cdapio/bugfix/CDAP-16430-fix-hub-plugin-deploy
    CDAP-16430 fix deploy hub plugin

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 349b7ae6c543ae3133b8d01265cece473e834076

    CDAP-16430 fix deploy hub plugin
    Fix the deploy hub plugin method to use the public CDAP REST APIs
    instead of some UI endpoint that is subject to change and which is
    currently broken.

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/ETLTestBase.java (version 349b7ae6c543ae3133b8d01265cece473e834076)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/HivePluginTest.java (version 349b7ae6c543ae3133b8d01265cece473e834076)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/hub/PluginAttributes.java (version 349b7ae6c543ae3133b8d01265cece473e834076)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 9607e8dc83a7b1ded1c4bbde09b852b44a08ae15

    Merge pull request #1053 from cdapio/bugfix/CDAP-16429-fix-pubsub-stop-flakiness
    CDAP-16429 log a warning if pipeline cleanup fails

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> adcdd7d6d9c5f7bf43bf2999bd9ebb286da2d5fc

    CDAP-16429 log a warning if pipeline cleanup fails
    The pubsub test was flaky because it would occasionally get a 400
    error that the pipeline is not running when the stop call is made.
    Changed the test to just log a warning in this scenario since the
    test was actually successful. CDAP may have some underlying bugs
    causing this behavior though.

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/PubSubTest.java (version adcdd7d6d9c5f7bf43bf2999bd9ebb286da2d5fc)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 0b7ed9478d8767c981b843acc196ef6713defe19

    Merge pull request #1042 from yeweidaniel/integration-docs
    Update steps for running integration test

  • Daniel <danielye@google.com>

    Daniel <danielye@google.com> e0762675907ce2347b96144fac6e1d046cb2cafa

    Update steps for running integration test

    • README.rst (version e0762675907ce2347b96144fac6e1d046cb2cafa)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> fb4b714c33cda5cc5d6cd072984b69f25513595e

    Merge pull request #1038 from AntonGensitskiy/PLUGIN-82
    PLUGIN-82: Added new tests to cover dedupe functionality.

  • Anton Gensitskiy

    Anton Gensitskiy 5de3b01a9a700d48c09a5346561dac6a92bb7b93

    PLUGIN-82: Update testUpsertOperationWithDedupeSourceData() integration test.

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/GoogleBigQueryTest.java (version 5de3b01a9a700d48c09a5346561dac6a92bb7b93)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 410b5d84501df62f1cae139af8f31c8ceffbd1d7

    Merge pull request #1040 from cdapio/remove-metric-assertions-datastore
    remove datastore metric assertions

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> a53b770fdebd8ed111c3bd73ee1b9fa9ef737b2e

    remove datastore metric assertions
    Checking metrics is very unreliable due to race conditions. It can
    fail incorrectly because they are not polled for a long enough time,
    and they can pass incorrectly because the number being checked is
    too low.

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/GoogleCloudDatastoreTest.java (version a53b770fdebd8ed111c3bd73ee1b9fa9ef737b2e)
  • Anton Gensitskiy

    Anton Gensitskiy c29ae8b6d648f1965f131adeeff596a7bb9f8099

    PLUGIN-82: Added new tests to cover dedupe functionality.

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/GoogleBigQueryTest.java (version c29ae8b6d648f1965f131adeeff596a7bb9f8099)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> a8b593de1f6d3fa0ca846ec73293e1c632bd2a85

    Merge pull request #1037 from cdapio/disable-yarn-pmem-check
    disable yarn pmem check

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> c1bf76daab79bd501d86326f5bf6638418de5afe

    disable yarn pmem check
    CDAP services are getting killed by yarn due to physical memory
    constraints. Disable the pmem check to avoid this.

    • conf/cdap.json (version c1bf76daab79bd501d86326f5bf6638418de5afe)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 08dcee5bdfd9c1094ccb72ac5f7ed3b87b65df69

    Merge pull request #1036 from cdapio/CDAP-15804-gcp-test-macros
    CDAP-15804 use macros in tests for gcp plugins

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 18c0f67a3bbdda9103e03fe200bf09e448749f2d

    CDAP-15804 use macros in tests for gcp plugins
    Enhanced bigquery, datastore, and spanner tests to use macros.

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/DataprocETLTestBase.java (version 18c0f67a3bbdda9103e03fe200bf09e448749f2d)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/GoogleBigQueryTest.java (version 18c0f67a3bbdda9103e03fe200bf09e448749f2d)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/GoogleCloudDatastoreTest.java (version 18c0f67a3bbdda9103e03fe200bf09e448749f2d)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/GoogleCloudSpannerTest.java (version 18c0f67a3bbdda9103e03fe200bf09e448749f2d)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> a8599f005650c36465311faf53190ff3cafd5044

    Merge pull request #1035 from cdapio/remove-validator-test-6-1
    remove validator test

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> ecb9085c50b2fd15f3e1e130f6524049ce99bee0

    remove validator test
    the validator plugin was removed, so removing the test as well

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/ETLMapReduceTest.java (version ecb9085c50b2fd15f3e1e130f6524049ce99bee0)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 3f3ea4522aec02a8d4a00544d0dda999b8711a87

    Merge pull request #1034 from cdapio/remove-validator-test
    remove validator test

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 34ef57cd3140949c766895eb2fd3d8386c7acbfc

    remove validator test
    the validator plugin was removed, so removing the test as well

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/ETLMapReduceTest.java (version 34ef57cd3140949c766895eb2fd3d8386c7acbfc)
  • yaojiefeng <yaojie@google.com>

    yaojiefeng <yaojie@google.com> 813f6bdf4ce06672675ac5928300a2e599575bd7

    Merge pull request #1032 from cdapio/feature/bump-version
    bump version

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 04e85e4a550226ea682b06dd26ab741d01ef604a

    Merge pull request #1033 from cdapio/disable-yarn-mem-check
    disable yarn pmem check

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 14b7298b067d7cb958fbb9008ece8f09ef979a95

    disable yarn pmem check
    CDAP services are getting killed by yarn due to physical memory
    constraints. Disable the pmem check to avoid this.

    • conf/cdap.json (version 14b7298b067d7cb958fbb9008ece8f09ef979a95)
  • yaojie <yaojie@google.com>

    yaojie <yaojie@google.com> c2ece85e06ffc90c138406b6feecfded61a38753

    bump version

    • pom.xml (version c2ece85e06ffc90c138406b6feecfded61a38753)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> fcfec769e9c31b9bb146700e0c07e75c044ab4c4

    Merge pull request #1031 from cdapio/feature/CDAP-15879-gcs-formats-test
    CDAP-15879 test GCS formats

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> c5257419f917eac4a2f22b4d2fa7fe0bb0ce18ad

    CDAP-15879 test GCS formats
    Modified the existing GCS test to test that all the formats can
    be used, and to also test the usage of regex.

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/GCSTest.java (version c5257419f917eac4a2f22b4d2fa7fe0bb0ce18ad)
    • integration-test-remote/src/test/resources/customers.csv (version c5257419f917eac4a2f22b4d2fa7fe0bb0ce18ad)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 0ff7d962a4c4f954a2f17dbc4feb2a2277d137d1

    Merge pull request #1030 from cdapio/set-dataproc-network
    Add ability to configure the network used for tests

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 6ce027035f91712b6672c9c5402f9562304e5c30

    Add ability to configure the network used for tests

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/DataprocETLTestBase.java (version 6ce027035f91712b6672c9c5402f9562304e5c30)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> f7ffcfd0e88141efda2949d695c99aa50273dc2f

    Merge pull request #1029 from cdapio/fix-profile-creation
    Fix dataproc profile creation to only set region

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 47bc1c2e6da0a36091cf075e53a008a545e8d52a

    Fix dataproc profile creation to only set region

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/DataprocETLTestBase.java (version 47bc1c2e6da0a36091cf075e53a008a545e8d52a)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> a445544d05ec43b670535a2de2b522768fceebfe

    Merge pull request #1027 from cdapio/group-cloud-tests
    Create a GCP test suite and fix tests to set project for plugins

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> eb57a1c2b4ea3b4df3927327a5f786c0da56323e

    Create a GCP test suite and fix tests to set project for plugins
    Fixed the bigtable test to create an instance at the start of the
    tests instead of requiring that an instance is already setup.

    Added artifact info to all of the pipelines so that they are
    viewable through the UI.

    Fixed datetime checks for failing BigQuery test

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/DataprocETLTestBase.java (version eb57a1c2b4ea3b4df3927327a5f786c0da56323e)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/GCSTest.java (version eb57a1c2b4ea3b4df3927327a5f786c0da56323e)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/PubSubTest.java (version eb57a1c2b4ea3b4df3927327a5f786c0da56323e)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/GoogleBigQueryTest.java (version eb57a1c2b4ea3b4df3927327a5f786c0da56323e)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/GoogleCloudBigtableTest.java (version eb57a1c2b4ea3b4df3927327a5f786c0da56323e)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/GoogleCloudDatastoreTest.java (version eb57a1c2b4ea3b4df3927327a5f786c0da56323e)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/GoogleCloudSpannerTest.java (version eb57a1c2b4ea3b4df3927327a5f786c0da56323e)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/DataprocETLTestBase.java (version eb57a1c2b4ea3b4df3927327a5f786c0da56323e)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/GCSTest.java (version eb57a1c2b4ea3b4df3927327a5f786c0da56323e)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/GoogleBigQueryTest.java (version eb57a1c2b4ea3b4df3927327a5f786c0da56323e)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/GoogleCloudBigtableTest.java (version eb57a1c2b4ea3b4df3927327a5f786c0da56323e)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/GoogleCloudDatastoreTest.java (version eb57a1c2b4ea3b4df3927327a5f786c0da56323e)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/GoogleCloudSpannerTest.java (version eb57a1c2b4ea3b4df3927327a5f786c0da56323e)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/gcp/PubSubTest.java (version eb57a1c2b4ea3b4df3927327a5f786c0da56323e)
    • integration-test-remote/src/test/java/io/cdap/cdap/test/suite/GCPSuite.java (version eb57a1c2b4ea3b4df3927327a5f786c0da56323e)
  • Ali Anwar

    Ali Anwar b4f627d73d515f12b67dcacc3ebcb88c7dff7b8d

    Merge pull request #1028 from cdapio/increase-program-timeout
    Increase default program timeout to 120 seconds.

  • Ali Anwar

    Ali Anwar 576da880881c9bdc228414c054ae83fb11ead5ec

    Increase default program timeout to 120 seconds.

    • integration-test-core/src/main/java/io/cdap/cdap/test/AudiTestBase.java (version 576da880881c9bdc228414c054ae83fb11ead5ec)
  • dereklwood <derekwood@google.com>

    dereklwood <derekwood@google.com> 7ba6c68eb6e98648e47f576cf52e5587b1627d20

    Merge pull request #1026 from cdapio/feature/update-cdap-version
    update cdap version to trigger new node.js version requirement in coo…

  • Derek Wood <derekwood@google.com>

    Derek Wood <derekwood@google.com> 69b849cffff528f9b71c13996a7519d61f21b748

    update cdap version to trigger new node.js version requirement in cookbook

    • conf/cdap_sdk.json (version 69b849cffff528f9b71c13996a7519d61f21b748)
  • Ali Anwar

    Ali Anwar de70731ab45f116b5958e22d4f728aa6544e62dd

    Merge pull request #1025 from cdapio/fix-install-plugin-from-market
    Fix HivePluginTest (installing from market)

  • Ali <alianwar@google.com>

    Ali <alianwar@google.com> 85edfae1341d87b4ae3974d200a3f919b7b97e61

    This change is needed due to https://github.com/cdapio/cdap/pull/11546 and https://github.com/cdapio/cdap/pull/11568.

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/ETLTestBase.java (version 85edfae1341d87b4ae3974d200a3f919b7b97e61)
  • Terence Yim

    Terence Yim d9ba849b35ebf98f4f19755c337efcf611976c5f

    Added travis configuration

    • .travis.yml (version d9ba849b35ebf98f4f19755c337efcf611976c5f)
  • Ali Anwar

    Ali Anwar 2f42b3263ee929be292ddb25654afdea8445b772

    Merge pull request #1024 from cdapio/fix-checkstyle
    Fix checkstyle; ITN build is failing across all distros.

  • Ali <alianwar@google.com>

    Ali <alianwar@google.com> 33054223f3b4dc4d17c05f3ecd766e8e810195c2

    Fix checkstyle; ITN build is failing across all distros.

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/GoogleCloudSpannerTest.java (version 33054223f3b4dc4d17c05f3ecd766e8e810195c2)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 4f62faccad0ed4fff8d94ea2ae079baf7321184e

    Merge pull request #1022 from AntonGensitskiy/bug/CDAP-15771
    CDAP-15771: Added check for BigQuery bytes type.

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 687c86190f950bd4c46151bb1082fbe50aa18126

    Merge pull request #1023 from vladhlinskiy/feature/CDAP-15735_Added_tests_for_Array_in_Spanner_plugin2
    CDAP-15735 Support all data types in integration tests for Spanner

  • Vlad Hlinskiy <vglinskiy@cybervisiontech.com>

    Vlad Hlinskiy <vglinskiy@cybervisiontech.com> 0a777940dd4d4a123e4218adf8a917fe4e176467

    CDAP-15735 Support all data types in integration tests for Spanner, including arrays and null values.

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/GoogleCloudSpannerTest.java (version 0a777940dd4d4a123e4218adf8a917fe4e176467)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> e53fac8e1947c40d5ddfbf815e247e88d2d867ef

    Merge pull request #1021 from cdapio/revert-1020-feature/CDAP-15735_Added_tests_for_Array_in_Spanner_plugin
    Revert "CDAP-15735 Support all data types in integration tests for Spanner"

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 9bed0fc6d921b6b2ebd1ad6150d07cb047a0da6e

    Revert "CDAP-15735 Support all data types in integration tests for Spanner"

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/GoogleCloudSpannerTest.java (version 9bed0fc6d921b6b2ebd1ad6150d07cb047a0da6e)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 1613b2626be0a90f53c09112f981216083d2dbbb

    Merge pull request #1020 from vladhlinskiy/feature/CDAP-15735_Added_tests_for_Array_in_Spanner_plugin
    CDAP-15735 Support all data types in integration tests for Spanner

  • Vlad Hlinskiy <vglinskiy@cybervisiontech.com>

    Vlad Hlinskiy <vglinskiy@cybervisiontech.com> 1a0178db3cd282d76d2b479fe13e6d33aeeb191a

    CDAP-15735 Support all data types in integration tests for Spanner

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/GoogleCloudSpannerTest.java (version 1a0178db3cd282d76d2b479fe13e6d33aeeb191a)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 3327f829fe5e740dae1974ba15e73ffd71b219e8

    Merge pull request #1019 from vladhlinskiy/feature/CDAP-15731-GCS-Data-Types
    CDAP-15731 Support all data types in integration tests for GCS

  • Vlad Hlinskiy <vglinskiy@cybervisiontech.com>

    Vlad Hlinskiy <vglinskiy@cybervisiontech.com> eab4bf2691c547469a405cad58a990143a70613a

    CDAP-15731 Support all data types in integration tests for GCS - dynamically generate avro file.

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/GCSTest.java (version eab4bf2691c547469a405cad58a990143a70613a)
    • integration-test-remote/src/test/resources/gcs-types/schema.json (version eab4bf2691c547469a405cad58a990143a70613a)
    • integration-test-remote/src/test/resources/gcs-types/test.avro (version eab4bf2691c547469a405cad58a990143a70613a)
  • Anton Gensitskiy

    Anton Gensitskiy 6932d8410fae5d93a147d35ed8e2d2c7f918bd3b

    CDAP-15771: Added check for BigQuery bytes type.

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/GoogleBigQueryTest.java (version 6932d8410fae5d93a147d35ed8e2d2c7f918bd3b)
  • Vlad Hlinskiy <vglinskiy@cybervisiontech.com>

    Vlad Hlinskiy <vglinskiy@cybervisiontech.com> 2d390ce6d9826081be19a49eb70bf77f4997ad4b

    CDAP-15731 Support all data types in integration tests for GCS - remove JsonObject usage, move to streams, add missing checks.

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/GCSTest.java (version 2d390ce6d9826081be19a49eb70bf77f4997ad4b)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 8ae032f2667fb2237f1ce2f0f146b269405403fe

    Merge pull request #1016 from AntonGensitskiy/feature/CDAP-15644-bigquery-intergation-tests
    CDAP-15644: Added BigQuery integration tests.

  • Anton Gensitskiy

    Anton Gensitskiy 34aff56de2fd648a3480304acf2bd0ef16f8a62f

    CDAP-15644: Added BigQuery integration tests.

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/GoogleBigQueryTest.java (version 34aff56de2fd648a3480304acf2bd0ef16f8a62f)
    • pom.xml (version 34aff56de2fd648a3480304acf2bd0ef16f8a62f)
  • Vlad Hlinskiy <vglinskiy@cybervisiontech.com>

    Vlad Hlinskiy <vglinskiy@cybervisiontech.com> f64cf66c4f53df51b5c89102d0060a5f1f2a2ef3

    CDAP-15731 Support all data types in integration tests for GCS

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/GCSTest.java (version f64cf66c4f53df51b5c89102d0060a5f1f2a2ef3)
    • integration-test-remote/src/test/resources/gcs-types/schema.json (version f64cf66c4f53df51b5c89102d0060a5f1f2a2ef3)
    • integration-test-remote/src/test/resources/gcs-types/test.avro (version f64cf66c4f53df51b5c89102d0060a5f1f2a2ef3)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 66df4985a333b898fe437731043f5b6126e791cf

    Merge pull request #1018 from vladhlinskiy/CDAP-15756-fix-spanner-tests
    CDAP-15756 GoogleCloudSpannerTest broken due to pubsub dependency

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 856d521dfbcc84340fad4049b3b0daab65c959e9

    Merge pull request #1017 from mlozbin-cybervisiontech/feature/CDAP-15643-bigtable-integration-tests
    CDAP-15643 & CDAP-15734 Support all data types in BigTable Sink & Source integration tests

  • Maksim Lozbin

    Maksim Lozbin 7660894a5e925b992713254bc55f82f2d0e06ec6

    CDAP-15643 added tests for Bigtable plugin

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/DataprocETLTestBase.java (version 7660894a5e925b992713254bc55f82f2d0e06ec6)
    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/GoogleCloudBigtableTest.java (version 7660894a5e925b992713254bc55f82f2d0e06ec6)
    • pom.xml (version 7660894a5e925b992713254bc55f82f2d0e06ec6)
  • Vlad Hlinskiy <vglinskiy@cybervisiontech.com>

    Vlad Hlinskiy <vglinskiy@cybervisiontech.com> b45dd6a836552a0740a99748c09cb8dd978a6e70

    CDAP-15756 GoogleCloudSpannerTest broken due to pubsub dependency

    • pom.xml (version b45dd6a836552a0740a99748c09cb8dd978a6e70)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> b00db1cd8fa0cc1042d605d036badd23e00aeec2

    Merge pull request #1015 from vladhlinskiy/feature/CDAP-15642-Pubsub-integration-tests-coverage
    CDAP-15642 Pubsub integration tests coverage

  • Vlad Hlinskiy <vglinskiy@cybervisiontech.com>

    Vlad Hlinskiy <vglinskiy@cybervisiontech.com> 681305e0b408f320aef1e1bb3cdb5a2c9ed5e0ac

    CDAP-15642 Pubsub integration tests coverage

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/PubSubTest.java (version 681305e0b408f320aef1e1bb3cdb5a2c9ed5e0ac)
    • integration-test-remote/src/test/resources/logback-test.xml (version 681305e0b408f320aef1e1bb3cdb5a2c9ed5e0ac)
    • pom.xml (version 681305e0b408f320aef1e1bb3cdb5a2c9ed5e0ac)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 594f487f26b0424af376a87687edda32de49eba2

    Merge pull request #1014 from vladhlinskiy/feature/CDAP-15641-spanner-integration-tests
    CDAP-15641 Spanner integration tests

  • Vlad Hlinskiy <vglinskiy@cybervisiontech.com>

    Vlad Hlinskiy <vglinskiy@cybervisiontech.com> fa1a14c3799b50d72804c8116fccf712b38c50b0

    CDAP-15641 Spanner integration tests created

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/batch/GoogleCloudSpannerTest.java (version fa1a14c3799b50d72804c8116fccf712b38c50b0)
    • pom.xml (version fa1a14c3799b50d72804c8116fccf712b38c50b0)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> a0220e91873d5dd80f9982c779e53e971270c3fc

    Merge pull request #1013 from vladhlinskiy/fetaure/CDAP-15640-GCS-integration-tests-coverage
    CDAP-15640 GCS integration tests coverage

  • Vlad Hlinskiy <vglinskiy@cybervisiontech.com>

    Vlad Hlinskiy <vglinskiy@cybervisiontech.com> ccce58e8b4de9183e4f2d5d59f88eddc373f6cdd

    CDAP-15640 GCS integration tests coverage

    • integration-test-remote/src/test/java/io/cdap/cdap/app/etl/GCSTest.java (version ccce58e8b4de9183e4f2d5d59f88eddc373f6cdd)