albertshau <ashau@google.com>: Author Summary

Builds triggered by albertshau <ashau@google.com>

Builds triggered by an author are those builds which contains changes committed by the author.
1123
333 (30%)
790 (70%)

Breakages and fixes

Broken means the build has failed but the previous build was successful.
Fixed means that the build was successful but the previous build has failed.
99 (9% of all builds triggered)
99 (9% of all builds triggered)
0
Build Completed Code commits Tests
HYP › BAD › #291 1 day ago
Merge pull request #1068 from cdapio/update-submodules
update submodules
update submodules
861 passed
IT › UIE2EMINI › #9 5 days ago
Merge pull request #1060 from cdapio/increase-time-between-polls
CDAP-16454 increase time between status polls
Merge pull request #1013 from vladhlinskiy/fetaure/CDAP-15640-GCS-integration-tests-coverage
CDAP-15640 GCS integration tests coverage
CDAP-16429 log a warning if pipeline cleanup fails
The pubsub test was flaky because it would occasionally get a 400
error that the pipeline is not running when the stop call is made.
Changed the test to just log a warning in this scenario since the
test was actually successful. CDAP may have some underlying bugs
causing this behavior though.
Merge pull request #1033 from cdapio/disable-yarn-mem-check
disable yarn pmem check
Revert "CDAP-15735 Support all data types in integration tests for Spanner"
Merge pull request #1053 from cdapio/bugfix/CDAP-16429-fix-pubsub-stop-flakiness
CDAP-16429 log a warning if pipeline cleanup fails
Merge branch 'release/6.1' into merge-release-6-1
remove validator test
the validator plugin was removed, so removing the test as well
explicitly list tests in the GCP suite
Explicitly listing out tests in the GCP suite to control the
order and see if some failures can be explained by ordering.
Merge pull request #1031 from cdapio/feature/CDAP-15879-gcs-formats-test
CDAP-15879 test GCS formats
Merge pull request #1040 from cdapio/remove-metric-assertions-datastore
remove datastore metric assertions
don't wait for status before starting
Merge pull request #1016 from AntonGensitskiy/feature/CDAP-15644-bigquery-intergation-tests
CDAP-15644: Added BigQuery integration tests.
Merge pull request #1042 from yeweidaniel/integration-docs
Update steps for running integration test
Merge pull request #1037 from cdapio/disable-yarn-pmem-check
disable yarn pmem check
disable yarn pmem check
CDAP services are getting killed by yarn due to physical memory
constraints. Disable the pmem check to avoid this.
CDAP-15879 test GCS formats
Modified the existing GCS test to test that all the formats can
be used, and to also test the usage of regex.
Create a GCP test suite and fix tests to set project for plugins
Fixed the bigtable test to create an instance at the start of the
tests instead of requiring that an instance is already setup.

Added artifact info to all of the pipelines so that they are
viewable through the UI.

Fixed datetime checks for failing BigQuery test
Merge pull request #1019 from vladhlinskiy/feature/CDAP-15731-GCS-Data-Types
CDAP-15731 Support all data types in integration tests for GCS
Merge pull request #1023 from vladhlinskiy/feature/CDAP-15735_Added_tests_for_Array_in_Spanner_plugin2
CDAP-15735 Support all data types in integration tests for Spanner
remove validator test
the validator plugin was removed, so removing the test as well
Merge pull request #1029 from cdapio/fix-profile-creation
Fix dataproc profile creation to only set region
CDAP-16454 increase time between status polls
Increase the time between program status polls, since most programs
take minutes to run and don't need to be polled every 50 milliseconds.
This also greatly reduces the amount of logging, making it
easier to investigate failures.
Merge pull request #1058 from cdapio/increase-dataproc-resources
make dataproc worker resources configurable
Merge pull request #1020 from vladhlinskiy/feature/CDAP-15735_Added_tests_for_Array_in_Spanner_plugin
CDAP-15735 Support all data types in integration tests for Spanner
remove datastore metric assertions
Checking metrics is very unreliable due to race conditions. It can
fail incorrectly because they are not polled for a long enough time,
and they can pass incorrectly because the number being checked is
too low.
Merge pull request #1030 from cdapio/set-dataproc-network
Add ability to configure the network used for tests
Merge pull request #1038 from AntonGensitskiy/PLUGIN-82
PLUGIN-82: Added new tests to cover dedupe functionality.
Merge pull request #1021 from cdapio/revert-1020-feature/CDAP-15735_Added_tests_for_Array_in_Spanner_plugin
Revert "CDAP-15735 Support all data types in integration tests for Spanner"
Merge pull request #1014 from vladhlinskiy/feature/CDAP-15641-spanner-integration-tests
CDAP-15641 Spanner integration tests
Merge pull request #1015 from vladhlinskiy/feature/CDAP-15642-Pubsub-integration-tests-coverage
CDAP-15642 Pubsub integration tests coverage
Merge pull request #1017 from mlozbin-cybervisiontech/feature/CDAP-15643-bigtable-integration-tests
CDAP-15643 & CDAP-15734 Support all data types in BigTable Sink & Source integration tests
Merge pull request #1035 from cdapio/remove-validator-test-6-1
remove validator test
Add ability to configure the network used for tests
CDAP-16430 fix deploy hub plugin
Fix the deploy hub plugin method to use the public CDAP REST APIs
instead of some UI endpoint that is subject to change and which is
currently broken.
make dataproc worker resources configurable
Add a way to modify the dataproc worker cpus and memory through
passing in system properties. Also set the default to a higher
value, since some pipelines seem to be stuck waiting for
resources.
Merge pull request #1063 from cdapio/fix-integration-tests
fix integration tests
increase dataproc cluster size
This fixes an issue where some pipelines in some environments
are stuck forever with MapReduce jobs that stay in pending state.
disable yarn pmem check
CDAP services are getting killed by yarn due to physical memory
constraints. Disable the pmem check to avoid this.
Merge pull request #1022 from AntonGensitskiy/bug/CDAP-15771
CDAP-15771: Added check for BigQuery bytes type.
Merge pull request #1055 from cdapio/bugfix/CDAP-16430-fix-hub-plugin-deploy
CDAP-16430 fix deploy hub plugin
Fix dataproc profile creation to only set region
fix integration tests
Fix the google library dependencies to versions that all use
compatible dependencies, and remove the direct gax-grpc depdendency
that was causing Spanner to fail.

Also use macros in the BQ tests to both test macros and to
workaround an issue when the CDAP instance is running on a GCE
VM.
Merge pull request #1034 from cdapio/remove-validator-test
remove validator test
Merge pull request #1018 from vladhlinskiy/CDAP-15756-fix-spanner-tests
CDAP-15756 GoogleCloudSpannerTest broken due to pubsub dependency
CDAP-15804 use macros in tests for gcp plugins
Enhanced bigquery, datastore, and spanner tests to use macros.
Merge pull request #1056 from cdapio/merge-release-6-1
Merge release 6 1
Merge pull request #1062 from cdapio/increase-cluster-size
increase dataproc cluster size
Merge pull request #1036 from cdapio/CDAP-15804-gcp-test-macros
CDAP-15804 use macros in tests for gcp plugins
Merge pull request #1061 from cdapio/try-ordering-gcp-tests
explicitly list tests in the GCP suite
Merge pull request #1027 from cdapio/group-cloud-tests
Create a GCP test suite and fix tests to set project for plugins
Testless build
IT › UIE2EMINI › #8 5 days ago
Merge pull request #1023 from vladhlinskiy/feature/CDAP-15735_Added_tests_for_Array_in_Spanner_plugin2
CDAP-15735 Support all data types in integration tests for Spanner
remove validator test
the validator plugin was removed, so removing the test as well
Merge pull request #1056 from cdapio/merge-release-6-1
Merge release 6 1
Merge pull request #1019 from vladhlinskiy/feature/CDAP-15731-GCS-Data-Types
CDAP-15731 Support all data types in integration tests for GCS
Merge pull request #1063 from cdapio/fix-integration-tests
fix integration tests
make dataproc worker resources configurable
Add a way to modify the dataproc worker cpus and memory through
passing in system properties. Also set the default to a higher
value, since some pipelines seem to be stuck waiting for
resources.
disable yarn pmem check
CDAP services are getting killed by yarn due to physical memory
constraints. Disable the pmem check to avoid this.
Merge pull request #1061 from cdapio/try-ordering-gcp-tests
explicitly list tests in the GCP suite
CDAP-15879 test GCS formats
Modified the existing GCS test to test that all the formats can
be used, and to also test the usage of regex.
Merge pull request #1017 from mlozbin-cybervisiontech/feature/CDAP-15643-bigtable-integration-tests
CDAP-15643 & CDAP-15734 Support all data types in BigTable Sink & Source integration tests
Merge pull request #1036 from cdapio/CDAP-15804-gcp-test-macros
CDAP-15804 use macros in tests for gcp plugins
Merge branch 'release/6.1' into merge-release-6-1
CDAP-15804 use macros in tests for gcp plugins
Enhanced bigquery, datastore, and spanner tests to use macros.
Revert "CDAP-15735 Support all data types in integration tests for Spanner"
Merge pull request #1042 from yeweidaniel/integration-docs
Update steps for running integration test
Merge pull request #1037 from cdapio/disable-yarn-pmem-check
disable yarn pmem check
remove datastore metric assertions
Checking metrics is very unreliable due to race conditions. It can
fail incorrectly because they are not polled for a long enough time,
and they can pass incorrectly because the number being checked is
too low.
Merge pull request #1033 from cdapio/disable-yarn-mem-check
disable yarn pmem check
Merge pull request #1016 from AntonGensitskiy/feature/CDAP-15644-bigquery-intergation-tests
CDAP-15644: Added BigQuery integration tests.
CDAP-16454 increase time between status polls
Increase the time between program status polls, since most programs
take minutes to run and don't need to be polled every 50 milliseconds.
This also greatly reduces the amount of logging, making it
easier to investigate failures.
Merge pull request #1015 from vladhlinskiy/feature/CDAP-15642-Pubsub-integration-tests-coverage
CDAP-15642 Pubsub integration tests coverage
Merge pull request #1021 from cdapio/revert-1020-feature/CDAP-15735_Added_tests_for_Array_in_Spanner_plugin
Revert "CDAP-15735 Support all data types in integration tests for Spanner"
Merge pull request #1060 from cdapio/increase-time-between-polls
CDAP-16454 increase time between status polls
Merge pull request #1013 from vladhlinskiy/fetaure/CDAP-15640-GCS-integration-tests-coverage
CDAP-15640 GCS integration tests coverage
Merge pull request #1040 from cdapio/remove-metric-assertions-datastore
remove datastore metric assertions
Merge pull request #1030 from cdapio/set-dataproc-network
Add ability to configure the network used for tests
Merge pull request #1031 from cdapio/feature/CDAP-15879-gcs-formats-test
CDAP-15879 test GCS formats
Create a GCP test suite and fix tests to set project for plugins
Fixed the bigtable test to create an instance at the start of the
tests instead of requiring that an instance is already setup.

Added artifact info to all of the pipelines so that they are
viewable through the UI.

Fixed datetime checks for failing BigQuery test
Merge pull request #1027 from cdapio/group-cloud-tests
Create a GCP test suite and fix tests to set project for plugins
disable yarn pmem check
CDAP services are getting killed by yarn due to physical memory
constraints. Disable the pmem check to avoid this.
Merge pull request #1014 from vladhlinskiy/feature/CDAP-15641-spanner-integration-tests
CDAP-15641 Spanner integration tests
Merge pull request #1038 from AntonGensitskiy/PLUGIN-82
PLUGIN-82: Added new tests to cover dedupe functionality.
Merge pull request #1020 from vladhlinskiy/feature/CDAP-15735_Added_tests_for_Array_in_Spanner_plugin
CDAP-15735 Support all data types in integration tests for Spanner
Merge pull request #1029 from cdapio/fix-profile-creation
Fix dataproc profile creation to only set region
remove validator test
the validator plugin was removed, so removing the test as well
Merge pull request #1035 from cdapio/remove-validator-test-6-1
remove validator test
Merge pull request #1058 from cdapio/increase-dataproc-resources
make dataproc worker resources configurable
don't wait for status before starting
Merge pull request #1018 from vladhlinskiy/CDAP-15756-fix-spanner-tests
CDAP-15756 GoogleCloudSpannerTest broken due to pubsub dependency
Merge pull request #1062 from cdapio/increase-cluster-size
increase dataproc cluster size
CDAP-16429 log a warning if pipeline cleanup fails
The pubsub test was flaky because it would occasionally get a 400
error that the pipeline is not running when the stop call is made.
Changed the test to just log a warning in this scenario since the
test was actually successful. CDAP may have some underlying bugs
causing this behavior though.
Fix dataproc profile creation to only set region
Merge pull request #1034 from cdapio/remove-validator-test
remove validator test
increase dataproc cluster size
This fixes an issue where some pipelines in some environments
are stuck forever with MapReduce jobs that stay in pending state.
explicitly list tests in the GCP suite
Explicitly listing out tests in the GCP suite to control the
order and see if some failures can be explained by ordering.
Merge pull request #1053 from cdapio/bugfix/CDAP-16429-fix-pubsub-stop-flakiness
CDAP-16429 log a warning if pipeline cleanup fails
Add ability to configure the network used for tests
CDAP-16430 fix deploy hub plugin
Fix the deploy hub plugin method to use the public CDAP REST APIs
instead of some UI endpoint that is subject to change and which is
currently broken.
fix integration tests
Fix the google library dependencies to versions that all use
compatible dependencies, and remove the direct gax-grpc depdendency
that was causing Spanner to fail.

Also use macros in the BQ tests to both test macros and to
workaround an issue when the CDAP instance is running on a GCE
VM.
Merge pull request #1022 from AntonGensitskiy/bug/CDAP-15771
CDAP-15771: Added check for BigQuery bytes type.
Merge pull request #1055 from cdapio/bugfix/CDAP-16430-fix-hub-plugin-deploy
CDAP-16430 fix deploy hub plugin
Testless build
IT › UIE2EMINI › #7 5 days ago
make dataproc worker resources configurable
Add a way to modify the dataproc worker cpus and memory through
passing in system properties. Also set the default to a higher
value, since some pipelines seem to be stuck waiting for
resources.
Merge pull request #1061 from cdapio/try-ordering-gcp-tests
explicitly list tests in the GCP suite
Merge pull request #1063 from cdapio/fix-integration-tests
fix integration tests
Add ability to configure the network used for tests
CDAP-15879 test GCS formats
Modified the existing GCS test to test that all the formats can
be used, and to also test the usage of regex.
Merge pull request #1017 from mlozbin-cybervisiontech/feature/CDAP-15643-bigtable-integration-tests
CDAP-15643 & CDAP-15734 Support all data types in BigTable Sink & Source integration tests
fix integration tests
Fix the google library dependencies to versions that all use
compatible dependencies, and remove the direct gax-grpc depdendency
that was causing Spanner to fail.

Also use macros in the BQ tests to both test macros and to
workaround an issue when the CDAP instance is running on a GCE
VM.
Merge pull request #1022 from AntonGensitskiy/bug/CDAP-15771
CDAP-15771: Added check for BigQuery bytes type.
disable yarn pmem check
CDAP services are getting killed by yarn due to physical memory
constraints. Disable the pmem check to avoid this.
Merge pull request #1040 from cdapio/remove-metric-assertions-datastore
remove datastore metric assertions
Merge pull request #1031 from cdapio/feature/CDAP-15879-gcs-formats-test
CDAP-15879 test GCS formats
CDAP-15804 use macros in tests for gcp plugins
Enhanced bigquery, datastore, and spanner tests to use macros.
Merge pull request #1037 from cdapio/disable-yarn-pmem-check
disable yarn pmem check
Merge pull request #1018 from vladhlinskiy/CDAP-15756-fix-spanner-tests
CDAP-15756 GoogleCloudSpannerTest broken due to pubsub dependency
Merge pull request #1056 from cdapio/merge-release-6-1
Merge release 6 1
Merge pull request #1023 from vladhlinskiy/feature/CDAP-15735_Added_tests_for_Array_in_Spanner_plugin2
CDAP-15735 Support all data types in integration tests for Spanner
Merge pull request #1036 from cdapio/CDAP-15804-gcp-test-macros
CDAP-15804 use macros in tests for gcp plugins
Merge pull request #1062 from cdapio/increase-cluster-size
increase dataproc cluster size
Merge pull request #1016 from AntonGensitskiy/feature/CDAP-15644-bigquery-intergation-tests
CDAP-15644: Added BigQuery integration tests.
Merge pull request #1020 from vladhlinskiy/feature/CDAP-15735_Added_tests_for_Array_in_Spanner_plugin
CDAP-15735 Support all data types in integration tests for Spanner
Merge pull request #1013 from vladhlinskiy/fetaure/CDAP-15640-GCS-integration-tests-coverage
CDAP-15640 GCS integration tests coverage
remove validator test
the validator plugin was removed, so removing the test as well
CDAP-16454 increase time between status polls
Increase the time between program status polls, since most programs
take minutes to run and don't need to be polled every 50 milliseconds.
This also greatly reduces the amount of logging, making it
easier to investigate failures.
Merge pull request #1060 from cdapio/increase-time-between-polls
CDAP-16454 increase time between status polls
remove validator test
the validator plugin was removed, so removing the test as well
Merge pull request #1030 from cdapio/set-dataproc-network
Add ability to configure the network used for tests
Revert "CDAP-15735 Support all data types in integration tests for Spanner"
remove datastore metric assertions
Checking metrics is very unreliable due to race conditions. It can
fail incorrectly because they are not polled for a long enough time,
and they can pass incorrectly because the number being checked is
too low.
Merge pull request #1021 from cdapio/revert-1020-feature/CDAP-15735_Added_tests_for_Array_in_Spanner_plugin
Revert "CDAP-15735 Support all data types in integration tests for Spanner"
Merge pull request #1014 from vladhlinskiy/feature/CDAP-15641-spanner-integration-tests
CDAP-15641 Spanner integration tests
Merge pull request #1015 from vladhlinskiy/feature/CDAP-15642-Pubsub-integration-tests-coverage
CDAP-15642 Pubsub integration tests coverage
Merge pull request #1035 from cdapio/remove-validator-test-6-1
remove validator test
CDAP-16430 fix deploy hub plugin
Fix the deploy hub plugin method to use the public CDAP REST APIs
instead of some UI endpoint that is subject to change and which is
currently broken.
Merge pull request #1034 from cdapio/remove-validator-test
remove validator test
increase dataproc cluster size
This fixes an issue where some pipelines in some environments
are stuck forever with MapReduce jobs that stay in pending state.
Fix dataproc profile creation to only set region
Merge pull request #1055 from cdapio/bugfix/CDAP-16430-fix-hub-plugin-deploy
CDAP-16430 fix deploy hub plugin
Merge pull request #1042 from yeweidaniel/integration-docs
Update steps for running integration test
Create a GCP test suite and fix tests to set project for plugins
Fixed the bigtable test to create an instance at the start of the
tests instead of requiring that an instance is already setup.

Added artifact info to all of the pipelines so that they are
viewable through the UI.

Fixed datetime checks for failing BigQuery test
Merge pull request #1027 from cdapio/group-cloud-tests
Create a GCP test suite and fix tests to set project for plugins
Merge pull request #1033 from cdapio/disable-yarn-mem-check
disable yarn pmem check
disable yarn pmem check
CDAP services are getting killed by yarn due to physical memory
constraints. Disable the pmem check to avoid this.
CDAP-16429 log a warning if pipeline cleanup fails
The pubsub test was flaky because it would occasionally get a 400
error that the pipeline is not running when the stop call is made.
Changed the test to just log a warning in this scenario since the
test was actually successful. CDAP may have some underlying bugs
causing this behavior though.
Merge pull request #1019 from vladhlinskiy/feature/CDAP-15731-GCS-Data-Types
CDAP-15731 Support all data types in integration tests for GCS
Merge pull request #1029 from cdapio/fix-profile-creation
Fix dataproc profile creation to only set region
Merge pull request #1053 from cdapio/bugfix/CDAP-16429-fix-pubsub-stop-flakiness
CDAP-16429 log a warning if pipeline cleanup fails
Merge pull request #1058 from cdapio/increase-dataproc-resources
make dataproc worker resources configurable
Merge branch 'release/6.1' into merge-release-6-1
don't wait for status before starting
explicitly list tests in the GCP suite
Explicitly listing out tests in the GCP suite to control the
order and see if some failures can be explained by ordering.
Merge pull request #1038 from AntonGensitskiy/PLUGIN-82
PLUGIN-82: Added new tests to cover dedupe functionality.
Testless build
IT › ITN › #347 1 week ago
fix integration tests
Fix the google library dependencies to versions that all use
compatible dependencies, and remove the direct gax-grpc depdendency
that was causing Spanner to fail.

Also use macros in the BQ tests to both test macros and to
workaround an issue when the CDAP instance is running on a GCE
VM.
Merge pull request #1063 from cdapio/fix-integration-tests
fix integration tests
674 passed
IT › SAN › #247 1 week ago
Merge pull request #1062 from cdapio/increase-cluster-size
increase dataproc cluster size
Merge pull request #1063 from cdapio/fix-integration-tests
fix integration tests
increase dataproc cluster size
This fixes an issue where some pipelines in some environments
are stuck forever with MapReduce jobs that stay in pending state.
fix integration tests
Fix the google library dependencies to versions that all use
compatible dependencies, and remove the direct gax-grpc depdendency
that was causing Spanner to fail.

Also use macros in the BQ tests to both test macros and to
workaround an issue when the CDAP instance is running on a GCE
VM.
42 passed
IT › DC › #279 1 week ago
Merge pull request #1062 from cdapio/increase-cluster-size
increase dataproc cluster size
fix integration tests
Fix the google library dependencies to versions that all use
compatible dependencies, and remove the direct gax-grpc depdendency
that was causing Spanner to fail.

Also use macros in the BQ tests to both test macros and to
workaround an issue when the CDAP instance is running on a GCE
VM.
Merge pull request #1063 from cdapio/fix-integration-tests
fix integration tests
increase dataproc cluster size
This fixes an issue where some pipelines in some environments
are stuck forever with MapReduce jobs that stay in pending state.
42 passed
IT › UPD2 › #468 1 week ago
fix integration tests
Fix the google library dependencies to versions that all use
compatible dependencies, and remove the direct gax-grpc depdendency
that was causing Spanner to fail.

Also use macros in the BQ tests to both test macros and to
workaround an issue when the CDAP instance is running on a GCE
VM.
Merge pull request #1063 from cdapio/fix-integration-tests
fix integration tests
Testless build
IT › IIT › #688 1 week ago
fix integration tests
Fix the google library dependencies to versions that all use
compatible dependencies, and remove the direct gax-grpc depdendency
that was causing Spanner to fail.

Also use macros in the BQ tests to both test macros and to
workaround an issue when the CDAP instance is running on a GCE
VM.
Merge pull request #1063 from cdapio/fix-integration-tests
fix integration tests
46 passed
IT › CLD › #339 1 week ago
Merge pull request #1063 from cdapio/fix-integration-tests
fix integration tests
fix integration tests
Fix the google library dependencies to versions that all use
compatible dependencies, and remove the direct gax-grpc depdendency
that was causing Spanner to fail.

Also use macros in the BQ tests to both test macros and to
workaround an issue when the CDAP instance is running on a GCE
VM.
3 of 30 failed
Build Completed Code commits Tests
IT › IIT › #687 1 week ago
Merge pull request #1062 from cdapio/increase-cluster-size
increase dataproc cluster size
increase dataproc cluster size
This fixes an issue where some pipelines in some environments
are stuck forever with MapReduce jobs that stay in pending state.
1 of 46 failed
IT › SAN › #246 1 week ago
Merge pull request #1061 from cdapio/try-ordering-gcp-tests
explicitly list tests in the GCP suite
explicitly list tests in the GCP suite
Explicitly listing out tests in the GCP suite to control the
order and see if some failures can be explained by ordering.
1 of 42 failed
IT › IIT › #682 2 weeks ago
CDAP-16454 increase time between status polls
Increase the time between program status polls, since most programs
take minutes to run and don't need to be polled every 50 milliseconds.
This also greatly reduces the amount of logging, making it
easier to investigate failures.
Merge pull request #1060 from cdapio/increase-time-between-polls
CDAP-16454 increase time between status polls
2 of 46 failed
You have insufficient permissions to see all of the builds.
Build Completed Code commits Tests
IT › UIE2EMINI › #9 5 days ago
Merge pull request #1060 from cdapio/increase-time-between-polls
CDAP-16454 increase time between status polls
Merge pull request #1013 from vladhlinskiy/fetaure/CDAP-15640-GCS-integration-tests-coverage
CDAP-15640 GCS integration tests coverage
CDAP-16429 log a warning if pipeline cleanup fails
The pubsub test was flaky because it would occasionally get a 400
error that the pipeline is not running when the stop call is made.
Changed the test to just log a warning in this scenario since the
test was actually successful. CDAP may have some underlying bugs
causing this behavior though.
Merge pull request #1033 from cdapio/disable-yarn-mem-check
disable yarn pmem check
Revert "CDAP-15735 Support all data types in integration tests for Spanner"
Merge pull request #1053 from cdapio/bugfix/CDAP-16429-fix-pubsub-stop-flakiness
CDAP-16429 log a warning if pipeline cleanup fails
Merge branch 'release/6.1' into merge-release-6-1
remove validator test
the validator plugin was removed, so removing the test as well
explicitly list tests in the GCP suite
Explicitly listing out tests in the GCP suite to control the
order and see if some failures can be explained by ordering.
Merge pull request #1031 from cdapio/feature/CDAP-15879-gcs-formats-test
CDAP-15879 test GCS formats
Merge pull request #1040 from cdapio/remove-metric-assertions-datastore
remove datastore metric assertions
don't wait for status before starting
Merge pull request #1016 from AntonGensitskiy/feature/CDAP-15644-bigquery-intergation-tests
CDAP-15644: Added BigQuery integration tests.
Merge pull request #1042 from yeweidaniel/integration-docs
Update steps for running integration test
Merge pull request #1037 from cdapio/disable-yarn-pmem-check
disable yarn pmem check
disable yarn pmem check
CDAP services are getting killed by yarn due to physical memory
constraints. Disable the pmem check to avoid this.
CDAP-15879 test GCS formats
Modified the existing GCS test to test that all the formats can
be used, and to also test the usage of regex.
Create a GCP test suite and fix tests to set project for plugins
Fixed the bigtable test to create an instance at the start of the
tests instead of requiring that an instance is already setup.

Added artifact info to all of the pipelines so that they are
viewable through the UI.

Fixed datetime checks for failing BigQuery test
Merge pull request #1019 from vladhlinskiy/feature/CDAP-15731-GCS-Data-Types
CDAP-15731 Support all data types in integration tests for GCS
Merge pull request #1023 from vladhlinskiy/feature/CDAP-15735_Added_tests_for_Array_in_Spanner_plugin2
CDAP-15735 Support all data types in integration tests for Spanner
remove validator test
the validator plugin was removed, so removing the test as well
Merge pull request #1029 from cdapio/fix-profile-creation
Fix dataproc profile creation to only set region
CDAP-16454 increase time between status polls
Increase the time between program status polls, since most programs
take minutes to run and don't need to be polled every 50 milliseconds.
This also greatly reduces the amount of logging, making it
easier to investigate failures.
Merge pull request #1058 from cdapio/increase-dataproc-resources
make dataproc worker resources configurable
Merge pull request #1020 from vladhlinskiy/feature/CDAP-15735_Added_tests_for_Array_in_Spanner_plugin
CDAP-15735 Support all data types in integration tests for Spanner
remove datastore metric assertions
Checking metrics is very unreliable due to race conditions. It can
fail incorrectly because they are not polled for a long enough time,
and they can pass incorrectly because the number being checked is
too low.
Merge pull request #1030 from cdapio/set-dataproc-network
Add ability to configure the network used for tests
Merge pull request #1038 from AntonGensitskiy/PLUGIN-82
PLUGIN-82: Added new tests to cover dedupe functionality.
Merge pull request #1021 from cdapio/revert-1020-feature/CDAP-15735_Added_tests_for_Array_in_Spanner_plugin
Revert "CDAP-15735 Support all data types in integration tests for Spanner"
Merge pull request #1014 from vladhlinskiy/feature/CDAP-15641-spanner-integration-tests
CDAP-15641 Spanner integration tests
Merge pull request #1015 from vladhlinskiy/feature/CDAP-15642-Pubsub-integration-tests-coverage
CDAP-15642 Pubsub integration tests coverage
Merge pull request #1017 from mlozbin-cybervisiontech/feature/CDAP-15643-bigtable-integration-tests
CDAP-15643 & CDAP-15734 Support all data types in BigTable Sink & Source integration tests
Merge pull request #1035 from cdapio/remove-validator-test-6-1
remove validator test
Add ability to configure the network used for tests
CDAP-16430 fix deploy hub plugin
Fix the deploy hub plugin method to use the public CDAP REST APIs
instead of some UI endpoint that is subject to change and which is
currently broken.
make dataproc worker resources configurable
Add a way to modify the dataproc worker cpus and memory through
passing in system properties. Also set the default to a higher
value, since some pipelines seem to be stuck waiting for
resources.
Merge pull request #1063 from cdapio/fix-integration-tests
fix integration tests
increase dataproc cluster size
This fixes an issue where some pipelines in some environments
are stuck forever with MapReduce jobs that stay in pending state.
disable yarn pmem check
CDAP services are getting killed by yarn due to physical memory
constraints. Disable the pmem check to avoid this.
Merge pull request #1022 from AntonGensitskiy/bug/CDAP-15771
CDAP-15771: Added check for BigQuery bytes type.
Merge pull request #1055 from cdapio/bugfix/CDAP-16430-fix-hub-plugin-deploy
CDAP-16430 fix deploy hub plugin
Fix dataproc profile creation to only set region
fix integration tests
Fix the google library dependencies to versions that all use
compatible dependencies, and remove the direct gax-grpc depdendency
that was causing Spanner to fail.

Also use macros in the BQ tests to both test macros and to
workaround an issue when the CDAP instance is running on a GCE
VM.
Merge pull request #1034 from cdapio/remove-validator-test
remove validator test
Merge pull request #1018 from vladhlinskiy/CDAP-15756-fix-spanner-tests
CDAP-15756 GoogleCloudSpannerTest broken due to pubsub dependency
CDAP-15804 use macros in tests for gcp plugins
Enhanced bigquery, datastore, and spanner tests to use macros.
Merge pull request #1056 from cdapio/merge-release-6-1
Merge release 6 1
Merge pull request #1062 from cdapio/increase-cluster-size
increase dataproc cluster size
Merge pull request #1036 from cdapio/CDAP-15804-gcp-test-macros
CDAP-15804 use macros in tests for gcp plugins
Merge pull request #1061 from cdapio/try-ordering-gcp-tests
explicitly list tests in the GCP suite
Merge pull request #1027 from cdapio/group-cloud-tests
Create a GCP test suite and fix tests to set project for plugins
Testless build
IT › UIE2EMINI › #8 5 days ago
Merge pull request #1023 from vladhlinskiy/feature/CDAP-15735_Added_tests_for_Array_in_Spanner_plugin2
CDAP-15735 Support all data types in integration tests for Spanner
remove validator test
the validator plugin was removed, so removing the test as well
Merge pull request #1056 from cdapio/merge-release-6-1
Merge release 6 1
Merge pull request #1019 from vladhlinskiy/feature/CDAP-15731-GCS-Data-Types
CDAP-15731 Support all data types in integration tests for GCS
Merge pull request #1063 from cdapio/fix-integration-tests
fix integration tests
make dataproc worker resources configurable
Add a way to modify the dataproc worker cpus and memory through
passing in system properties. Also set the default to a higher
value, since some pipelines seem to be stuck waiting for
resources.
disable yarn pmem check
CDAP services are getting killed by yarn due to physical memory
constraints. Disable the pmem check to avoid this.
Merge pull request #1061 from cdapio/try-ordering-gcp-tests
explicitly list tests in the GCP suite
CDAP-15879 test GCS formats
Modified the existing GCS test to test that all the formats can
be used, and to also test the usage of regex.
Merge pull request #1017 from mlozbin-cybervisiontech/feature/CDAP-15643-bigtable-integration-tests
CDAP-15643 & CDAP-15734 Support all data types in BigTable Sink & Source integration tests
Merge pull request #1036 from cdapio/CDAP-15804-gcp-test-macros
CDAP-15804 use macros in tests for gcp plugins
Merge branch 'release/6.1' into merge-release-6-1
CDAP-15804 use macros in tests for gcp plugins
Enhanced bigquery, datastore, and spanner tests to use macros.
Revert "CDAP-15735 Support all data types in integration tests for Spanner"
Merge pull request #1042 from yeweidaniel/integration-docs
Update steps for running integration test
Merge pull request #1037 from cdapio/disable-yarn-pmem-check
disable yarn pmem check
remove datastore metric assertions
Checking metrics is very unreliable due to race conditions. It can
fail incorrectly because they are not polled for a long enough time,
and they can pass incorrectly because the number being checked is
too low.
Merge pull request #1033 from cdapio/disable-yarn-mem-check
disable yarn pmem check
Merge pull request #1016 from AntonGensitskiy/feature/CDAP-15644-bigquery-intergation-tests
CDAP-15644: Added BigQuery integration tests.
CDAP-16454 increase time between status polls
Increase the time between program status polls, since most programs
take minutes to run and don't need to be polled every 50 milliseconds.
This also greatly reduces the amount of logging, making it
easier to investigate failures.
Merge pull request #1015 from vladhlinskiy/feature/CDAP-15642-Pubsub-integration-tests-coverage
CDAP-15642 Pubsub integration tests coverage
Merge pull request #1021 from cdapio/revert-1020-feature/CDAP-15735_Added_tests_for_Array_in_Spanner_plugin
Revert "CDAP-15735 Support all data types in integration tests for Spanner"
Merge pull request #1060 from cdapio/increase-time-between-polls
CDAP-16454 increase time between status polls
Merge pull request #1013 from vladhlinskiy/fetaure/CDAP-15640-GCS-integration-tests-coverage
CDAP-15640 GCS integration tests coverage
Merge pull request #1040 from cdapio/remove-metric-assertions-datastore
remove datastore metric assertions
Merge pull request #1030 from cdapio/set-dataproc-network
Add ability to configure the network used for tests
Merge pull request #1031 from cdapio/feature/CDAP-15879-gcs-formats-test
CDAP-15879 test GCS formats
Create a GCP test suite and fix tests to set project for plugins
Fixed the bigtable test to create an instance at the start of the
tests instead of requiring that an instance is already setup.

Added artifact info to all of the pipelines so that they are
viewable through the UI.

Fixed datetime checks for failing BigQuery test
Merge pull request #1027 from cdapio/group-cloud-tests
Create a GCP test suite and fix tests to set project for plugins
disable yarn pmem check
CDAP services are getting killed by yarn due to physical memory
constraints. Disable the pmem check to avoid this.
Merge pull request #1014 from vladhlinskiy/feature/CDAP-15641-spanner-integration-tests
CDAP-15641 Spanner integration tests
Merge pull request #1038 from AntonGensitskiy/PLUGIN-82
PLUGIN-82: Added new tests to cover dedupe functionality.
Merge pull request #1020 from vladhlinskiy/feature/CDAP-15735_Added_tests_for_Array_in_Spanner_plugin
CDAP-15735 Support all data types in integration tests for Spanner
Merge pull request #1029 from cdapio/fix-profile-creation
Fix dataproc profile creation to only set region
remove validator test
the validator plugin was removed, so removing the test as well
Merge pull request #1035 from cdapio/remove-validator-test-6-1
remove validator test
Merge pull request #1058 from cdapio/increase-dataproc-resources
make dataproc worker resources configurable
don't wait for status before starting
Merge pull request #1018 from vladhlinskiy/CDAP-15756-fix-spanner-tests
CDAP-15756 GoogleCloudSpannerTest broken due to pubsub dependency
Merge pull request #1062 from cdapio/increase-cluster-size
increase dataproc cluster size
CDAP-16429 log a warning if pipeline cleanup fails
The pubsub test was flaky because it would occasionally get a 400
error that the pipeline is not running when the stop call is made.
Changed the test to just log a warning in this scenario since the
test was actually successful. CDAP may have some underlying bugs
causing this behavior though.
Fix dataproc profile creation to only set region
Merge pull request #1034 from cdapio/remove-validator-test
remove validator test
increase dataproc cluster size
This fixes an issue where some pipelines in some environments
are stuck forever with MapReduce jobs that stay in pending state.
explicitly list tests in the GCP suite
Explicitly listing out tests in the GCP suite to control the
order and see if some failures can be explained by ordering.
Merge pull request #1053 from cdapio/bugfix/CDAP-16429-fix-pubsub-stop-flakiness
CDAP-16429 log a warning if pipeline cleanup fails
Add ability to configure the network used for tests
CDAP-16430 fix deploy hub plugin
Fix the deploy hub plugin method to use the public CDAP REST APIs
instead of some UI endpoint that is subject to change and which is
currently broken.
fix integration tests
Fix the google library dependencies to versions that all use
compatible dependencies, and remove the direct gax-grpc depdendency
that was causing Spanner to fail.

Also use macros in the BQ tests to both test macros and to
workaround an issue when the CDAP instance is running on a GCE
VM.
Merge pull request #1022 from AntonGensitskiy/bug/CDAP-15771
CDAP-15771: Added check for BigQuery bytes type.
Merge pull request #1055 from cdapio/bugfix/CDAP-16430-fix-hub-plugin-deploy
CDAP-16430 fix deploy hub plugin
Testless build
IT › UIE2EMINI › #7 5 days ago
make dataproc worker resources configurable
Add a way to modify the dataproc worker cpus and memory through
passing in system properties. Also set the default to a higher
value, since some pipelines seem to be stuck waiting for
resources.
Merge pull request #1061 from cdapio/try-ordering-gcp-tests
explicitly list tests in the GCP suite
Merge pull request #1063 from cdapio/fix-integration-tests
fix integration tests
Add ability to configure the network used for tests
CDAP-15879 test GCS formats
Modified the existing GCS test to test that all the formats can
be used, and to also test the usage of regex.
Merge pull request #1017 from mlozbin-cybervisiontech/feature/CDAP-15643-bigtable-integration-tests
CDAP-15643 & CDAP-15734 Support all data types in BigTable Sink & Source integration tests
fix integration tests
Fix the google library dependencies to versions that all use
compatible dependencies, and remove the direct gax-grpc depdendency
that was causing Spanner to fail.

Also use macros in the BQ tests to both test macros and to
workaround an issue when the CDAP instance is running on a GCE
VM.
Merge pull request #1022 from AntonGensitskiy/bug/CDAP-15771
CDAP-15771: Added check for BigQuery bytes type.
disable yarn pmem check
CDAP services are getting killed by yarn due to physical memory
constraints. Disable the pmem check to avoid this.
Merge pull request #1040 from cdapio/remove-metric-assertions-datastore
remove datastore metric assertions
Merge pull request #1031 from cdapio/feature/CDAP-15879-gcs-formats-test
CDAP-15879 test GCS formats
CDAP-15804 use macros in tests for gcp plugins
Enhanced bigquery, datastore, and spanner tests to use macros.
Merge pull request #1037 from cdapio/disable-yarn-pmem-check
disable yarn pmem check
Merge pull request #1018 from vladhlinskiy/CDAP-15756-fix-spanner-tests
CDAP-15756 GoogleCloudSpannerTest broken due to pubsub dependency
Merge pull request #1056 from cdapio/merge-release-6-1
Merge release 6 1
Merge pull request #1023 from vladhlinskiy/feature/CDAP-15735_Added_tests_for_Array_in_Spanner_plugin2
CDAP-15735 Support all data types in integration tests for Spanner
Merge pull request #1036 from cdapio/CDAP-15804-gcp-test-macros
CDAP-15804 use macros in tests for gcp plugins
Merge pull request #1062 from cdapio/increase-cluster-size
increase dataproc cluster size
Merge pull request #1016 from AntonGensitskiy/feature/CDAP-15644-bigquery-intergation-tests
CDAP-15644: Added BigQuery integration tests.
Merge pull request #1020 from vladhlinskiy/feature/CDAP-15735_Added_tests_for_Array_in_Spanner_plugin
CDAP-15735 Support all data types in integration tests for Spanner
Merge pull request #1013 from vladhlinskiy/fetaure/CDAP-15640-GCS-integration-tests-coverage
CDAP-15640 GCS integration tests coverage
remove validator test
the validator plugin was removed, so removing the test as well
CDAP-16454 increase time between status polls
Increase the time between program status polls, since most programs
take minutes to run and don't need to be polled every 50 milliseconds.
This also greatly reduces the amount of logging, making it
easier to investigate failures.
Merge pull request #1060 from cdapio/increase-time-between-polls
CDAP-16454 increase time between status polls
remove validator test
the validator plugin was removed, so removing the test as well
Merge pull request #1030 from cdapio/set-dataproc-network
Add ability to configure the network used for tests
Revert "CDAP-15735 Support all data types in integration tests for Spanner"
remove datastore metric assertions
Checking metrics is very unreliable due to race conditions. It can
fail incorrectly because they are not polled for a long enough time,
and they can pass incorrectly because the number being checked is
too low.
Merge pull request #1021 from cdapio/revert-1020-feature/CDAP-15735_Added_tests_for_Array_in_Spanner_plugin
Revert "CDAP-15735 Support all data types in integration tests for Spanner"
Merge pull request #1014 from vladhlinskiy/feature/CDAP-15641-spanner-integration-tests
CDAP-15641 Spanner integration tests
Merge pull request #1015 from vladhlinskiy/feature/CDAP-15642-Pubsub-integration-tests-coverage
CDAP-15642 Pubsub integration tests coverage
Merge pull request #1035 from cdapio/remove-validator-test-6-1
remove validator test
CDAP-16430 fix deploy hub plugin
Fix the deploy hub plugin method to use the public CDAP REST APIs
instead of some UI endpoint that is subject to change and which is
currently broken.
Merge pull request #1034 from cdapio/remove-validator-test
remove validator test
increase dataproc cluster size
This fixes an issue where some pipelines in some environments
are stuck forever with MapReduce jobs that stay in pending state.
Fix dataproc profile creation to only set region
Merge pull request #1055 from cdapio/bugfix/CDAP-16430-fix-hub-plugin-deploy
CDAP-16430 fix deploy hub plugin
Merge pull request #1042 from yeweidaniel/integration-docs
Update steps for running integration test
Create a GCP test suite and fix tests to set project for plugins
Fixed the bigtable test to create an instance at the start of the
tests instead of requiring that an instance is already setup.

Added artifact info to all of the pipelines so that they are
viewable through the UI.

Fixed datetime checks for failing BigQuery test
Merge pull request #1027 from cdapio/group-cloud-tests
Create a GCP test suite and fix tests to set project for plugins
Merge pull request #1033 from cdapio/disable-yarn-mem-check
disable yarn pmem check
disable yarn pmem check
CDAP services are getting killed by yarn due to physical memory
constraints. Disable the pmem check to avoid this.
CDAP-16429 log a warning if pipeline cleanup fails
The pubsub test was flaky because it would occasionally get a 400
error that the pipeline is not running when the stop call is made.
Changed the test to just log a warning in this scenario since the
test was actually successful. CDAP may have some underlying bugs
causing this behavior though.
Merge pull request #1019 from vladhlinskiy/feature/CDAP-15731-GCS-Data-Types
CDAP-15731 Support all data types in integration tests for GCS
Merge pull request #1029 from cdapio/fix-profile-creation
Fix dataproc profile creation to only set region
Merge pull request #1053 from cdapio/bugfix/CDAP-16429-fix-pubsub-stop-flakiness
CDAP-16429 log a warning if pipeline cleanup fails
Merge pull request #1058 from cdapio/increase-dataproc-resources
make dataproc worker resources configurable
Merge branch 'release/6.1' into merge-release-6-1
don't wait for status before starting
explicitly list tests in the GCP suite
Explicitly listing out tests in the GCP suite to control the
order and see if some failures can be explained by ordering.
Merge pull request #1038 from AntonGensitskiy/PLUGIN-82
PLUGIN-82: Added new tests to cover dedupe functionality.
Testless build
IT › SAN › #247 1 week ago
Merge pull request #1062 from cdapio/increase-cluster-size
increase dataproc cluster size
Merge pull request #1063 from cdapio/fix-integration-tests
fix integration tests
increase dataproc cluster size
This fixes an issue where some pipelines in some environments
are stuck forever with MapReduce jobs that stay in pending state.
fix integration tests
Fix the google library dependencies to versions that all use
compatible dependencies, and remove the direct gax-grpc depdendency
that was causing Spanner to fail.

Also use macros in the BQ tests to both test macros and to
workaround an issue when the CDAP instance is running on a GCE
VM.
42 passed
IT › UPD2 › #468 1 week ago
fix integration tests
Fix the google library dependencies to versions that all use
compatible dependencies, and remove the direct gax-grpc depdendency
that was causing Spanner to fail.

Also use macros in the BQ tests to both test macros and to
workaround an issue when the CDAP instance is running on a GCE
VM.
Merge pull request #1063 from cdapio/fix-integration-tests
fix integration tests
Testless build
IT › IIT › #688 1 week ago
fix integration tests
Fix the google library dependencies to versions that all use
compatible dependencies, and remove the direct gax-grpc depdendency
that was causing Spanner to fail.

Also use macros in the BQ tests to both test macros and to
workaround an issue when the CDAP instance is running on a GCE
VM.
Merge pull request #1063 from cdapio/fix-integration-tests
fix integration tests
46 passed
IT › IIT › #685 1 week ago
Merge pull request #1061 from cdapio/try-ordering-gcp-tests
explicitly list tests in the GCP suite
explicitly list tests in the GCP suite
Explicitly listing out tests in the GCP suite to control the
order and see if some failures can be explained by ordering.
46 passed
IT › ITN › #342 2 weeks ago
don't wait for status before starting
719 passed
IT › IIT › #683 2 weeks ago
don't wait for status before starting
46 passed
CDAP › RUT › #1049 2 weeks ago
CDAP-16454 add sleep time to waitForRuns method
Merge pull request #11986 from cdapio/feature/CDAP-16454-add-sleep-time-to-program-wait
CDAP-16454 add sleep time to waitForRuns method
2677 passed