Terence Yim <terencey@google.com>: Author Summary

Builds triggered by Terence Yim <terencey@google.com>

Builds triggered by an author are those builds which contains changes committed by the author.
1420
241 (17%)
1179 (83%)

Breakages and fixes

Broken means the build has failed but the previous build was successful.
Fixed means that the build was successful but the previous build has failed.
100 (7% of all builds triggered)
87 (6% of all builds triggered)
-13
Build Completed Code commits Tests
CDAP › DUT › #3191 1 week ago
Close connection in unit-test
(CDAP-17610) Added OAuth macro function
- Expanded plugin config to support object type fields
- Added a new OAuth macro function that uses the studio app for getting oauth information
(CDAP-17610) Refactor to pass in ServiceDiscoverer for Macro usage
(CDAP-17610) Added support for system app service discovery
- Allows runtime to discovery and communicate with system app services
2921 passed
CDAP › RUT › #1380 1 week ago
(CDAP-17610) Refactor to pass in ServiceDiscoverer for Macro usage
(CDAP-17610) Added support for system app service discovery
- Allows runtime to discovery and communicate with system app services
Close connection in unit-test
(CDAP-17610) Added OAuth macro function
- Expanded plugin config to support object type fields
- Added a new OAuth macro function that uses the studio app for getting oauth information
1 of 2421 failed
IT › UPD2 › #809 1 week ago
Close connection in unit-test
(CDAP-17610) Refactor to pass in ServiceDiscoverer for Macro usage
(CDAP-17610) Added support for system app service discovery
- Allows runtime to discovery and communicate with system app services
(CDAP-17610) Added OAuth macro function
- Expanded plugin config to support object type fields
- Added a new OAuth macro function that uses the studio app for getting oauth information
Testless build
CDAP › UDUT › #1287 1 week ago
Close connection in unit-test
(CDAP-17610) Added OAuth macro function
- Expanded plugin config to support object type fields
- Added a new OAuth macro function that uses the studio app for getting oauth information
(CDAP-17610) Added support for system app service discovery
- Allows runtime to discovery and communicate with system app services
(CDAP-17610) Refactor to pass in ServiceDiscoverer for Macro usage
Testless build
CDAP › URUT › #1297 1 week ago
(CDAP-17610) Added support for system app service discovery
- Allows runtime to discovery and communicate with system app services
(CDAP-17610) Refactor to pass in ServiceDiscoverer for Macro usage
Close connection in unit-test
(CDAP-17610) Added OAuth macro function
- Expanded plugin config to support object type fields
- Added a new OAuth macro function that uses the studio app for getting oauth information
Testless build
CDAP › DRC › #5277 1 week ago
(CDAP-17610) Added support for system app service discovery
- Allows runtime to discovery and communicate with system app services
(CDAP-17610) Refactor to pass in ServiceDiscoverer for Macro usage
Close connection in unit-test
(CDAP-17610) Added OAuth macro function
- Expanded plugin config to support object type fields
- Added a new OAuth macro function that uses the studio app for getting oauth information
Testless build
CDAP › BPP › #1552 1 week ago
Close connection in unit-test
(CDAP-17610) Added OAuth macro function
- Expanded plugin config to support object type fields
- Added a new OAuth macro function that uses the studio app for getting oauth information
(CDAP-17610) Refactor to pass in ServiceDiscoverer for Macro usage
(CDAP-17610) Added support for system app service discovery
- Allows runtime to discovery and communicate with system app services
Testless build
IT › UPD2 › #807 1 week ago
(CDAP-17630) Capture environment variables setup by spark-env.sh
- This is to handle case where the CDAP process is launched without spark-env.sh being ran prior, such as in remote process execution case
Testless build
CDAP › RUT › #1374 1 week ago
(CDAP-17630) Capture environment variables setup by spark-env.sh
- This is to handle case where the CDAP process is launched without spark-env.sh being ran prior, such as in remote process execution case
1 of 2028 failed
CDAP › UDUT › #1280 1 week ago
(CDAP-17630) Capture environment variables setup by spark-env.sh
- This is to handle case where the CDAP process is launched without spark-env.sh being ran prior, such as in remote process execution case
Testless build
Build Completed Code commits Tests
CDAP › RUT › #1380 1 week ago
(CDAP-17610) Refactor to pass in ServiceDiscoverer for Macro usage
(CDAP-17610) Added support for system app service discovery
- Allows runtime to discovery and communicate with system app services
Close connection in unit-test
(CDAP-17610) Added OAuth macro function
- Expanded plugin config to support object type fields
- Added a new OAuth macro function that uses the studio app for getting oauth information
1 of 2421 failed
CDAP › RUT › #1369 2 weeks ago
(CDAP-17691) Make it possible to store secret in the SYSTEM namespace
- Also fixed a pom issue that copy the test sources incorrectly
1 of 2417 failed
CDAP › DUT › #3167 1 month ago
(CDAP-17645) Applies SSH config from the system and default a timeout
1 of 2410 failed
CDAP › RUT › #1354 1 month ago
(CDAP-17645) Applies SSH config from the system and default a timeout
1 of 2410 failed
CDAP › RUT › #1336 1 month ago
(CDAP-17439) Add the missing Preconditions.checkNotNull methods for Hadoop 3
1 of 2014 failed
CDAP › DUT › #3149 1 month ago
(CDAP-17439) Add the missing Preconditions.checkNotNull methods for Hadoop 3
1 of 2014 failed
CDAP › RUT › #1328 2 months ago
(CDAP-17507) Default starting pipeline studio and dataprep system app
- Use capability manager to start
2 of 2014 failed
CDAP › RUT › #1267 4 months ago
(CDAP-17367) Use different config name for tracking and limiting InputFormat
1 of 2856 failed
CDAP › RUT › #1252 5 months ago
(CDAP-17287) Collection of Spark execution event logs
- Introducing the Spark event logging and upload of logs through the runtime service.
  Based on the cConf settings, the event logging collection will be on/off regardless
  of the Spark job spark.eventLog.* configurations.
- Refactoring of the SparkRuntimeEnv class to simplify the logic by using CompletableFuture.
- Make runtime system services shutdown after the SparkExecutionContext stopped.
  Currently they stop concurrently. It is not a problem before this PR since we
  are not doing any important work at the context stop.
  However, this PR introduces uploading the event logs at the context stop,
  which needs to be completed before the system services shutdown and process termination.
1 of 1984 failed
CDAP › RUT › #1242 6 months ago
(CDAP-17218) Launch Preview Runner Using TwillRunner
- Introduced PreviewRunnerTwillApplication and PreviewRunnerTwillRunnable
  for starting multiple instances of preview runners in the cluster
- Fixed preview level db binding to correctly use the preview local data directory
  instead of using the regular data directory
- Refactoring of preview related Guice bindings
1 of 1984 failed
Build Completed Code commits Tests
CDAP › DUT › #3181 2 weeks ago
(CDAP-17691) Make it possible to store secret in the SYSTEM namespace
- Also fixed a pom issue that copy the test sources incorrectly
2893 passed
CDAP › RUT › #1332 1 month ago
(CDAP-16008) Checks if kerberos settings are empty
2883 passed
CDAP › DUT › #3137 2 months ago
(CDAP-16008) Added Kerberos support to Remote Hadoop provisioner
2873 passed
CDAP › DUT › #3128 2 months ago
(CDAP-17474) Allow configuring ElasticSearch to trust all certs
2873 passed
CDAP › RUT › #1310 2 months ago
(CDAP-17474) Allow configuring ElasticSearch to trust all certs
2873 passed
CDAP › DUT › #3112 2 months ago
(CDAP-17439) Added runtime support for Hadoop + Spark 3
This change added support for Hadoop 3 and Spark 3, yet maintaining compatibility with
Hadoop 2 and Spark 1 and Spark 2.

Hadoop YARN
* Don’t use YARN localization to expand program jar file due to YARN bug YARN-9591
** In twill containers, expand the jar locally
** In Spark, renames the jar to .zip before localization

Guava depdendency
* Add missing methods to the Guava library
** Various missing Preconditions.checkArgument() methods
** Various missing Preconditions.checkState() methods
** MoreExecutors.directExecutor() method

Hadoop 3
* Upgrade to Avro 1.8.2
** SpecificData constructor becomes public
* Remove usage of ConcurrentHashSet from the jetty library as Hadoop 3 uses an incompatible version, and CDAP shouldn’t be using that library outside of the cdap-security module

Spark 3
* Use Reflection to alter SparkListenerApplicationStart event
** Spark 2 and 3 are binary incompatible for that class

Also, there is some small refactoring to modernize the code to use the standard Java library instead of Guava
2849 passed
CDAP › RUT › #1263 4 months ago
(CDAP-17333) Makes calls from the program context resilient against service outage
- Wrap calls from AbstractContext with the program RetryStrategy
2839 passed
IT › UPD2 › #673 4 months ago
(CDAP-17323) Skip network checking logic for Existing Dataproc
Testless build
CDAP › DUT › #3067 5 months ago
(CDAP-17287) Collection of Spark execution event logs
- Introducing the Spark event logging and upload of logs through the runtime service.
  Based on the cConf settings, the event logging collection will be on/off regardless
  of the Spark job spark.eventLog.* configurations.
- Refactoring of the SparkRuntimeEnv class to simplify the logic by using CompletableFuture.
- Make runtime system services shutdown after the SparkExecutionContext stopped.
  Currently they stop concurrently. It is not a problem before this PR since we
  are not doing any important work at the context stop.
  However, this PR introduces uploading the event logs at the context stop,
  which needs to be completed before the system services shutdown and process termination.
2839 passed
CDAP › DUT › #3062 5 months ago
(CDAP-17257) Wait till the TwillRunnable.run return on stopping
- This fixes the preview pod get killed too fast before it can stop itself gracefully
2839 passed