wyzhang <wyzhang@google.com>: Author Summary

Builds triggered by wyzhang <wyzhang@google.com>

Builds triggered by an author are those builds which contains changes committed by the author.
181
17 (9%)
164 (91%)

Breakages and fixes

Broken means the build has failed but the previous build was successful.
Fixed means that the build was successful but the previous build has failed.
8 (4% of all builds triggered)
8 (4% of all builds triggered)
0
Build Completed Code commits Tests
HYP › BAD › #298 1 month ago
Merge pull request #1085 from cdapio/feature/update-submodules
Update submodule
862 passed
IT › UPD2 › #512 1 month ago
Merge pull request #12137 from cdapio/bug/CDAP-16730/kubeMem
(CDAP-16730) Set correct -xmx for pods in k8s to run programs
(CDAP-16730) Set correct -xmx for pods in k8s to run programs
Bug:
We have introduced a multiplier to determine the actual CPU and RAM
to request for creating the pod to run a program. But we fail to
set the correct -xmx based on the RAM requested for the pod.

This change
- Set memory requested to be memory * multiplier
- Set xmx using org.apache.twill.internal.utils.Resources.computeMaxHeapSiz
  based on the memory request size
Testless build
CDAP › DUT › #2920 1 month ago
Merge pull request #12137 from cdapio/bug/CDAP-16730/kubeMem
(CDAP-16730) Set correct -xmx for pods in k8s to run programs
(CDAP-16730) Set correct -xmx for pods in k8s to run programs
Bug:
We have introduced a multiplier to determine the actual CPU and RAM
to request for creating the pod to run a program. But we fail to
set the correct -xmx based on the RAM requested for the pod.

This change
- Set memory requested to be memory * multiplier
- Set xmx using org.apache.twill.internal.utils.Resources.computeMaxHeapSiz
  based on the memory request size
2715 passed
CDAP › RUT › #1113 1 month ago
(CDAP-16730) Set correct -xmx for pods in k8s to run programs
Bug:
We have introduced a multiplier to determine the actual CPU and RAM
to request for creating the pod to run a program. But we fail to
set the correct -xmx based on the RAM requested for the pod.

This change
- Set memory requested to be memory * multiplier
- Set xmx using org.apache.twill.internal.utils.Resources.computeMaxHeapSiz
  based on the memory request size
Merge pull request #12137 from cdapio/bug/CDAP-16730/kubeMem
(CDAP-16730) Set correct -xmx for pods in k8s to run programs
2715 passed
CDAP › UDUT › #934 1 month ago
(CDAP-16730) Set correct -xmx for pods in k8s to run programs
Bug:
We have introduced a multiplier to determine the actual CPU and RAM
to request for creating the pod to run a program. But we fail to
set the correct -xmx based on the RAM requested for the pod.

This change
- Set memory requested to be memory * multiplier
- Set xmx using org.apache.twill.internal.utils.Resources.computeMaxHeapSiz
  based on the memory request size
Merge pull request #12137 from cdapio/bug/CDAP-16730/kubeMem
(CDAP-16730) Set correct -xmx for pods in k8s to run programs
Testless build
CDAP › URUT › #942 1 month ago
(CDAP-16730) Set correct -xmx for pods in k8s to run programs
Bug:
We have introduced a multiplier to determine the actual CPU and RAM
to request for creating the pod to run a program. But we fail to
set the correct -xmx based on the RAM requested for the pod.

This change
- Set memory requested to be memory * multiplier
- Set xmx using org.apache.twill.internal.utils.Resources.computeMaxHeapSiz
  based on the memory request size
Merge pull request #12137 from cdapio/bug/CDAP-16730/kubeMem
(CDAP-16730) Set correct -xmx for pods in k8s to run programs
Testless build
CDAP › DRC › #4892 1 month ago
Merge pull request #12137 from cdapio/bug/CDAP-16730/kubeMem
(CDAP-16730) Set correct -xmx for pods in k8s to run programs
(CDAP-16730) Set correct -xmx for pods in k8s to run programs
Bug:
We have introduced a multiplier to determine the actual CPU and RAM
to request for creating the pod to run a program. But we fail to
set the correct -xmx based on the RAM requested for the pod.

This change
- Set memory requested to be memory * multiplier
- Set xmx using org.apache.twill.internal.utils.Resources.computeMaxHeapSiz
  based on the memory request size
Testless build
IT › UPD2 › #507 2 months ago
Merge pull request #12142 from cdapio/bugfix/CDAP-16730/wyzhang/kubeMem
(CDAP-16730) Set correct -xmx for pods in k8s to run programs
(CDAP-16730) Set correct -xmx for pods in k8s to run programs
Bug:
We have introduced a multiplier to determine the actual CPU and RAM
to request for creating the pod to run a program. But we fail to
set the correct -xmx based on the RAM requested for the pod.

This change
- Set memory requested to be memory * multiplier
- Set xmx using org.apache.twill.internal.utils.Resources.computeMaxHeapSiz
  based on the memory request size
Testless build
CDAP › DUT › #2918 2 months ago
(CDAP-16730) Set correct -xmx for pods in k8s to run programs
Bug:
We have introduced a multiplier to determine the actual CPU and RAM
to request for creating the pod to run a program. But we fail to
set the correct -xmx based on the RAM requested for the pod.

This change
- Set memory requested to be memory * multiplier
- Set xmx using org.apache.twill.internal.utils.Resources.computeMaxHeapSiz
  based on the memory request size
Merge pull request #12142 from cdapio/bugfix/CDAP-16730/wyzhang/kubeMem
(CDAP-16730) Set correct -xmx for pods in k8s to run programs
2713 passed
CDAP › RUT › #1111 2 months ago
Merge pull request #12142 from cdapio/bugfix/CDAP-16730/wyzhang/kubeMem
(CDAP-16730) Set correct -xmx for pods in k8s to run programs
(CDAP-16730) Set correct -xmx for pods in k8s to run programs
Bug:
We have introduced a multiplier to determine the actual CPU and RAM
to request for creating the pod to run a program. But we fail to
set the correct -xmx based on the RAM requested for the pod.

This change
- Set memory requested to be memory * multiplier
- Set xmx using org.apache.twill.internal.utils.Resources.computeMaxHeapSiz
  based on the memory request size
2713 passed
Build Completed Code commits Tests
CDAP › RUT › #1108 2 months ago
Merge pull request #12092 from cdapio/feature/CDAP-16610/wyzhang/cherrypick/runtime
(CDAP-16610) Fixing DataprocUtils.getBucketName null pointer exception
Merge pull request #12126 from cdapio/feature/CDAP-16672/cherrypick/dataproc
(CDAP-16672) Support GCP Dataproc cluster property override via cdap-site.xml
(CDAP-16672) Support GCP Dataproc cluster property override via cdap-site.xml
Why:
When creating Dataproc cluster, overriding cluster properties
(i.e. property key-val pairs in config files) cannot be done dynamically.
Currenlty they are hard-coded.

What:
This change gets property overrides from ProvisionerSystemContext and set them
in the context used for dataproc creation, which get passed to Dataproc via
creation API.

How to use:
Format of property override in cdap-site.xml:
"[prefix].gcp-dataproc.[dataproc config file]:[property key]" : "[property val]"
where "prefix" is "provisioner.system.properties"

E.g.
provisioner.system.properties.gcp-dataproc.capacity-scheduler:yarn.scheduler.capacity.maximum-am-resource-percent: "0.5"

Needs to comply with format
"[prefix].gcp-dataproc.[dataproc config file]:[property key]" : "[property val]"
(CDAP-16610) Fixing DataprocUtils.getBucketName null pointer exception
Why:
Starting a pipline failed with the null pointer exception.

What:
This bucket is used for launching jobs in dataproc. Its name is set
via provisioner.system.properties.gcp-dataproc.bucket which is in
system context. But we failed to propagate it to the context
when setting one up for runtime job manager
1 of 1139 failed
CDAP › DUT › #2912 2 months ago
(CDAP-16672) Support GCP Dataproc cluster property override via cdap-site.xml
Why:
When creating Dataproc cluster, overriding cluster properties
(i.e. property key-val pairs in config files) cannot be done dynamically.
Currenlty they are hard-coded.

What:
This change gets property overrides from ProvisionerSystemContext and set them
in the context used for dataproc creation, which get passed to Dataproc via
creation API.

How to use:
Format of property override in cdap-site.xml:
"[prefix].gcp-dataproc.[dataproc config file]:[property key]" : "[property val]"
where "prefix" is "provisioner.system.properties"

E.g.
provisioner.system.properties.gcp-dataproc.capacity-scheduler:yarn.scheduler.capacity.maximum-am-resource-percent: "0.5"

Needs to comply with format
"[prefix].gcp-dataproc.[dataproc config file]:[property key]" : "[property val]"
Merge pull request #12122 from cdapio/feature/CDAP-16672/wyzhang/dataproc
(CDAP-16672) Support GCP Dataproc cluster property override via cdap-site.xml
address comments
1 of 2593 failed
CDAP › RUT › #1099 2 months ago
Merge pull request #12088 from cdapio/feature/CDAP-16610/wyzhang/fix/runtime
(CDAP-16610) Fixing DataprocUtils.getBucketName null pointer exception
(CDAP-16610) Fixing DataprocUtils.getBucketName null pointer exception
Why:
Starting a pipline failed with the null pointer exception.

What:
This bucket is used for launching jobs in dataproc. Its name is set
via provisioner.system.properties.gcp-dataproc.bucket which is in
system context. But we failed to propagate it to the context
when setting one up for runtime job manager
1 of 2701 failed
IT › UPD2 › #475 3 months ago
(CDAP-16353) cleanup/refactor: move bindings for preview runner into its module.
Motivation:
small refactoring to move bindings for preview runner into central location
(i.e. its own guice module DefaultPreviewRunnerModule)
Merge pull request #12024 from cdapio/feature/CDAP-16353/wyzhang/preview/testServiceMain
(CDAP-16353) Small refactoring of test for k8s service test base
Merge pull request #12023 from cdapio/feature/CDAP-16353/wyzhang/preview/binding
(CDAP-16353) cleanup/refactor: move bindings for preview runner into its module
(CDAP-16353) Small refactoring of test for k8s service test base
What:
Introduce APIs for starting and stopping service in k8s service main test base.

Why:
Will be used in a follow-up PR to allow preview test to start and stop
preview service.
Testless build
CDAP › DUT › #2869 3 months ago
{CDAP-16353) Implement LocalScheduleFetcher to get schedule from local store.
Motiviation:
Having local vs remote impliation for ScheduleFetcher allows us to
fetcher schedules from local when using shared SQL as storage and
from remote when using non-sharing NOSQL.

We have already implemented RemoteScheduleFetcher. This change is
to introduce a local version.

Currently neither local nor remote is being used. A follow-up PR
will make metadata service use either local or remote depending on
storage implementation configuration.
(CDAP-16353) parameterize preference fetcher test to cover local and remote
Motivation:
To ensure that local and remote implementations for PreferencesFetcher have
consistent behavior. Parameterizing the test to run exact same set of tests
for both implementations.
Merge pull request #12007 from cdapio/feature/CDAP-16353/wyzhang/localFetchers
{CDAP-16353) Implement LocalScheduleFetcher to get schedule from local store
Merge pull request #12008 from cdapio/feature/CDAP-16353/wyzhang/preview/propertyResolver
(CDAP-16353) parameterize preference fetcher test to ensure same behavior for local vs remote
1 of 2704 failed
IT › UPD2 › #469 3 months ago
Merge pull request #11984 from cdapio/feature/CDAP-16353/wyzhang/preview/artifactHttpRepo
(CDAP-16353) Introduce ArtifactRepositoryReader interface and implement local and remote versions
(CDAP-16353) Introduce ArtifactRepositoryReader interface and implement local and remote versions
Goal:
- Support ArtifactRepository impl to get artifact detail from either local store or remote Appfabric

Specially this change includes:
- Create a new interface ArtifactRepositoryReader and move getArtifact method over from ArtifactRepository
- Implement local and remote version of ArtifactRepositoryReader
- Use local version in all places (e.g. Preview and AppFabric).
- Remote version is unused at the moment. Will switch preview to use it later when storage is NOSQL
Testless build
CDAP › RUT › #1005 4 months ago
Merge pull request #11896 from cdapio/wyzhang/develop/leveldb/v5
(CDAP-16353) Add a preferences http handler returning PreferencesDetail (for internal usage only)
2656 passed
CDAP › DUT › #2808 4 months ago
Merge pull request #11896 from cdapio/wyzhang/develop/leveldb/v5
(CDAP-16353) Add a preferences http handler returning PreferencesDetail (for internal usage only)
2 of 2654 failed
Build Completed Code commits Tests
CDAP › RUT › #1111 2 months ago
Merge pull request #12142 from cdapio/bugfix/CDAP-16730/wyzhang/kubeMem
(CDAP-16730) Set correct -xmx for pods in k8s to run programs
(CDAP-16730) Set correct -xmx for pods in k8s to run programs
Bug:
We have introduced a multiplier to determine the actual CPU and RAM
to request for creating the pod to run a program. But we fail to
set the correct -xmx based on the RAM requested for the pod.

This change
- Set memory requested to be memory * multiplier
- Set xmx using org.apache.twill.internal.utils.Resources.computeMaxHeapSiz
  based on the memory request size
2713 passed
CDAP › RUT › #1073 3 months ago
Merge pull request #12022 from cdapio/feature/CDAP-16353/wyzhang/preview/providers
(CDAP-16353) Add binding providers to allow local vs remote bindings
(CDAP-16353) Add binding providers to allow local vs remote bindings
Motivation:
We have local vs remote implications for
* ApplicationDetailFetcher
* ArtifactRepositoryReader
* PluginFinder
* PreferencesFetcher

This change introduces providers for each of them to bind interface
to local impl for SQL and remote for NOSQL
2702 passed
CDAP › DUT › #2870 3 months ago
{CDAP-16353) Implement LocalApplicationDetailFetcher to get ApplicationDetail
Motiviation:
Having local vs remote impliation for ApplicationDetailFetcher allows us to
fetcher ApplicationDetail from local when using shared SQL as storage and
from remote when using non-sharing NOSQL.

We have already implemented RemoteApplicationDetailFetcher. This change is
to introduce a local version.

Currently neither local nor remote is being used. A follow-up PR
will make profile metadata service use either local or remote depending on
storage implementation configuration.
(CDAP-16353) Cleanup: remove MetadataServiceModule from AppFabricServiceRuntimeModule
Reason:
MetadataServiceModule contains metadata and lineage http handler bindings and
a binding for MetadataAdmin.  All these are specific to metadata service and are
not needed by AppFabric.
Merge pull request #12010 from cdapio/feature/CDAP-16353/wyzhang/fetcher
{CDAP-16353) Implement LocalApplicationDetailFetcher to get ApplicationDetail
Merge pull request #12012 from cdapio/feature/CDAP-16353/wyzhang/cleanup/module
(CDAP-16353) Cleanup: remove MetadataServiceModule from AppFabricServiceRuntimeModule
2691 passed
IT › UPD2 › #468 3 months ago
(CDAP-16353) Replace ArtifactRepository param with PluginFinder for AbstractConfigurer
Motivation:
AbstractConfigurer uses ArtifactRepository just to find the artifact for a given plugin,
this functionality is already provided by PluginFinder which has local and remote implementations,
thereby allowing us to use local impl when having shared SQL and remote version for non-shared
per service local levelDB.
Merge pull request #11988 from cdapio/feature/CDAP-16353/wyzhang/preview/pluginFinder
(CDAP-16353) Replace ArtifactRepository param with PluginFinder for AbstractConfigurer…
Merge pull request #12000 from cdapio/feature/CDAP-16353/wyzhang/cleanup
(CDAP-16353) Adding a log entry for twill program termination status
(CDAP-16353) Adding an info log entry for twill program termination status
This is to facilitate debugging.
Testless build
CDAP › RUT › #1018 4 months ago
Merge pull request #11914 from cdapio/feature/CDAP-16353/wyzhang/leveldb/logs
(CDAP-16353)Add local/remote program run record fetcher for LogHttpHandler
(CDAP-16353) Add local/remote program run record fetcher for LogHttpHandler
2666 passed
CDAP › DUT › #2818 4 months ago
(CDAP-16353) Add clients for fetching ApplicationDetail, PreferencesDetail, ScheduleDetail
Merge pull request #11903 from cdapio/feature/CDAP-16353/wyzhang/leveldb/fetcher
Add client to fetch Application/Schedule/Preferences details
2665 passed
IT › UPD2 › #438 4 months ago
(CDAP-16353) Adding last update time to ScheduleDetail that gets returned by get schedule API
Merge pull request #11900 from cdapio/feature/cdap-wyzhang/leveldb/v0
Testless build
IT › UPD2 › #265 9 months ago
Update netty-http version from 1.3.0 to 1.3.1 which fixes a high cpu usage bug due to spinning in an infinit loop
Merge pull request #11642 from cdapio/wyzhang/upgrade_netty_http_version
Upgrade netty-http version from 1.3.0 to 1.3.1 which fixes a high cpu …
Testless build