Build: #678 was successful Child of CDAP-DRC-4262
Code commits
caskdata/cdap
-
albertshau <ashau@google.com> 28456716dc26320ccfaae0c1ba2dcec3114197ef
Merge pull request #10993 from cdapio/feature/CDAP-14827-get-namespace-info
CDAP-14827 Added admin call to get namespace information -
albertshau <ashau@google.com> 2234e8f27b15c1d5a4a875ae77feaa25cac4c673 m
CDAP-14827 Added admin call to get namespace information
Added a method to get information about a namespace through the
Admin. Also added a generation field to namespaces that can
be used by consumers to know when a namespace has been deleted
and then recreated.
This feature will be used in dataprep to include the namespace
generation in entity row keys to avoid showing entities from
a previously existing namespace that was deleted and then
recreated.- cdap-api/src/main/java/co/cask/cdap/api/Admin.java (version 2234e8f27b15c1d5a4a875ae77feaa25cac4c673)
- cdap-api/src/main/java/co/cask/cdap/api/NamespaceSummary.java (version 2234e8f27b15c1d5a4a875ae77feaa25cac4c673)
- cdap-app-fabric/src/main/java/co/cask/cdap/app/runtime/scheduler/SchedulerQueueResolver.java (version 2234e8f27b15c1d5a4a875ae77feaa25cac4c673)
- cdap-app-fabric/src/main/java/co/cask/cdap/gateway/handlers/NamespaceHttpHandler.java (version 2234e8f27b15c1d5a4a875ae77feaa25cac4c673)
- cdap-app-fabric/src/main/java/co/cask/cdap/internal/app/runtime/DefaultAdmin.java (version 2234e8f27b15c1d5a4a875ae77feaa25cac4c673)
- cdap-app-fabric/src/main/java/co/cask/cdap/internal/bootstrap/executor/DefaultNamespaceCreator.java (version 2234e8f27b15c1d5a4a875ae77feaa25cac4c673)
- cdap-app-fabric/src/test/java/co/cask/cdap/internal/AppFabricClient.java (version 2234e8f27b15c1d5a4a875ae77feaa25cac4c673)
- cdap-app-fabric/src/test/java/co/cask/cdap/internal/app/services/http/handlers/NamespaceHttpHandlerTest.java (version 2234e8f27b15c1d5a4a875ae77feaa25cac4c673)
- cdap-app-fabric/src/test/java/co/cask/cdap/internal/app/store/DefaultStoreTest.java (version 2234e8f27b15c1d5a4a875ae77feaa25cac4c673)
- cdap-common/src/main/java/co/cask/cdap/common/namespace/NamespaceAdmin.java (version 2234e8f27b15c1d5a4a875ae77feaa25cac4c673)
- cdap-common/src/test/java/co/cask/cdap/common/test/NoopAdmin.java (version 2234e8f27b15c1d5a4a875ae77feaa25cac4c673)
- cdap-operational-stats-core/src/test/java/co/cask/cdap/operations/cdap/CDAPOperationalStatsTest.java (version 2234e8f27b15c1d5a4a875ae77feaa25cac4c673)
- cdap-proto/src/main/java/co/cask/cdap/proto/NamespaceMeta.java (version 2234e8f27b15c1d5a4a875ae77feaa25cac4c673)
- cdap-security/src/main/java/co/cask/cdap/security/authorization/DefaultAuthorizationContext.java (version 2234e8f27b15c1d5a4a875ae77feaa25cac4c673)
- cdap-unit-test/src/test/java/co/cask/cdap/admin/AdminApp.java (version 2234e8f27b15c1d5a4a875ae77feaa25cac4c673)
- cdap-unit-test/src/test/java/co/cask/cdap/admin/AdminAppTestRun.java (version 2234e8f27b15c1d5a4a875ae77feaa25cac4c673)
-
Terence Yim <885032+chtyim@users.noreply.github.com> 5ef98ddf2c2e0c1895199b6c0d7da24510883bb5
Merge pull request #11025 from cdapio/feature/CDAP-14646-isolate-master-services
(CDAP-14646) Create isolated main classes for TMS and Metrics services -
Terence Yim 0b3778ea45e569e6bdd2ca530c2350a7f153718c
(CDAP-14646) Introduct cdap-master-spi module
- Created MessagingServiceMain to run as isolated process
- Created MetricsServiceMain to run as isolated process
- Introduced K8s implementation of cdap-master-spi- cdap-common/src/main/java/co/cask/cdap/common/conf/CConfigurationUtil.java (version 0b3778ea45e569e6bdd2ca530c2350a7f153718c)
- cdap-common/src/main/java/co/cask/cdap/common/conf/Constants.java (version 0b3778ea45e569e6bdd2ca530c2350a7f153718c)
- cdap-common/src/main/resources/cdap-default.xml (version 0b3778ea45e569e6bdd2ca530c2350a7f153718c)
- cdap-common/src/test/java/co/cask/cdap/common/conf/CConfigurationMapTest.java (version 0b3778ea45e569e6bdd2ca530c2350a7f153718c)
- cdap-kubernetes/pom.xml (version 0b3778ea45e569e6bdd2ca530c2350a7f153718c)
- cdap-kubernetes/src/main/java/co/cask/cdap/k8s/discovery/KubeDiscoveryService.java (version 0b3778ea45e569e6bdd2ca530c2350a7f153718c)
- cdap-kubernetes/src/main/java/co/cask/cdap/master/environment/k8s/KubeMasterEnvironment.java (version 0b3778ea45e569e6bdd2ca530c2350a7f153718c)
- cdap-kubernetes/src/main/resources/META-INF/services/META-INF/services/co.cask.cdap.master.spi.environment.MasterEnvironment (version 0b3778ea45e569e6bdd2ca530c2350a7f153718c)
- cdap-master-spi/pom.xml (version 0b3778ea45e569e6bdd2ca530c2350a7f153718c)
- cdap-master-spi/src/main/java/co/cask/cdap/master/spi/environment/MasterEnvironment.java (version 0b3778ea45e569e6bdd2ca530c2350a7f153718c)
- cdap-master-spi/src/main/java/co/cask/cdap/master/spi/environment/MasterEnvironmentContext.java (version 0b3778ea45e569e6bdd2ca530c2350a7f153718c)
- cdap-master/pom.xml (version 0b3778ea45e569e6bdd2ca530c2350a7f153718c)
- cdap-master/src/main/java/co/cask/cdap/data/runtime/main/MessagingServiceTwillRunnable.java (version 0b3778ea45e569e6bdd2ca530c2350a7f153718c)
- cdap-master/src/main/java/co/cask/cdap/master/environment/DefaultMasterEnvironmentContext.java (version 0b3778ea45e569e6bdd2ca530c2350a7f153718c)
- cdap-master/src/main/java/co/cask/cdap/master/environment/MasterEnvironmentExtensionLoader.java (version 0b3778ea45e569e6bdd2ca530c2350a7f153718c)
- cdap-master/src/main/java/co/cask/cdap/master/environment/k8s/AbstractServiceMain.java (version 0b3778ea45e569e6bdd2ca530c2350a7f153718c)
- cdap-master/src/main/java/co/cask/cdap/master/environment/k8s/MessagingServiceMain.java (version 0b3778ea45e569e6bdd2ca530c2350a7f153718c)
- cdap-master/src/main/java/co/cask/cdap/master/environment/k8s/MetricsServiceMain.java (version 0b3778ea45e569e6bdd2ca530c2350a7f153718c)
- cdap-master/src/test/java/co/cask/cdap/master/environment/MockMasterEnvironment.java (version 0b3778ea45e569e6bdd2ca530c2350a7f153718c)
- cdap-master/src/test/java/co/cask/cdap/master/environment/k8s/MasterMainTestBase.java (version 0b3778ea45e569e6bdd2ca530c2350a7f153718c)
- cdap-master/src/test/java/co/cask/cdap/master/environment/k8s/MessagingServiceMainTest.java (version 0b3778ea45e569e6bdd2ca530c2350a7f153718c)
- cdap-master/src/test/java/co/cask/cdap/master/environment/k8s/MetricsServiceMainTest.java (version 0b3778ea45e569e6bdd2ca530c2350a7f153718c)
- cdap-master/src/test/resources/META-INF/services/co.cask.cdap.master.spi.environment.MasterEnvironment (version 0b3778ea45e569e6bdd2ca530c2350a7f153718c)
- cdap-tms/src/main/java/co/cask/cdap/messaging/server/MessagingHttpService.java (version 0b3778ea45e569e6bdd2ca530c2350a7f153718c)
- pom.xml (version 0b3778ea45e569e6bdd2ca530c2350a7f153718c)
-
Andreas Neumann <andreas@cask.co> 1d3ef51cac52561a081be8e1812d28bc8ea68af0
Merge pull request #11019 from cdapio/feature/CDAP-14584-elastic-meta-storage
[CDAP-15584] Add an Elasticsearch Metadata Storage Provider -
Andreas Neumann a06bee86d09aeb7c8baa81d74aaf6f1c63d9e0d5 m
[CDAP-15584] Add an Elasticsearch Metadata Storage Provider
- cdap-api/src/main/java/co/cask/cdap/api/metadata/MetadataScope.java (version a06bee86d09aeb7c8baa81d74aaf6f1c63d9e0d5)
- cdap-data-fabric/src/main/java/co/cask/cdap/spi/metadata/Metadata.java (version a06bee86d09aeb7c8baa81d74aaf6f1c63d9e0d5)
- cdap-data-fabric/src/main/java/co/cask/cdap/spi/metadata/MetadataKind.java (version a06bee86d09aeb7c8baa81d74aaf6f1c63d9e0d5)
- cdap-data-fabric/src/main/java/co/cask/cdap/spi/metadata/MetadataStorage.java (version a06bee86d09aeb7c8baa81d74aaf6f1c63d9e0d5)
- cdap-data-fabric/src/main/java/co/cask/cdap/spi/metadata/ScopedName.java (version a06bee86d09aeb7c8baa81d74aaf6f1c63d9e0d5)
- cdap-data-fabric/src/main/java/co/cask/cdap/spi/metadata/ScopedNameOfKind.java (version a06bee86d09aeb7c8baa81d74aaf6f1c63d9e0d5)
- cdap-data-fabric/src/main/java/co/cask/cdap/spi/metadata/dataset/DatasetMetadataStorage.java (version a06bee86d09aeb7c8baa81d74aaf6f1c63d9e0d5)
- cdap-data-fabric/src/main/java/co/cask/cdap/spi/metadata/dataset/SearchHelper.java (version a06bee86d09aeb7c8baa81d74aaf6f1c63d9e0d5)
- cdap-data-fabric/src/main/java/co/cask/cdap/spi/metadata/noop/NoopMetadataStorage.java (version a06bee86d09aeb7c8baa81d74aaf6f1c63d9e0d5)
- cdap-data-fabric/src/test/java/co/cask/cdap/spi/metadata/MetadataStorageTest.java (version a06bee86d09aeb7c8baa81d74aaf6f1c63d9e0d5)
- cdap-data-fabric/src/test/java/co/cask/cdap/spi/metadata/dataset/DatasetMetadataStorageTest.java (version a06bee86d09aeb7c8baa81d74aaf6f1c63d9e0d5)
- cdap-elastic/src/main/java/co/cask/cdap/metadata/elastic/Cursor.java (version a06bee86d09aeb7c8baa81d74aaf6f1c63d9e0d5)
- cdap-elastic/src/main/java/co/cask/cdap/metadata/elastic/ElasticsearchMetadataStorage.java (version a06bee86d09aeb7c8baa81d74aaf6f1c63d9e0d5)
- cdap-elastic/src/main/java/co/cask/cdap/metadata/elastic/MetadataDocument.java (version a06bee86d09aeb7c8baa81d74aaf6f1c63d9e0d5)
- cdap-elastic/src/main/java/co/cask/cdap/metadata/elastic/ScopedNameTypeAdapter.java (version a06bee86d09aeb7c8baa81d74aaf6f1c63d9e0d5)
- cdap-elastic/src/main/resources/index.mapping.json (version a06bee86d09aeb7c8baa81d74aaf6f1c63d9e0d5)
- cdap-elastic/src/test/java/co/cask/cdap/metadata/elastic/ElasticsearchMetadataStorageTest.java (version a06bee86d09aeb7c8baa81d74aaf6f1c63d9e0d5)
-
Terence Yim <885032+chtyim@users.noreply.github.com> b8b6db70b596a20f3bdaa165b2e961ee28784387
Merge pull request #11024 from cdapio/feature/CDAP-14869-remove-direct-metric-store-usage
(CDAP-14869) Remove direct metric store usage -
albertshau <ashau@google.com> f68219323ef4becba2f6b5df95fc2f6b8fbd75d2
Merge pull request #11016 from cdapio/feature/CDAP-14855-healthcheck-endpoints-system-apps
CDAP-14855 add healthcheck endpoints to pipeline and report apps -
Edwin Elia <edwinelia@google.com> c771f12259b04e7b2addfd13f222d15b53bfce93
Merge pull request #11017 from cdapio/bugfix-ui/CDAP-14856-minimap-interaction-detail-view
[CDAP-14856] fix minimap interaction in pipeline detail view -
Terence Yim 2520decea9972880704baab98bed89d794820d76 m
(CDAP-14869) Remove direct usage of MetricStore in non-metric system code
- Separate out the TMS polling logic into two classes
- The generic polling logic doesn’t manage subscriber state
- The more specific subscriber service added subscriber state management
- Remove reliance on DatasetFramework for Metrics table
- Make MetricDeleteQuery serializable
- Introduce MetricsSystemClient and implementation
- Introduce MetricsAdminSubscriberService for admin messages processing
- Replace all production usages of MetricStore to MetricsSystemClient- cdap-app-fabric/src/main/java/co/cask/cdap/app/mapreduce/DistributedMRJobInfoFetcher.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-app-fabric/src/main/java/co/cask/cdap/app/mapreduce/LocalMRJobInfoFetcher.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-app-fabric/src/main/java/co/cask/cdap/gateway/handlers/WorkflowStatsSLAHttpHandler.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-app-fabric/src/main/java/co/cask/cdap/internal/app/deploy/LocalApplicationManager.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-app-fabric/src/main/java/co/cask/cdap/internal/app/deploy/pipeline/DeletedProgramHandlerStage.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-app-fabric/src/main/java/co/cask/cdap/internal/app/namespace/DefaultNamespaceResourceDeleter.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-app-fabric/src/main/java/co/cask/cdap/internal/app/services/ApplicationLifecycleService.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-app-fabric/src/main/java/co/cask/cdap/internal/profile/ProfileService.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-common/src/main/java/co/cask/cdap/common/conf/Constants.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-common/src/main/java/co/cask/cdap/common/metrics/NoOpMetricsSystemClient.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-common/src/main/resources/cdap-default.xml (version 2520decea9972880704baab98bed89d794820d76)
- cdap-data-fabric-tests/src/test/java/co/cask/cdap/data2/dataset2/lib/table/hbase/MetricHBaseTableUtilTest.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-data-fabric/src/main/java/co/cask/cdap/data/runtime/SystemDatasetRuntimeModule.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-data-fabric/src/main/java/co/cask/cdap/data2/dataset2/lib/table/hbase/HBaseMetricsTableDefinition.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-master/src/main/java/co/cask/cdap/data/runtime/main/MetricsProcessorTwillRunnable.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-master/src/main/java/co/cask/cdap/data/runtime/main/MetricsTwillRunnable.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-master/src/main/java/co/cask/cdap/data/tools/UpgradeTool.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-master/src/main/java/co/cask/cdap/master/startup/ConfigurationCheck.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-master/src/test/java/co/cask/cdap/master/startup/ConfigurationCheckTest.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-operational-stats-core/src/main/java/co/cask/cdap/operations/cdap/CDAPLoad.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-operational-stats-core/src/main/java/co/cask/cdap/operations/cdap/CDAPTransactions.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-tms/src/main/java/co/cask/cdap/messaging/subscriber/AbstractMessagingPollingService.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-tms/src/main/java/co/cask/cdap/messaging/subscriber/AbstractMessagingSubscriberService.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-watchdog-api/src/main/java/co/cask/cdap/api/metrics/MetricDeleteQuery.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-watchdog-api/src/main/java/co/cask/cdap/api/metrics/MetricsSystemClient.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-watchdog/src/main/java/co/cask/cdap/metrics/guice/DistributedMetricsClientModule.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-watchdog/src/main/java/co/cask/cdap/metrics/guice/MetricsClientRuntimeModule.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-watchdog/src/main/java/co/cask/cdap/metrics/process/DirectMetricsSystemClient.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-watchdog/src/main/java/co/cask/cdap/metrics/process/MessagingMetricsProcessorService.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-watchdog/src/main/java/co/cask/cdap/metrics/process/MetricsAdminMessage.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-watchdog/src/main/java/co/cask/cdap/metrics/process/MetricsAdminSubscriberService.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-watchdog/src/main/java/co/cask/cdap/metrics/process/MetricsConsumerMetaTable.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-watchdog/src/main/java/co/cask/cdap/metrics/process/RemoteMetricsSystemClient.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-watchdog/src/main/java/co/cask/cdap/metrics/store/DefaultMetricDatasetFactory.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-watchdog/src/main/java/co/cask/cdap/metrics/store/LocalMetricsDatasetFactory.java (version 2520decea9972880704baab98bed89d794820d76)
- cdap-watchdog/src/test/java/co/cask/cdap/metrics/process/MetricsAdminSubscriberServiceTest.java (version 2520decea9972880704baab98bed89d794820d76)
-
Edwin Elia <edwinelia@google.com> 916c1eaff2a794ec3d62812ed730d2d74bfeb806 m
[CDAP-14856] fix minimap interaction in pipeline detail view
- cdap-ui/app/directives/dag-minimap/dag-minimap-ctrl.js (version 916c1eaff2a794ec3d62812ed730d2d74bfeb806)
-
albertshau <ashau@google.com> 9770e847d05ca29b6022f4f10092a6c26051ed2b m
CDAP-14855 add healthcheck endpoints to pipeline and report apps
- cdap-app-templates/cdap-etl/cdap-data-pipeline/src/main/java/co/cask/cdap/datapipeline/service/ValidationHandler.java (version 9770e847d05ca29b6022f4f10092a6c26051ed2b)
- cdap-app-templates/cdap-program-report/src/main/java/co/cask/cdap/report/ReportGenerationSpark.java (version 9770e847d05ca29b6022f4f10092a6c26051ed2b)