Cask Data Application Platform - Develop Build and Test

Build: #1556 was successful Child of CDAP-DRC-2638

Code commits

CDAP

  • GitHub <noreply@github.com>

    GitHub <noreply@github.com> 7f42d979446fcd1e92f67c18fef3aac6dbfb5e74

    Copy cdap-kms and security extension jars to cdap master lib and to m… (#6146)
    * Copy cdap-kms and security extension jars to cdap master lib and to master ext

    * Added the new option to specify security extensions.

    • BUILD.rst (version 7f42d979446fcd1e92f67c18fef3aac6dbfb5e74)
    • cdap-master/pom.xml (version 7f42d979446fcd1e92f67c18fef3aac6dbfb5e74)
  • GitHub <noreply@github.com>

    GitHub <noreply@github.com> cf4ca23adbb3244ed5840b0ea3455a27150b6076

    Merge pull request #6110 from caskdata/feature/authentication-context
    (CDAP-6580) Added an AuthenticationContext that can be injected to de…

  • Bhooshan Mogal

    Bhooshan Mogal 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c

    (CDAP-6580) Added an AuthenticationContext that can be injected to determine the UserId depending on the context.
    - HTTPAuthenticationContext uses SecurityRequestContext to determine the UserId
    - ProgramContainerAuthenticationContext uses UserGroupInformation.getCurrentUser() to determine the UserId
    - Currently, AuthenticationContext is injected into RemoteDatasetFramework for DatasetServiceClient to use it while making requests to dataset service
    - However, it is also available as AuthorizationContext in Authorizer#initialize, so that Authorizer implementations like SentryAuthorizer can get the requesting UserId without CDAP passing it to every method (PR for that will be a follow-up)
    - Also added some missing tests for TwillRunnable (StreamHandler, Explore, Log Saver and MetricsProcessor) injectors, so injection failures can be caught in unit tests
    - Use AuthenticationContext in DatasetInstanceService instead of SecurityRequestContext
    - Fixed DatasetServiceTestBase
      - to rely more upon injector
      - to make tests faster by not initializing and restarting dataset service before every test
    - Refactored RemoteDatasetFrameworkTest to rely more upon injector

    • cdap-app-fabric/src/main/java/co/cask/cdap/app/guice/AppFabricServiceRuntimeModule.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-app-fabric/src/main/java/co/cask/cdap/app/guice/DistributedProgramRunnableModule.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-app-fabric/src/test/java/co/cask/cdap/internal/app/runtime/schedule/store/DatasetBasedTimeScheduleStoreTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-app-fabric/src/test/java/co/cask/cdap/stream/store/MDSStreamMetaStoreTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-common/src/main/java/co/cask/cdap/common/namespace/guice/NamespaceClientRuntimeModule.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-data-fabric-tests/src/test/java/co/cask/cdap/data/stream/DFSStreamFileJanitorTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-data-fabric-tests/src/test/java/co/cask/cdap/data/stream/DistributedStreamCoordinatorClientTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-data-fabric-tests/src/test/java/co/cask/cdap/data/stream/service/DFSStreamHeartbeatsTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-data-fabric-tests/src/test/java/co/cask/cdap/data2/transaction/distributed/TransactionServiceClientTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-data-fabric-tests/src/test/java/co/cask/cdap/data2/transaction/distributed/TransactionServiceTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-data-fabric/src/main/java/co/cask/cdap/data2/datafabric/dataset/DatasetServiceClient.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-data-fabric/src/main/java/co/cask/cdap/data2/datafabric/dataset/RemoteDatasetFramework.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-data-fabric/src/main/java/co/cask/cdap/data2/datafabric/dataset/service/DatasetInstanceService.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-data-fabric/src/test/java/co/cask/cdap/data/view/MDSViewStoreTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-data-fabric/src/test/java/co/cask/cdap/data2/datafabric/dataset/DatasetsUtilTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-data-fabric/src/test/java/co/cask/cdap/data2/datafabric/dataset/RemoteDatasetFrameworkTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-data-fabric/src/test/java/co/cask/cdap/data2/datafabric/dataset/service/DatasetInstanceHandlerTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-data-fabric/src/test/java/co/cask/cdap/data2/datafabric/dataset/service/DatasetInstanceServiceTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-data-fabric/src/test/java/co/cask/cdap/data2/datafabric/dataset/service/DatasetServiceAuthorizationTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-data-fabric/src/test/java/co/cask/cdap/data2/datafabric/dataset/service/DatasetServiceTestBase.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-data-fabric/src/test/java/co/cask/cdap/data2/datafabric/dataset/service/DatasetTypeHandlerTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-data-fabric/src/test/java/co/cask/cdap/data2/datafabric/dataset/service/executor/DatasetOpExecutorServiceTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-data-fabric/src/test/java/co/cask/cdap/data2/dataset2/lib/table/leveldb/LevelDBTableServiceTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-data-fabric/src/test/java/co/cask/cdap/data2/dataset2/lib/table/leveldb/LevelDBTableTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-data-fabric/src/test/java/co/cask/cdap/data2/transaction/queue/LocalQueueTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-data-fabric/src/test/java/co/cask/cdap/data2/transaction/queue/leveldb/LevelDBQueueTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-explore/src/main/java/co/cask/cdap/hive/context/ContextManager.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-explore/src/test/java/co/cask/cdap/explore/service/BaseHiveExploreServiceTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-explore/src/test/java/co/cask/cdap/explore/service/ExploreDisabledTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-explore/src/test/java/co/cask/cdap/explore/service/InMemoryExploreServiceTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-gateway/src/test/java/co/cask/cdap/gateway/handlers/metrics/MetricsSuiteTestBase.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-gateway/src/test/java/co/cask/cdap/gateway/run/MetricsTwillRunnableTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-master/src/main/java/co/cask/cdap/data/runtime/main/DatasetOpExecutorServerTwillRunnable.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-master/src/main/java/co/cask/cdap/data/runtime/main/ExploreServiceTwillRunnable.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-master/src/main/java/co/cask/cdap/data/runtime/main/StreamHandlerRunnable.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-master/src/main/java/co/cask/cdap/data/runtime/main/TransactionServiceTwillRunnable.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-master/src/main/java/co/cask/cdap/data/tools/DatasetServiceManager.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-master/src/main/java/co/cask/cdap/data/tools/UpgradeTool.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-master/src/test/java/co/cask/cdap/data/runtime/main/DatasetOpExecutorServerTwillRunnableTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-master/src/test/java/co/cask/cdap/data/runtime/main/TwillRunnableTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-notifications/src/test/java/co/cask/cdap/notifications/NotificationTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-security-spi/src/main/java/co/cask/cdap/security/spi/authentication/AuthenticationContext.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-security-spi/src/main/java/co/cask/cdap/security/spi/authorization/AuthorizationContext.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-security/src/main/java/co/cask/cdap/security/auth/context/AuthenticationContextModules.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-security/src/main/java/co/cask/cdap/security/auth/context/AuthenticationTestContext.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-security/src/main/java/co/cask/cdap/security/auth/context/MasterAuthenticationContext.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-security/src/main/java/co/cask/cdap/security/auth/context/ProgramContainerAuthenticationContext.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-security/src/main/java/co/cask/cdap/security/authorization/DefaultAuthorizationContext.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-security/src/test/java/co/cask/cdap/security/authorization/NoOpAuthorizationContextFactory.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-watchdog/src/main/java/co/cask/cdap/logging/run/LogSaverTwillRunnable.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-watchdog/src/main/java/co/cask/cdap/metrics/runtime/MetricsProcessorTwillRunnable.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-watchdog/src/main/java/co/cask/cdap/metrics/runtime/MetricsTwillRunnable.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-watchdog/src/test/java/co/cask/cdap/logging/appender/TestResilientLogging.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-watchdog/src/test/java/co/cask/cdap/logging/run/LogSaverTwillRunnableTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
    • cdap-watchdog/src/test/java/co/cask/cdap/metrics/runtime/MetricsTwillRunnableTest.java (version 6478384c63ecc0dc0f98c4ffd6652d0b41ff613c)
  • sreevatsan raman <sreevatsan@gmail.com>

    sreevatsan raman <sreevatsan@gmail.com> 476aa9204aef956b7ebb5223640faa50ed29c40f

    Remove Tephra and replace with "Apache Tephra" as appropriate. (#6130)

    • cdap-docs/_common/_themes/cdap/casksites.html (version 476aa9204aef956b7ebb5223640faa50ed29c40f)
    • cdap-docs/developers-manual/source/building-blocks/transaction-system.rst (version 476aa9204aef956b7ebb5223640faa50ed29c40f)
    • cdap-docs/developers-manual/source/overview/abstractions.rst (version 476aa9204aef956b7ebb5223640faa50ed29c40f)
    • cdap-docs/reference-manual/source/licenses/index.rst (version 476aa9204aef956b7ebb5223640faa50ed29c40f)
  • GitHub <noreply@github.com>

    GitHub <noreply@github.com> 80d643758db5c80f4e44c3594b7d1c329d08c997

    Merge pull request #6135 from caskdata/docs/3.5_5890_FAQ_Audit_Log
    CDAP-5890, 6101 FAQ entries

  • John Jackson

    John Jackson e1c71c11f4424e95dceaccdd12a8a503b8e6370c

    Fix build by removing Script plugin, now deleted in hydrator-plugins.

    • cdap-docs/cdap-apps/build.sh (version e1c71c11f4424e95dceaccdd12a8a503b8e6370c)
  • sreevatsan raman <sreevatsan@gmail.com>

    sreevatsan raman <sreevatsan@gmail.com> ba2066470fd7a787de0fc52b07b906dcd2e58c00

    CDAP-6606 Avoid caching the FileContext object in FileContextLocationFactory, when the calling UGI is different than the cached FileContext's UGI. (#6142)

    • cdap-common/src/main/java/org/apache/twill/filesystem/FileContextLocationFactory.java (version ba2066470fd7a787de0fc52b07b906dcd2e58c00)
  • John Jackson

    John Jackson bd042132365b5f71a48729419d2462e7c3b9261b

    FAQ entries

    • cdap-docs/faqs/source/databases-transactions.txt (version bd042132365b5f71a48729419d2462e7c3b9261b)
    • cdap-docs/faqs/source/general.rst (version bd042132365b5f71a48729419d2462e7c3b9261b)