Build #3,433

Cask Data Application Platform - Develop Build and Test

Build: #3433 was successful Child of CDAP-DRC-5596

Code commits

caskdata/cdap

  • rahul1995 <rljain@google.com>

    rahul1995 <rljain@google.com> acfbeda1fcd524e4667f266420954679f25e1f7c

    Merge pull request #13688 from cdapio/feature/CDAP-18532-config-declared-errors
    CDAP-18532: Provide a way to declare server error through Configuration file

  • Rahul Jain <rljain@google.com>

    Rahul Jain <rljain@google.com> 8ddadae452052661a0597ca4326b308ddeb172e6 m

    Define config-reloading interval in seconds instead of minutes, and 0 interval should be treated as a way to disable config-reload scheduling

    • cdap-common/src/main/java/io/cdap/cdap/common/conf/Constants.java (version 8ddadae452052661a0597ca4326b308ddeb172e6)
    • cdap-common/src/main/resources/cdap-default.xml (version 8ddadae452052661a0597ca4326b308ddeb172e6)
    • cdap-gateway/src/main/java/io/cdap/cdap/gateway/router/NettyRouter.java (version 8ddadae452052661a0597ca4326b308ddeb172e6)
    • cdap-gateway/src/test/java/io/cdap/cdap/gateway/router/NettyRouterTestBase.java (version 8ddadae452052661a0597ca4326b308ddeb172e6)
  • wyzhang <wyzhang@google.com>

    wyzhang <wyzhang@google.com> 4ca718da41c6ecbd1ab644fba264572308ee4c54

    Merge pull request #13695 from cdapio/feature/CDAP-18542/wyzhang-metrics-levelDB-compaction
    [CDAP-18542] Adding a periodic full compaction to LevelDBTableService.

  • Wangyuan Zhang <wyzhang@google.com>

    Wangyuan Zhang <wyzhang@google.com> 9702a5f9e821b227ef6bbbd1b5aea88cbb887ba4 m

    [CDAP-18542] Adding a periodic full compaction to LevelDBTableService.
    What:
    Adding a periodic full compaction across tables managed by LevelDBTableService.
    This background periodic compaction is controlled by a cdap-site configuration.
    Typically this the comapction interval should be a few days (e.g. 7 days) since
    it could be a heavy disk operation depending on the size of the table.

    Why:
    LevelDB by default compacts across level N and subset of files in level N+1, this could
    potentially lead to deletion-marker-masked values in N+1 or N+2 not being discarded
    after the compaction, thus degrading read performance and causing disk overhead.

    • cdap-common/src/main/java/io/cdap/cdap/common/conf/Constants.java (version 9702a5f9e821b227ef6bbbd1b5aea88cbb887ba4)
    • cdap-data-fabric/src/main/java/io/cdap/cdap/data2/dataset2/lib/table/leveldb/LevelDBTableService.java (version 9702a5f9e821b227ef6bbbd1b5aea88cbb887ba4)
    • cdap-data-fabric/src/test/java/io/cdap/cdap/data2/dataset2/lib/table/leveldb/LevelDBTableServiceTest.java (version 9702a5f9e821b227ef6bbbd1b5aea88cbb887ba4)
  • Rahul Jain <rljain@google.com>

    Rahul Jain <rljain@google.com> fbb3ad3e621dd2c03193c9af9c904afd3946f15c m

    Refactoring: Moved ConfigBasedRequestBlocking constants inside Router constants class

    • cdap-common/src/main/java/io/cdap/cdap/common/conf/Constants.java (version fbb3ad3e621dd2c03193c9af9c904afd3946f15c)
    • cdap-common/src/main/resources/cdap-default.xml (version fbb3ad3e621dd2c03193c9af9c904afd3946f15c)
    • cdap-gateway/src/main/java/io/cdap/cdap/gateway/router/NettyRouter.java (version fbb3ad3e621dd2c03193c9af9c904afd3946f15c)
    • cdap-gateway/src/main/java/io/cdap/cdap/gateway/router/handlers/ConfigBasedRequestBlockingHandler.java (version fbb3ad3e621dd2c03193c9af9c904afd3946f15c)
    • cdap-gateway/src/test/java/io/cdap/cdap/gateway/router/ConfigBasedRequestBlockingTest.java (version fbb3ad3e621dd2c03193c9af9c904afd3946f15c)
  • Rahul Jain <rljain@google.com>

    Rahul Jain <rljain@google.com> 5db27a1b7e12994daa9cbf7f85d9052c5d40e6e7 m

    Add test-cases for Config-based request-blocking

    • cdap-gateway/pom.xml (version 5db27a1b7e12994daa9cbf7f85d9052c5d40e6e7)
    • cdap-gateway/src/test/java/io/cdap/cdap/gateway/router/ConfigBasedRequestBlockingTest.java (version 5db27a1b7e12994daa9cbf7f85d9052c5d40e6e7)
    • cdap-gateway/src/test/java/io/cdap/cdap/gateway/router/NettyRouterTestBase.java (version 5db27a1b7e12994daa9cbf7f85d9052c5d40e6e7)
  • Rahul Jain <rljain@google.com>

    Rahul Jain <rljain@google.com> 368042997cccc40749710157c31ace7e6270f03c m

    Rename config-declared errors to config-based request-blocking to generalize the concept in non-error cases

    • cdap-common/src/main/java/io/cdap/cdap/common/conf/Constants.java (version 368042997cccc40749710157c31ace7e6270f03c)
    • cdap-common/src/main/resources/cdap-default.xml (version 368042997cccc40749710157c31ace7e6270f03c)
    • cdap-gateway/src/main/java/io/cdap/cdap/gateway/router/NettyRouter.java (version 368042997cccc40749710157c31ace7e6270f03c)
    • cdap-gateway/src/main/java/io/cdap/cdap/gateway/router/handlers/ConfigBasedRequestBlockingHandler.java (version 368042997cccc40749710157c31ace7e6270f03c)
    • cdap-gateway/src/main/java/io/cdap/cdap/gateway/router/handlers/ConfigDeclaredErrorHandler.java (version 368042997cccc40749710157c31ace7e6270f03c)
  • Rahul Jain <rljain@google.com>

    Rahul Jain <rljain@google.com> 98600bd426ccb18cd6f60904e99fd23db8f53bfb m

    Move default configs from Constants.java to cdap-default.xml

    • cdap-common/src/main/java/io/cdap/cdap/common/conf/Constants.java (version 98600bd426ccb18cd6f60904e99fd23db8f53bfb)
    • cdap-common/src/main/resources/cdap-default.xml (version 98600bd426ccb18cd6f60904e99fd23db8f53bfb)
    • cdap-gateway/src/main/java/io/cdap/cdap/gateway/router/handlers/ConfigDeclaredErrorHandler.java (version 98600bd426ccb18cd6f60904e99fd23db8f53bfb)
  • Rahul Jain <rljain@google.com>

    Rahul Jain <rljain@google.com> ba53d813aca12950c84ff1c4cdc23a6253cf5eb1 m

    Handle config-declared errors and reload Router periodically to incorporate new changes in cconf

    • cdap-common/src/main/java/io/cdap/cdap/common/conf/Constants.java (version ba53d813aca12950c84ff1c4cdc23a6253cf5eb1)
    • cdap-common/src/main/resources/cdap-default.xml (version ba53d813aca12950c84ff1c4cdc23a6253cf5eb1)
    • cdap-gateway/src/main/java/io/cdap/cdap/gateway/router/NettyRouter.java (version ba53d813aca12950c84ff1c4cdc23a6253cf5eb1)
    • cdap-gateway/src/main/java/io/cdap/cdap/gateway/router/handlers/ConfigDeclaredErrorHandler.java (version ba53d813aca12950c84ff1c4cdc23a6253cf5eb1)