Build #344

Build: #344 was successful

Job: Unit Test Job was successful

Stages & jobs

  1. Unit Test Stage

Code commits

CDAP

  • Terence Yim

    Terence Yim 2fb9dfa5c4c63d454f607be5f4f9ddd838dc52b2

    Merge pull request #9980 from caskdata/feature/merge-4.3
    Feature/merge 4.3

  • Terence Yim

    Terence Yim a935eeb5bfd5585ff27523137981a0737e9ea420 m

    Merge remote-tracking branch 'origin/release/4.3' into feature/merge-4.3

  • albertshau <albert@cask.co>

    albertshau <albert@cask.co> fa3451f1ac38591838ff537caad73878ee8da00b m

    Merge pull request #9982 from caskdata/fix-race-in-test
    fix race in creating app.meta dataset in test

  • Albert Shau <albert@cask.co>

    Albert Shau <albert@cask.co> 15ef6d1effef0457563a2fa38bbd4246c98c8300 m

    fix race in creating app.meta dataset in test
    Both the test and the service itself create the dataset. Use
    getOrCreate so there is no conflict exception.

    • cdap-app-fabric/src/test/java/co/cask/cdap/internal/app/services/ProgramNotificationSubscriberServiceTest.java (version 15ef6d1effef0457563a2fa38bbd4246c98c8300)
    • cdap-data-fabric/src/main/java/co/cask/cdap/data2/datafabric/dataset/DatasetsUtil.java (version 15ef6d1effef0457563a2fa38bbd4246c98c8300)
  • Terence Yim

    Terence Yim e48c7631e419953f22451c1587c70a9cb56d7f3b m

    Fix the wrong conflict resolution and compilation error

    • cdap-app-fabric/src/main/java/co/cask/cdap/internal/app/store/AppMetadataStore.java (version e48c7631e419953f22451c1587c70a9cb56d7f3b)
    • cdap-app-fabric/src/test/java/co/cask/cdap/internal/app/services/ProgramNotificationSubscriberServiceTest.java (version e48c7631e419953f22451c1587c70a9cb56d7f3b)
    • cdap-data-fabric/src/main/java/co/cask/cdap/data2/dataset2/lib/partitioned/PartitionedFileSetDefinition.java (version e48c7631e419953f22451c1587c70a9cb56d7f3b)
  • Terence Yim

    Terence Yim 66bd42d992f96a63d9cb4122aae4961acee4d425 m

    Merge branch 'release/4.3' into feature/merge-4.3

  • Rohit Sinha

    Rohit Sinha 689c995f8c49e6bae8dedb75397393ba9cded0ce m

    Merge pull request #9973 from caskdata/feature/CDAP-13154-auth-imp
    CDAP-13154 Use just the username and not kerberos principal for auth…

  • Rohit Sinha

    Rohit Sinha 17e838dd0ffe3b5872035435b33113594f2f88a3 m

    CDAP-13154 Use just the username and not kerberos principal for authorization

    • cdap-app-fabric/src/main/java/co/cask/cdap/internal/app/namespace/DefaultNamespaceAdmin.java (version 17e838dd0ffe3b5872035435b33113594f2f88a3)
    • cdap-data-fabric/src/main/java/co/cask/cdap/data2/datafabric/dataset/DatasetServiceClient.java (version 17e838dd0ffe3b5872035435b33113594f2f88a3)
    • cdap-security/src/main/java/co/cask/cdap/security/authorization/AuthorizationUtil.java (version 17e838dd0ffe3b5872035435b33113594f2f88a3)
    • cdap-security/src/test/java/co/cask/cdap/security/authorization/AuthorizationUtilTest.java (version 17e838dd0ffe3b5872035435b33113594f2f88a3)
  • Bhooshan Mogal

    Bhooshan Mogal 0a89dc000f7a9ca597bd2d4ac78a4130538dd98b m

    Merge pull request #9978 from caskdata/bugfix/format-date
    (CDAP-13223) Fixed single quotes in format strings

  • Bhooshan Mogal

    Bhooshan Mogal 88ce7ee3c3b763e19642ac7f8f2f511060fd3ff4 m

    (CDAP-13223) Fixed single quotes in format strings
    - Fixed format date directive to lose un-necessary single quote
    - Removed on ENTER to submit a directive while concatenating string to a column
    - Added a missing format, which caused the ordering between text and code to be messed up

    • cdap-ui/app/cdap/components/DataPrep/Directives/Format/index.js (version 88ce7ee3c3b763e19642ac7f8f2f511060fd3ff4)
    • cdap-ui/app/cdap/components/DataPrep/Directives/Parse/Modals/SimpleDateModal.js (version 88ce7ee3c3b763e19642ac7f8f2f511060fd3ff4)
    • cdap-ui/app/cdap/text/text-en.yaml (version 88ce7ee3c3b763e19642ac7f8f2f511060fd3ff4)
  • Terence Yim

    Terence Yim 740c9c2aefeda1dd5f667ab066cbe4a86fda9bce m

    Merge pull request #9977 from caskdata/feature/CDAP-13218-fix-workflow-state
    (CDAP-13218) Distinguish program state based on user actions

  • Terence Yim

    Terence Yim 91345fac0cd8a1b747a7ba0be791669a1da02cef m

    (CDAP-13218) Distinguish program state based on user actions
    - Fix for Workflow, MR, and Spark that when user initiate a stop, emit KILLED instead of FAILED
    - Ignore program run status event that won’t change the state in run record

    • cdap-app-fabric/src/main/java/co/cask/cdap/common/twill/TwillAppLifecycleEventHandler.java (version 91345fac0cd8a1b747a7ba0be791669a1da02cef)
    • cdap-app-fabric/src/main/java/co/cask/cdap/internal/app/store/AppMetadataStore.java (version 91345fac0cd8a1b747a7ba0be791669a1da02cef)
  • albertshau <albert@cask.co>

    albertshau <albert@cask.co> 84336f909fee1bf5a6993680abedad6ff4fbdd8a m

    Merge pull request #9975 from caskdata/bugfix_release/CDAP-13191-fix-completed-run-record-read
    CDAP-13191 fix bug in reading existing run records

  • Terence Yim

    Terence Yim 0bfd935f333c2bee04d30d672d5611edde512065 m

    Merge pull request #9956 from caskdata/feature/CDAP-13179-am-resilience
    (CDAP-13179) Integrates TWILL-61 changes for AM resilience

  • Yaojie Feng

    Yaojie Feng 8cc156c7a5d8f8f34cf8d409dc612ecba21681ae m

    Merge pull request #9945 from caskdata/feature_release/log-login-user-who-starts-the-program
    log user who executes the program

  • Yaojie Feng

    Yaojie Feng 3c0dec65256a8293b7bfd3124df9441881abb7fc m

    log user who executes the program

    • cdap-app-fabric/src/main/java/co/cask/cdap/internal/app/runtime/distributed/DistributedFlowProgramRunner.java (version 3c0dec65256a8293b7bfd3124df9441881abb7fc)
    • cdap-app-fabric/src/main/java/co/cask/cdap/internal/app/services/ProgramLifecycleService.java (version 3c0dec65256a8293b7bfd3124df9441881abb7fc)
    • cdap-spark-core-base/src/main/java/co/cask/cdap/app/runtime/spark/distributed/DistributedSparkProgramRunner.java (version 3c0dec65256a8293b7bfd3124df9441881abb7fc)
  • albertshau <albert@cask.co>

    albertshau <albert@cask.co> 04edc152787e138a8188ec6d605158c969cbeb06 m

    Merge pull request #9974 from caskdata/bugfix_release/CDAP-13217-remove-program-check-state-store
    CDAP-13217 remove app spec check when recording run state

  • Albert Shau <albert@cask.co>

    Albert Shau <albert@cask.co> 03b647a2fbd2952a3440f8c307e356fea6510934 m

    rename method and mark argument as nullable

    • cdap-app-fabric/src/main/java/co/cask/cdap/internal/app/store/AppMetadataStore.java (version 03b647a2fbd2952a3440f8c307e356fea6510934)
    • cdap-app-fabric/src/test/java/co/cask/cdap/internal/app/store/DefaultStoreTest.java (version 03b647a2fbd2952a3440f8c307e356fea6510934)
  • Albert Shau <albert@cask.co>

    Albert Shau <albert@cask.co> 28fe532ca8e9d63ade78fbe10bc0b7526563fda2 m

    CDAP-13217 remove app spec check when recording run state
    When we are recording run record state, we should not check for
    existence of the application or program. Otherwise, there can
    be a situation where an app is modified to remove a running
    program. If that program is killed on the cluster, then the run
    record will be in the 'running' state, which will eventually
    be fixed by the run record corrector. However, when it is time
    to save the run state, the program will not be found and the
    state update will be ignored. This will loop infinitely.

    • cdap-app-fabric/src/main/java/co/cask/cdap/internal/app/services/ProgramNotificationSubscriberService.java (version 28fe532ca8e9d63ade78fbe10bc0b7526563fda2)
    • cdap-app-fabric/src/main/java/co/cask/cdap/internal/app/services/RunRecordCorrectorService.java (version 28fe532ca8e9d63ade78fbe10bc0b7526563fda2)
    • cdap-app-fabric/src/test/java/co/cask/cdap/internal/app/services/ProgramNotificationSubscriberServiceTest.java (version 28fe532ca8e9d63ade78fbe10bc0b7526563fda2)
  • Albert Shau <albert@cask.co>

    Albert Shau <albert@cask.co> f94ea55265aa32b4e9e1429a76b8690bf9f81873 m

    CDAP-13191 fix bug in reading existing run records
    Completed run records use a different rowkey than the other types
    of run records. The previous code had a different and buggy method
    to read existing run records, where the completed run record key
    was missing the inverted timestamp part.

    Fixed the code to use the working existing method to read a run
    record for a given program run id.

    • cdap-app-fabric/src/main/java/co/cask/cdap/internal/app/store/AppMetadataStore.java (version f94ea55265aa32b4e9e1429a76b8690bf9f81873)
  • sagarkapare <sagar@cask.co>

    sagarkapare <sagar@cask.co> 39aa9ffd33cf4d70d1b41a894056e2c1ff7e755b m

    CDAP-13205 Fix inconsistency when the Job is deleted but Trigger still exist in the schedulestore.

    • cdap-app-fabric/src/main/java/co/cask/cdap/internal/app/runtime/schedule/store/DatasetBasedTimeScheduleStore.java (version 39aa9ffd33cf4d70d1b41a894056e2c1ff7e755b)
    • cdap-app-fabric/src/test/java/co/cask/cdap/internal/app/runtime/schedule/store/DatasetBasedTimeScheduleStoreTest.java (version 39aa9ffd33cf4d70d1b41a894056e2c1ff7e755b)
  • sagarkapare <sagar@cask.co>

    sagarkapare <sagar@cask.co> 7178ff89c58eb6553b8e16a894d6271ddabdd2d0 m

    Merge pull request #9969 from caskdata/bugfix_release/cdap-13205
    CDAP-13205 Fix inconsistency when the Job is deleted but Trigger still exist in the schedulestore.

  • Terence Yim

    Terence Yim 88ee552239d0f872a20d8efb0b43e99fd777bc3b m

    (CDAP-13179) Integrates TWILL-61 changes for AM resilience

    • cdap-app-fabric/src/main/java/co/cask/cdap/internal/app/runtime/SystemArguments.java (version 88ee552239d0f872a20d8efb0b43e99fd777bc3b)
    • cdap-app-fabric/src/main/java/co/cask/cdap/internal/app/runtime/distributed/DistributedFlowProgramRunner.java (version 88ee552239d0f872a20d8efb0b43e99fd777bc3b)
    • cdap-app-fabric/src/main/java/co/cask/cdap/internal/app/runtime/distributed/DistributedProgramRunner.java (version 88ee552239d0f872a20d8efb0b43e99fd777bc3b)
    • cdap-app-fabric/src/main/java/co/cask/cdap/internal/app/runtime/distributed/DistributedServiceProgramRunner.java (version 88ee552239d0f872a20d8efb0b43e99fd777bc3b)
    • cdap-app-fabric/src/main/java/co/cask/cdap/internal/app/runtime/distributed/LongRunningDistributedProgramRunner.java (version 88ee552239d0f872a20d8efb0b43e99fd777bc3b)
    • cdap-common/src/main/java/co/cask/cdap/common/conf/Constants.java (version 88ee552239d0f872a20d8efb0b43e99fd777bc3b)
    • cdap-common/src/main/resources/cdap-default.xml (version 88ee552239d0f872a20d8efb0b43e99fd777bc3b)
    • cdap-master/src/main/java/co/cask/cdap/data/runtime/main/MasterServiceMain.java (version 88ee552239d0f872a20d8efb0b43e99fd777bc3b)
    • pom.xml (version 88ee552239d0f872a20d8efb0b43e99fd777bc3b)
  • Ajai Narayan

    Ajai Narayan 6e38610e9f7d1ffc1eed6954b4878250fd5b6f9f m

    Merge pull request #9960 from caskdata/bugfix_release/ui-CDAP-13162
    [CDAP-13162] Retain code editors if macro'd

  • Ajai Narayan

    Ajai Narayan fb4d935dc41ffbed45f938c09951401f0e61b863 m

    [CDAP-13162] Retain code editors if macro'd in plugin configuration in pipelines
     - Fixes code editors in plugin config to not swap with input textboxes

    • cdap-ui/app/directives/macro-widget-toggle/macro-widget-toggle.html (version fb4d935dc41ffbed45f938c09951401f0e61b863)
    • cdap-ui/app/directives/macro-widget-toggle/macro-widget-toggle.js (version fb4d935dc41ffbed45f938c09951401f0e61b863)
  • Ajai Narayan

    Ajai Narayan 39cf43b72833b062ea3f4d71347321d2b6f18e07 m

    Merge pull request #9950 from caskdata/bugfix/ui-fix-node-uuids-for-dom-4.3.4
    Fix references of node-uuid used for id for DOM elements

  • Ajai Narayan

    Ajai Narayan d5425ef81f590d97291c28331adec376bc8e20a5 m

    Fix references of node-uuid used for id for DOM elements
     - More patches to node uuids being set as ids for DOM elements
     - Fixes btn color for actions in dataset/stream detailed view in tracker

    • cdap-ui/app/cdap/components/FastAction/DeleteAction/index.js (version d5425ef81f590d97291c28331adec376bc8e20a5)
    • cdap-ui/app/cdap/components/FastAction/ExploreAction/ExploreModal.js (version d5425ef81f590d97291c28331adec376bc8e20a5)
    • cdap-ui/app/cdap/components/FastAction/ExploreAction/index.js (version d5425ef81f590d97291c28331adec376bc8e20a5)
    • cdap-ui/app/cdap/components/FastAction/SendEventAction/index.js (version d5425ef81f590d97291c28331adec376bc8e20a5)
    • cdap-ui/app/cdap/components/FastAction/SetPreferenceAction/index.js (version d5425ef81f590d97291c28331adec376bc8e20a5)
    • cdap-ui/app/cdap/components/FastAction/TruncateAction/index.js (version d5425ef81f590d97291c28331adec376bc8e20a5)
    • cdap-ui/app/cdap/components/Overview/OverviewMetaSection/index.js (version d5425ef81f590d97291c28331adec376bc8e20a5)
    • cdap-ui/app/cdap/components/ProgramTable/index.js (version d5425ef81f590d97291c28331adec376bc8e20a5)
    • cdap-ui/app/cdap/components/UncontrolledComponents/Popover.js (version d5425ef81f590d97291c28331adec376bc8e20a5)
    • cdap-ui/app/tracker/directives/jump/jump.less (version d5425ef81f590d97291c28331adec376bc8e20a5)
  • Terence Yim

    Terence Yim 5ff02702a9aee52909abcf24a53922205b25809a m

    Merge pull request #9948 from caskdata/feature/CDAP-13158-fix-spark-conf-zip-race
    (CDAP-13158) Fix a race condition when running concurrent spark in wo…

  • anew <andreas@cask.co>

    anew <andreas@cask.co> 77ad17ef12e0a23884e55af1ce4eeeb747a0efe1 m

    [CDAP-13171] Only set group on namespace directories if the group was explicitly configured

    • cdap-app-fabric/src/main/java/co/cask/cdap/internal/app/namespace/AbstractStorageProviderNamespaceAdmin.java (version 77ad17ef12e0a23884e55af1ce4eeeb747a0efe1)
    • cdap-docs/reference-manual/source/http-restful-api/namespace.rst (version 77ad17ef12e0a23884e55af1ce4eeeb747a0efe1)
  • Terence Yim

    Terence Yim aae3ad5c32e791f44cee9e44a28f552d79269289 m

    (CDAP-13158) Fix a race condition when running concurrent spark in workflow

    • cdap-spark-core-base/src/main/java/co/cask/cdap/app/runtime/spark/SparkRuntimeUtils.java (version aae3ad5c32e791f44cee9e44a28f552d79269289)
    • cdap-spark-core-base/src/main/java/co/cask/cdap/app/runtime/spark/classloader/SparkClassRewriter.java (version aae3ad5c32e791f44cee9e44a28f552d79269289)
    • cdap-spark-core-base/src/test/java/co/cask/cdap/app/runtime/spark/SparkRuntimeUtilsTest.java (version aae3ad5c32e791f44cee9e44a28f552d79269289)
  • albertshau <albert@cask.co>

    albertshau <albert@cask.co> 106144e704948965323ea9b45896fb8d97180b9f m

    Merge pull request #9932 from caskdata/bugfix_release/CDAP-13120-leftover-local-filesets
    CDAP-13120 use correct directory for file in PartitionedFileSet

  • Ajai Narayan

    Ajai Narayan 3a12659c3201e7786df1bed8cd1c08627fd27ec6 m

    Merge pull request #9934 from caskdata/bugfix_release/ui-CDAP-13112
    [CDAP-13111] - Fix performance issue with pipeline detailed view

  • Ajai Narayan

    Ajai Narayan e02c56b38216f71f4e469a94edd348c2843d7a87 m

    [CDAP-13111] Fixes performance of pipeline detailed view for a large pipeline + minor styling fix for run level section in pipeline detailed view
     - Fixes metrics for pipeline nodes to have proper ellipsis + renames a filter based on its function
     - Simplifies comma-separated-number filter to just add commas to a number (removed ellipsis adding functionality)

    • cdap-ui/app/directives/node-metrics/node-metrics.html (version e02c56b38216f71f4e469a94edd348c2843d7a87)
    • cdap-ui/app/directives/node-metrics/node-metrics.js (version e02c56b38216f71f4e469a94edd348c2843d7a87)
    • cdap-ui/app/filters/comma-separated-number.js (version e02c56b38216f71f4e469a94edd348c2843d7a87)
    • cdap-ui/app/filters/leading-ellipsis.js (version e02c56b38216f71f4e469a94edd348c2843d7a87)
    • cdap-ui/app/filters/number-with-ellipsis.js (version e02c56b38216f71f4e469a94edd348c2843d7a87)
    • cdap-ui/app/hydrator/adapters.less (version e02c56b38216f71f4e469a94edd348c2843d7a87)
    • cdap-ui/app/hydrator/templates/partial/metrics-popover.html (version e02c56b38216f71f4e469a94edd348c2843d7a87)
  • Sreevatsan Raman

    Sreevatsan Raman 33662d0588d5ebf8a1f9a549d558f8da9585bbdc m

    Merge pull request #9937 from caskdata/feature/cdap-13110-fix-build
    fix build break

  • anew <andreas@cask.co>

    anew <andreas@cask.co> 9486adc2843b762519836ae8bd5a952a41f39f48 m

    fix build break

    • cdap-data-fabric/src/main/java/co/cask/cdap/data2/datafabric/dataset/service/executor/DatasetAdminService.java (version 9486adc2843b762519836ae8bd5a952a41f39f48)
  • anew <andreas@cask.co>

    anew <andreas@cask.co> 20fc691d3ffc3d576126371160be8e0a5f804a74 m

    fix an error message

    • cdap-data-fabric/src/main/java/co/cask/cdap/data2/dataset2/lib/file/FileSetDefinition.java (version 20fc691d3ffc3d576126371160be8e0a5f804a74)
  • anew <andreas@cask.co>

    anew <andreas@cask.co> 57d3cb319deb8ae24bf61d99a1a248287dc5d537 m

    [CDAP-13110] Fix dataset class loading in some places

    • cdap-data-fabric/src/main/java/co/cask/cdap/data2/datafabric/dataset/service/executor/DatasetAdminService.java (version 57d3cb319deb8ae24bf61d99a1a248287dc5d537)
    • cdap-explore/src/main/java/co/cask/cdap/explore/executor/ExploreExecutorHttpHandler.java (version 57d3cb319deb8ae24bf61d99a1a248287dc5d537)
    • cdap-explore/src/main/java/co/cask/cdap/explore/service/ExploreTableManager.java (version 57d3cb319deb8ae24bf61d99a1a248287dc5d537)
  • Albert Shau <albert@cask.co>

    Albert Shau <albert@cask.co> 4096b2b42ad8ed0b36bf98f58446e19414af95fe m

    CDAP-13120 fix unit tests

    • cdap-data-fabric/src/test/java/co/cask/cdap/api/dataset/lib/SystemDatasetDefinitionTest.java (version 4096b2b42ad8ed0b36bf98f58446e19414af95fe)
    • cdap-data-fabric/src/test/java/co/cask/cdap/data2/datafabric/dataset/DatasetsUtilTest.java (version 4096b2b42ad8ed0b36bf98f58446e19414af95fe)
    • cdap-data-fabric/src/test/java/co/cask/cdap/data2/dataset2/lib/partitioned/PartitionedFileSetTest.java (version 4096b2b42ad8ed0b36bf98f58446e19414af95fe)
  • Sreevatsan Raman

    Sreevatsan Raman 4acec72d78c044359cf94d263f433cb31ac076c1 m

    Merge pull request #9921 from caskdata/bugfix_release/CDAP-13129-ensure-sink-in-single-mapred
    CDAP-13129 ensure a sink is only ever in one mapreduce phase

  • Albert Shau <albert@cask.co>

    Albert Shau <albert@cask.co> 52deb03874d791689fe5d368c3b81642fc8574c4 m

    CDAP-13120 make defaulting base path backwards compatible
    Modify the reconfigure logic so that the base path is not
    defaulted unless it was defaulted when it was created. Also adding
    unit tests.

    • cdap-data-fabric/src/main/java/co/cask/cdap/data2/dataset2/lib/partitioned/PartitionedFileSetDefinition.java (version 52deb03874d791689fe5d368c3b81642fc8574c4)
    • cdap-data-fabric/src/test/java/co/cask/cdap/api/dataset/lib/SystemDatasetDefinitionTest.java (version 52deb03874d791689fe5d368c3b81642fc8574c4)
    • cdap-data-fabric/src/test/java/co/cask/cdap/data2/datafabric/dataset/DatasetsUtilTest.java (version 52deb03874d791689fe5d368c3b81642fc8574c4)
    • cdap-data-fabric/src/test/java/co/cask/cdap/data2/dataset2/lib/partitioned/PartitionedFileSetTest.java (version 52deb03874d791689fe5d368c3b81642fc8574c4)
  • Albert Shau <albert@cask.co>

    Albert Shau <albert@cask.co> 9943b0d25c3684d83b7bd46c53c54b3c21da686d m

    CDAP-13129 ensure a sink is only ever in one mapreduce phase
    Fixes a bug in the planner that could cause a sink to be present
    in multiple phases of a pipeline. The planner will now insert a
    connector in front of a sink if it has multiple reduce nodes
    as parents, or if it has a reduce node and a source as a parent.

    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/co/cask/cdap/etl/planner/ConnectorDag.java (version 9943b0d25c3684d83b7bd46c53c54b3c21da686d)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/test/java/co/cask/cdap/etl/planner/ConnectorDagTest.java (version 9943b0d25c3684d83b7bd46c53c54b3c21da686d)
  • Yaojie Feng

    Yaojie Feng 9b0ca0cd7578afad13b569b5895cd66f732f0761 m

    Merge pull request #9931 from caskdata/feature_release/CDAP-13123-fix-stream-explore
    CDAP-13123 fix stream exploration

  • Yaojie Feng

    Yaojie Feng 395981b6c709fdd95199be078e831dd8006fe905 m

    CDAP-13123 fix stream exploration

    • cdap-security/src/main/java/co/cask/cdap/security/impersonation/DefaultImpersonator.java (version 395981b6c709fdd95199be078e831dd8006fe905)
  • albertshau <albert@cask.co>

    albertshau <albert@cask.co> 124c80b45e45055111bd04fe17dcb23ca196c2f3 m

    Merge pull request #9930 from caskdata/bugfix_release/CDAP-13122-connector-input-format
    CDAP-13122 use combine input format for connectors

  • albertshau <albert@cask.co>

    albertshau <albert@cask.co> ac6295b60701dcfd92d67cf5b224689b2531d27f m

    Merge pull request #9923 from caskdata/4-3-4-snapshot
    bump version to 4.3.4-SNAPSHOT

  • Albert Shau <albert@cask.co>

    Albert Shau <albert@cask.co> 82424d37d36d0715c879c72851cfa77123e41f7e m

    bump version to 4.3.4-SNAPSHOT

    • cdap-api-common/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-api-spark/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-api-spark2_2.11/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-api/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-app-fabric-tests/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-app-fabric/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-app-templates/cdap-data-quality/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-app-templates/cdap-etl/cdap-data-pipeline/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-app-templates/cdap-etl/cdap-data-pipeline2_2.11/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-app-templates/cdap-etl/cdap-data-streams/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-app-templates/cdap-etl/cdap-data-streams2_2.11/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-app-templates/cdap-etl/cdap-etl-api-spark/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-app-templates/cdap-etl/cdap-etl-api/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-app-templates/cdap-etl/cdap-etl-archetypes/cdap-data-pipeline-plugins-archetype/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-app-templates/cdap-etl/cdap-etl-archetypes/cdap-data-pipeline-plugins-archetype/src/main/resources/archetype-resources/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-app-templates/cdap-etl/cdap-etl-archetypes/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-app-templates/cdap-etl/cdap-etl-batch/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-app-templates/cdap-etl/cdap-etl-core/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-app-templates/cdap-etl/cdap-etl-proto/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-app-templates/cdap-etl/cdap-etl-tools/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-app-templates/cdap-etl/hydrator-spark-core/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-app-templates/cdap-etl/hydrator-spark-core2_2.11/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-app-templates/cdap-etl/hydrator-test/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-app-templates/cdap-etl/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-app-templates/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-archetypes/cdap-app-archetype/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-archetypes/cdap-app-archetype/src/main/resources/archetype-resources/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-archetypes/cdap-mapreduce-archetype/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-archetypes/cdap-mapreduce-archetype/src/main/resources/archetype-resources/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-archetypes/cdap-spark-java-archetype/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-archetypes/cdap-spark-java-archetype/src/main/resources/archetype-resources/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-archetypes/cdap-spark-scala-archetype/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-archetypes/cdap-spark-scala-archetype/src/main/resources/archetype-resources/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-archetypes/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-cli-tests/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-cli/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-client-tests/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-client/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-common-unit-test/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-common/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-data-fabric-tests/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-data-fabric/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-distributions/bin/build_docs_bucket.sh (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-distributions/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-distributions/src/emr/install.sh (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-distributions/src/hdinsight/pkg/install.sh (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-distributions/src/hdinsight/pkg/mainTemplate.json (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-distributions/src/packer/cdap-sdk-ubuntu16-with-uri.json (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-distributions/src/packer/files/cdap-sdk.json (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-docs-gen/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-examples/ClicksAndViews/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-examples/CountRandom/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-examples/CubeService/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-examples/DataCleansing/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-examples/DecisionTreeRegression/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-examples/FileSetExample/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-examples/HelloWorld/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-examples/LogAnalysis/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-examples/Purchase/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-examples/SpamClassifier/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-examples/SparkKMeans/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-examples/SparkPageRank/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-examples/SportResults/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-examples/StreamConversion/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-examples/UserProfiles/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-examples/WebAnalytics/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-examples/WikipediaPipeline/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-examples/WordCount/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-examples/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-examples/resources/weblog-analytics-config.json (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-examples/resources/weblog-analytics.txt (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-examples/resources/weblog-service.txt (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-explore-client/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-explore-jdbc/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-explore/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-formats/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-gateway/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-hbase-compat-0.96/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-hbase-compat-0.98/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-hbase-compat-1.0-cdh/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-hbase-compat-1.0-cdh5.5.0/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-hbase-compat-1.0/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-hbase-compat-1.1/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-hbase-compat-1.2-cdh5.7.0/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-hbase-compat-base/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-hbase-spi/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-integration-test/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-kafka/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-kms/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-master/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-notifications-api/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-notifications/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-operational-stats-core/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-proto/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-security-spi/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-security/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-spark-core/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-spark-core2_2.11/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-spark-python/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-spi/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-standalone/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-test/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-tms-tests/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-tms/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-ui/package.json (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-ui/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-unit-test-spark2_2.11/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-unit-test/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-watchdog-api/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • cdap-watchdog/pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
    • pom.xml (version 82424d37d36d0715c879c72851cfa77123e41f7e)
  • Albert Shau <albert@cask.co>

    Albert Shau <albert@cask.co> 71fd9530af78d7ae06759f881e9d17050c13e42e m

    CDAP-13120 use correct directory for file in PartitionedFileSet
    If there was no base path set for a PartitionedFileSet, it would
    default to a 'files' directory whose parent is a directory named
    after the dataset. This means when the dataset is deleted, only
    the 'files' directory is deleted, but not the parent directory.
    This behavior is especially problematic when using a
    PartitionedFileSet as a local fileset in a workflow, as each run
    of the workflow would leave a directory per PartitionedFileSet.

    • cdap-data-fabric/src/main/java/co/cask/cdap/data2/dataset2/lib/partitioned/PartitionedFileSetDefinition.java (version 71fd9530af78d7ae06759f881e9d17050c13e42e)
  • Albert Shau <albert@cask.co>

    Albert Shau <albert@cask.co> 8e7e76d5cc6bbfff78d39c2eef9321a98dbe951f m

    CDAP-13122 use combine input format for connectors
    This is a small optimization in case connector output is small
    but in multiple files. In that scenario, this change will use
    fewer mappers, lowering the overhead and resource usage of the
    pipeline.

    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/co/cask/cdap/etl/batch/connector/ConnectorSource.java (version 8e7e76d5cc6bbfff78d39c2eef9321a98dbe951f)
  • albertshau <albert@cask.co>

    albertshau <albert@cask.co> 3c2b879a766246dae8c7c73b1f7e12e9484060ca m

    Merge pull request #9928 from caskdata/fix-checkstyle
    fix checkstyle

  • Albert Shau <albert@cask.co>

    Albert Shau <albert@cask.co> 9f66e84319430b1541af00a7dfccc3dff3b152e8 m

    fix checkstyle

    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/co/cask/cdap/etl/planner/Dag.java (version 9f66e84319430b1541af00a7dfccc3dff3b152e8)
  • albertshau <albert@cask.co>

    albertshau <albert@cask.co> 3c413736410b765b46e84c44fe258b5a640c0cf9 m

    Merge pull request #9927 from caskdata/feature_release/CDAP-13119-planner-reduce-connectors
    CDAP-13119 planner optimization to reduce connector datasets

  • Albert Shau <albert@cask.co>

    Albert Shau <albert@cask.co> ef3c917be430c5ccd9a51f495cbf575da2f1ba37 m

    CDAP-13119 planner optimization to reduce connector datasets
    Optimizing the planner to add a pass at the end that will merge
    connectors if possible. Connectors can be merged if they have the
    same branch head, which reduces the amount of I/O required while
    still preserving the property that each record is only processed
    once.

    • cdap-app-templates/cdap-etl/cdap-data-pipeline/src/main/java/co/cask/cdap/datapipeline/SmartWorkflow.java (version ef3c917be430c5ccd9a51f495cbf575da2f1ba37)
    • cdap-app-templates/cdap-etl/cdap-etl-batch/src/main/java/co/cask/cdap/etl/batch/ETLBatchApplication.java (version ef3c917be430c5ccd9a51f495cbf575da2f1ba37)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/co/cask/cdap/etl/planner/ConnectorDag.java (version ef3c917be430c5ccd9a51f495cbf575da2f1ba37)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/co/cask/cdap/etl/planner/Dag.java (version ef3c917be430c5ccd9a51f495cbf575da2f1ba37)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/main/java/co/cask/cdap/etl/planner/PipelinePlanner.java (version ef3c917be430c5ccd9a51f495cbf575da2f1ba37)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/test/java/co/cask/cdap/etl/planner/ConnectorDagTest.java (version ef3c917be430c5ccd9a51f495cbf575da2f1ba37)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/test/java/co/cask/cdap/etl/planner/DagTest.java (version ef3c917be430c5ccd9a51f495cbf575da2f1ba37)
    • cdap-app-templates/cdap-etl/cdap-etl-core/src/test/java/co/cask/cdap/etl/planner/PipelinePlannerTest.java (version ef3c917be430c5ccd9a51f495cbf575da2f1ba37)