Build #7

GCP Cloud Sandbox built from DA Artifacts

Build: #7 failed Manual run by Matt Wuenschel

Code commits

caskdata/cdap

  • mattwuenschel <mwuenschel@google.com>

    mattwuenschel <mwuenschel@google.com> dac04700dd129bb5fd0546ad346fdbc582841dd3

    Merge pull request #10569 from caskdata/feature/distributions-packer-gce
    add packer googlecompute builder

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 1d44ef638930c243786a4c0976e5738a7b901f50

    Merge pull request #10563 from caskdata/feature/CDAP-14104-provisioner-ip-choice
    CDAP-14104 add ability to determine which ip to use for dataproc

  • mattwuenschel <mwuenschel@google.com>

    mattwuenschel <mwuenschel@google.com> 9a8e914e68a9ee40a50cb1d156f1f7bb778d1374 m

    add packer googlecompute builder

    • cdap-distributions/src/packer/cdap-sdk-ubuntu16-with-uri.json (version 9a8e914e68a9ee40a50cb1d156f1f7bb778d1374)
    • cdap-distributions/src/packer/files/cdap-sdk-with-uri.json.template (version 9a8e914e68a9ee40a50cb1d156f1f7bb778d1374)
    • cdap-distributions/src/packer/files/cdap-sdk.json (version 9a8e914e68a9ee40a50cb1d156f1f7bb778d1374)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 24600efe45a8cc374dcad101df8d502c32eee5eb m

    CDAP-14104 add ability to determine which ip to use for dataproc
    Changed the Dataproc provisioner so that if it is running on GCP
    in the same network as the dataproc cluster, it will by default
    use the internal ip address when communicating with the cluster.
    This makes it less likely that firewall rules will be an issue.
    If users still want to use the external ip address, that is left
    as an option.

    Performed some refactoring so that CDAP no longer looks for the
    'ip.external' node property, since that is no longer accurate.
    Instead, added ip address as a required field in the Node class,
    since it was functionally required anyway. This allows us to make
    the contract explicit, instead of having required properties in
    some Map.

    Also made network and zone optional properties. When the provisioner
    is running on GCP, the network and zone can be picked up from the
    metadata server running on the VM.

    • cdap-app-fabric/src/main/java/co/cask/cdap/internal/app/runtime/distributed/remote/RemoteExecutionTwillRunnerService.java (version 24600efe45a8cc374dcad101df8d502c32eee5eb)
    • cdap-runtime-ext-dataproc/src/main/java/co/cask/cdap/runtime/spi/provisioner/dataproc/DataProcClient.java (version 24600efe45a8cc374dcad101df8d502c32eee5eb)
    • cdap-runtime-ext-dataproc/src/main/java/co/cask/cdap/runtime/spi/provisioner/dataproc/DataProcConf.java (version 24600efe45a8cc374dcad101df8d502c32eee5eb)
    • cdap-runtime-ext-dataproc/src/main/java/co/cask/cdap/runtime/spi/provisioner/dataproc/DataProcProvisioner.java (version 24600efe45a8cc374dcad101df8d502c32eee5eb)
    • cdap-runtime-ext-dataproc/src/main/resources/gcp-dataproc.json (version 24600efe45a8cc374dcad101df8d502c32eee5eb)
    • cdap-runtime-ext-emr/src/main/java/co/cask/cdap/runtime/spi/provisioner/emr/EMRClient.java (version 24600efe45a8cc374dcad101df8d502c32eee5eb)
    • cdap-runtime-ext-emr/src/main/java/co/cask/cdap/runtime/spi/provisioner/emr/ElasticMapReduceProvisioner.java (version 24600efe45a8cc374dcad101df8d502c32eee5eb)
    • cdap-runtime-ext-remote-hadoop/src/main/java/co/cask/cdap/runtime/spi/provisioner/remote/RemoteHadoopProvisioner.java (version 24600efe45a8cc374dcad101df8d502c32eee5eb)
    • cdap-runtime-spi/src/main/java/co/cask/cdap/runtime/spi/Constants.java (version 24600efe45a8cc374dcad101df8d502c32eee5eb)
    • cdap-runtime-spi/src/main/java/co/cask/cdap/runtime/spi/provisioner/Node.java (version 24600efe45a8cc374dcad101df8d502c32eee5eb)