Build #298

Code commits

Wrangler Transform

  • Vinisha Shah <CuriousVini@users.noreply.github.com>

    Vinisha Shah <CuriousVini@users.noreply.github.com> c41f485c2fd6da500f91db55221a6ab65842a7e5

    Merge pull request #332 from data-integrations/fix/merge-release-develop
    Fix/merge release develop

  • Vinisha Shah

    Vinisha Shah 5a8a804fd69efeb61335ad465b0143160c641b62

    remove conflict

    • pom.xml (version 5a8a804fd69efeb61335ad465b0143160c641b62)
  • Vinisha Shah

    Vinisha Shah dcc6e14be874b2390798a7c865bcc070bf77d53e

    merge release to develop

  • Vinisha Shah <CuriousVini@users.noreply.github.com>

    Vinisha Shah <CuriousVini@users.noreply.github.com> 082fbf7f4c00c20c193a121558ba7feee9567824 m

    Merge pull request #330 from data-integrations/fix/remove-snapshot
    Remove snapshot

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 3b7974a7b6e4fddf838bbdae47f006ad602c7178 m

    Merge pull request #331 from data-integrations/bugfix_release/CDAP-15397-fix-udd-execution
    CDAP-15397 fix execution of user scoped UDDs

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 9a5d2e1a07a365c299e2a9e9e51df38ca7627319 m

    CDAP-15397 fix execution of user scoped UDDs
    User directives must be reloaded before they are used, as
    in-memory objects are not shared across Service requests.

    Also improving wrangler errors to also include the cause when
    throwing an exception.

    • wrangler-service/src/main/java/io/cdap/wrangler/service/directive/DirectivesHandler.java (version 9a5d2e1a07a365c299e2a9e9e51df38ca7627319)
    • wrangler-transform/src/main/java/io/cdap/wrangler/Wrangler.java (version 9a5d2e1a07a365c299e2a9e9e51df38ca7627319)
  • Vinisha Shah

    Vinisha Shah 8c2b2b995c0b3ae6639b5b26d74906e4f9debd0a m

    Remove snapshot

    • pom.xml (version 8c2b2b995c0b3ae6639b5b26d74906e4f9debd0a)
    • wrangler-transform/pom.xml (version 8c2b2b995c0b3ae6639b5b26d74906e4f9debd0a)
  • Vinisha Shah <CuriousVini@users.noreply.github.com>

    Vinisha Shah <CuriousVini@users.noreply.github.com> bea8de38da55dfa5b09432c0da4f41ee578b2e2a m

    Merge pull request #328 from data-integrations/feature/array-support
    [CDAP-15256] support arrays in bq connection

  • Vinisha Shah

    Vinisha Shah 4fef98e813a0221cc53708a6b37eca53799a9590 m

    iterate fast by not calling getschema upon null values

    • wrangler-core/src/main/java/io/cdap/wrangler/utils/Json2Schema.java (version 4fef98e813a0221cc53708a6b37eca53799a9590)
    • wrangler-core/src/test/java/io/cdap/wrangler/utils/Json2SchemaTest.java (version 4fef98e813a0221cc53708a6b37eca53799a9590)
  • Vinisha Shah

    Vinisha Shah da0ed5dc7c993920e0b508e8fda67153ee702ad7 m

    Add ByteBuffer and cause of exception in RecordConvertorException

    • wrangler-core/src/main/java/io/cdap/wrangler/utils/Json2Schema.java (version da0ed5dc7c993920e0b508e8fda67153ee702ad7)
    • wrangler-core/src/main/java/io/cdap/wrangler/utils/RecordConvertor.java (version da0ed5dc7c993920e0b508e8fda67153ee702ad7)
    • wrangler-core/src/main/java/io/cdap/wrangler/utils/RecordConvertorException.java (version da0ed5dc7c993920e0b508e8fda67153ee702ad7)
    • wrangler-service/src/main/java/io/cdap/wrangler/service/bigquery/BigQueryHandler.java (version da0ed5dc7c993920e0b508e8fda67153ee702ad7)
  • Vinisha Shah <CuriousVini@users.noreply.github.com>

    Vinisha Shah <CuriousVini@users.noreply.github.com> 79cca7ab91269df4c4d5cf6cb3e0a44ee9d0afa3 m

    Merge pull request #329 from data-integrations/fix/test
    Fix Json2SchemaTest

  • Vinisha Shah

    Vinisha Shah f61cacbe1e18176af3721f2cc450e2fed10f325d m

    Fix Json2SchemaTest

    • wrangler-core/src/test/java/io/cdap/wrangler/utils/Json2SchemaTest.java (version f61cacbe1e18176af3721f2cc450e2fed10f325d)
  • Vinisha Shah

    Vinisha Shah 7884d356ea004d7f5c27405a5217bb165a7859e8 m

    Add tests

    • wrangler-core/src/test/java/io/cdap/wrangler/utils/Json2SchemaTest.java (version 7884d356ea004d7f5c27405a5217bb165a7859e8)
    • wrangler-core/src/test/java/io/cdap/wrangler/utils/RecordConvertorTest.java (version 7884d356ea004d7f5c27405a5217bb165a7859e8)
  • Vinisha Shah

    Vinisha Shah 8d8bba793d9495bdb4f6503116244b3ef4435c18 m

    [CDAP-15256] support arrays in bq connection

    • wrangler-core/src/main/java/io/cdap/wrangler/utils/Json2Schema.java (version 8d8bba793d9495bdb4f6503116244b3ef4435c18)
    • wrangler-core/src/main/java/io/cdap/wrangler/utils/RecordConvertor.java (version 8d8bba793d9495bdb4f6503116244b3ef4435c18)
    • wrangler-service/src/main/java/io/cdap/wrangler/service/bigquery/BigQueryHandler.java (version 8d8bba793d9495bdb4f6503116244b3ef4435c18)
  • Vinisha Shah <CuriousVini@users.noreply.github.com>

    Vinisha Shah <CuriousVini@users.noreply.github.com> cbae4a0ccd865fee579d3064e9231c4ecd209a41 m

    Merge pull request #327 from data-integrations/feature/bq-decimal-schema
    [CDAP-15256] support decimal type in big query connection

  • Vinisha Shah

    Vinisha Shah 92bb2e3df2085e16c97e669505aa43da22277172 m

    Use 38 as max precision due to CDAP-15361

    • wrangler-core/src/main/java/io/cdap/wrangler/utils/Json2Schema.java (version 92bb2e3df2085e16c97e669505aa43da22277172)
    • wrangler-service/src/main/java/io/cdap/wrangler/service/bigquery/BigQueryHandler.java (version 92bb2e3df2085e16c97e669505aa43da22277172)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 1dea3effb60648ffc5764eed1bddd14982a5326f m

    Merge pull request #326 from data-integrations/bugfix_release/CDAP-15352-fix-drivers-endpoint
    CDAP-15352 fix installed drivers list endpoint

  • Vinisha Shah

    Vinisha Shah 6c103bb7e06bd86fea5c854323cbd3cb09f8b60d m

    [CDAP-15256] support decimal type in big query connection

    • pom.xml (version 6c103bb7e06bd86fea5c854323cbd3cb09f8b60d)
    • wrangler-core/src/main/java/io/cdap/wrangler/utils/Json2Schema.java (version 6c103bb7e06bd86fea5c854323cbd3cb09f8b60d)
    • wrangler-core/src/main/java/io/cdap/wrangler/utils/RecordConvertor.java (version 6c103bb7e06bd86fea5c854323cbd3cb09f8b60d)
    • wrangler-core/src/test/java/io/cdap/wrangler/utils/Json2SchemaTest.java (version 6c103bb7e06bd86fea5c854323cbd3cb09f8b60d)
    • wrangler-service/src/main/java/io/cdap/wrangler/service/bigquery/BigQueryHandler.java (version 6c103bb7e06bd86fea5c854323cbd3cb09f8b60d)
    • wrangler-transform/src/main/java/io/cdap/wrangler/Wrangler.java (version 6c103bb7e06bd86fea5c854323cbd3cb09f8b60d)
  • Joltie Root <nitinmotgi@users.noreply.github.com>

    Joltie Root <nitinmotgi@users.noreply.github.com> e9f8274b03abca73dd12f36d2d7396db54294f4c m

    [CDAP-15338] Fixed TextList extraction and also added a test case (#325)
    * [CDAP-15338] Fixed TextList extraction and also added a test case

    • wrangler-api/src/main/java/io/cdap/wrangler/api/parser/TextList.java (version e9f8274b03abca73dd12f36d2d7396db54294f4c)
    • wrangler-core/src/main/java/io/cdap/wrangler/parser/RecipeVisitor.java (version e9f8274b03abca73dd12f36d2d7396db54294f4c)
    • wrangler-core/src/test/java/io/cdap/wrangler/parser/MapArgumentsTest.java (version e9f8274b03abca73dd12f36d2d7396db54294f4c)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> 0f513a0979df65e291b30a56d43989a99931f135 m

    CDAP-15352 fix installed drivers list endpoint
    The database handler was incorrectly joining hardcoded information
    about the mysql and oracle jdbc drivers to the actual drivers
    installed. When the generic mysql driver was installed, the drivers
    endpoint would return two entries, one for the generic mysql
    driver and one incorrect mix between the generic mysql driver and
    the information for the google cloud mysql driver.

    A similar problem existed with the oracle and oracle thin drivers.
    The short term fix done here was to look up driver information by
    the driver name instead of the driver class name, and to make sure
    all driver names are unique (at least when going through wrangler).

    • wrangler-service/src/main/java/io/cdap/wrangler/service/database/DatabaseHandler.java (version 0f513a0979df65e291b30a56d43989a99931f135)
    • wrangler-service/src/main/resources/drivers.mapping (version 0f513a0979df65e291b30a56d43989a99931f135)
    • wrangler-service/src/test/java/io/cdap/wrangler/service/database/DatabaseHandlerTest.java (version 0f513a0979df65e291b30a56d43989a99931f135)
  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> ffb38b90980de15c124afa7cb0138c650386336e m

    Merge pull request #324 from data-integrations/bugfix_release/CDAP-14691-fix-bad-workspace-ids
    CDAP-14691 fix conflicts in workspace id

  • albertshau <ashau@google.com>

    albertshau <ashau@google.com> d5581465173d4d7512e4686c06b37b542ce13407 m

    CDAP-14691 fix conflicts in workspace id
    Changed workspace creation such that each new workspace is given
    an actual unique identifier instead of hashing some non-unique
    attribute of the connection.

    This fixes bugs where a workspace for a table in one database
    could clobber the workspace for a table from a different database
    that has the same name, and similar conflict problems with all
    of the other connection types.

    A side effect of this is that there can now be more than one
    workspace for the same data object being wrangled. For example,
    a database table can now have more than one namespace. This was
    already the case for files, but not for other connection types.

    • wrangler-service/src/main/java/io/cdap/wrangler/PropertyIds.java (version d5581465173d4d7512e4686c06b37b542ce13407)
    • wrangler-service/src/main/java/io/cdap/wrangler/service/bigquery/BigQueryHandler.java (version d5581465173d4d7512e4686c06b37b542ce13407)
    • wrangler-service/src/main/java/io/cdap/wrangler/service/database/DatabaseHandler.java (version d5581465173d4d7512e4686c06b37b542ce13407)
    • wrangler-service/src/main/java/io/cdap/wrangler/service/directive/DirectivesHandler.java (version d5581465173d4d7512e4686c06b37b542ce13407)
    • wrangler-service/src/main/java/io/cdap/wrangler/service/explorer/FilesystemExplorer.java (version d5581465173d4d7512e4686c06b37b542ce13407)
    • wrangler-service/src/main/java/io/cdap/wrangler/service/gcs/GCSHandler.java (version d5581465173d4d7512e4686c06b37b542ce13407)
    • wrangler-service/src/main/java/io/cdap/wrangler/service/kafka/KafkaHandler.java (version d5581465173d4d7512e4686c06b37b542ce13407)
    • wrangler-service/src/main/java/io/cdap/wrangler/service/s3/S3Handler.java (version d5581465173d4d7512e4686c06b37b542ce13407)
    • wrangler-service/src/main/java/io/cdap/wrangler/service/spanner/SpannerHandler.java (version d5581465173d4d7512e4686c06b37b542ce13407)
    • wrangler-storage/src/main/java/io/cdap/wrangler/dataset/workspace/Workspace.java (version d5581465173d4d7512e4686c06b37b542ce13407)
    • wrangler-storage/src/main/java/io/cdap/wrangler/dataset/workspace/WorkspaceDataset.java (version d5581465173d4d7512e4686c06b37b542ce13407)
    • wrangler-storage/src/main/java/io/cdap/wrangler/dataset/workspace/WorkspaceMeta.java (version d5581465173d4d7512e4686c06b37b542ce13407)
    • wrangler-storage/src/test/java/io/cdap/wrangler/dataset/WorkspaceDatasetTest.java (version d5581465173d4d7512e4686c06b37b542ce13407)
  • Ali <alianwar@google.com>

    Ali <alianwar@google.com> 3b2258e6ebb8106d936f87a73a706aab5609b531 m

    v4.0.0

    • pom.xml (version 3b2258e6ebb8106d936f87a73a706aab5609b531)
    • wrangler-api/pom.xml (version 3b2258e6ebb8106d936f87a73a706aab5609b531)
    • wrangler-core/pom.xml (version 3b2258e6ebb8106d936f87a73a706aab5609b531)
    • wrangler-proto/pom.xml (version 3b2258e6ebb8106d936f87a73a706aab5609b531)
    • wrangler-service/pom.xml (version 3b2258e6ebb8106d936f87a73a706aab5609b531)
    • wrangler-storage/pom.xml (version 3b2258e6ebb8106d936f87a73a706aab5609b531)
    • wrangler-test/pom.xml (version 3b2258e6ebb8106d936f87a73a706aab5609b531)
    • wrangler-transform/pom.xml (version 3b2258e6ebb8106d936f87a73a706aab5609b531)