albertshau <ashau@google.com>: Author Summary

Builds triggered by albertshau <ashau@google.com>

Builds triggered by an author are those builds which contains changes committed by the author.
1447
400 (28%)
1047 (72%)

Breakages and fixes

Broken means the build has failed but the previous build was successful.
Fixed means that the build was successful but the previous build has failed.
123 (9% of all builds triggered)
120 (8% of all builds triggered)
-3
Build Completed Code commits Tests
HYP › BAD › #365 1 day ago
Merge pull request #1337 from cdapio/fix-join-test
CDAP-17718 fix joiner config test
CDAP-17718 fix joiner config test
1059 passed
IT › UPD2 › #822 2 days ago
CDAP-17718 fix advanced join metrics
Cache join input for advanced joins when it is not a broadcast
join in order to prevent re-processing and confusing metrics.
Merge pull request #13165 from cdapio/CDAP-17718-fix-advanced-join-metrics
CDAP-17718 fix advanced join metrics
Testless build
HYP › BAD › #364 2 days ago
Merge pull request #1334 from cdapio/CDAP-17718-advanced-outer-join-improvements
CDAP-17718 force advanced outer join to be a broadcast
CDAP-17718 force advanced outer join to be a broadcast
Since Spark executes advanced outer joins as
BroadcastNestedLoopJoins, force the user to specify a side to
broadcast so that the behavior is clear and they don't wonder
why the pipeline fails with memory errors.
1 of 259 failed
CDAP › RUT › #1397 2 days ago
CDAP-17718 fix advanced join metrics
Cache join input for advanced joins when it is not a broadcast
join in order to prevent re-processing and confusing metrics.
Merge pull request #13165 from cdapio/CDAP-17718-fix-advanced-join-metrics
CDAP-17718 fix advanced join metrics
2942 passed
CDAP › UDUT › #1310 2 days ago
Merge pull request #13165 from cdapio/CDAP-17718-fix-advanced-join-metrics
CDAP-17718 fix advanced join metrics
CDAP-17718 fix advanced join metrics
Cache join input for advanced joins when it is not a broadcast
join in order to prevent re-processing and confusing metrics.
Testless build
CDAP › URUT › #1321 2 days ago
CDAP-17718 fix advanced join metrics
Cache join input for advanced joins when it is not a broadcast
join in order to prevent re-processing and confusing metrics.
Merge pull request #13165 from cdapio/CDAP-17718-fix-advanced-join-metrics
CDAP-17718 fix advanced join metrics
Testless build
CDAP › DUT › #3210 2 days ago
Merge pull request #13165 from cdapio/CDAP-17718-fix-advanced-join-metrics
CDAP-17718 fix advanced join metrics
CDAP-17718 fix advanced join metrics
Cache join input for advanced joins when it is not a broadcast
join in order to prevent re-processing and confusing metrics.
2942 passed
CDAP › DRC › #5306 2 days ago
Merge pull request #13165 from cdapio/CDAP-17718-fix-advanced-join-metrics
CDAP-17718 fix advanced join metrics
CDAP-17718 fix advanced join metrics
Cache join input for advanced joins when it is not a broadcast
join in order to prevent re-processing and confusing metrics.
Testless build
CDAP › BPP › #1580 2 days ago
Merge pull request #13165 from cdapio/CDAP-17718-fix-advanced-join-metrics
CDAP-17718 fix advanced join metrics
CDAP-17718 fix advanced join metrics
Cache join input for advanced joins when it is not a broadcast
join in order to prevent re-processing and confusing metrics.
Testless build
HYP › WT › #362 3 days ago
Merge pull request #473 from data-integrations/fix-lower-bound-compat
fix CDAP lower bound, as it uses newer APIs
fix CDAP lower bound, as it uses newer APIs
420 passed
Build Completed Code commits Tests
CDAP › RUT › #1392 5 days ago
CDAP-17719 exclude conflicting asm version
Merge pull request #13152 from cdapio/bugfix/CDAP-17719-exclude-asm-from-pipelines
CDAP-17719 exclude conflicting asm version
1 of 2425 failed
HYP › BAD › #362 1 week ago
Merge pull request #1331 from cdapio/fix-parent-lower-bound
fix the parent version lower bound
fix the parent version lower bound
since the code relies on CDAP 6.4.0, the lower bound should also
be 6.4.0.
1 of 260 failed
CDAP › DUT › #3182 3 weeks ago
Merge pull request #13120 from cdapio/feature/CDAP-17556-join-condition-expr-api
CDAP-17556 add api for expression join conditions
CDAP-17556 add api for expression join conditions
1 of 2421 failed
IT › ITM › #241 2 months ago
Merge pull request #1043 from yeweidaniel/integration-doc2
Update docs
45 of 45 failed
CDAP › DUT › #3124 2 months ago
Merge pull request #12899 from cdapio/feature/CDAP-17059-fail-middle-action-pipeline
CDAP-17059 validate no actions in the middle of a pipeline
CDAP-17059 validate no actions in the middle of a pipeline
1 of 1993 failed
CDAP › RUT › #1306 2 months ago
CDAP-17059 validate no actions in the middle of a pipeline
Merge pull request #12899 from cdapio/feature/CDAP-17059-fail-middle-action-pipeline
CDAP-17059 validate no actions in the middle of a pipeline
1 of 2394 failed
HYP › BAD › #339 2 months ago
Merge pull request #1268 from cdapio/feature/CDAP-17249-regex-path-docs
CDAP-17249 add docs for regex path
CDAP-17249 add docs for regex path
955 passed
CDAP › RUT › #1290 3 months ago
CDAP-17425 expose max preview records to sources
Merge pull request #12874 from cdapio/feature/CDAP-17425-expose-preview-info
CDAP-17425 expose max preview records to sources
1 of 1169 failed
CDAP › DUT › #3061 5 months ago
Merge pull request #12763 from cdapio/bugfix/CDAP-17237-fix-pipeline-hconf
CDAP-17237 fix pipeline hconf clearing
CDAP-17237 fix pipeline hconf clearing
Fixed a bug where the hadoop conf is cleared before adding sink
specific properties. This ensures that cluster specific defaults
are correctly included in the conf instead of being wiped.
2356 passed
CDAP › DUT › #3022 7 months ago
Merge pull request #12558 from cdapio/feature/CDAP-17078-spark-stage-consolidation
CDAP-17078 consolidate stages within a group
CDAP-17078 consolidate stages within a group
Changed the SparkPipelineRunner to use a CombinerDag to group
sinks and their preceding transforms together. These grouped
stages are treated similarly to how a single sink is treated,
with flatMapToPair() called on the input RDD to transform it
into a PairRDD, then calling save() to write the RDD out.
This capability is off by default, but can be turned on by
setting a runtime argument.

Instead of flatMapToPair() calling just the sink's transform
method, a new MultiSinkFunction class is used to direct incoming
records to the correct logical branches of the pipeline.
This requires that each input be tagged with which stage it
came from (stage and port), as well as its type (output, or error).
In order to do this, refactored the SparkPipelineRunner a bit
to maintain the RDD<RecordInfo> for each stage rather than
RDD<StructuredRecord>, as the RecordInfo class contains that
extra information.

Also added a MultiOutputFormat that will take the output of the
MultiSinkFunction and delegate writes to the correct underlying
OutputFormat. Since the OutputFormat lives in the pipeline
app, this approach means CDAP datasets cannot be combined.
This caused a problem with dataset lineage, since it is
implemented by implemented by wrapping OutputFormats into a hidden
ExternalDataset class in CDAP. Instead of doing this indirect
wrapping, changed the SparkSinkFactory class to explicitly
register lineage through direct calls instead of hiding it
under several layers of abstraction.
1 of 1969 failed
Build Completed Code commits Tests
CDAP › DUT › #3210 2 days ago
Merge pull request #13165 from cdapio/CDAP-17718-fix-advanced-join-metrics
CDAP-17718 fix advanced join metrics
CDAP-17718 fix advanced join metrics
Cache join input for advanced joins when it is not a broadcast
join in order to prevent re-processing and confusing metrics.
2942 passed
CDAP › RUT › #1397 2 days ago
CDAP-17718 fix advanced join metrics
Cache join input for advanced joins when it is not a broadcast
join in order to prevent re-processing and confusing metrics.
Merge pull request #13165 from cdapio/CDAP-17718-fix-advanced-join-metrics
CDAP-17718 fix advanced join metrics
2942 passed
CDAP › DUT › #3190 2 weeks ago
Merge pull request #13134 from cdapio/feature/CDAP-17606-join-expr-validate
CDAP-17606 basic validation for join condition expressions
CDAP-17606 basic validation for join condition expressions
Added validation to ensure that join condition expressions are
not allowed in MapReduce pipelines or streaming pipelines.
Also added validation to ensure that input schemas are always
known when using expressions.
2920 passed
HYP › BAD › #341 2 months ago
Merge pull request #1277 from cdapio/feature/PLUGIN-102-fix-file-sink-compatibility
PLUGIN-102 fix backwards incompatibility from format fix
PLUGIN-102 fix backwards incompatibility from format fix
Keep the same method signature for protected method.
958 passed
HYP › WT › #355 2 months ago
PLUGIN-464 fix flatten to be a no-op on empty lists
Fixed a bug where input rows with an empty list would get
filtered out by the flatten directive.
Merge pull request #462 from data-integrations/feature/PLUGIN-464-flatten-fix
PLUGIN-464 fix flatten to be a no-op on empty lists
396 passed
CDAP › DUT › #3126 2 months ago
CDAP-16527 include field name is casting errors for records
For logical type utility methods, including the field name when
the type is not as expected.

This is only required because the builder does not verify that
a valid type is being set for a field. If/when that verification
is added, this logic can be removed.
Merge pull request #12898 from cdapio/feature/CDAP-16527-include-field-name-is-classcast
CDAP-16527 include field name in casting errors for records
CDAP-16527 fix decimal getter and check messages in tests
2873 passed
CDAP › RUT › #1292 3 months ago
Merge pull request #12876 from cdapio/feature/CDAP-17428-consolidation-default-on
CDAP-17428 default stage consolidation to true
Merge pull request #12877 from cdapio/feature/CDAP-17408-fix-aggregator-partitions
CDAP-17408 fix to honor partitions set by aggregators
CDAP-17428 default stage consolidation to true
CDAP-17408 fix to honor partitions set by aggregators
2847 passed
CDAP › RUT › #1234 6 months ago
CDAP-17232 wait longer for program state to reduce flakiness
Increase the time to wait for program state in gateway tests from
10 seconds to 30 seconds to reduce test flakiness on slow machines.
Merge pull request #12700 from cdapio/bugfix/CDAP-17232-wait-longer-for-programs
CDAP-17232 wait longer for program state to reduce flakiness
2839 passed
CDAP › DUT › #3028 6 months ago
Merge pull request #12594 from cdapio/feature/CDAP-17078-consolidate-multi-output-sinks
CDAP-17078 consolidate multiple outputs from same sink
CDAP-17078 consolidate multiple outputs from same sink
Some sinks have multiple outputs, which can cause a lot of
data recomputation. Added logic to consolidate these outputs.
2827 passed
CDAP › DUT › #3010 7 months ago
CDAP-17078 add a combiner dag to consolidate nodes
Merge pull request #12483 from cdapio/feature/CDAP-17078-dag-consolidation-logic
CDAP-17078 add a combiner dag to consolidate nodes
2805 passed