crp-order-quantity-kpi-service - Details for Query 2989

Submitted Time: 2026/03/20 05:59:07
Duration: 3 s
Succeeded Jobs: 5352 5353 5354 5355 5356 5357 5358 5359 5360 5361

Plan Visualization

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [id="node0" labelType="html" label=" AdaptiveSparkPlan " tooltip="AdaptiveSparkPlan isFinalPlan=true"]; 1 [id="node1" labelType="html" label="CollectLimit shuffle records written: 37 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 1 local bytes read: 806.0 B merged fetch fallback count: 0 local blocks read: 13 remote merged chunks fetched: 0 remote blocks read: 0 local merged bytes read: 0.0 B remote reqs duration: 0 ms local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 5 ms (0 ms, 0 ms, 0 ms (stage 100797.0: task 323950)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 fetch wait time: 0 ms remote bytes read: 0.0 B remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 2.2 KiB (62.0 B, 62.0 B, 62.0 B (stage 100797.0: task 323936))" tooltip="CollectLimit 1"]; 2 [id="node2" labelType="html" label="InMemoryTableScan number of output rows: 146" tooltip="InMemoryTableScan"]; 3 [id="node3" labelType="html" label=" AdaptiveSparkPlan " tooltip="AdaptiveSparkPlan isFinalPlan=true"]; subgraph cluster4 { isCluster="true"; id="cluster4"; label="WholeStageCodegen (11)\n \nduration: total (min, med, max (stageId: taskId))\n3.6 s (0 ms, 12 ms, 656 ms (stage 100792.0: task 323914))"; tooltip="WholeStageCodegen (11)"; 5 [id="node5" labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build total (min, med, max (stageId: taskId)) 296 ms (0 ms, 0 ms, 133 ms (stage 100792.0: task 323918)) peak memory total (min, med, max (stageId: taskId)) 9.3 MiB (256.0 KiB, 256.0 KiB, 256.0 KiB (stage 100792.0: task 323900)) number of output rows: 146 number of sort fallback tasks: 0 avg hash probes per key: 0" tooltip="HashAggregate(keys=[product#944305, plant#944306], functions=[])"]; } 6 [id="node6" labelType="html" label="Exchange shuffle records written: 146 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 146 local bytes read total (min, med, max (stageId: taskId)) 12.6 KiB (144.0 B, 288.0 B, 1008.0 B (stage 100792.0: task 323897)) merged fetch fallback count: 0 local blocks read: 89 remote merged chunks fetched: 0 remote blocks read: 54 data size total (min, med, max (stageId: taskId)) 12.5 KiB (176.0 B, 352.0 B, 1056.0 B (stage 100787.0: task 323873)) local merged bytes read: 0.0 B local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 21 ms (0 ms, 0 ms, 0 ms (stage 100787.0: task 323854)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 fetch wait time total (min, med, max (stageId: taskId)) 276 ms (0 ms, 0 ms, 132 ms (stage 100792.0: task 323918)) remote bytes read total (min, med, max (stageId: taskId)) 7.6 KiB (144.0 B, 288.0 B, 1008.0 B (stage 100792.0: task 323909)) number of partitions: 37 remote reqs duration total (min, med, max (stageId: taskId)) 312 ms (0 ms, 3 ms, 134 ms (stage 100792.0: task 323918)) remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 20.2 KiB (288.0 B, 576.0 B, 1727.0 B (stage 100787.0: task 323873))" tooltip="Exchange hashpartitioning(product#944305, plant#944306, 37), ENSURE_REQUIREMENTS, [plan_id=1457554]"]; subgraph cluster7 { isCluster="true"; id="cluster7"; label="WholeStageCodegen (10)\n \nduration: total (min, med, max (stageId: taskId))\n1.5 s (2 ms, 5 ms, 158 ms (stage 100787.0: task 323867))"; tooltip="WholeStageCodegen (10)"; 8 [id="node8" labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build total (min, med, max (stageId: taskId)) 182 ms (1 ms, 3 ms, 15 ms (stage 100787.0: task 323861)) peak memory total (min, med, max (stageId: taskId)) 8.5 MiB (256.0 KiB, 256.0 KiB, 256.0 KiB (stage 100787.0: task 323868)) number of output rows: 146 number of sort fallback tasks: 0 avg hash probes per key: 0" tooltip="HashAggregate(keys=[product#944305, plant#944306], functions=[])"]; 9 [id="node9" labelType="html" label=" Project " tooltip="Project [product#944305, plant#944306]"]; 10 [id="node10" labelType="html" label="Filter number of output rows: 146" tooltip="Filter (((size(demandTimeBuckets#944819, true) = 0) OR isnull(demandTimeBuckets#944819)) AND ((size(demandPointInTime#944822, true) = 0) OR isnull(demandPointInTime#944822)))"]; 11 [id="node11" labelType="html" label="SortMergeJoin number of output rows: 434 spill size total (min, med, max (stageId: taskId)) 0.0 B (0.0 B, 0.0 B, 0.0 B (stage 100787.0: task 323868))" tooltip="SortMergeJoin [coalesce(plant#944306, ), isnull(plant#944306), coalesce(product#944305, ), isnull(product#944305)], [coalesce(plant#944815, ), isnull(plant#944815), coalesce(product#944814, ), isnull(product#944814)], LeftOuter"]; } subgraph cluster12 { isCluster="true"; id="cluster12"; label="WholeStageCodegen (8)\n \nduration: total (min, med, max (stageId: taskId))\n5.6 s (4 ms, 6 ms, 631 ms (stage 100787.0: task 323867))"; tooltip="WholeStageCodegen (8)"; 13 [id="node13" labelType="html" label="Sort sort time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 100787.0: task 323868)) peak memory total (min, med, max (stageId: taskId)) 70.1 MiB (2.1 MiB, 2.1 MiB, 2.1 MiB (stage 100787.0: task 323868)) spill size total (min, med, max (stageId: taskId)) 0.0 B (0.0 B, 0.0 B, 0.0 B (stage 100787.0: task 323868))" tooltip="Sort [coalesce(plant#944306, ) ASC NULLS FIRST, isnull(plant#944306) ASC NULLS FIRST, coalesce(product#944305, ) ASC NULLS FIRST, isnull(product#944305) ASC NULLS FIRST], false, 0"]; } 14 [id="node14" labelType="html" label="AQEShuffleRead number of partitions: 34 partition data size total (min, med, max (stageId: taskId)) 60.6 KiB (831.0 B, 1788.0 B, 2.9 KiB (driver)) number of coalesced partitions: 3" tooltip="AQEShuffleRead coalesced"]; 15 [id="node15" labelType="html" label="Exchange shuffle records written: 434 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 434 local bytes read total (min, med, max (stageId: taskId)) 25.9 KiB (288.0 B, 720.0 B, 1471.0 B (stage 100787.0: task 323880)) merged fetch fallback count: 0 local blocks read: 170 remote merged chunks fetched: 0 remote blocks read: 202 data size total (min, med, max (stageId: taskId)) 37.3 KiB (528.0 B, 968.0 B, 1760.0 B (stage 100783.0: task 323847)) local merged bytes read: 0.0 B local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 31 ms (0 ms, 0 ms, 1 ms (stage 100783.0: task 323841)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 100787.0: task 323868)) remote bytes read total (min, med, max (stageId: taskId)) 30.3 KiB (288.0 B, 933.0 B, 1584.0 B (stage 100787.0: task 323861)) number of partitions: 37 remote reqs duration total (min, med, max (stageId: taskId)) 104 ms (0 ms, 3 ms, 7 ms (stage 100787.0: task 323857)) remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 56.1 KiB (863.0 B, 1433.0 B, 2.5 KiB (stage 100783.0: task 323848))" tooltip="Exchange hashpartitioning(coalesce(plant#944306, ), isnull(plant#944306), coalesce(product#944305, ), isnull(product#944305), 37), ENSURE_REQUIREMENTS, [plan_id=1457460]"]; subgraph cluster16 { isCluster="true"; id="cluster16"; label="WholeStageCodegen (7)\n \nduration: total (min, med, max (stageId: taskId))\n150 ms (1 ms, 3 ms, 11 ms (stage 100783.0: task 323823))"; tooltip="WholeStageCodegen (7)"; 17 [id="node17" labelType="html" label=" Project " tooltip="Project [product#944305, plant#944306]"]; 18 [id="node18" labelType="html" label="Generate number of output rows: 434" tooltip="Generate explode(_extract_quantityDenominator#944849), [product#944305, plant#944306], true, [quantityDenominator#944420]"]; 19 [id="node19" labelType="html" label=" Project " tooltip="Project [product#944305, plant#944306, filteredUnitOfMeasureConversions#944409.quantityDenominator AS _extract_quantityDenominator#944849]"]; 20 [id="node20" labelType="html" label="Generate number of output rows: 434" tooltip="Generate explode(filteredUnitOfMeasureConversions#944409.quantityNumerator), [product#944305, plant#944306, filteredUnitOfMeasureConversions#944409], true, [quantityNumerator#944417]"]; } 21 [id="node21" labelType="html" label=" Project " tooltip="Project [product#944305, plant#944306, filter(unitOfMeasureConversions#944403, lambdafunction((((lambda u#944410.measurementUnit1.code = _extract_code#944928) AND (lambda u#944410.measurementUnit1.internalRefUUID = _extract_internalRefUUID#944929)) AND ((lambda u#944410.measurementUnit2.code = baseUnitOfMeasure#944401.code) AND (lambda u#944410.measurementUnit2.internalRefUUID = baseUnitOfMeasure#944401.internalRefUUID))), lambda u#944410, false)) AS filteredUnitOfMeasureConversions#944409]"]; subgraph cluster22 { isCluster="true"; id="cluster22"; label="WholeStageCodegen (6)\n \nduration: total (min, med, max (stageId: taskId))\n1.6 s (2 ms, 3 ms, 174 ms (stage 100783.0: task 323823))"; tooltip="WholeStageCodegen (6)"; 23 [id="node23" labelType="html" label="BroadcastHashJoin number of output rows: 434" tooltip="BroadcastHashJoin [product#944305], [internalUUID#944397], LeftOuter, BuildRight, false"]; 24 [id="node24" labelType="html" label=" Project " tooltip="Project [product#944305, plant#944306, _extract_code#944928, _extract_internalRefUUID#944929]"]; 25 [id="node25" labelType="html" label="BroadcastHashJoin number of output rows: 434" tooltip="BroadcastHashJoin [coalesce(product#944305, ), isnull(product#944305), coalesce(plant#944306, ), isnull(plant#944306)], [coalesce(_extract_internalRefUUID#944930, ), isnull(_extract_internalRefUUID#944930), coalesce(_extract_internalRefUUID#944931, ), isnull(_extract_internalRefUUID#944931)], LeftOuter, BuildRight, false"]; } 26 [id="node26" labelType="html" label="Exchange shuffle records written: 434 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 434 local bytes read total (min, med, max (stageId: taskId)) 22.5 KiB (399.0 B, 581.0 B, 911.0 B (stage 100783.0: task 323847)) merged fetch fallback count: 0 local blocks read: 37 remote merged chunks fetched: 0 remote blocks read: 0 data size total (min, med, max (stageId: taskId)) 37.3 KiB (528.0 B, 968.0 B, 1760.0 B (stage 100775.0: task 323686)) local merged bytes read: 0.0 B local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 19 ms (0 ms, 0 ms, 0 ms (stage 100775.0: task 323810)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 100783.0: task 323819)) remote bytes read: 0.0 B number of partitions: 37 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 22.5 KiB (399.0 B, 581.0 B, 911.0 B (stage 100775.0: task 323686))" tooltip="Exchange hashpartitioning(product#944305, plant#944306, 37), REPARTITION_BY_NUM, [plan_id=1456881]"]; subgraph cluster27 { isCluster="true"; id="cluster27"; label="WholeStageCodegen (1)\n \nduration: total (min, med, max (stageId: taskId))\n3.8 s (70 ms, 86 ms, 190 ms (stage 100775.0: task 323801))"; tooltip="WholeStageCodegen (1)"; 28 [id="node28" labelType="html" label="ColumnarToRow number of output rows: 434 number of input batches: 37" tooltip="ColumnarToRow"]; } 29 [id="node29" labelType="html" label="Scan parquet number of files read: 37 scan time total (min, med, max (stageId: taskId)) 3.2 s (69 ms, 81 ms, 162 ms (stage 100775.0: task 323698)) metadata time: 0 ms size of files read: 78.2 KiB number of output rows: 434" tooltip="FileScan parquet [product#944305,plant#944306] Batched: true, DataFilters: [], Format: Parquet, Location: InMemoryFileIndex(1 paths)[hdlfs://770f0770-bd0c-456f-b83b-9649926e0857.files.hdl.prod-eu20.hanac..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<product:string,plant:string>"]; 30 [id="node30" labelType="html" label="BroadcastExchange time to broadcast: 2 ms time to build: 9 ms time to collect: 118 ms number of output rows: 1,462 data size: 2.1 MiB" tooltip="BroadcastExchange HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=1457064]"]; subgraph cluster31 { isCluster="true"; id="cluster31"; label="WholeStageCodegen (5)\n \nduration: total (min, med, max (stageId: taskId))\n0 ms (0 ms, 0 ms, 0 ms (stage 100779.0: task 323712))"; tooltip="WholeStageCodegen (5)"; 32 [id="node32" labelType="html" label=" Project " tooltip="Project [_extract_code#944928, _extract_internalRefUUID#944929, _extract_internalRefUUID#944930, _extract_internalRefUUID#944931]"]; } 33 [id="node33" labelType="html" label="Exchange shuffle records written: 1,462 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 1,462 local bytes read total (min, med, max (stageId: taskId)) 77.5 KiB (1958.0 B, 2.1 KiB, 2.3 KiB (stage 100779.0: task 323783)) merged fetch fallback count: 0 local blocks read: 37 remote merged chunks fetched: 0 remote blocks read: 0 data size: 202.9 KiB local merged bytes read: 0.0 B local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 100779.0: task 323712)) remote bytes read: 0.0 B number of partitions: 37 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 77.5 KiB" tooltip="Exchange RoundRobinPartitioning(37), REPARTITION_BY_NUM, [plan_id=1456895]"]; subgraph cluster34 { isCluster="true"; id="cluster34"; label="WholeStageCodegen (2)\n \nduration: 89 ms"; tooltip="WholeStageCodegen (2)"; 35 [id="node35" labelType="html" label=" Project " tooltip="Project [plant#944363.internalRefUUID AS _extract_internalRefUUID#944931, product#944364.internalRefUUID AS _extract_internalRefUUID#944930, productionAspect#944366.productMovementPlants.unitOfIssue.code AS _extract_code#944928, productionAspect#944366.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#944929]"]; 36 [id="node36" labelType="html" label="ColumnarToRow number of output rows: 1,462 number of input batches: 1" tooltip="ColumnarToRow"]; } 37 [id="node37" labelType="html" label="Scan parquet number of files read: 1 scan time: 86 ms dynamic partition pruning time: 0 ms metadata time: 0 ms size of files read: 80.3 KiB number of output rows: 1,462 number of partitions read: 1" tooltip="FileScan parquet [plant#944363,product#944364,productionAspect#944366,systemId#944361] Batched: true, DataFilters: [], Format: Parquet, Location: PreparedDeltaFileIndex(1 paths)[hdlfs://770f0770-bd0c-456f-b83b-9649926e0857.files.hdl.prod-eu20...., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspe..."]; 38 [id="node38" labelType="html" label="BroadcastExchange time to broadcast: 1 ms time to build: 7 ms time to collect: 82 ms number of output rows: 145 data size: 2.0 MiB" tooltip="BroadcastExchange HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=1456926]"]; 39 [id="node39" labelType="html" label="Exchange shuffle records written: 145 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 145 local bytes read total (min, med, max (stageId: taskId)) 28.8 KiB (573.0 B, 786.0 B, 999.0 B (stage 100781.0: task 323760)) merged fetch fallback count: 0 local blocks read: 37 remote merged chunks fetched: 0 remote blocks read: 0 data size: 88.5 KiB local merged bytes read: 0.0 B local merged chunks fetched: 0 shuffle write time: 1 ms remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 100781.0: task 323714)) remote bytes read: 0.0 B number of partitions: 37 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written: 28.8 KiB" tooltip="Exchange RoundRobinPartitioning(37), REPARTITION_BY_NUM, [plan_id=1456916]"]; subgraph cluster40 { isCluster="true"; id="cluster40"; label="WholeStageCodegen (3)\n \nduration: 118 ms"; tooltip="WholeStageCodegen (3)"; 41 [id="node41" labelType="html" label=" Project " tooltip="Project [internalUUID#944397, baseUnitOfMeasure#944401, unitOfMeasureConversions#944403]"]; 42 [id="node42" labelType="html" label="Filter number of output rows: 145" tooltip="Filter isnotnull(internalUUID#944397)"]; 43 [id="node43" labelType="html" label="ColumnarToRow number of output rows: 145 number of input batches: 1" tooltip="ColumnarToRow"]; } 44 [id="node44" labelType="html" label="Scan parquet number of files read: 1 scan time: 116 ms dynamic partition pruning time: 0 ms metadata time: 0 ms size of files read: 33.6 KiB number of output rows: 145 number of partitions read: 1" tooltip="FileScan parquet [internalUUID#944397,baseUnitOfMeasure#944401,unitOfMeasureConversions#944403,systemId#944396] Batched: true, DataFilters: [isnotnull(internalUUID#944397)], Format: Parquet, Location: PreparedDeltaFileIndex(1 paths)[hdlfs://770f0770-bd0c-456f-b83b-9649926e0857.files.hdl.prod-eu20...., PartitionFilters: [], PushedFilters: [IsNotNull(internalUUID)], ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMea..."]; subgraph cluster45 { isCluster="true"; id="cluster45"; label="WholeStageCodegen (9)\n \nduration: total (min, med, max (stageId: taskId))\n2.9 s (3 ms, 5 ms, 317 ms (stage 100787.0: task 323867))"; tooltip="WholeStageCodegen (9)"; 46 [id="node46" labelType="html" label="Sort sort time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 100787.0: task 323868)) peak memory total (min, med, max (stageId: taskId)) 70.1 MiB (2.1 MiB, 2.1 MiB, 2.1 MiB (stage 100787.0: task 323868)) spill size total (min, med, max (stageId: taskId)) 0.0 B (0.0 B, 0.0 B, 0.0 B (stage 100787.0: task 323868))" tooltip="Sort [coalesce(plant#944815, ) ASC NULLS FIRST, isnull(plant#944815) ASC NULLS FIRST, coalesce(product#944814, ) ASC NULLS FIRST, isnull(product#944814) ASC NULLS FIRST], false, 0"]; } 47 [id="node47" labelType="html" label="AQEShuffleRead number of partitions: 34 partition data size total (min, med, max (stageId: taskId)) 2.8 MiB (50.8 KiB, 85.9 KiB, 146.0 KiB (driver)) number of coalesced partitions: 3" tooltip="AQEShuffleRead coalesced"]; 48 [id="node48" labelType="html" label="Exchange shuffle records written: 434 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 434 local bytes read total (min, med, max (stageId: taskId)) 759.0 KiB (168.0 B, 23.0 KiB, 54.0 KiB (stage 100787.0: task 323861)) merged fetch fallback count: 0 local blocks read: 96 remote merged chunks fetched: 0 remote blocks read: 277 data size total (min, med, max (stageId: taskId)) 4.9 MiB (97.1 KiB, 145.1 KiB, 222.9 KiB (stage 100774.0: task 323671)) local merged bytes read: 0.0 B local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 32 ms (0 ms, 0 ms, 1 ms (stage 100774.0: task 323671)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 100787.0: task 323868)) remote bytes read total (min, med, max (stageId: taskId)) 1954.0 KiB (21.0 KiB, 57.8 KiB, 101.2 KiB (stage 100787.0: task 323880)) number of partitions: 37 remote reqs duration total (min, med, max (stageId: taskId)) 162 ms (1 ms, 3 ms, 13 ms (stage 100787.0: task 323858)) remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 2.6 MiB (54.8 KiB, 80.7 KiB, 124.2 KiB (stage 100774.0: task 323671))" tooltip="Exchange hashpartitioning(coalesce(plant#944815, ), isnull(plant#944815), coalesce(product#944814, ), isnull(product#944814), 37), ENSURE_REQUIREMENTS, [plan_id=1456940]"]; subgraph cluster49 { isCluster="true"; id="cluster49"; label="WholeStageCodegen (4)\n \nduration: total (min, med, max (stageId: taskId))\n4.4 s (71 ms, 98 ms, 305 ms (stage 100774.0: task 323685))"; tooltip="WholeStageCodegen (4)"; 50 [id="node50" labelType="html" label="ColumnarToRow number of output rows: 434 number of input batches: 33" tooltip="ColumnarToRow"]; } 51 [id="node51" labelType="html" label="Scan parquet number of files read: 33 scan time total (min, med, max (stageId: taskId)) 3.5 s (69 ms, 91 ms, 268 ms (stage 100774.0: task 323685)) metadata time: 0 ms size of files read: 1745.1 KiB number of output rows: 434" tooltip="FileScan parquet [product#944814,plant#944815,demandTimeBuckets#944819,demandPointInTime#944822] Batched: true, DataFilters: [], Format: Parquet, Location: InMemoryFileIndex(1 paths)[hdlfs://770f0770-bd0c-456f-b83b-9649926e0857.files.hdl.prod-eu20.hanac..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<product:string,plant:string,demandTimeBuckets:array<struct<demandStartDateTime:timestamp,d..."]; 1->0; 2->1; 3->2; 5->3; 6->5; 8->6; 9->8; 10->9; 11->10; 13->11; 14->13; 15->14; 17->15; 18->17; 19->18; 20->19; 21->20; 23->21; 24->23; 25->24; 26->25; 28->26; 29->28; 30->25; 32->30; 33->32; 35->33; 36->35; 37->36; 38->23; 39->38; 41->39; 42->41; 43->42; 44->43; 46->11; 47->46; 48->47; 50->48; 51->50; }

Details

== Physical Plan ==
AdaptiveSparkPlan (81)
+- == Final Plan ==
   ResultQueryStage (79), Statistics(sizeInBytes=8.0 EiB)
   +- CollectLimit (78)
      +- TableCacheQueryStage (77), Statistics(sizeInBytes=10.3 KiB, rowCount=146)
         +- InMemoryTableScan (1)
               +- InMemoryRelation (2)
                     +- AdaptiveSparkPlan (76)
                     +- == Final Plan ==
                        ResultQueryStage (48)
                        +- * HashAggregate (47)
                           +- ShuffleQueryStage (46), Statistics(sizeInBytes=12.5 KiB, rowCount=146)
                              +- Exchange (45)
                                 +- * HashAggregate (44)
                                    +- * Project (43)
                                       +- * Filter (42)
                                          +- * SortMergeJoin LeftOuter (41)
                                             :- * Sort (34)
                                             :  +- AQEShuffleRead (33)
                                             :     +- ShuffleQueryStage (32), Statistics(sizeInBytes=37.3 KiB, rowCount=434)
                                             :        +- Exchange (31)
                                             :           +- * Project (30)
                                             :              +- * Generate (29)
                                             :                 +- * Project (28)
                                             :                    +- * Generate (27)
                                             :                       +- Project (26)
                                             :                          +- * BroadcastHashJoin LeftOuter BuildRight (25)
                                             :                             :- * Project (16)
                                             :                             :  +- * BroadcastHashJoin LeftOuter BuildRight (15)
                                             :                             :     :- ShuffleQueryStage (6), Statistics(sizeInBytes=37.3 KiB, rowCount=434)
                                             :                             :     :  +- Exchange (5)
                                             :                             :     :     +- * ColumnarToRow (4)
                                             :                             :     :        +- Scan parquet  (3)
                                             :                             :     +- BroadcastQueryStage (14), Statistics(sizeInBytes=2.1 MiB, rowCount=1.46E+3)
                                             :                             :        +- BroadcastExchange (13)
                                             :                             :           +- * Project (12)
                                             :                             :              +- ShuffleQueryStage (11), Statistics(sizeInBytes=202.9 KiB, rowCount=1.46E+3)
                                             :                             :                 +- Exchange (10)
                                             :                             :                    +- * Project (9)
                                             :                             :                       +- * ColumnarToRow (8)
                                             :                             :                          +- Scan parquet  (7)
                                             :                             +- BroadcastQueryStage (24), Statistics(sizeInBytes=2.0 MiB, rowCount=145)
                                             :                                +- BroadcastExchange (23)
                                             :                                   +- ShuffleQueryStage (22), Statistics(sizeInBytes=88.5 KiB, rowCount=145)
                                             :                                      +- Exchange (21)
                                             :                                         +- * Project (20)
                                             :                                            +- * Filter (19)
                                             :                                               +- * ColumnarToRow (18)
                                             :                                                  +- Scan parquet  (17)
                                             +- * Sort (40)
                                                +- AQEShuffleRead (39)
                                                   +- ShuffleQueryStage (38), Statistics(sizeInBytes=4.9 MiB, rowCount=434)
                                                      +- Exchange (37)
                                                         +- * ColumnarToRow (36)
                                                            +- Scan parquet  (35)
                     +- == Initial Plan ==
                        HashAggregate (75)
                        +- Exchange (74)
                           +- HashAggregate (73)
                              +- Project (72)
                                 +- Filter (71)
                                    +- SortMergeJoin LeftOuter (70)
                                       :- Sort (67)
                                       :  +- Exchange (66)
                                       :     +- Project (65)
                                       :        +- Generate (64)
                                       :           +- Project (63)
                                       :              +- Generate (62)
                                       :                 +- Project (61)
                                       :                    +- BroadcastHashJoin LeftOuter BuildRight (60)
                                       :                       :- Project (55)
                                       :                       :  +- BroadcastHashJoin LeftOuter BuildRight (54)
                                       :                       :     :- Exchange (49)
                                       :                       :     :  +- Scan parquet  (3)
                                       :                       :     +- BroadcastExchange (53)
                                       :                       :        +- Project (52)
                                       :                       :           +- Exchange (51)
                                       :                       :              +- Project (50)
                                       :                       :                 +- Scan parquet  (7)
                                       :                       +- BroadcastExchange (59)
                                       :                          +- Exchange (58)
                                       :                             +- Project (57)
                                       :                                +- Filter (56)
                                       :                                   +- Scan parquet  (17)
                                       +- Sort (69)
                                          +- Exchange (68)
                                             +- Scan parquet  (35)
+- == Initial Plan ==
   CollectLimit (80)
   +- InMemoryTableScan (1)
         +- InMemoryRelation (2)
               +- AdaptiveSparkPlan (76)
               +- == Final Plan ==
                  ResultQueryStage (48)
                  +- * HashAggregate (47)
                     +- ShuffleQueryStage (46), Statistics(sizeInBytes=12.5 KiB, rowCount=146)
                        +- Exchange (45)
                           +- * HashAggregate (44)
                              +- * Project (43)
                                 +- * Filter (42)
                                    +- * SortMergeJoin LeftOuter (41)
                                       :- * Sort (34)
                                       :  +- AQEShuffleRead (33)
                                       :     +- ShuffleQueryStage (32), Statistics(sizeInBytes=37.3 KiB, rowCount=434)
                                       :        +- Exchange (31)
                                       :           +- * Project (30)
                                       :              +- * Generate (29)
                                       :                 +- * Project (28)
                                       :                    +- * Generate (27)
                                       :                       +- Project (26)
                                       :                          +- * BroadcastHashJoin LeftOuter BuildRight (25)
                                       :                             :- * Project (16)
                                       :                             :  +- * BroadcastHashJoin LeftOuter BuildRight (15)
                                       :                             :     :- ShuffleQueryStage (6), Statistics(sizeInBytes=37.3 KiB, rowCount=434)
                                       :                             :     :  +- Exchange (5)
                                       :                             :     :     +- * ColumnarToRow (4)
                                       :                             :     :        +- Scan parquet  (3)
                                       :                             :     +- BroadcastQueryStage (14), Statistics(sizeInBytes=2.1 MiB, rowCount=1.46E+3)
                                       :                             :        +- BroadcastExchange (13)
                                       :                             :           +- * Project (12)
                                       :                             :              +- ShuffleQueryStage (11), Statistics(sizeInBytes=202.9 KiB, rowCount=1.46E+3)
                                       :                             :                 +- Exchange (10)
                                       :                             :                    +- * Project (9)
                                       :                             :                       +- * ColumnarToRow (8)
                                       :                             :                          +- Scan parquet  (7)
                                       :                             +- BroadcastQueryStage (24), Statistics(sizeInBytes=2.0 MiB, rowCount=145)
                                       :                                +- BroadcastExchange (23)
                                       :                                   +- ShuffleQueryStage (22), Statistics(sizeInBytes=88.5 KiB, rowCount=145)
                                       :                                      +- Exchange (21)
                                       :                                         +- * Project (20)
                                       :                                            +- * Filter (19)
                                       :                                               +- * ColumnarToRow (18)
                                       :                                                  +- Scan parquet  (17)
                                       +- * Sort (40)
                                          +- AQEShuffleRead (39)
                                             +- ShuffleQueryStage (38), Statistics(sizeInBytes=4.9 MiB, rowCount=434)
                                                +- Exchange (37)
                                                   +- * ColumnarToRow (36)
                                                      +- Scan parquet  (35)
               +- == Initial Plan ==
                  HashAggregate (75)
                  +- Exchange (74)
                     +- HashAggregate (73)
                        +- Project (72)
                           +- Filter (71)
                              +- SortMergeJoin LeftOuter (70)
                                 :- Sort (67)
                                 :  +- Exchange (66)
                                 :     +- Project (65)
                                 :        +- Generate (64)
                                 :           +- Project (63)
                                 :              +- Generate (62)
                                 :                 +- Project (61)
                                 :                    +- BroadcastHashJoin LeftOuter BuildRight (60)
                                 :                       :- Project (55)
                                 :                       :  +- BroadcastHashJoin LeftOuter BuildRight (54)
                                 :                       :     :- Exchange (49)
                                 :                       :     :  +- Scan parquet  (3)
                                 :                       :     +- BroadcastExchange (53)
                                 :                       :        +- Project (52)
                                 :                       :           +- Exchange (51)
                                 :                       :              +- Project (50)
                                 :                       :                 +- Scan parquet  (7)
                                 :                       +- BroadcastExchange (59)
                                 :                          +- Exchange (58)
                                 :                             +- Project (57)
                                 :                                +- Filter (56)
                                 :                                   +- Scan parquet  (17)
                                 +- Sort (69)
                                    +- Exchange (68)
                                       +- Scan parquet  (35)


(1) InMemoryTableScan
Output: []

(2) InMemoryRelation
Arguments: [product#944305, plant#944306], StorageLevel(disk, memory, 2 replicas)

(3) Scan parquet 
Output [2]: [product#944305, plant#944306]
Batched: true
Location: InMemoryFileIndex [hdlfs://770f0770-bd0c-456f-b83b-9649926e0857.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000000678/shardId=0_1_10000000678]
ReadSchema: struct<product:string,plant:string>

(4) ColumnarToRow [codegen id : 1]
Input [2]: [product#944305, plant#944306]

(5) Exchange
Input [2]: [product#944305, plant#944306]
Arguments: hashpartitioning(product#944305, plant#944306, 37), REPARTITION_BY_NUM, [plan_id=1456881]

(6) ShuffleQueryStage
Output [2]: [product#944305, plant#944306]
Arguments: 0

(7) Scan parquet 
Output [4]: [plant#944363, product#944364, productionAspect#944366, systemId#944361]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://770f0770-bd0c-456f-b83b-9649926e0857.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>

(8) ColumnarToRow [codegen id : 2]
Input [4]: [plant#944363, product#944364, productionAspect#944366, systemId#944361]

(9) Project [codegen id : 2]
Output [4]: [plant#944363.internalRefUUID AS _extract_internalRefUUID#944931, product#944364.internalRefUUID AS _extract_internalRefUUID#944930, productionAspect#944366.productMovementPlants.unitOfIssue.code AS _extract_code#944928, productionAspect#944366.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#944929]
Input [4]: [plant#944363, product#944364, productionAspect#944366, systemId#944361]

(10) Exchange
Input [4]: [_extract_internalRefUUID#944931, _extract_internalRefUUID#944930, _extract_code#944928, _extract_internalRefUUID#944929]
Arguments: RoundRobinPartitioning(37), REPARTITION_BY_NUM, [plan_id=1456895]

(11) ShuffleQueryStage
Output [4]: [_extract_internalRefUUID#944931, _extract_internalRefUUID#944930, _extract_code#944928, _extract_internalRefUUID#944929]
Arguments: 1

(12) Project [codegen id : 5]
Output [4]: [_extract_code#944928, _extract_internalRefUUID#944929, _extract_internalRefUUID#944930, _extract_internalRefUUID#944931]
Input [4]: [_extract_internalRefUUID#944931, _extract_internalRefUUID#944930, _extract_code#944928, _extract_internalRefUUID#944929]

(13) BroadcastExchange
Input [4]: [_extract_code#944928, _extract_internalRefUUID#944929, _extract_internalRefUUID#944930, _extract_internalRefUUID#944931]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=1457064]

(14) BroadcastQueryStage
Output [4]: [_extract_code#944928, _extract_internalRefUUID#944929, _extract_internalRefUUID#944930, _extract_internalRefUUID#944931]
Arguments: 4

(15) BroadcastHashJoin [codegen id : 6]
Left keys [4]: [coalesce(product#944305, ), isnull(product#944305), coalesce(plant#944306, ), isnull(plant#944306)]
Right keys [4]: [coalesce(_extract_internalRefUUID#944930, ), isnull(_extract_internalRefUUID#944930), coalesce(_extract_internalRefUUID#944931, ), isnull(_extract_internalRefUUID#944931)]
Join type: LeftOuter
Join condition: None

(16) Project [codegen id : 6]
Output [4]: [product#944305, plant#944306, _extract_code#944928, _extract_internalRefUUID#944929]
Input [6]: [product#944305, plant#944306, _extract_code#944928, _extract_internalRefUUID#944929, _extract_internalRefUUID#944930, _extract_internalRefUUID#944931]

(17) Scan parquet 
Output [4]: [internalUUID#944397, baseUnitOfMeasure#944401, unitOfMeasureConversions#944403, systemId#944396]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://770f0770-bd0c-456f-b83b-9649926e0857.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>

(18) ColumnarToRow [codegen id : 3]
Input [4]: [internalUUID#944397, baseUnitOfMeasure#944401, unitOfMeasureConversions#944403, systemId#944396]

(19) Filter [codegen id : 3]
Input [4]: [internalUUID#944397, baseUnitOfMeasure#944401, unitOfMeasureConversions#944403, systemId#944396]
Condition : isnotnull(internalUUID#944397)

(20) Project [codegen id : 3]
Output [3]: [internalUUID#944397, baseUnitOfMeasure#944401, unitOfMeasureConversions#944403]
Input [4]: [internalUUID#944397, baseUnitOfMeasure#944401, unitOfMeasureConversions#944403, systemId#944396]

(21) Exchange
Input [3]: [internalUUID#944397, baseUnitOfMeasure#944401, unitOfMeasureConversions#944403]
Arguments: RoundRobinPartitioning(37), REPARTITION_BY_NUM, [plan_id=1456916]

(22) ShuffleQueryStage
Output [3]: [internalUUID#944397, baseUnitOfMeasure#944401, unitOfMeasureConversions#944403]
Arguments: 2

(23) BroadcastExchange
Input [3]: [internalUUID#944397, baseUnitOfMeasure#944401, unitOfMeasureConversions#944403]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=1456926]

(24) BroadcastQueryStage
Output [3]: [internalUUID#944397, baseUnitOfMeasure#944401, unitOfMeasureConversions#944403]
Arguments: 5

(25) BroadcastHashJoin [codegen id : 6]
Left keys [1]: [product#944305]
Right keys [1]: [internalUUID#944397]
Join type: LeftOuter
Join condition: None

(26) Project
Output [3]: [product#944305, plant#944306, filter(unitOfMeasureConversions#944403, lambdafunction((((lambda u#944410.measurementUnit1.code = _extract_code#944928) AND (lambda u#944410.measurementUnit1.internalRefUUID = _extract_internalRefUUID#944929)) AND ((lambda u#944410.measurementUnit2.code = baseUnitOfMeasure#944401.code) AND (lambda u#944410.measurementUnit2.internalRefUUID = baseUnitOfMeasure#944401.internalRefUUID))), lambda u#944410, false)) AS filteredUnitOfMeasureConversions#944409]
Input [7]: [product#944305, plant#944306, _extract_code#944928, _extract_internalRefUUID#944929, internalUUID#944397, baseUnitOfMeasure#944401, unitOfMeasureConversions#944403]

(27) Generate [codegen id : 7]
Input [3]: [product#944305, plant#944306, filteredUnitOfMeasureConversions#944409]
Arguments: explode(filteredUnitOfMeasureConversions#944409.quantityNumerator), [product#944305, plant#944306, filteredUnitOfMeasureConversions#944409], true, [quantityNumerator#944417]

(28) Project [codegen id : 7]
Output [3]: [product#944305, plant#944306, filteredUnitOfMeasureConversions#944409.quantityDenominator AS _extract_quantityDenominator#944849]
Input [4]: [product#944305, plant#944306, filteredUnitOfMeasureConversions#944409, quantityNumerator#944417]

(29) Generate [codegen id : 7]
Input [3]: [product#944305, plant#944306, _extract_quantityDenominator#944849]
Arguments: explode(_extract_quantityDenominator#944849), [product#944305, plant#944306], true, [quantityDenominator#944420]

(30) Project [codegen id : 7]
Output [2]: [product#944305, plant#944306]
Input [3]: [product#944305, plant#944306, quantityDenominator#944420]

(31) Exchange
Input [2]: [product#944305, plant#944306]
Arguments: hashpartitioning(coalesce(plant#944306, ), isnull(plant#944306), coalesce(product#944305, ), isnull(product#944305), 37), ENSURE_REQUIREMENTS, [plan_id=1457460]

(32) ShuffleQueryStage
Output [2]: [product#944305, plant#944306]
Arguments: 6

(33) AQEShuffleRead
Input [2]: [product#944305, plant#944306]
Arguments: coalesced

(34) Sort [codegen id : 8]
Input [2]: [product#944305, plant#944306]
Arguments: [coalesce(plant#944306, ) ASC NULLS FIRST, isnull(plant#944306) ASC NULLS FIRST, coalesce(product#944305, ) ASC NULLS FIRST, isnull(product#944305) ASC NULLS FIRST], false, 0

(35) Scan parquet 
Output [4]: [product#944814, plant#944815, demandTimeBuckets#944819, demandPointInTime#944822]
Batched: true
Location: InMemoryFileIndex [hdlfs://770f0770-bd0c-456f-b83b-9649926e0857.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-demand-service/out/demand/10000000678/0_1_10000000678]
ReadSchema: struct<product:string,plant:string,demandTimeBuckets:array<struct<demandStartDateTime:timestamp,demandQuantity:double,demandVariance:double>>,demandPointInTime:array<struct<demandDateTime:timestamp,demandQuantity:double,demandVariance:double>>>

(36) ColumnarToRow [codegen id : 4]
Input [4]: [product#944814, plant#944815, demandTimeBuckets#944819, demandPointInTime#944822]

(37) Exchange
Input [4]: [product#944814, plant#944815, demandTimeBuckets#944819, demandPointInTime#944822]
Arguments: hashpartitioning(coalesce(plant#944815, ), isnull(plant#944815), coalesce(product#944814, ), isnull(product#944814), 37), ENSURE_REQUIREMENTS, [plan_id=1456940]

(38) ShuffleQueryStage
Output [4]: [product#944814, plant#944815, demandTimeBuckets#944819, demandPointInTime#944822]
Arguments: 3

(39) AQEShuffleRead
Input [4]: [product#944814, plant#944815, demandTimeBuckets#944819, demandPointInTime#944822]
Arguments: coalesced

(40) Sort [codegen id : 9]
Input [4]: [product#944814, plant#944815, demandTimeBuckets#944819, demandPointInTime#944822]
Arguments: [coalesce(plant#944815, ) ASC NULLS FIRST, isnull(plant#944815) ASC NULLS FIRST, coalesce(product#944814, ) ASC NULLS FIRST, isnull(product#944814) ASC NULLS FIRST], false, 0

(41) SortMergeJoin [codegen id : 10]
Left keys [4]: [coalesce(plant#944306, ), isnull(plant#944306), coalesce(product#944305, ), isnull(product#944305)]
Right keys [4]: [coalesce(plant#944815, ), isnull(plant#944815), coalesce(product#944814, ), isnull(product#944814)]
Join type: LeftOuter
Join condition: None

(42) Filter [codegen id : 10]
Input [6]: [product#944305, plant#944306, product#944814, plant#944815, demandTimeBuckets#944819, demandPointInTime#944822]
Condition : (((size(demandTimeBuckets#944819, true) = 0) OR isnull(demandTimeBuckets#944819)) AND ((size(demandPointInTime#944822, true) = 0) OR isnull(demandPointInTime#944822)))

(43) Project [codegen id : 10]
Output [2]: [product#944305, plant#944306]
Input [6]: [product#944305, plant#944306, product#944814, plant#944815, demandTimeBuckets#944819, demandPointInTime#944822]

(44) HashAggregate [codegen id : 10]
Input [2]: [product#944305, plant#944306]
Keys [2]: [product#944305, plant#944306]
Functions: []
Aggregate Attributes: []
Results [2]: [product#944305, plant#944306]

(45) Exchange
Input [2]: [product#944305, plant#944306]
Arguments: hashpartitioning(product#944305, plant#944306, 37), ENSURE_REQUIREMENTS, [plan_id=1457554]

(46) ShuffleQueryStage
Output [2]: [product#944305, plant#944306]
Arguments: 7

(47) HashAggregate [codegen id : 11]
Input [2]: [product#944305, plant#944306]
Keys [2]: [product#944305, plant#944306]
Functions: []
Aggregate Attributes: []
Results [2]: [product#944305, plant#944306]

(48) ResultQueryStage
Output [2]: [product#944305, plant#944306]
Arguments: 8

(49) Exchange
Input [2]: [product#944305, plant#944306]
Arguments: hashpartitioning(product#944305, plant#944306, 37), REPARTITION_BY_NUM, [plan_id=1456767]

(50) Project
Output [4]: [plant#944363.internalRefUUID AS _extract_internalRefUUID#944931, product#944364.internalRefUUID AS _extract_internalRefUUID#944930, productionAspect#944366.productMovementPlants.unitOfIssue.code AS _extract_code#944928, productionAspect#944366.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#944929]
Input [4]: [plant#944363, product#944364, productionAspect#944366, systemId#944361]

(51) Exchange
Input [4]: [_extract_internalRefUUID#944931, _extract_internalRefUUID#944930, _extract_code#944928, _extract_internalRefUUID#944929]
Arguments: RoundRobinPartitioning(37), REPARTITION_BY_NUM, [plan_id=1456770]

(52) Project
Output [4]: [_extract_code#944928, _extract_internalRefUUID#944929, _extract_internalRefUUID#944930, _extract_internalRefUUID#944931]
Input [4]: [_extract_internalRefUUID#944931, _extract_internalRefUUID#944930, _extract_code#944928, _extract_internalRefUUID#944929]

(53) BroadcastExchange
Input [4]: [_extract_code#944928, _extract_internalRefUUID#944929, _extract_internalRefUUID#944930, _extract_internalRefUUID#944931]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=1456791]

(54) BroadcastHashJoin
Left keys [4]: [coalesce(product#944305, ), isnull(product#944305), coalesce(plant#944306, ), isnull(plant#944306)]
Right keys [4]: [coalesce(_extract_internalRefUUID#944930, ), isnull(_extract_internalRefUUID#944930), coalesce(_extract_internalRefUUID#944931, ), isnull(_extract_internalRefUUID#944931)]
Join type: LeftOuter
Join condition: None

(55) Project
Output [4]: [product#944305, plant#944306, _extract_code#944928, _extract_internalRefUUID#944929]
Input [6]: [product#944305, plant#944306, _extract_code#944928, _extract_internalRefUUID#944929, _extract_internalRefUUID#944930, _extract_internalRefUUID#944931]

(56) Filter
Input [4]: [internalUUID#944397, baseUnitOfMeasure#944401, unitOfMeasureConversions#944403, systemId#944396]
Condition : isnotnull(internalUUID#944397)

(57) Project
Output [3]: [internalUUID#944397, baseUnitOfMeasure#944401, unitOfMeasureConversions#944403]
Input [4]: [internalUUID#944397, baseUnitOfMeasure#944401, unitOfMeasureConversions#944403, systemId#944396]

(58) Exchange
Input [3]: [internalUUID#944397, baseUnitOfMeasure#944401, unitOfMeasureConversions#944403]
Arguments: RoundRobinPartitioning(37), REPARTITION_BY_NUM, [plan_id=1456777]

(59) BroadcastExchange
Input [3]: [internalUUID#944397, baseUnitOfMeasure#944401, unitOfMeasureConversions#944403]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=1456795]

(60) BroadcastHashJoin
Left keys [1]: [product#944305]
Right keys [1]: [internalUUID#944397]
Join type: LeftOuter
Join condition: None

(61) Project
Output [3]: [product#944305, plant#944306, filter(unitOfMeasureConversions#944403, lambdafunction((((lambda u#944410.measurementUnit1.code = _extract_code#944928) AND (lambda u#944410.measurementUnit1.internalRefUUID = _extract_internalRefUUID#944929)) AND ((lambda u#944410.measurementUnit2.code = baseUnitOfMeasure#944401.code) AND (lambda u#944410.measurementUnit2.internalRefUUID = baseUnitOfMeasure#944401.internalRefUUID))), lambda u#944410, false)) AS filteredUnitOfMeasureConversions#944409]
Input [7]: [product#944305, plant#944306, _extract_code#944928, _extract_internalRefUUID#944929, internalUUID#944397, baseUnitOfMeasure#944401, unitOfMeasureConversions#944403]

(62) Generate
Input [3]: [product#944305, plant#944306, filteredUnitOfMeasureConversions#944409]
Arguments: explode(filteredUnitOfMeasureConversions#944409.quantityNumerator), [product#944305, plant#944306, filteredUnitOfMeasureConversions#944409], true, [quantityNumerator#944417]

(63) Project
Output [3]: [product#944305, plant#944306, filteredUnitOfMeasureConversions#944409.quantityDenominator AS _extract_quantityDenominator#944849]
Input [4]: [product#944305, plant#944306, filteredUnitOfMeasureConversions#944409, quantityNumerator#944417]

(64) Generate
Input [3]: [product#944305, plant#944306, _extract_quantityDenominator#944849]
Arguments: explode(_extract_quantityDenominator#944849), [product#944305, plant#944306], true, [quantityDenominator#944420]

(65) Project
Output [2]: [product#944305, plant#944306]
Input [3]: [product#944305, plant#944306, quantityDenominator#944420]

(66) Exchange
Input [2]: [product#944305, plant#944306]
Arguments: hashpartitioning(coalesce(plant#944306, ), isnull(plant#944306), coalesce(product#944305, ), isnull(product#944305), 37), ENSURE_REQUIREMENTS, [plan_id=1456804]

(67) Sort
Input [2]: [product#944305, plant#944306]
Arguments: [coalesce(plant#944306, ) ASC NULLS FIRST, isnull(plant#944306) ASC NULLS FIRST, coalesce(product#944305, ) ASC NULLS FIRST, isnull(product#944305) ASC NULLS FIRST], false, 0

(68) Exchange
Input [4]: [product#944814, plant#944815, demandTimeBuckets#944819, demandPointInTime#944822]
Arguments: hashpartitioning(coalesce(plant#944815, ), isnull(plant#944815), coalesce(product#944814, ), isnull(product#944814), 37), ENSURE_REQUIREMENTS, [plan_id=1456805]

(69) Sort
Input [4]: [product#944814, plant#944815, demandTimeBuckets#944819, demandPointInTime#944822]
Arguments: [coalesce(plant#944815, ) ASC NULLS FIRST, isnull(plant#944815) ASC NULLS FIRST, coalesce(product#944814, ) ASC NULLS FIRST, isnull(product#944814) ASC NULLS FIRST], false, 0

(70) SortMergeJoin
Left keys [4]: [coalesce(plant#944306, ), isnull(plant#944306), coalesce(product#944305, ), isnull(product#944305)]
Right keys [4]: [coalesce(plant#944815, ), isnull(plant#944815), coalesce(product#944814, ), isnull(product#944814)]
Join type: LeftOuter
Join condition: None

(71) Filter
Input [6]: [product#944305, plant#944306, product#944814, plant#944815, demandTimeBuckets#944819, demandPointInTime#944822]
Condition : (((size(demandTimeBuckets#944819, true) = 0) OR isnull(demandTimeBuckets#944819)) AND ((size(demandPointInTime#944822, true) = 0) OR isnull(demandPointInTime#944822)))

(72) Project
Output [2]: [product#944305, plant#944306]
Input [6]: [product#944305, plant#944306, product#944814, plant#944815, demandTimeBuckets#944819, demandPointInTime#944822]

(73) HashAggregate
Input [2]: [product#944305, plant#944306]
Keys [2]: [product#944305, plant#944306]
Functions: []
Aggregate Attributes: []
Results [2]: [product#944305, plant#944306]

(74) Exchange
Input [2]: [product#944305, plant#944306]
Arguments: hashpartitioning(product#944305, plant#944306, 37), ENSURE_REQUIREMENTS, [plan_id=1456813]

(75) HashAggregate
Input [2]: [product#944305, plant#944306]
Keys [2]: [product#944305, plant#944306]
Functions: []
Aggregate Attributes: []
Results [2]: [product#944305, plant#944306]

(76) AdaptiveSparkPlan
Output [2]: [product#944305, plant#944306]
Arguments: isFinalPlan=true

(77) TableCacheQueryStage
Output: []
Arguments: 0

(78) CollectLimit
Input: []
Arguments: 1

(79) ResultQueryStage
Output: []
Arguments: 1

(80) CollectLimit
Input: []
Arguments: 1

(81) AdaptiveSparkPlan
Output: []
Arguments: isFinalPlan=true

SQL / DataFrame Properties

Name	Value
spark.activity.hdlfsEnabled	true
spark.activity.isolateSession	true
spark.sql.parquet.fieldId.read.enabled	true
spark.sql.parquet.fieldId.write.enabled	true
temporal.activity.id	372339e3-e78f-3202-9d96-da74ae3e269f
temporal.activity.type	BufferDataSnapShot