You are creating a new pipeline in Google Cloud to stream IoT data from Cloud Pub/Sub through Cloud Dataflow to BigQuery. While previewing the data, you notice that roughly 2% of the data appears to be corrupt. You need to modify the Cloud Dataflow pipeline to filter out this corrupt data. What should you do?
A) Add a SideInput that returns a Boolean if the element is corrupt.
B) Add a ParDo transform in Cloud Dataflow to discard corrupt elements.
C) Add a Partition transform in Cloud Dataflow to separate valid data from corrupt data.
D) Add a GroupByKey transform in Cloud Dataflow to group all of the valid data together and discard the rest.

Question

Quizplus · Accepted Answer

A ParDo transform can be used to process each element in the pipeline, allowing you to apply a function that filters out corrupt data by discarding elements that do not meet certain criteria.

You Are Creating a New Pipeline in Google Cloud to Stream