An external customer provides you with a daily dump of data from their database. The data flows into Google Cloud Storage GCS as comma-separated values (CSV) files. You want to analyze this data in Google BigQuery, but the data could have rows that are formatted incorrectly or corrupted. How should you build this pipeline?
A) Use federated data sources, and check data in the SQL query.
B) Enable BigQuery monitoring in Google Stackdriver and create an alert.
C) Import the data into BigQuery using the gcloud CLI and set max_bad_records to 0 . Import the data into BigQuery using the gcloud CLI and set max_bad_records to 0 .
D) Run a Google Cloud Dataflow batch pipeline to import the data into BigQuery, and push errors to another dead-letter table for analysis.

Question

Quizplus · Accepted Answer

Running a Google Cloud Dataflow batch pipeline allows you to process and clean the data before importing it into BigQuery. Errors can be pushed to a dead-letter table for further analysis, ensuring that only correctly formatted data is imported.

An External Customer Provides You with a Daily Dump of Data