Google Cloud Professional Data Engineer Exam

For large amounts of data stored on both Cloud Storage and BigQuery, what should you create to facilitate internal data discovery?

1. Create a lake for Cloud Storage data and a zone for BigQuery data.

2. Create a lake for BigQuery data and a zone for Cloud Storage data.

3. Create a lake for unprocessed data and assets for processed data.

4. Create a raw zone for the unprocessed data and a curated zone for the processed data.

The correct answer is: Create a raw zone for the unprocessed data and a curated zone for the processed data.

Creating a raw zone for unprocessed data and a curated zone for processed data is the most effective approach to facilitate internal data discovery, especially when dealing with large amounts of data across Cloud Storage and BigQuery. A raw zone typically serves as a centralized repository where original, unprocessed data can be stored. This is essential for preserving the integrity of the original datasets and allows for easy access when data exploration or reprocessing is needed. It acts as a comprehensive data lake, enabling users to discover unrefined data that may be useful for various analytics. On the other hand, the curated zone is where processed data resides. This area contains data that has been cleansed, transformed, and structured in a way that is optimized for querying and analysis. By segregating processed data in a curated environment, users can effectively discover and utilize high-quality datasets tailored for specific analytical tasks or reporting requirements. This two-zone architecture enhances data governance, simplifies data management, and promotes data literacy among users, thereby aiding in data discovery across the organization. It establishes a clear workflow from raw data ingestion to meaningful insights, making it easier for data engineers and analysts to navigate through the vast amounts of data.