Google Cloud Professional Data Engineer Exam

What is a key consideration for disaster recovery when using Dataflow for streaming data?

1. Take Dataflow snapshots periodically.

2. Create Dataflow jobs from templates.

3. Enable vertical autoscaling.

4. Enable Dataflow shuffle.

The correct answer is: Take Dataflow snapshots periodically.

A key consideration for disaster recovery when using Dataflow for streaming data is the need for resilience and the ability to recover from unexpected failures or interruptions in processing. Taking Dataflow snapshots periodically allows you to capture the state of your streaming job at specific points in time. This is crucial because in the event of a failure, you can restore the job from the most recent snapshot, minimizing data loss and ensuring a more efficient recovery process. By capturing snapshots, you create restore points that can help in diagnosing issues that may arise and assist in rolling back to a stable state without significant data loss. This capability is particularly important in streaming contexts, where data is continuously ingested, and being able to recover quickly can significantly affect the integrity and availability of data processing. Other considerations, while important for job performance and management, do not specifically address the resilience and recovery aspect required for disaster recovery. For instance, creating jobs from templates is useful for consistency and ease of deployment, enabling vertical autoscaling pertains more to performance optimization, and enabling Dataflow shuffle is related to efficiency in data processing but does not contribute directly to the layering of resilience in the disaster recovery strategy.