Migrate Cron Jobs to Dataflow

Proposing a migration of current cron jobs from individual microservices to a more centralized and scalable system using Google Cloud Dataflow (or a similar scheduling pipeline architecture).

Context
Currently, we have several cron jobs scattered across different microservices handling time-sensitive logic — for example:

• Scheduled escrow releases(payment)

• emitting storage usage (storage)

• handling course visibility (course)

This setup makes the system harder to manage, debug, and scale, especially as the number of scheduled operations grows.

Proposal
Move these types of time-based operational tasks to a centralized Dataflow job (or other batch-processing pipelines), scheduled via Cloud Scheduler, and fed by a reliable data source.

Benefits

• Centralization: All scheduled logic handled in one place

• Scalability: Dataflow can handle thousands of scheduled actions in parallel

• Decoupling: Business logic separated from microservice infrastructure

• Monitoring & retries: More robust than ad-hoc cron implementations