Skip to main content
← Back to board (資訊系統設計)

#Slack replaced SSH-based execution in Amazon EMR pipelines with Quarry - a…

by InfoQ InfoQ@techhub.social ·
Posted in 資訊系統設計

#Slack replaced SSH-based execution in Amazon EMR pipelines with Quarry - a REST-driven orchestration layer.

The migration covered 700+ Airflow operators, improving security, reliability & observability while removing direct SSH access to production clusters and enabling server-side job lifecycle management.

🔗 Learn more: https://bit.ly/4eJs0V6

#InfoQ #Observability #Migration #DataPipelines #SoftwareArchitecture

Preview image for Slack Eliminates SSH in EMR Pipelines, Migrates 700+ Jobs to Rest-Based Architecture

Slack Eliminates SSH in EMR Pipelines, Migrates 700+ Jobs to Rest-Based Architecture

Slack modernized its data platform by replacing SSH based execution in Amazon EMR pipelines with a REST driven orchestration layer called Quarry. The migration covered 700 plus Airflow operators, improving security, reliability, and observability while eliminating direct SSH access across production clusters and enabling a server side job lifecycle model.

bit.ly
Image 1
View original 0 Likes 0 Boosts

Comments (0)

No comments yet.