Ep193: The Conductor Behind Your Data Orchestra: Astronomer's Approach to AI Pipeline Management
Échec de l'ajout au panier.
Veuillez réessayer plus tard
Échec de l'ajout à la liste d'envies.
Veuillez réessayer plus tard
Échec de la suppression de la liste d’envies.
Veuillez réessayer plus tard
Échec du suivi du balado
Ne plus suivre le balado a échoué
Ep193: The Conductor Behind Your Data Orchestra: Astronomer's Approach to AI Pipeline Management
-
Narrateur(s):
-
Auteur(s):
À propos de cet audio
Astronomer's Steven Hillion reveals how OpenAI, Anthropic, Uber, and Lyft use Apache Airflow to orchestrate AI and machine learning pipelines at scale on AWS.
Topics Include:
- Steven Hillion leads data and AI at Astronomer
- Apache Airflow surpassed Spark and Kafka in community metrics
- Astronomer coordinates data flow like conductor orchestrating instrumental platforms
- Organizations with data engineering teams use Airflow at scale
- Customers already used Airflow for ML before official promotion
- Uber and Lyft orchestrate pricing models using Airflow
- Astronomer runs on AWS with close integration partnerships
- OpenAI Anthropic and GitHub Copilot use Airflow for operations
- Internal data team uses Airflow creating feedback loops
- Evolved from constrained AI reports to agentic workflows
- Platform monitors generative AI output quality at user interactions
- Metadata and context increasingly critical for AI applications
- Learn more at Astronomer’s Data FlowCast podcast
Participants:
- Steven Hillion – SVP, Data and AI, Astronomer
See how Amazon Web Services gives you the freedom to migrate, innovate, and scale your software company at https://aws.amazon.com/isv/
Pas encore de commentaire