Data Engineer – Data Platform

Sobre a vaga

⭐ JOIN THE EXITLAG CLAN Are you passionate about games, technology, and challenges that truly make a difference? We’re the first AI-powered game optimization platform, ensuring maximum performance and connection stability for millions of players in over 190 countries. We were founded in Brazil, but we operate with a global mindset, a high-performance culture, and a total focus on the user experience. Here, gaming is serious business, and we always deliver in PRO mode.🎮 🚀 Why join ExitLag? ✅ GPTW – Great Place To Work A workplace recognized by people, with a strong culture, collaboration, and a team that plays together every day. ✅ Real global impact Your work impacts millions of gamers around the world and helps transform the gaming experience every day. ✅ Accelerated growth Here you’ll find autonomy, leadership opportunities, real challenges, and room to grow both technically and professionally. ✅ A borderless mindset We’re Remote First and operate globally. Where you are never limits the scope of your career. Our EVP | Employee Value Proposition At ExitLag, you’ll find more than just a job, you’ll find a journey with purpose. We want you to: 🎯 Connect with your purpose 🚀 Take pride in what you build 🤝 Grow alongside people who challenge and support you every day 🌍 Be part of a global, innovative, and constantly evolving company 🎮 Experience a culture that truly believes in the power of gaming and technology Job Purpose Build and evolve ExitLag’s data platform, ensuring that data is ingested, stored, and made available in a reliable, scalable, and governed manner. The individual will be responsible for the technical foundation that supports the Analytics, Data Science, Product, and Growth teams. We have a culture of in-house development: our lakehouse on Apache Iceberg is being built from scratch alongside the infrastructure team, on top of Kubernetes, and the new ingestion suite will be managed by dbt, with data contracts and dead-letter queues from day one. Key Responsibilities Build and evolve the lakehouse using Apache Iceberg, in close collaboration with the infrastructure team. Design and develop the ingestion suite managed by dbt and orchestrated on Kubernetes, with data contracts, schema validation, and a DLQ for handling invalid records. Define and maintain data contracts with producer teams, including rules for schema evolution and compatibility. Implement and operate CDC from transactional databases (MySQL/MariaDB and PostgreSQL). Develop and operate streaming applications in Flink, consuming and producing data in Kafka, with a focus on low latency and fault tolerance. Develop and maintain batch pipelines orchestrated in Airflow. Manage the analytics layer in ClickHouse Cloud, optimizing modeling, query performance, and costs. Ensure data quality, observability, and reliability (testing, monitoring, alerts, and reprocessing from the DLQ). Document all processes, pipelines, and implementations clearly, comprehensively, and in a timely manner, including architecture, technical decisions, runbooks, and data contracts. Build robust, durable, and easily maintainable solutions, prioritizing clean code, automated testing, and consistent standards across projects. Create reusable components and best practices that enable self-service for user teams. Participate in decisions regarding the platform's architecture and roadmap. Technical Requirements 3+ years of experience in data engineering or software engineering with a focus on data. Advanced Python and SQL; Java or Scala (a plus, especially for Flink). Streaming with Apache Flink (Spark Structured Streaming, Kafka Streams, Bytewax, or similar technologies are also considered), including event time, watermarks, windows, and state management. Kafka (Pulsar, Kinesis, or Pub/Sub are also considered). CDC from relational databases (MySQL/MariaDB binlog, PostgreSQL WAL), using Debezium, Flink CDC, or in-house solutions. Data contracts and schema management (Avro, Protobuf, or J

Habilidades

O que você vai fazer

• Construir e evoluir a lakehouse com Apache Iceberg.
• Desenvolver a suíte de ingestão com dbt e Kubernetes.
• Definir e manter contratos de dados com equipes produtoras.
• Implementar e operar CDC de bancos de dados transacionais.
• Desenvolver e operar aplicações de streaming em Flink.

O que você precisa ter

• 3+ anos de experiência em engenharia de dados ou software.
• Python e SQL avançados.
• Experiência com streaming em Apache Flink.
• Experiência com Kafka.
• Experiência com CDC de bancos de dados relacionais.

Benefícios

Perguntas frequentes