The Junior Data Engineer is Now an AI Agent - The Data Exchange Podcast Recap

Podcast: The Data Exchange Podcast

Published: 2026-01-08

Duration: 55 minutes

Guests: Matthew Glickman

Summary

Matthew Glickman discusses how Genesis Computing is transforming data engineering by using AI agents to automate routine tasks, freeing up human engineers for complex work. This shift aims to enhance productivity and address challenges in deploying AI models in enterprise environments.

What Happened

Matthew Glickman, Co-founder and CEO of Genesis Computing, dives into the challenges and solutions in automating enterprise data workflows. He describes how Genesis Computing's AI agents act like junior data engineers, automating routine tasks and allowing human engineers to focus on more complex issues. This is crucial in data engineering, where teams are often overwhelmed by the sheer volume of work.

Glickman emphasizes the difficulty enterprises face in deploying AI models, particularly the last 10% of a project where challenges become significantly steep. Genesis Computing addresses this by ensuring its AI systems are installed within customer environments, maintaining data security and proprietary knowledge.

The AI agents are designed to connect with platforms like Databricks and Snowflake, acquiring institutional knowledge over time to reduce the risk of losing critical information when employees leave. This capability is particularly beneficial in legacy system migrations, such as transitioning from SAP to modern platforms.

Genesis Computing utilizes commercial AI models from top-tier companies like Anthropic and OpenAI, which continue to improve for data engineering use cases. Glickman notes that benchmarks for AI models are often gamed, making real workflow testing essential for accurate evaluation.

The conversation also covers the impact of AI on data engineering jobs. Glickman predicts that AI systems are more likely to replace entry-level data engineering roles, affecting the pipeline for senior positions. He stresses the importance of educational systems adapting to include AI as a core skill to prepare students for future job markets.

Glickman argues that the real value of AI systems lies in the time they save, not just in productivity increases. This shift in measurement reflects a broader trend in the industry where AI systems are valued for the efficiency and time savings they bring to enterprise data workflows.

Key Insights

• Genesis Computing's AI agents automate routine tasks in data engineering, acting as junior data engineers and allowing human engineers to focus on complex issues. This approach helps manage the overwhelming volume of work in enterprise data workflows.
• Deploying AI models in enterprises faces significant challenges, particularly in the final 10% of a project. Genesis Computing addresses this by installing AI systems within customer environments to maintain data security and proprietary knowledge.
• AI agents from Genesis Computing connect with platforms like Databricks and Snowflake to acquire institutional knowledge over time, reducing the risk of losing critical information during legacy system migrations, such as moving from SAP to modern platforms.
• AI's impact on data engineering jobs is likely to replace entry-level roles, affecting the pipeline for senior positions. Educational systems need to adapt by including AI as a core skill to prepare students for future job markets.