Data Engineering

The AI-Ready Enterprise: Preparing Data for the Age of AI

A practical framework for building data systems that AI can find, understand, trust, and use.Conversations around enterprise AI are moving rapidly from models to applications. Foundational models are becoming increasingly capable, and organisations are experimenting with retrieval augmented generation (RAG), tool calling, AI agents and multi-agent systems to bring intelligence into their business processes. Yet, as these systems move closer to production, a familiar constraint emerges: the data they need was rarely designed for AI consumption. Enterprise data has traditionally been...

Read

Building the Next-Generation Data and AI Platform for SiteMinder

SiteMinder is the global leader in hotel commerce, with 56,000 hotel customers that use its platform to generate more than AU$85 billion in revenue each year. As a category leader, maintaining its competitive edge was important and so it embarked on a journey of pioneering advanced AI capabilities, including an intelligent "Room Nights Forecasting" engine for its groundbreaking Dynamic Revenue Plus product, which uniquely provides hotels live market intelligence as local demand changes.The company faced significant bottlenecks upfront: Firstly, SiteMinder needed...

Read

Observability for AI-Assisted Development

Developers using AI estimate they’re 24% faster. A randomised controlled trial measured them at 19% slower.That’s from METR’s 2025 study. These were experienced open-source developers working on their own codebases with tools they chose. Their self-assessment was off by over 40 percentage points.If your perception of AI’s impact is that unreliable, what are you actually measuring?If you didn’t measure before AI, measuring with AI won’t work.You can’t attribute improvements to AI if you don’t know what “before” looked like. Cycle...

Read

The Judgement Pyramid: Reasoning vs Measurement

A team I was talking to had built a code-review skill in Claude Code. It looked good: a careful prompt, a set of rules, and examples. Then it shipped a PR with a nested recursive loop inside another recursive loop. Cyclomatic complexity any static analyser (PMD, radon, ESLint’s complexity rule) would have flagged in milliseconds. The skill missed it.The team’s response was to iterate the skill. More examples. More rules. A few more lines of prompt. The context window kept...

Read

The Unix Philosophy for Agentic Coding

Most people use AI coding agents backwards. They hand the agent a problem and ask it to solve the whole thing. The agent reads, reasons, generates, and hopes for the best.There’s a better way. One that’s cheaper, more predictable, and already well understood. It’s the Unix philosophy, applied to how we work with AI.The Unix philosophy boils down to: do one thing well, compose small tools, let the shell orchestrate. When you work with an AI coding agent, the agent...

Read

A Skill Map for Learning AI-Assisted Development

Principles are useful, but they don’t tell you what to practice.In my previous post on intelligent Engineering principles, I outlined the ideas that guide how I build software with AI. Since then, I’ve had people ask: “Where do I start? What skills should I build first?”This post answers that: a map of the skills that make up intelligent Engineering, organised into a learning path you can follow whether you’re an individual contributor looking to level up or a tech leader...

Read

Context Engineering for AI-Assisted Development

Same model, different tools, different results.If you’ve used Claude Sonnet in Claude Code, Cursor, Copilot, and Windsurf, you’ve noticed this. The model is identical, but the behavior varies. This isn’t magic. It’s context engineering.In Principles for Building With AI, I mentioned that “context is everything” and that “context engineering matters more than prompt engineering.” But I didn’t explain what that means or how to do it. This post fills that gap.Imagine you’re in a day-long strategy meeting. There’s one whiteboard...

Read

Structuring Claude Code for Multi-Repo Workspaces

Claude Code understands one repo at a time. Most teams have thirty.Microservices, shared libraries, infrastructure-as-code, frontend apps, data pipelines, all in separate git repos. Start Claude Code in one and ask about another, and it has no context. It doesn’t know the workspace exists.Here’s how I’ve been setting this up to work across repositories.When you start Claude Code in orders/order-service, it has no idea that orders/orders-ui exists next door, or that shared libraries live in shared/, or that the data...

Read

The Comfort Plateau AI Built For You

I’ve spent the last year watching engineers and writers adopt these tools, and the same pattern keeps showing up. The first three months look transformative. The next nine look almost identical to the first three.The output keeps coming. The confidence keeps building. The skill stops moving.This isn’t a story about AI making people dumber. It’s a story about a comfort plateau that didn’t exist before, and what it costs to stay on it.Draw a spider chart. Hundreds of axes: SQL,...

Read

Prompting Is Architecture: Why Understanding Systems Matters More Than Ever

A Spark story about scale, grain, and why speed makes system awareness non-negotiable.Image generated using ChatGPTRecently, I ran into a Spark job that looked perfectly fine. I used Claude in this case to generate code, which wasSyntactically correctSemantically correctEasy to read and reason aboutThis worked on the sample dataset. Assuming Spark will manage the scale, we pushed it to production. However, on prod, it took more than 3 hours to complete on a 40-node EMR cluster. It was trying to...

Read