“What I enjoy the most is getting to the fundamental heart of the problem that people are trying to solve, which isn’t always what they may think they need to solve.”

At Sahaj Software, people own the dreams, ideas, ideologies, successes and failures as a collective–a synergy that is rooted in the ethos of honesty, respect, trust, and equitability. 

In the “Inside Sahaj” series, we meet the people of Sahaj Software and find out about what inspires them, what keeps them motivated and going, and discover about their journeys in the world of tech.

Here, we meet Carmen Madiros and find out how she got her start in data engineering and what superpower she wishes she had at work.

We know you’re an expert on sleep routines. What’s the most interesting fact about sleep that you have come across? 

The most interesting fact about sleep I’ve learned is that it is the most important thing you can do for your health. It trumps everything else including nutrition and exercise. So, those of us who may decide to wake up earlier to hit the gym or go running are doing themselves a disservice by cutting down on sleep to exercise.

And in terms of a sleep routine, I don’t know if it’s fun, but I brush my teeth in the dark. Once I go upstairs and get ready for bed, the upstairs is completely dark. My whole nighttime routine is in the dark to help me prepare for sleep.

What was your dream job as a child and how does that differ from what you do now?

I didn’t actually have a dream job as a child; I just thought about things I wanted to do. I was really interested in computers, Lego, and putting things together and fixing them. I guess what I do now is just a slight variation of that—it still involves computers and figuring out how things work, putting them together, and taking them apart.

Did you start your tech career as a data engineer or has your role evolved into what you do today?

It has evolved into this. I started my Tech Career building and managing my content websites, which I monetised through advertising and affiliate marketing. That was at the start of 2002, a ridiculous time ago. I studied marketing, so I didn’t study anything related to computers, I just up-skilled myself.

I started off managing my own sites, working more around the analytics routes. I enjoyed finding insights into data to optimise my websites and their revenue potential. Gradually, I had to become more tech-savvy to collect better data and make better decisions. 

I later decided to join an e-commerce company as an analyst. In the role, I realised I did not have the data I needed to do my job. So, that led me to the path of data engineering, which is a common theme for many data scientists. They end up in positions where they have to leverage data that isn’t easily available, or not curated or collected properly. It was an opportunity that allowed me to up-skill in technology and for data engineering I set up pipelines for the first time. That allowed me to bring data in, transform it, and make it usable so I could do my job. 

Fast forwarding to today, I realise I like the process of building capabilities to answer questions with data more than doing actual insights. I enjoy enabling others to extract insights from data much more and ensuring they have the infrastructure and the capabilities to do that.

What inspired you to start your own websites?

It started when I was in my first year of University. I created a website about my favourite band at the time. That somehow led me down the rabbit hole of realising I could make money from displaying ads. I would then deliberately organise topics for different websites that would fit that business model effectively.

This was in the early days when Search Engine Optimisation (SEO) was still very young. The Google algorithm was simple and worked well for a while. I then became more interested in the data than the SEO side.

I have to ask, what was your favourite band?

It was an Irish band called The Corrs. They were quite popular in the North East a very very long time ago. 

What advice would you give to someone who is thinking about a career in data engineering?

It is a very exciting field. It’s fast-moving and very rewarding, so the first thing that comes to mind is not really advice but more to say good for you for considering data engineering because it really sets you up well for the future.

In terms of advice, try to get a sense of what type of data engineering you’d like to do, as the field is becoming extremely specialised. For example, if you have an interest in business questions and problems, you might find analytics engineering more interesting, as it involves getting data into a format that supports business insights and drives business optimisation. It’s closer to where decisions are made and where the value of data is realised.

If you prefer backend work, infrastructure, and systems, then data platform and related infrastructure might be more your area. This involves moving huge amounts of data around cloud infrastructures and handling incremental transformations of billions of events. The technology stack and approaches are very different. In many ways, it’s simpler, less nuanced, and less opinionated than analytics engineering, which always involves adding interpretation to data for a specific purpose.

There’s also the third class of ‘Ops’—whether that is machine learning Ops or anything related to putting machine learning and AI systems into production. This requires a very distinct skill set and involves different types of problems.

In summary, the advice would be to have a chat with someone, or several people, to get a flavour of the different types of data engineering. This will help you get a sense of which one you would enjoy more and then take it from there.

What do you enjoy most about consulting?

This is my first time in consulting,  but in many ways, I find it similar to my previous roles. 

What I enjoy the most is getting to the fundamental heart of the problem that people are trying to solve, which isn’t always what they may think they need to solve. The process of asking clarifying questions to get to the real heart of what is needed in a project is interesting. It’s a useful skill for anyone in business or any working professional to have, it’s not exclusive to consulting. 

The other thing I enjoy is presenting the pros and cons of different choices and helping people arrive at a decision themselves. Also, trying to be quite dispassionate about which option I favour, even though I still offer my perspective on my recommendations. It’s interesting to be a trusted advisor and offer a balanced view. “These are the pros or cons… There’s no free lunch here”.  You do need to make a decision and accept the trade-offs and the benefits of whichever option you go with.

Almost like empowering decision-makers to make decisions…

Yeah… because sometimes the other party needs to take responsibility for the outcome and they can only do that if they take an active part in making the decision.

What most excites you about the future of data engineering? 

I wonder whether we’ll ever get to the point, or how long it will be before we get to a point, where LLMs can truly replace the ability to peel back the layers of a data engineering problem. Particularly, in the field of analytics engineering, where it’s all about business interpretation and business logic, which is very nuanced and requires a deep understanding of the domain space and business priorities.

I keep thinking that LLMs are going to replace much of the work we do in data engineering, potentially making us more productive, much like in other fields. However, I don’t know whether we’ll ever reach a point where LLMs can peel back the layers of a problem as effectively as a human. But we may well be proven wrong.

Linking back to the advice you would give for a career in data engineering, do you think that LLMs could quickly change what you could look at as a future career?

I don’t know because even with what they call big data engineering, it still requires a lot of architectural systems-level thinking to understand all the nuances of a business problem. So again, it’s like seeing the forest for the trees. I don’t know if LLMs will be able to replace a human in doing that.

For younger people starting in data engineering, it might help with the process of getting a sense of which flavour of data engineering they would enjoy. It would be really nice to reach a point where LLMs can create different ways to experience the various flavours of data engineering. This would help someone decide, “This is the flavour I would enjoy and find rewarding,” without needing to find a company to be an apprentice or take an internship. It would be really cool if young people could find it easier to choose.

If you could have a superpower that you could only use at work, what would it be?

This one is easy! If I could get LLMs to read my mind and type for me so I don’t end up with repetitive strain injury from typing at a computer that’ll be amazing. I don’t need it to think for me, it just needs to read my mind and type. Copilot and other tools make it easier, but it’s not the same. 

These days, I have a treadmill so I can do my walking and get my movement for the day, but my hands are still on a keyboard in one position for eight hours a day.