Building Teams That Work Well Together: In Conversation with Andrew Spiegelman, Service NSW

Why do data, engineering, and AI teams often struggle to collaborate, even when they share the same organisational goals?

Data/AI teams’ greatest struggle is that organisations are unsure how to use them. C-suites want advanced data/AI but are often unsure what data/AI can deliver. C-suites are even less certain about what investments (people, effort, resources, etc.) are required to realise any goals. Thus, data/AI leaders at many organisations have vague mandates and insufficient resources to meet said mandates. Alternatively, engineering, is a long-established field with better defined goals. So, data/AI teams try to find their identities and prove their value, while engineering teams know better what’s expected of them. This difference often makes data/AI incentives/goals misalign with engineering incentives/goals – harming collaboration. But even further, data/AI’s need to find identity and prove value means they need significant entrepreneurial and bureaucratic talents – but those talents are often secondary to tech talent when hiring data/AI leaders.

What team structures or operating models have you seen work best for cross-functional delivery in AI and data programs?

One great model is to have data/AI people embedded across various organisation functions, thus connecting data/AI into all critical functions instead of isolating it. However, this is rarely practical because most functions are unfamiliar with data/AI and so cannot incorporate it (as opposed to the ease of incorporating an engineer or product manager). This model requires a data literate workforce.

The next best thing is to have centralised data/AI teams who are excellent at dealing with many different business areas. That means tenacity, proactiveness, entrepreneurial spirit, and above all, comfort and confidence in spending significant time with customer teams. Many data/AI people are hired purely on tech talent but data/AI teams are support functions: they help other areas of the business (unless the org makes flagship customer-facing AI products, which is still relatively rare). Because data/AI teams prove value by helping other areas, they shouldn’t just sit back and take tickets; they should intimately understand their internal customers’ contexts, goals, and needs. Such a data/AI team can be very successful.

How can leaders design clearer decision rights to reduce friction, misalignment, and delays across teams?

This topic is vast, with many volumes written. I’ll just say that consistently making one person accountable for a goal tends to either achieve or at least reveal that you need a different accountable person.

What cultural or behavioural shifts are essential to creating high-trust, high-performance technical teams?

As I mentioned earlier, we must stop focusing so much on tech talent and start focusing more on soft skills. I’ve been on both sides of dozens of data/AI job interviews and soft skills almost never arise as requirements (though they do when I’m the hiring manager). This feels counterintuitive because tech talent is still important (I give technical exercises to anyone I’m considering hiring). But tech talent should no longer be the only thing that matters. Especially now that AI can draft code, we need more people who can get along with others, who can clearly communicate and listen to others, and who understand what’s important to the business. Many organisations casually think they want this but few focus their hiring practices on achieving it.

One great way to start is to stop listing 10+ specific technologies as “requirements” in data/AI job ads. Doing so discourages women from applying; men will often apply even if they lack many “required” technologies – only to learn that those technologies were, in fact, not required. In my experience, SQL and Python get most people very far – far enough to learn most other technologies on the job.

When teams are siloed or misaligned, where should leaders start to rebuild collaboration and shared accountability?

Start at the top: leaders of misaligned teams must ensure that their own leaders are both aligned on strategy and aware of problems realising that alignment. And then the next level down of leaders must align on the relevant sub-strategies, and so on. If you’re not at the top, the best you can do is make the appeal to your own leaders. But leadership comes from the top and if the people at the top cannot align, those below will struggle.