Deep Dive: Orchestrating AI-Infused Research

Olivia Carter

Good afternoon, Mark. Today, we’re diving into what it really takes to build an AI-infused research engine at a major research university. And, Mark, I feel like this is one of those topics where the stakes are just… massive, right?

Mark Putnam

Absolutely, Olivia. For a research university, AI isn’t just a buzzword to sprinkle into strategic plans. It’s a fundamental shift in how research is done. If you want to tackle deep learning, genomic sequencing, infectious disease modeling—any of these frontiers—you need serious computing muscle. And not just as a one-off capital project, but as a continual, planned investment in three areas: the hardware itself, the facilities to house and power it, and, maybe most importantly, the people who can manage, optimize, and support it day in and day out.

Olivia Carter

Right. And I think sometimes people outside the university world imagine that all this AI work just “happens in the cloud,” or that you can just buy a couple of powerful servers and be done. The reality is very different. These models are hungry in a way that’s hard to overstate. They demand GPU acceleration for training, high-throughput storage for massive datasets, robust low-latency networking to move data efficiently, and cooling systems that keep all of it from overheating. Without that backbone, your AI strategy isn’t a research capability—it’s just an idea on a PowerPoint slide.

Mark Putnam

That’s exactly right. I still remember when the first GPU cluster arrived at my university. There was real skepticism from some faculty—concerns about whether the investment was justified, whether anyone would really use it. And then the early adopters got their hands on it. Suddenly, simulations that took days finished in hours, datasets that couldn’t fit on a personal workstation were being analyzed, and entirely new lines of inquiry opened up. Within a semester, those same skeptics were submitting job requests and lobbying for more GPUs. That’s the cycle—skepticism, trial, and then the “I can’t work without this” moment.

Olivia Carter

And that cycle is speeding up with AI. The demands keep escalating, but the solution isn’t just buying the biggest, fastest machine you can find. It’s about building a balanced ecosystem—central high-performance clusters for the largest jobs, advanced desktop workstations for day-to-day heavy lifting, and cloud resources for those specialized or short-term needs.

Mark Putnam

Exactly. If you don’t have that mix, you run into real problems. Researchers are either stuck waiting in long queues for compute time or, worse, scaling back their ambitions because they can’t get the horsepower they need when they need it. And that’s where the people part comes in. You need leaders who can understand the research priorities, technical staff who can keep the systems tuned and efficient, and a governance process that ensures resources are allocated where they’ll have the most impact.

Olivia Carter

Which is why I think the universities that will really thrive in this AI era are the ones that don’t just invest in technology, but also embed faculty leadership into the process. People who live in the research trenches and can see where the next bottleneck is going to be—before it actually slows down the work.

Mark Putnam

That's right, Olivia.

Olivia Carter

Let’s talk about leadership in research computing, because this is where I think a lot of universities either get it right or stumble badly. Mark, you’ve been through enough technology transitions to know—this isn’t just about machines and software. There’s a real shift happening in how leadership is structured.

Mark Putnam

There is. In high-performance and AI-supported research, the biggest breakthroughs happen when the leadership team includes faculty voices at the table, not just IT and administrative leaders. Too often, technology decisions are made in a kind of vacuum—IT builds for what they think researchers need, based on usage data or broad assumptions, but without truly understanding the nuances of the work. Faculty researchers live in those nuances every day. They know the bottlenecks, the quirks in workflows, and the trade-offs that make the difference between getting results in a week versus a month.

Olivia Carter

Right. I’ve seen so many universities roll out new systems or infrastructure without deep faculty involvement. You get the ribbon-cutting moment, the announcement about new capabilities, but then you hear, “This doesn’t actually solve my problem,” or worse, you see faculty quietly ignoring it and spinning up their own solutions on the side. That’s a sign the investment missed the mark, even if the technology itself is impressive.

Mark Putnam

Exactly. And in an AI-driven research space, that gap gets even more costly. If you’re trying to run large-scale simulations, train a model on terabytes of genomic data, or do multi-institutional climate modeling, you don’t just need raw compute power. You need a setup that matches your project’s data flows, integrates with your preferred analysis tools, and fits the way your team collaborates. That only happens when the people making infrastructure decisions are in direct conversation with the people pushing the research forward.

Olivia Carter

So it’s not just about consultation, it’s about embedding faculty leadership in the process. Someone who can speak both “research” and “infrastructure” fluently. They’re not just a guest at the table—they’re part of the team making decisions about priorities, resource allocation, and system design.

Mark Putnam

Exactly. That’s the difference between a system that looks good on paper and one that actually moves the needle. When faculty leadership is present, you don’t just get technically sound systems, you get strategically relevant systems—built around the realities of research rather than abstract specifications. And in this AI era, where demands on computing resources are accelerating, that alignment is what keeps a university competitive and innovative.

Olivia Carter

Alright, let’s get into the nuts and bolts. The smartest universities aren’t just saying, “Buy a big cluster and call it a day.” The forward-looking ones are taking a more nuanced, three-tiered approach: desktop powerhouses, cloud elasticity, and the central HPC. Mark, can you break down how that works in practice?

Mark Putnam

Absolutely. The old model was simple: if you had serious computational work, it went straight to the central high-performance computing cluster. That made sense when desktop machines couldn’t come close to the horsepower of the cluster. But that’s no longer the world we live in. Today, a lot of meaningful breakthroughs are happening right on advanced desktop workstations sitting in labs and offices.

Mark Putnam

You’ve got researchers in physics, infectious diseases, computational chemistry—running advanced AI models on machines that, a decade ago, would have been science fiction. An 80-core, GPU-enabled workstation with half a terabyte of RAM can now be bought for around ten thousand dollars. For many research groups, that’s a one-time investment that pays off for years, letting them process massive datasets, train sophisticated models, or run highly complex simulations without ever entering the cluster queue. And that’s a big deal—it keeps the central HPC free for the truly massive jobs while giving individual researchers independence and speed.

Olivia Carter

Yeah, I actually interviewed a physics PhD student last year who was working on particle simulations. She had access to one of these high-powered desktops, and she said it completely changed her workflow. No more waiting in line for compute time. No more trimming datasets to fit arbitrary limits. She could tweak her model at lunchtime, re-run it in the afternoon, and have results to analyze by evening. That kind of rapid iteration is a game-changer, especially in AI-heavy research where refinement is everything.

Mark Putnam

And then there’s the cloud. For short-term spikes in demand, or when you need highly specialized computational environments, cloud resources from providers like Microsoft, Amazon, or Oracle are invaluable. You don’t want to overbuild your on-premises capacity for those rare moments when demand peaks. The cloud gives you elasticity—you can spin up resources quickly, run the job, and shut it down without the ongoing cost of hardware that sits idle most of the year. It’s also a lifeline for interdisciplinary projects with external partners who already operate in the cloud, making data sharing and workflow integration smoother.

Olivia Carter

And then you have the central high-performance computing cluster—the real heavy lifter. This is still the backbone of any serious research computing strategy. It’s where the largest, most resource-intensive projects live: global climate models, whole-genome alignments, exoplanet data processing, large-scale AI training runs that might consume terabytes of GPU memory and run for weeks. It’s a shared investment, often across disciplines, and that shared nature makes it both cost-effective and strategically important.

Mark Putnam

Exactly. And the point is, it’s not about choosing one of these approaches and declaring it “the solution.” It’s about having all three, and matching the right tool to the right job. That means researchers have options—so they’re not stuck waiting in a queue or compromising their science because the infrastructure can’t handle the workload. When the right resource is available at the right time, research moves faster, collaborations are smoother, and the institution as a whole gets more value out of its computing investments.

Olivia Carter

Let’s talk about bottlenecks. Because, honestly, nothing kills momentum like waiting for compute resources. How does this balanced portfolio actually help researchers avoid those stalls?

Mark Putnam

It’s all about flexibility. If you’ve got a heavy workload that needs the cluster, it’s there. But if the queue is full, or your job is better suited to a desktop or the cloud, you have options. Cloud bursting—using cloud resources when internal capacity is maxed out—means you don’t have to put your project on hold. And for a lot of day-to-day work, those desktop powerhouses are more than enough.

Olivia Carter

But there are trade-offs, right? I mean, local hardware is great for continuous workloads, but it’s a big upfront investment. Cloud is flexible, but costs can spiral if you’re not careful. And the central HPC is efficient, but only if you’re running jobs that really need that scale. It’s a balancing act.

Mark Putnam

That’s right. The key is matching the resource to the need. Not every project needs the biggest machine. Sometimes, the fastest path is the simplest one. And having faculty leadership in the room means those decisions are grounded in real research needs, not just what’s convenient for IT.

Olivia Carter

It’s like what we talked about in our episode on “sweating the details together.” You need that shared accountability, that willingness to get into the weeds and solve problems as a team. Otherwise, you end up with bottlenecks and frustration.

Mark Putnam

That's right, Olivia.

Olivia Carter

So, looking ahead, the demands are only going to get bigger. Datasets are exploding, AI models are getting more complex, and collaborations are stretching across institutions and even countries. How do research Universities stay ahead?

Mark Putnam

It comes down to adaptive leadership and continued investment. The technology will keep evolving—what’s cutting-edge today will be standard tomorrow. But the institutions that thrive are the ones that invest intentionally, that keep faculty at the table, and that aren’t afraid to rethink their approach as needs change. I’ve seen this play out before—every wave of technological change brings new challenges, but also new opportunities. The universities that stay competitive are the ones that lead with purpose, not just with hardware.

Olivia Carter

And it’s not just about the machines, it’s about the people and the process. You need leaders who can see around corners, who can build trust, and who can keep the institution human, even as the tech gets more powerful. That’s something we keep coming back to on this show—whether it’s ERP, negotiation, or AI, it’s always about the culture and the leadership.

Mark Putnam

Exactly. And as the demands grow, so does the need for thoughtful, intentional leadership. balancing investment, embedding faculty, and staying agile—is a model others can learn from. The work is never finished, but that’s what makes it meaningful.

Olivia Carter

Alright, that’s a wrap for today. Thanks for joining us as we unpacked what it really means to orchestrate AI at a research university. Mark, always a pleasure.

Mark Putnam

Likewise, Olivia. Looking forward to our next conversation.

Olivia Carter

And thanks to all of you for listening. We’ll be back soon with more on tech, leadership, and the future of higher ed. Take care, everyone.