The Current Architecture of a Basic LLM Application

We’re starting to design a new type of software. What does it look like?

Matthew MacDonald
Young Coder
Published in
9 min readMar 26, 2024

--

A few years ago, artificial intelligence was a specialized topic for a small set of tech companies with deep expertise and even deeper pockets. Fast forward to today, and it’s hard to find a tech company that isn’t rushing to put AI into its processes and products — even if they don’t know exactly what that looks like.

In this article, we’ll look at this new frontier of development. How do today’s AI-assisted applications work? What architecture do they use, and why might they succeed where other approaches don’t?

“Off the shelf” AI

As impressive as AI may be, it certainly isn’t new. For years, leading tech companies have used machine learning in massive applications — for example, optimizing content encoding at Netflix, calculating food recommendations at Uber, or planning deliveries at Amazon. The difference is that those AI features were highly specialized. They weren’t in the toolbox of your average software vendor.

The machine learning pipeline for Uber Eats recommendations

--

--

Published in Young Coder

Insights into the .NET stack. Thought pieces about the craft of software development. Real advice for teaching kids to code. And a shot of humor.

Written by Matthew MacDonald

Teacher, coder, long-ago Microsoft MVP. Author of heavy books. Join Young Coder for a creative take on science and technology. Queries: matthew@prosetech.com

Responses (4)

What are your thoughts?