The Current Architecture of a Basic LLM Application
We’re starting to design a new type of software. What does it look like?
A few years ago, artificial intelligence was a specialized topic for a small set of tech companies with deep expertise and even deeper pockets. Fast forward to today, and it’s hard to find a tech company that isn’t rushing to put AI into its processes and products — even if they don’t know exactly what that looks like.
In this article, we’ll look at this new frontier of development. How do today’s AI-assisted applications work? What architecture do they use, and why might they succeed where other approaches don’t?
“Off the shelf” AI
As impressive as AI may be, it certainly isn’t new. For years, leading tech companies have used machine learning in massive applications — for example, optimizing content encoding at Netflix, calculating food recommendations at Uber, or planning deliveries at Amazon. The difference is that those AI features were highly specialized. They weren’t in the toolbox of your average software vendor.