Did you know that “T” in Chat-GPT stands for Transformer, which is Google’s revolutionary architecture that brings the concept of “self-attention” to AI? And that Google pioneered silicon for deep learning workloads with TPUs? After combing through dozens of technical papers and posts, I summarized my learnings in one visual below.
All the recent AI talk brought back the memory of the fall semester of 2003, when I signed up for a Neural Networks course 👩🎓. After several classes of advanced algebra and calculus, I was excited to see their practical applications in natural language processing and speech recognition use cases. Little did I know that in 2023, computers would not only be able to almost perfectly understand human speech, they would also gain a voice of their own thanks to decision making capability similar to humans.
I majored in Telecommunications Engineering and always found the Open Systems Interconnection Reference Model, more commonly known as the OSI model, extremely useful in visually depicting all the key layers of the networking tech stack. So I thought to myself: what if I build a similar reference model for AI? At the end, at the core of AI lies a neural network. And I’ve successfully demystified a variety of tech stacks using the good ol’ OSI model before, from PaaS to SDN/NFV. Let me know what you think!
Thank you for inspiration to Philip Moyer and to Priyanka Vergadia and Neama Dadkhahnikoo for technical review.
#artificialintelligence#deeplearning#machinelearning#selfattention#naturallanguageprocessing#googlecloud
And here’s an animated version of “The 6 Layers of Generative AI Technology Stack”. To me, it’s like watching a delicious multi-layer cake being assembled layer by layer, except instead of vanilla cake, lemon custard and cream-cheese frosting, our recipe calls for infrastructure, modeling and application layers as key ingredients. Who knew that a stack of AI layers could be so captivating?
Check out my LinkedIn post The 6 Layers of Generative AI Technology Stack
Tech Surprises 