Category: Uncategorized

  • The attention mechanism

    —

    by

    Aisha Katusi
    in

    The attention mechanism is one of the key pieces of technology inside LLMs and many other prominent systems in robotics, computer vision etc. This article attempts to explain how it processes data. It was first introduced in a 2017 landmark paper called Attention is all you need Imagine you are given a sentence : “The…

  • The mathematics and concepts in LLM’s

    —

    by

    Aisha Katusi
    in

    Today, state-of-the-art LLMs like Gemini, Claude, DeepSeek, GPT etc exhibit extraordinary capabilities resembling aspects of human cognition. They are able to write poems, produce powerful codebases, summarize text etc. and interact with users in a multimodal way. Their performance is largely correlated to their architecture. An LLM architecture is a parametrized function that applies a…