Delving into LLMs' business impact, featuring a NordAxon case study on custom conversational agents and process optimization.

<h4>Introduction</h4><p>The landscape of Large Language Models (LLMs) has undergone a dramatic transformation, with recent advancements like the release of ChatGPT and the emergence of open models, opening up new possibilities for businesses across sectors. At NordAxon we're observing an increasing number of companies seeking our expertise to leverage these technologies for a range of applications. Among the emerging uses is the development of LLM-powered conversational agents, such as our own <a href="https://www.emely.se/">Emely</a>, but customized for individual companies. These specialized systems can also be crafted to intuitively interact with and process a company's unique data and internal context. Another area where LLMs are making a significant impact is in optimizing internal processes, streamlining tasks that range from data analysis to automated report generation.</p><p>In this article, we'll explore these innovations, using one of our projects as a practical example to illustrate the benefits and real-world applications of these advanced models in a corporate context. Our focus also extends beyond the case study, encompassing various aspects of LLMs and their transformative potential in the AI landscape.</p><h4>Closed vs. Open Language Models</h4><p>The LLM arena has mainly been dominated by OpenAI's GPT-3, 3.5, and 4 models. These closed models have captured the spotlight with their advanced capabilities and widespread applications. However, the LLM landscape extends beyond these well-known players, including powerful yet less publicized open alternatives such as Meta’s Llama and Mistral AI’s models. Each of these models presents unique advantages and challenges from a business perspective, and it's crucial to understand their distinctions. When evaluating LLMs for business integration, it's essential to recognize a fundamental distinction between two categories: closed and open models. This distinction shapes accessibility, control, and innovation potential in different ways.</p><h2>Closed Models</h2><p>Closed models, such as OpenAI’s GPT-models, offer a mix of benefits and considerations for businesses. On the positive side, they typically deliver superior performance compared to open counterparts and relieve companies from the complexities of hosting, as there's no need to manage infrastructure like GPUs. However, they are structured on a pay-per-usage basis and require sending data off-premises for processing. The pay-per-usage model, initially economical for smaller tasks, can escalate in cost significantly when processing large volumes of text data. Meanwhile, the need to send data off-premises for processing can present a separate concern, particularly for organizations with the highest privacy needs or those dealing with exceptionally sensitive data, as it may conflict with their strict confidentiality protocols.</p><h2>Open Models</h2><p>Open models present economical and flexible options for businesses, eliminating pay-per-usage fees and allowing for hosting on private servers, thus enhancing data privacy. While there are open models that exhibit impressive performance, surpassing some of OpenAI's earlier models in areas like complex question answering and detailed text summarization, they still fall short of matching the capabilities of OpenAI's most advanced and latest closed models. Quantized versions of closed models are also available, designed to run on consumer GPUs, and even on less powerful devices such as laptops and phones, albeit with a compromise in performance. This makes them more accessible to organizations with a range of resource availabilities. While they require more technical oversight and infrastructure management, their cost-effectiveness and adaptability make them a viable choice for many, especially those prioritizing data control and customization.</p><h2>A Leaderboard Snapshot of Models</h2><p>Performance benchmarks for instruct-models, as highlighted on the <a href="https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard">HuggingFace LMSys Chatbot Arena Leaderboard</a>, offer a snapshot of the competitive LLM landscape. OpenAI's GPT-4 models currently dominate the leaderboard. Notably, the recently released Mixtral-8x7B, an open model from Mistral AI, achieves a strong 8th position. This stands as a significant testament to the capabilities of open models in the field, particularly notable in a field dominated by proprietary systems. Although not visible in the image, the Llama-2-70b-chat model is currently ranked 18th, illustrating that more parameters do not necessarily guarantee superior performance.</p><p><img alt="A screenshot of a computer

Description automatically generated" /></p><p>We now shift focus to an emerging technology that's particularly interesting for businesses: Retrieval-Augmented Generation, or RAG systems. These systems introduce an innovative way to enhance LLM’s capabilities by integrating company-specific data retrieval with generative techniques.</p><h4>RAG Systems: Unleashing New AI Capabilities</h4><p>Retrieval-Augmented Generation (RAG) systems merge internal data retrieval with the generative capabilities of LLMs, significantly enhancing the customization potential of LLM-based applications. In essence, RAG allows for ChatGPT-like functionality, but uniquely tailored to a company's specific data, effectively creating what might be called a 'company-ChatGPT'.</p><p>In summary, RAG operates by archiving text-based materials, like corporate documents or web content, in a specialized database. Typically, this content is converted into embeddings and stored in a vector database, where text segments with similar meanings are positioned in close proximity to each other in a mathematical space. When a query, or user input, is received, the system performs a semantic search to pinpoint text segments that are most relevant to the query. These segments are then fed to the LLM along with the query, enabling the model to generate answers that are informed by, and directly utilize, the organization's own content.</p><p>In the following example, an LLM (a quantized Mistral-7B running on my laptop) is used out-of-the-box and prompted about NordAxon's services. As can be seen in the image, the model’s response indicates a lack of knowledge about the company. At this point, it is important to recognize that that the LLM’s knowledge is limited to its initial training data, which includes a significant portion of the internet, but notably lacks specific information about NordAxon.</p><p><img alt="A screenshot of a computer

Description automatically generated" /></p><p><span class="text-small"><i>Image 1: When prompted with a query requiring detailed knowledge about NordAxon, the model's response reveals its limitations in providing specific information.</i></span></p><p>Contrastingly, in the second scenario, the model has been provided with access to text from NordAxon's <a href="https://www.nordaxon.com/services">webpage</a> that has been integrated into a RAG-chain. When prompted with the same question, the LLM now incorporates this webpage information into its response, and delivers an accurate answer derived from the content of the webpage.</p><p><img alt="A screenshot of a computer

Description automatically generated" /></p><p><span class="text-small"><i>Image 2: Demonstrating the effectiveness of RAG, the model accurately responds to the query when granted access to specific information about NordAxon.</i></span></p><p>While our example illustrates the creation of a 'company-ChatGPT', it is important to keep in mind that RAG systems offer a broader range of capabilities beyond just this. The upcoming section will feature a project where NordAxon was hired by a large organization, highlighting our utilization of RAG technology in a real-world context.</p><h4>RAG Technology in Action: A Three-Case Project with a Swedish Company</h4><p>In a recent 12-week project at a prominent Swedish company, NordAxon undertook three proof-of-concept cases, each implementing RAG technology. Two consultants from NordAxon took on the project, beginning with the company's specific needs, and developed a solution using RAG technology. This encompassed everything from creating a user-friendly interface to architectural design and implementation, while also maintaining an awareness of essential privacy aspects. To safeguard data privacy, the open model Mistral-7B-v0.1 was employed, ensuring that sensitive information remained on-premises.</p><p>The first case focused on developing a system to enhance information retrieval across the organization's various platforms. Given the challenge of dealing with vast amounts of extensive internal documents, the consultants closely collaborated with stakeholders who frequently interact with complex process documents. The result was a user-friendly system designed to answer company-specific questions using information from internal documents. It was capable of tasks such as outlining the next step in a subprocess or identifying the right contact for additional information related to various tasks within the processes, among other functions. </p><p>The second case, similar to the first, was also a RAG-based chatting application. However, it sourced its information from emails instead of documents. This project aimed to allow employees involved in purchasing to easily ask questions through their email inboxes, especially regarding specific components. In working closely with these stakeholders, the project naturally evolved to include a feature that displays relevant documents related to the orders on the same dashboard, offering a more comprehensive and convenient overview. </p><p>In the third and final case, although it also employed a RAG-based system, it departed from the earlier scenarios by not being a conventional chat application where users submit questions in free text. Instead, its focus was on utilizing primarily external data, such as text extracted from websites, to generate detailed reports about manufacturers with a simple click of a button. The LLM composed the content of each report, addressing and providing information related to each manufacturer in a systematic manner. Aimed at streamlining the tasks for those in global sourcing, this project stood out for its effectiveness and potential. Motivated by its success and positive reception, NordAxon is now set to advance this case further. We aim to refine and embed the system within the company's existing operations, targeting improved efficiency in sourcing.</p><p>While the decision was made not to continue the development of the two other cases, they clearly demonstrated the capabilities of LLMs in the workspace and the potential impact of fully developed tools like these. These projects further illustrated that it is feasible to explore and customize these cutting-edge technologies within relatively short timeframes, even with a small team.</p><h4>Wrap-Up</h4><p>In this exploration of Large Language Models (LLMs), we've seen their significant impact on various aspects of business operations. The flexibility of LLMs, particularly in creating tailored conversational agents and enhancing process efficiency, is a testament to their value in today's business environment. The practical applications of technologies like RAG systems, as demonstrated in real-world scenarios, highlight their potential for customization and efficiency in handling business-specific data. As the field of LLMs continues to grow and evolve, it presents compelling opportunities for businesses to innovate and improve their operations. At NordAxon, we remain committed to exploring these advancements, helping organizations leverage the full potential of LLMs to meet their unique challenges and goals.</p>