Machine Learning

Intrinsic Dimension Part 2: Measuring the True Complexity of a Model via Random Subspace Training

In Part I of this series, we delved into the concept of Intrinsic Dimension (ID) and its implications on fine-tuning. To recap, the intrinsic dimension of an objective function measures the minimal number of parameters needed to achieve satisfactory performance on a given task (Li et al.,¹). For example, in their seminal work, the authors demonstrated that a fully connected neural network with a total of parameters (architecture: 784–200–200–10) achieved a 90% performance threshold on the MNIST dataset with an...

Read

Intrinsic Dimension Part 1: How Learning in Large Models Is Driven by a Few Parameters and Its Impact on Fine-Tuning

Learned over-parameterized models inherently exist within a low intrinsic dimension {Li et al.¹ and Aghajanyan et al.³}. To understand this concept better, let’s delve into the following questions:What is a model’s dimension?What is an Objective function and its landscape?What is ID and how can it be visualized through graphs of convex and non-convex landscapes?How do you identify the ID of a network?If the ID of an objective function is so low, why do we have such large networks in the...

Read

Data Drives Innovation and Growth at Australian Fintech

A tailored strategy combining software engineering, data engineering, and ML operations enabled data-led business decisioning for a fintech enterprise. Our client, a leading Australian non-banking fintech providing digital instalments and lending services to customers across Australia, New Zealand, and Singapore, was facing a unique challenge. Having acquired and integrated several small businesses over the years, the company was grappling with a mix of disparate and legacy systems and processes, making it extremely arduous to build critical components around Machine Learning...

Read

Improving LegalTech Text Search

Improving the performance and response time of a legal-tech software enables it to handle millions of documents in milliseconds. A Norwegian legal-tech start-up developed an e-discovery software designed specifically for the local legal market, utilising a proprietary AI algorithm to streamline handling of extensive data in the context of a legal case, such as documents, emails, and attachments. This software allows legal professionals to conduct text searches for specific words or phrases, identify patterns, and understand relationships amongst key stakeholders...

Read

Simplifying and Humanising LegalTech

A Norwegian legal-tech start-up developed a niche product with proprietary and a unique AI algorithm capable of identifying patterns and relationships from a sea of data. While the product’s core capability worked well, its adoption was limited to a few users. In early 2024, the company collaborated with Sahaj Software to integrate AI algorithms into its case-management process for better adoption in areas such as disputes, litigations, investigations, and transactions. Typically, the legal space in Norway did not face any...

Read

Leveraging Large Language Models in Real-time Ad Bidding and Placement: Possible Perspectives for Enhanced User Targeting

Written by Oshin Anand and Kulbhushan Singhal In the ever-evolving digital advertising landscape, Real-Time Bidding (RTB) is a cornerstone of modern ad placement strategies. The integration of Large Language Models (LLMs) such as GPT-4, Bing Chat, and Gemini promises to revolutionize this process. This blog post explores the potential of LLMs to enhance the RTB value chain and both the opportunities and challenges that come with this technological advancement. The integration of Large Language Models (LLMs) into the RTB process...

Read

Exploring LoRA - Part 2: Analyzing LoRA through its Implementation on an MLP

Source: ChatGPT+ Part 1 delves into the concept and necessity of fine-tuning pre-trained large models for specialized tasks. It introduces the conventional method of fine-tuning, where only the top layers of the model are adjusted, and highlights its limitations, particularly in terms of computational and storage demands. To address these challenges, the article shifts focus to Parameter-Efficient Fine-Tuning (PEFT) methods, specifically the use of adapter modules, as proposed by Houlsby and colleagues. These adapters are small, inserted layers that allow for...

Read

Exploring LoRA — Part 1: The Idea Behind Parameter Efficient Fine-Tuning and LoRA

Source: ChatGPT+ Pre-trained large language models undergo extensive training on vast data from the internet, resulting in exceptional performance across a broad spectrum of tasks. Nonetheless, in most real-world scenarios, there arises a necessity for the model to possess expertise in a particular, specialized domain. Numerous applications in the fields of natural language processing and computer vision rely on the adaptation of a single large-scale, pre-trained language model for multiple downstream applications. This adaptation process is typically achieved through a...

Read

Soaring to New Heights: AI, Machine Learning, and Data Strategy in the Airline Industry

Written by Ravindrababu T and mckimmer In the fiercely competitive airline industry, the race to deliver a frictionless passenger experience is on. Air travel, once merely about getting from A to B, is now a journey laden with potential touch-points where airlines can either win loyalty or lose business. The key to success lies in leveraging machine learning, artificial intelligence (AI), and a robust data strategy to not only streamline this journey but also to unlock avenues for increased revenue. The application of...

Read

Data-Driven Customer Insights: A New Altitude for Airline Revenue Management and Retail Pricing

The aviation industry, with its dynamic pricing and complex service delivery, presents a fertile ground for AI driven data insights to revolutionize customer experience and revenue management. Understanding customer behavior through meticulous data mining is not just a competitive advantage; it’s a strategic imperative that is increasingly separating the successful airlines from those struggling to achieve profitability. The crux of the matter lies in mining and interpreting the vast array of customer data at each touchpoint — from booking a...

Read