Introduction to LangChain for Data Engineering & Data Applications
Introduction to LangChain
Large language models (LLMs) like OpenAI GPT, Google Bert, and Meta LLaMA are revolutionizing every industry through their power to generate almost any text you can imagine, from marketing copy to data science code to poetry.
While ChatGPT has taken the lion's share of attention through its intuitive chat interface, there are many more opportunities for making use of LLMs by incorporating them into other software.
As an illustration, the DataCamp Workspace incorporates an AI Assistant feature that empowers you to enhance both code and text within your analysis.
Additionally, DataCamp's interactive courses offer a valuable "explain my error" capability, enabling you to pinpoint and comprehend any mistakes you may have made.
These impressive functionalities are made possible by the GPT-powered engine, seamlessly accessed through the OpenAI API.
LangChain is a groundbreaking framework built upon multiple APIs for Large Language Models (LLMs).
Its primary objective is to enhance the productivity of software developers and data engineers by facilitating the seamless integration of LLM-based AI into their applications and data pipelines.
By leveraging LangChain, developers and engineers can significantly boost their efficiency and effectiveness in harnessing the power of LLMs for their projects..
This tutorial details the problems that LangChain solves and its main use cases, so you can understand why and where to use it.
Before reading, it is helpful to understand the basic idea of an LLM. The How NLP is Changing the Future of Data Science tutorial is a good place to refresh your knowledge.
What problems does LangChain solve?
There are essentially two workflows for interacting with LLMs.
- "Chatting" involves writing a prompt, sending it to the AI, and getting a text response back.
- "Embedding" involves writing a prompt, sending it to the AI, and getting a numeric array response back.
Both the chatting and embedding workflows have some problems that LangChain tries to address.
Prompts Are Full of BoilerPlate Text
Both chatting and embedding pose a challenge, as crafting an effective prompt entails more than simply outlining the task at hand.
It requires a thoughtful description of the AI's personality and writing style, along with instructions to ensure factual accuracy.
In other words, you may need to compose a concise and straightforward prompt that not only conveys the desired task but also encompasses the desired AI attributes and guidelines for maintaining factual precision.
Write an outline for a 500-word blog post targeted at teenagers about the health benefits of doing yoga.
However, to get a good response, you need to write something more like:
You are an expert sports scientist. Your writing style is casual but terse. Write an outline for a 500-word blog post targeted at teenagers about the health benefits of doing yoga. Only include factually correct information. Explain your reasoning.
To minimize the repetitive nature of boilerplate copy in prompts, LangChain offers a compelling solution through prompt templates.
By leveraging these templates, you can conveniently merge the pertinent prompt input, such as writing a blog post on yoga, with the boilerplate elements encompassing the desired writing style and the request for factual accuracy.
This allows you to write the boilerplate text just once and effortlessly incorporate it into any prompts as needed, streamlining the prompt creation process and ensuring consistency across different prompts.
Responses Are Unstructured
In the context of chatting workflows, the output generated by the model typically consists of plain text.
Nevertheless, when integrating AI within software applications, there is often a need for structured output that can be easily programmed against.
For instance, if the objective is to generate a dataset, it becomes crucial to receive the response in a predetermined format, such as CSV or JSON.
This enables seamless processing and utilization of the AI-generated output in a structured manner, aligning it with the specific requirements of the software environment.
Assuming that you can write a prompt that will get the AI to consistently provide a response in a suitable format, you need a way to handle that output. LangChain provides output parser tools for just this purpose.
It's Hard to Switch Between LLMs
While GPT is wildly successful, there are a lot of other LLMs available.
By programming directly against one company's API, you are locking your software into that ecosystem.
It's perfectly plausible that after building your AI features on GPT, you realize stronger multilingual capabilities are necessary for your product.
You may then want to switch to Polyglot or switch from using an AI to including the AI in your product and require a more compact model like Stability AI's StableLM.
LangChain provides an LLM class, and this abstraction makes it much easier to swap one model for another or even make use of multiple models within your software.
LLMs Have Short Memories
The response generated by a Large Language Model (LLM) relies on the context of the preceding conversation, including user prompts and the model's previous responses.
However, LLMs have inherent limitations in their memory capacity. Even the state-of-the-art GPT-4, for instance, defaults to a memory limit of 8,000 tokens, equivalent to approximately 6,000 words.
This implies that the LLM's ability to retain and reference past information is constrained by this finite memory boundary.
In a chatting workflow, if the conversation continues beyond the memory limits, the responses from the AI can become inconsistent (since it has no recollection of the start of the conversation).
Chatbots are an example where this can be a problem. Ideally, you want the chatbot to recall the entire conversation with the customer so as not to provide contradictory information.
LangChain solves the problem of memory by providing chat message history tools. These allow you to feed previous messages back to the LLM to remind it of what it has been talking about.
It's Hard to Integrate LLM Usage Into Pipelines
When used in data pipelines or in software applications, the AI is often only part of a large piece of functionality.
For example, you may wish to retrieve some data from a database, pass it to the LLM, then process the response and feed it into another system.
LangChain offers a robust set of tools specifically designed for pipeline-type workflows, leveraging the concepts of chains and agents.
Chains serve as straightforward objects that effectively connect multiple components, enabling the creation of linear pipelines.
On the other hand, agents offer enhanced capabilities by incorporating business logic, granting you the flexibility to determine how these components interact with each other.
This includes the ability to employ conditional logic based on the output from a Large Language Model (LLM), empowering you to make informed decisions about the subsequent steps in the pipeline.
With LangChain, you have the freedom to customize and orchestrate your workflow according to your specific requirements and desired outcomes.
Passing Data to the LLM is Tricky
When working with text-based Large Language Models (LLMs), it is often challenging to determine the best approach for passing data to the model.
This problem consists of two main components.
Firstly, it is necessary to store the data in a format that allows for a controlled selection of specific portions of the dataset to be sent to the LLM.
In the case of structured datasets like DataFrames or SQL tables, the typical approach is to send data row by row, enabling fine-grained control.
Secondly, deciding how to incorporate the data into the prompt presents another consideration.
While a straightforward approach is to include the entire dataset directly within the prompt (known as "prompt stuffing"), there are more advanced options available for handling data inclusion in prompts, catering to specific requirements and scenarios.
Thus, LangChain provides solutions for both aspects, empowering users with the flexibility to effectively manage data input and choose the most suitable method for including the data within prompts, be it the straightforward approach or more sophisticated alternatives.
LangChain solves the first part of this with indexes.
These provide functionality for importing data from databases, JSON files, pandas DataFrames, CSV files, and other formats and storing them in a format suitable for serving them row-wise into an LLM.
LangChain provides index-related chains to solve the second part of the problem and has classes for four techniques for passing data to the LLM.
Prompt stuffing inserts the whole dataset into the prompt. It's very simple but only works when you have a small dataset.
The Map-Reduce technique divides the data into smaller chunks, invoking the Large Language Model (LLM) with an initial prompt for each data chunk ("map" phase).
Subsequently, the LLM is called again, this time with a modified prompt that includes the responses generated from the initial prompts.
This approach proves effective in scenarios where a "group by" command would typically be employed.
On the other hand, the Refine approach follows an iterative Bayesian-style methodology. Initially, a prompt is executed on the first data chunk, and for each subsequent chunk, a refined prompt is used to request the LLM to enhance the results based on the new dataset.
This approach is particularly suitable when seeking to converge the LLM's responses towards a specific output.
In the Map-Rerank strategy, a variation of Map-Reduce, the initial phase remains the same: data segmentation into chunks followed by prompt invocation on each chunk.
However, the distinguishing factor lies in requesting the LLM to provide a confidence score for its response, enabling the ranking of outputs.
This approach is highly advantageous for recommendation-based tasks where identifying a single "best" answer is crucial.
Which Programming Languages are Supported by LangChain?
LangChain can be used from JavaScript via the langchain node package. This is suitable for embedding AI into web applications.
It can also be used from Python via the langchain (PyPI, conda) package. This is suitable for including AI into data pipelines or Python-based software.
What are the Main Use Cases of LangChain?
LangChain can be used wherever you might want to use an LLM. Here, we'll cover several examples related to data, explaining which features of LangChain are relevant.
Querying Datasets with Natural Language
The application of Large Language Models (LLMs) for data analysis has brought about transformative use cases, particularly in the realm of writing SQL queries or equivalent Python/R code using natural language.
This breakthrough enables individuals without coding skills to engage in exploratory data analysis effectively.
There exist several variations in the workflow to achieve this goal. For small datasets, direct result generation by the LLM is possible.
This involves utilizing LangChain's document loaders to format the data appropriately, passing the data to the LLM using index-related chains, and parsing the response with the aid of an output parser.
More commonly, the approach involves providing the LLM with details about the data structure, such as table names, column names, types, and any specific information like missing values.
Subsequently, the LLM is requested to generate SQL/Python/R code, which can then be executed. This flow is simpler, as it eliminates the need to pass data. Nonetheless, LangChain remains valuable in modularizing the steps involved.
Another variation is to incorporate a second call to the LLM, enabling it to interpret the results.
This workflow, involving multiple interactions with the LLM, is where LangChain significantly contributes. In such cases, the chat message history tools provided by LangChain ensure consistent interpretation of the results based on the previously provided data structure.
Overall, LangChain facilitates diverse workflows within LLM-based data analysis, making it accessible, modular, and ensuring coherence throughout the analytical process.
Interacting with APIs
For data use cases such as creating a data pipeline, including AI from an LLM is often part of a longer workflow that involves other API calls.
For example, you may wish to use an API to retrieve stock data or to interact with a cloud platform.
LangChain's chain and agent features that allow you to connect these steps in sequence (and use additional business logic for branching pipelines) are ideal for this use case.
Building a Chatbot
Chatbots are one of the most popular use cases of AI, and generative AI holds a great deal of promise for chatbots that behave more realistically.
However, it can be cumbersome to control the personality of the chatbot and get it to remember the context of the conversation.
LangChain's prompt templates give you control of both the chatbot's personality (its tone of voice and its style of communication) and the responses it gives.
Additionally, the message history tools are useful for giving the chatbot a memory that is longer than the few hundred or thousand words that LLMs provide by default, allowing for greater consistency within a conversation or even across multiple conversations.
Other uses
This tutorial only scratches the surface of the possibilities. There are many more use cases of LLMs for data professionals.
Whether you are interested in creating a personal assistant, summarizing reports, or answering questions about support docs or a knowledge base, LangChain provides a framework for including AI into any data pipeline or data application that you can think of.
Take it to the Next Level
LangChain content is coming soon to DataCamp. In the meantime, you can learn about one of the use cases discussed here by taking the Building Chatbots in Python course.