Tools and Routing

Tutorial 5

Jan Kirenz

Tools and Routing

Setup

import os
import openai

from dotenv import load_dotenv, find_dotenv
_ = load_dotenv(find_dotenv()) 
openai.api_key = os.environ['OPENAI_API_KEY']
import datetime
import requests
import wikipedia
from pydantic import BaseModel, Field

from langchain.agents import tool
from langchain.tools.render import format_tool_to_openai_function
from langchain.chains.openai_functions.openapi import openapi_spec_to_openai_fn
from langchain.utilities.openapi import OpenAPISpec
from langchain.chat_models import ChatOpenAI
from langchain.prompts import ChatPromptTemplate
from langchain.agents.output_parsers import OpenAIFunctionsAgentOutputParser
from langchain.schema.agent import AgentFinish

Define a tool

Create tool

@tool
def search(query: str) -> str:
    """Search for weather online"""
    return "42f"

Inspect tool

search.name
  • ‘search’
search.description
  • ‘search(query: str) -> str - Search for weather online’ . . .
search.args
  • {‘query’: {‘title’: ‘Query’, ‘type’: ‘string’}}

Add a description

class SearchInput(BaseModel):
    query: str = Field(description="Thing to search for")
@tool(args_schema=SearchInput)
def search(query: str) -> str:
    """Search for the weather online."""
    return "42f"
search.args
  • {‘query’: {‘title’: ‘Query’, ‘description’: ‘Thing to search for’, ‘type’: ‘string’}}

Call function

search.run("sf")
  • ‘42f’

Create weather tool

Create tool

# Define the input schema
class OpenMeteoInput(BaseModel):
    latitude: float = Field(..., description="Latitude of the location to fetch weather data for")
    longitude: float = Field(..., description="Longitude of the location to fetch weather data for")

@tool(args_schema=OpenMeteoInput)
def get_current_temperature(latitude: float, longitude: float) -> dict:
    """Fetch current temperature for given coordinates."""
    
    BASE_URL = "https://api.open-meteo.com/v1/forecast"
    
    # Parameters for the request
    params = {
        'latitude': latitude,
        'longitude': longitude,
        'hourly': 'temperature_2m',
        'forecast_days': 1,
    }

    # Make the request
    response = requests.get(BASE_URL, params=params)
    
    if response.status_code == 200:
        results = response.json()
    else:
        raise Exception(f"API Request failed with status code: {response.status_code}")

    current_utc_time = datetime.datetime.utcnow()
    time_list = [datetime.datetime.fromisoformat(time_str.replace('Z', '+00:00')) for time_str in results['hourly']['time']]
    temperature_list = results['hourly']['temperature_2m']
    
    closest_time_index = min(range(len(time_list)), key=lambda i: abs(time_list[i] - current_utc_time))
    current_temperature = temperature_list[closest_time_index]
    
    return f'The current temperature is {current_temperature}°C'

Inspect function

get_current_temperature.name
  • ‘get_current_temperature’
get_current_temperature.description
  • ‘get_current_temperature’
get_current_temperature.args
  • {‘latitude’: {‘title’: ‘Latitude’, ‘description’: ‘Latitude of the location to fetch weather data for’, ‘type’: ‘number’}, ‘longitude’: {‘title’: ‘Longitude’, ‘description’: ‘Longitude of the location to fetch weather data for’, ‘type’: ‘number’}}

Convert this tool into OpenAI function

format_tool_to_openai_function(get_current_temperature)
  • {‘name’: ‘get_current_temperature’, ‘description’: ‘get_current_temperature(latitude: float, longitude: float) -> dict - Fetch current temperature for given coordinates.’, ‘parameters’: {‘title’: ‘OpenMeteoInput’, ‘type’: ‘object’, ‘properties’: {‘latitude’: {‘title’: ‘Latitude’, ‘description’: ‘Latitude of the location to fetch weather data for’, ‘type’: ‘number’}, ‘longitude’: {‘title’: ‘Longitude’, ‘description’: ‘Longitude of the location to fetch weather data for’, ‘type’: ‘number’}}, ‘required’: [‘latitude’, ‘longitude’]}}

Use tool

get_current_temperature({"latitude": 48.741400, "longitude": 9.1006302})
  • ‘The current temperature is 8.2°C’

Wikikedia tool

Create tool

@tool
def search_wikipedia(query: str) -> str:
    """Run Wikipedia search and get page summaries."""
    page_titles = wikipedia.search(query)
    summaries = []
    for page_title in page_titles[: 3]:
        try:
            wiki_page =  wikipedia.page(title=page_title, auto_suggest=False)
            summaries.append(f"Page: {page_title}\nSummary: {wiki_page.summary}")
        except (
            self.wiki_client.exceptions.PageError,
            self.wiki_client.exceptions.DisambiguationError,
        ):
            pass
    if not summaries:
        return "No good Wikipedia Search Result was found"
    return "\n\n".join(summaries)

Inspect

search_wikipedia.name
  • ‘search_wikipedia’
search_wikipedia.description
  • {‘name’: ‘search_wikipedia’, ‘description’: ‘search_wikipedia(query: str) -> str - Run Wikipedia search and get page summaries.’, ‘parameters’: {‘title’: ‘search_wikipediaSchemaSchema’, ‘type’: ‘object’, ‘properties’: {‘query’: {‘title’: ‘Query’, ‘type’: ‘string’}}, ‘required’: [‘query’]}}

Inspect

format_tool_to_openai_function(search_wikipedia)
  • {‘name’: ‘search_wikipedia’, ‘description’: ‘search_wikipedia(query: str) -> str - Run Wikipedia search and get page summaries.’, ‘parameters’: {‘title’: ‘search_wikipediaSchemaSchema’, ‘type’: ‘object’, ‘properties’: {‘query’: {‘title’: ‘Query’, ‘type’: ‘string’}}, ‘required’: [‘query’]}}

Use tool

search_wikipedia({"query": "Hochschule der Medien Stuttgart"})
  • ‘Page: Stuttgart Media University: The Stuttgart Media University or Media University (German: Hochschule der Medien) is a state university of media studies in Stuttgart, Germany, offering nearly 30 accredited bachelor's and master's degree programs within three faculties.: Stuttgart: Stuttgart (German: [ˈʃtʊtɡaʁt] ; Swabian: Schduagert [ˈʒ̊d̥ua̯ɡ̊ɛʕd̥]; names in other languages) is the capital and largest city of the German state of Baden-Württemberg. It is located on the Neckar river in a fertile valley known as the Stuttgarter Kessel (Stuttgart Cauldron) and lies an hour from the Swabian Jura and the Black Forest. Stuttgart has a population of 635,911, making it the sixth largest city in Germany, while over 2.8 million people live in the city's administrative region and nearly 5.5 million people in its metropolitan area, making it the fourth largest metropolitan area in Germany. The city and metropolitan area are consistently ranked among the top 20 European metropolitan areas by GDP; Mercer listed Stuttgart as 21st on its 2015 list of cities by quality of living; innovation agency 2thinknow ranked the city 24th globally out of 442 cities in its Innovation Cities Index; and the Globalization and World Cities Research Network ranked the city as a Beta-status global city in their 2020 survey. Stuttgart was one of the host cities for the official tournaments of the 1974 and 2006 FIFA World Cups.is unusual in the scheme of German cities. It is spread across a variety of hills (some of them covered in vineyards), valleys (especially around the Neckar river and the Stuttgart basin) and parks. The city is known as the “cradle of the automobile”. As such, it is home to famous automobile museums like the Mercedes-Benz Museum and Porsche Museum, as well as numerous auto-enthusiast magazines, which contributes to Stuttgart's status as Germany's “Autohauptstadt” (“car capital city”). The city's tourism slogan is “Stuttgart offers more”. Under current plans to improve transport links to the international infrastructure (as part of the Stuttgart 21 project), Stuttgart unveiled a new city logo and slogan in March 2008, describing itself as “Das neue Herz Europas” (“The new Heart of Europe”). For business, it describes itself as “Where business meets the future”. In July 2010, the city unveiled a new logo, designed to entice more business people to stay in the city and enjoy breaks in the area.Since the seventh millennium BC, the Stuttgart area has been an important agricultural area and has been host to a number of cultures seeking to utilize the rich soil of the Neckar valley. The Roman Empire conquered the area in AD 83 and built a massive castrum near Bad Cannstatt, making it the most important regional centre for several centuries. Stuttgart's roots were truly laid in the tenth century with its founding by Liudolf, Duke of Swabia, as a stud farm for his warhorses. Initially overshadowed by nearby Bad Cannstatt, the town grew steadily and was granted a charter in 1320. The fortunes of Stuttgart turned with those of the House of Württemberg, and they made it the capital of their county, duchy, and kingdom from the 15th century to 1918. Stuttgart prospered despite setbacks in the Thirty Years' War and devastating air raids by the Allies on the city and its automobile production during World War II. However, by 1952, the city had bounced back and became the major cultural, economic, industrial, financial, tourism and publishing centre it is today.Stuttgart is known for its strong high-tech industry, especially in the automotive sector. It has the highest general standard of prosperity of any German city. In addition to many medium-sized companies, several major corporations are headquartered in Stuttgart, including Porsche, Bosch, and Mercedes-Benz Group. Stuttgart is an important financial center; the Stuttgart Stock Exchange is the second largest in Germany (after Frankfurt), and the Landesbank Baden-Württemberg (LBBW) is Germany's largest Landesbank. Stuttgart is also a major transport junction; it is among the most congested conurbations of Europe, and its airport is the sixth-busiest in Germany (2019). Stuttgart is a city with a high number of immigrants; according to Dorling Kindersley's Eyewitness Travel Guide to Germany, “In the city of Stuttgart, every third inhabitant is a foreigner.” 40% of Stuttgart's residents, and 64% of the population below the age of five, are of immigrant background.: Menschenliebe: Menschenliebe is an independent German feature film directed by Alexander Tuschinski. It had its premiere in Stuttgart, Germany in December 2010. It was screened and received numerous awards at international film-festivals, was additionally shown in various cinemas and screening events in Germany, and was officially released online in June 2013. It is the first instalment of Tuschinski's informal Trilogy of Rebellion - three very different feature films connected by the same thoughts, ideas and main characters, although each tells an independent story: Menschenliebe, Timeless and an upcoming project called Revolution!. Additionally, the film Break-Up refers to some events of Menschenliebe.’

Routing

Include tools in function

functions = [
    format_tool_to_openai_function(f) for f in [
        search_wikipedia, get_current_temperature
    ]
]

model = ChatOpenAI(temperature=0).bind(functions=functions)

Invoke functions

model.invoke("what is the weather in stuttgart right now")
  • AIMessage(content=’‘, additional_kwargs={’function_call’: {‘name’: ‘get_current_temperature’, ‘arguments’: ‘{“latitude”: 48.7758,“longitude”: 9.1829}’}})
model.invoke("what is langchain")
  • AIMessage(content=’‘, additional_kwargs={’function_call’: {‘name’: ‘search_wikipedia’, ‘arguments’: ‘{“query”: “langchain”}’}})

Create prompt and chain

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are helpful but sassy assistant"),
    ("user", "{input}"),
])

chain = prompt | model

Invoke chain

chain.invoke({"input": "what is the weather in stuttgart right now"})

AIMessage(content=’‘, additional_kwargs={’function_call’: {‘name’: ‘get_current_temperature’, ‘arguments’: ‘{“latitude”: 48.7758,“longitude”: 9.1829}’}})

Use output parser

chain = prompt | model | OpenAIFunctionsAgentOutputParser()
result = chain.invoke({"input": "what is the weather in stuttgart right now"})
type(result)
  • langchain.schema.agent.AgentActionMessageLog

Inspect result

result.tool
  • ‘get_current_temperature’
result.tool_input
  • {‘latitude’: 48.7758, ‘longitude’: 9.1829}
get_current_temperature(result.tool_input)
  • ‘The current temperature is 10.1°C’

Try a new input

result = chain.invoke({"input": "Hi! How are you?"})
type(result)
  • langchain.schema.agent.AgentFinish
result.return_values
  • {‘output’: “Hello! I’m an AI assistant, so I don’t have feelings, but I’m here to help you. How can I assist you today?”}

Define output of function

Define route function

def route(result):
    if isinstance(result, AgentFinish):
        return result.return_values['output']
    else:
        tools = {
            "search_wikipedia": search_wikipedia, 
            "get_current_temperature": get_current_temperature,
        }
        return tools[result.tool].run(result.tool_input)

Create chain

chain = prompt | model | OpenAIFunctionsAgentOutputParser() | route

Input a weather quaestion

result = chain.invoke({"input": "What is the weather in stuttgart right now?"})
result
  • ‘The current temperature is 10.1°C’

Input a general queation

result = chain.invoke({"input": "What is langchain?"})
result

‘Page: LangChain: LangChain is a framework designed to simplify the creation of applications using large language models (LLMs). As a language model integration framework, LangChain's use-cases largely overlap with those of language models in general, including document analysis and summarization, chatbots, and code analysis.: Prompt engineering: Prompt engineering is the process of structuring text that can be interpreted and understood by a generative AI model. A prompt is natural language text describing the task that an AI should perform.A prompt for a text-to-text model can be a query such as “what is Fermat's little theorem?”, a command such as “write a poem about leaves falling”, a short statement of feedback (for example, “too verbose”, “too formal”, “rephrase again”, “omit this word”) or a longer statement including context, instructions, and input data. Prompt engineering may involve phrasing a query, specifying a style, providing relevant context or assigning a role to the AI such as “Act as a native French speaker”. A prompt may include a few examples for a model to learn from, such as “maison -> house, chat -> cat, chien ->”, an approach called few-shot learning.When communicating with a text-to-image or a text-to-audio model, a typical prompt is a description of a desired output such as “a high-quality photo of an astronaut riding a horse” or “Lo-fi slow BPM electro chill with organic samples”. Prompting a text-to-image model may involve adding, removing, emphasizing and re-ordering words to achieve a desired subject, style, layout, lighting, and aesthetic.: Sentence embedding: In natural language processing, a sentence embedding refers to a numeric representation of a sentence in the form of a vector of real numbers which encodes meaningful semantic information.State of the art embeddings are based on the learned hidden layer representation of dedicated sentence transformer models. BERT pioneered an approach involving the use of a dedicated [CLS] token prepended to the beginning of each sentence inputted into the model; the final hidden state vector of this token encodes information about the sentence and can be fine-tuned for use in sentence classification tasks. In practice however, BERT's sentence embedding with the [CLS] token achieves poor performance, often worse than simply averaging non-contextual word embeddings. SBERT later achieved superior sentence embedding performance by fine tuning BERT's [CLS] token embeddings through the usage of a siamese neural network architecture on the SNLI dataset. approaches are loosely based on the idea of distributional semantics applied to sentences. Skip-Thought trains an encoder-decoder structure for the task of neighboring sentences predictions. Though this has been shown to achieve worse performance than approaches such as InferSent or SBERT. alternative direction is to aggregate word embeddings, such as those returned by Word2vec, into sentence embeddings. The most straightforward approach is to simply compute the average of word vectors, known as continuous bag-of-words (CBOW). However, more elaborate solutions based on word vector quantization have also been proposed. One such approach is the vector of locally aggregated word embeddings (VLAWE), which demonstrated performance improvements in downstream text classification tasks.’

Just say hi

chain.invoke({"input": "hi!"})
  • ‘Hello! How can I assist you today?’

Acknowledgments

What’s next?

Congratulations! You have completed this tutorial 👍

Next, you may want to go back to the lab’s website