{ "cells": [ { "cell_type": "code", "execution_count": null, "metadata": { "id": "ijGzTHJJUCPY" }, "outputs": [], "source": [ "# Copyright 2024 Google LLC\n", "#\n", "# Licensed under the Apache License, Version 2.0 (the \"License\");\n", "# you may not use this file except in compliance with the License.\n", "# You may obtain a copy of the License at\n", "#\n", "# https://www.apache.org/licenses/LICENSE-2.0\n", "#\n", "# Unless required by applicable law or agreed to in writing, software\n", "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", "# See the License for the specific language governing permissions and\n", "# limitations under the License." ] }, { "cell_type": "markdown", "metadata": { "id": "VEqbX8OhE8y9" }, "source": [ "# Working with Parallel Function Calls and Multiple Function Responses in Gemini\n", "\n", "\n", " \n", " \n", " \n", " \n", "
\n", " \n", " \"Google
Open in Colab\n", " \n", " 		\n", " \n", " \"Google
Open in Colab Enterprise\n", " \n", " 		\n", " \n", " \"Vertex
Open in Workbench\n", " \n", " 		\n", " \n", " \"GitHub
View on GitHub\n", " \n", "
\n", "\n", "
\n", "\n", "Share to:\n", "\n", "\n", " \"LinkedIn\n", "\n", "\n", "\n", " \"Bluesky\n", "\n", "\n", "\n", " \"X\n", "\n", "\n", "\n", " \"Reddit\n", "\n", "\n", "\n", " \"Facebook\n", " " ] }, { "cell_type": "markdown", "metadata": { "id": "ZNJC1SkrsJY3" }, "source": [ "| | |\n", "|-|-|\n", "|Author(s) | [Kristopher Overholt](https://github.com/koverholt) |" ] }, { "cell_type": "markdown", "metadata": { "id": "CkHPv2myT2cx" }, "source": [ "## Overview\n", "\n", "Gemini is a family of generative AI models developed by Google DeepMind that is designed for multimodal use cases. The Gemini API gives you access to the Gemini models.\n", "\n", "[Function Calling](https://cloud.google.com/vertex-ai/docs/generative-ai/multimodal/function-calling) in Gemini lets you create a description of a function in your code, then pass that description to a language model in a request. The response from the model includes the name of a function that matches the description and the arguments to call it with.\n", "\n", "In this tutorial, you'll learn how to work with parallel function calling within Gemini Function Calling, including:\n", " \n", "- Handling parallel function calls for repeated functions\n", "- Working with parallel function calls across multiple functions\n", "- Extracting multiple function calls from a Gemini response\n", "- Calling multiple functions and returning them to Gemini\n", "\n", "### What is parallel function calling?\n", "\n", "In previous versions of the Gemini models (prior to May 2024), Gemini would return two or more chained function calls if the model determined that more than one function call was needed before returning a natural language summary. Here, a chained function call means that you get the first function call response, return the API data to Gemini, get a second function call response, return the API data to Gemini, and so on.\n", "\n", "In recent versions of specific Gemini models (from May 2024 and on), Gemini has the ability to return two or more function calls in parallel (i.e., two or more function call responses within the first function call response object). Parallel function calling allows you to fan out and parallelize your API calls or other actions that you perform in your application code, so you don't have to work through each function call response and return one-by-one! Refer to the [Gemini Function Calling documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling) for more information on which Gemini model versions support parallel function calling.\n", "\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": { "id": "r11Gu7qNgx1p" }, "source": [ "## Getting Started\n" ] }, { "cell_type": "markdown", "metadata": { "id": "No17Cw5hgx12" }, "source": [ "### Install Google Gen AI SDK" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "tFy3H3aPgx12" }, "outputs": [], "source": [ "%pip install --upgrade --quiet google-genai" ] }, { "cell_type": "markdown", "metadata": { "id": "R5Xep4W9lq-Z" }, "source": [ "### Restart runtime\n", "\n", "To use the newly installed packages in this Jupyter runtime, you must restart the runtime. You can do this by running the cell below, which restarts the current kernel.\n", "\n", "The restart might take a minute or longer. After it's restarted, continue to the next step." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "XRvKdaPDTznN" }, "outputs": [], "source": [ "import IPython\n", "\n", "app = IPython.Application.instance()\n", "app.kernel.do_shutdown(True)" ] }, { "cell_type": "markdown", "metadata": { "id": "SbmM4z7FOBpM" }, "source": [ "
\n", "⚠️ The kernel is going to restart. Please wait until it is finished before continuing to the next step. ⚠️\n", "
\n" ] }, { "cell_type": "markdown", "metadata": { "id": "dmWOrTJ3gx13" }, "source": [ "### Authenticate your notebook environment (Colab only)\n", "\n", "If you are running this notebook on Google Colab, run the cell below to authenticate your environment." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "NyKGtVQjgx13" }, "outputs": [], "source": [ "import sys\n", "\n", "if \"google.colab\" in sys.modules:\n", " from google.colab import auth\n", "\n", " auth.authenticate_user()" ] }, { "cell_type": "markdown", "metadata": { "id": "DF4l8DTdWgPY" }, "source": [ "### Set Google Cloud project information\n", "\n", "To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n", "\n", "Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "Nqwi-5ufWp_B" }, "outputs": [], "source": [ "import os\n", "\n", "PROJECT_ID = \"[your-project-id]\" # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n", "if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n", " PROJECT_ID = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n", "\n", "LOCATION = os.environ.get(\"GOOGLE_CLOUD_REGION\", \"us-central1\")\n", "\n", "from google import genai\n", "\n", "client = genai.Client(vertexai=True, project=PROJECT_ID, location=LOCATION)" ] }, { "cell_type": "markdown", "metadata": { "id": "jXHfaVS66_01" }, "source": [ "## Code Examples\n", "\n", "### Import libraries" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "lslYAvw37JGQ" }, "outputs": [], "source": [ "from typing import Any\n", "\n", "from IPython.display import Markdown, display\n", "from google.genai.types import (\n", " FunctionDeclaration,\n", " GenerateContentConfig,\n", " GenerateContentResponse,\n", " Part,\n", " Tool,\n", ")\n", "import wikipedia" ] }, { "cell_type": "markdown", "metadata": { "id": "b2acd610d52c" }, "source": [ "### Define helper function" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "32c90d8c452a" }, "outputs": [], "source": [ "# Helper function to extract one or more function calls from a Gemini Function Call response\n", "\n", "\n", "def extract_function_calls(response: GenerateContentResponse) -> list[dict]:\n", " function_calls: list[dict] = []\n", " for function_call in response.function_calls:\n", " function_call_dict: dict[str, dict[str, Any]] = {function_call.name: {}}\n", " for key, value in function_call.args.items():\n", " function_call_dict[function_call.name][key] = value\n", " function_calls.append(function_call_dict)\n", " return function_calls" ] }, { "cell_type": "markdown", "metadata": { "id": "j3KHAr6BsJY6" }, "source": [ "## Example: Parallel function calls on the same function\n", "\n", "A great use case for parallel function calling is when you have a function that only accepts one parameter per API call and you need to make repeated calls to that function.\n", "\n", "With Parallel Function Calling, rather than having to send N number of API requests to Gemini for N number function calls, instead you can send a single API request to Gemini, receive N number of Function Call Responses within a single response, make N number of external API calls in your code, then return all of the API responses to Gemini in bulk. And you can do all of this without any extra configuration in your function declarations, tools, or requests to Gemini.\n", "\n", "In this example, you'll do exactly that and use Parallel Function Calling in Gemini to ask about multiple topics on [Wikipedia](https://www.wikipedia.org/). Let's get started!\n", "\n", "### Write function declarations and wrap them in a tool\n", "\n", "First, define your function declarations and tool using the Vertex AI Python SDK:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "31a4ae78030e" }, "outputs": [], "source": [ "search_wikipedia = FunctionDeclaration(\n", " name=\"search_wikipedia\",\n", " description=\"Search for articles on Wikipedia\",\n", " parameters={\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"query\": {\n", " \"type\": \"string\",\n", " \"description\": \"Query to search for on Wikipedia\",\n", " },\n", " },\n", " },\n", ")\n", "\n", "wikipedia_tool = Tool(\n", " function_declarations=[\n", " search_wikipedia,\n", " ],\n", ")" ] }, { "cell_type": "markdown", "metadata": { "id": "11c5abb4bbb1" }, "source": [ "### Initialize the Gemini model\n", "\n", "Now you can initialize Gemini using a [model version that supports parallel function calling](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling):" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "521d7dbb44a7" }, "outputs": [], "source": [ "MODEL_ID = \"gemini-2.0-flash\"\n", "\n", "chat = client.chats.create(\n", " model=MODEL_ID, config=GenerateContentConfig(temperature=0, tools=[wikipedia_tool])\n", ")" ] }, { "cell_type": "markdown", "metadata": { "id": "2a8ac4fd2f45" }, "source": [ "### Send prompt to Gemini\n", "\n", "Send a prompt to Gemini that includes a phrase that you expect to invoke two or more function calls.\n", "\n", "In this case we'll ask about three different article topics to search for on Wikipedia:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "aD4UJ6BcsJY6" }, "outputs": [], "source": [ "prompt = \"Search for articles related to solar panels, renewable energy, and battery storage and provide a summary of your findings\"\n", "\n", "response = chat.send_message(prompt)" ] }, { "cell_type": "markdown", "metadata": { "id": "f8e67fe6825f" }, "source": [ "### Extract function names and parameters\n", "\n", "Use the helper function that we created earlier to extract the function names and function parameters for each Function Call that Gemini responded with:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "468e7308ebb8" }, "outputs": [], "source": [ "function_calls = extract_function_calls(response)\n", "function_calls" ] }, { "cell_type": "markdown", "metadata": { "id": "5d69b21870f1" }, "source": [ "Note that the helper function is just one way to extract fields from the structured Function Call response. You can modify the helper function or write your own custom code to extract and get the fields into your preferred format or data structure!\n", "\n", "### Make external API calls\n", "\n", "Next, you'll loop through the Function Calls and use the `wikipedia` Python package to make an API call for each search query:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "754bfbf1864a" }, "outputs": [], "source": [ "api_response = []\n", "\n", "# Loop over multiple function calls\n", "for function_call in function_calls:\n", " print(function_call)\n", "\n", " # Make external API call\n", " result = wikipedia.summary(function_call[\"search_wikipedia\"][\"query\"])\n", "\n", " # Collect all API responses\n", " api_response.append(result)" ] }, { "cell_type": "markdown", "metadata": { "id": "90340a8c2949" }, "source": [ "### Get a natural language summary\n", "\n", "Now you can return all of the API responses to Gemini so that it can generate a natural language summary:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "3409d561a84b" }, "outputs": [], "source": [ "# Return the API response to Gemini\n", "response = chat.send_message(\n", " [\n", " Part.from_function_response(\n", " name=\"search_wikipedia\",\n", " response={\n", " \"content\": api_response[0],\n", " },\n", " ),\n", " Part.from_function_response(\n", " name=\"search_wikipedia\",\n", " response={\n", " \"content\": api_response[1],\n", " },\n", " ),\n", " Part.from_function_response(\n", " name=\"search_wikipedia\",\n", " response={\n", " \"content\": api_response[2],\n", " },\n", " ),\n", " ],\n", ")\n", "display(Markdown(response.text))" ] }, { "cell_type": "markdown", "metadata": { "id": "6e55e0b45931" }, "source": [ "And you're done with no extra configuration necessary!\n", "\n", "Note that Gemini will use the information in your `FunctionDeclarations` to determine if and when it should return parallel Function Call responses, or it will determine which Function Calls need to be made before others to get information that a subsequent Function Call depends on. So be sure to account for this behavior in your logic and application code!" ] }, { "cell_type": "markdown", "metadata": { "id": "j3KHAr6BsJY6" }, "source": [ "## Example: Parallel function calls across multiple functions\n", "\n", "Another good fit for parallel function calling is when you have multiple, independent functions that you want to be able to call in parallel, which reduces the number of consecutive Gemini API calls that you need to make and (ideally) reduces the overall response time to the end user who is waiting for a natural language response.\n", "\n", "In this example, you'll use Parallel Function Calling in Gemini to ask about multiple aspects of topics and articles on [Wikipedia](https://www.wikipedia.org/).\n", "\n", "### Write function declarations and wrap them in a tool\n", "\n", "First, define your function declarations and tool using the Vertex AI Python SDK:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "577bd3ad36ed" }, "outputs": [], "source": [ "search_wikipedia = FunctionDeclaration(\n", " name=\"search_wikipedia\",\n", " description=\"Search for articles on Wikipedia\",\n", " parameters={\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"query\": {\n", " \"type\": \"string\",\n", " \"description\": \"Query to search for on Wikipedia\",\n", " },\n", " },\n", " },\n", ")\n", "\n", "suggest_wikipedia = FunctionDeclaration(\n", " name=\"suggest_wikipedia\",\n", " description=\"Get suggested titles from Wikipedia for a given term\",\n", " parameters={\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"query\": {\n", " \"type\": \"string\",\n", " \"description\": \"Query to search for suggested titles on Wikipedia\",\n", " },\n", " },\n", " },\n", ")\n", "\n", "summarize_wikipedia = FunctionDeclaration(\n", " name=\"summarize_wikipedia\",\n", " description=\"Get article summaries from Wikipedia\",\n", " parameters={\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"topic\": {\n", " \"type\": \"string\",\n", " \"description\": \"Query to search for article summaries on Wikipedia\",\n", " },\n", " },\n", " },\n", ")\n", "\n", "wikipedia_tool = Tool(\n", " function_declarations=[\n", " search_wikipedia,\n", " suggest_wikipedia,\n", " summarize_wikipedia,\n", " ],\n", ")" ] }, { "cell_type": "markdown", "metadata": { "id": "4884fb361482" }, "source": [ "### Initialize the Gemini model\n", "\n", "Now you can initialize Gemini using a [model version that supports parallel function calling](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling):" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "470894a34e87" }, "outputs": [], "source": [ "chat = client.chats.create(\n", " model=MODEL_ID, config=GenerateContentConfig(temperature=0, tools=[wikipedia_tool])\n", ")" ] }, { "cell_type": "markdown", "metadata": { "id": "4a7026e6a6a4" }, "source": [ "### Send prompt to Gemini\n", "\n", "Send a prompt to Gemini that includes a phrase that you expect to invoke two or more function calls.\n", "\n", "In this case we'll ask about three types of details to look up for a given topic on Wikipedia:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "aD4UJ6BcsJY6" }, "outputs": [], "source": [ "prompt = \"Show the search results, variations, and article summaries about Wikipedia articles related to the solar system\"\n", "\n", "response = chat.send_message(prompt)" ] }, { "cell_type": "markdown", "metadata": { "id": "add3c553b773" }, "source": [ "### Extract function names and parameters\n", "\n", "Use the helper function that we created earlier to extract the function names and function parameters for each Function Call that Gemini responded with:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "f02643f2344b" }, "outputs": [], "source": [ "function_calls = extract_function_calls(response)\n", "function_calls" ] }, { "cell_type": "markdown", "metadata": { "id": "0e86bbd26969" }, "source": [ "### Make external API calls\n", "\n", "Next, you'll loop through the Function Calls and use the `wikipedia` Python package to make APIs calls and gather information from Wikipedia:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "b3ca4ba3118b" }, "outputs": [], "source": [ "api_response: dict[str, Any] = {} # type: ignore\n", "\n", "# Loop over multiple function calls\n", "for function_call in function_calls:\n", " print(function_call)\n", " for function_name, function_args in function_call.items():\n", " # Determine which external API call to make\n", " if function_name == \"search_wikipedia\":\n", " result = wikipedia.search(function_args[\"query\"])\n", " if function_name == \"suggest_wikipedia\":\n", " result = wikipedia.suggest(function_args[\"query\"])\n", " if function_name == \"summarize_wikipedia\":\n", " result = wikipedia.summary(function_args[\"topic\"], auto_suggest=False)\n", "\n", " # Collect all API responses\n", " api_response[function_name] = result" ] }, { "cell_type": "markdown", "metadata": { "id": "5edff88c747e" }, "source": [ "### Get a natural language summary\n", "\n", "Now you can return all of the API responses to Gemini so that it can generate a natural language summary:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "0c94849ed303" }, "outputs": [], "source": [ "# Return the API response to Gemini\n", "response = chat.send_message(\n", " [\n", " Part.from_function_response(\n", " name=\"search_wikipedia\",\n", " response={\n", " \"content\": api_response.get(\"search_wikipedia\", \"\"),\n", " },\n", " ),\n", " Part.from_function_response(\n", " name=\"suggest_wikipedia\",\n", " response={\n", " \"content\": api_response.get(\"suggest_wikipedia\", \"\"),\n", " },\n", " ),\n", " Part.from_function_response(\n", " name=\"summarize_wikipedia\",\n", " response={\n", " \"content\": api_response.get(\"summarize_wikipedia\", \"\"),\n", " },\n", " ),\n", " ],\n", ")\n", "\n", "display(Markdown(response.text))" ] }, { "cell_type": "markdown", "metadata": { "id": "GpDvGrmtsJY8" }, "source": [ "And you're done! You successfully made parallel function calls for a couple of different use cases. Feel free to adapt the code samples here for your own use cases and applications. Or try another notebook to continue exploring other functionality in the Gemini API.\n", "\n", "Happy parallel function calling!" ] } ], "metadata": { "colab": { "name": "parallel_function_calling.ipynb", "toc_visible": true }, "kernelspec": { "display_name": "Python 3", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 0 }