{ "cells": [ { "cell_type": "code", "execution_count": null, "metadata": { "id": "ur8xi4C7S06n" }, "outputs": [], "source": [ "# Copyright 2024 Google LLC\n", "#\n", "# Licensed under the Apache License, Version 2.0 (the \"License\");\n", "# you may not use this file except in compliance with the License.\n", "# You may obtain a copy of the License at\n", "#\n", "# https://www.apache.org/licenses/LICENSE-2.0\n", "#\n", "# Unless required by applicable law or agreed to in writing, software\n", "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", "# See the License for the specific language governing permissions and\n", "# limitations under the License." ] }, { "cell_type": "markdown", "metadata": { "id": "JAPoU8Sm5E6e" }, "source": [ "# Text Summarization with Generative Models on Vertex AI\n", "\n", "\n", "\n", " \n", " \n", " \n", " \n", "
\n", " \n", " \"Google
Open in Colab\n", " \n", " 		\n", " \n", " \"Google
Open in Colab Enterprise\n", " \n", " 		\n", " \n", " \"Vertex
Open in Workbench\n", " \n", " 		\n", " \n", " \"GitHub
View on GitHub\n", " \n", "
\n", "\n", "
\n", "\n", "Share to:\n", "\n", "\n", " \"LinkedIn\n", "\n", "\n", "\n", " \"Bluesky\n", "\n", "\n", "\n", " \"X\n", "\n", "\n", "\n", " \"Reddit\n", "\n", "\n", "\n", " \"Facebook\n", " \n" ] }, { "cell_type": "markdown", "metadata": { "id": "7Rf0CRrzrgHd" }, "source": [ "| | | |\n", "|-|-|-|\n", "|Author(s) | [Polong Lin](https://github.com/polong-lin) | [Deepak Moonat](https://github.com/dmoonat)|" ] }, { "cell_type": "markdown", "metadata": { "id": "tvgnzT1CKxrO" }, "source": [ "## Overview\n", "Text summarization produces a concise and fluent summary of a longer text document. There are two main text summarization types: extractive and abstractive. Extractive summarization involves selecting critical sentences from the original text and combining them to form a summary. Abstractive summarization involves generating new sentences representing the original text's main points. In this notebook, you go through a few examples of how large language models can help with generating summaries based on text.\n", "\n", "Learn more about text summarization in the [official documentation](https://cloud.google.com/vertex-ai/docs/generative-ai/text/summarization-prompts)." ] }, { "cell_type": "markdown", "metadata": { "id": "d975e698c9a4" }, "source": [ "### Objective\n", "\n", "In this tutorial, you will learn how to use generative models to summarize information from text by working through the following examples:\n", "- Transcript summarization\n", "- Summarizing text into bullet points\n", "- Dialogue summarization with to-dos\n", "- Hashtag tokenization\n", "- Title & heading generation\n", "\n", "You also learn how to evaluate model-generated summaries by comparing to human-created summaries using ROUGE as an evaluation framework." ] }, { "cell_type": "markdown", "metadata": { "id": "f6d865e68adb" }, "source": [ "### Costs\n", "\n", "This tutorial uses billable components of Google Cloud:\n", "\n", "* Vertex AI\n", "\n", "Learn about [Vertex AI pricing](https://cloud.google.com/vertex-ai/pricing),\n", "and use the [Pricing Calculator](https://cloud.google.com/products/calculator/)\n", "to generate a cost estimate based on your projected usage." ] }, { "cell_type": "markdown", "metadata": { "id": "bs9TZo0GJKCR" }, "source": [ "## Getting Started" ] }, { "cell_type": "markdown", "metadata": { "id": "2a5AEr0lkLKD" }, "source": [ "### Install Vertex AI SDK and other required packages\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "148dd6321946" }, "outputs": [], "source": [ "%pip install google-cloud-aiplatform --upgrade --user -q\n", "%pip install \"bigframes<1.0.0\" -q\n", "%pip install rouge -q" ] }, { "cell_type": "markdown", "metadata": { "id": "FLVWFKFwkLKE" }, "source": [ "### Restart runtime\n", "\n", "To use the newly installed packages in this Jupyter runtime, you must restart the runtime. You can do this by running the cell below, which restarts the current kernel.\n", "\n", "The restart might take a minute or longer. After its restarted, continue to the next step.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "_Hsqwn4hkLKE" }, "outputs": [], "source": [ "import IPython\n", "\n", "app = IPython.Application.instance()\n", "app.kernel.do_shutdown(True)" ] }, { "cell_type": "markdown", "metadata": { "id": "GruF-GNqfU-2" }, "source": [ "
\n", "⚠️ The kernel is going to restart. Please wait until it is finished before continuing to the next step. ⚠️\n", "
" ] }, { "cell_type": "markdown", "metadata": { "id": "Xe7OuYuGkLKF" }, "source": [ "### Authenticate your notebook environment (Colab only)\n", "\n", "If you are running this notebook on Google Colab, run the cell below to authenticate your environment." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "U9Gx2SAZkLKF" }, "outputs": [], "source": [ "import sys\n", "\n", "if \"google.colab\" in sys.modules:\n", " from google.colab import auth\n", "\n", " auth.authenticate_user()" ] }, { "cell_type": "markdown", "metadata": { "id": "eutX8N98fhan" }, "source": [ "### Set Google Cloud project information and initialize Vertex AI SDK\n", "\n", "To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n", "\n", "Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment).\"" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "t917lBaOfmyN" }, "outputs": [], "source": [ "PROJECT_ID = \"your-project-id\" # @param {type:\"string\"}\n", "LOCATION = \"us-central1\" # @param {type:\"string\"}\n", "\n", "import vertexai\n", "\n", "vertexai.init(project=PROJECT_ID, location=LOCATION)" ] }, { "cell_type": "markdown", "metadata": { "id": "960505627ddf" }, "source": [ "### Import libraries\n", "\n", "Let's start by importing the libraries that we will need for this tutorial" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "PyQmSRbKA8r-" }, "outputs": [], "source": [ "from rouge import Rouge\n", "from vertexai.generative_models import GenerationConfig, GenerativeModel" ] }, { "cell_type": "markdown", "metadata": { "id": "UP76a2la7O-a" }, "source": [ "### Import models\n", "\n", "Here we load the pre-trained model called `gemini-2.0-flash`." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "7isig7e07O-a" }, "outputs": [], "source": [ "generation_model = GenerativeModel(\"gemini-2.0-flash\")" ] }, { "cell_type": "markdown", "metadata": { "id": "-W2JWi4pmeS0" }, "source": [ "#### Generation config\n", "\n", "- Each call that you send to a model includes parameter values that control how the model generates a response. The model can generate different results for different parameter values\n", "- Experiment with different parameter values to get the best values for the task\n", "\n", "Refer to following [link](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/prompt-design-strategies#experiment-with-different-parameter-values) for understanding different parameters" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "fb3Y2f58sQGC" }, "outputs": [], "source": [ "generation_config = GenerationConfig(temperature=0.1, max_output_tokens=256)" ] }, { "cell_type": "markdown", "metadata": { "id": "Mu1UAhoTKn51" }, "source": [ "## Text Summarization" ] }, { "cell_type": "markdown", "metadata": { "id": "TgZvJeBpJKCS" }, "source": [ "### Transcript summarization\n", "\n", "In this first example, you summarize a piece of text on quantum computing." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "UA2NjngeJKCS" }, "outputs": [], "source": [ "prompt = \"\"\"\n", "Provide a very short summary, no more than three sentences, for the following article:\n", "\n", "Our quantum computers work by manipulating qubits in an orchestrated fashion that we call quantum algorithms.\n", "The challenge is that qubits are so sensitive that even stray light can cause calculation errors — and the problem worsens as quantum computers grow.\n", "This has significant consequences, since the best quantum algorithms that we know for running useful applications require the error rates of our qubits to be far lower than we have today.\n", "To bridge this gap, we will need quantum error correction.\n", "Quantum error correction protects information by encoding it across multiple physical qubits to form a \"logical qubit,\" and is believed to be the only way to produce a large-scale quantum computer with error rates low enough for useful calculations.\n", "Instead of computing on the individual qubits themselves, we will then compute on logical qubits. By encoding larger numbers of physical qubits on our quantum processor into one logical qubit, we hope to reduce the error rates to enable useful quantum algorithms.\n", "\n", "Summary:\n", "\n", "\"\"\"\n", "\n", "response = generation_model.generate_content(\n", " contents=prompt, generation_config=generation_config\n", ").text\n", "print(response)" ] }, { "cell_type": "markdown", "metadata": { "id": "aade04b2e86a" }, "source": [ "Instead of a summary, we can ask for a TL;DR (\"too long; didn't read\"). You can compare the differences between the outputs generated." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "0c0c0f3dfe10" }, "outputs": [], "source": [ "prompt = \"\"\"\n", "Provide a TL;DR for the following article:\n", "\n", "Our quantum computers work by manipulating qubits in an orchestrated fashion that we call quantum algorithms.\n", "The challenge is that qubits are so sensitive that even stray light can cause calculation errors — and the problem worsens as quantum computers grow.\n", "This has significant consequences, since the best quantum algorithms that we know for running useful applications require the error rates of our qubits to be far lower than we have today.\n", "To bridge this gap, we will need quantum error correction.\n", "Quantum error correction protects information by encoding it across multiple physical qubits to form a \"logical qubit,\" and is believed to be the only way to produce a large-scale quantum computer with error rates low enough for useful calculations.\n", "Instead of computing on the individual qubits themselves, we will then compute on logical qubits. By encoding larger numbers of physical qubits on our quantum processor into one logical qubit, we hope to reduce the error rates to enable useful quantum algorithms.\n", "\n", "TL;DR:\n", "\"\"\"\n", "\n", "response = generation_model.generate_content(\n", " contents=prompt, generation_config=generation_config\n", ").text\n", "print(response)" ] }, { "cell_type": "markdown", "metadata": { "id": "5PATmHivJKCS" }, "source": [ "### Summarize text into bullet points\n", "In the following example, you use same text on quantum computing, but ask the model to summarize it in bullet-point form. Feel free to change the prompt." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "2orkDF2VJKCT" }, "outputs": [], "source": [ "prompt = \"\"\"\n", "Provide a very short summary in four bullet points for the following article:\n", "\n", "Our quantum computers work by manipulating qubits in an orchestrated fashion that we call quantum algorithms.\n", "The challenge is that qubits are so sensitive that even stray light can cause calculation errors — and the problem worsens as quantum computers grow.\n", "This has significant consequences, since the best quantum algorithms that we know for running useful applications require the error rates of our qubits to be far lower than we have today.\n", "To bridge this gap, we will need quantum error correction.\n", "Quantum error correction protects information by encoding it across multiple physical qubits to form a \"logical qubit,\" and is believed to be the only way to produce a large-scale quantum computer with error rates low enough for useful calculations.\n", "Instead of computing on the individual qubits themselves, we will then compute on logical qubits. By encoding larger numbers of physical qubits on our quantum processor into one logical qubit, we hope to reduce the error rates to enable useful quantum algorithms.\n", "\n", "Bullet points:\n", "\n", "\"\"\"\n", "\n", "generation_config = GenerationConfig(\n", " temperature=0.2, max_output_tokens=256, top_k=1, top_p=0.8\n", ")\n", "\n", "response = generation_model.generate_content(\n", " contents=prompt, generation_config=generation_config\n", ").text\n", "print(response)" ] }, { "cell_type": "markdown", "metadata": { "id": "yE7y-clBJKCT" }, "source": [ "### Dialogue summarization with to-dos\n", "Dialogue summarization involves condensing a conversation into a shorter format so that you don't need to read the whole discussion but can leverage a summary. In this example, you ask the model to summarize an example conversation between an online retail customer and a support agent, and include to-dos at the end." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "SV-BWzRhJKCT" }, "outputs": [], "source": [ "prompt = \"\"\"\n", "Please generate a summary of the following conversation and at the end summarize the to-do's for the support Agent:\n", "\n", "Customer: Hi, I'm Larry, and I received the wrong item.\n", "\n", "Support Agent: Hi, Larry. How would you like to see this resolved?\n", "\n", "Customer: That's alright. I want to return the item and get a refund, please.\n", "\n", "Support Agent: Of course. I can process the refund for you now. Can I have your order number, please?\n", "\n", "Customer: It's [ORDER NUMBER].\n", "\n", "Support Agent: Thank you. I've processed the refund, and you will receive your money back within 14 days.\n", "\n", "Customer: Thank you very much.\n", "\n", "Support Agent: You're welcome, Larry. Have a good day!\n", "\n", "Summary:\n", "\"\"\"\n", "\n", "generation_config = GenerationConfig(\n", " temperature=0.2, max_output_tokens=256, top_k=40, top_p=0.8\n", ")\n", "\n", "response = generation_model.generate_content(\n", " contents=prompt, generation_config=generation_config\n", ").text\n", "print(response)" ] }, { "cell_type": "markdown", "metadata": { "id": "LlOgWzmNJKCT" }, "source": [ "### Hashtag tokenization\n", "Hashtag tokenization is the process of taking a piece of text and getting the hashtag \"tokens\" out of it. You can use this, for example, if you want to generate hashtags for your social media campaigns. In this example, you take [this tweet from Google Cloud](https://twitter.com/googlecloud/status/1649127992348606469) and generate some hashtags you can use." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "BWa8rNV0JKCT" }, "outputs": [], "source": [ "prompt = \"\"\"\n", "Tokenize the hashtags of this tweet:\n", "\n", "Google Cloud\n", "@googlecloud\n", "How can data help our changing planet? 🌎\n", "\n", "In honor of #EarthDay this weekend, we're proud to share how we're partnering with\n", "@ClimateEngine\n", " to harness the power of geospatial data and drive toward a more sustainable future.\n", "\n", "Check out how → https://goo.gle/3mOUfts\n", "\"\"\"\n", "\n", "generation_config = GenerationConfig(\n", " temperature=0.8, max_output_tokens=1024, top_k=40, top_p=0.8\n", ")\n", "\n", "response = generation_model.generate_content(\n", " contents=prompt, generation_config=generation_config\n", ").text\n", "print(response)" ] }, { "cell_type": "markdown", "metadata": { "id": "7f-w7mUxJKCT" }, "source": [ "### Title & heading generation\n", "Below, you ask the model to generate five options for possible title/heading combos for a given piece of text." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "MWNri4DTJKCU" }, "outputs": [], "source": [ "prompt = \"\"\"\n", "Write a title for this text, give me five options:\n", "Whether helping physicians identify disease or finding photos of \"hugs,\" AI is behind a lot of the work we do at Google. And at our Arts & Culture Lab in Paris, we've been experimenting with how AI can be used for the benefit of culture.\n", "Today, we're sharing our latest experiments—prototypes that build on seven years of work in partnership the 1,500 cultural institutions around the world.\n", "Each of these experimental applications runs AI algorithms in the background to let you unearth cultural connections hidden in archives—and even find artworks that match your home decor.\"\n", "\"\"\"\n", "\n", "generation_config = GenerationConfig(\n", " temperature=0.8, max_output_tokens=256, top_k=1, top_p=0.8\n", ")\n", "\n", "response = generation_model.generate_content(\n", " contents=prompt, generation_config=generation_config\n", ").text\n", "print(response)" ] }, { "cell_type": "markdown", "metadata": { "id": "zcpmZnwKJKCU" }, "source": [ "## Evaluation\n", "You can evaluate the outputs from summarization tasks using [ROUGE](https://en.wikipedia.org/wiki/ROUGE_(metric)) as an evaluation framework. ROUGE (Recall-Oriented Understudy for Gisting Evaluation) are measures to automatically determine the quality of a summary by comparing it to other (ideal) summaries created by humans. The measures count the number of overlapping units such as n-gram, word sequences, and word pairs between the computer-generated summary to be evaluated and the ideal summaries created by humans." ] }, { "cell_type": "markdown", "metadata": { "id": "iD9eKq3SJKCU" }, "source": [ "Create a summary from a language model that you can use to compare against a human-generated summary." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "37m_fb-HJKCU" }, "outputs": [], "source": [ "ROUGE = Rouge()\n", "\n", "prompt = \"\"\"\n", "Provide a very short, maximum four sentences, summary for the following article:\n", "\n", "Our quantum computers work by manipulating qubits in an orchestrated fashion that we call quantum algorithms.\n", "The challenge is that qubits are so sensitive that even stray light can cause calculation errors — and the problem worsens as quantum computers grow.\n", "This has significant consequences, since the best quantum algorithms that we know for running useful applications require the error rates of our qubits to be far lower than we have today.\n", "To bridge this gap, we will need quantum error correction.\n", "Quantum error correction protects information by encoding it across multiple physical qubits to form a \"logical qubit,\" and is believed to be the only way to produce a large-scale quantum computer with error rates low enough for useful calculations.\n", "Instead of computing on the individual qubits themselves, we will then compute on logical qubits. By encoding larger numbers of physical qubits on our quantum processor into one logical qubit, we hope to reduce the error rates to enable useful quantum algorithms.\n", "\n", "Summary:\n", "\n", "\"\"\"\n", "\n", "generation_config = GenerationConfig(\n", " temperature=0.1, max_output_tokens=1024, top_k=40, top_p=0.8\n", ")\n", "\n", "candidate = generation_model.generate_content(\n", " contents=prompt, generation_config=generation_config\n", ").text\n", "\n", "print(candidate)" ] }, { "cell_type": "markdown", "metadata": { "id": "b44f9872e1ba" }, "source": [ "You will also need a human-generated summary that we will use to compare to the `candidate` generated by the model. We will call this `reference`." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "D0qNdPzOJKCc" }, "outputs": [], "source": [ "reference = 'Quantum computers are sensitive to noise and errors. To bridge this gap, we will need quantum error correction. Quantum error correction protects information by encoding across multiple physical qubits to form a \"logical qubit\".'" ] }, { "cell_type": "markdown", "metadata": { "id": "1KKaYhzwJKCc" }, "source": [ "Now you can take the candidate and reference to evaluate the performance. In this case, ROUGE will give you:\n", "\n", "- `rouge-1`, which measures unigram overlap\n", "- `rouge-2`, which measures bigram overlap\n", "- `rouge-l`, which measures the longest common subsequence" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "FHUH6VuTJKCc" }, "outputs": [], "source": [ "ROUGE.get_scores(candidate, reference)" ] } ], "metadata": { "colab": { "name": "text_summarization.ipynb", "toc_visible": true }, "kernelspec": { "display_name": "Python 3", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 0 }