Skip to content

Latest commit

 

History

History

beam-ml

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
data_preprocessing
data_preprocessing
 
 
rag_usecase
rag_usecase
 
 
README.md
README.md
 
 
alloydb_product_catalog_embeddings.ipynb
alloydb_product_catalog_embeddings.ipynb
 
 
automatic_model_refresh.ipynb
automatic_model_refresh.ipynb
 
 
bigquery_enrichment_transform.ipynb
bigquery_enrichment_transform.ipynb
 
 
bigquery_vector_ingestion_and_search.ipynb
bigquery_vector_ingestion_and_search.ipynb
 
 
bigtable_enrichment_transform.ipynb
bigtable_enrichment_transform.ipynb
 
 
custom_remote_inference.ipynb
custom_remote_inference.ipynb
 
 
dataframe_api_preprocessing.ipynb
dataframe_api_preprocessing.ipynb
 
 
gemma_2_sentiment_and_summarization.ipynb
gemma_2_sentiment_and_summarization.ipynb
 
 
image_processing_tensorflow.ipynb
image_processing_tensorflow.ipynb
 
 
mltransform_basic.ipynb
mltransform_basic.ipynb
 
 
nlp_tensorflow_streaming.ipynb
nlp_tensorflow_streaming.ipynb
 
 
per_key_models.ipynb
per_key_models.ipynb
 
 
run_custom_inference.ipynb
run_custom_inference.ipynb
 
 
run_inference_gemma.ipynb
run_inference_gemma.ipynb
 
 
run_inference_generative_ai.ipynb
run_inference_generative_ai.ipynb
 
 
run_inference_huggingface.ipynb
run_inference_huggingface.ipynb
 
 
run_inference_multi_model.ipynb
run_inference_multi_model.ipynb
 
 
run_inference_pytorch.ipynb
run_inference_pytorch.ipynb
 
 
run_inference_pytorch_tensorflow_sklearn.ipynb
run_inference_pytorch_tensorflow_sklearn.ipynb
 
 
run_inference_sklearn.ipynb
run_inference_sklearn.ipynb
 
 
run_inference_tensorflow.ipynb
run_inference_tensorflow.ipynb
 
 
run_inference_tensorflow_with_tfx.ipynb
run_inference_tensorflow_with_tfx.ipynb
 
 
run_inference_vertex_ai.ipynb
run_inference_vertex_ai.ipynb
 
 
run_inference_vllm.ipynb
run_inference_vllm.ipynb
 
 
run_inference_windowing.ipynb
run_inference_windowing.ipynb
 
 
run_inference_with_tensorflow_hub.ipynb
run_inference_with_tensorflow_hub.ipynb
 
 
run_inference_xgboost.ipynb
run_inference_xgboost.ipynb
 
 
speech_emotion_tensorflow.ipynb
speech_emotion_tensorflow.ipynb
 
 
tfma_beam.ipynb
tfma_beam.ipynb
 
 
vertex_ai_feature_store_enrichment.ipynb
vertex_ai_feature_store_enrichment.ipynb
 
 

README.md

ML sample notebooks

Starting with the Apache Beam SDK version 2.40, users have access to a RunInference transform.

This transform allows you to make predictions and inference on data with machine learning (ML) models. The model handler abstracts the user from the configuration needed for specific frameworks, such as Tensorflow, PyTorch, and others. For a full list of supported frameworks, see the About Beam ML page.

Use the notebooks

These notebooks illustrate ways to use Apache Beam's RunInference transforms, as well as different use cases for ModelHandler implementations. Beam comes with multiple ModelHandler implementations.

Load the notebooks

  1. To get started quickly with notebooks, use Colab.
  2. In Colab, open the notebook from GitHub using the notebook URL, for example:
https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_tensorflow.ipynb
  1. To run most notebooks, you need to change the Google Cloud project and bucket to your project and bucket.

Notebooks

This section contains the following example notebooks.

Data processing

Data enrichment

Prediction and inference with pretrained models

Custom inference

Machine Learning Use Cases

Automatic Model Refresh

Multi-model pipelines

Model Evaluation