This module contains a series of Beam YAML code samples that can be run using the command:
python -m apache_beam.yaml.main --pipeline_spec_file=/path/to/example.yaml
A good starting place is the Wordcount example under the root example directory. This example reads in a text file, splits the text on each word, groups by each word, and counts the occurrence of each word. This is a classic example used in the other SDK's and shows off many of the functionalities of Beam YAML.
Examples in this directory show off the various built-in transforms of the Beam YAML framework.
These examples leverage the built-in mapping transforms including MapToFields
,
Filter
and Explode
. More information can be found about mapping transforms
here.
These examples leverage the built-in Combine
transform for performing simple
aggregations including sum, mean, count, etc.
More information can be found about aggregation transforms here.