TinyML: Audio Classification By Utilizing Synthetic Data

Traditionally, training machine learning models for audio classification requires large amounts of real-world data, which can be expensive, time-consuming, and difficult to obtain. Machine-learning models trained on synthetic data can outperform models trained using real data in certain circumstances. So, how can synthetic data be generated? This is where Generative AI comes into play. Generative AI offers a powerful solution by allowing us to create synthetic data sets that are highly realistic and diverse. This synthetic data can be generated quickly and easily, allowing for faster and more efficient model development.

The goal of this project is to demonstrate how we can develop an Audio Classification system that can distinguish between different classes. To achieve this, we will be using OpenAI's ChatGPT, Meta's AudioCraft, Nvidia's Jetson AGX Orin Developer Kit 64 GB andEdge Impulse platform to train our model and then deploy it to an edge device such as Arduino GIGA R1 WiFi.

Our classes are:

- Speech
- Music
- Noise

Below flow diagram can better explain the overall architecture of the project.

This flow diagram illustrates the process for audio classification on the Arduino GIGA R1 using synthetic data generated with Meta's Audiocraft and ChatGPT. The process consists of three stages: data processing, training, and deployment. In the data processing stage, ChatGPT is used to generate text prompts for three different classes: noise, speech, and music. Then, Meta's Audiocraft is utilized on the NVIDIA Jetson AGX Orin 64GB Developer Kit to generate a synthetic dataset. In the training stage, the Edge Impulse cloud platform is used to train the model. Finally, in the deployment stage, the trained model is deployed on the Arduino GIGA R1.

Text prompts generation using ChatGPT

I plan to leverage the capabilities of ChatGPT to generate diverse text prompts for various classes. The prompt utilized to gather examples from ChatGPT is as follows:

I am creating a dataset of synthetic audio data for an audio classification task, which generates audio from text prompts. The first class in this dataset is background noise. Here is a list of examples for this class: 

A hammer is hitting a wooden surface,
A noise of nature,
The sound of waves crashing on the shore,
A thunderstorm in the distance,
Traffic noise on a busy street,
The hum of an air conditioning unit,
Birds chirping in the morning,
The sound of a train passing

Could you please provide other 100 examples?

Please note that these are just examples. You can use these prompts as a starting point, or you can create your own prompts that are specific to your needs.

Here are examples of the different types of output generated by ChatGPT for each category.

Below is list of a variety of speech samples encompassing diverse genders, emotions, and speaking styles. These samples can be in the form of dialogues, monologues, narratives, or any other format that effectively showcases the range of human speech.

Woman delivering a motivational speech
Man conducting a job interview
Group discussion in a business meeting
Woman giving a lecture in a classroom
Man reading a poem aloud
Casual conversation at a park
Woman participating in a radio talk show
Man narrating a story
Group discussion at a social gathering
Woman giving instructions in a workshop
Man reciting a monologue
Casual conversation at a coffee shop
Woman conducting a podcast interview
Man providing commentary for a sports event
Group discussion at a book club meeting
Woman leading a team meeting
Man practicing a speech in front of a mirror
Casual conversation at a birthday party
Woman participating in a panel discussion
Man delivering a eulogy at a funeral
Group discussion at a community forum
Woman giving a guided tour
Man explaining a scientific concept
Casual conversation during a road trip
Woman leading a support group session
Man giving a toast at a wedding
Group discussion at a technology conference
Woman conducting a product demonstration
Man providing commentary for a documentary
Casual conversation at a picnic
Woman participating in a debate
Man teaching a language lesson
Group discussion in a political debate
Woman leading a mindfulness meditation session
Man reciting lines from a play
Casual conversation at a fitness class
Woman conducting a cooking class
Man giving a live commentary for a gaming session
Group discussion at a cultural event
Woman narrating a travel vlog
Man participating in a mock trial
Casual conversation at a museum
Woman leading a fitness workout
Man providing commentary for a fashion show
Group discussion at a charity event
Woman giving a weather report
Man delivering a farewell speech
Casual conversation at a farmers' market
Woman conducting a parenting workshop
Man participating in a courtroom drama
Group discussion at a science fair
Woman leading a creative writing workshop
Man providing commentary for a cooking competition
Casual conversation at a community garden
Woman delivering a TED talk
Man explaining a historical event
Group discussion at a film screening
Woman conducting a music appreciation class
Man participating in a comedy improv show
Casual conversation at a beach bonfire
Woman giving a news report
Man reciting famous speeches
Group discussion at an art exhibition
Woman leading a dog training session
Man providing commentary for a wildlife documentary
Casual conversation at a yoga retreat
Woman participating in a reality TV show confessional
Man conducting a virtual meeting
Group discussion at a gaming convention
Woman giving a tech tutorial
Man leading a poetry slam
Casual conversation at a comedy club
Woman narrating a historical documentary
Man providing commentary for a car race
Group discussion at a space exploration conference
Woman leading a birdwatching expedition
Man explaining a mathematical concept
Casual conversation at a dog park
Woman conducting a DIY crafting workshop
Man participating in a radio play
Group discussion at a psychology seminar
Woman giving a gardening tutorial
Man delivering a commencement speech
Casual conversation at a school reunion
Woman participating in a language exchange
Man providing commentary for a fashion runway
Group discussion at a podcasting conference
Woman leading a nature photography workshop
Man explaining a philosophical theory
Casual conversation at a tech startup office
Woman conducting a virtual fitness class
Man reciting famous movie quotes
Group discussion at an environmental sustainability summit
Woman giving a cybersecurity tutorial
Man providing commentary for a live painting session
Casual conversation at a rooftop party
Woman leading a mindfulness retreat
Man participating in a quiz show
Group discussion at a cultural diversity forum
Woman delivering a voiceover for an audiobook
Man reading a bedtime story
Woman leading a fitness class
Man explaining a coding tutorial
Woman participating in a radio drama
Man reciting a famous speech
Woman giving a guided meditation
Man narrating a wildlife documentary
Woman discussing travel experiences
Man participating in a language exchange
Woman conducting a cooking demonstration
Man leading a team-building exercise
Woman providing commentary for a fashion show
Man delivering a comedy stand-up routine
Woman participating in a debate
Man giving a motivational talk
Woman narrating a historical documentary
Man leading a podcast interview
Woman leading a children's storytelling session
Man conducting a virtual meeting
Woman participating in a book club discussion
Man explaining a scientific discovery
Woman providing commentary for a sports match
Man reciting a love poem
Woman leading a seminar on personal development
Man giving a live commentary for a gaming tournament
Woman conducting a mindfulness workshop
Man participating in a quiz show
Woman delivering a weather report
Man leading a community town hall meeting
Woman discussing art and creativity
Man reading aloud from a novel
Woman participating in a technology panel
Man providing commentary for a car race
Woman leading a workshop on public speaking
Man explaining a complex mathematical concept
Woman giving a TED Talk
Man participating in a courtroom drama
Woman conducting a music appreciation class
Man reciting lines from a Shakespearean play
Woman providing commentary for a reality TV show
Man leading a discussion on artificial intelligence
Woman leading a session on stress management
Man narrating a fantasy adventure story
Woman participating in a podcasting workshop
Man giving a demonstration on DIY home improvement
Woman conducting a virtual language lesson
Man explaining the rules of a board game
Woman leading a discussion on environmental sustainability
Man providing commentary for a live art demonstration
Woman delivering a commencement address
Man participating in a mock trial
Woman conducting a DIY crafting tutorial
Man leading a community poetry slam
Woman participating in a political debate
Man reciting famous movie quotes
Woman leading a parenting support group
Man explaining the history of a cultural festival
Woman providing commentary for a cooking competition
Man giving a guided tour of a historical site
Woman participating in a radio talk show
Man leading a virtual science experiment
Woman conducting a fitness workout session
Man explaining the principles of mindfulness
Woman discussing fashion trends
Man participating in a comedy improv show
Woman providing commentary for a technology expo
Man leading a discussion on space exploration
Woman conducting a podcast interview with a celebrity
Man explaining the art of storytelling
Woman leading a virtual painting class
Man participating in a panel discussion on innovation
Woman delivering a seminar on financial literacy
Man reciting lines from a classic play
Woman conducting a cooking class for beginners
Man leading a virtual trivia night
Woman participating in a panel discussion on diversity
Man explaining the process of scientific research
Woman providing commentary for a fashion runway show
Man leading a workshop on effective communication
Woman discussing the latest trends in technology
Man participating in a debate on ethics
Woman narrating a true crime podcast episode
Man delivering a eulogy at a memorial service
Woman leading a session on goal setting
Man reading and interpreting a poem
Woman providing commentary for a live painting exhibition
Man conducting a virtual town hall meeting
Woman participating in a radio play
Man explaining the rules of a card game
Woman leading a workshop on sustainable living
Man reciting a famous historical speech
Woman conducting a podcast interview with an expert
Man delivering a virtual keynote address
Woman participating in a technology innovation panel
Man explaining the process of scientific discovery
Woman leading a virtual book club discussion
Man narrating a nature documentary
Woman providing commentary for a travel documentary
Man participating in a discussion on artificial intelligence
Woman conducting a workshop on creative writing
Man giving a demonstration on computer programming
Woman leading a workshop on effective leadership
Man participating in a virtual reality gaming discussion
Woman delivering a speech on climate change
Man reciting a monologue from a Shakespearean play
Woman providing commentary for a live fashion show
Man leading a discussion on space exploration
Woman conducting a virtual travel storytelling session
Man explaining the principles of artificial intelligence
Woman participating in a panel discussion on social issues
Man narrating a historical reenactment
Woman leading a guided mindfulness meditation
Man giving a motivational talk on personal growth
Woman participating in a radio talk show interview
Man conducting a virtual seminar on public speaking
Woman delivering a virtual lecture on psychology
Man reciting lines from a classic literature piece
Woman leading a panel discussion on innovation
Man providing commentary for a live eSports tournament
Woman discussing the latest trends in digital marketing
Man participating in a discussion on cultural diversity
Woman conducting a virtual workshop on creative writing
Man explaining the principles of quantum mechanics
Woman delivering a TED Talk on technology trends
Man leading a virtual town hall meeting on community issues
Woman participating in a panel discussion on entrepreneurship
Man reciting famous quotes from historical figures
Woman providing commentary for a live cooking competition
Man leading a virtual discussion on artificial intelligence ethics
Woman conducting a virtual workshop on effective communication
Man delivering a commencement address to graduating students
Woman participating in a virtual book club discussion
Man explaining the principles of sustainable living
Woman leading a virtual seminar on mental health awareness
Man narrating a virtual guided tour of a historical landmark
Woman discussing the impact of technology on society
Man giving a demonstration on scientific experimentation
Woman participating in a virtual panel discussion on women in STEM
Man reciting a famous speech from a political leader
Woman providing commentary for a virtual fashion runway show
Man leading a virtual workshop on team collaboration
Woman conducting a virtual fitness and wellness seminar
Man explaining the process of innovation in technology
Woman participating in a panel discussion on sustainability
Man delivering a virtual keynote address at a conference
Woman leading a virtual workshop on personal finance
Man participating in a panel discussion on artificial intelligence in education
Woman conducting a virtual parenting support group
Man reciting lines from a virtual reality play
Woman providing commentary for a virtual reality gaming championship
Man leading a virtual discussion on the future of space exploration
Woman discussing the role of women in technology
Man explaining the principles of quantum computing
Woman delivering a TED Talk on the power of storytelling
Man participating in a virtual debate on ethical dilemmas
Woman conducting a virtual workshop on digital literacy
Man narrating a virtual nature documentary
Woman providing commentary for a virtual travel documentary
Man leading a virtual panel discussion on artificial intelligence in healthcare
Woman participating in a virtual town hall meeting on community development
Man reciting lines from a virtual reality theater performance
Woman discussing the impact of social media on society
Man giving a virtual demonstration on renewable energy technologies
Woman leading a virtual workshop on mindfulness in the workplace
Man explaining the principles of genetic engineering
Woman participating in a virtual panel discussion on climate action
Man delivering a virtual TED Talk on the future of education
Woman providing commentary for a virtual cooking show
Man leading a virtual workshop on effective teamwork
Woman conducting a virtual seminar on mental resilience
Man participating in a virtual panel discussion on the future of transportation
Woman discussing the potential of artificial intelligence in art
Man reciting lines from a virtual reality poetry slam
Woman providing commentary for a virtual fashion design showcase
Man narrating a virtual documentary on historical innovations
Woman leading a virtual workshop on personal branding
Man explaining the principles of sustainable urban planning
Woman participating in a virtual panel discussion on diversity in technology
Man delivering a virtual keynote address on social justice
Woman conducting a virtual workshop on effective communication in virtual teams
Man reciting famous lines from a virtual reality movie
Woman discussing the impact of artificial intelligence on the job market
Man giving a virtual demonstration on the use of augmented reality in education
Woman leading a virtual seminar on inclusive leadership
Man participating in a virtual panel discussion on the ethics of AI in healthcare
Woman providing commentary for a virtual reality gaming championship
Man leading a virtual workshop on sustainable business practices
Woman conducting a virtual town hall meeting on environmental conservation
Man explaining the principles of bioinformatics
Woman participating in a virtual panel discussion on the future of transportation
Man reciting lines from a virtual reality poetry slam
Woman providing commentary for a virtual fashion design showcase
Man narrating a virtual documentary on historical innovations
Woman leading a virtual workshop on personal branding
Man explaining the principles of sustainable urban planning
Woman participating in a virtual panel discussion on diversity in technology
Man delivering a virtual keynote address on social justice
Woman conducting a virtual workshop on effective communication in virtual teams
Man reciting famous lines from a virtual reality movie
Woman discussing the impact of artificial intelligence on the job market

Then, another class is music. Musical pieces in a variety of genres and styles, showcasing different musical instrument, tempos, rhythms, melodies, and harmonies.

A classical piano composition with a melancholic tone.
Upbeat jazz ensemble featuring saxophone, trumpet, and drums.
Electronic dance music (EDM) track with a pulsating beat.
Acoustic guitar solo playing a lively flamenco piece.
Ambient instrumental with soothing synthesizers and gentle percussion.
Rock ballad with powerful vocals and electric guitar solos.
Traditional Indian sitar and tabla duet.
African drum ensemble with rhythmic patterns and tribal chants.
Bossa nova jazz with smooth guitar and soft percussion.
Celtic folk song featuring violin, flute, and bodhran.
Reggae track with a laid-back groove and Caribbean influences.
Funky bassline-driven groove with brass instruments.
Minimalist piano piece reminiscent of Erik Satie.
Heavy metal guitar riff with intense drumming.
Up-tempo bluegrass with banjo, fiddle, and mandolin.
Synthwave track inspired by 80s electronic music.
Flamboyant Broadway musical number with vocals and orchestration.
Ambient chillout music with ethereal synthesizers and gentle beats.
Japanese traditional koto and shakuhachi composition.
Hip-hop beat with catchy samples and rap vocals.
Spanish flamenco fusion with a modern twist.
Progressive rock epic with intricate instrumental sections.
Caribbean steel drum ensemble playing a festive tune.
Traditional Irish pub song with accordion and tin whistle.
Upbeat ska with a lively horn section.
Synthesized chiptune inspired by classic video game music.
Brazilian samba with percussion and lively rhythms.
Ambient soundscape with nature sounds and electronic elements.
Electric blues guitar solo with a soulful vibe.
Baroque chamber music featuring strings and harpsichord.
Gospel choir singing a powerful and uplifting hymn.
Psychedelic rock with trippy effects and distorted guitars.
Hard-hitting techno track with pulsating beats.
Mariachi band performing a festive Mexican folk song.
Traditional Chinese erhu and guzheng duet.
Smooth R&B ballad with soulful vocals and jazzy instrumentation.
Celtic punk rock with energetic guitar and accordion.
Caribbean calypso with steel drums and catchy melodies.
Ambient dubstep with deep bass and atmospheric textures.
Russian folk song with balalaika and accordion.
Upbeat swing jazz with a lively big band arrangement.
Bollywood-inspired film score with traditional instruments.
Synth-pop track with catchy hooks and electronic beats.
Middle Eastern oud and darbuka instrumental.
Avant-garde experimental piece with unconventional sounds.
South American tango with accordion and passionate melodies.
Up-tempo bluegrass with a driving banjo and fiddle.
Traditional Maori haka with powerful vocal chants and percussion.
Dark and moody post-rock instrumental with atmospheric guitars.
Spanish bolero with romantic vocals and orchestral accompaniment.
Caribbean dancehall with a rhythmic beat and reggae influences.
Ambient drone music with slowly evolving textures.
Latin jazz fusion with a mix of traditional and modern instruments.
Synthesized ambient sounds inspired by deep space.
Italian opera aria with dramatic vocals and orchestration.
Electronic glitch-hop with stuttering beats and digital effects.
Native American flute solo with nature sounds in the background.
Upbeat folk-pop with acoustic guitars and harmonica.
African highlife music with upbeat rhythms and joyful melodies.
Hard-hitting dub reggae with heavy bass and echoing effects.
Japanese anime-inspired pop song with energetic vocals.
Celtic harp solo playing a whimsical melody.
Progressive metal with complex guitar riffs and intricate drumming.
Russian balalaika ensemble playing a lively folk dance.
Ambient rainforest sounds blended with electronic elements.
Irish step dance music with fast-paced fiddle and accordion.
Traditional Scottish bagpipe and drum march.
Caribbean soca with lively beats and infectious energy.
Experimental noise rock with dissonant guitar and chaotic sounds.
Traditional Native American powwow drumming and singing.
Upbeat ska punk with horns and energetic vocals.
Chinese guqin solo, capturing the essence of classical Chinese music.
Brazilian forró with accordion and triangle.
Ambient post-rock with atmospheric guitars and gradual build-ups.
Bollywood dance track with vibrant instrumentation.
Jazz fusion with a mix of electronic and acoustic instruments.
Synthesized ambient sounds inspired by an underwater world.
Italian tarantella folk dance with accordion and tambourine.
Traditional Hawaiian slack-key guitar instrumental.
Up-tempo klezmer music with clarinet and accordion.
Ambient electronica with spacey synthesizers and ethereal vibes.
Spanish paso doble with a dramatic orchestral arrangement.
Jamaican dub with deep bass and echoing effects.
Celtic folk-rock with a mix of traditional and modern instruments.
Traditional Persian santur and daf duet.
Electronic trance with a driving beat and uplifting melodies.
Turkish belly dance music with darbuka and ney.
Experimental ambient noise with unconventional instrumentation.
Latin pop with a catchy melody and rhythmic percussion.
African kora and djembe improvisation.
Upbeat country swing with twangy guitars and fiddle.
Indian classical raga with sitar and tabla.
Synthesized space exploration soundtrack with futuristic elements.
French chanson with romantic vocals and accordion.
Bluesy harmonica solo accompanied by acoustic guitar.
Indonesian gamelan ensemble playing a traditional piece.
Upbeat electro-swing with a fusion of jazz and electronic beats.
Middle Eastern belly dance fusion with modern production.
Experimental ambient piece inspired by a dream-like state.
Japanese taiko drum ensemble performing a powerful rhythmic composition.
Ambient trance with atmospheric pads and pulsating beats.
South American chacarera with folk guitar and percussion.
Upbeat indie pop with catchy hooks and jangly guitars.
Traditional Swedish polska with fiddle and nyckelharpa.
Hip-hop instrumental with a laid-back groove and sampled vinyl scratches.
Irish sean-nós singing with emotive vocals and minimal accompaniment.
Funk rock with a groovy bassline and brass section.
Japanese enka ballad with soulful vocals and orchestral backing.
Ambient electronic piece inspired by the sounds of nature.
Gypsy jazz swing with Django Reinhardt-style guitar solos.
Latin bolero with romantic lyrics and lush orchestration.
Up-tempo electronic folk fusion with bagpipes and synthesizers.
Spanish rumba with flamenco guitar and percussion.
Progressive metalcore with heavy guitar breakdowns and intense vocals.
Traditional Appalachian bluegrass with banjo, mandolin, and harmonizing vocals.
Synthesized futuristic sci-fi soundtrack with electronic elements.
Cuban son with lively Afro-Cuban rhythms and trumpet solos.
Ambient vocal choir piece with ethereal harmonies.
Brazilian batucada percussion ensemble playing a carnival rhythm.
Psybient track blending psychedelic and ambient electronic elements.
Traditional Japanese Noh theater music with flute and drums.
Upbeat ska-reggae fusion with a horn section and reggae-style vocals.
Darkwave synth-pop with haunting melodies and atmospheric synths.
Maqam improvisation in Arabic music with oud and percussion.
Traditional Russian folk song with balalaika and accordion.
Jazz-funk fusion with a tight rhythm section and brass stabs.
Indian classical thumri with expressive vocal ornamentation.
Ambient post-metal with atmospheric guitar swells and heavy drums.
Caribbean zouk with a danceable rhythm and tropical vibes.
Synthesized cyberpunk-inspired soundtrack with electronic beats.
Traditional Scottish pipe and drum march with bagpipes.
Brazilian chorinho with flute and cavaquinho.
Progressive electronic rock with dynamic shifts and synthesizer solos.
West African highlife with palm wine guitar and percussion.
Irish tin whistle and bodhran duet playing a lively reel.
Spanish copla with dramatic vocals and orchestral accompaniment.
Upbeat electro-pop with catchy vocal hooks and synthesizers.
Middle Eastern belly dance fusion with electronic dance beats.
Experimental noise piece with unconventional instrumentation and effects.
Japanese shakuhachi meditation music with nature sounds.
Country ballad with pedal steel guitar and heartfelt lyrics.
Ambient industrial techno with mechanical sounds and electronic textures.
Brazilian maracatu with syncopated rhythms and percussion ensemble.
Progressive rock with complex time signatures and instrumental virtuosity.
Traditional Irish sean-nós dance music with accordion and fiddle.
Synthesized video game boss battle music with epic orchestration.
Turkish saz and ney improvisation in a modal scale.
Reggaeton with a catchy dembow rhythm and Latin-style vocals.
Ambient soundscape inspired by a bustling cityscape.
Spanish pasodoble with a festive and lively tempo.
Hawaiian hula music with ukulele, slack-key guitar, and vocals.
Dark ambient industrial with distorted textures and eerie atmospheres.
Indian raga on santoor and tabla, exploring melodic and rhythmic patterns.
Synthesized space exploration soundtrack with cosmic atmospheres.
French musette waltz with accordion, violin, and guitar.
Blues rock with soulful guitar solos and gritty vocals.
Indonesian angklung ensemble playing a traditional piece.
Up-tempo swing jazz with a dynamic brass section.
Synthwave-inspired retrowave track with nostalgic electronic sounds.
Latin jazz mambo with vibrant percussion and horn arrangements.
Traditional Chinese pipa solo with intricate fingerpicking.
Funky electro-soul with a blend of electronic and soulful elements.
Russian ballad with emotive vocals and orchestral accompaniment.
Upbeat Caribbean merengue with lively accordion and percussion.
Ambient electronic piece with pulsating rhythms and evolving textures.
Irish folk-rock with a mix of acoustic and electric instruments.
Japanese nohkan and taiko drum duet in a traditional style.
Upbeat indie folk with acoustic guitar and harmonized vocals.
African mbira and kalimba ensemble playing a joyful tune.
Synthesized ambient sounds inspired by a mystical forest.
Italian tarantella folk dance with accordion and tambourine.
Traditional Balkan brass band playing a celebratory dance.
Synth-pop ballad with emotional vocals and electronic production.
Middle Eastern oud solo with expressive improvisation.
Blues harmonica and slide guitar duo playing a Delta blues.
Traditional Scottish bagpipe and fiddle strathspey.
Up-tempo Brazilian forró with accordion, triangle, and zabumba drum.
Ambient electronic meditation with calming tones and textures.
Japanese pop-rock with catchy guitar riffs and upbeat vocals.
Caribbean dancehall with a mix of reggae and electronic elements.
Traditional Indian bansuri flute and tabla duet.
Progressive metalcore with intricate guitar riffs and intense drumming.
Spanish flamenco fusion with modern electronic beats.
Psychedelic ambient soundscape with otherworldly textures.
Caribbean reggaeton with a blend of Latin and urban influences.
Ambient post-rock with ethereal guitars and gradual build-ups.
Traditional Irish jig with fiddle, tin whistle, and bodhran.
Upbeat ska punk with energetic horn lines and catchy vocals.
Japanese koto and shakuhachi ensemble playing a traditional piece.
Experimental noise rock with dissonant guitar and chaotic rhythms.
African drum circle with polyrhythmic patterns and communal energy.
Up-tempo country bluegrass with banjo, fiddle, and vocal harmonies.
Synthesized ambient sounds inspired by a futuristic cityscape.
Italian opera aria with powerful vocals and orchestral accompaniment.
Electronic glitch-hop with glitchy beats and intricate rhythms.
Native American powwow drumming and singing with traditional regalia.
Traditional Chinese guzheng solo with melodic improvisation.
Upbeat klezmer with clarinet, accordion, and lively dance rhythms.
Ambient space-inspired soundscape with cosmic tones.
Caribbean steel drum ensemble playing a joyful and festive tune.
Indian classical vocal alaap with intricate melodic phrases.
Ambient electronic piece with a blend of organic and synthetic sounds.
Afrobeat fusion with high-energy percussion and brass arrangements.
Upbeat indie rock with catchy guitar riffs and anthemic vocals.
Traditional Japanese shamisen and koto duet.
Brazilian bossa nova with smooth guitar and gentle percussion.
Psychedelic trance with pulsating beats and evolving synthesizers.
Traditional Irish ballad with heartfelt vocals and acoustic guitar.
Electronic hip-hop with sampled beats and futuristic soundscapes.
Middle Eastern maqam-inspired jazz fusion with oud and saxophone.
Up-tempo Scottish ceilidh dance music with accordion and fiddle.
Caribbean dancehall reggae with a rhythmic beat and danceable groove.
Synthesized ambient sounds inspired by a rainstorm in the jungle.
Spanish bolero flamenco with passionate vocals and expressive guitar.
Progressive metal instrumental with complex guitar arrangements.
Japanese enka pop-rock ballad with emotional vocals.
Ambient drone music with deep bass and atmospheric textures.
Traditional African griot storytelling with kora accompaniment.
French electro-swing with a modern twist and jazzy instrumentation.
Celtic harp and flute duet playing a whimsical melody.
Russian folk dance with accordion, balalaika, and lively rhythms.
Latin jazz salsa with a vibrant horn section and percussive elements.
Synthesized ambient sounds inspired by a journey through space.
Irish sean-nós dance music with accordion and fiddle.
Indian classical jugalbandi with sitar and tabla.
Upbeat indie pop-rock with catchy melodies and layered harmonies.
Caribbean steel drum calypso with a tropical island feel.
Synthwave-inspired retrowave track with nostalgic 80s vibes.
Traditional Chinese erhu solo with expressive phrasing.
Brazilian samba-reggae with a mix of African and Brazilian rhythms.
Ambient post-rock with atmospheric guitars and gradual build-ups.
Japanese taiko drum ensemble performing a powerful rhythmic composition.
Up-tempo bluegrass with banjo, fiddle, and vocal harmonies.
Russian balalaika and accordion polka with a lively tempo.
Funky electronic soul with groovy basslines and soulful vocals.
Caribbean zouk with a tropical feel and sensual melodies.
Synthesized cyberpunk-inspired soundtrack with futuristic elements.
Traditional Japanese noh theater music with flute and drums.
Upbeat swing jazz with a lively brass section and danceable rhythm.
Ambient soundscape with ethereal textures and contemplative tones.
Spanish paso doble with a festive and dramatic orchestral arrangement.
Jamaican dub reggae with deep bass and echoing effects.
Celtic folk-rock with a fusion of traditional and modern instruments.
Traditional Persian santur and daf duet.
Electronic trip-hop with atmospheric beats and moody melodies.
Russian ballad with emotive vocals and orchestral accompaniment.
Upbeat Caribbean merengue with lively accordion and percussion.
Ambient electronic meditation with calming tones and textures.
Japanese pop-rock with catchy guitar riffs and energetic vocals.
Caribbean dancehall with a mix of reggae and electronic elements.
Traditional Indian bansuri flute and tabla duet.
Progressive metalcore with intricate guitar riffs and intense drumming.
Spanish flamenco fusion with modern electronic beats.
Psychedelic ambient soundscape with otherworldly textures.
Caribbean reggaeton with a blend of Latin and urban influences.
Ambient post-rock with ethereal guitars and gradual build-ups.
Traditional Irish jig with fiddle, tin whistle, and bodhran.
Upbeat ska punk with energetic horn lines and catchy vocals.
Japanese koto and shakuhachi ensemble playing a traditional piece.
Experimental noise rock with dissonant guitar and chaotic rhythms.
African drum circle with polyrhythmic patterns and communal energy.
Up-tempo country bluegrass with banjo, fiddle, and vocal harmonies.
Synthesized ambient sounds inspired by a futuristic cityscape.
Italian opera aria with powerful vocals and orchestral accompaniment.
Electronic glitch-hop with glitchy beats and intricate rhythms.
Native American powwow drumming and singing with traditional regalia.
Traditional Chinese guzheng solo with melodic improvisation.
Upbeat klezmer with clarinet, accordion, and lively dance rhythms.
Ambient space-inspired soundscape with cosmic tones.
Caribbean steel drum ensemble playing a joyful and festive tune.
Indian classical vocal alaap with intricate melodic phrases.
Ambient electronic piece with a blend of organic and synthetic sounds.
Afrobeat fusion with high-energy percussion and brass arrangements.
Upbeat indie rock with catchy guitar riffs and anthemic vocals.
Traditional Japanese shamisen and koto duet.
Brazilian bossa nova with smooth guitar and gentle percussion.
Psychedelic trance with pulsating beats and evolving synthesizers.
Traditional Irish ballad with heartfelt vocals and acoustic guitar.
Electronic hip-hop with sampled beats and futuristic soundscapes.
Middle Eastern maqam-inspired jazz fusion with oud and saxophone.
Up-tempo Scottish ceilidh dance music with accordion and fiddle.
Caribbean dancehall reggae with a rhythmic beat and danceable groove.
Synthesized ambient sounds inspired by a rainstorm in the jungle.
Spanish bolero flamenco with passionate vocals and expressive guitar.
Progressive metal instrumental with complex guitar arrangements.
Japanese enka pop-rock ballad with emotional vocals.
Ambient drone music with deep bass and atmospheric textures.
Traditional African griot storytelling with kora accompaniment.
French electro-swing with a modern twist and jazzy instrumentation.
Celtic harp and flute duet playing a whimsical melody.
Russian folk dance with accordion, balalaika, and lively rhythms.
Latin jazz salsa with a vibrant horn section and percussive elements.
Synthesized ambient sounds inspired by a journey through space.
Irish sean-nós dance music with accordion and fiddle.
Indian classical jugalbandi with sitar and tabla.
Upbeat indie pop-rock with catchy melodies and layered harmonies.
Caribbean steel drum calypso with a tropical island feel.
Synthwave-inspired retrowave track with nostalgic 80s vibes.
Traditional Chinese erhu solo with expressive phrasing.
Brazilian samba-reggae with a mix of African and Brazilian rhythms.

And finally, below is a range of realistic and diverse noise samples encompassing different environments and phenomena. These can include natural sounds like wind, rain, thunder, animal calls, and ocean waves, as well as artificial sounds like machinery, traffic, crowds, and electronic devices.

Raindrops tapping on a tin roof
Leaves rustling in the wind
Distant howling of wolves
Crackling of a fireplace
Cicadas buzzing on a summer night
Water flowing in a gentle stream
Creaking of a wooden ship at sea
Rustling of grass in a meadow
Purring of a cat
Sizzling of food on a barbecue
Chirping crickets in the evening
Drip-dropping of water in a cave
Buzzing bees in a garden
Distant rumble of a waterfall
Footsteps on a gravel path
Hooting of an owl in the night
Cracking of ice in a frozen lake
Rustling of pages in a book
Bubble popping in a pond
Hooves trotting on a dirt road
Bubbling of a hot spring
Swishing of wind through tall grass
Crackling of a bonfire
Humming of electrical wires
Squeaking of a playground swing
Beeping of distant car alarms
Mechanical whirring of a windmill
Pattering of rain on a tent
Swirling of leaves in a vortex
Distant rumble of thunder
Water dripping in a cave
Chattering of squirrels in a forest
Hiss of steam escaping
Crackling of a vinyl record
Slurping of a drink through a straw
Clattering of dishes in a restaurant
Scuffling of small animals in leaves
Creaking of a rocking chair
Pinging of sonar underwater
Rattling of a distant train
Scratching of a pen on paper
Whirring of a distant helicopter
Creaking of a ship's mast
Rhythmic tapping of a dripping faucet
Gurgling of a coffee percolator
Jingling of keys in a pocket
Crackling of dry leaves underfoot
Sizzling of food in a frying pan
Beeping of a supermarket checkout
Droning of a distant lawnmower
Clanking of metal pipes
Beeping of a reversing truck
Swirling of wind through canyon walls
Creaking of a door in the wind
Pattering of rain on a tin roof
Hissing of a steaming teapot
Squeaking of a playground seesaw
Humming of a refrigerator
Whistling of the wind through a keyhole
Ticking of a grandfather clock
Snapping of a campfire
Whirring of a distant helicopter
Popping of popcorn in a microwave
Clattering of a distant construction site
Slapping of water against a boat
Whirring of a ceiling fan
Cracking of ice cubes in a glass
Rumbling of distant machinery
Beeping of a digital alarm clock
Creaking of a swing set in a breeze
Scratching of a record needle
Pulsing of a heartbeat
Snoring in a quiet room
Fluttering of a flag in the wind
Tapping of a pencil on a desk
Sizzling of a barbecue grill
Whirring of a distant wind turbine
Flapping of wings in a birdcage
Purring of a distant cat
Squeaking of a rocking chair
Ticking of a bicycle wheel
Scratching of a cat's claws on a post
Buzzing of a distant chainsaw
Rattling of a window in the wind
Crackling of footsteps on gravel
Beeping of a truck backing up
Hissing of a balloon losing air
Slurping of a soup being eaten
Clanging of metal in a junkyard
Whirring of a distant lawnmower
Beeping of a hospital heart monitor
Fluttering of pages in a breeze
Tapping of rain on a car roof
Swishing of a broom sweeping
Distant honking of geese
Sizzling of food on a hot skillet
Pattering of rain on a tent
Humming of a computer server room
Chattering of teeth in the cold
Crackling of a bonfire in the snow
Scraping of shoes on gravel
Creaking of an old wooden door
Whirring of a distant helicopter
Clattering of dishes in a busy kitchen
Distant rumble of a passing subway
Fluttering of leaves caught in a breeze
Cackling of a distant hyena
Pattering of rain on a metal roof
Crackling of footsteps on fallen leaves
Hooting of a distant foghorn
Tinkling of wind chimes in a garden
Flapping of sails on a sailing boat
Distant hum of a factory
Swishing of wind through a canyon
Creaking of a porch swing
Pinging of rain on a car roof
Hissing of a distant steam engine
Clattering of a construction jackhammer
Fluttering of a flag in the wind
Rattling of a distant shopping cart
Beeping of a truck reversing
Pulsing of a distant nightclub
Creaking of a tree branch in the wind
Crackling of a vinyl record needle
Slurping of a drink through a straw
Ticking of a distant clock tower
Whirring of a distant circular saw
Beeping of a supermarket checkout scanner
Swirling of wind through city streets
Pattering of rain on a plastic surface
Creaking of an old wooden floor
Scratching of a cat's claws on a post
Buzzing of a distant lawnmower
Rattling of a chain-link fence
Crackling of a bonfire in the rain
Whirring of a ceiling fan on low speed
Clattering of a skateboard on pavement
Fluttering of pages turning in a book
Slapping of water against a dock
Beeping of a crosswalk signal
Hissing of a deflating balloon
Swishing of wind through tree branches
Scratching of a record player needle
Tapping of rain on a window pane
Humming of a vending machine
Purring of a distant lawnmower
Squeaking of a rocking horse
Ticking of a distant clock
Cracking of footsteps on icy pavement
Beeping of a microwave finishing
Rumbling of a distant thunderstorm
Hissing of a teapot about to boil
Sizzling of food on a hot grill
Clattering of a busy coffee shop
Flapping of a tent in the wind
Pattering of rain on a car windshield
Crackling of a burning candle
Whirring of a distant chainsaw
Beeping of a car alarm in the distance
Fluttering of a butterfly's wings
Scratching of a pencil on paper
Swishing of wind through tall reeds
Tapping of fingers on a desk
Hissing of a distant steam iron
Rattling of a distant roller coaster
Crackling of footsteps on a gravel path
Buzzing of a distant beehive
Clanging of metal in a construction site
Slurping of a soup being eaten
Whirring of a distant wind turbine
Beeping of a heart rate monitor
Fluttering of a bird's wings
Pattering of rain on a tent roof
Humming of a distant generator
Creaking of a gate in the wind
Ticking of a bicycle wheel in motion
Scratching of a pen on paper
Crackling of ice cubes in a glass
Hooting of an owl in the night
Beeping of a reversing garbage truck
Swishing of wind through a canyon
Whirring of a distant washing machine
Pinging of rain on a metal surface
Fluttering of a flag in the breeze
Squeaking of a rocking chair in use
Crackling of a radio tuning between stations
Rumbling of a distant avalanche
Slapping of water against a riverbank
Hissing of a distant steam engine
Clattering of a busy restaurant kitchen
Swirling of wind through a tunnel
Pattering of rain on a car roof
Beeping of a crosswalk signal
Fluttering of pages turning in a book
Ticking of a distant clock
Cracking of footsteps on icy pavement
Beeping of a microwave finishing
Rumbling of a distant thunderstorm
Hissing of a teapot about to boil
Sizzling of food on a hot grill
Clattering of dishes in a busy cafeteria
Pinging of rain on a metal balcony railing
Distant hum of a busy urban street
Fluttering of leaves in a gentle breeze
Cackling of seagulls near the coast
Tapping of heels on a marble floor
Rattling of a distant rollercoaster ride
Hissing of a distant steam locomotive
Swishing of wind through desert sand
Scratching of a cat's claws on a scratching post
Crackling of a vinyl record playing at low volume
Beeping of a construction vehicle backing up
Pulsing of distant music from a nightclub
Sizzling of food on a hotplate
Clattering of keys on a busy typewriter
Fluttering of a flagpole rope in the wind
Pattering of rain on a car roof with occasional thunder
Crackling of footsteps on a forest trail
Swirling of wind through mountainous terrain
Hooting of an owl echoing through a canyon
Beeping of a reversing garbage truck
Whirring of a distant factory machinery
Slurping of a hot beverage being enjoyed
Ticking of a grandfather clock in a silent room
Crackling of a campfire with occasional wildlife sounds
Pinging of rain on a metal fence
Squeaking of a playground swing in use
Hissing of a distant steamboat on a river
Rumbling of a distant thunderstorm with light rain
Clattering of dishes and cutlery in a bustling restaurant
Fluttering of pages turning in a breeze near an open window
Swishing of wind through a bamboo forest
Scratching of a record needle on vinyl at a low volume
Tapping of rain on a skylight or glass roof
Beeping of a distant construction vehicle
Buzzing of bees around a flowering garden
Crackling of footsteps on a snowy path
Pattering of rain on a plastic surface with distant traffic
Whirring of a distant helicopter with intermittent rotor sounds
Rattling of leaves caught in a gust of wind
Slapping of water against the side of a boat
Hooting of a distant foghorn near the coast
Creaking of a rusty gate in the wind
Beeping of a medical equipment in a hospital room
Whirring of a distant circular saw at a construction site
Pulsing of a distant nightclub with muffled bass beats
Fluttering of leaves caught in a gentle breeze
Crackling of a bonfire with occasional popping sounds
Ticking of rain on a car roof with distant thunder
Hissing of a teapot about to boil on a gas stove
Swirling of wind through a dense forest
Scratching of a pen on paper in a quiet room
Rattling of a distant train passing over tracks
Clattering of keys on a computer keyboard
Slurping of a smoothie being enjoyed
Beeping of a distant truck backing up
Crackling of footsteps on gravel with occasional bird calls
Buzzing of a distant lawnmower in a suburban area
Pinging of rain on a metal rooftop
Whirring of a ceiling fan at medium speed
Fluttering of a flag in a light breeze
Squeaking of a rusty swing set in use
Ticking of a distant clock tower with distant traffic
Creaking of an old wooden floor with distant footsteps
Crackling of footsteps on a wooden dock near water
Beeping of a supermarket checkout scanner
Hissing of a distant steam engine with intermittent whistles
Swishing of wind through a field of tall grass
Scratching of a cat's claws on a scratching post
Pulsing of a distant nightclub with occasional laughter
Sizzling of food on a barbecue with distant traffic
Clattering of dishes in a busy restaurant kitchen
Fluttering of leaves in a gentle breeze near an open window
Pattering of rain on a car roof with distant thunder
Crackling of a campfire with occasional wildlife sounds
Pinging of rain on a metal fence
Squeaking of a playground swing in use
Hissing of a distant steamboat on a river
Rumbling of a distant thunderstorm with light rain
Clattering of dishes and cutlery in a bustling restaurant
Fluttering of pages turning in a breeze near an open window
Swishing of wind through a bamboo forest
Scratching of a record needle on vinyl at a low volume
Tapping of rain on a skylight or glass roof
Beeping of a distant construction vehicle
Buzzing of bees around a flowering garden
Crackling of footsteps on a snowy path
Pattering of rain on a plastic surface with distant traffic
Whirring of a distant helicopter with intermittent rotor sounds
Rattling of leaves caught in a gust of wind
Slapping of water against the side of a boat
Hooting of a distant foghorn near the coast
Creaking of a rusty gate in the wind
Beeping of medical equipment in a hospital room
Whirring of a distant circular saw at a construction site
Pulsing of a distant nightclub with muffled bass beats
Fluttering of leaves caught in a gentle breeze
Crackling of a bonfire with occasional popping sounds
Ticking of rain on a car roof with distant thunder
Hissing of a teapot about to boil on a gas stove
Swirling of wind through a dense urban alley
Scratching of a dog's nails on a hardwood floor
Pinging of rain on an umbrella
Creaking of a porch swing in a light breeze
Fluttering of papers in an office with distant chatter
Rumbling of distant thunder with faint city noises
Crackling of footsteps on a gravel trail with occasional bird songs
Beeping of a digital watch with distant traffic
Whirring of a distant drone in the sky
Tapping of fingers on a glass surface
Hooting of an owl in a silent forest
Sizzling of food on a hot griddle in a food market
Clattering of coins in a pocket with distant footsteps
Buzzing of mosquitoes in a humid environment
Crackling of footsteps on a forest floor with occasional rustling leaves
Fluttering of a flag in a strong gust of wind
Pattering of rain on a tent with distant animal sounds
Crackling of footsteps on a sandy beach with crashing waves
Whirring of a distant lawnmower with intermittent gusts of wind
Ticking of a distant clock with occasional distant car horns
Hissing of a distant steam engine with intermittent train whistles
Swishing of wind through a field of wheat
Scratching of a record needle on vinyl at a moderate volume
Rattling of a distant tram on tracks
Clattering of dishes and silverware in a busy restaurant
Slurping of a drink being enjoyed with distant laughter
Beeping of a microwave with distant thunder
Crackling of footsteps on a gravel path with distant traffic
Buzzing of a distant chainsaw with intermittent bird calls
Pinging of rain on a metal balcony railing
Squeaking of a playground swing in use with distant voices
Hissing of a distant steam locomotive with intermittent train bells
Fluttering of leaves in a suburban backyard with distant dog barks
Rumbling of a distant thunderstorm with light rain
Clattering of dishes and cutlery in a bustling food court
Swirling of wind through a residential neighborhood with distant footsteps
Scratching of a cat's claws on a scratching post with distant city hum
Crackling of a bonfire with occasional popping sounds
Ticking of rain on a car roof with distant thunder
Hissing of a teapot about to boil on a gas stove
Pulsing of a distant nightclub with occasional laughter
Sizzling of food on a barbecue with distant traffic
Clattering of dishes in a busy restaurant kitchen
Fluttering of pages turning in a breeze near an open window
Pattering of rain on a car roof with distant thunder
Crackling of a campfire with occasional wildlife sounds
Pinging of rain on a metal fence
Squeaking of a playground swing in use
Hissing of a distant steamboat on a river
Rumbling of a distant thunderstorm with light rain
Clattering of dishes and cutlery in a bustling restaurant
Fluttering of pages turning in a breeze near an open window
Swishing of wind through a bamboo forest
Scratching of a record needle on vinyl at a low volume
Tapping of rain on a skylight or glass roof
Beeping of a distant construction vehicle
Buzzing of bees around a flowering garden
Crackling of footsteps on a snowy path
Pattering of rain on a plastic surface with distant traffic
Whirring of a distant helicopter with intermittent rotor sounds
Rattling of leaves caught in a gust of wind
Slapping of water against the side of a boat
Hooting of a distant foghorn near the coast
Creaking of a rusty gate in the wind
Beeping of medical equipment in a hospital room
Whirring of a distant circular saw at a construction site
Pulsing of a distant nightclub with muffled bass beats
Fluttering of leaves caught in a gentle breeze
Crackling of a bonfire with occasional popping sounds
Ticking of rain on a car roof with distant thunder
Hissing of a teapot about to boil on a gas stove
Sizzling of food on a hot grill with distant traffic
Clattering of dishes in a busy cafeteria
Pinging of rain on a metal balcony railing
Distant hum of a busy urban street
Fluttering of leaves in a gentle breeze
Cackling of seagulls near the coast
Tapping of heels on a marble floor
Rattling of a distant rollercoaster ride
Hissing of a distant steam locomotive
Swishing of wind through desert sand
Scratching of a dog's nails on a hardwood floor
Crackling of a vinyl record playing at low volume
Beeping of a construction vehicle backing up
Pulsing of distant music from a nightclub
Sizzling of food on a hotplate
Clattering of keys on a busy typewriter
Fluttering of a flagpole rope in the wind
Pattering of rain on a car roof with occasional thunder
Crackling of footsteps on a forest trail
Swirling of wind through mountainous terrain
Hooting of an owl echoing through a canyon
Beeping of a reversing garbage truck
Whirring of a distant factory machinery
Slurping of a hot beverage being enjoyed
Ticking of a grandfather clock in a silent room
Crackling of a campfire with occasional wildlife sounds
Pinging of rain on a metal fence
Squeaking of a playground swing in use
Hissing of a distant steamboat on a river
Swishing of wind through tall palm trees
Scratching of a cat's claws on a carpet
Pinging of rain on a metal awning
Creaking of an old wooden rocking chair
Fluttering of papers in a quiet office
Rumbling of distant thunder with light drizzle
Crackling of footsteps on a dirt path
Beeping of a digital alarm clock with distant traffic
Whirring of a distant drone flying overhead
Tapping of rain on a car roof with distant city sounds
Hooting of an owl in a quiet woodland
Sizzling of food on a hot griddle in a food market
Clattering of coins in a pocket with distant footsteps
Buzzing of mosquitoes in a calm evening
Crackling of footsteps on a forest trail with occasional bird chirps
Fluttering of a flag in a mild breeze
Pattering of rain on a tent with distant animal calls
Crackling of footsteps on a sandy beach with gentle waves
Whirring of a distant lawnmower with intermittent gusts of wind
Ticking of a distant clock with occasional distant car horns
Hissing of a distant steam engine with intermittent train whistles
Swishing of wind through a field of sunflowers
Scratching of a record needle on vinyl at a moderate volume
Rattling of a distant tram on tracks
Clattering of dishes and silverware in a quiet cafe
Slurping of a drink being enjoyed with distant laughter
Beeping of a microwave with distant thunder
Crackling of footsteps on a gravel path with distant traffic
Buzzing of a distant chainsaw with intermittent bird songs
Pinging of rain on a metal balcony railing
Squeaking of a playground swing in use with distant voices
Hissing of a distant steam locomotive with intermittent train bells
Fluttering of leaves in a suburban backyard with distant dog barks
Rumbling of a distant thunderstorm with light rain
Clattering of dishes and cutlery in a bustling food court
Swirling of wind through a residential neighborhood with distant footsteps
Scratching of a cat's claws on a scratching post with distant city hum
Crackling of a bonfire with occasional popping sounds
Ticking of rain on a car roof with distant thunder
Hissing of a teapot about to boil on a gas stove
Pulsing of a distant nightclub with occasional laughter
Sizzling of food on a barbecue with distant traffic
Clattering of dishes in a busy restaurant kitchen
Fluttering of pages turning in a breeze near an open window
Pattering of rain on a car roof with distant thunder
Crackling of a campfire with occasional wildlife sounds
Pinging of rain on a metal fence
Squeaking of a playground swing in use
Hissing of a distant steamboat on a river
Rumbling of a distant thunderstorm with light rain
Clattering of dishes and cutlery in a bustling restaurant
Fluttering of pages turning in a breeze near an open window
Swishing of wind through a bamboo forest
Scratching of a record needle on vinyl at a low volume
Tapping of rain on a skylight or glass roof
Beeping of a distant construction vehicle
Buzzing of bees around a flowering garden
Crackling of footsteps on a snowy path
Pattering of rain on a plastic surface with distant traffic
Whirring of a distant helicopter with intermittent rotor sounds
Rattling of leaves caught in a gust of wind
Slapping of water against the side of a boat
Hooting of a distant foghorn near the coast
Creaking of a rusty gate in the wind
Beeping of medical equipment in a hospital room
Whirring of a distant circular saw at a construction site
Pulsing of a distant nightclub with muffled bass beats
Fluttering of leaves caught in a gentle breeze
Crackling of a bonfire with occasional popping sounds
Ticking of rain on a car roof with distant thunder
Hissing of a teapot about to boil on a gas stove
Sizzling of food on a hot grill with distant traffic
Clattering of dishes in a busy cafeteria
Pinging of rain on a metal balcony railing
Distant hum of a busy urban street
Fluttering of leaves in a gentle breeze
Cackling of seagulls near the coast
Tapping of heels on a marble floor
Rattling of a distant rollercoaster ride
Hissing of a distant steam locomotive
Swishing of wind through desert sand
Scratching of a dog's nails on a hardwood floor
Crackling of a vinyl record playing at low volume
Beeping of a construction vehicle backing up
Pulsing of distant music from a nightclub
Sizzling of food on a hotplate
Clattering of keys on a busy typewriter
Fluttering of a flagpole rope in the wind
Pattering of rain on a car roof with occasional thunder
Crackling of footsteps on a forest trail
Swirling of wind through mountainous terrain
Hooting of an owl echoing through a canyon
Beeping of a reversing garbage truck
Whirring of a distant factory machinery
Slurping of a hot beverage being enjoyed
Ticking of a grandfather clock in a silent room
Crackling of a campfire with occasional wildlife sounds
Pinging of rain on a metal fence
Squeaking of a playground swing in use
Hissing of a distant steamboat on a river

These are just examples, and the prompts can be further customized to generate more specific and detailed outputs from ChatGPT. The possibilities are endless when it comes to utilizing ChatGPT for creative text prompt generation.

Dataset Generation for Audio Classification using AudioCraft

To classify audio based on different categories, we will need to gather some sample WAV files of music, background noise, and speech classes. This will enable the system to differentiate between the three categories. Here, I will be using Audiocraft developed by Meta to generate from our prompts wav audio files.

AudioCraft is a powerful tool in synthetic music generation, and sound effects. It utilizes three AI models developed by Meta AI:

• MusicGen: Generates music based on textual prompts.
• AudioGen: Creates sound effects from simple descriptions.
• EnCodec: Compresses and tokenizes audio data for efficient processing.

To use AudioCraft, you will need a standalone computer with a GPU. I have the Nvidia Jetson AGX Orin Developer Kit 64GB, which is suitable for this purpose. Additionally, you can consider using Google Colab if you prefer a cloud-based solution. The Nvidia Jetson AGX Orin Developer Kit 64GB offers the following features:

• AI Performance: 275 TOPS
• GPU: 2048-core NVIDIA Ampere architecture GPU with 64 Tensor Cores
• CPU: 12-core Arm® Cortex®-A78AE v8.2 64-bit CPU
• Memory: 64GB 256-bit LPDDR5 204.8GB/s
• Storage: 64GB eMMC 5.1
• Power: 15W - 60W (with the Enclosure accessory)

To set up your Jetson AGX Orin Developer Kit, you can follow the instructions provided in the user guide. This will allow you to complete the initial software setup in a stand-alone manner. Then, follow the instructions here from the NVIDIA Jetson Generative AI Lab by Dustin Franklin in order to install AudioCraft.

After running docker container, you should see the following output:

Namespace(disable=[''], output='/tmp/autotag', packages=['audiocraft'], prefer=['local', 'registry', 'build'], quiet=False, user='dustynv', verbose=False)
-- L4T_VERSION=35.2.1  JETPACK_VERSION=5.1  CUDA_VERSION=11.8.89
-- Finding compatible container image for ['audiocraft']
dustynv/audiocraft:r35.4.1
+ docker run --runtime nvidia -it --rm --network host --volume /tmp/argus_socket:/tmp/argus_socket --volume /etc/enctune.conf:/etc/enctune.conf --volume /etc/nv_tegra_release:/etc/nv_tegra_release --volume /tmp/nv_jetson_model:/tmp/nv_jetson_model --volume /mnt/storage/Projects/jetson-containers/data:/data --device /dev/snd --device /dev/bus/usb dustynv/audiocraft:r35.4.1
allow 10 sec for JupyterLab to start @ http://192.168.0.104:8888 (password nvidia)
JupterLab logging location:  /var/log/jupyter.log  (inside the container)

JupyterLab will be launched within the container and accessible at http://192.168.0.104:8888via browser.

Open the interface in your browser and type the password nvidia. In a terminal within JupyterLab, update the package list and install ffmpeg:

sudo apt-get update
sudo apt-get install ffmpeg

The below python script utilizes the audiocraft library to generate audio samples based on textual descriptions using the facebook/audiogen-medium(a 1.5B model for text to sound) model. The prompt texts are read from the descriptions.txt file, and the generated audio samples will be saved as WAV files in the 'output' folder. This audio generation script has been successfully tested on the NVIDIA Jetson AGX Orin 64GB Developer Kit.

import torchaudio
from audiocraft.models import AudioGen
from audiocraft.data.audio import audio_write
import os
import torch
import gc

def purge():
    torch.cuda.empty_cache()
    torch.cuda.ipc_collect()
    gc.collect()

# Load descriptions from a text file
descriptions_file = 'descriptions.txt'  # Update with your file name
with open(descriptions_file, 'r') as file:
    descriptions = file.read().splitlines()

# Initialize the AudioGen model
model = AudioGen.get_pretrained('facebook/audiogen-medium')
model.set_generation_params(duration=2)  # generate 2 seconds.

# Create an output folder if it doesn't exist
output_folder = 'output'
os.makedirs(output_folder, exist_ok=True)

# Generate and save audio samples in chunks of 50 descriptions
chunk_size = 50
num_chunks = (len(descriptions) + chunk_size - 1) // chunk_size

for chunk_idx in range(num_chunks):
    start_idx = chunk_idx * chunk_size
    end_idx = (chunk_idx + 1) * chunk_size
    current_descriptions = descriptions[start_idx:end_idx]

    # Generate audio samples based on descriptions
    wav = model.generate(current_descriptions, progress=True)

    # Save generated audio samples to the output folder
    for idx, one_wav in enumerate(wav):
        # Save each sample as a WAV file in the output folder
        output_file_path = os.path.join(output_folder, f'{start_idx + idx}')
        audio_write(output_file_path, one_wav.cpu(), model.sample_rate, strategy="loudness", loudness_compressor=True)

    # Purge memory after processing each chunk
    purge()

print(f'Audio samples saved in the "{output_folder}" folder.')

If you encounter out-of-memory errors, consider reducing the chunk_size to decrease the amount of GPU memory required in each iteration. The purge function in the provided code is responsible for freeing up GPU resources.

MusicGen comes with four pretrained models:

• facebook/musicgen-small: A 300M model, for text to music generation only.
• facebook/musicgen-medium: A 1.5B model, for text to music generation only.
• facebook/musicgen-melody: A 1.5B model, for text to music and text+melody to music.
• facebook/musicgen-large: A 3.3B model, for text to music generation only.

Here is the code snippet for music generation"

import torchaudio
from audiocraft.models import MusicGen
from audiocraft.data.audio import audio_write
import os
import torch
import gc

def purge():
    torch.cuda.empty_cache()
    torch.cuda.ipc_collect()
    gc.collect()

# Load descriptions from a text file
descriptions_file = 'descriptions.txt'  # Update with your file name
with open(descriptions_file, 'r') as file:
    descriptions = file.read().splitlines()

# Initialize the AudioGen model
model = MusicGen.get_pretrained('facebook/musicgen-small')
model.set_generation_params(duration=2)  # generate 2 seconds of music.
#wav = model.generate_unconditional(4)  

# Create an output folder if it doesn't exist
output_folder = 'output'
os.makedirs(output_folder, exist_ok=True)

# Generate and save audio samples in chunks of 50 descriptions
chunk_size = 50
num_chunks = (len(descriptions) + chunk_size - 1) // chunk_size

for chunk_idx in range(num_chunks):
    start_idx = chunk_idx * chunk_size
    end_idx = (chunk_idx + 1) * chunk_size
    current_descriptions = descriptions[start_idx:end_idx]

    # Generate audio samples based on descriptions
    wav =model.generate(current_descriptions, progress=True)

    # Save generated audio samples to the output folder
    for idx, one_wav in enumerate(wav):
        # Save each sample as a WAV file in the output folder
        output_file_path = os.path.join(output_folder, f'{start_idx + idx}')
        audio_write(output_file_path, one_wav.cpu(), model.sample_rate, strategy="loudness", loudness_compressor=True)

    # Purge memory after processing each chunk
    purge()

print(f'Audio samples saved in the "{output_folder}" folder.')

You can use the Jupyter notebook that has been prepared for you. The notebook includes all the necessary steps and code to generate audio files using Audiocraft.

However, keep in mind that the model is quite large, so it may take some time to generate, especially if you don't have a GPU. Once the wav audio samples have been collected, they can be fed into the neural network to initiate the training process.

Model Training using Edge Impulse platform

Now we can simply upload these wav files to the Edge Impulse Studioin order to train our model. Edge Impulseoffers an end-to-end solution for deploying models on TinyML devices. Here I assume that you are familiar with Edge Impulse

Create a new project. Select the Data acquisition tab and select upload data. Make sure that under the label option, you have mentioned the Label name. Upload your wav files.

In total, I have gathered 1 hour 52 minutes and 30 seconds worth of data that can be categorized into three distinct classes.

• Speech class
• Background noise class
• Music

Also, you can visualize complex datasets in Edge Impulse. The data explorer is a visual tool to explore your dataset, find outliers or mislabeled data, and to help label unlabeled data. I highly recommend to use this great feature.

Once you have set up all of your classes and are happy with your datasets, it is time to train the model. Navigate to Create Impulse on the left navigation menu.

I settled with the following parameters:

• Window Size = 2000 ms
• Window Increase = 300 ms
• Audio(MFE)
• Classification

Click Save Impulse.

Then navigate to MFE. In this step we will generate the features from the input data. Features are unique properties that will be used by the classification algorithm to classify audio.

Once you're done playing with the feature explorer, click on the Classifier item in the left navigation menu.

For this project, the number of training cycles was set to 50 and the learning rate was set to 0.0005.

Train the model by pressing the Start training button. This process might take around 10-20 minutes, depends on your data set sizes.

After training is complete, Edge Impulse Studio will display the performance of the model, a confusion matrix, feature explorer, and on-device performance details.

The accuracy of 90.7% is pretty good for the training set.

Then, go to the Deployment tab of the Edge Impulse. Click on the deployment options. In my case it is Arduino library.

And at the bottom of the section, press the button Build. A zip file will be automatically downloaded to your computer.

Connect the Arduino GIGA R1 WiFi device to your computer using a USB cable.

Run Inferencing Sketch on the Arduino GIGA R1 WiFi

Launch Arduino IDE. Go to Sketch > Include Library > Add.ZIP Library. Select the.zip file you just downloaded from Edge Impulse and click Open.

In the below code, I added the PDM.setGain(5). This will make the microphone more sensitive to quiet sounds. Also, I replaced the sampleRate(16000) with sampleRate(20000).

/* Edge Impulse ingestion SDK
 * Copyright (c) 2022 EdgeImpulse Inc.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 * http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 *
 */

// If your target is limited in memory remove this macro to save 10K RAM
#define EIDSP_QUANTIZE_FILTERBANK   0


/*
 ** NOTE: If you run into TFLite arena allocation issue.
 **
 ** This may be due to may dynamic memory fragmentation.
 ** Try defining "-DEI_CLASSIFIER_ALLOCATION_STATIC" in boards.local.txt (create
 ** if it doesn't exist) and copy this file to
 ** `<ARDUINO_CORE_INSTALL_PATH>/arduino/hardware/<mbed_core>/<core_version>/`.
 **
 ** See
 ** (https://support.arduino.cc/hc/en-us/articles/360012076960-Where-are-the-installed-cores-located-)
 ** to find where Arduino installs cores on your machine.
 **
 ** If the problem persists then there's not enough memory for this model and application.
 */

/* Includes ---------------------------------------------------------------- */
#include <PDM.h>
#include <Audio_classification_v1_inferencing.h>
#include "Arduino_GigaDisplay_GFX.h"

#define screen_size_x 480
#define screen_size_y 800

GigaDisplay_GFX display; // create the object
#define BLACK 0x0000

/** Audio buffers, pointers and selectors */
typedef struct {
    int16_t *buffer;
    uint8_t buf_ready;
    uint32_t buf_count;
    uint32_t n_samples;
} inference_t;

static inference_t inference;
static signed short sampleBuffer[2048];
static bool debug_nn = false; // Set this to true to see e.g. features generated from the raw signal
static const int frequency = 20000; 
/**
 * @brief      Arduino setup function
 */
void setup()
{
    // put your setup code here, to run once:
    Serial.begin(115200);
    // comment out the below line to cancel the wait for USB connection (needed for native USB)
    while (!Serial);
    Serial.println("Edge Impulse Inferencing Demo");
    display.begin();
    display.fillScreen(BLACK);
    display.setTextSize(2);
    display.setRotation(1);
    delay(1000);
    // summary of inferencing settings (from model_metadata.h)
    ei_printf("Inferencing settings:\n");
    ei_printf("\tInterval: %.2f ms.\n", (float)EI_CLASSIFIER_INTERVAL_MS);
    ei_printf("\tFrame size: %d\n", EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE);
    ei_printf("\tSample length: %d ms.\n", EI_CLASSIFIER_RAW_SAMPLE_COUNT / 16);
    ei_printf("\tNo. of classes: %d\n", sizeof(ei_classifier_inferencing_categories) / sizeof(ei_classifier_inferencing_categories[0]));

    if (microphone_inference_start(EI_CLASSIFIER_RAW_SAMPLE_COUNT) == false) {
        ei_printf("ERR: Could not allocate audio buffer (size %d), this could be due to the window length of your model\r\n", EI_CLASSIFIER_RAW_SAMPLE_COUNT);
        return;
    }
}

/**
 * @brief      Arduino main function. Runs the inferencing loop.
 */
void loop()
{
    ei_printf("Starting inferencing in 2 seconds...\n");

    delay(2000);

    ei_printf("Recording...\n");

    bool m = microphone_inference_record();
    if (!m) {
        ei_printf("ERR: Failed to record audio...\n");
        return;
    }

    ei_printf("Recording done\n");

    signal_t signal;
    signal.total_length = EI_CLASSIFIER_RAW_SAMPLE_COUNT;
    signal.get_data = &microphone_audio_signal_get_data;
    ei_impulse_result_t result = { 0 };

    EI_IMPULSE_ERROR r = run_classifier(&signal, &result, debug_nn);
    if (r != EI_IMPULSE_OK) {
        ei_printf("ERR: Failed to run classifier (%d)\n", r);
        return;
    }

    // print the predictions
    ei_printf("Predictions ");
    ei_printf("(DSP: %d ms., Classification: %d ms., Anomaly: %d ms.)",
        result.timing.dsp, result.timing.classification, result.timing.anomaly);
    ei_printf(": \n");
    for (size_t ix = 0; ix < EI_CLASSIFIER_LABEL_COUNT; ix++) {
        ei_printf("    %s: %.5f\n", result.classification[ix].label, result.classification[ix].value);
    }

    const char *resultLabel = getMax(result.classification);
    display.fillScreen(BLACK);
    Serial.println(resultLabel);
    //display.println(resultLabel);
    int artLength = 9; // Change this value based on the length of your ASCII art
    int centerX = (screen_size_x - (artLength * 10)) / 2; // 10 is the approximate width of each character in the font

   if (strcmp(resultLabel, "speech") == 0) {
        display.fillScreen(BLACK);
        // Display ASCII art for speech
        display.setCursor(100, 150);
        display.println("\n");
        // Print each line of the ASCII art
        display.setCursor(100, 160);
        display.println(" #####  ######  ####### #######  #####  #     # ");
        display.setCursor(100, 180);
        display.println("#     # #     # #       #       #     # #     # ");
        display.setCursor(100, 200);
        display.println("#       #     # #       #       #       #     # ");
        display.setCursor(100, 220);
        display.println(" #####  ######  #####   #####   #       ####### ");
        display.setCursor(100, 240);
        display.println("      # #       #       #       #       #     # ");
        display.setCursor(100, 260);
        display.println("#     # #       #       #       #     # #     # ");
        display.setCursor(100, 280);
        display.println(" #####  #       ####### #######  #####  #     # \n");
    } else if (strcmp(resultLabel, "music") == 0) {
      display.fillScreen(BLACK);
      // Display ASCII art for music
      display.setCursor(200, 150);
      display.println("\n");
      // Print each line of the ASCII art
      display.setCursor(200, 160);
      display.println("#     # #     #  #####  ###  #####  ");
      display.setCursor(200, 180);
      display.println("##   ## #     # #     #  #  #     # ");
      display.setCursor(200, 200);
      display.println("# # # # #     # #        #  #       ");
      display.setCursor(200, 220);
      display.println("#  #  # #     #  #####   #  #       ");
      display.setCursor(200, 240);
      display.println("#     # #     #       #  #  #       ");
      display.setCursor(200, 260);
      display.println("#     # #     # #     #  #  #     # ");
      display.setCursor(200, 280);
      display.println("#     #  #####   #####  ###  #####  \n");
    } else if (strcmp(resultLabel, "noise") == 0) {
        display.fillScreen(BLACK);
      // Display ASCII art for noise
      display.setCursor(200, 150);
      display.println("\n");
      // Print each line of the ASCII art
      display.setCursor(200, 160);
      display.println("#     # ####### ###  #####  ####### ");
      display.setCursor(200, 180);
      display.println("##    # #     #  #  #     # #       ");
      display.setCursor(200, 200);
      display.println("# #   # #     #  #  #       #       ");
      display.setCursor(200, 220);
      display.println("#  #  # #     #  #   #####  #####   ");
      display.setCursor(200, 240);
      display.println("#   # # #     #  #        # #       ");
      display.setCursor(200, 260);
      display.println("#    ## #     #  #  #     # #       ");
      display.setCursor(200, 280);
      display.println("#     # ####### ###  #####  ####### \n");
        }


#if EI_CLASSIFIER_HAS_ANOMALY == 1
    ei_printf("    anomaly score: %.3f\n", result.anomaly);
#endif
}

const char* getMax(ei_impulse_result_classification_t *classifications)
{
    uint8_t maxLabelIndex = 0;
    for (size_t i = 1; i < EI_CLASSIFIER_LABEL_COUNT; i++) {
        if (classifications[i].value > classifications[maxLabelIndex].value) {
            maxLabelIndex = i;
        }
    }

    return classifications[maxLabelIndex].label;
}

/**
 * @brief      PDM buffer full callback
 *             Get data and call audio thread callback
 */
static void pdm_data_ready_inference_callback(void)
{
    int bytesAvailable = PDM.available();

    // read into the sample buffer
    int bytesRead = PDM.read((char *)&sampleBuffer[0], bytesAvailable);

    if (inference.buf_ready == 0) {
        for(int i = 0; i < bytesRead>>1; i++) {
            inference.buffer[inference.buf_count++] = sampleBuffer[i];

            if(inference.buf_count >= inference.n_samples) {
                inference.buf_count = 0;
                inference.buf_ready = 1;
                break;
            }
        }
    }
}

/**
 * @brief      Init inferencing struct and setup/start PDM
 *
 * @param[in]  n_samples  The n samples
 *
 * @return     { description_of_the_return_value }
 */
static bool microphone_inference_start(uint32_t n_samples)
{
    inference.buffer = (int16_t *)malloc(n_samples * sizeof(int16_t));

    if(inference.buffer == NULL) {
        return false;
    }

    inference.buf_count  = 0;
    inference.n_samples  = n_samples;
    inference.buf_ready  = 0;

    // configure the data receive callback
    PDM.onReceive(&pdm_data_ready_inference_callback);

    PDM.setBufferSize(512);

    // initialize PDM with:
    // - one channel (mono mode)
    // - a 20 kHz sample rate
    if (!PDM.begin(1, frequency)) {
        ei_printf("Failed to start PDM!");
        microphone_inference_end();

        return false;
    }

    PDM.setGain(5)

    return true;
}

/**
 * @brief      Wait on new data
 *
 * @return     True when finished
 */
static bool microphone_inference_record(void)
{
    inference.buf_ready = 0;
    inference.buf_count = 0;

    while(inference.buf_ready == 0) {
        delay(10);
    }

    return true;
}

/**
 * Get raw audio signal data
 */
static int microphone_audio_signal_get_data(size_t offset, size_t length, float *out_ptr)
{
    numpy::int16_to_float(&inference.buffer[offset], out_ptr, length);

    return 0;
}

/**
 * @brief      Stop PDM and release buffers
 */
static void microphone_inference_end(void)
{
    PDM.end();
    free(inference.buffer);
}

#if !defined(EI_CLASSIFIER_SENSOR) || EI_CLASSIFIER_SENSOR != EI_CLASSIFIER_SENSOR_MICROPHONE
#error "Invalid model for current sensor."
#endif

I used the ASCII art generator from here to generate ASCII art for our classes.

Open the Arduino IDE Serial Monitor. Set the baud rate to 115200. If everything goes correctly, you should see the following:

02:38:59.938 -> Recording...
02:39:01.551 -> Recording done
02:39:02.130 -> Predictions (DSP: 26 ms., Classification: 530 ms., Anomaly: 0 ms.):
02:39:02.130 ->     music: 0.10147
02:39:02.130 ->     noise: 0.63983
02:39:02.130 ->     speech: 0.25870
02:39:02.130 -> noise
02:39:02.161 -> Starting inferencing in 2 seconds...
02:39:04.147 -> Recording...
02:39:05.759 -> Recording done
02:39:06.323 -> Predictions (DSP: 26 ms., Classification: 530 ms., Anomaly: 0 ms.):
02:39:06.323 ->     music: 0.05066
02:39:06.323 ->     noise: 0.84657
02:39:06.323 ->     speech: 0.10276

Here is my demo setup with the Arduino Giga R1.

And, finally here is a demonstration video of what the final result looks like.

This project demonstrated the potential of generative AI, synthetic dataset and edge computing in developing efficient and practical audio classification systems for various applications. By utilizing synthetic data generated through AI, we can overcome limitations of real-world data acquisition, like cost and diversity, and train our model more effectively.

This audio classifier is publicly accessible on Edge Impulse as a project. All the code referenced in this story is available in my github repo. Thank you for reading!

References

• NVIDIA Jetson Generative AI Lab
• Reading Microphone Data on Nano RP2040 Connect
• Audio classification using transfer learning approach
• Audio Classification using Librosa and Pytorch
• Nano 33 BLE Sense Rev2 Cheat Sheet
• Generative Sound AI: How To Get Started With Text-to-Music and Text-to-Sound with AudioCraft
• Generative Text to Audio with Audiocraft 2: Generating audio from the command line.
• Text to ASCII Art Generator
• Data explorer - Edge Impulse