How to Read File Word By Word in Golang?

File reading is such an important aspect of a programming language. We need to perform certain operations programmatically, using automation in file reading/writing allows us to create certain programs/projects which might have looked impossible otherwise.

File Input

To work with files, we have to input an existing file as we are reading from a file. We can surely take input by using the name of the file, but here we are taking it simple and reading from a pre-defined file. We can open a file for reading by using the os package/module in Golang. The Open function in the os package lets us give the filename as a parameter to it. 

// Golang program to open file
package main
import (
    "os"
    "log"
)
func main() {
    file, err := os.Open("sample.txt")
    if err != nil {
        log.Fatal(err)
    }
}

The Open function either returns a pointer reference to the file or it returns an error in case the file is not found, can't be open for reading, etc. So, we store either value whichever s applicable. If we have an error we simply want to log the error as Fatal and exit out of the program via the log package in Go.

Output:

&{0x11934420} 

Using bufio to Scan File

To actually get the contents from the file pointer, we need to use the Scanner functions which have functionalities to read the contents of the file line by line, word by word, character by character(rune), and so on. 

Before that we need a Reader to read from a stream, we use the NewScanner function which takes in a parameter as a stream. As we want to read from a file, we will parse the file pointer as a parameter. This will create a Scanner that can read from the provided file. 

After we have our Scanner initialized we can provide a splitter(Split function) or kind of format that will read according to a specific pattern. As we want to read word by word we will parse the bufio.ScanWords function. This function acts as a splitter of the given stream i,e, it will split the entire file into the provided split function in our case into words. 

// Golang program to scan files
package main

import (
    "fmt"
    "os"
    "log"
    "bufio"
)

func main() {
    file, err := os.Open("sample.txt")
    if err != nil {
        log.Fatal(err)
    }else{
        fmt.Println(file)
    }
    Scanner := bufio.NewScanner(file)
    Scanner.Split(bufio.ScanWords)
    fmt.Println(Scanner)
}

Output:

&{0x11864420}
&{0x118061a8 0x48e410 65536 [] [] 0 0 <nil> 0 false false}

So, using bufio Scanner and its relevant functions like NewScanner, Split, and ScanWords we are able to set up the file scanner.  We can now read from the scanner each word in the file. 

Scanning Words

To iterate over the file we created the scanner and all information about the file is stored in Scanner. We can access the contents of the file with another function called Scan. This function advances the Scanner to the next split token. So it will return true if there is content/tokens in the Scanner else return false if we reach the end of the file, but when it is EOF, the error message is nil and hence we exit without any valid error. Inside the loop, we can use the function Text. This function returns the bytes of the recently allocated Scanner in our case it is the file that we provided. So it basically gets the text for the current token which the Scan function is iterating on. Thus we can print the value returned by the Text function.

// Go program to scan the words
package main

import (
    "fmt"
    "os"
    "bufio"
    "log"
)

func main() {

    file, err := os.Open("sample.txt")
    if err != nil {
        log.Fatal(err)
    }

    Scanner := bufio.NewScanner(file)
    Scanner.Split(bufio.ScanWords)

    for Scanner.Scan() {
        fmt.Println(Scanner.Text())
    }
    if err := Scanner.Err(); err != nil {
        log.Fatal(err)
    }
}

As we can see we are able to iterate over the file word by word. The error checking using the err variable, we first store the error in the variable err, and then if and only if the err is not nil then that is considered as an error and we have an exit out of the program after logging the error. The semi-colon(;) is used to chain statements in go lang before we evaluate a condition. 

Append to a String Slice

We have simply printed the file contents word by word which might not be a good use case of the package. We can even store the context in the form of a slice in this case a slice of string where each element is a word. 

// Go program to append to a String slice
package main

import (
    "fmt"
    "os"
    "bufio"
    "log"
)

func main() {

    file, err := os.Open("sample.txt")

    var words []string

    if err != nil {
        log.Fatal(err)
    }

    Scanner := bufio.NewScanner(file)
    Scanner.Split(bufio.ScanWords)

    for Scanner.Scan() {
        words = append(words, Scanner.Text())
    }

    fmt.Println(words)
    for _, word := range words {
        fmt.Println(word)
    }
    if err := Scanner.Err(); err != nil {
        log.Fatal(err)
    }

}

The things that changed in this code are only the way we append and iterate over the created string slice. We first initialize a string slice by the syntax name []string, after the Scanner is initialized and we are iterating with the Scan function, we simply append the value returned from the Scanner.Text() function into the string slice. We can print the slice or iterate over it and do the required processing on it. So this is how we can append the contents of a file word by word to a string slice in Golang.