Golang read file, update in place, and overwrite safely

People who want golang read file or golang read entire file usually reach for os.ReadFile (the modern golang os readfile API). Golang overwrite file is then os.WriteFile or Truncate. Update one field of big file golang searches map to the same engineering problem: you must not tear a large text or log file if the process stops halfway—stream to a temporary file and os.Rename when done, or load fully only when memory fits. Golang open file and go open file details live in os.Open and os.OpenFile flags. This page contrasts small-file read–modify–write, safe big-file replacement, append patterns, and WriteAt limits. For more read patterns, see ways to read a file in Go.

Checked with Go 1.24 on 64-bit Linux (Fedora and Ubuntu family).

golang read file and golang read entire file

os.ReadFile reads the whole path into a []byte and closes the handle—best when the file fits in memory. Pair it with os.WriteFile to replace content atomically at the syscall level (new inode, rename semantics depend on OS).

package main

import (
	"bytes"
	"fmt"
	"os"
	"path/filepath"
)

func main() {
	dir, err := os.MkdirTemp("", "rw")
	if err != nil {
		panic(err)
	}
	defer os.RemoveAll(dir)
	p := filepath.Join(dir, "note.txt")
	if err := os.WriteFile(p, []byte("hello world\n"), 0644); err != nil {
		panic(err)
	}
	data, err := os.ReadFile(p)
	if err != nil {
		panic(err)
	}
	data = bytes.ReplaceAll(data, []byte("world"), []byte("GoLinuxCloud"))
	if err := os.WriteFile(p, data, 0644); err != nil {
		panic(err)
	}
	out, _ := os.ReadFile(p)
	fmt.Print(string(out))
}

You should see hello GoLinuxCloud on the printed line.

For streams that do not fit in RAM, open with os.Open, wrap bufio.NewScanner or bufio.Reader, and copy with io.Copy to a temp path instead of os.ReadFile.

golang open file: os.Open and os.OpenFile flags

os.Open is OpenFile(name, os.O_RDONLY, 0). os.OpenFile combines bitwise flags:

• os.O_RDONLY, os.O_WRONLY, or os.O_RDWR (exactly one direction base).
• os.O_CREATE — create if missing (use with a perm such as 0644).
• os.O_TRUNC — truncate on successful open (typical golang overwrite file open when paired with O_WRONLY or O_RDWR).
• os.O_APPEND — each Write goes to the end; do not mix with WriteAt appends—WriteAt returns an error when O_APPEND is set.

To read existing lines then append, open with os.O_RDWR|os.O_APPEND so reads start at the current offset while writes still land at the end.

package main

import (
	"bufio"
	"fmt"
	"os"
	"path/filepath"
)

func main() {
	dir, _ := os.MkdirTemp("", "ap")
	defer os.RemoveAll(dir)
	p := filepath.Join(dir, "app.log")
	_ = os.WriteFile(p, []byte("seed line\n"), 0644)

	f, err := os.OpenFile(p, os.O_RDWR|os.O_APPEND, 0644)
	if err != nil {
		panic(err)
	}
	defer f.Close()

	fmt.Println("before append:")
	sc := bufio.NewScanner(f)
	for sc.Scan() {
		fmt.Println(sc.Text())
	}
	if err := sc.Err(); err != nil {
		panic(err)
	}
	if _, err := f.WriteString("new tail line\n"); err != nil {
		panic(err)
	}
	b, _ := os.ReadFile(p)
	fmt.Println("after append:")
	fmt.Print(string(b))
}

You should see the seed line, then a file body that ends with new tail line.

golang overwrite file and WriteAt for fixed offsets

os.WriteFile replaces the entire file (it opens with truncate semantics). WriteAt patches bytes at an absolute offset—useful for fixed-width records, not for arbitrary UTF-8 text where character width varies.

package main

import (
	"fmt"
	"os"
	"path/filepath"
)

func main() {
	dir, _ := os.MkdirTemp("", "wa")
	defer os.RemoveAll(dir)
	p := filepath.Join(dir, "fix.bin")
	_ = os.WriteFile(p, []byte("Gello GoLinuxCloud member!\n"), 0644)

	f, err := os.OpenFile(p, os.O_RDWR, 0644)
	if err != nil {
		panic(err)
	}
	defer f.Close()
	if _, err := f.WriteAt([]byte("These"), 0); err != nil {
		panic(err)
	}
	b, _ := os.ReadFile(p)
	fmt.Print(string(b))
}

You should see the greeting start with These instead of Gello.

update one field of big file golang (stream, then rename)

Opening the same path twice and interleaving reads and writes on two *os.File values is easy to get wrong (offset races, partial writes). A robust pattern for read file golang transforms while streaming: read from the original path, write to a sibling temp file, Flush, close both, then os.Rename the temp over the source (same parent directory for atomic replace on Unix).

package main

import (
	"bufio"
	"fmt"
	"io"
	"os"
	"path/filepath"
	"strings"
)

func main() {
	dir, _ := os.MkdirTemp("", "big")
	defer os.RemoveAll(dir)
	src := filepath.Join(dir, "story.txt")
	_ = os.WriteFile(src, []byte("hello Midas world\n"), 0644)
	tmp := filepath.Join(dir, "story.txt.tmp")

	in, _ := os.Open(src)
	out, _ := os.Create(tmp)
	r := bufio.NewReader(in)
	w := bufio.NewWriter(out)
	for {
		line, err := r.ReadString('\n')
		if len(line) > 0 {
			line = strings.ReplaceAll(line, "Midas", "GoLinuxCloud")
			if _, werr := w.WriteString(line); werr != nil {
				panic(werr)
			}
		}
		if err == io.EOF {
			break
		}
		if err != nil {
			panic(err)
		}
	}
	if err := w.Flush(); err != nil {
		panic(err)
	}
	in.Close()
	out.Close()
	if err := os.Rename(tmp, src); err != nil {
		panic(err)
	}
	b, _ := os.ReadFile(src)
	fmt.Print(string(b))
}

You should see hello GoLinuxCloud world in the printed output.

Summary

Golang read entire file workloads fit os.ReadFile; go read file flows that must stay bounded use os.Open plus bufio. Golang open file choices boil down to os.OpenFile flags: add O_TRUNC to overwrite, O_APPEND to append, O_RDWR when you need both sides. WriteAt only suits fixed-byte layouts. For update one field of big file golang style edits, stream through a temp file and Rename instead of two handles on one path. That is the practical core of golang read update same file work without corrupting live data.

References

• Package os
• Package bufio
• FileMode (permissions)