AWK Command in Linux: Complete Guide with Practical Examples

AWK is one of the most powerful and lightweight text-processing tools available on Linux and Unix systems. If you work with log files, CSV data, system reports, or structured text, AWK can replace complex scripts with simple one‑liners.

This guide is written for beginners, but it scales all the way to real-world automation.

In this tutorial, you will learn:

• What AWK is and how it works internally
• AWK syntax, patterns, and actions
• Printing and reordering columns
• BEGIN and END blocks
• Filtering rows using conditions
• Writing and executing AWK scripts
• Practical Linux examples you can reuse daily

What is AWK in Linux or Unix?

AWK is an interpreted programming language designed for pattern scanning and text processing. It reads input line by line, splits each line into fields, and applies actions when patterns match.

Unlike traditional programming languages, AWK is data-driven — you describe what to look for and what to do when you find it.

AWK is commonly used for:

• Parsing log files
• Extracting columns from text
• Generating reports
• Filtering structured data

Types of AWK implementations

Most modern Linux distributions ship with GNU AWK (gawk), which is backward compatible with classic AWK.

• AWK – Original implementation
• NAWK – New AWK
• GAWK – GNU AWK (most common)

On Linux, running awk usually executes gawk.

Common AWK variables (quick reference)

These built-in variables are used frequently in real-world AWK scripts:

Bookmark this table — it answers most AWK questions instantly.

AWK vs sed vs cut (When to Use What)

Understanding when to use the right text-processing tool avoids overengineering simple tasks and keeps commands readable and efficient.

Rule of thumb:

• Need logic, math, or conditions → awk
• Need simple search or replace → sed
• Need fast column extraction → cut
• Need only pattern matching → grep

AWK vs grep (Pattern-Only Use Cases)

Use grep when you only need to match lines based on a pattern.

Example using grep:

grep "ERROR" app.log

Use AWK when you need pattern matching with field-level control.

Equivalent AWK command:

awk '/ERROR/ { print }' app.log

Match a pattern in a specific field (not possible with plain grep):

awk '$3 ~ /ERROR/ { print }' app.log

Combine pattern matching with conditions:

awk '/ERROR/ && $5 > 100 { print }' app.log

AWK Basics & Syntax

AWK command syntax (pattern and action)

An AWK program consists of patterns and actions:

awk 'pattern { action }' file

Both pattern and action are optional:

• Pattern only → print matching lines
• Action only → run action on all lines

AWK syntax: pattern-only example

The awk command syntax with a pattern only is as follows:

awk '/mail/' /etc/passwd

This prints all lines containing the word mail.

mail:x:8:12:mail:/var/spool/mail:/sbin/nologin

Sample input file used in all examples

All examples below use the same input file so you can focus on learning AWK, not guessing the data.

userdata.txt

id   Name    Age  username
1    Deepak  31   deepak
2    Rahul   32   rahul
3    Amit    33   amit
4    Sumit   33   sumit

Running and Writing AWK Programs

Running AWK from a source file

Instead of writing long one-liners, you can store AWK programs in a file and execute them using the -f option. This improves readability and maintainability.

Create an AWK source file (script.awk):

{
    print $1, $2
}

Run the AWK program on an input file:

awk -f script.awk data.txt

Use multiple input files:

awk -f script.awk file1.txt file2.txt

Writing Executable AWK Scripts

AWK scripts can be made executable like shell scripts by adding a shebang line.

Create an executable AWK script:

#!/usr/bin/awk -f
{
    print $1, $2
}

Make the script executable:

chmod +x script.awk

Run the script directly:

./script.awk data.txt

You can also pass command-line arguments:

./script.awk file1.txt file2.txt

Comments in AWK

Comments are used to document AWK scripts and improve readability. Any text after # is ignored during execution.

# This is a comment
BEGIN { print "Hello" }

Comments are especially helpful in long or shared AWK scripts.

Basic Printing & Line Processing

Print every line (basic AWK example)

This is the most basic use of AWK. It prints every input line exactly as it is, without modifying the content.

awk '{ print }' userdata.txt

Sample output:

id   Name    Age  username
1    Deepak  31   deepak
2    Rahul   32   rahul
3    Amit    33   amit
4    Sumit   33   sumit

Print a specific line number

You can selectively print a particular line by matching NR with the desired line number.

awk 'NR == 2' userdata.txt

Sample output:

1    Deepak  31   deepak

Print even-numbered lines

Using the modulo operator with NR, you can filter even-numbered lines from the file.

awk 'NR % 2 == 0' userdata.txt

Sample output:

1    Deepak  31   deepak
3    Amit    33   amit

Print odd-numbered lines

This command prints only odd-numbered lines using a modulo operation on NR.

awk 'NR % 2 == 1' userdata.txt

Sample output:

id   Name    Age  username
2    Rahul   32   rahul
4    Sumit   33   sumit

Print a range of lines using NR

The range operator allows printing a continuous block of lines based on line numbers.

awk 'NR == 2, NR == 4' userdata.txt

Sample output:

1    Deepak  31   deepak
2    Rahul   32   rahul
3    Amit    33   amit

Print a range of lines using patterns

Instead of line numbers, you can define a start and end pattern to print a block of lines.

awk '/Deepak/,/Amit/' userdata.txt

Sample output:

1    Deepak  31   deepak
2    Rahul   32   rahul
3    Amit    33   amit

BEGIN and END Blocks

BEGIN and END blocks in AWK

The BEGIN block runs before any input is processed, and the END block runs after all input lines are processed. These blocks are commonly used for headers, footers, and initialization.

awk 'BEGIN { print "== Start ==" }
     { print }
     END { print "== End ==" }' userdata.txt

Sample output:

== Start ==
id   Name    Age  username
1    Deepak  31   deepak
2    Rahul   32   rahul
3    Amit    33   amit
4    Sumit   33   sumit
== End ==

Print header using BEGIN block

The BEGIN block is executed before input processing starts, making it ideal for printing headers.

awk 'BEGIN { print "ID Name Age User" } { print }' userdata.txt

Sample output:

ID Name Age User
id   Name    Age  username
1    Deepak  31   deepak
2    Rahul   32   rahul
3    Amit    33   amit
4    Sumit   33   sumit

Print the last line using END block

Since variables retain their last assigned value, the END block can be used to print the final record.

awk '{ last = $0 } END { print last }' userdata.txt

Sample output:

4    Sumit   33   sumit

Field and Column Operations

Print selected columns

AWK treats each column as a field. Fields are accessed using $1, $2, $3, and so on.

awk '{ print $1, $2 }' userdata.txt

Sample output:

id Name
1 Deepak
2 Rahul
3 Amit
4 Sumit

This is commonly used to extract specific columns from structured data.

Print columns with conditions

Patterns can be combined with actions so that columns are printed only when a condition matches.

awk '/deepak/ { print $1, $2, $3 }' userdata.txt

Sample output:

1 Deepak 31

This helps filter and format output in a single command.

Reorder columns with custom text

This example reorders fields and formats the output using custom text. It prints the Name ($2) followed by the username ($4) inside parentheses.

awk '{ print $2 " (" $4 ")" }' userdata.txt

Sample output:

Deepak (deepak)
Rahul (rahul)
Amit (amit)
Sumit (sumit)

This is useful for generating readable reports.

Printing the number of columns in a line (NF)

NF is a built-in AWK variable that stores the number of fields in the current line. It is often used for validation and dynamic field access.

awk '{ print NF, $1, $NF }' userdata.txt

Sample output:

4 id username
4 1 deepak
4 2 rahul
4 3 amit
4 4 sumit

This example prints the field count, the first field, and the last field of each line.

Change the field separator (FS)

By default, AWK uses spaces as field separators. You can change this using -F or the FS variable. This example uses /etc/passwd, where fields are separated by colons.

awk -F: '{ print $1 }' /etc/passwd

Sample output:

root
bin
daemon

Pattern Matching & Regular Expressions

Match pattern and print lines

When only a pattern is specified, AWK automatically prints lines that match the pattern.

awk '/rahul/' userdata.txt

Sample output:

2    Rahul   32   rahul

This is useful for quickly filtering records based on text content.

Match a pattern in a specific field using (~)

The match operator (~) is used to check whether a specific field matches a regular expression.

Command used:

awk '$2 ~ /a/' userdata.txt

Explanation:

• $2 refers to the second column, which is the Name field.
• ~ means “matches the pattern”.
• /a/ is a regular expression that matches any value containing the lowercase letter a.
• AWK prints the entire line when the condition is true.

Sample output:

id   Name    Age  username
1    Deepak  31   deepak
2    Rahul   32   rahul

Match beginning of a field using (^)

The caret (^) is a regex anchor that matches the start of a field.

Command used:

awk '$2 ~ /^A/' userdata.txt

Explanation:

• $2 refers to the Name column.
• ^A means the field must start with the letter A.
• Only names beginning with uppercase A are matched.
• AWK prints the entire line when the condition is true.

Sample output:

3    Amit    33   amit

Why this line matched:

• Amit starts with the letter A
• Other names do not start with A

Match character classes using []

Character classes ([]) allow matching one character from a defined set.

Command used:

awk '$2 ~ /^[AD]/' userdata.txt

Explanation:

• ^[AD] means the field must start with either A or D.
• Character classes match exactly one character at that position.
• Matching is case-sensitive by default.

Sample output:

1    Deepak  31   deepak
3    Amit    33   amit

Why these lines matched:

• Deepak starts with D
• Amit starts with A

Match end of a field using ($)

The dollar sign ($) is a regex anchor that matches the end of a field.

Command used:

awk '$2 ~ /k$/' userdata.txt

Explanation:

• k$ means the field must end with the letter k.
• $ ensures the match occurs at the end, not anywhere in the field.

Sample output:

1    Deepak  31   deepak

Why this line matched:

• Deepak ends with the letter k
• Other names end with different characters

Conditional Logic & Comparisons

Numeric comparison using relational operators

AWK treats fields as numbers when used with relational operators such as >, <, >=, <=, ==, and !=.

awk '$3 >= 32' userdata.txt

Explanation:

• $3 refers to the Age column.
• >= 32 selects records where the numeric value of Age is greater than or equal to 32.
• AWK automatically performs numeric comparison without explicit type conversion.

Sample output:

2    Rahul   32   rahul
3    Amit    33   amit
4    Sumit   33   sumit

String comparison in a field

String comparisons in AWK must use double quotes around the string value.

awk '$4 == "deepak"' userdata.txt

Explanation:

• $4 refers to the username column.
• == "deepak" performs an exact string match.
• String comparisons are case-sensitive by default.

Sample output:

1    Deepak  31   deepak

Why this line matched:

• The username value exactly matches deepak

Combine multiple conditions using logical operators

AWK supports logical operators such as:

• && → AND
• || → OR
• ! → NOT

awk '$3 >= 30 && $3 <= 32' userdata.txt

Explanation:

• The first condition checks Age ≥ 30
• The second condition checks Age ≤ 32
• Both conditions must be true for the record to be printed.

Sample output:

1    Deepak  31   deepak
2    Rahul   32   rahul

Why these lines matched:

• Deepak → Age 31 (within range)
• Rahul → Age 32 (within range)

Counting, Statistics & Built-in Variables

Count total number of lines in a file

NR is a built-in AWK variable that stores the current record (line) number. Since NR keeps increasing as lines are read, its final value represents the total number of lines in the file.

awk 'END { print NR }' userdata.txt

Explanation:

• AWK processes the file line by line.
• NR increments automatically for each line.
• The END block runs after the last line is processed.
• Printing NR in the END block outputs the total line count.

This is functionally similar to:

wc -l userdata.txt

Sample output:

5

Count characters using length()

The length() function returns the number of characters in a string. When applied to $0, it counts characters in the entire line.

awk '{ print length($0), $0 }' userdata.txt

Explanation:

• $0 represents the entire current line.
• length($0) counts all characters, including spaces.
• The output shows the character count followed by the original line.

Sample output:

23 id   Name    Age  username
22 1    Deepak  31   deepak
22 2    Rahul   32   rahul
22 3    Amit    33   amit
22 4    Sumit   33   sumit

Remove empty lines using NF

NF represents the number of fields in the current line. For empty or blank lines, NF evaluates to 0.

awk 'NF > 0' userdata.txt

Explanation:

• Lines with at least one field have NF > 0.
• Empty or whitespace-only lines have NF == 0.
• AWK prints only lines that satisfy the condition.

This effectively removes:

• Blank lines
• Lines containing only spaces or tabs

Sample output:

id   Name    Age  username
1    Deepak  31   deepak
2    Rahul   32   rahul
3    Amit    33   amit
4    Sumit   33   sumit

Print line numbers using NR

NR is a built-in variable that stores the current line number. Prefixing it prints line numbers alongside each record.

awk '{ print NR, $0 }' userdata.txt

Explanation:

• NR prefixes each line with its line number.
• $0 prints the full line content.
• The output shows numbered lines, similar to nl or cat -n.

Sample output:

1 id   Name    Age  username
2 1    Deepak  31   deepak
3 2    Rahul   32   rahul
4 3    Amit    33   amit
5 4    Sumit   33   sumit

Control Structures and Programming in AWK

Control structures: AWK while loop

AWK supports control structures such as loops. This example uses a while loop to print numbers from 1 to 4.

awk 'BEGIN { n=1; while (n < 5) { print n; n++ } }'

Explanation:

• The BEGIN block runs before any input is read.
• n=1 initializes a variable.
• while (n < 5) keeps looping as long as the condition is true.
• print n outputs the current value of n.
• n++ increments n by 1 on each iteration.

Sample output:

1
2
3
4

Using Variables in AWK

Variables in AWK are created automatically when first used. They can store numbers, strings, or computed values.

Assign and use a variable:

awk '{ total = total + $3 } END { print total }' userdata.txt

Explanation:

• total is a user-defined variable.
• $3 refers to the Age column.
• The variable accumulates values for each line.
• The END block prints the final result.

Pass variables from the command line using -v:

awk -v min=32 '$3 >= min { print $1, $3 }' userdata.txt

Explanation:

• -v min=32 defines a variable before processing starts.
• This avoids hardcoding values inside the AWK program.
• Useful for reusable and parameterized scripts.

Use variables inside BEGIN block:

awk -v header="Name Age" 'BEGIN { print header } { print $1, $2 }' userdata.txt

Mental model:

• Variables are untyped (AWK decides number vs string automatically)
• Variables persist across lines
• -v is the preferred way to pass external values

Using if-else Conditions in AWK

AWK supports conditional logic using if, else if, and else.

Classify values using if-else:

awk '{
    if ($2 >= 30)
        print $1, "Senior"
    else
        print $1, "Junior"
}' data.txt

Multiple conditions using else if:

awk '{
    if ($2 >= 40)
        print $1, "Expert"
    else if ($2 >= 30)
        print $1, "Intermediate"
    else
        print $1, "Beginner"
}' data.txt

Use if-else inside BEGIN block:

awk 'BEGIN {
    if (1)
        print "AWK condition check passed"
    else
        print "This will never run"
}'

Using for Loops in AWK

AWK supports for loops for iterating over fields, records, or arrays.

Print all fields in each line using a loop:

awk '{
    for (i = 1; i <= NF; i++)
        print $i
}' file.txt

Print field number along with value:

awk '{
    for (i = 1; i <= NF; i++)
        print "Field", i ":", $i
}' file.txt

Associative Arrays in AWK

AWK arrays are associative (key-value based), not index-based.

Count occurrences of the first column:

awk '{ count[$1]++ } END {
    for (item in count)
        print item, count[item]
}' data.txt

Group and sum values by key:

awk '{ sum[$1] += $2 } END {
    for (item in sum)
        print item, sum[item]
}' data.txt

Track unique values:

awk '!seen[$1]++ { print $1 }' data.txt

Using Functions in AWK

You can define and reuse functions inside AWK programs.

Define and use a function:

awk '
function classify(age) {
    if (age >= 40)
        return "Expert"
    else if (age >= 30)
        return "Intermediate"
    else
        return "Beginner"
}
{
    print $1, classify($2)
}' data.txt

Use functions with BEGIN block:

awk '
function banner() {
    print "=== Processing File ==="
}
BEGIN {
    banner()
}
{ print $0 }
' file.txt

Advanced Data Handling

AWK for CSV and Delimiter-Based Files

By default, AWK uses whitespace as the field separator. For CSV or other delimiter-based files, specify the delimiter using -F.

Example CSV file (data.csv):

Name,Age,City
Alice,30,Bangalore
Bob,25,Chennai
Carol,28,Hyderabad

Print the first and third columns from a CSV file:

awk -F',' '{ print $1, $3 }' data.csv

Skip the header row:

awk -F',' 'NR > 1 { print $1, $3 }' data.csv

Using OFS to Control Output Formatting

OFS (Output Field Separator) controls how fields are separated in the output.

Print columns separated by |:

awk '{ OFS=" | "; print $1, $2, $3 }' file.txt

Set both input and output field separators:

awk -F',' '{ OFS=" | "; print $1, $2, $3 }' data.csv

Format output as CSV explicitly:

awk '{ OFS=","; print $1, $2, $3 }' file.txt

Practical AWK One-Liners for Logs

Extract IP addresses from access logs:

awk '{ print $1 }' access.log

Count requests per IP:

awk '{ count[$1]++ } END {
    for (ip in count)
        print ip, count[ip]
}' access.log

Filter logs by HTTP status code:

awk '$9 == 404 { print $0 }' access.log

Print timestamp and request URL:

awk '{ print $4, $7 }' access.log

Summary

AWK is a powerful yet lightweight text-processing tool that excels at working with structured data such as logs, command output, and delimited files. In this tutorial, you learned AWK progressively—from basic syntax to practical, real-world use cases—without unnecessary complexity.

By the end of this guide, you should be comfortable with:

• Understanding how AWK processes input line by line
• Using patterns and actions effectively
• Printing, filtering, and reordering columns
• Working with built-in variables like NR, NF, and FS
• Writing reusable AWK scripts with BEGIN and END blocks
• Applying conditions, comparisons, and regular expressions
• Using AWK loops and control structures

This tutorial is intentionally structured as a cheat sheet + hands-on guide, making it easy to revisit specific examples whenever you need them in day-to-day Linux or Unix work.

If you practice these examples regularly, AWK can replace many complex shell scripts with simple, fast, and readable one-liners.

References

Use the following references to deepen your understanding or explore advanced AWK features:

• GNU AWK Manual – Official and most comprehensive reference for AWK
• AWK man page – Quick command-line reference
• Learning AWK Programming (O’Reilly) – In-depth book for serious learners
• POSIX AWK Specification – Standard behavior across Unix systems

Next steps

To continue improving your AWK skills:

1. Apply AWK to real log files and command output
2. Combine AWK with shell pipelines such as grep, sort, and uniq
3. Write small AWK scripts instead of long shell loops
4. Explore advanced topics such as associative arrays and functions

Mastering these concepts will make AWK one of the most productive tools in your Linux tool