Creators Aho, Weinberger, and Kernighan gave the Unix and Linux community a wonderful and powerful text-processing language. The AWK commands can be used either to simply scan a text pattern and also to process text. In real world conditions, AWK command can empower your scripting skills to master manipulation and analysis of text files. In this article, let us learn the basics of the AWK command, followed by some advanced cases where you can apply them. These basic concepts with some practical examples will guide you use AWK commands to your requirements efficiently.
AWK: What is it?
AWK is a programming language usually used as a data reporting and extraction tool. It was primarily designed for processing text. AWK is used to perform desired operations as per the inputs specified in the command by processing every line in a file.
AWK Command Basic Syntax
Here is the very basic syntax of the AWK command:
Here, pattern refers to the pattern to search for, replace “action to be done” with your required action, and file is the name of the file to process.
There are two common pitfalls where most of the users make mistakes in executing this AWK command. They are:
- There should be a space before and after the action.
- Always use ‘ instead of “.
What are Fields and Records in AWK?
In AWK, a single line of text is a record. The individual pieces of data on that line (record) are fields.These individual pieces of data in a record could be separated by either a whitespace or a specified delimiter. A variable is assigned to each field by AWK. For example, an entire line is represented by $0. The first field is represented by $1, the second field is represented by $2, the third by $3, and so on.
Some Commonly Used AWK Commands
Print Columns using AWK
One of the most common uses of AWK is to print specific columns from a file. To print the first and fourth columns of each line in a file, execute the following command.
Print Lines that Match a Pattern using AWK
With AWK, you can perform actions only on the lines that match a specified pattern. Customize the following command as per your requirements.
This command prints all the lines that have the pattern specified.
AWK Command to Extract Specific Fields from a File
Use the following variation of AWK command to extract and print specific fields in a file. This command helps you when you are sifting through a huge text or CSV file but you want only specific fields and not the entire dataset.
This command extracts and prints the first and fourth fields from the file. The comma after “-F” tells AWK that the separator is comma in the file.
How to Add all the Values in a Column
Execute this command to add all the values in a specified column and print the total value after processing all the lines in the file.
This command adds all the values in the first column and prints the total value.
How to Count the Non-Empty Lines in a File
AWK command uses the NF operator to count the non-empty lines in a file.
This command counts all the lines that have at least one field in them i.e. not empty.
How to Show Messages Before and After Executing AWK Command
To perform any action before and after executing an AWK command, use the “BEGIN” and “END” parameters.
This command prints a message before processing each line to print the first field, and then prints an end message.
How to Use Variables in AWK
With AWK, you can use variables to manipulate and store data.
This example gathers the sum of the first column in a variable and prints it at the end.
Wrapping Up
AWK is a user-friendly, easy to learn, and efficient tool for processing text in Linux. Though it seems like a plain text processing tool, the uses are many like extracting pin-pointed data, filter log files, and perform calculations. Use AWK and breeze through your text files.
Related Links
AWK Programming language documentation
