xargs - The generic tool for arguments

    xargs is one of the most versatile tool used by command line warriors. There is a sort of aura around this command. Numerous videos are out there explaining the usage of this tool. The discussions below are to be taken as one for the entry level command line users. Kindly refer to the man pages for further advanced usage. In particular more information and its usage can be better appreciated from books and articles on shell programming, where a discussion on this is a mainstay.

    I will be using music files and c source files as examples wherever appropriate according to the easiness with which the underlying usage of xargs is more clear.

In the present directory as we start this discussion, we have the following files

music1.mp3
music2.mp3
music 3.mp3(notice the space in between music and 3)
music 4.mp3
big_program1.c
big_program2.c
big_program3.c

ls *.mp3 | xargs -t

echo music1.mp3 music2.mp3 music 3.mp3 music 4.mp3
music1.mp3 music2.mp3 music 3.mp3 music 4.mp3

    The output above is a little cryptic. Let us clear it first. If there is no other command to which xargs passes the arguments, the by default xargs pipes it to echo. The -t option is same as --verbose option with GNU extensions(by the way, I like GNU extensions for any program, they are more verbose and use the long form to extend  and explain what that particular option would do). By specifying the -t option, we are telling xargs to be more verbose in its activities. The result was that the xargs command took the output of ls and piped it to echo. We can see that exact executed by xargs in the first line of the output. The total result of the command is the output in the second line. All the arguments were passed on at once to the echo command.

    Now, let us change the above command such that the echo command is run on every argument. For that,

ls *.mp3 | xargs -t -n 1

echo music1.mp3
music1.mp3
echo music2.mp3
music2.mp3
echo music
music
echo 3.mp3
3.mp3
echo music
music
echo 4.mp3
4.mp3

    If we see the output, we met our task, partially. echo command is run on every argument. However, the file names with spaces in them have created a problem. By default, the delimiter for xargs is the space/blank character. To ensure that the above command works properly, we have to include the information of the delimiter in the xargs command, like so.


ls *.mp3 | xargs -t -n 1 -d'\n'

echo music1.mp3
music1.mp3
echo music2.mp3
music2.mp3
echo 'music 3.mp3'
music 3.mp3
echo 'music 4.mp3'
music 4.mp3

    Now, we see that the xargs command met our requirements. Now we are pretty confident that we can now pass on these xargs to other programs and are confident of them working properly. Let us now play the mp3s.


ls *.mp3 | xargs -t -n 1 -d'\n' mpg123

    This will now play all the mp3 files in alphanumeric order as listed by the default ls command, one after the another.

    Let us now take up the next key strength of xargs. xargs has the capability to spawn multiple processes so that arguments can be passed on to multiple instances of the program. For eg,

ls *.mp3 | xargs -t -n 1 -d'\n' -P 8 mpg123

    That was indeed funny, right. Seriously, that is a very wrong example to illustrate the parallel processing achieved by the above command. The above command starts multiple instances of mpg123, with all the instance playing an argument supplied. You can increase the argument for -P based on the number of files piped into the final command. Here you have all the four songs playing in parallel. The sound output might be interesting for music producers. For us listeners, nothing.

    Let us now consider a proper, practical example for this spawning of multiple processes,

ls *.c | xargs -t -n 1 -P 8 gcc

    The above command will run multiple instances of gcc at the same time and the compilation will happen in parallel on all the 3 files in the present directory. To do this kindly ensure that your programs can be compiled so and are stand alone and not dependent on each other being compiled already. Here we do not have the problem of file name containing spaces. If you are comfortable with

make -j 8

    For compiling your linux kernel, you know exactly what is happening here. The linux kernel is one of the greatest and practical examples for compiling sources in parallel. What a great feat. Kudos to all the kernel devs.

    With this, we come to the end of this gentle introduction to one of the most important tools for shell programming. The knowledge of xargs is a must for all shell programming aficionados.

    The standard thanks and regards to all the FLOSS users and devs. You people are great. Thanks again.