Examancer

I recently had to find a specific line in a large (28GB) file equipped with nothing more than the line number. I thought it would take me just a few seconds to find a cool *nix utility to accomplish this task. Instead, it took me a bit of scouring to find something that works, and works well on large files. That’s OK though since I had to wait for the 28GB file to uncompress from a tarball… which, obviously, takes a while.

What I learned about while I waited was the *nix command ‘sed’. This is a tool built for command line processing of data files. Apparently it was birthed as an evolution of our trusty friend ‘grep’. The forum post I found which hinted that ‘sed’ was my solution didn’t provide much real information and the Wikipedia article was mostly background and provided examples that won’t help me.

Where I found the most useful info was the sed page on sourceforge… go figure. The docs page pointed me to ‘The sed one-liners‘ by Eric Pement. Here I found, through example, the power of ‘sed’ and an example that is more efficient on large files than the ones I found elsewhere.

So here is how you do it:

’34005050′ is the line number. ‘q’ tells sed you are looking for that line number, and ‘;d’ tells it to stop after that line. ‘filename’ is of course the file you are trying to coax a specific line out of. To do an inclusive range of lines all at once (lines 8 through 12, for example), do this:

I’m still learning about ‘sed’ but its already saving my ass. Have fun.

Tags: linux, reference, sed, unix

This entry was posted on Friday, December 4th, 2009 at 12:24 pm and is filed under Programming. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.