/ 
PerlPowerTools-1.045
River stage one • 1 direct dependent • 1 total dependent
/ sort

NAME

sort - sort or merge text files

SYNOPSIS

sort [-cmudfinrbD] [-o output_file]
  [-t field_separator] [-X regex_field_separator] [-R record_separator]
  [-k pos1[,pos2]] [+pos1 [-pos2]]
  [-y max_records] [-F max_files]
  input_file1 [input_file2 ...]

DESCRIPTION

The sort utility sorts text files by lines (or records). Comparisons are based on one or more sort keys extracted from each line of input, and are performed lexicographically. By default, if keys are not given, sort regards each input line as a single field. The sort is a merge sort. If you don't like that, feel free to change it.

OPTIONS

The following options are available:

I INPUT

Pass in the input file(s). This can be either a single string with the filename, or an array reference containing multiple filename strings.

c

Check that single input file is ordered as specified by the arguments and the collating sequence of the current locale. No output is produced; only the exit code is affected.

m

Merge only; the input files are assumed to already be sorted.

o OUTPUT

Specify the name of an OUTPUT file to be used instead of the standard output.

u

Unique: Suppresses all but one in each set of lines having equal keys. If used with the c option check that there are no lines with consecutive lines with duplicate keys, in addition to checking that the input file is sorted.

y MAX_SORT_RECORDS

Maximum number of lines (records) read before writing to temp file. Default is 200,000. This may eventually change to be kbytes instead of lines. Lines was easier to implement. Can also specify with MAX_SORT_RECORDS environment variable.

F MAX_SORT_FILES

Maximum number of temp files to be held open at once. Default to 40, as older Windows ports had quite a small limit. Can also specify with MAX_SORT_FILES environment variable. No temp files will be used at all if MAX_SORT_RECORDS is never reached.

D

Send debugging information to STDERR. Behavior subject to change.

The following options override the default ordering rules. When ordering options appear independent of any key field specifications, the requested field ordering rules are applied globally to all sort keys. When attached to a specific key (see k), the specified ordering options override all global ordering options for that key.

d

Specify that only blank characters and alphanumeric characters, according to the current locale setting, are significant in comparisons. d overrides i.

f

Consider all lower-case characters that have upper-case equivalents, according to the current locale setting, to be the upper-case equivalent for the purposes of comparison.

i

Ignores all characters that are non-printable, according to the current locale setting.

n

Does numeric instead of string compare, using whatever perl considers to be a number in numeric comparisons.

r

Reverse the sense of the comparisons.

b

Ignore leading blank characters when determining the starting and ending positions of a restricted sort key. If the b option is specified before the first k option, it is applied to all k options. Otherwise, the b option can be attached independently to each field_start or field_end option argument (see below).

t STRING

Use STRING as the field separator character; char is not considered to be part of a field (although it can be included in a sort key). Each occurrence of char is significant (for example, <char><char> delimits an empty field). If t is not specified, blank characters are used as default field separators; each maximal non-empty sequence of blank characters that follows a non-blank character is a field separator.

X STRING

Same as t, but STRING is interpreted as a Perl regular expression instead. Do not escape any characters (/ characters need to be escaped internally, and will be escaped for you).

The string matched by STRING is not included in the fields themselves, unless demanded by perl's regex and split semantics (e.g., regexes in parentheses will add that matched expression as an extra field). See perlre and "split" in perlfunc.

R STRING

Record separator, defaults to newline.

k pos1[,pos2]

The keydef argument is a restricted sort key field definition. The format of this definition is:

field_start[.first_char][type][,field_end[.last_char][type]]

where field_start and field_end define a key field restricted to a portion of the line, and type is a modifier from the list of characters b, d, f, i, n, r. The b modifier behaves like the b option, but applies only to the field_start or field_end to which it is attached. The other modifiers behave like the corresponding options, but apply only to the key field to which they are attached; they have this effect if specified with field_start, field_end, or both. If any modifier is attached to a field_start or a field_end, no option applies to either.

Occurrences of the k option are significant in command line order. If no k option is specified, a default sort key of the entire line is used. When there are multiple keys fields, later keys are compared only after all earlier keys compare equal.

Except when the u option is specified, lines that otherwise compare equal are ordered as if none of the options d, f, i, n or k were present (but with r still in effect, if it was specified) and with all bytes in the lines significant to the comparison. The order in which lines that still compare equal are written is unspecified.

+pos1 [-pos2]

Similar to k, these are mostly obsolete switches, but some people like them and want to use them. Usage is:

+field_start[.first_char][type] [-field_end[.last_char][type]]

Where field_end in k specified the last position to be included, it specifies the last position to NOT be included. Also, numbers are counted from 0 instead of 1. pos2 must immediately follow corresponding +pos1. The rest should be the same as the k option.

Mixing +pos1 pos2 with k is allowed, but will result in all of the +pos1 pos2 options being ordered AFTER the k options. It is best if you Don't Do That. Pick one and stick with it.

Here are some equivalencies:

+1 -2           =>  -k 2,2
+1.1 -1.2       =>  -k 2.2,2.2
+1 -2 +3 -5     =>  -k 2,2 -k 4,5
+2 +0b -1       =>  -k 3 -k 1b,1
+2.1 -2.4       =>  -k 3.2,3.4
+2.0 -3.0       =>  -k 3.1,4.0

Not Implemented

If the options are not listed as implemented above, or are not listed in TODO below, they are not in the plan for implementation. This includes T and z.

EXAMPLES

Sort file by straight string compare of each line.

sort file

Sort contents of file by second key in file.

sort -k 2 file

Sort, in reverse order, contents of file1 and file2, placing output in outfile and using second character of second field as the sort key.

sort -r -k 2.2,2.2 -o outfile file1 file2

Same sort but sorting numerically on characters 3 through 5 of the fifth field first, and only return records with unique keys.

sort -u -r -k 5.3,5.5rn -k 2.2,2.2 -o outfile file1 file2

Print passwd(4) file sorted by numeric user ID.

sort -t : -k 3n /etc/passwd

For the anal sysadmin, check that passwd(4) file is sorted by numeric user ID.

sort -c -t : -k 3n /etc/passwd

ENVIRONMENT

Note that if you change the locale settings after the program has started up, you must call setlocale() for the new settings to take effect. For example:

# get constants
use POSIX 'locale_h';

# e.g., blank out locale
$ENV{LC_ALL} = $ENV{LANG} = '';

# use new ENV settings
setlocale(LC_CTYPE, '');
setlocale(LC_COLLATE, '');
LC_COLLATE

Determine the locale for ordering rules.

LC_CTYPE

Determine the locale for the interpretation of sequences of bytes of text data as characters (for example, single- versus multi-byte characters in arguments and input files) and the behaviour of character classification for the b, d, f, i and n options.

MAX_SORT_RECORDS

Default is 200,000. Maximum number of records to use before writing to a temp file. Overridden by y option.

MAX_SORT_FILES

Maximum number of open temp files to use before merging open temp files. Overridden by F option.

TODO

Better debugging and error reporting
Performance hit with -u
Do bytes instead of lines
Better test suite
Switch for turning off locale ... ?

HISTORY

v1.00, Tuesday, November 13, 2001

Long overdue release.

Add O_TRUNC to output open (D'oh!).

Played with somem of the -k options (Marco A. Romero).

Fix filehandle close test of STDOUT (Gael Marziou).

Some cleanup.

v0.68, Saturday, February 12, 2000

Closed all files in test.pl so they could be unlinked on some platforms. (Hubert Toullec)

Documented I option. (Hubert Toullec)

Removed O_EXCL flag from sort_file.

Fixed bug in sorting multiple files. (Paul Eckert)

v0.67 Friday, April 30, 1999

Merged sources back into File::Sort module.

Fixed a few bugs, including accepting input via STDIN if no input files given.

v0.66 Monday, April 5, 1999

Added +pos1 -pos2, and X (regex field separator).

More minor fixes to sort subs.

v0.65 Sunday, April 4, 1999

Added b and R (record separator).

Made more fixes to sort subs. Seems to be getting pretty stable now, after some serious rewriting. Borrowed several ideas from Albert Dvornik's implementation.

Tidied up docs.

Added usage() function.

v0.60 Saturday, April 3, 1999

Rewrote most of the sort sub stuff to be more efficient and in some cases to just make it work right.

Can now use multiple k switches. Need Getopt::Long 2.17 (comes with perl 5.005_02). Earlier versions of module work?

Still need to go back and implement +pos1 -pos2 if I feel like it.

v0.51 Tuesday, March 30, 1999

First round of fixes.

v0.50 Monday, March 29, 1999

Rewrote much of the code for the PPT project. Will integrate changes back into the original source, the File::Sort module.

THANKS

Mike Blazer <blazer@mail.nevalink.ru>, Vicki Brown <vlb@cfcl.com>, Tom Christiansen <tchrist@perl.com>, Albert Dvornik <bert@mit.edu>, Paul Eckert <peckert@epicrealm.com>, Gene Hsu <gene@moreinfo.com>, Andrew M. Langmead <aml@world.std.com>, Gael Marziou <gael_marziou@hp.com>, Brian L. Matthews <blm@halcyon.com>, Rich Morin <rdm@cfcl.com>, Matthias Neeracher <neeri@iis.ee.ethz.ch>, Miko O'Sullivan <miko@idocs.com>, Tom Phoneix <rootbeer@teleport.com>, Marco A. Romero <mromero@iglou.com>, Gurusamy Sarathy <gsar@activestate.com>, Hubert Toullec <Hubert.Toullec@wanadoo.fr>.

AUTHOR

Chris Nandor <pudge@pobox.com>, http://pudge.net/

Copyright (c) 1997-2001 Chris Nandor. All rights reserved. This program is free software; you can redistribute it and/or modify it under the terms of the Artistic License, distributed with Perl.

VERSION

v1.00, Tuesday, November 13, 2001

SEE ALSO

PPT project, <URL:http://sf.net/projects/ppt/>, sort(1), File::Sort, locale.