Home of the Bioinformatics Perl Workshop

perl workshop > courses > data mining and analysis at the command line (2.1.2.4) > lecture notes

course 2.1.2.4

Level: intermediate

Data Mining and Analysis at the Command Line

You don't need to write scripts extract information from text files. Nor do you always want to. This 4-week series will take Perl's idiom philosophy and apply it to getting things done at the prompt. Learn how to leverage standard UNIX tools to perform rapid data analysis without writing any code. We'll introduce a set of custom Perl scripts to add to your prompt toolbox - these have been designed to provide additional functionality, such as histogramming, random sampling, conditional line extraction and others.

lecture notes <

legend

course code

cat.course.level.sessions.session

e.g. 1.0.1.8

categories

0 | introduction and orientation

1 | perl fundamentals

2 | shell and prompt tools

3 | web development

4 | CPAN Modules

5 | Ruby

levels

level: all all ( 0 )

level: beginner beginner ( 1 )

level: intermediate intermediate ( 2 )

level: advanced advanced ( 3 )

[ Camels are known to spit up to 36 feet in the US and 11 meters everywhere else. ]

2.1.2.4 Data Mining and Analysis at the Command Line

lecture notes

1 | Happy UNIX Friends | 2.1.2.4.1

Learn the do's and don'ts of creating flat-file reports. Get familiar with cut, join, sort, and uniq to sift through data quickly.

2.1.2.4.1.d1 | Happy UNIX Friends | Martin Krzywinski | data

2.1.2.4.1.d2 | Happy UNIX Friends | Martin Krzywinski | data

2.1.2.4.1.d3 | Happy UNIX Friends | Martin Krzywinski | data

2.1.2.4.1.a1 | Happy UNIX Friends | Martin Krzywinski | pdf

2.1.2.4.1.p1 | Happy UNIX Friends | Martin Krzywinski | ppt

2.1.2.4.1.s1 | Happy UNIX Friends | Martin Krzywinski | slides

2 | More Happy UNIX Friends | 2.1.2.4.2

Sort, join and agrep have a lot to offer.

2.1.2.4.2.c1 | dumpout | Martin Krzywinski | code

2.1.2.4.2.c2 | readin | Martin Krzywinski | code

2.1.2.4.2.c3 | c.txt | Martin Krzywinski | code

2.1.2.4.2.c4 | colors.txt | Martin Krzywinski | code

2.1.2.4.2.c5 | forsort.txt | Martin Krzywinski | code

2.1.2.4.2.c6 | makesort | Martin Krzywinski | code

2.1.2.4.2.c7 | nummax.txt | Martin Krzywinski | code

2.1.2.4.2.c8 | nummin.txt | Martin Krzywinski | code

2.1.2.4.2.c9 | redir | Martin Krzywinski | code

2.1.2.4.2.c10 | sounds.txt | Martin Krzywinski | code

2.1.2.4.2.c11 | t.tgz | Martin Krzywinski | code

2.1.2.4.2.c12 | test.txt | Martin Krzywinski | code

2.1.2.4.2.c13 | text.txt | Martin Krzywinski | code

2.1.2.4.2.c14 | tmp.1 | Martin Krzywinski | code

2.1.2.4.2.a1 | More Happy UNIX Friends | Martin Krzywinski | pdf

2.1.2.4.2.p1 | More Happy UNIX Friends | Martin Krzywinski | ppt

2.1.2.4.2.s1 | More Happy UNIX Friends | Martin Krzywinski | slides

3 | Perl Prompt Tools | 2.1.2.4.3

Collection of Perl scripts to extend and add to command-line functionality.

2.1.2.4.3.c1 | alphabet.fa | Martin Krzywinski | code

2.1.2.4.3.c2 | args.txt | Martin Krzywinski | code

2.1.2.4.3.a1 | Perl Prompt Tools | Martin Krzywinski | pdf

2.1.2.4.3.p1 | Perl Prompt Tools | Martin Krzywinski | ppt

2.1.2.4.3.s1 | Perl Prompt Tools | Martin Krzywinski | slides

4 | Perl Prompt Tools | 2.1.2.4.4

Collection of Perl scripts to extend and add to command-line functionality (cont'd)

2.1.2.4.4.a1 | Perl Prompt Tools | Martin Krzywinski | pdf

2.1.2.4.4.p1 | Perl Prompt Tools | Martin Krzywinski | ppt

2.1.2.4.4.s1 | Perl Prompt Tools | Martin Krzywinski | slides

other in this category

2.0.0.3 | Introduction to Unix

2.2.2.2 | Prompt Tools

other by same level

1.1.2.8 | Intermediate Perl

1.2.2.1 | Effective use of map, sort and grep in Perl

2.2.2.2 | Prompt Tools

4.0.2.1 | Spans and Sets

4.1.2.2 | Random Numbers and Distributions

other by same instructor

Other courses by Martin Krzywinski.

0.0.0.1 | Orientation Session

0.1.0.1 | Two Problems

1.0.1.8 | Introduction to Perl

1.1.2.8 | Intermediate Perl

1.2.2.1 | Effective use of map, sort and grep in Perl

2.2.2.2 | Prompt Tools

4.0.2.1 | Spans and Sets

4.1.2.2 | Random Numbers and Distributions