Friday, September 23, 2016

Linux Grep
       -exec command ;
              Execute command; true if 0 status is returned.  All following
              arguments to find are taken to be arguments to the command
              until an argument consisting of `;' is encountered.  The
              string `{}' is replaced by the current file name being
              processed everywhere it occurs in the arguments to the
              command, not just in arguments where it is alone, as in some
              versions of find.  Both of these constructions might need to
              be escaped (with a `\') or quoted to protect them from
              expansion by the shell.  See the EXAMPLES section for examples
              of the use of the -exec option.  The specified command is run
              once for each matched file.  The command is executed in the
              starting directory.  There are unavoidable security problems
              surrounding use of the -exec action; you should use the
              -execdir option instead.

       -exec command {} +
              This variant of the -exec action runs the specified command on
              the selected files, but the command line is built by appending
              each selected file name at the end; the total number of
              invocations of the command will be much less than the number
              of matched files.  The command line is built in much the same
              way that xargs builds its command lines.  Only one instance of
              `{}' is allowed within the command, and (when find is being
              invoked from a shell) it should be quoted (for example, '{}')
              to protect it from interpretation by shells.  The command is
              executed in the starting directory.  If any invocation returns
              a non-zero value as exit status, then find returns a non-zero
              exit status.  If find encounters an error, this can sometimes
              cause an immediate exit, so some pending commands may not be
              run at all.  This variant of -exec always returns true.

-s, --no-messages
             Silent mode.  Nonexistent and unreadable files are ignored (i.e. their error messages are suppressed).
  • -e pattern
  • -i: Ignore uppercase vs. lowercase.
  • -v: Invert match.
  • -c: Output count of matching lines only.
  • -l: Output matching files only.
  • -n: Precede each matching line with a line number.
  • -b: A historical curiosity: precede each matching line with a block number.

[Find multiple patterns across multiple lines](
- find . | xargs grep 'pattern1' -sl | xargs grep 'pattern2' -sl
The standard option grep -l (that is a lowercase L) could do this.
  1. How can I list just the names of matching files?
    grep -l 'main' *.c
    lists the names of all C files in the current directory whose contents mention `main'.
  2. How do I search directories recursively?
    grep -r 'hello' /home/gigi
    searches for `hello' in all files under the directory `/home/gigi'. For more control of which files are searched, use @command{find}, @command{grep} and @command{xargs}. For example, the following command searches only C files:
    find /home/gigi -name '*.c' -print | xargs grep 'hello' /dev/null
  3. What if a pattern has a leading `-'?
    grep -e '--cut here--' *
    searches for all lines matching `--cut here--'. Without `-e', @command{grep} would attempt to parse `--cut here--' as a list of options.
  4. Suppose I want to search for a whole word, not a part of a word?
    grep -w 'hello' *
    searches only for instances of `hello' that are entire words; it does not match `Othello'. For more control, use `\<' and `\>' to match the start and end of words. For example:
    grep 'hello\>' *
    searches only for words ending in `hello', so it matches the word `Othello'.
  5. How do I output context around the matching lines?
    grep -C 2 'hello' *
    prints two lines of context around each matching line.
  6. How do I force grep to print the name of the file? Append `/dev/null':
    grep 'eli' /etc/passwd /dev/null
  7. Why do people use strange regular expressions on @command{ps} output?
    ps -ef | grep '[c]ron'
    If the pattern had been written without the square brackets, it would have matched not only the @command{ps} output line for @command{cron}, but also the @command{ps} output line for @command{grep}.
  8. Why does @command{grep} report "Binary file matches"? If @command{grep} listed all matching "lines" from a binary file, it would probably generate output that is not useful, and it might even muck up your display. So GNU @command{grep} suppresses output from files that appear to be binary files. To force GNU @command{grep} to output lines even from files that appear to be binary, use the `-a' or `--text'option.
  9. Why doesn't `grep -lv' print nonmatching file names? `grep -lv' lists the names of all files containing one or more lines that do not match. To list the names of all files that contain no matching lines, use the `-L' or `--files-without-match' option.
  10. I can do OR with `|', but what about AND?
    grep 'paul' /etc/motd | grep 'franc,ois'
    finds all lines that contain both `paul' and `franc,ois'.
  11. How can I search in both standard input and in files? Use the special file name `-':
    cat /etc/passwd | grep 'alain' - /etc/motd
tail -f logs/production.log | grep "500 Internal Server Error"

-A NUM, --after-context=NUM Print NUM lines of trailing context after matching lines. Places a line containing -- between contiguous groups of matches. -a, --text Process a binary file as if it were text; this is equivalent to the --binary-files=text option. -B NUM, --before-context=NUM Print NUM lines of leading context before matching lines. Places a line containing -- between contiguous groups of matches. -C NUM, --context=NUM Print NUM lines of output context. Places a line containing -- between contiguous groups of matches.
tail -f logs/production.log | grep "500 Internal Server Error" -B 2 -A 5
For BSD or GNU grep you can use -B num to set how many lines before the match and -A numfor the number of lines after the match.
grep -B 3 -A 2 foo README.txt
If you want the same number of lines before and after you can use -C num.
grep -C 3 foo README.txt
This will show 3 lines before and 3 lines after.
Line numbers are printed with grep -n:
grep -n pattern file.txt
To get only the line number (without the matching line), one may use cut:
grep -n pattern file.txt | cut -d : -f 1
Lines not containing a pattern are printed with grep -v:
grep -v pattern file.txt

The -n or --line-number grep option

You can pass either -n or --line-number option to the grep command to prefix each line of output with the line number within its input file. The syntax is:
grep -n 'patten' file
grep -n 'patten' file1 file2
grep -n [options] 'pattens' file
In basic regular expressions the meta-characters ‘?’, ‘+’, ‘{’, ‘|’, ‘(’, and ‘)’ lose their special meaning; instead use the backslashed versions ‘\?’, ‘\+’, ‘\{’, ‘\|’, ‘\(’, and ‘\)’.
Traditional egrep did not support the ‘{’ meta-character, and some egrep implementations support ‘\{’ instead, so portable scripts should avoid ‘{’ in ‘grep -E’ patterns and should use ‘[{]’ to match a literal ‘{’.
GNU grep -E attempts to support traditional usage by assuming that ‘{’ is not special if it would be the start of an invalid interval specification. For example, the command ‘grep -E '{1'’ searches for the two-character string ‘{1’ instead of reporting a syntax error in the regular expression. POSIX allows this behavior as an extension, but portable scripts should avoid it.
  • egrep is 100% equivalent to grep -E
  • fgrep is 100% equivalent to grep -F
Historically these switches were provided in separate binaries. On some really old Unix systems you will find that you need to call the separate binaries, but on all modern systems the switches are preferred. The man page for grep has details about this.
As for what they do, -E switches grep into a special mode so that the expression is evaluated as an ERE (Extended Regular Expression) as opposed to its normal pattern matching. Details of this syntax are on the man page.
-E, --extended-regexp
Interpret PATTERN as an extended regular expression
The -F switch switches grep into a different mode where it accepts a pattern to match, but then splits that pattern up into one search string per line and does an OR search on any of the strings without doing any special pattern matching.
-F, --fixed-strings
Interpret PATTERN as a list of fixed strings, separated by newlines, any of which is to be matched.
Here are some example scenarios:
  • You have a file with a list of say ten Unix usernames in plain text. You want to search the group file on your machine to see if any of the ten users listed are in any special groups:
    grep -F -f user_list.txt /etc/group
    The reason the -F switch helps here is that the usernames in your pattern file are interpreted as plain text strings. Dots for example would be interpreted as dots rather than wild-cards.
  • You want to search using a fancy expression. For example parenthesis () can be used to indicate groups with | used as an OR operator. You could run this search using -E:
    grep -E '^no(fork|group)' /etc/group return lines that start with either "nofork" or "nogroup". Without the -E switch you would have to escape the special characters involved because with normal pattern matching they would just search for that exact pattern;
    grep '^no\(fork\|group\)' /etc/group
The following command will do what you asked for "extract the lines between 1234 and 5555" in someFile.
sed -n '1234,5555p' someFile
Example 1. Beginning of line ( ^ )
Example 2. End of the line ( $)
$ grep "terminating.$" messages
$ grep "^Nov 10" messages.1

Example 4. Single Character (.)
$ grep ".ello" input

Example 5. Zero or more occurrence (*)
$ grep "kernel: *." *

Example 6. One or more occurrence (\+)

No comments:

Post a Comment


Review (561) System Design (304) System Design - Review (196) Java (179) Coding (75) Interview-System Design (65) Interview (60) Book Notes (59) Coding - Review (59) to-do (45) Linux (40) Knowledge (39) Interview-Java (35) Knowledge - Review (32) Database (31) Design Patterns (29) Product Architecture (28) Big Data (27) Soft Skills (27) Concurrency (26) MultiThread (26) Miscs (25) Cracking Code Interview (24) Distributed (24) Career (22) Interview - Review (21) Java - Code (21) Operating System (21) Interview Q&A (20) OOD Design (20) System Design - Practice (19) How to Ace Interview (16) Security (16) Algorithm (15) Brain Teaser (14) Google (14) Redis (14) Linux - Shell (13) Spark (13) Spring (13) Code Quality (12) How to (12) Interview-Database (12) Interview-Operating System (12) Tools (12) Architecture Principles (11) Company - LinkedIn (11) Solr (11) Testing (11) Resource (10) Search (10) Amazon (9) Cache (9) Web Dev (9) Architecture Model (8) Better Programmer (8) Company - Uber (8) Interview - MultiThread (8) Java67 (8) Math (8) OO Design principles (8) SOLID (8) Scalability (8) Trouble Shooting (8) Cassandra (7) Company - Facebook (7) Design (7) Git (7) Interview Corner (7) JVM (7) Java Basics (7) Kafka (7) Machine Learning (7) NoSQL (7) C++ (6) File System (6) Highscalability (6) How to Better (6) Network (6) Restful (6) CareerCup (5) Code Review (5) Hash (5) How to Interview (5) JDK Source Code (5) JavaScript (5) Leetcode (5) Must Known (5) API Design (4) Be Architect (4) Big Fata (4) C (4) Company Product Architecture (4) Data structures (4) Design Principles (4) Facebook (4) GeeksforGeeks (4) Generics (4) Google Interview (4) Hardware (4) JDK8 (4) Optimization (4) Product + Framework (4) Puzzles (4) Python (4) Shopping System (4) Source Code (4) Web Service (4) node.js (4) Back-of-Envelope (3) Chrome (3) Company - Pinterest (3) Company - Twiiter (3) Company - Twitter (3) Consistent Hash (3) Elasticsearch (3) GOF (3) Game Design (3) GeoHash (3) Growth (3) Guava (3) Html (3) Interview-Big Data (3) Interview-Linux (3) Interview-Network (3) Java EE Patterns (3) Javarevisited (3) Map Reduce (3) Math - Probabilities (3) Performance (3) RateLimiter (3) Resource-System Desgin (3) Scala (3) UML (3) ZooKeeper (3) geeksquiz (3) AI (2) Advanced data structures (2) AngularJS (2) Behavior Question (2) Bugs (2) Coding Interview (2) Company - Netflix (2) Crawler (2) Cross Data Center (2) Data Structure Design (2) Database-Shard (2) Debugging (2) Docker (2) Garbage Collection (2) Go (2) Hadoop (2) Interview - Soft Skills (2) Interview-Miscs (2) Interview-Web (2) JDK (2) Logging (2) POI (2) Papers (2) Programming (2) Project Practice (2) Random (2) Software Desgin (2) System Design - Feed (2) Thread Synchronization (2) Video (2) reddit (2) Ads (1) Algorithm - Review (1) Android (1) Approximate Algorithms (1) Base X (1) Bash (1) Books (1) C# (1) CSS (1) Client-Side (1) Cloud (1) CodingHorror (1) Company - Yelp (1) Counter (1) DSL (1) Dead Lock (1) Difficult Puzzles (1) Distributed ALgorithm (1) Eclipse (1) Facebook Interview (1) Function Design (1) Functional (1) GoLang (1) How to Solve Problems (1) ID Generation (1) IO (1) Important (1) Internals (1) Interview - Dropbox (1) Interview - Project Experience (1) Interview Stories (1) Interview Tips (1) Interview-Brain Teaser (1) Interview-How (1) Interview-Mics (1) Interview-Process (1) Java Review (1) Jeff Dean (1) Joda (1) LeetCode - Review (1) Library (1) LinkedIn (1) LintCode (1) Mac (1) Micro-Services (1) Mini System (1) MySQL (1) Nigix (1) NonBlock (1) Process (1) Productivity (1) Program Output (1) Programcreek (1) Quora (1) RPC (1) Raft (1) Reactive (1) Reading (1) Reading Code (1) Refactoring (1) Resource-Java (1) Resource-System Design (1) Resume (1) SQL (1) Sampling (1) Shuffle (1) Slide Window (1) Spotify (1) Stability (1) Storm (1) Summary (1) System Design - TODO (1) Tic Tac Toe (1) Time Management (1) Web Tools (1) algolist (1) corejavainterviewquestions (1) martin fowler (1) mitbbs (1)

Popular Posts