Posts Tagged ‘bash’

Ruby: Call Bash commands, run shell scripts

http://stackoverflow.com/questions/2232/calling-bash-commands-from-ruby

Ways to execute a shell script

cmd = "echo 'hi'" # Sample string that can be used

1. Kernel#`, commonly called backticks – `cmd`

This is like many other languages, including Bash, PHP, and Perl

Returns the result of the shell command

Docs: http://ruby-doc.org/core/Kernel.html#method-i-60

value = `echo 'hi'`
value = `#{cmd}`

2. Built-in syntax, %x( cmd )

Following the x character is a delimiter, which can be any character. If the delimiter is one of the characters ([{, or <, the literal consists of the characters up to the matching closing delimiter, taking account of nested delimiter pairs. For all other delimiters, the literal comprises the characters up to the next occurrence of the delimiter character. String interpolation #{ ... } is allowed.

Returns the result of the shell command, just like the backticks

Docs: http://www.ruby-doc.org/docs/ProgrammingRuby/html/language.html

value = %x( echo 'hi' )
value = %x[ #{cmd} ]

3. Kernel#system

Executes the given command in a subshell,

Return: true if the command was found and ran successfully, false otherwise

Docs: http://ruby-doc.org/core/Kernel.html#method-i-system

wasGood = system( "echo 'hi'" )
wasGood = system( cmd )

4. Kernel#exec

Replaces the current process by running the given external command.

Return: none, the current process is replaced and never continues

Docs: http://ruby-doc.org/core/Kernel.html#method-i-exec

exec( "echo 'hi'" )exec( cmd ) # Note: this will never be reached beacuse of the line above

Extra Advice

$?, which is the same as $CHILD_STATUS, accesses the status of the last system executed command if you use the backticks, system() or %{}. You can then access the exitstatus and pid properties

$?.exitstatus

More Reading

Hive: Make CLI output files comma delimited

bash >> hive -e ‘select * from some_Table’ | sed ‘s/[\t]/,/g’ > outputfile.txt

Here [\t] means Control+V and then the tab button, i.e.
sed ‘s//,/g’

Example:

[user@server]$ hive -e "use dbname ; select * from tablename" | sed ‘s/ /,/g’ > kpi_event_jan8.csv

Bash: $,* and special parameters

The shell treats several parameters specially. These parameters may only be referenced; assignment to them is not allowed.

*
Expands to the positional parameters, starting from one. When the expansion occurs within double quotes, it expands to a single word with the value of each parameter separated by the first character of the IFS special variable. That is, "$*" is equivalent to "$1c$2c…", where c is the first character of the value of the IFS variable. If IFS is unset, the parameters are separated by spaces. If IFS is null, the parameters are joined without intervening separators.

@
Expands to the positional parameters, starting from one. When the expansion occurs within double quotes, each parameter expands to a separate word. That is, "$@" is equivalent to "$1" "$2" …. If the double-quoted expansion occurs within a word, the expansion of the first parameter is joined with the beginning part of the original word, and the expansion of the last parameter is joined with the last part of the original word. When there are no positional parameters, "$@" and $@ expand to nothing (i.e., they are removed).

#
Expands to the number of positional parameters in decimal.

?
Expands to the exit status of the most recently executed foreground pipeline.

-
(A hyphen.) Expands to the current option flags as specified upon invocation, by the set builtin command, or those set by the shell itself (such as the -i option).

$
Expands to the process ID of the shell. In a () subshell, it expands to the process ID of the invoking shell, not the subshell.

!
Expands to the process ID of the most recently executed background (asynchronous) command.

0
Expands to the name of the shell or shell script. This is set at shell initialization. If Bash is invoked with a file of commands (see Shell Scripts), $0 is set to the name of that file. If Bash is started with the -c option (seeInvoking Bash), then $0 is set to the first argument after the string to be executed, if one is present. Otherwise, it is set to the filename used to invoke Bash, as given by argument zero.

_
(An underscore.) At shell startup, set to the absolute pathname used to invoke the shell or shell script being executed as passed in the environment or argument list. Subsequently, expands to the last argument to the previous command, after expansion. Also set to the full pathname used to invoke each command executed and placed in the environment exported to that command. When checking mail, this parameter holds the name of the mail file.

Bash: store the return value and/or output of a command in a variable

depends on whether you want to store the command’s output (either stdout, or stdout + stderr) or its exit status (0 to 255, with 0 typically meaning “success”).

If you want to capture the output, you use command substitution:

    output=$(command)      # stdout only; stderr remains uncaptured
    output=$(command 2>&1) # both stdout and stderr will be captured

If you want the exit status, you use the special parameter $? after running the command:

    command
    status=$?

If you want both:

    output=$(command)
    status=$?

The assignment to output has no effect on command‘s exit status, which is still in $?.

If you don’t actually want to store the exit status, but simply want to take an action upon success or failure, just use if:

    if command; then
        echo "it succeeded"
    else
        echo "it failed"
    fi

Or if you want to capture stdout as well as taking action on success/failure, without explicitly storing or checking $?:

    if output=$(command); then
        echo "it succeeded"
    ...

Quick Tip: Shred a Directory in 1 Line

find directory -type f | xargs shred –remove
normal (recursive) directory removal:
rm -rf directory

Quick Tip: IPTables port forwarding in 1 Line

iptables -A PREROUTING -t nat -p tcp -i eth0 –dport 88 -j DNAT –to 192.168.1.2:22

Quick Tip: Erase a drive (insecurely) in 1 Line

sudo dd if=/dev/zero of=/dev/sdx bs=512 count=1
where sdx is the drive you want to zero (this is an INsecure wipe)

Quick Tip: Remove Lines with sed in 1 Line

cat sample.txt | sed -e '1,15d'

Quick Tip: Change the size of a VirtualBox VM’s hdd in 1 Line

$ VBoxManage modifyhd <path to your vdi> –resize <new size in megabytes>

Sec: How to Import and Verfiy GPG Keys

How to Generate PGP Keys

Run the following commands on your local workstation:

* Note: You will have to run the same command to initialize the directory (step 1) and to generate the key (step 2).

  1. Initialize the GPG Directory*

    gpg --gen-key 

    gpg (GnuPG) 1.0.6; Copyright (C) 2001 Free Software Foundation, Inc.
    This program comes with ABSOLUTELY NO WARRANTY.
    This is free software, and you are welcome to redistribute it
    under certain conditions. See the file COPYING for details.
    gpg: /home/keymaster/.gnupg: directory created
    gpg: /home/keymaster/.gnupg/options: new options file created
    gpg: you have to start GnuPG again, so it can read the new options file 

  2. Generate a Private Key*

    > gpg --gen-key 

    gpg (GnuPG) 1.0.6; Copyright (C) 2001 Free Software Foundation, Inc.
    This program comes with ABSOLUTELY NO WARRANTY.
    This is free software, and you are welcome to redistribute it
    under certain conditions. See the file COPYING for details.
    gpg: /home/keymaster/.gnupg/secring.gpg: keyring created
    gpg: /home/keymaster/.gnupg/pubring.gpg: keyring created
    Please select what kind of key you want:
    (1) DSA and ElGamal? (default)
    (2) DSA (sign only)
    (4) ElGamal? (sign and encrypt)
    Your selection? 1
    DSA keypair will have 1024 bits.
    About to generate a new ELG-E keypair.
    minimum keysize is 768 bits
    default keysize is 1024 bits
    highest suggested keysize is 2048 bits
    What keysize do you want? (1024) 4096
    you really need such a large keysize? y
    Requested keysize is 4096 bits
    Please specify how long the key should be valid.
    0 = key does not expire
    = key expires in n days
    w = key expires in n weeks
    m = key expires in n months
    y = key expires in n years
    Key is valid for? (0) 0
    Key does not expire at all
    Is this correct (y/n)? y

    You need a User-ID to identify your key; the software constructs the user id
    from Real Name, Comment and Email Address in this form:
    "Heinrich Heine (Der Dichter) "
    Real name: Your Real Name
    Email address: youremail@foo.upenn.edu
    You selected this USER-ID:
    "Your Real Name"
    Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o
    You need a Passphrase to protect your secret key.
    Enter passphrase: pickapassphrase
    Repeat passphrase: pickapassphrase
    We need to generate a lot of random bytes. It is a good idea to perform
    some other action (type on the keyboard, move the mouse, utilize the
    disks) during the prime generation; this gives the random number
    generator a better chance to gain enough entropy.
    +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++. ...
    you may see
    ...
    Not enough random bytes available. Please do some other work to give
    the OS a chance to collect more entropy! (Need 290 more bytes)
    We need to generate a lot of random bytes. It is a good idea to perform
    some other action (type on the keyboard, move the mouse, utilize the
    disks) during the prime generation; this gives the random number
    generator a better chance to gain enough entropy.
    +++++++++++++++.+++++++++++++++.+++++++++++++++++++++++++.

  3. public and secret key created and signed.
  4. Generate and print a copy of your public key ID and fingerprint 

    >gpg --fingerprint 

    An an example of the fingerprint output, here is the output of the NMLUG Keymaster's fingerprint:

    pub 1024D/14931423 2003-02-13 New Mexico Linux User Group Keymaster
    Key fingerprint = 724D 2514 93EC 83D4 E5BC D534 C5BC C12C 1493 1423
    sub 2048g/42685AC7 2003-02-13