Shell Scripts usage

Color codes : http://misc.flogisoft.com/bash/tip_colors_and_formatting
Text to ascii generator : http://patorjk.com/software/taag/#p=display&f=Graffiti&t=Type%20Something

function test {
    "$@"
    local status=$?
    if [ $status -ne 0 ]; then
        echo "error with $1" >&2
    fi
    return $status
}
 
test command1

From http://stackoverflow.com/questions/5195607/checking-bash-exit-status-of-several-commands-efficiently

The aim here was to use the following ascii art :

                                 P___----....
                                ! __
                      ' ~~ ---.#..__ `  ~  ~    -  -  .   .:
                       `             ~~--.               .F~~___-__.
                       ;                   ,       .- . _!  
                      ,                     '       ;      ~ .
                     ,        ____           ;      ' _ ._    ; 
                    ,_ . - '___#,  ~~~ ---. _,   . '  .#'  ~ .;
                  =---==~~~    ~~~==--__     ; '~ -. ,#_     .'
                   '                     `~=.;           `  /
                                             '  '          '.      
                    '                         '               
            \                                  ' '            '
            `.`\    '                          . ;             ,
              \  `  '                            '             ;
               ;   '                           '               '
             /_ .,                           /   __...---./   '
             ',_,   __.--- ~~;#~ --..__    _'.-~;#     //  `.'
             / / ~~ .' .     #;         ~~  /// #;   //   /
           /    ' . __ .  ' ;#;_ .        ////.;#;./ ;  /
           \ .        /    ,##' /   _   /. '(/    ~||~\'
            \  ` - . /_ . -==-  ~ '   / (/ '     . ;;. ', 
           /' .       ' -^^^...--- ``(/'    _  '   '' `,;
##,. .#...(       '   .c  c .c  c  c.    '..      ;; ../ 
%%#%;,..##.\_                           ,;###;,. ;;.:##;,.    raf
%%%%########%%%%;,.....,;%%%%%%;,.....,;%%%%%%%%%%%%%%%%%%%%............

Into the bashrc at login, but only in interactive mode, and using colors. To use colors, you need to use echo -e, however, you will face difficulties with special characters. An excellent way around I found (do not remember where) was to use variables to bypass the problem. The only variable to modify is `, change them by ' :

if [ "$PS1" ]; then
echo
echo
toto="                                 P___----...."
echo -e "$toto"
toto='                                ! __'
echo -e "$toto"
toto="                      ' ~~ ---.#..__ '  ~  ~    -  -  .   .:"
echo -e "$toto"
toto="                       '   "
toto0="_"
toto1="         ~~--.               .F~~___-__."
echo -e "$toto\e[1m\e[38;5;202m$toto0\e[0m$toto1"
toto="                       ;  "
toto0="/_| _  / _ _"
toto1="     ,       .- . _!  "
echo -e "$toto\e[1m\e[38;5;202m$toto0\e[0m$toto1"
toto="                      ,  "
toto0="(  |//)()/ (-"
toto1="      '       ;      ~ ."
echo -e "$toto\e[1m\e[38;5;202m$toto0\e[0m$toto1"
toto="                     ,        ____           ;      ' _ ._    ; "
echo -e "$toto"
toto="                    ,_ . - '___#,  ~~~ ---. _,   . '  .#'  ~ .;"
echo -e "$toto"
toto="                  =---==~~~    ~~~==--__     ; '~ -. ,#_     .'"
echo -e "$toto"
toto="                   '                     '~=.;           '  /"
echo -e "$toto"
toto="                                             '  '          '.      "
echo -e "$toto"
toto="                    '                         '               "
echo -e "$toto"
toto="            \                                  ' '            '"
echo -e "$toto"
toto="            '.'\    '                          . ;             ,"
echo -e "$toto"
toto="              \  '  '    Experimental            '             ;"
echo -e "$toto"
toto="               ;   '    Cluster                '               '"
echo -e "$toto"
toto="             /_ .,                           /   __...---./   '"
echo -e "$toto"
toto="             ',_,   __.--- ~~;#~ --..__    _'.-~;#     //  '.'"
echo -e "$toto"
toto="             / / ~~ .' .     #;         ~~  /// #;   //   /"
echo -e "$toto"
toto="           /    ' . __ .  ' ;#;_ .        ////.;#;./ ;  /"
echo -e "$toto"
toto="           \ .        /    ,##' /   _   /. '(/    ~||~\'"
echo -e "$toto"
toto="            \  ' - . /_ . -==-  ~ '   / (/ '     . ;;. ', "
echo -e "$toto"
toto="           /' .       ' -^^^...--- ''(/'    _  '   '' ',;"
echo -e "$toto"
toto="##,. .#..."
toto0="(       '   .c  c .c  c  c.    '"
toto1=".."
toto2="      ;; "
toto3=".."
toto4="/ "
echo -e "\e[1m\e[38;5;27m$toto\e[0m$toto0\e[1m\e[38;5;27m$toto1\e[0m$toto2\e[1m\e[38;5;27m$toto3\e[0m$toto4"
toto="%%#%;,..##."
toto0="\_                           "
toto1=",;###;,."
toto2=" ;;"
toto3=".:##;,.    raf"
echo -e "\e[1m\e[38;5;27m$toto\e[0m$toto0\e[1m\e[38;5;27m$toto1\e[0m$toto2\e[1m\e[38;5;27m$toto3\e[0m"
toto="%%%%########%%%%;,.....,;%%%%%%;,.....,;%%%%%%%%%%%%%%%%%%%%............"
echo -e "\e[1m\e[38;5;27m$toto\e[0m"
echo
echo
fi;

Add this into your .bashrc or in the /etc/bashrc for all users, and you obtain the following result :

All :

๑•ิ.•ั๑ ๑۩۞۩๑ ♬✿.｡.:* ★ ☆ εїз℡❣·۰•●○●ōゃ ♥ ♡๑۩ﺴ ☜ ☞ ☎ ☏♡ ⊙◎ ☺ ☻✖╄ஐﻬ ► ◄ ▧ ▨ ♨ ◐ ◑ ↔ ↕ ▪ ▫ ☼ ♦ ▀ ▄ █▌ 
▐░ ▒ ▬♦ ◊ ◦ ☼ ♠♣ ▣ ▤ ▥ ▦ ▩ ◘ ◙ ◈ ♫ ♬ ♪ ♩ ♭ ♪ の ☆ → あ ぃ ￡ ❤＃ ＠ ＆ ＊ ❁ ❀ ✿ ✾ ❃ ✺ ❇ ❈ ❊ ❉ ✱ ✲ ✩ ✫
 ✬ ✭ ✮ ✰ ☆ ★ ✪ ¤ ☼ ☀ ☽ ☾ ❤ ♡ ღ☻ ☺ ❂ ◕ ⊕ ☉ Θ o O ♋ ☯ ㊝ ⊙ ◎◑ ◐ ۰ • ● ▪ ▫ ｡ ﾟ ๑ ☜ ☞ ☂ ♨ ☎ ☏ × ÷ ＝ ≠
 ≒ ∞ ˇ ± √ ⊥▶ ▷ ◀ ◁ ☀ ☁ ☂ ☃ ☄ ★ ☆ ☇ ☈ ☉ ☊ ☋ ☌ ☍ ☑ ☒☢ ☸ ☹ ☺ ☻ ☼ ☽ ☾ ♠ ♡ ♢ ♣ ♤ ♥ ♦ ♧ ♨ ♩ ✙ ✈ ✉ ✌ ✁
♝ ♞♯♩♪♫♬♭♮ ☎ ☏ ☪ ♈ ♨ ₪ ™ ♂✿ ♥ の ↑ ↓ ← → ↖ ↗ ↙ ↘ ㊣ ◎ ○ ● ⊕ ⊙ ○　 △ ▲ ☆ ★ ◇ ◆ ■ □ ▽ ▼ § ￥ 〒 ￠ ￡ ※
 ♀ ♂ &⁂ ℡ ↂ░ ▣ ▤ ▥ ▦ ▧ ✐✌✍✡✓✔✕✖ ♂ ♀ ♥ ♡ ☜ ☞ ☎ ☏ ⊙ ◎ ☺ ☻ ► ◄ ▧ ▨ ♨ ◐ ◑ ↔ ↕ ♥ ♡ ▪ ▫ ☼ ♦ ▀ ▄ █ ▌ ▐ 
░ ▒ ▬ ♦ ◊ ◘ ◙ ◦ ☼ ♠ ♣ ▣ ▤ ▥ ▦ ▩ ◘ ◙ ◈ ♫ ♬ ♪ ♩ ♭ ♪ ✄☪☣☢☠░ ▒ ▬ ♦ ◊ ◦ ♠ ♣ ▣ ۰•● ❤ ●•۰► ◄ ▧ ▨ ♨ ◐ ◑ ↔ ↕
 ▪ ▫ ☼ ♦♧♡♂♀♠♣♥❤☜☞☎☏⊙◎ ☺☻☼▧▨♨◐◑↔↕▪ ▒ ◊◦▣▤▥ ▦▩◘ ◈◇♬♪♩♭♪の★☆→あぃ￡Ю〓§♤♥▶¤๑⊹⊱⋛⋌⋚⊰⊹ ๑۩۩.. ..۩۩๑ 
๑۩۞۩๑ ✲ ❈ ✿ ✲ ❈ ➹ ~.~ ◕‿- ❣ ✚ ✪ ✣ ✤ ✥ ✦❉ ❥ ❦ ❧ ❃ ❂ ❁ ❀ ✄ ☪ ☣ ☢ ☠ ☭ღღღ ▶ ▷ ◀ ◁ ☀ ☁ ☂ ☃ ☄ ★
 ☆ ☇ ☈ ⊙ ☊ ☋ ☌ ☍ⓛⓞⓥⓔ๑•ิ.•ั๑ ๑۩۞۩๑ ♬✿ ☉♡ ♢ ♣ ♤ ♥ ♦ ♧ ♨ ♩ ✙✈ ✉ ✌ ✁ ✎ ✐ ❀ ✰ ❁ ❤ ❥ ❦❧ ➳ ➽ 
εїз℡❣·۰•●○●ゃōゃ♥ ♡๑۩ﺴ ☜ ☞ ☎ ☏♡ ⊙◎ ☺ ☻✖╄ஐﻬ ► ◄ ▧ ▨ ♨ ◐ ◑ ↔ ↕ ▪ ▫ ☼ ♦ ▀ ▄ █▌ ▐░ ▒ ▬♦ ◊ ◦ ☼ ♠♣ ▣ ▤ ▥ ▦ 
▩ ◘ ◙ ◈ ♫ ♬ ♪ ♩ ♭ ♪ の ☆ → あ ぃ ￡ ❤ ❁ ❀ ✿ ✾ ❃ ✺ ❇ ❈ ❊ ❉ ✱ ✲ ✩ ✫ ✬ ✭ ✮ ✰ ☆ ★ ✪ ¤ ☼ ☀ ☽ ☾ ❤ ♡ ღ☻ 
☺ ❂ ◕ ⊕ ☉ Θ o O ♋ ☯ ㊝ ⊙ ◎ ◑ ◐ ۰ • ● ▪ ▫ ｡ ﾟ ๑ ☜ ☞ ☂ ♨ ☎ ☏▶ ▷ ◀ ◁ ☀ ☁ ☂ ☃ ☄ ★ ☆ ☇ ☈ ☉ ☊ ☋ ☌ ☍ ☑
 ☒☢ ☸ ☹ ☺ ☻ ☼ ☽ ☾ ♠ ♝ ♞♯♩♪♫♬♭♮ ☎ ☏ ☪ ♈ ♨ ºº ₪ ¤ 큐 « »™ ♂✿ ♥ の ↑ ↓ ← → ↖ ↗ ↙ ↘ ㊣ ◎ ○ ● ⊕ ⊙ ○　 △ ▲ 
☆ ★ ◇ ◆ ■ □ ▽ ▼ § ￥〒 ￠ ￡ ※ ♀ ♂ © ® ⁂ ℡ ↂ░ ▣ ▤ ▥ ▦ ▧ ✐✌✍✡✓✔✕✖ ♂ ♀ ♥ ♡ ☜ ☞ ☎ ☏ ⊙ ◎ ☺ ☻ ► ◄ ▧ ▨
 ♨ ◐ ◑ ↔ ↕ ♥ ♡ ▪ ▫ ☼ ♦ ▀ ▄ █ ▌ ▐ ░ ▒ ▬ ♦ ◊ ◘ ◙ ◦ ☼ ♠ ♣ ▣ ▤ ▥ ▦ ▩ ◘ ◙ ◈ ♫ ♬ ♪ ♩ ♭ ♪ ✄☪☣☢☠㊊㊋㊌㊍㊎㊏ ㊐㊑
㊒㊓㊔㊕㊖㊗㊘㊜㊝㊞㊟㊠㊡㊢ ㊣㊤㊥㊦㊧㊨㊩㊪㊫㊬㊭㊮㊯㊰✗✘✚✪✣✤✥✦✧✩✫✬✭✮✯✰ ✱✲✳❃❂❁❀✿✾✽✼✻✺✹✸✷ 
✶✵✴❄❅❆❇❈❉❊❋❖☀☂☁【】┱ ┲ ❣ ✪ ✣ ✤ ✥ ✦ ❉ ❥ ❦ ❧ ❃ ❂ ❁ ❀ ✄ ☪ ☣ ☢ ☠ ☭ ♈ ➸ ✓ ✔ ✕ ✖ .: ◢ ◣ ◥ ◤ ▽ ▧ 
▨ ▣ ▤ ▥ ▦ ▩ ◘ ◙ ▓ ▒ ░ ™ ℡ 凸 の ๑۞๑ ๑۩ﺴ ﺴ۩๑ o(‧”’‧)o ❆ べò⊹⊱⋛⋋ ⋌⋚⊰⊹ ⓛⓞⓥⓔ ☀ ☼ ☜ ☞ ⊙® ◈ ♦ ◊ ◦ ◇ ◆ 
εїз❃❂❁❀✿✾✽✼✻✺✹✸✷ ✶✵✴❄❅❆❇❈❉ ❊❋❖❤❥❦❧↔ ↕ ▪ → ︷╅╊✿ (¯`•._.• •._.•´¯)(¯`•¸•´¯) ❤`•.¸¸.•´´¯`•• 
.¸¸.•´¯`•.•●•۰• ••.•´¯`•.•• ••.•´¯`•.••—¤÷(`[¤* *¤]´)÷¤——(•·÷[ ]÷·•)— ①②③④⑤⑥⑦⑧⑨⑩ ⑪⑫⑬⑭⑮⑯⑰⑱⑲⑳ ⒶⒷⒸⒹⒺⒻ 
ⒼⒽⒾⒿⓀⓁ ⓂⓃⓄⓅⓆⓇ ⓈⓉⓊⓋⓌⓍ ⓎⓏ ⓐⓑⓒⓓⓔⓕ ⓖⓗⓘⓙⓚⓛ ⓜⓝⓞⓟⓠⓡ ⓢⓣⓤⓥⓦⓧ ⓨⓩ(⊙▂⊙✖ )(づ ￣ ³￣)づ 
( c//”-}{-*\x)(-’๏_๏’-)(◐ o ◑ )(⊙…⊙ )๏[-ิ_•ิ]๏(•ิ_•ิ)(•ิ_•ิ) (/•ิ_•ิ)(︶︹︺)

Arrows:

← 	↑ 	→ 	↓ 	↔ 	↕ 	↖ 	↗ 	↘ 	↙ 	↚ 	↛ 	↜ 	↝ 	↞ 	↟
↠ 	↡ 	↢ 	↣ 	↤ 	↥ 	↦ 	↧ 	↨ 	↩ 	↪ 	↫ 	↬ 	↭ 	↮ 	↯
↰ 	↱ 	↲ 	↳ 	↴ 	↵ 	↶ 	↷ 	↸ 	↹ 	↺ 	↻ 	↼ 	↽ 	↾ 	↿
⇀ 	⇁ 	⇂ 	⇃ 	⇄ 	⇅ 	⇆ 	⇇ 	⇈ 	⇉ 	⇊ 	⇋ 	⇌ 	⇍ 	⇎ 	⇏
⇐ 	⇑ 	⇒ 	⇓ 	⇔ 	⇕ 	⇖ 	⇗ 	⇘ 	⇙ 	⇚ 	⇛ 	⇜ 	⇝ 	⇞ 	⇟
⇠ 	⇡ 	⇢ 	⇣ 	⇤ 	⇥ 	⇦ 	⇧ 	⇨ 	⇩ 	⇪ 	⇫ 	⇬ 	⇭ 	⇮ 	⇯
⇰ 	⇱ 	⇲ 	⇳ 	⇴ 	⇵ 	⇶ 	⇷ 	⇸ 	⇹ 	⇺ 	⇻ 	⇼ 	⇽ 	⇾ 	⇿
⟰ 	⟱ 	⟲ 	⟳ 	⟴ 	⟵ 	⟶ 	⟷ 	⟸ 	⟹ 	⟺ 	⟻ 	⟼ 	⟽ 	⟾ 	⟿
⤀ 	⤁ 	⤂ 	⤃ 	⤄ 	⤅ 	⤆ 	⤇ 	⤈ 	⤉ 	⤊ 	⤋ 	⤌ 	⤍ 	⤎ 	⤏
⤐ 	⤑ 	⤒ 	⤓ 	⤔ 	⤕ 	⤖ 	⤗ 	⤘ 	⤙ 	⤚ 	⤛ 	⤜ 	⤝ 	⤞ 	⤟
⤠ 	⤡ 	⤢ 	⤣ 	⤤ 	⤥ 	⤦ 	⤧ 	⤨ 	⤩ 	⤪ 	⤫ 	⤬ 	⤭ 	⤮ 	⤯
⤰ 	⤱ 	⤲ 	⤳ 	⤴ 	⤵ 	⤶ 	⤷ 	⤸ 	⤹ 	⤺ 	⤻ 	⤼ 	⤽ 	⤾ 	⤿
⥀ 	⥁ 	⥂ 	⥃ 	⥄ 	⥅ 	⥆ 	⥇ 	⥈ 	⥉ 	⥊ 	⥋ 	⥌ 	⥍ 	⥎ 	⥏
⥐ 	⥑ 	⥒ 	⥓ 	⥔ 	⥕ 	⥖ 	⥗ 	⥘ 	⥙ 	⥚ 	⥛ 	⥜ 	⥝ 	⥞ 	⥟
⥠ 	⥡ 	⥢ 	⥣ 	⥤ 	⥥ 	⥦ 	⥧ 	⥨ 	⥩ 	⥪ 	⥫ 	⥬ 	⥭ 	⥮ 	⥯
⥰ 	⥱ 	⥲ 	⥳ 	⥴ 	⥵ 	⥶ 	⥷ 	⥸ 	⥹ 	⥺ 	⥻ 	⥼ 	⥽ 	⥾ 	⥿

Geometric shapes:

■ 	□ 	▢ 	▣ 	▤ 	▥ 	▦ 	▧ 	▨ 	▩ 	▪ 	▫ 	▬ 	▭ 	▮ 	▯
▰ 	▱ 	▲ 	△ 	▴ 	▵ 	▶ 	▷ 	▸ 	▹ 	► 	▻ 	▼ 	▽ 	▾ 	▿
◀ 	◁ 	◂ 	◃ 	◄ 	◅ 	◆ 	◇ 	◈ 	◉ 	◊ 	○ 	◌ 	◍ 	◎ 	●
◐ 	◑ 	◒ 	◓ 	◔ 	◕ 	◖ 	◗ 	◘ 	◙ 	◚ 	◛ 	◜ 	◝ 	◞ 	◟
◠ 	◡ 	◢ 	◣ 	◤ 	◥ 	◦ 	◧ 	◨ 	◩ 	◪ 	◫ 	◬ 	◭ 	◮ 	◯
◰ 	◱ 	◲ 	◳ 	◴ 	◵ 	◶ 	◷ 	◸ 	◹ 	◺ 	◻ 	◼ 	◽ 	◾ 	◿

╮(─▽─)╭ 	¯\_(ツ)_/¯ 	٩(-̮̮̃-̃)۶

ヽ（´ー｀）┌ 	┗┐ヽ(′Д、`*)ﾉ┌┛ 	ヽ( ´¬`)ノ

┗┃・ ■ ・┃┛ 	ヾ(´A｀)ノﾟ 	ヽ（・＿・；)ノ

ヽ(　￣д￣;)ノ 	＼（〇_ｏ）／ 	ヽ(。_°)ノ

＼(;´□｀)/ 	ヾ(*´ー`)ノ 	ヽ(‘ー`)ノ

ヽ(ー_ー )ノ 	ヽ(´～｀；） 	┐(‘～`；)┌

ヽ（*ω。）ノ 	(;´・`)>   	（＾～＾）

٩(-̮̮̃•̃)۶ 	     ┐(￣ー￣)┌    	t(ツ)_/¯

┐(￣ヮ￣)┌ 	╰(　´◔　ω　◔ `)╯ 	     ಠ_ಠ

ლ(ಠ_ಠლ) 	¯\(°_o)/¯ 	╮(╯▽╰)╭

When using tools like KCacheGrind, you will encounter difficulties because KGhostView has been replaced by Okular. To bypass it, and make your tools use Okular: (note that this “bypass” can be used with all programs, including compilers!)

mkdir $HOME/bin
touche $HOME/bin/kghostview

edit the file, using vim for example :

vim $HOME/bin/kghostview

add :

#!/bin/bash
okular "$@"

save and close, and make it executable :

chmod +x $HOME/bin/kghostview

add it to your PATH :

export PATH=$HOME/bin/:$PATH

Now you can use KCacheGrind or other tools that need to use KGhostView, they will use okular instead.

Redirect everything to file, even when using make or configure :

> output.txt 2>&1

To add a string at the end of file (for example, adding an export at the end of your bashrc to make it permanent) :

echo 'export PATH=/soft/gnuplot-4.4/bin:$PATH' >> ~/.bashrc

General redirection table :

To wget multiple of files at once, if the files have the same names and differ only by a numerous, and if you are using bash > 4.0 :

wget http://theadress.com/thegile{001..030}.dat 

Read a file line per line, line is stored in $line, and work on values (users), a progress is displayed :

while read line  
do
   compteur=$(expr $compteur + 1)
   echo -ne "User $compteur/$nb_ldap_users\r"
   # Get ids
   user_uid=$(id "$line" | awk -F'(' '{print $2;}' | awk -F')' '{print $1;}')
   user_gid=$(id "$line" | awk -F'(' '{print $3;}' | awk -F')' '{print $1;}')
   # Get home from passwd
   user_home=$(getent passwd "$line" | cut -d: -f6)
   # Get real home position
   # First check if home exist
   ls -l "$user_home" 2>> errors > /dev/null
   if [ $? -eq 0 ];then
      user_home_real=$(ls -l "$user_home" | awk -F' ' '{print $10;}')
      # Get real home uid and gid and check them
      user_home_real_uid=$(ls -ld $user_home_real | awk -F' ' '{print $3;}')
      user_home_real_gid=$(ls -ld $user_home_real | awk -F' ' '{print $4;}')
      # check
      if [ "$user_uid" != "$user_home_real_uid" ]; then
         echo "Bad home uid for user $line. Real : $user_uid, got : $user_home_real_uid " >> errors
      fi 
      if [ "$user_gid" != "$user_home_real_gid" ]; then
         echo "Bad home gid for user $line. Real : $user_gid, got : $user_home_real_gid " >> errors
      fi
   else
      echo "Could not find home for user $line" >> errors   
   fi
 
   # Write result
   echo $line $user_gid $user_home $user_home_real $user_home_real_uid $user_home_real_gid  >> result
done < ldap_users
echo -ne '\n'

Now, using the same, to get the size of all repertories listed in a text file :

rm size.txt; compteur=0
# Get informations on user
while read line  
do
   compteur=$(expr $compteur + 1)
   echo -ne "User $compteur/516\r"
   du -ch -sB K "$line" | grep total | awk -F' ' '{print $1;}' >> size.txt  
done < home_users_nr_d
echo -ne '\n'

All size are noted like “132K\n”, easy to manipulate after.

In a shell, you can 'pipe' output to another command. tutu.dat contains :

Y X
1.2 3.4
2.3 3.5
-0.6 3.6

To extract the value 2.3 :

grep 3.4 tyty

Result :

1.2 3.4

To take only the first value (1.2), ask awk to keep only the first value of the line :

grep 3.4 tyty | awk -F' ' '{print $1;}'

Result :

1.2

If the separator is different: echo “1.2/1.3” | awk -F'/' '{print $1;}', it still gives 1.2. To sum all values of the first row : two ways, remove the first line when piping, or remove all lines using symbol Y for example.

First way :

sed '1d' tyty

Result :

1.2 3.4
2.3 3.5
-0.6 3.6

(of course, replace 1d with 2d for second line, or with '5,10d' for lines 5 and 10, etc)

Second way :

grep -v 'Y' tyty

Result :

1.2 3.4
2.3 3.5
-0.6 3.6

Note that to add another string, use \| :

grep -v 'Y\|1.2' tyty

Result :

2.3 3.5
-0.6 3.6

Then, pipe it on awk to recover only first row :

grep -v 'Y' tyty | awk -F' ' '{print $1;}'

Result :

1.2
2.3
-0.6

Then, you want all of this to be the same as an addition expression, compatible with bc command (the gnu calculator).

grep -v 'Y' tyty | awk -F' ' '{print $1;}' | paste -sd+

Result :

1.2+2.3+-0.6

Of course, using -sd* will give you 1.2*2.3*-0.6, etc. Last step is to pipe it to bc :

grep -v 'Y' tyty | awk -F' ' '{print $1;}' | paste -sd+ | bc

Result :

2.9

Tip: do no try to merge steps. You can do very shorts commands that do the same thing. Very useful to show off, but bad idea to explain to others, to recover a single step, or simply to remember how it works few months later.

To put this into a variable :

myvalue=$( grep -v 'Y' tyty | awk -F' ' '{print $1;}' | paste -sd+ | bc )

echo $myvalue should give 2.9.

In bash, there are multiple ways to use loops, by I prefer to use the way like C :

#!/bin/bash 
for (( c=1; c<=5; c++ )) 
do 
   echo "Welcome $c times" 
done 

#!/bin/bash 
for (( c=2; c<=10; c=c+2 )) 
do 
   echo "Welcome $c times" 
done 

etc. Use if statement to break the loop :

if [[ $myvalue == 0 ]] 
  then 
    echo "Test $c Passed" 
    break 
  else 
    echo "Test $c Failed, Restart" 
fi 

In this example, we can start a program, and recover its exit statue to check if it executed and exited normally. Then, we break the loop if yes, or we restart it if no.

for (( c=1; c<=30; c++ )) # 30 Tests 
do 
 
grep -i "$c" Test 
 
RETVAL=$? 
if [[ $RETVAL == 0 ]] 
  then 
    echo "Test $c Passed" 
    break 
  else 
    echo "Test $c Failed, Restart" 
fi 
done 

$? contains the value of the exist status of the last command.

If you want to pass variables/arguments to launched/son scripts, and recover variables from them. Consider the following script, in myscript.sh (do not forget to do a chmod +x to execute it):

In_Buffer="$@" 
 
val1=$( echo $In_Buffer | awk '{print $1;}') 
val2=$( echo $In_Buffer | awk '{print $2;}') 
val3=$( echo $In_Buffer | awk '{print $3;}') 
 
echo $val1 $val2 $val3 

./myscript.sh

give nothing.

./myscript.sh 10 13 16 9

returns :

10 13 16

The son's script has recovered the values. They can be seen has subroutine/function arguments. Now, to return values, just do a final echo in the son script, using a special word, let's say “SON_RETURN”.

In_Buffer="$@" 
 
val1=$( echo $In_Buffer | awk '{print $1;}') 
val2=$( echo $In_Buffer | awk '{print $2;}') 
val3=$( echo $In_Buffer | awk '{print $3;}') 
 
echo $val1 $val2 $val3 
 
sumval=$(echo "$val1+$val2+$val3" | bc) 
 
echo '\nSON_RETURN' $sumval 

Now, using the command you get :

./myscript.sh 10 13 16
10 13 16
\nSON_RETURN 39

You can then recover this result to a variable :

returnval=$(./myscript.sh 10 13 16)

then

echo -e $returnval | grep 'SON_RETURN'
SON_RETURN 39

So a simple awk will allow you to extract 39 or other returned variables. Note that the -e argument to echo allow the \n character to be used has a return. Without it, the grep could not work properly.

Now, I want to use gnuplot to plot things into png files. You can use everything seen before to create gnuplot scripts, and launch them.

Suppose we want to plot the tyty file seen above. Create a file “gnuplotscript”, and add :

# Set png terminal and png size 
set terminal png size 640,480 
# Set output file name 
set output 'myfile.png' 
# Ask gnuplot to use first line as title name 
set key autotitle columnhead 
# Plot (exit automaticaly after) 
plot 'tyty' with line 

Now, use gnuplot :

gnuplot gnuplotscript

And it will output your graph in png.

I present here a small project I worked on. The idea was to be able to monitor multiple systems, using only simple user accounts and without installing anything on remote machines. This script had to render everything in HTML, and had to actualise information every 15 minutes.

First, we need to acquire infos on the system :

Date :

current_date=$(date +"%s")

Network :

- ipv4 and ipv6 in and out :
ipv4_in=$( echo "`egrep -v bond\|lo\|face\|Inter /proc/net/dev | awk -F' ' '{print $2;}' | paste -sd+ | bc` * 8" | bc )
ipv4_out=$( echo "`egrep -v bond\|lo\|face\|Inter /proc/net/dev | awk -F' ' '{print $10;}' | paste -sd+ | bc` * 8" | bc )
ipv6_in=$( echo "`egrep Ip6InOctets /proc/net/snmp6 | awk -F' ' '{print $2;}'` * 8" | bc )
ipv6_out=$( echo "`egrep Ip6OutOctets /proc/net/snmp6 | awk -F' ' '{print $2;}'` * 8" | bc )
totalip_in=$( echo "$ipv4_in + $ipv6_in" | bc )
totalip_out=$( echo "$ipv4_out + $ipv6_out" | bc )
totalip_in=$( echo "$totalip_in*0.001" | bc ) # in Kilooctets (from Octets)
totalip_out=$( echo "$totalip_out*0.001" | bc ) # in Kilooctets (from Octets)

Flux if you now the previous get :

difftotalip_in=$( echo "scale=4;( $totalip_in - $previous_totalip_in ) / ( $current_date - $previous_date )" | bc) # in Ko/s
difftotalip_out=$( echo "scale=4;( $totalip_out - $previous_totalip_out ) / ( $current_date - $previous_date )" | bc) # in Ko/s

Ssh :

Number of correct connexions for the user spheniscus :

grep -i "spheniscus" /var/log/auth.log | wc -l

Number of invalid user authentications and failed password :

grep -i "Invalid user" /var/log/auth.log | wc -l
grep -i "Failed password" /var/log/auth.log | wc -l

I/O :

Total HDD I/O :

totalio_write=$(echo "`grep -i "sda " /proc/diskstats | awk '{print $10;}'` * 512 * 0.001" | bc) # In Ko
totalio_read=$(echo "`grep -i "sda " /proc/diskstats | awk '{print $6;}'` * 512 * 0.001" | bc) # In Ko

Flux if you now the last get :

difftotalio_write=$( echo "scale=4;( $totalio_write - $previous_totalio_write ) / ( $current_date - $previous_date )" | bc) # in Ko/s
difftotalio_read=$( echo "scale=4;( $totalio_read - $previous_totalio_read ) / ( $current_date - $previous_date )" | bc) # in Ko/s

CPU :

cpu_idle=$( echo "`top -b -n 5 | grep -i "Cpu(s)" | tr -d '%id,' | awk '{print $2;}' | awk 'NR==5'`");
cpu_used=$( echo "100 - $cpu_idle" | bc )

Memory :

MemTotal=$( echo "`grep -i "MemTotal" /proc/meminfo | tr -d 'kB' | awk '{print $2;}'`")
MemFree=$( echo "`grep -i "MemFree" /proc/meminfo | tr -d 'kB' | awk '{print $2;}'`")
Cached=$( echo "`grep -i "Cached" /proc/meminfo | tr -d 'kB' | awk '{print $2;}' | awk 'NR==1'`")
Buffers=$( echo "`grep -i "Buffers" /proc/meminfo | tr -d 'kB' | awk '{print $2;}'`")
MemUsed=$( echo "$MemTotal - $MemFree - $Buffers - $Cached" | bc )
MemCached=$( echo "$Buffers + $Cached" | bc )
MemUsed=$( echo "$MemUsed*0.001" | bc ) # in MegaBytes
MemCached=$( echo "$MemCached*0.001" | bc ) # in MegaBytes
MemFree=$( echo "$MemFree*0.001" | bc ) # in MegaBytes

SwapTotal=$( echo "`grep -i "SwapTotal" /proc/meminfo | tr -d 'kB' | awk '{print $2;}'`")
SwapFree=$( echo "`grep -i "SwapFree" /proc/meminfo | tr -d 'kB' | awk '{print $2;}'`")
SwapUsed=$( echo "$SwapTotal - $SwapFree" | bc )
SwapUsed=$( echo "$SwapUsed*0.001" | bc ) # in MegaBytes

We then launch these information gathering scripts using ssh, and using input and output system seen in the basis section of this page (to keep in memory previous records and date, to evaluate flux for example). I assume you already have a key authentication system for ssh, and that you do not have passphrase (bad idea on a server, but ok four your monitoring station).

The script will be (file name : ssh_llcheck.bash):

# /bin/bash 
 
In_Buffer="$@" 
 
previous_date=$( echo $In_Buffer | awk '{print $1;}') 
previous_totalip_in=$( echo $In_Buffer | awk '{print $2;}') 
previous_totalip_out=$( echo $In_Buffer | awk '{print $3;}') 
previous_totalio_write=$( echo $In_Buffer | awk '{print $4;}') 
previous_totalio_read=$( echo $In_Buffer | awk '{print $5;}') 
 
current_date=$(date +"%s") 
 
# 
#   NETWORK 
# 
 
ipv4_in=$( echo "`egrep -v bond\|lo\|face\|Inter /proc/net/dev | awk -F' ' '{print $2;}' | paste -sd+ | bc` * 8" | bc ) 
ipv4_out=$( echo "`egrep -v bond\|lo\|face\|Inter /proc/net/dev | awk -F' ' '{print $10;}' | paste -sd+ | bc` * 8" | bc ) 
ipv6_in=$( echo "`egrep Ip6InOctets /proc/net/snmp6 | awk -F' ' '{print $2;}'` * 8"  | bc ) 
ipv6_out=$( echo "`egrep Ip6OutOctets /proc/net/snmp6 | awk -F' ' '{print $2;}'` * 8"  | bc ) 
totalip_in=$( echo "$ipv4_in + $ipv6_in" | bc ) 
totalip_out=$( echo "$ipv4_out + $ipv6_out" | bc ) 
totalip_in=$( echo "$totalip_in*0.001" | bc ) # in Kilooctets (from Octets) 
totalip_out=$( echo "$totalip_out*0.001" | bc ) # in Kilooctets (from Octets) 
difftotalip_in=$( echo "scale=4;( $totalip_in - $previous_totalip_in ) / ( $current_date - $previous_date )" | bc) # in Ko/s 
difftotalip_out=$( echo "scale=4;( $totalip_out - $previous_totalip_out ) / ( $current_date - $previous_date )" | bc) # in Ko/s 
 
 
# 
#   HARDWARE 
# 
 
totalio_write=$(echo  "`grep -i "sda " /proc/diskstats | awk '{print $10;}'` * 512 * 0.001" | bc) # In Ko 
totalio_read=$(echo  "`grep -i "sda " /proc/diskstats | awk '{print $6;}'` * 512 * 0.001" | bc) # In Ko 
difftotalio_write=$( echo "scale=4;( $totalio_write - $previous_totalio_write ) / ( $current_date - $previous_date )" | bc) # in Ko/s 
difftotalio_read=$( echo "scale=4;( $totalio_read - $previous_totalio_read ) / ( $current_date - $previous_date )" | bc) # in Ko/s 
 
cpu_idle=$( echo "`top -b -n 5 | grep -i "Cpu(s)" | tr -d '%id,' | awk '{print $5;}' | awk 'NR==5'`"); 
cpu_used=$( echo "100 - $cpu_idle" | bc ) 
 
MemTotal=$( echo "`grep -i "MemTotal" /proc/meminfo | tr -d 'kB' | awk '{print $2;}'`") 
MemFree=$( echo "`grep -i "MemFree" /proc/meminfo | tr -d 'kB' | awk '{print $2;}'`") 
Cached=$( echo "`grep -i "Cached" /proc/meminfo | tr -d 'kB' | awk '{print $2;}' | awk 'NR==1'`") 
Buffers=$( echo "`grep -i "Buffers" /proc/meminfo | tr -d 'kB' | awk '{print $2;}'`") 
MemUsed=$( echo "$MemTotal - $MemFree - $Buffers - $Cached" | bc ) 
MemCached=$( echo "$Buffers + $Cached" | bc ) 
MemUsed=$( echo "$MemUsed*0.001" | bc ) # in MegaBytes 
MemCached=$( echo "$MemCached*0.001" | bc ) # in MegaBytes 
MemFree=$( echo "$MemFree*0.001" | bc ) # in MegaBytes 
 
SwapTotal=$( echo "`grep -i "SwapTotal" /proc/meminfo | tr -d 'kB' | awk '{print $2;}'`") 
SwapFree=$( echo "`grep -i "SwapFree" /proc/meminfo | tr -d 'kB' | awk '{print $2;}'`") 
SwapUsed=$( echo "$SwapTotal - $SwapFree" | bc ) 
SwapUsed=$( echo "$SwapUsed*0.001" | bc ) # in MegaBytes 
 
 
# First echo, returned variables 
echo "$current_date $totalip_in $totalip_out $totalio_write $totalio_read" 
# Middle echo, to cut the returned line 
echo "Next" 
# Second echo, to be written in the Log File 
echo "$( date +%Y-%d-%m-%H:%M ) $cpu_used $difftotalip_in $difftotalip_out $MemUsed $MemCached $MemFree $SwapUsed $totalio_write $totalio_read" 

Now, you can use an upper script that will call this script for multiple machines, and store information in directories (one per server) :

#!/bin/bash 
 
# 
# VARIABLES 
# 
 
#TallGeese 
TallGeese_previous_date="0" 
TallGeese_previous_totalip_in="0" 
TallGeese_previous_totalip_out="0" 
TallGeese_previous_totalio_write="0" 
TallGeese_previous_totalio_read="0" 
TallGeese_repertory_name="TallGeese" 
TallGeese_login="spheniscus" 
TallGeese_ip="tallgeese.fr" 
 
# 
# MAIN LOOP 
# 
 
 while true 
 do 
    echo "Check" 
 
       # ssh the script 
       Out_buffer=$(ssh " $TallGeese_login@$TallGeese_ip" 'bash -s' < ssh_llcheck.bash $TallGeese_previous_date $TallGeese_previous_totalip_in \
         $TallGeese_previous_totalip_out $TallGeese_previous_totalio_write $TallGeese_previous_totalio_read) 
 
       # store values for next check 
       TallGeese_previous_date=$( echo $Out_buffer | awk '{print $1;}') 
       TallGeese_previous_totalip_in=$( echo $Out_buffer | awk '{print $2;}') 
       TallGeese_previous_totalip_out=$( echo $Out_buffer | awk '{print $3;}') 
       TallGeese_previous_totalio_write=$( echo $Out_buffer | awk '{print $4;}') 
       TallGeese_previous_totalio_read=$( echo $Out_buffer | awk '{print $5;}') 
 
       # log values 
       echo $Out_buffer | awk 'match($0,"Next"){print substr($0,RSTART+5)}' >> "./$TallGeese_repertory_name/Log.dat" 
 
       # plot 
       cd "$TallGeese_repertory_name" 
       gnuplot plotscript.gmp 
       cd .. 
 
    echo "Check OK, Sleep" 
    sleep 10s 
 done 

The script to plot values is plotscript.gmp :

set terminal png size 642,462 small 
set key autotitle columnheader 
set timefmt "%Y-%d-%m-%H:%M" 
set xdata time 
set format x "%H:%M" 
set grid 
 
 
set output 'CPU.png' 
plot 'Log.dat' using 1:2 w l linewidth 3 lc rgb "#f7d301" 
 
set output 'Net.png' 
plot 'Log.dat' using 1:3 w l linewidth 3 lc rgb "#f7d301",  '' using 1:4 w l linewidth 3 lc rgb "#1cac77" 
 
set output 'Memory.png' 
plot 'Log.dat' using 1:7 w l linewidth 3 lc rgb "#f7d301", '' using 1:6 w l linewidth 3 lc rgb "#1f75fe", '' using 1:5 w l linewidth 3 lc rgb "#ff2d2e" 
 
set style fill transparent solid 0.5 noborder 
 
quit 

And I first prepared the file log.dat in the directory of the server, by adding at top :

Date %CPU NetTrafic_in NetTrafic_out Memory_used Memory_cached Memory_free Swap_used IO_write IO_read 

(Important, do not forget to pass line after the first one.)

This gives you png like that, that you can incorporate into an HTML display based system :