毎日叩ける
シェル芸を覚えよう！

Transcript

1. ຖ೔ୟ͚Δ
 γΣϧܳΛ֮͑Α͏ʂ ʙேͷγΣϧܳษڧձʙ Yasuhiro Yamada (@grethlen) 2016-08-27 ୈ6ճ΋͏ॳ৺ऀ޲͚Ͱͳ͍ͳΜͯݴΘͳ͍Α
 ઈରޕલͷγΣϧษڧձ/ୈ24ճ̋̋őཪ൪૊γΣϧܳษڧձ ޕલͷ෦

2. $ whoami ͳ·͑: Yasuhiro Yamada ͓͠͝ͱ: ΞϓϦέʔγϣϯΤϯδχΞ
 ిࢠܾࡁ·ΘΓ͕ຊۀ ͠ΎΈ: Ξχϝؑ৆

   γΣϧܳ Twitter: @grethlen Qiita: greymd ةݥ͡Όͳ͍Α

3. γΣϧܳྺ ܳྺ ຊ֨తʹ୺຤։͖࢝Ίͨͷ͕͓͓Αͦ6೥લ͘Β͍ γΣϧܳؔ࿈Ͱ։ൃ egzact — γΣϧͷऑ఺Λิ͏ίϚϯυୡ
 Github: greymd/egzact cureutils

   — ࣍ੈ୅ITਓࡐҭ੒πʔϧ
 Github: greymd/cureutils هࣄ(Qiita) ʲͨͷ͍͠ͳʳ༷ʑͳίϚϯυୡΛԿ΋ߟ͑ͣʹͭͳ͍Ͱ༡Ϳ (1079 stock) ςετͷ਺ΛݮΒͦ͏ʂϓϦΩϡΞͰֶͿPICT (305 stock) ϓϦΩϡΞͰֶͿϫϯϥΠφʔWebεΫϨΠϐϯά (122 stock)

4. γΣϧܳͷֶͼํ ϘτϜΞοϓ
 →஌ࣝΛ౔୆͔ΒݻΊΔ →ྫ: ମܥతʹ·ͱ·ͬͨ
 ࢿྉΛಡΉ τοϓμ΢ϯ →΍Γ͍ͨ͜ͱΛઌʹܾΊΔ →࠷௿ݶඞཁͳ஌͔ࣝΒಘΔ →ྫ:

   άάͬͨ݁ՌΛ
 ϦόʔεΤϯδχΞϦϯά ඪ४ೖग़ྗɺ੍ޚߏ଄
 ૊ࠐΈίϚϯυɺ֎෦ίϚϯυɺ ύΠϓϥΠϯɺऴྃεςʔλεɺ ύϥϝʔλల։……etc ֦ுਖ਼نදݱɺ AWKͷGolfɺevalࠇຐज़ ίϚϯυͷෳ߹ɺ ศརͳΦϓγϣϯ……etc ΍Γ͍ͨ ͜ͱ

5. ϘτϜΞοϓͳΞϓϩʔν ମܥతʹ·ͱ·ͬͨࢿྉΛಡΉ • γΣϧεΫϦϓτجຊϦϑΝϨϯε
 (ஶ: ࢁ৿ ৎൣ) • Software DesignͰϐϯͱ͖ͨΒങ͏


   (ྫ: 2016೥6݄߸ bash ࠶ೖ໳) • γΣϧܳຊ • ͳͲ…(࠷ޙͷํͷεϥΠυʹ·ͱΊͯ·͢)

6. τοϓμ΢ϯͳΞϓϩʔν ΍Γ͍ͨ͜ͱΛάάΔ →ग़͖ͯͨίϚϯυΛϦόʔεΤϯδχΞϦϯά ษڧձͰ՝୊Λղ͘ → ෼͔Βͳ͍՝୊͸γΣϧܳਓʹҰ୴΍ͬͯ΋Β͏ → ؾʹͳͬͨϑϨʔζΛ֮͑ͯؼͬͯ࢖ͬͯΈΔ ຖ೔୺຤Λ։͍֮ͯ͑ͨϑϨʔζΛୟ͘ͷΦεεϝ ΧϨϯμʔͷ֬ೝͱ͔ɺ


   ੜ׆ʹඞཁͳ࡞ۀ͕GOODʢޙड़ʣ

7. ͲͬͪͷΞϓϩʔν͕͍͍ͷʁ ࠓͷ্ୡϨϕϧʹԠͯ͡όϥϯεΛऔΔͷ͕େ੾ ϘτϜΞοϓ͚ͩͩͱख͕ಈ͔ͳ͍ɻ τοϓμ΢ϯ͚ͩͩͱ஌͕ࣝ๘ͷ૥ʹɻ

8. ϓνγΣϧܳษڧձΛ
 ͜Ε͔Β։͖·͢

9. ਐΊํ(1/3) ೖ໳ऀʢࠇ͍ը໘͕·ͩ͜Θ͍͓ͱ΋ͩͪʣ ը໘ʹग़ΔίϚϯυࣸܦ͋Δ͍͸άάͬͯԼ͍͞ɻ
 ๞͖ͨΒมଶΛ؍࡯ͯ͠Լ͍͞ɻ͜Θ͘ͳ͍Ͱ͢ɻ ॳ৺ऀʢgrepͰϩάΛ֬ೝ͢Δͷ͕͍͍ͤͥͷ͓ͱ΋ͩͪʣ ͋ͨΒ͍͠ίϚϯυͱΦϓγϣϯΛ֮͑·͠ΐ͏ɻ ޙͰࢿྉΛݟฦͯ͠Έ·͠ΐ͏ɻ தڃऀʢxargsͰվߦΛআ͘ͷ͕श׳Խͯ͠Δ͓ͱ΋ͩͪʣ ஌Βͳ͍Tips͕͋Δͱࢥ͏ͷͰɺੋඇ࣋ͪؼͬͯԼ͍͞ɻ มଶʢAWKͰGolf͢Δ͓ͱ΋ͩͪʣ

   Ғͦ͏ͳ͜ͱݴͬͯͯ͢Έ·ͤΜɻมଶճ౴ظ଴ͯ͠·͢ɻ

10. ਐΊํ(2/3) ಈ࡞؀ڥ ໰୊͸Ubuntu14.14ͷbashͰಈ࡞֬ೝࡁΈɻ ࢿྉࣗମ͸طʹWebͰݟΕ·͢ɻ Ͱ͖Δ͚ͩ౴͑Λݟͳ͍ͰਐΊͯԼ͍͞ɻ ࣮ࢪத͸τοϓμ΢ϯͳֶͼΛɻ ࢿྉࣗମ͸ޙ͔ΒಡΈฦͯ͠ϘτϜΞοϓͳֶͼͷ ͨΊͷࡧҾͱͯ͠͝׆༻Լ͍͞ɻ

11. ਐΊํ(3/3) ୺຤ͷઆ໌ $ ίϚϯυ -Φϓγϣϯ΍ͦͷଞ 'จࣈྻϦςϥϧ' ϑΝΠϧ # ίϝϯτ -

    ઌ಄ͷ $ ͸ϓϩϯϓτΛද͢ه߸Ͱೖྗ͸ෆཁɻ - จ຤ͷٯεϥογϡ \ ͸վߦʢ͋ͬͯ΋ͳͯ͘΋OKʣ [࣋ͪؼΓϙΠϯτ] આ໌ͩΒͩΒ

12. Լ४උ (1/2) 1. ୺຤Λ։͍ͯԼ͍͞ 2. ԼهͷίϚϯυΛ࣮ߦͯ͠ɺϑΝΠϧΛμ΢ϯϩʔ υ͍ͯͩ͘͠͞ɻ $ wget goo.gl/PGDjKb

    -O holidays ※ wget͕ͳ͍ਓ͸ͪ͜Β $ curl -L goo.gl/PGDjKb > holidays

13. Լ४උ (2/2) 3. ls ΛଧͬͯϑΝΠϧ͕͋Δ͜ͱΛ֬ೝɻ $ ls
 holidays 4. ϑΝΠϧͷ಺༰Λ֬ೝ

    $ head holidays 2015-01-01 ໦ ݩ೔ 1 2015-01-02 ۚ ࡾ͕೔ 1 2015-01-03 ౔ ࡾ͕೔ 1 …

14. ͜ͷϑΝΠϧ͸ʁ 2015ʙ2021೥·Ͱͷ೔ຊͷ೔෇Ұཡ ϑΥʔϚοτʢ [ ] ͸εϖʔεʣ YYYY-MM-DD[ ]༵೔[ ]Πϕϯτ໊[ ]ٳ೔ϑϥά

    ٳ೔ϑϥάʢٳ೔=1, ฏ೔=0) 12/30ʙ01/03·Ͱ͸ٳ೔ѻ͍ ʢ࢓ࣄೲΊɺେᷢ೔ɺࡾ͕೔ʣ ͓ຍ͸8/13ʙ8/16ʹઃఆ దٓݸਓͷঢ়گʹԠͯ͡ॻ͖׵͑ͯԼ͍͞ɻ

15. Q1 2016೥ͷ೔෇͚ͩΛඪ४ग़ྗʹදࣔͯ͠Լ͍͞ɻ ※தڃऀҎ্ͷํ͸”g”ͷͭ͘ίϚϯυېࢭ $ ? ? ? 2016-01-01 ۚ ݩ೔

    1 2016-01-02 ౔ ࡾ͕೔ 1 2016-01-03 ೔ ࡾ͕೔ 1 2016-01-04 ݄ ฏ೔ 0 …

16. ύλʔϯϚονΛ͠Α͏(1/2) ղ౴ྫ $ grep '2016' holidays 2016-01-01 ۚ ݩ೔ 1

    2016-01-02 ౔ ࡾ͕೔ 1 2016-01-03 ೔ ࡾ͕೔ 1 … [grepίϚϯυͷ࢖͍ํͷجຊ] 1. grep <நग़͍ͨ͠ύλʔϯ> <ϑΝΠϧ໊> 2. cat <ϑΝΠϧ໊> | grep <நग़͍ͨ͠ύλʔϯ>

17. ύλʔϯϚονΛ͠Α͏(2/2) ผղ $ cat holidays | grep 2016 grepͷ࢖͍ํ͕Θ͔Βͳ͍ਓʹ͸͜Μͳ΍Γํ΋ɻ $

    cat holidays | sed -n '/2016/p' $ cat holidays | awk '/2016/' $ cat holidays | perl -ne '/2016/ and print' ஫: grep͸଎͍ͷͰɺେ఍ͷ৔߹grepΛ࢖ͬͨ΄͏͕͍͍ Ͱ͢ɻ

18. Q2 2016೥2݄͸Կ೔͋Δ͔
 ਺͑ͯΈ·͠ΐ͏ɻ $ ? ? ? 29

19. ߦ਺ΛΧ΢ϯτ͠Α͏(1/2) ղ౴ྫ $ cat holidays | grep '2016-02' | wc

    -l 29

20. ߦ਺ΛΧ΢ϯτ͠Α͏(2/2) Լه͸ wc -l (ߦ਺Χ΢ϯτ)ͱ
 ΄΅ಉ͡ڍಈΛ͢Δɻ • awk 'END{print NR}'

    • grep -c . • nl -nln | cut -f 1 | tail -n 1 • perl -pE '}{ say $.'

21. Q3 2016೥9݄19೔͸Կͷ೔ʁ Πϕϯτ໊ͷΈΛදࣔͯ͠Έ·͠ΐ͏ɻ தڃऀҎ্ͷਓ͸”a”ͷͭ͘ίϚϯυېࢭ $ ? ? ? ܟ࿝ͷ೔

22. ಛఆͷϑΟʔϧυΛநग़(1/2) ղ౴ྫ $ cat holidays | grep '2016-09-19' | \

    awk '{print $3}' [ϑΟʔϧυ] จࣈྻΛεϖʔεͰ۠੾ͬͨͱ͖ͷจࣈͷ·ͱ·Γͷ ͜ͱɻ্هྫ͸ࠨ͔Β̏൪໨ͷϑΟʔϧυΛදࣔɻ 2015-01-01[εϖʔε]໦[εϖʔε]ݩ೔[εϖʔε]1 $1 $2 $3 $4

23. ಛఆͷϑΟʔϧυΛநग़(2/2) Լهͷྫ͸શͯɺ̏ϑΟʔϧυ໨Λநग़͢Δྫ • awk '$0=$3' • cut -d ' '

    -f 3 • perl -anle 'print $F[2]' # Perl͸0ΠϯσοΫε • perl -aple '$_=$F[2]' • self 3 # Open usp Tukubai ࢖༻

24. Q4 ࠓ೔ͷ৘ใΛදࣔͯ͠Έ·͠ΐ͏ɻ
 ͨͩ͠ɺίϚϯυʹҰ੾਺ࣈΛ࢖Θͣʹɻ $ ? ? ? 2016-08-27 ౔ ٳ೔

    1

25. ίϚϯυஔ׵Λ͔͓ͭ͏(1/2) ղ౴ྫ $ cat holidays | grep $(date +%F) [ίϚϯυஔ׵]

    $(ʙ) ͋Δ͍͸ όοΫΫΦʔτʢ`ʣΛ࢖ͬͨ `ʙ` ͱ͍͏දݱ͸ʮίϚϯυஔ׵ʯɻ ʙ ʹॻ͍ͨίϚϯυͷ݁ՌΛจࣈྻʹ ͨ͠΋ͷͷΑ͏ʹ;Δ·͏ɻ

26. ίϚϯυஔ׵Λ͔͓ͭ͏(2/2) ίϚϯυஔ׵ͷྫ $ date +%F 2016-08-27 $ echo $( date

    +%F ) #(1) 2016-08-27 $ echo '2016-08-27' #(2) 2016-08-27 (1)ͱ(2)ͷίϚϯυ͸݁Ռతʹಉ͡ɻ

27. Q5 2016೥ʹɺ֤༵೔͕͍ͭ͋͘Δ͔දࣔͯ͠Έ·͠ΐ͏ɻ $ ? ? ? 53 ౔ 52 ೔

    52 ݄ 52 ໦ 52 ਫ 52 Ր 53 ۚ

28. ਺্͑͛(1/2) ղ౴ྫ $ cat holidays | grep '2016' | awk

    '{print $2}' | \ sort | uniq -c [Α͘࢖͏਺͑ํͷϑϨʔζ] • sort | uniq -c • sort | uniq -c | sort -n

29. ਺্͑͛(2/2) sortͱuniqͷ࢖͍ํ͕෼͔Βͳ͍ਓ޲͚ճ౴ $ cat holidays | \ grep '2016' |

    awk '{print $2}' | \ awk '{a[$1]++}END{for (k in a){print a[k],k}}' $ cat holidays | \ grep '2016' | awk '{print $2}' | \ perl -0777 -anlE 'grep{!$h{$_}++} @F; say "$h{$_} $_" for keys(%h)'

30. Q6 2016೥08݄03೔ ͔Β
 2016೥09݄30೔ ·Ͱදࣔͯ͠Έ·͠ΐ͏ɻ $ ? ? ? 2016-08-03

    ໦ ฏ೔ 0 ʙ 2016-09-30 ۚ ฏ೔ 0

31. ͋Δύλʔϯ͔Β
 ͋Δύλʔϯ·Ͱநग़(1/2) ղ౴ྫ $ cat holidays | \ awk '/2016-08-03/,/2016-09-30/'

    [ಛఆͷ2ύλʔϯؒͷग़ྗ] awk '/։࢝ύλʔϯ/,/ऴྃύλʔϯ/'

32. ͋Δύλʔϯ͔Β
 ͋Δύλʔϯ·Ͱநग़(2/2) ผղ # sedΛ࢖͏ $ cat holidays | \

    sed -n '/2016-08-03/,/2016-09-30/p' # perlΛ࢖͏ $ cat holidays | \ perl -nle 'print if /2016-08-03/../2016-09-30/' ൒͹ڧҾʹ $ cat holidays | \ grep '2016-08-03' -A 50 | grep '2016-09-30' -B 50

33. Q7 2015೥ͱ2016೥Λൺֱͯ͠ɺ
 ยํʹ͔͠ଘࡏ͠ͳ͍ॕ೔Λදࣔͯ͠Լ͍͞ɻ
 ̎ͭ͋Γ·͢ɻ $ ? ? ? # ࣮ࡍղ͍ͯ֬ೝͯ͠ΈͯͶ

34. ϓϩηεஔ׵Λ͔͓ͭ͏(1/3) ղ౴ྫ $ diff \ <(grep 2015 holidays | awk

    '{print $3}' | sort | uniq) <(grep 2016 holidays | awk '{print $3}' | sort | uniq) [ϓϩηεஔ׵] <(ʙ) ͱ͍͏දݱ͸ʮϓϩηεஔ׵ʯɻ <(ʙ) ͷՕॴ͸ɺ ʙ ʹॻ͍ͨίϚϯυͷ݁ՌΛهड़ͨ͠ ϑΝΠϧͷΑ͏ʹৼΔ෣͏ɻ

35. ϓϩηεஔ׵Λ͔͓ͭ͏(2/3) ϓϩηεஔ׵ྫ $ echo A B C > abc.txt $

    cat abc.txt | sed 's/A/B/' # (1) B B C $ cat <( echo A B C ) | sed 's/A/B/' # (2) B B C (1)ͱ(2)ͷίϚϯυ͸݁Ռతʹಉ͡ɻ

36. ϓϩηεஔ׵Λ͔͓ͭ͏(3/3) ϓϩηεஔ׵Λ࢖Θͳ͍ผղ $ cat holidays | grep -e '2015' -e

    '2016' | \ awk '{print $3}' | sort | uniq -u

37. Q8 ※ ্ڃऀ޲͚໰୊
 ༗څΛ࢖ͬͯܭըతʹ࿈ٳΛ࡞ΔͨΊʹ
 2015 ʙ 2021೥ͷؒͰ
 ٳ೔→ฏ೔→ٳ೔
 ͱͳ͍ͬͯΔ3೔ؒΛશͯநग़ͯ͠Έ·͠ΐ͏ɻ $

    ? ? ? 2015-11-01 2015-11-02 2015-11-03 2016-02-11 2016-02-12 2016-02-13 2016-05-01 2016-05-02 2016-05-03 … ೥݄೔͕Θ͔Ε͹Α͍ͷͰɺ༵೔΍ٳ೔ϑϥάΛग़ྗͯ͠΋OKɻ ग़ྗํ๏͸͓·͔ͤ͠·͢ɻ

38. ஫ҙ ෳ਺ߦʹ·͕ͨͬͯ
 ॏෳ͢Δ೔͕͋Δ͔΋ɻ {08-09, 08-10, 08-11} ͷτϦΦ {08-11, 08-12, 08-13}

    ͷτϦΦ ೔෇ ༵೔ ಺༰ 2020-08-09 ೔ ٳ೔ 2020-08-10 ݄ ฏ೔ 2020-08-11 Ր ࢁͷ೔ 2020-08-12 ਫ ฏ೔ 2020-08-13 ໦ ͓ຍ

39. Ұߦ಺ͰϧʔϓΛճͦ͏ ղ౴ྫ $ cat holidays | tr '\n' '\t'| awk

    -F'\t' \ '{for(i=3;i<=NF;i++){print $(i-2),$(i-1),$i}}' \ | grep ' 1.* 0.* 1$' গ͠Θ͔Γ΍͘͢ cat holidays | tr '\n' '\t'| awk -F'\t' \ # λϒจࣈΛ۠੾Γจࣈʹࢦఆ '{ for(i=3;i<=NF;i++){ # NF(Number of Field)…ϑΟʔϧυ਺ print $(i-2),$(i-1),$i # ྫ: i͕3ͷ࣌$(i-1)͸$2ͱͯ͠ධՁ͞ΕΔ } }' \ | grep ' 1.* 0.* 1$' # 1(ٳ೔) " 0(ฏ೔) " 1(ٳ೔) (εϖʔε஫ҙ)

40. ศརͳಓ۩Λ׆༻͠Α͏ egzactʢ࠷ऴεϥΠυࢀরʣͷ
 convίϚϯυ͕͋Ε͹γϯϓϧʹ૊ΊΔʂ ʢखલຯḩࣦྱʣ $ cat holidays | tr '\n'

    '\t' | \ conv fs='\t' 3 | \ grep ' 1.* 0.* 1$' convίϚϯυ ϑΟʔϧυ۠੾ΓͷೖྗΛཐટঢ়ʹ܁Γฦ͢ɻ $ echo 'A B C D E' | conv 3 A B C B C D C D E

41. ͜ͷϑΝΠϧΛ࢖ͬͨศརٕ(1/3) ʮ2ϲ݄ͰͰ͖ΔΑͶʁʯͱݴΘΕͨ࣌ → ࣮ࡍͷӦۀ೔਺͸ʁ → େܕ࿈ٳΛڬΉͱҙ֎ʹগͳ͍ $ cat holidays |

    \ awk '/2016-08-01/,/2016-09-30/' |\ grep '0$' | nl Ӧۀ೔͚ͩநग़ → ΤΫηϧʹషΓ෇͚ͯਐḿ؅ཧʢന໨ʣ

42. ͜ͷϑΝΠϧΛ࢖ͬͨศརٕ(2/3) ࠓ೥ͷ༗څ͍ͭ͘Թଘ͠Α͏͔ͳ……ɻ ٳ೔ˠฏ೔ˠٳ೔ $ cat holidays | grep '2016' |

    \ tr '\n' '\t' | conv fs='\t' 3 | \ grep ' 1.* 0.* 1$' ٳ೔ˠฏ೔ˠฏ೔ˠٳ೔ $ cat holidays | grep '2016' | \ tr '\n' '\t' | conv fs='\t' 4 | \ grep ' 1.* 0.* 0.* 1$'

43. ͜ͷϑΝΠϧΛ࢖ͬͨศརٕ(3/3) ࣍ͷૉ਺ͷ೔͸͍ͭͩΖ͏ʁؾʹͳΔͳ͋ $ cat holidays | grep '2016' | \

    tr -d '-' | awk '$0=$1' | \ factor | awk '$0*=!$3' | \ sed -r 's/(..)(..)$/-\1-\2/' | \ grep -f - holidays

44. ·ͱΊ ೖ໳ऀ͸ϘτϜΞοϓֶशॏࢹ͕٢ͩͱࢥ͍·͢ ׳Ε͖ͯͨΒ΍Γ͍ͨ͜ͱΛ࡞ͬͯτοϓμ΢ϯ ΍Γํ͕Θ͔Βͳ͍࣌͸…… → άάͬͯϦόʔεΤϯδχΞϦϯά → ͭΑ͍γΣϧܳਓʹฉ͘ ຖ೔ୟ͘ͱϚΠφʔͳૢ࡞ɾΦϓγϣϯ΋֮͑ͯ͘Δ ͳΜͱ͔श׳Խͤ͞ΔͱͭΑ͘ͳΕΔɻ

    ࠓ೔ͷࢿྉͷؾʹͳͬͨ෦෼ΛҰ୴ؙ҉ه͢Δͷ΋खɻ

45. ࢀߟจݙɾURL ੴҪ͞ΜͷʮγΣϧܳೖ໳ʯ http://slideck.io/github.com/hisaharu/shellgei/slides/shellgei23AM2.md#/ ࡈ౻͞ΜͷAWKҰߦ໺࿠ඦ྽ݓ http://gauc.no-ip.org/awk-users-jp/material/100_one_liners_20131223.pdf γΣϧϓϩάϥϛϯά࣮༻ςΫχοΫ http://gihyo.jp/book/2015/978-4-7741-7344-3 γΣϧεΫϦϓτجຊϦϑΝϨϯε http://gihyo.jp/book/2011/978-4-7741-4643-0 Software

    Design 2016೥6݄߸ bash ࠶ೖ໳ http://gihyo.jp/magazine/SD/archive/2016/201606 Perl One-Liners (※༸ॻ) http://shop.oreilly.com/product/9781593275204.do

46. ࢧԉπʔϧURL egzact — ૊Έ߹Θͤྻڍ͕ܰʑͰ͖ΔΑ͏ʹͳΔ https://github.com/greymd/egzact Open usp Tukubai — DB͍ΒͣͰूܭ࡞ۀ͕ϥΫνϯʹͳΔ

    https://github.com/usp-engineers-community/Open-usp-Tukubai nixar — ஍ຯʹ΍Γʹ͍͘࡞ۀָ͕ʹͳΔ͔΋ https://github.com/askucher/nixar MCMD — ԯ୯Ґͷσʔλʹෳࡶͳ౷ܭॲཧ͕Ͱ͖Δ http://www.nysol.jp/ Cureutils — ϓϦςΟͰΩϡΞͳศརίϚϯυ https://github.com/greymd/cureutils