690 lines
94 KiB
XML
690 lines
94 KiB
XML
<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="4.2.1">Jekyll</generator><link href="https://sahinakkaya.dev/feed.xml" rel="self" type="application/atom+xml" /><link href="https://sahinakkaya.dev/" rel="alternate" type="text/html" /><updated>2023-02-14T09:02:59+00:00</updated><id>https://sahinakkaya.dev/feed.xml</id><title type="html">Şahin Akkaya’s Personal Page</title><subtitle>Şahin Akkaya's personal blog - a perfectionist who likes to tinker everything until it is just right. Get ready to find some sweet tips that will boost your productivity and make you fall in love with your computer.</subtitle><author><name>Şahin Akkaya</name></author><entry><title type="html">Hot-Reload Long Running Shell Scripts (feat. trap / kill)</title><link href="https://sahinakkaya.dev/2023/01/15/hot-reloading-with-trap-and-kill.html" rel="alternate" type="text/html" title="Hot-Reload Long Running Shell Scripts (feat. trap / kill)" /><published>2023-01-15T21:48:08+00:00</published><updated>2023-01-15T21:48:08+00:00</updated><id>https://sahinakkaya.dev/2023/01/15/hot-reloading-with-trap-and-kill</id><content type="html" xml:base="https://sahinakkaya.dev/2023/01/15/hot-reloading-with-trap-and-kill.html"><h2 id="trap-them-and-kill-them"><code class="language-plaintext highlighter-rouge">trap</code> them and <code class="language-plaintext highlighter-rouge">kill</code> them!</h2>
|
||
<p>There is a beautiful command in Linux called <a href="https://man7.org/linux/man-pages/man1/trap.1p.html"><code class="language-plaintext highlighter-rouge">trap</code></a> which <em>trap</em>s signals and let you run specific commands when they invoked. There is also good ol’ <a href="https://man7.org/linux/man-pages/man1/kill.1.html"><code class="language-plaintext highlighter-rouge">kill</code></a> command which not only kills processes but allows you to specify a signal to send. By combining these two, you can run specific functions from your scripts any time!</p>
|
||
|
||
<h3 id="basic-example">Basic Example</h3>
|
||
<p>Let’s start by creating something very simple and build up from there. Create a script with the following contents:</p>
|
||
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/bin/bash</span>
|
||
|
||
<span class="nb">echo</span> <span class="s2">"My pid is </span><span class="nv">$$</span><span class="s2">. Send me SIGUSR1!"</span>
|
||
|
||
func<span class="o">()</span> <span class="o">{</span>
|
||
<span class="nb">echo</span> <span class="s2">"Got SIGUSR1"</span>
|
||
<span class="o">}</span>
|
||
|
||
<span class="c"># here we are telling that run 'func' when USR1 signal is</span>
|
||
<span class="c"># received. You can run anything. Combine commands with ; etc.</span>
|
||
<span class="nb">trap</span> <span class="s2">"func"</span> USR1
|
||
|
||
<span class="c"># The while loop is important here otherwise our script will exit</span>
|
||
<span class="c"># before we manage to get a chance to send a signal.</span>
|
||
<span class="k">while </span><span class="nb">true</span> <span class="p">;</span> <span class="k">do
|
||
</span><span class="nb">echo</span> <span class="s2">"waiting SIGUSR1"</span>
|
||
<span class="nb">sleep </span>1
|
||
<span class="k">done</span>
|
||
</code></pre></div></div>
|
||
<p>Now make it executable and run it:</p>
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>❯ <span class="nb">chmod</span> +x trap_example
|
||
❯ ./trap_example
|
||
My pid is 2811137. Send me SIGUSR1!
|
||
waiting SIGUSR1
|
||
waiting SIGUSR1
|
||
waiting SIGUSR1
|
||
waiting SIGUSR1
|
||
waiting SIGUSR1
|
||
</code></pre></div></div>
|
||
|
||
<p>Open another terminal and send your signal with <code class="language-plaintext highlighter-rouge">kill</code> to the specified pid.</p>
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>❯ <span class="nb">kill</span> <span class="nt">-s</span> USR1 2811137
|
||
</code></pre></div></div>
|
||
|
||
<p>You should receive <code class="language-plaintext highlighter-rouge">"Got SIGUSR!"</code> from the other process. That’s it! Now, imagine you write whatever thing you want to execute in <code class="language-plaintext highlighter-rouge">func</code> and then you can simply <code class="language-plaintext highlighter-rouge">kill -s ...</code> anytime and as many times you want!</p>
|
||
|
||
<p>Let’s move the while loop into the <code class="language-plaintext highlighter-rouge">func</code> and add some variables so you can see how powerful this is.</p>
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/bin/bash</span>
|
||
|
||
<span class="nb">echo</span> <span class="s2">"My pid is </span><span class="nv">$$</span><span class="s2">. Send me SIGUSR1!"</span>
|
||
|
||
func<span class="o">()</span> <span class="o">{</span>
|
||
<span class="nv">i</span><span class="o">=</span>1
|
||
<span class="k">while </span><span class="nb">true</span> <span class="p">;</span> <span class="k">do
|
||
</span><span class="nb">echo</span> <span class="s2">"i: </span><span class="nv">$i</span><span class="s2">"</span>
|
||
<span class="nv">i</span><span class="o">=</span><span class="k">$((</span> i <span class="o">+</span> <span class="m">1</span> <span class="k">))</span>
|
||
<span class="nb">sleep </span>1
|
||
<span class="k">done</span>
|
||
<span class="o">}</span>
|
||
|
||
<span class="nb">trap</span> <span class="s2">"echo 'Got SIGUSR1!'; func"</span> USR1
|
||
|
||
<span class="c"># we need to call the function once, otherwise script</span>
|
||
<span class="c"># will exit before we manage to send a signal</span>
|
||
func
|
||
|
||
</code></pre></div></div>
|
||
|
||
<p>Now run the script and send <code class="language-plaintext highlighter-rouge">SIGUSR1</code>. Here is the result:</p>
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>❯ ./trap_example
|
||
My pid is 2880704. Send me SIGUSR1!
|
||
i: 1
|
||
i: 2
|
||
i: 3
|
||
i: 4
|
||
i: 5
|
||
i: 6
|
||
i: 7
|
||
Got SIGUSR1!
|
||
i: 1
|
||
i: 2
|
||
i: 3
|
||
i: 4
|
||
i: 5
|
||
Got SIGUSR1!
|
||
i: 1
|
||
i: 2
|
||
^C
|
||
</code></pre></div></div>
|
||
|
||
<p>Isn’t this neat?</p>
|
||
|
||
<h3 id="more-useful-example">More useful example</h3>
|
||
|
||
<p>Let’s imagine you have multiple long running (infinite loops basically) scripts and you want to restart them without manually searching for their pid’s and killing them. <code class="language-plaintext highlighter-rouge">trap</code> is for the rescue, again! <sup><a href="##" title="Yeah, I know you can run a systemd service if you want but I think it is an overkill for this situation. Plus, I don't like dealing with them.">*</a></sup> This command is awesome.</p>
|
||
|
||
<p>Without further ado, let’s get started. Create a script called <code class="language-plaintext highlighter-rouge">script1</code> with the following contents:</p>
|
||
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/bin/bash</span>
|
||
<span class="c"># file: script1</span>
|
||
|
||
<span class="nv">i</span><span class="o">=</span>1
|
||
<span class="k">while </span><span class="nb">true</span> <span class="p">;</span> <span class="k">do
|
||
</span><span class="nb">echo</span> <span class="s2">"Hello from </span><span class="nv">$0</span><span class="s2">. i is </span><span class="nv">$i</span><span class="s2">"</span>
|
||
<span class="nv">i</span><span class="o">=</span><span class="k">$((</span> i <span class="o">+</span> <span class="m">1</span> <span class="k">))</span>
|
||
<span class="nb">sleep </span>1
|
||
<span class="k">done</span>
|
||
</code></pre></div></div>
|
||
|
||
<p>And symlink it to another name just for fun:</p>
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>❯ <span class="nb">chmod</span> +x script1
|
||
❯ <span class="nb">ln</span> <span class="nt">-s</span> script1 script2
|
||
</code></pre></div></div>
|
||
|
||
<p>Now we can pretend they are two different scripts as their outputs differ:</p>
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>❯ ./script1
|
||
Hello from ./script1. i is 1
|
||
Hello from ./script1. i is 2
|
||
Hello from ./script1. i is 3
|
||
Hello from ./script1. i is 4
|
||
^C
|
||
❯ ./script2
|
||
Hello from ./script2. i is 1
|
||
Hello from ./script2. i is 2
|
||
Hello from ./script2. i is 3
|
||
^C
|
||
</code></pre></div></div>
|
||
|
||
<p>Finally, create the main script which will start child scripts and restart them on our signals:</p>
|
||
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/bin/bash</span>
|
||
<span class="nb">echo</span> <span class="s2">"My pid is </span><span class="nv">$$</span><span class="s2">. You know what to do ( ͡° ͜ʖ ͡°)"</span>
|
||
<span class="nb">echo</span> <span class="s2">"You can also kill me with 'kill -s INT -</span><span class="se">\`</span><span class="s2">pgrep -f </span><span class="sb">`</span><span class="nb">basename</span> <span class="nv">$0</span><span class="sb">`</span><span class="se">\`</span><span class="s2">'"</span>
|
||
|
||
<span class="nv">pids</span><span class="o">=()</span> <span class="c"># we will store the pid's of child scripts here</span>
|
||
<span class="nv">scripts_to_be_executed</span><span class="o">=(</span><span class="s2">"./script1"</span> <span class="s2">"./script2"</span><span class="o">)</span>
|
||
|
||
kill_childs<span class="o">(){</span> <span class="c"># wow, this sounded wild</span>
|
||
<span class="k">for </span>pid <span class="k">in</span> <span class="s2">"</span><span class="k">${</span><span class="nv">pids</span><span class="p">[@]</span><span class="k">}</span><span class="s2">"</span>
|
||
<span class="k">do
|
||
</span><span class="nb">echo </span>killing <span class="s2">"</span><span class="nv">$pid</span><span class="s2">"</span>
|
||
<span class="c"># -P: kill all the processes whose parent process is 'pid'</span>
|
||
<span class="c"># see how we are creating processes below</span>
|
||
pkill <span class="nt">-P</span> <span class="s2">"</span><span class="nv">$pid</span><span class="s2">"</span>
|
||
<span class="k">done
|
||
</span><span class="nv">pids</span><span class="o">=()</span>
|
||
<span class="o">}</span>
|
||
|
||
<span class="c"># kill childs and restart all the scripts</span>
|
||
restart_scripts<span class="o">(){</span>
|
||
kill_childs
|
||
<span class="c"># for each script in the list</span>
|
||
<span class="k">for </span>script <span class="k">in</span> <span class="s2">"</span><span class="k">${</span><span class="nv">scripts_to_be_executed</span><span class="p">[@]</span><span class="k">}</span><span class="s2">"</span>
|
||
<span class="k">do</span>
|
||
<span class="c"># Run the script and store its pid.</span>
|
||
<span class="c"># note the '&amp;' at the end of command. Without it the script will</span>
|
||
<span class="c"># block until its execution is finished. Also we are putting it</span>
|
||
<span class="c"># into braces because we want to create a "process group" so that</span>
|
||
<span class="c"># we can kill all its children later by specifying parent pid</span>
|
||
<span class="c"># (useful if you have pipes (|) or other &amp;'s in your script!)</span>
|
||
<span class="o">(</span><span class="nv">$script</span><span class="o">)</span> &amp;
|
||
pids+<span class="o">=(</span><span class="s2">"</span><span class="nv">$!</span><span class="s2">"</span><span class="o">)</span>
|
||
<span class="k">done</span>
|
||
<span class="o">}</span>
|
||
|
||
<span class="c"># we will restart_scripts with SIGUSR1 signal</span>
|
||
<span class="nb">trap</span> <span class="s1">'echo "restarting scripts"; restart_scripts'</span> USR1
|
||
|
||
<span class="c"># we will kill all the childs and exit the main script with SIGINT</span>
|
||
<span class="c"># which is same signal as when you press &lt;Control-C&gt; on your terminal</span>
|
||
<span class="nb">trap</span> <span class="s1">'echo exiting; kill_childs; exit'</span> INT
|
||
|
||
<span class="c"># run the function once</span>
|
||
restart_scripts
|
||
|
||
<span class="c"># infinite loop, otherwise main script will exit before we send signal.</span>
|
||
<span class="c"># remember, we started child processes with '&amp;' so they won't block this script</span>
|
||
<span class="k">while </span><span class="nb">true</span><span class="p">;</span> <span class="k">do
|
||
</span><span class="nb">sleep </span>1
|
||
<span class="k">done</span>
|
||
</code></pre></div></div>
|
||
|
||
<p>Now, you can run your main script and reload your child scripts any time with <code class="language-plaintext highlighter-rouge">killall main_script -USR1</code></p>
|
||
|
||
<p>Here is an example run:</p>
|
||
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>❯ ./trap_multiple
|
||
My pid is 3124123. You know what to do ( ͡° ͜ʖ ͡°)
|
||
You can also kill me with 'kill -s INT -`pgrep -f trap_multiple`'
|
||
Hello from ./script1. i is 1
|
||
Hello from ./script2. i is 1
|
||
Hello from ./script2. i is 2
|
||
Hello from ./script1. i is 2
|
||
Hello from ./script2. i is 3
|
||
Hello from ./script1. i is 3
|
||
restarting scripts
|
||
killing 3124125
|
||
killing 3124126
|
||
Hello from ./script1. i is 1
|
||
Hello from ./script2. i is 1
|
||
Hello from ./script2. i is 2
|
||
Hello from ./script1. i is 2
|
||
Hello from ./script2. i is 3
|
||
Hello from ./script1. i is 3
|
||
Hello from ./script2. i is 4
|
||
Hello from ./script1. i is 4
|
||
restarting scripts
|
||
killing 3124304
|
||
killing 3124305
|
||
Hello from ./script1. i is 1
|
||
Hello from ./script2. i is 1
|
||
Hello from ./script1. i is 2
|
||
Hello from ./script2. i is 2
|
||
^Cexiting
|
||
killing 3124875
|
||
killing 3124876
|
||
</code></pre></div></div>
|
||
|
||
<h3 id="final-words">Final words</h3>
|
||
<p>I think I am started to getting obsessed with <code class="language-plaintext highlighter-rouge">trap</code> command because it has such a good name and purpose. FOSS people are really on another level when it comes to naming. Here is another good one:</p>
|
||
|
||
<blockquote>
|
||
<p>- How can you see the contents of a file? <br />
|
||
+ You <em><code class="language-plaintext highlighter-rouge">cat</code></em> it. <br />
|
||
- What if you want to see them in reverse order? <br />
|
||
+ You <em><code class="language-plaintext highlighter-rouge">tac</code></em> it. <br /></p>
|
||
</blockquote>
|
||
|
||
<p>No, it is not just a joke. Try it… Man I love Gnoo slash Linux.</p>
|
||
|
||
<p>Anyway, I hope now you know how to <code class="language-plaintext highlighter-rouge">trap</code> and <code class="language-plaintext highlighter-rouge">kill</code>. Next week I will explain how to <code class="language-plaintext highlighter-rouge">unzip; strip; touch; finger; grep; mount; fsck; more; yes; fsck; fsck; umount; clean; sleep</code> <nobr>( ͡° ͜ʖ ͡°)</nobr>. <sup><a href="##" title="jk :D">*</a></sup></p></content><author><name>Şahin Akkaya</name></author><category term="trap" /><category term="kill" /><category term="linux" /><summary type="html">trap them and kill them! There is a beautiful command in Linux called trap which traps signals and let you run specific commands when they invoked. There is also good ol’ kill command which not only kills processes but allows you to specify a signal to send. By combining these two, you can run specific functions from your scripts any time!</summary></entry><entry><title type="html">Recap of 2022</title><link href="https://sahinakkaya.dev/2022/12/29/recap-of-2022.html" rel="alternate" type="text/html" title="Recap of 2022" /><published>2022-12-29T20:22:08+00:00</published><updated>2022-12-29T20:22:08+00:00</updated><id>https://sahinakkaya.dev/2022/12/29/recap-of-2022</id><content type="html" xml:base="https://sahinakkaya.dev/2022/12/29/recap-of-2022.html"><p>It’s been a while… It has been so long that I forgot how I was writing my blogs back then. My life didn’t change that much. Actually, it is getting worse.</p>
|
||
|
||
<p>The biggest problem of my life is the graduation project. Oh, God it is making me sick! I simply don’t have any interest for the subject I am supposed to work on. One part of me saying that “come on, you came this far. You are nearly finished. One last push!” and other part of me saying “oh no, don’t do it. You have never done something you don’t like in your entire life. F*ck it!”. So I am wasting my time each term with the dilemma I just described. I really don’t know what to do. This thing is f’ed up.</p>
|
||
|
||
<p>Second biggest problem is I live in Turkey. I feel like all my friends somehow get rid of this sh*thole and I am locked here. I use Twitter and Reddit to consume daily news and almost everyday I encounter something that make me say “F*ck me, why I am still here? There is no hope”. Actually, the situation was much worse while I was following pages that shares “street interviews”. At first I started watching them <em>for fun</em> but the stupidity of people was real and harming my mental health. Since that day, I started consuming only news. My experience got better but I feel like it is still affecting me in a bad manner because everyday something bad happens and there is not much I can do to fix. Today, I decided to delete Twitter and Reddit. I’ll see how it goes.</p>
|
||
|
||
<p>I am living with my parents for the past 6 months, I break up with my girlfriend, I left the place I was working. Man, this could be the worst year of my life!</p>
|
||
|
||
<p><img src="/assets/images/recap-2022/jackie-chan-wtf.jpg" alt="Jackie Chan WTF meme" /></p>
|
||
|
||
<p>You know what? I am not gonna give up. <em>“… It ain’t about how hard you get hit. It’s about how hard you get hit and keep moving forward. How much you can take, and keep moving forward…”</em> No, seriously things really will be different for me in 2023 I can feel it. I learn from my mistakes, they are making me even more perfect :D I love myself, I got this.</p>
|
||
|
||
<p><img src="/assets/images/recap-2022/i-got-this.jpg" alt="Disaster girl saying &quot;I got this&quot;" /></p></content><author><name>Şahin Akkaya</name></author><summary type="html">It’s been a while… It has been so long that I forgot how I was writing my blogs back then. My life didn’t change that much. Actually, it is getting worse.</summary></entry><entry><title type="html">Rant: Stop whatever you are doing and learn how licenses work</title><link href="https://sahinakkaya.dev/2022/06/22/rant-on-peoples-reaction-to-copilot.html" rel="alternate" type="text/html" title="Rant: Stop whatever you are doing and learn how licenses work" /><published>2022-06-22T07:46:00+00:00</published><updated>2022-06-22T07:46:00+00:00</updated><id>https://sahinakkaya.dev/2022/06/22/rant-on-peoples-reaction-to-copilot</id><content type="html" xml:base="https://sahinakkaya.dev/2022/06/22/rant-on-peoples-reaction-to-copilot.html"><p>Recently, Github <a href="https://github.blog/changelog/2022-06-21-github-copilot-is-now-available-to-individual-developers/">announced</a>
|
||
that they are making Github Copilot available for everyone. Previously, it was in Beta and you could get it through the waiting list.
|
||
When I saw the news, I thought I can give it a try. But not so surprisingly it was not free. You have 3 ways to get it:</p>
|
||
<ul>
|
||
<li>Pay the subscription fee and get it.</li>
|
||
<li>Prove you are a student and get it for free.</li>
|
||
<li>Be a maintainer of <em>a popular repository</em> and get it for free.</li>
|
||
</ul>
|
||
|
||
<p>I think I should be able to use it for free because I am a student but apparently they are not convinced yet. Anyways, that is a different
|
||
story. I don’t care if they will give me access to Github Copilot or not. It is not a big deal for me.</p>
|
||
|
||
<p>But some people were really angry about how Github Team being vague while defining the criteria as “being a maintainer of a popular open source project”.
|
||
I think they are right to some extent. If all you need is having a few thousands stars for a project, you could easily get that. I know a lot of troll
|
||
or low effort repositories that get a lot of stars because they are funny.</p>
|
||
|
||
<p>Later, I found <a href="https://twitter.com/fatih/status/1539574219629105156">another tweet</a> that explains how Github decides what is <em>popular</em>. According to
|
||
this tweet, if you have a repository that is in top 1000 in one of the most popular 34 languages, you are eligible to get Github Copilot for free.
|
||
This is better than the previous definition but you can still argue that it is not fair because one can create a package for checking if a number is
|
||
even or not and get thousands of stars.</p>
|
||
|
||
<p>You can criticize this, I get that. But do not come up with silly arguments to justify yourself. Like how on earth would you think that Github is doing
|
||
something bad because $10/month is too much for this service? It is business man, you pay if you think it is worth it. That’s it. <em>“I joined beta program
|
||
and it was free, now they want to charge me if I want to continue using it. They did not tell me that.”</em> Uhhm… What? Are you aware that what you are using
|
||
is another company’s service and they have all rights to do whatever they want with their service? How you guys even can build up arguments like that?! This is crazy!</p>
|
||
|
||
<p>Some people argue that <em>“what Github is doing is wrong because they used open source projects <strong>without consent</strong>.”</em> Another similar argument is that <em>“what
|
||
Github is doing is evil because they used projects developed by community and now they are selling it without giving any money to the contributors of
|
||
these projects.”</em> Do you guys even have an idea what licenses stands for? If you don’t want to some random person use your code, just license it that
|
||
way. And if you licensed it with a GPL compatible or similar license you already gave rights anyone to use or sell your code. That is not Github’s
|
||
problem. That is your problem not understanding how licenses work. Stop complaining.</p></content><author><name>Şahin Akkaya</name></author><category term="copilot" /><category term="license" /><category term="github" /><summary type="html">Recently, Github announced that they are making Github Copilot available for everyone. Previously, it was in Beta and you could get it through the waiting list. When I saw the news, I thought I can give it a try. But not so surprisingly it was not free. You have 3 ways to get it: Pay the subscription fee and get it. Prove you are a student and get it for free. Be a maintainer of a popular repository and get it for free.</summary></entry><entry><title type="html">Confession Time</title><link href="https://sahinakkaya.dev/2022/04/08/confession-time.html" rel="alternate" type="text/html" title="Confession Time" /><published>2022-04-08T15:46:00+00:00</published><updated>2022-04-13T00:00:00+00:00</updated><id>https://sahinakkaya.dev/2022/04/08/confession-time</id><content type="html" xml:base="https://sahinakkaya.dev/2022/04/08/confession-time.html"><h2 id="a-failure-story">A failure story</h2>
|
||
<p>Last week, I received an email from <a href="https://letsencrypt.org/">Let’s Encrypt</a> reminding me to renew my certificates. I forgot to renew it and the certificate expired. Now I can’t send or receive any emails. If you send me email in the last week and wonder why I didn’t respond, this is the reason.</p>
|
||
|
||
<p>Anyway, I thought it will be easy to fix. Just run <code class="language-plaintext highlighter-rouge">certbot</code> again and let him do the job, right? NOPE. It is not that easy. It is just giving me errors with some success messages. If I was not so clueless about what the heck I am doing, I could fix the error. But I don’t know anything about how SSL works and it is a shame.</p>
|
||
|
||
<p>I don’t even know the subject enough to Google it. I feel like I am the only guy in the planet whose certificate is expired. Seriously, how tf I can’t find a solution to a such common problem? There was a saying like, <em>“If you can’t find something on the internet, there is a high chance that you are being stupid”</em>. It was not exactly like this but I can’t find the original quote either. Argghh…</p>
|
||
|
||
<p>If you know the original quote, email me… No, do not email because it does not work. F%ck this thing. F*%k everything. I deserved this. Do not help. If I can’t fix this by myself, I should not call myself computer engineer. I am out.</p>
|
||
|
||
<h3 id="update"><strong>Update</strong></h3>
|
||
<p>The problem is fixed. One of my colleagues told me to reboot the server so that it will (<em>possibly</em>) trigger a script to get a new certificate. I did not think it would work because I already try to get a new certificate manually running <code class="language-plaintext highlighter-rouge">certbot renew</code>. And yeah, it didn’t change anything but gave me courage to try other <em>dead simple</em> solutions.</p>
|
||
|
||
<ul>
|
||
<li>
|
||
<p>One of them was adding missing MX records for my domain. <code class="language-plaintext highlighter-rouge">certbot</code> was telling me that it can’t find any <code class="language-plaintext highlighter-rouge">A</code> or <code class="language-plaintext highlighter-rouge">AAAA</code> records for <code class="language-plaintext highlighter-rouge">www.mail</code>. I didn’t think this is related with my problem because how would I receive emails before then? Anyway, I added the records and the errors are gone. It was only giving me success messages now. Everything seemed to be fine. But I still could not connect to my mail account.</p>
|
||
</li>
|
||
<li>
|
||
<p>And here is the solution: <code class="language-plaintext highlighter-rouge">sudo systemctl restart dovecot</code>. Kill me. I am <em>guessing</em> I had to restart the mail service because certificate has changed and it had to pick up the new one. I bet if I had run this command right after <code class="language-plaintext highlighter-rouge">certbot renew</code> I would not face any issues. The error messages caused by missing mx records were not related with this problem but I was confused by them and I thought something wrong with my certificates.</p>
|
||
</li>
|
||
</ul>
|
||
|
||
<p>Any way, I am happy that it is finally fixed. Did I learn something from this? Not much. But yeah, sometimes all you need is a simple restart :D</p></content><author><name>Şahin Akkaya</name></author><category term="ssl" /><summary type="html">A failure story Last week, I received an email from Let’s Encrypt reminding me to renew my certificates. I forgot to renew it and the certificate expired. Now I can’t send or receive any emails. If you send me email in the last week and wonder why I didn’t respond, this is the reason.</summary></entry><entry><title type="html">Never Get Trapped in Grub Rescue Again!</title><link href="https://sahinakkaya.dev/2022/03/03/never-get-trapped-in-grub-rescue-again.html" rel="alternate" type="text/html" title="Never Get Trapped in Grub Rescue Again!" /><published>2022-03-03T00:46:00+00:00</published><updated>2022-03-03T00:46:00+00:00</updated><id>https://sahinakkaya.dev/2022/03/03/never-get-trapped-in-grub-rescue-again</id><content type="html" xml:base="https://sahinakkaya.dev/2022/03/03/never-get-trapped-in-grub-rescue-again.html"><p>Anytime I install a new system on my machine, I pray God for nothing bad happens. But it usually happens. When I reboot, I find myself in the “Grub rescue” menu and I don’t know how to fix things from that point.</p>
|
||
|
||
<p>I go into the live environment and run some random commands that I found on the internet and hope for the best.</p>
|
||
|
||
<p><img src="/assets/images/grub-rescue/random-bullshit.jpg" alt="me trying to fix grub" /></p>
|
||
|
||
<p>What a nice way to shoot myself in the foot! But this time is different. This time, I f*cked up so much that even the random commands on the internet could not help me. I was on my own and I needed to figure out what is wrong with my system. Let me tell you what I did:</p>
|
||
|
||
<p>I decided to install another OS just to try it in a real machine. I wanted to shrink one of my partitions to create a space for the new system. I run <code class="language-plaintext highlighter-rouge">fdisk /dev/sdb</code>, the very first message that it tells me was</p>
|
||
<blockquote>
|
||
<p><code class="language-plaintext highlighter-rouge">This disk is currently in use - repartitioning is probably a bad idea. It's recommended to umount all file systems, and swapoff all swap partitions on this disk.</code></p>
|
||
</blockquote>
|
||
|
||
<p>Yes, it just screams <strong><em>“Do not do it!”</em></strong> but come on. I will not try to shrink the partition I am using (<code class="language-plaintext highlighter-rouge">sdb3</code>). So it should not be a problem. I ignored the message and shrink it anyway. No problem. Installed and tested the new OS a little bit. Time to reboot and hope for the best. And of course it did not boot. What would I even expecting?</p>
|
||
|
||
<p><img src="/assets/images/grub-rescue/cj.png" alt="ah sh*t here we go again" /></p>
|
||
|
||
<p>As always, I booted into a live environment and run <code class="language-plaintext highlighter-rouge">boot-repair</code> command. It was always working but this time… Even after finishing the operation successfully I could not boot into neither Arch nor Ubuntu (the two systems I had previously).</p>
|
||
|
||
<p>Arch was originally mounted in <code class="language-plaintext highlighter-rouge">sdb3</code> and Ubuntu was in <code class="language-plaintext highlighter-rouge">sda2</code>. Considering the fact that I only messed with <code class="language-plaintext highlighter-rouge">sdb</code>, I should be able to boot Ubuntu, right? Well, yeah. Technically I did boot into Ubuntu but I didn’t see the login screen. It was dropping me into something called <em>“Emergency mode”</em> which just makes me panic! <code class="language-plaintext highlighter-rouge">sudo update-grub</code>… Nope. Nothing changes. Arch does not boot and Ubuntu partially boots.</p>
|
||
|
||
<p>Let me tell you what the problem was and how my ignorance made it worse:</p>
|
||
|
||
<ul>
|
||
<li>
|
||
<p>While installing the new system, I saw a partition <strong>labelled</strong> <em>“Microsoft Basic Data”</em>. I deleted it thinking it is not required because I don’t use W*ndows. It turns out, it was my <em>boot</em> partition for Arch, just labelled incorrectly… Big lolz :D But we will see this is not even important because I had to rewrite my boot partition anyway.</p>
|
||
</li>
|
||
<li>My Arch was installed in <code class="language-plaintext highlighter-rouge">sdb3</code>. When I created a new partition and installed the new system, <code class="language-plaintext highlighter-rouge">sdb3</code> was shifted to <code class="language-plaintext highlighter-rouge">sdb5</code> even though I did not ask for it. But the grub configuration to boot my system was still pointing to <code class="language-plaintext highlighter-rouge">sdb3</code>. That was the reason why Arch does not boot. It was trying to boot from <code class="language-plaintext highlighter-rouge">sdb3</code>. So I had to recreate grub configuration and reinstall grub to fix it. I run the following commands that I found <a href="https://www.jeremymorgan.com/tutorials/linux/how-to-reinstall-boot-loader-arch-linux/">here</a> in a live Arch environment:
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">mkdir</span> /mnt/arch
|
||
mount <span class="nt">-t</span> auto /dev/sdb5 /mnt/arch
|
||
arch-chroot /mnt/arch
|
||
mount <span class="nt">-t</span> auto /dev/sdb4 /boot/efi
|
||
os-prober
|
||
grub-mkconfig <span class="o">&gt;</span> /boot/grub/grub.cfg
|
||
grub-install <span class="nt">--efi-directory</span><span class="o">=</span>/boot/efi <span class="nt">--target</span><span class="o">=</span>x86_64-efi /dev/sdb
|
||
<span class="nb">exit
|
||
</span>reboot
|
||
</code></pre></div> </div>
|
||
|
||
<p>And it fixed my grub. I can now boot into Arch, hooray!</p>
|
||
</li>
|
||
<li>
|
||
<p>Ubuntu was not still booting properly. I checked the logs with <code class="language-plaintext highlighter-rouge">journalctl -xb</code> and saw something related with <code class="language-plaintext highlighter-rouge">sdb</code>. Ubuntu was installed in <code class="language-plaintext highlighter-rouge">sda2</code>, why <code class="language-plaintext highlighter-rouge">sdb</code> should be a problem? Then I remembered something. Back in times when I was using Ubuntu, I was using <code class="language-plaintext highlighter-rouge">sdb1</code> as a secondary storage. So I had a configuration where it automatically mounts <code class="language-plaintext highlighter-rouge">sdb1</code> on startup. Since I messed with <code class="language-plaintext highlighter-rouge">sdb1</code> , it was failing to mount it. I opened <code class="language-plaintext highlighter-rouge">/etc/fstab</code>, and deleted the related line. Bingo! It started booting properly.</p>
|
||
|
||
<p><img src="/assets/images/grub-rescue/hackerman.jpg" alt="i am something of a hackerman myself" /></p>
|
||
</li>
|
||
<li>
|
||
<p>I started feeling like Hackerman, and I said to myself <em>“You know what, Imma fix everything.”</em> I had a very sh*tty grub menu with useless grub entries from old systems that I don’t use anymore. The UEFI also had the same problem. It had ridiculous amount of boot entries that most of them are just trash.</p>
|
||
|
||
<p><img src="/assets/images/grub-rescue/shitty-uefi.jpg" alt="the pictures i took while trying to figure out which boot options are useless" /></p>
|
||
|
||
<p>These are the pictures I took for reference while trying to figure out which boot options are useless. Sorry for the bad quality. I didn’t think I would use them in a blog post.</p>
|
||
|
||
<ul>
|
||
<li>While trying to fix the previous problems, I’ve spent enough time in the <code class="language-plaintext highlighter-rouge">/boot/efi</code> directory that make me understand where these grub entries are coming from. There were a lot of files belong to old systems. I simply deleted them and updated grub. All of the bad entries were gone. I want to draw your attention here: <em>I did not search for how to delete the unused grub entries. I just knew deleting their directories from <code class="language-plaintext highlighter-rouge">/boot/efi</code> will do the job. I am doing this sh*t! (Another hackerman moment :D )</em></li>
|
||
<li>In order to delete useless boot options from UEFI menu, I used <code class="language-plaintext highlighter-rouge">efibootmgr</code>. I searched for it on the internet, of course!
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>efibootmgr <span class="nt">-v</span> <span class="c"># Check which entries you want to delete, say it is 0003.</span>
|
||
<span class="nb">sudo </span>efibootmgr <span class="nt">-b</span> 0003 <span class="nt">-B</span> <span class="c"># This will delete third boot option. </span>
|
||
</code></pre></div> </div>
|
||
</li>
|
||
</ul>
|
||
</li>
|
||
</ul>
|
||
|
||
<p>And finally! I know everything about how all these work. Another shady part of Linux is clear for me. Now:</p>
|
||
|
||
<p><img src="/assets/images/grub-rescue/quote.jpg" alt="Give me a ruined computer and an Arch ISO, and I shall fix it for you." /></p></content><author><name>Şahin Akkaya</name></author><category term="linux" /><category term="grub" /><category term="partition" /><category term="uefi" /><summary type="html">Anytime I install a new system on my machine, I pray God for nothing bad happens. But it usually happens. When I reboot, I find myself in the “Grub rescue” menu and I don’t know how to fix things from that point.</summary></entry><entry><title type="html">Creating a *Useless* User</title><link href="https://sahinakkaya.dev/2022/02/27/creating-a-useless-user.html" rel="alternate" type="text/html" title="Creating a *Useless* User" /><published>2022-02-27T13:40:00+00:00</published><updated>2022-02-27T13:40:00+00:00</updated><id>https://sahinakkaya.dev/2022/02/27/creating-a-useless-user</id><content type="html" xml:base="https://sahinakkaya.dev/2022/02/27/creating-a-useless-user.html"><h2 id="story">Story</h2>
|
||
<p>In my <a href="/2022/02/26/ssh-into-machine-that-is-behind-private-network.html">previous post</a>, I explained how to do port forwarding to access some machine behind private network. I will use this method to fix some issues in our desktop at home or my girlfriend’s computer. Now, of course I don’t want to give them access to my server. But they also need to have a user in my server to be able to perform port forwarding via ssh. So I wanted to create a user with least privileges to make sure nothing goes wrong.</p>
|
||
|
||
<h2 id="the-solution">The solution</h2>
|
||
<p>I searched the problem in it turned out to be very simple. You just need to add two additional flags to <code class="language-plaintext highlighter-rouge">adduser</code> command while creating the user.</p>
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>adduser uselessuser <span class="nt">--shell</span><span class="o">=</span>/bin/false <span class="nt">--no-create-home</span>
|
||
</code></pre></div></div>
|
||
<p>Now, <code class="language-plaintext highlighter-rouge">uselessuser</code> can’t do anything useful in your server. If they try to login, the connection will be closed immediately.</p>
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>❯ ssh uselessuser@remote.host
|
||
uselessuser@remote.host<span class="se">\'</span>s password:
|
||
Could not chdir to home directory /home/uselessuser: No such file or directory
|
||
Connection to remote.host closed.
|
||
</code></pre></div></div>
|
||
<p>But they can still do forward the remote port to their local machine.</p>
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>❯ ssh <span class="nt">-Nf</span> <span class="nt">-R</span> 7777:localhost:22 uselessuser@remote.host
|
||
uselessuser@remote.host<span class="se">\'</span>s password:
|
||
</code></pre></div></div>
|
||
<p>The <code class="language-plaintext highlighter-rouge">-N</code> option is the most important one here. From the documentation:</p>
|
||
<blockquote>
|
||
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> -N Do not execute a remote command. This is useful
|
||
for just forwarding ports. Refer to the description
|
||
of SessionType in ssh_config(5) for details.
|
||
</code></pre></div> </div>
|
||
</blockquote>
|
||
|
||
<h2 id="last-words">Last words</h2>
|
||
<p>I love learning new things everyday. I knew setting the shell of a user to <code class="language-plaintext highlighter-rouge">/bin/false</code> will prevent them from logging in. The reason I wrote this blog post is because 2 things I wanted to share:</p>
|
||
<ul>
|
||
<li>While looking for a solution to the problem I mentioned, I searched <em>“create a user with no privileges in linux”</em> and <a href="https://askubuntu.com/questions/1174376/how-to-create-a-user-with-the-least-privileges-permissions-but-enough-to-do-ssh">this</a> came out. It is really interesting for me that another person wanted to do the same thing for the <em>exact same reasons</em>. They were also trying port forwarding via ssh and they wanted to create a limited user in their server to give friends. So the question was a <strong>perfect fit</strong> to the problem.</li>
|
||
<li>The <code class="language-plaintext highlighter-rouge">-N</code> flag of the ssh command was also surprising for me. It was like as if someone had encountered these problems before and just took the exact steps required to solve this problem for me. I mean look at the documentation. Crazy!</li>
|
||
</ul></content><author><name>Şahin Akkaya</name></author><category term="linux" /><category term="permissions" /><category term="privileges" /><summary type="html">Story In my previous post, I explained how to do port forwarding to access some machine behind private network. I will use this method to fix some issues in our desktop at home or my girlfriend’s computer. Now, of course I don’t want to give them access to my server. But they also need to have a user in my server to be able to perform port forwarding via ssh. So I wanted to create a user with least privileges to make sure nothing goes wrong.</summary></entry><entry><title type="html">SSH into Machine That Is Behind a Private Network</title><link href="https://sahinakkaya.dev/2022/02/26/ssh-into-machine-that-is-behind-private-network.html" rel="alternate" type="text/html" title="SSH into Machine That Is Behind a Private Network" /><published>2022-02-26T21:40:00+00:00</published><updated>2022-02-26T21:40:00+00:00</updated><id>https://sahinakkaya.dev/2022/02/26/ssh-into-machine-that-is-behind-private-network</id><content type="html" xml:base="https://sahinakkaya.dev/2022/02/26/ssh-into-machine-that-is-behind-private-network.html"><h2 id="story">Story</h2>
|
||
<p>I believe there is always a “tech support person” in every home. Everyone knows that when there is a problem with any electronic device, they should ask this person. I am the tech support in our house. Today, I had to fix a problem in our desktop. Since I was not at home, I had to fix the problem remotely.</p>
|
||
|
||
<h2 id="possible-solutions">Possible solutions</h2>
|
||
<ul>
|
||
<li>
|
||
<p>Just tell the non-tech people at home to configure the router to forward ssh traffic to desktop, right? Well, this is not an option for me, not because people are non-tech, but there is no router! The desktop is connected to internet via hotspot from mobile phone. There is no root access in the phone and even if there was, it is a really big pain to forward the packets manually. Trust me. Been there, done that!</p>
|
||
</li>
|
||
<li>
|
||
<p>There are tools like <a href="https://www.ngrok.com">ngrok</a>, <a href="localtunnel.me">localtunnel</a> which exposes your localhost to the internet and gives you a URL to access it but I did not want to use them.</p>
|
||
<ul>
|
||
<li>I did not want to use <code class="language-plaintext highlighter-rouge">ngrok</code> because it is not open source and it might have security issues. They are also charging you.</li>
|
||
<li><code class="language-plaintext highlighter-rouge">localtunnel</code> seemed perfect. The code of both client and server is open. That is great news! But it did not last long because it is just forwarding http/https traffic :(</li>
|
||
</ul>
|
||
</li>
|
||
</ul>
|
||
|
||
<h2 id="solution">Solution</h2>
|
||
<p>I was thinking of extending the functionality of <code class="language-plaintext highlighter-rouge">localtunnel</code>, but I learned a very simple way. You don’t need any external program to overcome this issue. The good old <code class="language-plaintext highlighter-rouge">ssh</code> can do that! All you need is another machine (a remote server) that both computers can access via ssh.</p>
|
||
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># local machine (my home computer)</span>
|
||
ssh <span class="nt">-R</span> 7777:localhost:22 remote-user@remote.host
|
||
</code></pre></div></div>
|
||
<p>This command forwards all the incoming connections to port 7777 of remote machine to port 22 of our current machine. In order for this to work, you need to make sure <code class="language-plaintext highlighter-rouge">GatewayPorts</code> is set to <code class="language-plaintext highlighter-rouge">yes</code> in the remote server ssh configuration. It also assumes our current machine accepts ssh connections via port 22.</p>
|
||
|
||
<hr />
|
||
<p>Now, go to any machine and connect to the remote server first. When we are connected, we will create another ssh connection to port 7777 to connect our home computer.</p>
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># another local machine (my laptop)</span>
|
||
ssh remote-user@remote.host
|
||
|
||
<span class="c"># connected remote</span>
|
||
ssh <span class="nt">-p</span> 7777 homeuser@localhost
|
||
<span class="c"># we are now connected to home computer</span>
|
||
</code></pre></div></div>
|
||
|
||
<p>The last two command can also combined so that we directly hop into the home computer.</p>
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ssh <span class="nt">-t</span> remote-user@remote.host ssh <span class="nt">-p</span> 7777 homeuser@localhost
|
||
</code></pre></div></div>
|
||
|
||
<h3 id="result">Result</h3>
|
||
<p>As a result, it only took us 2 simple ssh commands to do this. This is just unbelievable! Now, I need to find a way to make non-tech people at home run this command when there is a problem. Too bad Linux can’t help me there :D</p></content><author><name>Şahin Akkaya</name></author><category term="ssh" /><category term="private-network" /><category term="remote-port-forwarding" /><summary type="html">Story I believe there is always a “tech support person” in every home. Everyone knows that when there is a problem with any electronic device, they should ask this person. I am the tech support in our house. Today, I had to fix a problem in our desktop. Since I was not at home, I had to fix the problem remotely.</summary></entry><entry><title type="html">Using ffmpeg for Simple Video Editing</title><link href="https://sahinakkaya.dev/2022/01/21/ffmpeg-to-rescue.html" rel="alternate" type="text/html" title="Using ffmpeg for Simple Video Editing" /><published>2022-01-21T20:40:00+00:00</published><updated>2022-01-21T20:40:00+00:00</updated><id>https://sahinakkaya.dev/2022/01/21/ffmpeg-to-rescue</id><content type="html" xml:base="https://sahinakkaya.dev/2022/01/21/ffmpeg-to-rescue.html"><h2 id="story">Story</h2>
|
||
<p>Today, I have recorded a video for one of my classes and I was required to upload it till midnight. The video was perfect except for a few seconds where I misspelled some words and started again. I had to remove that part from the video before uploading it. Since I was low on time, I thought that I better use a GUI program to do this job. I opened up <a href="https://kdenlive.org/en/">Kdenlive</a> and jumped into editing my video. It was my first time using it so I spent some time to cut and delete the parts that I want to get rid of. When I was ready, I clicked Render button to render my video. It was waaay too slow than I expected. Since I have nothing to do while waiting for render to finish, I thought I could give <code class="language-plaintext highlighter-rouge">ffmpeg</code> a shot.</p>
|
||
|
||
<h2 id="let-the-show-begin">Let the show begin</h2>
|
||
<p>Like Kdenlive, I have never used <code class="language-plaintext highlighter-rouge">ffmpeg</code> before. Like every normal Linux user do, I opened up a terminal and typed <code class="language-plaintext highlighter-rouge">man ffmpeg</code> to learn how to use it… Just kidding :D I opened a browser and typed <em>“ffmpeg cut video by time”</em>. Not the best search query, but it was good enough to find what I am looking for as the <a href="https://stackoverflow.com/questions/18444194/cutting-the-videos-based-on-start-and-end-time-using-ffmpeg">first result</a>.</p>
|
||
|
||
<h3 id="cutting-the-videos-based-on-start-and-end-time">Cutting the videos based on start and end time</h3>
|
||
<p>According to answers on the page I mentioned, I run the following commands to cut my video into two parts:</p>
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ffmpeg <span class="nt">-ss</span> 00:00:00 <span class="nt">-to</span> 00:01:55 <span class="nt">-i</span> input.mov <span class="nt">-c</span> copy part1.mp4 <span class="c"># take from 00:00 to 01:55</span>
|
||
ffmpeg <span class="nt">-ss</span> 00:02:03 <span class="nt">-to</span> 00:05:17 <span class="nt">-i</span> input.mov <span class="nt">-c</span> copy part2.mp4 <span class="c"># take from 02:03 to 05:17</span>
|
||
</code></pre></div></div>
|
||
|
||
<p>These two commands run <strong>instantly</strong>! Kdenlive was still rendering… The progress was 46%. Meh… I said “Duck it, I am gonna use ffmpeg only” and cancelled the rendering.</p>
|
||
|
||
<h3 id="concatenating-the-video-files">Concatenating the video files</h3>
|
||
<p>Now we have two videos that we want to join. Guess what will be our next search query? <em>“ffmpeg join videos”</em>. And <a href="https://stackoverflow.com/questions/7333232/how-to-concatenate-two-mp4-files-using-ffmpeg">here</a> is the first result:</p>
|
||
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">echo </span>file part1.mp4 <span class="o">&gt;&gt;</span> mylist.txt
|
||
<span class="nb">echo </span>file part2.mp4 <span class="o">&gt;&gt;</span> mylist.txt
|
||
ffmpeg <span class="nt">-f</span> concat <span class="nt">-i</span> mylist.txt <span class="nt">-c</span> copy result.mp4
|
||
</code></pre></div></div>
|
||
|
||
<p>And we are DONE! How easy was that? Whole process took about 10 minutes including my search on the internet. If I continued waiting for Kdenlive to finish rendering, I would probably be still waiting at that time. I love the power of command line!</p></content><author><name>Şahin Akkaya</name></author><category term="cli" /><category term="ffmpeg" /><summary type="html">Story Today, I have recorded a video for one of my classes and I was required to upload it till midnight. The video was perfect except for a few seconds where I misspelled some words and started again. I had to remove that part from the video before uploading it. Since I was low on time, I thought that I better use a GUI program to do this job. I opened up Kdenlive and jumped into editing my video. It was my first time using it so I spent some time to cut and delete the parts that I want to get rid of. When I was ready, I clicked Render button to render my video. It was waaay too slow than I expected. Since I have nothing to do while waiting for render to finish, I thought I could give ffmpeg a shot.</summary></entry><entry><title type="html">Automatically Build and Deploy Your Site using GitHub Actions and Webhooks</title><link href="https://sahinakkaya.dev/2022/01/04/build-and-deploy-automatically.html" rel="alternate" type="text/html" title="Automatically Build and Deploy Your Site using GitHub Actions and Webhooks" /><published>2022-01-04T17:40:00+00:00</published><updated>2022-01-04T17:40:00+00:00</updated><id>https://sahinakkaya.dev/2022/01/04/build-and-deploy-automatically</id><content type="html" xml:base="https://sahinakkaya.dev/2022/01/04/build-and-deploy-automatically.html"><p>In this post I will explain how you can use GitHub to automate the build and deployment processes that you have. I am going to automate the deployment of this site but you can do whatever you want. Just understanding the basics will be enough.</p>
|
||
|
||
<h2 id="introduction-to-github-actions-and-webhooks">Introduction to GitHub Actions and Webhooks</h2>
|
||
<p>Let me start by explaining what are GitHub Actions and GitHub Webhooks.</p>
|
||
|
||
<blockquote>
|
||
<p><strong>Github Actions</strong> is a continuous integration and continuous delivery (CI/CD) platform that allows you to automate your build, test, and deployment pipeline. You can create workflows that build and test every pull request to your repository, or deploy merged pull requests to production.</p>
|
||
</blockquote>
|
||
|
||
<blockquote>
|
||
<p><strong>Webhooks</strong> provide a way for notifications to be delivered to an external web server whenever certain actions occur on a repository or organization. … For example, you can configure a webhook to execute whenever:</p>
|
||
<ul>
|
||
<li>A repository is pushed to</li>
|
||
<li>A pull request is opened</li>
|
||
<li>A GitHub Pages site is built</li>
|
||
<li>A new member is added to a team</li>
|
||
</ul>
|
||
</blockquote>
|
||
|
||
<h2 id="defining-the-problem-and-solution">Defining the problem and solution</h2>
|
||
<p>As I said, my example will be automating the deployment of this site. Here is the normal workflow of me doing it manually:
|
||
<img src="/assets/images/gh-actions-and-webhooks/workflow.png" alt="My workflow" /></p>
|
||
|
||
<p>As you can see, the only place where my work is really required is writing the post. Other two steps can be automated. We will use GitHub Actions to generate the site content and Webhooks to let our server know about the new content so it can pull the changes. Let’s get started.</p>
|
||
|
||
<h3 id="setting-up-github-actions">Setting up GitHub Actions</h3>
|
||
<p>Setting up a GitHub Action is as easy as creating a <code class="language-plaintext highlighter-rouge">.yml</code> file in <code class="language-plaintext highlighter-rouge">.github/workflows/</code> directory in your repository. Let us create a new action to build our site. Fortunately, there is already a <a href="https://github.com/marketplace/actions/jekyll-actions">GitHub action</a> to do it for us. Create a file called <code class="language-plaintext highlighter-rouge">.github/workflows/jekyll.yml</code> in your root directory of your repository and put the following contents:</p>
|
||
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">name</span><span class="pi">:</span> <span class="s">Jekyll site CI</span>
|
||
|
||
<span class="na">on</span><span class="pi">:</span>
|
||
<span class="na">push</span><span class="pi">:</span>
|
||
<span class="na">branches</span><span class="pi">:</span> <span class="pi">[</span> <span class="nv">main</span> <span class="pi">]</span>
|
||
<span class="na">pull_request</span><span class="pi">:</span>
|
||
<span class="na">branches</span><span class="pi">:</span> <span class="pi">[</span> <span class="nv">main</span> <span class="pi">]</span>
|
||
|
||
<span class="na">jobs</span><span class="pi">:</span>
|
||
<span class="na">build</span><span class="pi">:</span>
|
||
<span class="na">runs-on</span><span class="pi">:</span> <span class="s">ubuntu-latest</span>
|
||
|
||
<span class="na">steps</span><span class="pi">:</span>
|
||
<span class="pi">-</span> <span class="na">uses</span><span class="pi">:</span> <span class="s">actions/checkout@v2</span>
|
||
|
||
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">Jekyll Actions</span>
|
||
<span class="na">uses</span><span class="pi">:</span> <span class="s">helaili/jekyll-action@2.2.0</span>
|
||
<span class="na">with</span><span class="pi">:</span>
|
||
<span class="na">token</span><span class="pi">:</span> <span class="s">${{ secrets.GITHUB_TOKEN }}</span>
|
||
<span class="na">keep_history</span><span class="pi">:</span> <span class="no">true</span>
|
||
<span class="na">target_branch</span><span class="pi">:</span> <span class="s1">'</span><span class="s">gh-pages'</span>
|
||
</code></pre></div></div>
|
||
<p>That’s it! We have created our first Action. When we push this change, GitHub will start building our site and push the result to <code class="language-plaintext highlighter-rouge">gh-pages</code> branch. Currently, it will take a while to build because we don’t use caching. So let’s include it to build faster. Add the following piece as a second step:</p>
|
||
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Use GitHub Actions' cache to shorten build times and decrease load on servers</span>
|
||
<span class="pi">-</span> <span class="na">uses</span><span class="pi">:</span> <span class="s">actions/cache@v2</span>
|
||
<span class="na">with</span><span class="pi">:</span>
|
||
<span class="na">path</span><span class="pi">:</span> <span class="s">vendor/bundle</span>
|
||
<span class="na">key</span><span class="pi">:</span> <span class="s">${{ runner.os }}-gems-${{ hashFiles('**/Gemfile') }}</span>
|
||
<span class="na">restore-keys</span><span class="pi">:</span> <span class="pi">|</span>
|
||
<span class="s">${{ runner.os }}-gems-</span>
|
||
</code></pre></div></div>
|
||
<p>We are done with the Actions part. You can see the final code <a href="https://github.com/sahinakkaya/sahinakkayadotdev/blob/main/.github/workflows/jekyll.yml">here</a>. When you are also done with the code, just push it to trigger the action.</p>
|
||
|
||
<h3 id="setting-up-the-webhook-and-related-endpoint">Setting up the Webhook and related endpoint</h3>
|
||
<p>Now that we set up our Action to build the site, we need to let our server know about the changes so that it can pull the changes.</p>
|
||
|
||
<h4 id="creating-a-webhook-from-github">Creating a Webhook from GitHub</h4>
|
||
|
||
<p>To add a Webhook, open your repository in browser and navigate to <em>‘Settings &gt; Webhooks’</em> and click <em>‘Add Webhook’</em>. Fill in the form with appropriate values. Here is an example:
|
||
<img src="/assets/images/gh-actions-and-webhooks/add-webhook.png" alt="Webhook form example" /></p>
|
||
|
||
<p>This is all you have to do from GitHub. Now, whenever there is a <em><code class="language-plaintext highlighter-rouge">push</code></em> event to your repository, GitHub will send a POST request to your <em>payload url</em> with the details.</p>
|
||
|
||
<p class="notice--info"><strong>Note:</strong> Our Action is configured to push to a branch in our repository, so it will also trigger this hook and we will catch it.</p>
|
||
|
||
<h4 id="creating-an-endpoint-to-handle-the-requests">Creating an endpoint to handle the requests</h4>
|
||
<p>I will use <a href="https://flask.palletsprojects.com/en/2.0.x/">Flask</a> framework to handle the post requests coming to our endpoint. You can use whatever programming language or framework you want. It will be very simple code with just one job: Validate the secret keys and run a specific code.</p>
|
||
|
||
<p>Let’s start by creating a new project and a virtual environment:</p>
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>mdkir post_receiver
|
||
<span class="nb">cd </span>post_receiver
|
||
python3 <span class="nt">-m</span> venv venv
|
||
<span class="nb">source </span>venv/bin/activate
|
||
</code></pre></div></div>
|
||
<p>Install the required packages:</p>
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pip <span class="nb">install </span>Flask gunicorn
|
||
</code></pre></div></div>
|
||
<p>Create a new file for storing our environment variables:</p>
|
||
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># config.py
|
||
</span>
|
||
<span class="n">APP_KEY</span> <span class="o">=</span> <span class="s">"your-secret-key"</span> <span class="c1"># same key that is used in github while creating the webhook
|
||
</span><span class="n">PROJECT_PATH</span> <span class="o">=</span> <span class="s">"/path/to/your/project/"</span> <span class="c1"># you will want to cd into this path and perform commands such as git pull etc.
|
||
</span></code></pre></div></div>
|
||
|
||
<p>And create the Flask application:</p>
|
||
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># post_receiver.py
|
||
</span>
|
||
<span class="kn">import</span> <span class="nn">hashlib</span>
|
||
<span class="kn">import</span> <span class="nn">hmac</span>
|
||
<span class="kn">import</span> <span class="nn">subprocess</span>
|
||
|
||
<span class="kn">from</span> <span class="nn">flask</span> <span class="kn">import</span> <span class="n">Flask</span><span class="p">,</span> <span class="n">request</span>
|
||
|
||
<span class="kn">import</span> <span class="nn">config</span>
|
||
|
||
<span class="n">application</span> <span class="o">=</span> <span class="n">Flask</span><span class="p">(</span><span class="n">__name__</span><span class="p">)</span>
|
||
|
||
<span class="o">@</span><span class="n">application</span><span class="p">.</span><span class="n">route</span><span class="p">(</span><span class="s">'/'</span><span class="p">,</span> <span class="n">methods</span><span class="o">=</span><span class="p">[</span><span class="s">'GET'</span><span class="p">,</span> <span class="s">'POST'</span><span class="p">])</span>
|
||
<span class="k">def</span> <span class="nf">index</span><span class="p">():</span>
|
||
<span class="k">if</span> <span class="n">request</span><span class="p">.</span><span class="n">method</span> <span class="o">==</span> <span class="s">'GET'</span><span class="p">:</span>
|
||
<span class="k">return</span> <span class="s">'OK'</span>
|
||
<span class="k">elif</span> <span class="n">request</span><span class="p">.</span><span class="n">method</span> <span class="o">==</span> <span class="s">'POST'</span><span class="p">:</span>
|
||
<span class="n">content</span> <span class="o">=</span> <span class="n">request</span><span class="p">.</span><span class="n">data</span>
|
||
<span class="n">secret</span> <span class="o">=</span> <span class="nb">bytes</span><span class="p">(</span><span class="n">config</span><span class="p">.</span><span class="n">APP_KEY</span><span class="p">,</span> <span class="s">'utf-8'</span><span class="p">)</span>
|
||
<span class="n">digester</span> <span class="o">=</span> <span class="n">hmac</span><span class="p">.</span><span class="n">new</span><span class="p">(</span><span class="n">secret</span><span class="p">,</span> <span class="n">content</span><span class="p">,</span> <span class="n">hashlib</span><span class="p">.</span><span class="n">sha256</span><span class="p">)</span>
|
||
<span class="n">calculated_signature</span> <span class="o">=</span> <span class="s">'sha256='</span> <span class="o">+</span> <span class="n">digester</span><span class="p">.</span><span class="n">hexdigest</span><span class="p">()</span>
|
||
<span class="n">actual_signature</span> <span class="o">=</span> <span class="n">request</span><span class="p">.</span><span class="n">headers</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">'X-Hub-Signature-256'</span><span class="p">)</span>
|
||
<span class="k">if</span> <span class="n">calculated_signature</span> <span class="o">==</span> <span class="n">actual_signature</span><span class="p">:</span>
|
||
<span class="n">subprocess</span><span class="p">.</span><span class="n">Popen</span><span class="p">(</span>
|
||
<span class="p">[</span><span class="s">'./perform-git-pull.sh'</span><span class="p">,</span> <span class="n">config</span><span class="p">.</span><span class="n">PROJECT_PATH</span><span class="p">])</span>
|
||
<span class="k">return</span> <span class="s">'OK'</span>
|
||
<span class="k">else</span><span class="p">:</span>
|
||
<span class="k">return</span> <span class="s">'Error'</span>
|
||
|
||
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s">"__main__"</span><span class="p">:</span>
|
||
<span class="n">application</span><span class="p">.</span><span class="n">run</span><span class="p">(</span><span class="n">host</span><span class="o">=</span><span class="s">'0.0.0.0'</span><span class="p">)</span>
|
||
</code></pre></div></div>
|
||
<p>I will not go into details explaining what each line does. Basically, we are checking if the request is a POST request and if so we are comparing the secret keys to make sure that the request is coming from GitHub. In our case, this is not too important because when the keys match we are running simple git commands in our repository but you might need it if you are doing something more complicated. And here is the contents of <code class="language-plaintext highlighter-rouge">perform-git-pull.sh</code> file:</p>
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/bin/bash</span>
|
||
|
||
<span class="nb">cd</span> <span class="nv">$1</span>
|
||
git checkout gh-pages
|
||
git pull
|
||
</code></pre></div></div>
|
||
<p>We are almost done! All we need to do is create a service to automatically run our code and let nginx handle our endpoint correctly.</p>
|
||
|
||
<p>Create a new file <code class="language-plaintext highlighter-rouge">post_receiver.service</code> in <code class="language-plaintext highlighter-rouge">/etc/systemd/system/</code>:</p>
|
||
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>#/etc/systemd/system/post_receiver.service
|
||
|
||
# change &lt;user&gt; to your actual username
|
||
[Unit]
|
||
Description=post_receiver
|
||
After=network.target multi-user.target
|
||
|
||
[Service]
|
||
User=&lt;user&gt;
|
||
Environment="PYTHONPATH=/home/&lt;user&gt;/post_receiver/venv/bin/python"
|
||
WorkingDirectory=/home/&lt;user&gt;/post_receiver
|
||
ExecStart=/home/&lt;user&gt;/post_receiver/venv/bin/gunicorn -b 127.0.0.1:5000 -w 2 --log-file /home/&lt;user&gt;/post_receiver/post_receiver.log post_receiver
|
||
|
||
[Install]
|
||
WantedBy=multi-user.target
|
||
</code></pre></div></div>
|
||
|
||
<p>Make sure port <code class="language-plaintext highlighter-rouge">5000</code> is reachable from outside.</p>
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>ufw allow 5000
|
||
<span class="nb">sudo </span>ufw <span class="nb">enable</span>
|
||
</code></pre></div></div>
|
||
|
||
<p>Finally, edit your nginx configuration, <code class="language-plaintext highlighter-rouge">/etc/nginx/sites-available/yoursite</code></p>
|
||
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>location = /postreceive/ {
|
||
proxy_pass http://localhost:5000/;
|
||
}
|
||
</code></pre></div></div>
|
||
|
||
<p>Start, restart the services</p>
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>systemctl daemon-reload
|
||
<span class="nb">sudo </span>systemctl start post_receiver
|
||
<span class="nb">sudo </span>systemctl <span class="nb">enable </span>post_receiver
|
||
<span class="nb">sudo </span>systemctl restart nginx
|
||
</code></pre></div></div>
|
||
|
||
<p>That’s it! <code class="language-plaintext highlighter-rouge">curl https://yourdomain.com/postreceive/</code> should return <code class="language-plaintext highlighter-rouge">"OK"</code> and we are ready to accept POST requests from GitHub.</p>
|
||
|
||
<h3 id="notes-for-debugging">Notes for debugging</h3>
|
||
<p>In case anything goes wrong, here are a few tips to debug:</p>
|
||
<ul>
|
||
<li>Every GitHub Action produces a log that you can examine. Check them to see if anything is odd.</li>
|
||
<li>In the <em>Webhooks</em> tab, there is a sub-tab called <em>Recent Deliveries</em>. You can take a look at there to see the results of the requests from your hooks.</li>
|
||
<li>You can always test your code locally with <code class="language-plaintext highlighter-rouge">curl</code>:
|
||
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>❯ curl -i -X POST -H 'Content-Type: application/json' -d '{"foo": "bar", "bar": "baz"}' https://yourdomain.com/postreceive/
|
||
</code></pre></div> </div>
|
||
</li>
|
||
</ul>
|
||
|
||
<p>Happy hacking!</p></content><author><name>Şahin Akkaya</name></author><category term="github-actions" /><category term="github-webhooks" /><category term="ci-cd" /><summary type="html">In this post I will explain how you can use GitHub to automate the build and deployment processes that you have. I am going to automate the deployment of this site but you can do whatever you want. Just understanding the basics will be enough.</summary></entry><entry><title type="html">Stop cat-pipe’ing, You Are Doing It Wrong!</title><link href="https://sahinakkaya.dev/2022/01/01/stop-cat-pipeing.html" rel="alternate" type="text/html" title="Stop cat-pipe’ing, You Are Doing It Wrong!" /><published>2022-01-01T15:00:00+00:00</published><updated>2022-01-01T15:00:00+00:00</updated><id>https://sahinakkaya.dev/2022/01/01/stop-cat-pipeing</id><content type="html" xml:base="https://sahinakkaya.dev/2022/01/01/stop-cat-pipeing.html"><div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">cat </span>some_file | <span class="nb">grep </span>some_pattern
|
||
</code></pre></div></div>
|
||
<p>I’m sure that you run a command something like above at least once if you are using terminal. You know how <code class="language-plaintext highlighter-rouge">cat</code> and <code class="language-plaintext highlighter-rouge">grep</code> works and you also know what pipe (<code class="language-plaintext highlighter-rouge">|</code>) does. So you naturally combine all of these to make the job done. I was also doing it this way. What I didn’t know is that <code class="language-plaintext highlighter-rouge">grep</code> already accepts file as an argument. So the above command could be rewritten as:</p>
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">grep </span>some_pattern some_file
|
||
</code></pre></div></div>
|
||
|
||
<p>… which can make you save a few keystrokes and a few nanoseconds of CPU cycles. Phew! Not a big deal if you are not working files that contains GBs of data, right? I agree but you should still use the latter command because it will help you solve some other problems better. Here is a real life scenario: You want to search for some specific pattern in all the files in a directory.</p>
|
||
|
||
<ul>
|
||
<li>If you use the first approach, you may end up running commands like this:</li>
|
||
</ul>
|
||
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>❯ <span class="nb">ls</span>
|
||
config.lua Git.lua init.lua markdown.lua palette.lua util.lua
|
||
diff.lua highlights.lua LSP.lua Notify.lua Treesitter.lua Whichkey.lua
|
||
|
||
❯ <span class="nb">cat </span>config.lua | <span class="nb">grep </span>light
|
||
❯ <span class="nb">cat </span>diff.lua | <span class="nb">grep </span>light
|
||
❯ <span class="nb">cat </span>Git.lua | <span class="nb">grep </span>light
|
||
❯ <span class="nb">cat </span>highlights.lua | <span class="nb">grep </span>light
|
||
Pmenu <span class="o">=</span> <span class="o">{</span> <span class="nb">fg</span> <span class="o">=</span> C.light_gray, <span class="nb">bg</span> <span class="o">=</span> C.popup_back <span class="o">}</span>,
|
||
CursorLineNr <span class="o">=</span> <span class="o">{</span> <span class="nb">fg</span> <span class="o">=</span> C.light_gray, style <span class="o">=</span> <span class="s2">"bold"</span> <span class="o">}</span>,
|
||
Search <span class="o">=</span> <span class="o">{</span> <span class="nb">fg</span> <span class="o">=</span> C.light_gray, <span class="nb">bg</span> <span class="o">=</span> C.search_blue <span class="o">}</span>,
|
||
IncSearch <span class="o">=</span> <span class="o">{</span> <span class="nb">fg</span> <span class="o">=</span> C.light_gray, <span class="nb">bg</span> <span class="o">=</span> C.search_blue <span class="o">}</span>,
|
||
|
||
❯ <span class="nb">cat </span>init.lua | <span class="nb">grep </span>light
|
||
<span class="nb">local </span>highlights <span class="o">=</span> require <span class="s2">"onedarker.highlights"</span>
|
||
highlights,
|
||
❯ <span class="c"># You still have a lot to do :/</span>
|
||
</code></pre></div></div>
|
||
|
||
<ul>
|
||
<li>If you use the second approach, you will immediately realize that you can send all the files with <code class="language-plaintext highlighter-rouge">*</code> operator and you will finish the job with just one command (2 if you include mandatory <code class="language-plaintext highlighter-rouge">ls</code> :D):</li>
|
||
</ul>
|
||
|
||
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>❯ <span class="nb">ls</span>
|
||
config.lua Git.lua init.lua markdown.lua palette.lua util.lua
|
||
diff.lua highlights.lua LSP.lua Notify.lua Treesitter.lua Whichkey.lua
|
||
|
||
❯ <span class="nb">grep </span>light <span class="k">*</span>
|
||
highlights.lua: Pmenu <span class="o">=</span> <span class="o">{</span> <span class="nb">fg</span> <span class="o">=</span> C.light_gray, <span class="nb">bg</span> <span class="o">=</span> C.popup_back <span class="o">}</span>,
|
||
highlights.lua: CursorLineNr <span class="o">=</span> <span class="o">{</span> <span class="nb">fg</span> <span class="o">=</span> C.light_gray, style <span class="o">=</span> <span class="s2">"bold"</span> <span class="o">}</span>,
|
||
highlights.lua: Search <span class="o">=</span> <span class="o">{</span> <span class="nb">fg</span> <span class="o">=</span> C.light_gray, <span class="nb">bg</span> <span class="o">=</span> C.search_blue <span class="o">}</span>,
|
||
highlights.lua: IncSearch <span class="o">=</span> <span class="o">{</span> <span class="nb">fg</span> <span class="o">=</span> C.light_gray, <span class="nb">bg</span> <span class="o">=</span> C.search_blue <span class="o">}</span>,
|
||
init.lua:local highlights <span class="o">=</span> require <span class="s2">"onedarker.highlights"</span>
|
||
init.lua: highlights,
|
||
LSP.lua: NvimTreeNormal <span class="o">=</span> <span class="o">{</span> <span class="nb">fg</span> <span class="o">=</span> C.light_gray, <span class="nb">bg</span> <span class="o">=</span> C.alt_bg <span class="o">}</span>,
|
||
LSP.lua: LirFloatNormal <span class="o">=</span> <span class="o">{</span> <span class="nb">fg</span> <span class="o">=</span> C.light_gray, <span class="nb">bg</span> <span class="o">=</span> C.alt_bg <span class="o">}</span>,
|
||
markdown.lua: markdownIdDelimiter <span class="o">=</span> <span class="o">{</span> <span class="nb">fg</span> <span class="o">=</span> C.light_gray <span class="o">}</span>,
|
||
markdown.lua: markdownLinkDelimiter <span class="o">=</span> <span class="o">{</span> <span class="nb">fg</span> <span class="o">=</span> C.light_gray <span class="o">}</span>,
|
||
palette.lua: light_gray <span class="o">=</span> <span class="s2">"#abb2bf"</span>,
|
||
palette.lua: light_red <span class="o">=</span> <span class="s2">"#be5046"</span>,
|
||
util.lua:local <span class="k">function </span>highlight<span class="o">(</span>group, properties<span class="o">)</span>
|
||
util.lua: <span class="s2">"highlight"</span>,
|
||
util.lua: highlight<span class="o">(</span>group, properties<span class="o">)</span>
|
||
</code></pre></div></div>
|
||
|
||
<p>Isn’t this neat? You might say that <em>“This is cheating! You are using a wild card, of course it will be easier.”</em> Well, yes. Technically I could use the same wild card in the first command like <code class="language-plaintext highlighter-rouge">cat * | grep light</code> but:</p>
|
||
<ul>
|
||
<li>I figured that out only after using wild card in the second command. So I think it is does not feel natural.</li>
|
||
<li>It is still not giving the same output. Try and see the difference! <a href="##" title="You will not be able to see which file contains which line. 'cat' will just concatenate all the input.">*</a></li>
|
||
</ul></content><author><name>Şahin Akkaya</name></author><category term="cat" /><category term="grep" /><category term="linux" /><category term="command-line" /><summary type="html">cat some_file | grep some_pattern I’m sure that you run a command something like above at least once if you are using terminal. You know how cat and grep works and you also know what pipe (|) does. So you naturally combine all of these to make the job done. I was also doing it this way. What I didn’t know is that grep already accepts file as an argument. So the above command could be rewritten as: grep some_pattern some_file</summary></entry></feed> |