Terminal

Introduction

A terminal is a simple display to enter “commands” for your computer. Common tasks are managing the filesystem and running programs.

Some programs are said “to be on the command-line”. It usually means there is no graphical user interface (GUI). But the reverse is often not the case, at least for many technical programs: when they offer a GUI, they also often offer a command-line interface (CLI).

The terminology does not stop there, as we have 3 layers. From top to bottom:

A terminal is a simple display to enter “commands” for your computer. By layers:

Layervisual?Role
1 Terminal yes Shows the text of the commands and their results.
2 Shell no Defines the commands and their options, calls the kernel accordingly.
3 Kernel no Provides all the low-level operations.

The commands are simple and well-thought accesses to the kernel, so that we do not have to create programs over the kernel ourselves. It makes sense for two reasons: on one hand, many tasks are trivial (like creating a file or calling one program) so it is better to provide a standard way to perform them.

On the other hand, being trivial, we usually need more than one command. And as they can fail, we have to get their result back, to react accordingly. All this points to the necessity of a visual and interactive program, where simple and short calls to the kernel are available.

This gives you the split terminal / shell / kernel to separate the visual from the non-visual, with the shell as the interactive facilitator in the middle. In a way, it is like the outermost layer of the kernel, hence its name.

The commands are simple accesses to the kernel. As we usually need more than one command, and as commands can fail, the terminal has to be visual and interactive. It relies on the shell as the facilitator to the kernel. In a way, the shell is like the outermost layer of the kernel, hence its name.

Environment

Pick the terminal you want. I picked WezTerm because it is fast to boot, configurable by programming (with Lua) and very well documented.

I work on Windows but I actually use a Unix shell thanks to busybox-w32. I do not know if Busybox is easy to install or not. I got it through the whole environment setup by Chris Wellons with his w64devkit, which is very easy to install.

Pick the terminal you want. I chose WezTerm paired with busybox-w32 installed through w64devkit to have a Unix shell on Windows. Reasons are:

But why a Unix shell on Window? Because:

  • it is superior to the historical Windows shell, which is not good on its own anyway.
  • it is used a lot in the industry, especially for test & production machines. Meaning that when you need to do things on these machines, you have to know the Unix shell as the CLI is often the only tool available.
  • it goes the extra mile thanks to powerful data processing commands (like cut, sort, sed, awk)
  • a lot of programs expects to be executed from a Unix shell and cannot work properly otherwise.
  • the Unix shell is superior to the historical Windows shell.
  • it is used a lot in the industry.
  • it provides powerful data processing commands (like cut, sort, sed, awk)
  • it is often the expected environment of many CLI.

So take a moment to setup a Unix environment for your terminal, it will never be a waste of your time.

Installing a Unix environment for your terminal will never be a waste of your time.

Here is my .profile and my configuration file for Wezterm.

Basic visual

At its simplest, the command-line often shows two lines:

The terminal often shows these two lines:

current-working-directory
> _

The current working directory is abbreviated as CWD and the second line is called the prompt. Look at the documentation to configure this, as it is not intuitive. I would advise you to always include the CWD as it is the default path for all kinds of commands.

The first line tells you the path of the current working directory, often shorten as CWD. The second is called the “prompt” as it is where you type your command. It is indicated by a character, usually > or $. Technically though, the prompt is both lines as you can fully configure its content.

For instance, here is my configuration: PS1="$(printf '\n\x1b[32;1m\\w\x1b[0m\n$ ')". It means I set the PS1 variable to a string, obtained by the convoluted processing of the inner string '\n\x1b[32;1m\\w\x1b[0m\n$ '. It is necessary to go through printf because of the presence of the escape sequence \x1b. The string itself means:

\n newline, to ensure to have an empty line between my new prompt and the last result
\x1b[32;1m switch to a green color
\\w show the path of CWD
\x1b[0m reset the color to the default value
\n newline
$ indicate where the command will appear, add one space to separate the dollar sign and the command

I would advise you to always include the CWD in your prompt as it is the default path for all kinds of commands. The CWD is key to keep your commands short and also to quickly know where the effect of your command takes place.

Useful shortcuts
run a command Enter
auto-complete Tab
stop a running command Ctrl + C
copy Ctrl + Shift + C
paste Right click
clear the terminal1 Ctrl + Shift + O
close the terminal1 Ctrl + Shift + D
search for a previous command Ctrl + R
go to the previous/next command2 Up / Down

(1): totally mine. Check your terminal for how to do this.

(2): this is a bit quicker than Ctrl + R when you check a handful of commands repeatedly (because you are writing a script or trying to find the appropriate options of a command). But it is more error-prone too, as you may run the wrong command if you are not careful. Ctrl + R is more precise as you explicitly type part of the command you need.

Commands

A good first step is to search for cheatsheets. For Unix in general but also for the complex commands grep, sed and awk. Cheatsheets give you a better overview than the classic “man” pages since some options can repurpose a command.

In the table below, I have left out the commands for file management. You should use your file explorer instead because these tasks are visual by nature. Anyway, any good Unix cheatsheet will tell you about mkdir, touch, mv and cp. And about the one for removing files and folders…

Triple-read before pressing Enter, especially when using the options -r/R & -f. Because you are performing a full-depth removal without confirmation on the terminal. Pair that with absolute paths and the shorter the path, the more you will delete! Here is your Nemesis: rm -rf /.

It wipes out the whole machine without confirmation. Now, user accounts & groups will limit the effects. All well-setup Unix machines will give you less than administrative rights by default. Still, the conciseness of commands, the mental flow caused by the interactive prompt and the lack of visuals can all conspire against you.

General advices:

  • use cheatsheets for an overview as Unix commands often accept some options that can repurpose them entirely.
  • use your file explorer for file management.
  • if you ever remove files or folders from the CLI, triple-read before pressing Enter especially with absolute paths and options -r/R & -f

Here are the commands that proved useful to me over the years.

CommandNotes
man <command-name>

Open the manual of the given command.
Manuals may not be installed on the machine so this command can fail. You can always search on the web though.

cd <relative-pathname>

Modify the CWD.
Short for current-directory
Use .. to refer to the parent folder (hence ../.. refers to the grand-parent folder and so on).

pwd

Return the CWD.
Necessary when the prompt does not contain the CWD.

ls

List the content of the CWD.
Often used as ls -l, aliased as ll. In scripts, better used as ls -1 to parse the result.

wc -l <file-path>

Count the number of lines of the given file.
wc is short for “word-count” and the option -l repurposes the command to count the lines.

cut -d',' -f1,2,4 <file-path>

Extract fields from each line of the given file.
Here, cut at the delimiter , resulting in several fields of which we take the n°1, 2 and 4.

sort -u <other-options> <file-path>

Sort the lines of the given file.
Counter-intuitive: if you want to remove duplicated lines, use -u, short for “unique”.

grep <text> <file-path>

Search for the given text in the given file.
Prefer ripgrep on your machine.

-ccount rather than show the matches
-vinvert, show what does not match
-Ethe <text> can be a regular expression
-A <number>show <number> lines after a match
-B <number>show <number> lines before a match
find . -name "*.page" -type f

Search for files.
Prefer <fd on your machine.

-type <arg>f files, d directories, l symbolic links
-name <regexp>describe the item's name
-maxdepth <number>limit the search
sed -i -e 's/old/new/' <file-path>

Replace on each line the first occurence of old by new.
“replace all” with 's/old/new/g'.
sed can do much more but it is too complicated for me.

awk

Works like cut but more powerful as you write a program to process the fields.
Too complicated for me.

Aliases

As a rule, any command that is key to your workflow should have a short alias, two to three letters. You should have an alias to launch your tests, and probably some aliases over ripgrep for your most used searches.

CommandNotes
alias ll='ls -al'

List all the content of the CWD, in long format.
Always useful.

alias fresh='source ~/.profile'

Reload my .profile file.
Useful to get the newest changes without restarting the terminal.

alias ex='explorer .'

Windows specific: open a file explorer at the CWD.
Going visual.

alias cls='clear'

Windows specific: clear the terminal.
Uniformize how to clear the terminal by configuring Unix to use the same command as Windows.

Programs

Two programs greatly improve the classic Unix commands find & grep. Please give them a try:

CLI vs GUI

In general, you should use the CLI of a program, as it is the most powerful way to interact with a program, with the greatest number of options. It is hardly surprising as making a CLI is much easier than making a GUI. Extending it too. Making a GUI that works on a variety of machines, that's another story entirely.

The catch of course is that you have to read the documentation to use the CLI at all. A well-thought-out GUI will show what matters, no effort on your part. This beats reading the documentation, especially when the number of options is so large that the most basic ways to use the program are excruciating to understand.

You should still try to use the CLI. To that effect, your goal is to find good tutorials. But if the visual is truly important, use a GUI. That is why I use Sourcetree & WinMerge and it's no coincidence: they both diff things (code, file, folder and image). The result is visual by nature. Using the CLI makes little sense here.

The actual reason that often prevents GUIs to be my go-to tools is that they are either too generic and superficial, or too dense and incomprehensible. First case may not be a deal-breaker. For instance, Sourcetree is fine as the basic tasks of Git cover 90% of my needs. And the CLI complements the GUI when I need finer commands.

Second case is much worse: not only the GUI is as hard to use as the CLI out-of-the-box, but all its complexity stays in your way forever as you cannot trim the display down to what is important to you. In fact, you may even be limited when configuring the features themselves!

So invest even a bit of time in a CLI because:

Use a CLI as it is the most powerful way to interact with a program, with the greatest number of options. But if the visual is truly important, use a GUI. Good examples are Sourcetree and WinMerge. Do remember that GUIs always have some limitations, whereas investing even a bit of time in a CLI is always good:

  • you will train searching through documentation.
  • you will know more about the programs you use daily.
  • you will develop scripts for yourself sooner.
  • you will cater to your needs better than what generic softwares offer you.

Which leads to the next subject.

Scripting

The most basic reason to write scripts is to make your everyday operations as fluid, clear and bullet-proof as possible and I would lay out your options on four levels:

Make your everyday operations as fluid, clear and bullet-proof as possible with:

  1. aliases & basic Unix functions
    For straightforward sequence of commands.
  2. involved scripts with sed & awk
    Do not get stuck at that level for too long!
  3. a scripting language
    Like Javascript or Lua or Python.
  4. a statically-typed language to make a GUI
    Like Typescript + Electron + Mithril.

The progression is natural. When a simple command keeps coming back, define an alias. If it is a small sequence of commands, define a Unix function. For simple data processing, look at sed & awk as they are often able to do what you need. Other Unix functions like grep & sort complement them well.

But there is a huge problem here. Unix commands and regular expressions give the false impression you are just one character away from a working solution. Do not fall into this trap. When you have to define several variables, or make manual loops and intricate conditionals, bail out.

Indeed, Unix commands have built-in loops and options to select the proper situations. If you get manual about these, well, maybe you are using them wrong. You should correct that. But if you are using them right or if you found a smart “fix” that is a workaround, that is a clear sign to stop and to reach out to a scripting language.

By “scripting”, I mean dynamically-typed and minimal. Such a language can do a splendid job for self-contained one-file scripting. And it is usually straightforward to install in a variety of environments, i.e. on your local machine as well as on remote machines where you have limited rights.

This variety of environments is why there are four levels in general. But on my local machine, I only have two levels: the first and the last. I find the codes at levels n°2 & 3 to be too hard to maintain and reuse. And the program such codes provide is not even visual! So when level n°1 does not cut it, I go straight to level n°4.

Beware of level n°2. With Unix commands and regular expressions, it is easy to think that you are just one character away from a working solution. A scripting language (= dynamically-typed and minimal) will be better.

On your local machine though, I would advise you to go straight to level n°4 if level n°1 is not enough.