Mushycode

Podman and Colima as docker desktop alternatives for enterprise users

Sat, 04 Mar 2023 00:00:00 +0000

Docker Desktop Alternatives for Enterprise Users

Docker has changed its Service Agreement and requires a paid subscription for Docker Desktop. It is a popular tool used by many developers to manage containers on their Mac. However, some users may want to explore alternatives to Docker Desktop / Docker for various reasons, such as performance issues or the desire for daemonless docker. Two possible alternatives to Docker Desktop on a Mac are Podman and Colima. In this blog, I just wanted to quickly document my experience on try out these two alternatives as well as the pros and cons of each.

Replacing Docker Desktop with Podman on Mac

Podman is a container management tool that allows users to manage containers without requiring a daemon to be running in the background. It is an open-source project developed by Red Hat and provides a CLI that is compatible with Docker.

Install Podman: Podman can be installed on a Mac using Homebrew. Open a terminal window and enter the following commands:

brew install podman

brew install podman-completion

This will install Podman and its completion script.

Start using Podman: Once Podman is installed, you can start using it to manage your containers. To run a container, use the podman run command. For example, to run a simple hello-world container, enter the following command in the terminal:

podman run hello-world

Pros and Cons of Podman

Pros:

Podman does not require a daemon to be running in the background, which can improve performance and security.
Podman provides a Docker-compatible CLI, making it easy to switch from Docker to Podman.
Podman supports rootless containers, which allows non-root users to run containers.

Cons:

Podman is not as widely used as Docker, so finding support or tutorials may be more difficult.
Podman does not currently support Docker Compose, although this may change in the future.

Note: I had issues with volume mounting in podman, but looks like passing the mount to podman machine during its creation fixes it. For ex,

podman machine init --disk-size 20 -v \$HOME:$HOME

podman machine start

Replacing Docker Desktop with Colima on Mac

Colima is a Docker and Kubernetes development environment for Mac. It is built on top of macOS Hypervisor framework and provides a lightweight, native experience for managing containers.

To replace Docker Desktop with Colima on a Mac, follow these steps:

Install Colima: Colima can be downloaded from the official website. Once downloaded, open the DMG file and drag the Colima app to the Applications folder.

Start using Colima: Once Colima is installed, open the app and follow the setup instructions. You can then start using Colima to manage your containers.

Pros and Cons of Colima

Pros:

Colima provides a lightweight, native experience for managing containers on a Mac.
Colima supports Docker Compose and Kubernetes, making it a versatile tool for container management.
Colima provides a web-based interface for managing containers, which can be more user-friendly than a CLI.

Cons:

Colima is a relatively new tool and may have bugs or compatibility issues. C- olima is not as widely used as Docker, so finding support or tutorials may be more difficult.

Conclusion

Replacing Docker Desktop with Podman or Colima on a Mac is relatively easy, and both tools offer certain pros and cons. Podman is a lightweight tool that provides a Docker-compatible CLI, while Colima provides a native, web-based interface for managing containers. Ultimately, the choice of which tool to use depends on the user.

Lessons from Sectigo's Expired Certificates

Sat, 30 May 2020 20:43:52 +0000

Lessons from Sectigo’s Expired Certificates - Why it is important to validate SSL certificate chain and not just root SSL certificate in monitoring systems

How it all started today?

Today we noticed that in some of our applications connection to our internal API’s are failing with the following error messages -

RestClient::SSLCertificateNotVerified (SSL_connect returned=1 errno=0 state=error: certificate verify failed)

When we checked the nginx logs of the API’s we saw the following error -

SSL_shutdown() failed (SSL: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init) while SSL handshaking

API’s health check from browser did not show any issue, all seemed fine. Infact, the SSL certificate was not expiring before 2021. So what was causing this issue? Upon further investigation it was found that one of the intermediate certificate in the certificate chain has expired! Sectigo’s External CA root expired and thus our certifactes had issues.

How did we fix it?

The fix was to update our full certificate file with the new roots. In our case, it looked like the following -

Subject CN: *.ourdomain.com > Issuer CN: Sectigo RSA Domain Validation Secure Server CA
Subject CN: Sectigo RSA Domain Validation Secure Server CA > Issuer CN: USERTrust RSA Certification Authority
Subject CN: USERTrust RSA Certification Authority > Issuer CN: AddTrust External CA Root
Subject CN: AddTrust External CA Root > Issuer CN: AddTrust External CA Root

We updated the new UserTrust certs for Sectigo from this link.

The modern root certificates are availble under the heading USERTrust RSA Certification Authority. A couple of sections below the cross certificates are also availble.

The certificates need not be re-issued and can be updated with these new ones. In our case 3 and 4 in the full chain were replaced.

Hope this helps someone. Let me know in case there is some correction.

Commonly used Elasticsearch REST APIs at work

Wed, 13 May 2020 13:13:55 +0000

Helpful Elasticsearch APIs

This is a simple post listing the Elasticsearch rest APIs that I use most at work.

List all indices

curl "localhost:9200/_cat/indices"

List all shards

curl "localhost:9200/_cat/shards"

List all elasticserach nodes

curl "localhost:9200/_nodes?pretty"

curl "localhost:9200/_cluster/allocation/explain?pretty"

curl -s 'localhost:9200/_cat/allocation?v'

curl -XPOST localhost:9200/_cluster/reroute?retry_failed=true

curl -X POST localhost:9200/_reindex \
-H 'cache-control: no-cache' \
-H 'content-type: application/json' \
-d '{
  "source": {
    "index": "<<index_old_ones_with_data>>",
    "query": {
        "match_all": {}
    }
  },
  "dest": {
    "index": "<<new_index_with_new_mapping>>"
  }
}'

curl -X POST localhost:9200/_reindex \
-H 'cache-control: no-cache' \
-H 'content-type: application/json' \
-d '{
  "source": {
    "remote": {
      "host": "http://<some-ip>:9200"
    },
    "index": "<<index_old_ones_with_data>>",
    "query": {
        "match_all": {}
    }
  },
  "dest": {
    "index": "<<new_index_with_new_mapping>>"
  }
}'

curl -X POST "localhost:9200/<index_name>/_search?pretty" -H 'Content-Type: application/json' -d'
{
  "query": {
    "match": {
      "<field>": "<field-value>"
    }
  }
}
'

curl -X POST "localhost:9200/<index_name>/_delete_by_query?pretty" -H 'Content-Type: application/json' -d'
{
  "query": {
    "match": {
      "<field>": "<field-value>"
    }
  }
}
'

PS: I will add description for each soon

Understanding docker layers and overlay filesystem

Tue, 19 Nov 2019 12:47:46 +0000

What are docker layers and overlay filesystems

A docker image consists of layers. Each layer is an instruction in the Dockerfile. A container is an image with a writable(or readable) layer or top of other read only layers. This is where an overlayfs comes in to picture. This was one of the things that I wanted to understand better - What exactly is an overlay file system and how is docker using it. From archlinux wiki definition “Overlayfs allows one, usually read-write, directory tree to be overlaid onto another, read-only directory tree. All modifications go to the upper, writable layer.”

An apt example is a linux live CD.

Lets get down to how this overlay file system works

Overlays and docker

It consists of three layers for the sake of simplicity.

Lower Read Only Layer
Upper Layer that allows read and write
The Overlay layer itself that gives the user entire view of the filesystem and from where user actually reads a file or writes to them.

Lower Layer

The read only layer where base files of the filesystem are stored. It can be visualized as the base image of our docker image or root of a live CD example.

Upper Layer

Any changes made in the overlay layer gets stored in the upper layer. Whenever a process reads a file from the overlay layer, it is checked in upper layer first, if not available it is read from the lower layer.

Overlay

This is the consolidated or union view of lower and upper layers and where the user works on. Whenever any changes are made to a file, overlay layer presents the union of upper and lower layers such that files in upper layer supersede the ones in lower layer.

When objects with the same name exist in both directory trees of upper and lower layers, then their treatment depends on the object type:

File: We see the file in the upper layer, and the file in the lower layer is hidden.

Directory: content of both upper and lower layers of the directory tree is combined and shown in the overlay

Copy-on-Write

Whenever a file already present in lower layer is modified, the file is first copied over to the upper layer and then the modifications are made here.

Overlay in action

Lets create directories

root@dd4a4d50aefc:~# mkdir lower upper merged workdir
root@dd4a4d50aefc:~# echo "I am in lower layer" > lower/lower.txt
root@dd4a4d50aefc:~# echo "I am in upper layer" > upper/upper.txt
root@dd4a4d50aefc:~# echo "I am common file lower layer" > lower/common.txt
root@dd4a4d50aefc:~# echo "I am common file upper layer" > upper/common.txt
root@dd4a4d50aefc:~# sudo mount -t overlay overlay -o lowerdir=/root/lower,upperdir=/root/upper,workdir=/root/workdir /root/merged
root@dd4a4d50aefc:~# ls merged/
common.txt  lower.txt  upper.txt
root@dd4a4d50aefc:~# cat merged/common.txt 
I am common file upper layer
root@dd4a4d50aefc:~# ls lower/
common.txt  lower.txt
root@dd4a4d50aefc:~# ls upper/
common.txt  upper.txt

The lower layer can be read-only and an overlay itself, while the upper layer is normally writeable. In order to create an overlay of two directories, dir1 and dir2, we can use the following mount command:

root@dd4a4d50aefc:~# sudo mount -t overlay overlay -o lowerdir=/root/lower2:/root/lower1,upperdir=/root/upper,workdir=/root/workdir /root/merged

while specifying multiple lower layers, they are separated by a :, with the rightmost lower directory on the bottom, and the leftmost lower directory on the top of the overlay.

New file creation in overlay

root@dd4a4d50aefc:~# echo "I am the new file on the block" > merged/new.txt
root@dd4a4d50aefc:~# ls merged/
common.txt  lower.txt  new.txt  upper.txt
root@dd4a4d50aefc:~# ls upper/
common.txt  new.txt  upper.txt
root@dd4a4d50aefc:~# ls lower/
common.txt  lower.txt

References

https://docs.docker.com/storage/storagedriver/#images-and-layers
https://wiki.archlinux.org/index.php/Overlay_filesystem
https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt

How to write safe shell scripts

Wed, 13 Nov 2019 15:52:57 +0000

Good Practices in bash scripts

Shebang (#!)

It is called a shebang or a “bang” line. It is nothing but the absolute path to the Bash interpreter. It consists of a number sign and an exclamation point character (#!), followed by the full path to the interpreter such as /bin/bash. All scripts under Linux execute using the interpreter specified on a first line. This ensures that the correct interpreter will be used to interpret the script, even if it is executed under another shell.

set -e

this will make the shell script exit as soon as any line in the bash script fails. for example, a shell file like below will execute every line

arun@home:~$ ./set_e_without.sh 
true
true
false
true

After adding set -e, it will stop executing after the line that fails, in this case the one that returns false.

arun@home:~$ ./with_set_e.sh 
true
true

Ignore failure in scripts

if we don’t want the script to fail after certain failing statements, we can append these certain statements with || true.

arun@home:~$ ./with_set_e_and_ignore_fail.sh           
true
true
failing foo was ignored
true

set -x

this will make the shell print each line before execution. Combining this with previous set statement and same example, it will look like

arun@home:~$ ./with_set_x.sh           
++ true
++ echo true
true
++ true
++ echo true
true
++ false

set -u

The following example has variable b which is not set. The run of the script will be successful.

arun@home:~$ ./without_set_u.sh 
++ a=0
++ echo 0
0
++ echo 0
0
++ echo

++ echo 0
0

But adding -u to the same script will force bash to treat unset variables as an error and exit immediately.

arun@home:~$ ./with_set_u.sh 
++ a=0
++ echo 0
0
++ echo 0
0
./with_set_u.sh: line 5: b: unbound variable

set -o pipefail

bash usually looks at the exit code of the last command in a pipeline. This can cause a problem for -e option as it will only consider the leftmost command’s exit code in a pipeline. This particular option sets the exit code of pipeline commands to that of the rightmost command to exit with a non-zero status or 0 if all exit successfully.

arun@home:~$ ./without_pipefail.sh 
./test.sh: line 3: a: unbound variable
++ echo 'pipe chain failed'
pipe chain failed
++ echo 'but I execute'
but I execute

arun@home:~$ ./with_pipefail.sh 
./test.sh: line 3: a: unbound variable
++ echo 'pipe chain failed'
pipe chain failed

arun@home:~$ echo $?
1

the echo $? is a special variable in bash that shows the exit code of last run command.

How to remove file in linux using regex

Fri, 19 Jul 2019 23:20:00 +0000

Deleting older files within a date range

Generally compressed log files created by utilities contain a date in the filename. For cleaning such accumulated compressed files we can use rm command with regex.

$ ls

2013-12-23-nginx.tar.gz                2016-04-29-nginx.tar.gz
2013-12-24-nginx.tar.gz                2016-05-21-nginx.tar.gz
2016-04-19-nginx.tar.gz                2016-05-22-nginx.tar.gz
2016-04-20-nginx.tar.gz                2016-05-23-nginx.tar.gz
2016-04-21-nginx.tar.gz                2016-05-24-nginx.tar.gz
2016-04-22-nginx.tar.gz                2016-05-25-nginx.tar.gz
2016-04-23-nginx.tar.gz                2016-06-11-nginx.tar.gz
2016-04-24-nginx.tar.gz                2016-06-12-nginx.tar.gz
2016-04-25-nginx.tar.gz                2016-06-13-nginx.tar.gz
2016-04-26-nginx.tar.gz                2016-12-24-nginx.tar.gz
2016-04-27-nginx.tar.gz                2019-07-20-nginx.tar.gz
2016-04-28-nginx.tar.gz


$ rm 201[3-6]*