Category Archives: Linux

Get started with InfluxDB and Python on Linux

Posted on by
2

If you are building a script, application or anything else that collects measurements or data over time intervals, a time-series database might be a good match for you. In a TSDB you simply store the data and read it back for analysis. For these use cases, there is no need for updates or changes to the data as it’s time-based. The data can’t change back in the past. InfluxDB is the most popular of these time-series databases and this article describes how to get started with it, on Debian or Ubuntu, and how to use it from Python.

Continue reading →

Upgrade Debian 9 (Stretch) to Debian 10 (Buster)

Posted on by
Reply

When there is a need to upgrade an existing Linux installation that has been running for some time, there is always a certain fear of breaking things. Especially if the machine has been doing it’s job fine for a long time. Unfortunately we can’t escape these kind of things and in most cases it’s better to move to a new version before it gets unsupported. Fortunately the procedure and success of this process has been greatly improved with recent versions of (Debian) Linux. This post will take you through the required steps to upgrade an existing Debian 9 to the current stable Debian 10 installation.

Continue reading →

Read data from the Belgian digital meter through the P1 port

Posted on by
45

Like some other Belgians, unfortunately, my analog electricity meter was replaced by a digital one since a few months. Initially, I was quite happy with the change as it allows a much easier look into your consumption and solar panel injection to the net. But exactly that last turned out to be a bummer as the government decided to not keep their word regarding reversing consumption with solar panel injection. Nevertheless, I have the thing so I might as well take advantage of it. This post describes how to connect to and use the data from the P1 port on a digital electricity meter in Belgium.

Continue reading →

Data lezen van de P1 poort op de Belgische digitale elektriciteitsmeter

Posted on by
86

Zoals enkele andere Belgen werd, jammer genoeg, ook mijn analoge elektriciteitsmeter vervangen door een digitaal exemplaar. Initieel was ik daar best wel blij mee gezien het hiermee gemakkelijker is om het verbruik en opbrengst van de zonnepanelen bij te houden. Dat laatste is net he probleem, gezien de overheid heeft beslist dat de teller niet langer kan “terugdraaien”. Los daarvan, ik heb de meter nu dus kan ik er zowel gebruik van maken. Dit artikel beschrijft hoe je de meter kunt verbinden met je computer en hoe je de data van de P1 poort kunt gebruiken voor een Belgische digital elektriciteitsmeter.

Continue reading →

Cross compiling for arm or aarch64 on Debian or Ubuntu

Posted on by
29

ARM is gaining more and more traction and is growing a lot in popularity. It’s not always possible to build directly on these ARM-based devices, especially when they are limited in resources. The majority of build and developer machines are still on x86 and by using cross compiling, it is possible to build binaries or executables usable on another architecture. For example, to use your standard PC, most likely x86, to build something that is usable on another machine or device that’s on another architecture, like ARM. In this post, I’ll explain how to do cross compiling for 32bit ARM (arm) or 64bit ARM (aarch64) using Debian 10 or Ubuntu 20.04 LTS.

Continue reading →

Forward a TCP port to another IP or port using NAT with nftables

Posted on by
20

Besides using NAT for accessing the internet with multiple machines using a single IP address, there are many other uses of NAT. One of them is to forward all traffic that is sent to a certain TCP port to another host. In practice, this technique can be used to test a service on a new host without adjusting anything on the client. The users or the clients do not need to be pointed to a new machine in order to test it. When the test would be unsuccessful, removing the NAT-rule is all it takes to switch back.

Continue reading →

BIOS or SPI programming on Windows or Linux using a CH341a MiniProgrammer

Posted on by
43

Without a working BIOS, even the most powerful hardware can’t be brought to life. In some rare cases, the BIOS chip in your system can become corrupt, which will return your system completely unusable. To fix this situation but also for other reasons like simply a backup, tuning, modding or enabling hidden features, flashing your BIOS ROM is interesting. Recently I got in the situation where the BIOS chip in my Lenovo X1 Yoga got corrupt. To fix this, I purchased the CH341a MiniProgrammer. In this post you can find the documented steps I took to reflash the BIOS and bring my laptop back to life.

Continue reading →

Install a newer kernel in Debian 10 (buster) stable

Posted on by
6

When using the latest version of Debian 10 stable, also known as buster, even with all updates installed, by default, you can’t get the most recent kernel via the standard repositories in your package manager. While the idea of using Debian stable is to remain stable and rather conservative, there are several benefits with installing a newer kernel and in some cases it’s the only option to get the OS to support all your hardware. The risk and impact on stability is small and the process is rather simple.

Continue reading →

Build/configure a Linux-based NAS

Posted on by
10

There are several options when you would like a NAS or Network Attached Storage. One of them is, instead of using dedicated NAS hardware, to build and configure your own (Linux-based) NAS system, using a standard distro. Getting dedicated NAS hardware has an advantage in terms of ease of use. On the other hand, building your own system and configuring the OS yourself has the advantage of flexibility, potential additional features and might even save you some money. But what about the power consumption? This post covers my journey for a self-built, Linux-based, NAS for home

Continue reading →

Build Docker images for deployment on non-x86 architecture

Posted on by
1

When building a Docker container image, in most cases, the machine where you build it on runs on the same CPU architecture as where you eventually want to use it. Today, more and more, there is a need for deploying containers on a different architecture as where it was built on. The most common scenario would be to build on your regular x86 desktop and to deploy to an ARM variant.

Continue reading →

Install and use CentOS 8 or RHEL 8 as KVM virtualization host

Posted on by
1

There are many possibilities to turn your machine into a host for virtual machines, emulating one or more separate and isolated computers. Among the popular commercial products from VMWare or Hyper-V on Windows, KVM is a great and mature alternative for full virtualization with great stability and performance. This guide will explain you step by step, from the start how to get this up and running.

Continue reading →

Install and use Zabbix 3.2 on CentOS 7 or RHEL 7

Posted on by
54

Zabbix is a monitoring tool which is widely used in all kinds of environments. Zabbix is very flexible, information can be retrieved using HTTP/SNMP or by installing a Zabbix agent on the machines to monitor, and allows a lot of customisation. Unfortunately this also means that the learning curve can be rather high. This post will go a little deeper into the installation process and the first steps in Zabbix.

Continue reading →

Install a newer kernel in Debian 9 (stretch) stable

Posted on by
30

When using the latest version of Debian 9 stable, even with all updates installed, by default, you can’t get a very recent kernel via the standard repositories in your package manager. While the idea of using Debian stable is to remain stable and rather conservative, there are several benefits with installing a newer kernel and in some cases it’s the only option to get the OS to support all your hardware. The risk and impact on stability is small and the process is rather simple.

Continue reading →

Install a newer kernel in CentOS 7

Posted on by
6

When using the latest version of CentOS 7, even with all updates installed, by default, you can’t get a very recent kernel via the standard repositories in your package manager. While the idea is to remain stable and rather conservative, there are several benefits with installing a newer kernel and in some cases it’s the only option to get the OS to support all your hardware. The risk and impact on stability is small and the process is rather simple.

Continue reading →

Cross compiling for ARM with Ubuntu 16.04 LTS

Posted on by
14

The goal of cross compiling is to compile for one architecture on machine running another one. In this post, I’ll try to explain the steps required to be able to compile software that is executable on ARM-based hardware using a “normal” x64-based PC or virtual machine. ARM-based devices are usually limited in processing power and are mostly running stripped-down, embedded versions of Linux. This makes it sometimes difficult to compile on the target device directly.

Continue reading →

Install the Zabbix 3.0.0 beta from source on CentOS or RHEL

Posted on by
2

It’s been a while since I was busy with Zabbix but I have been following their alpha releases for version 3 quite closely. Now that Zabbix released the first beta release for version 3, I thought that it was time to have a good look at this new and improved version. Zabbix is used a lot and more and more companies count on it as their primary monitoring solution. In this post, I’ll explain how to get started with the beta of the latest Zabbix release on CentOS 7 or RHEL 7.

Continue reading →

Create a minimal SSH-accessible chroot jail with Dropbear and BusyBox on CentOS

Posted on by
2

Sometimes you just need some kind of setup which is not so standard. This was the case which lead to this post. For some testing, I needed a minimal environment containing only a BusyBox shell and that was accessible via Dropbear SSH-server. Probably this setup is quite useless for daily use but it can be used as a base for testing with a minimum set of libraries available. Using this setup gives you the flexibility to experiment with a minimal environment without rebuilding small Linux images.

Continue reading →

Linux live kernel patching with kpatch on CentOS 7

Posted on by
7

Live or dynamic kernel patching allows you to patch a running kernel with no impact to running applications and without rebooting the system. Since the announcement and release of Linux kernel version 4, live kernel patching got got a lot of extra attention. Most probably this is because there weren’t a lot of big changes that one would expect with an increase in the version number. While v4 of the kernel does introduce some new stuff related to live or dynamic kernel patching, Live kernel patching was and is already available for earlier kernel releases. In this post, I’ll try to explain the differences between the new and old method and more important how to do live patching of a v3 kernel on CentOS 7.

Continue reading →

SELinux in a practical way

Posted on by
4

SELinux is often seen as an evil, complex, unnecessary and especially annoying security component which exists in a lot of Linux distributions. Often you can hear something like: “Disable SELinux and try again” or , “The first thing I do on a new server is to disable SELinux”. The problem with SELinux is that it looks very complex and that it looks like you need to spend ages to understand it. In this post, I’ll try to explain a few basic SELinux principles and especially focus on daily, practical problems related to SELinux and their solutions. Don’t forget that there’s a very good reason for SELinux and it would be a shame to not use it.

Continue reading →

Setup roundcube webmail as a proxy for Gmail

Posted on by
4

Recently, I was asked to find a solution or workaround for people using Gmail and are visiting countries where there’s limited or no access to Gmail. I’m not talking about bad internet connection quality but about an explicit (government) block on Gmail or related websites. An example is the block on Gmail by the Chinese Great Firewall. While there is a possibility to avoid the limitation by using a normal proxy or VPN connection, those methods are also actively blocked and monitored. A workaround is to setup your own webmail, as a proxy for Gmail.

Continue reading →

Setup a redundant PostgreSQL database with repmgr and pgpool

Posted on by
59

Using a database to store information is very common. The more information gets stored in a database, the more critical it becomes. At a certain point, the need for a redundant, high(er) available database-setup raises. When you’re using PostgreSQL, you can setup streaming replication quite easily in order to have your data redundant on two or more nodes. But wouldn’t it be a shame if you wouldn’t use the copies of your data to balance the load of your primary database-server? In this post, I’ll try to explain how to setup a redundant PostgreSQL database with load balancing.

Continue reading →

Tips & tricks for Ansible

Posted on by
9

Ansible is a very powerful configuration management tool. It can be used to deploy settings and software to one or more (virtual) machines. Using a CM-tool ensures that every deployment is done in exactly the same way. This doesn’t only make your job as a system engineer a whole lot easier, it also results in a more stable environment since every machine is consistent and more predictable. I’m using Ansible intensively for quite some time now and found that it was time to write a post about some small, but usually hard to find or annoying to solve, problems or challenges.

Continue reading →

Complile and use a realtime kernel on CentOS 7 or RHEL 7

Posted on by
15

While the need for a realtime kernel or kernel-rt hasn’t been very high recently, there are these special cases where you really need one. A “standard” linux kernel is fast but also balanced in order to treat all workload fair and give each kind of task an equal share of the system resources. A realtime kernel allows you to squeeze that extra bit of performance out of the kernel. It basically allows you to tune it in a more aggressive way in regards to timing and priorities. Since I couldn’t find any clear instructions on how to get a realtime kernel for CentOS 7 and RHEL 7, I decided to write them myself.

Continue reading →

Install and use Spacewalk 2.4 on CentOS 7 and RHEL 7

Posted on by
22

Spacewalk is the upstream-project for Redhat Satellite. It’s a tool which is mainly used to list, deploy and manage packages and updates to Fedora, CentOS, SLES and even Debian installations. Recently a new version, 2.4, came out, so let’s have a look at the installation procedure and how to perform some basic Spacewalk actions

Continue reading →

Install oVirt on CentOS 7

Posted on by
30

Recently, as you could see in previous posts, oVirt got to my attention. oVirt is the upstream project of Red Hat Enterprise Visualization or RHEV. While oVirt it isn’t providing everything that competitor VMWare ESXi does, it comes close and is a very good alternative for smaller or home setups. For a long time, oVirt wasn’t supported on Enterprise Linux 7 (like RHEL 7 or CentOS 7) but since a short time it is. Here you can find how to install oVirt and get started.

Continue reading →

Install the Cinnamon desktop environment on CentOS 7

Posted on by
7

As you might know, Linux is everything about diversity. With the numerous desktop environments available nothing is different in that area. Besides Mate and XFCE, Cinnamon is one of my favorite desktop environments on Linux. Honestly, I can’t really choose. Cinnamon is fast, (not too) fancy, customizable and familiar in some way. Here is how to install Cinnamon on a fresh minimal CentOS 7 installation.

Continue reading →

Create a PXE bootserver to install multiple Linux distributions

Posted on by
2

When regularly installing Linux hosts or VM’s, it easily becomes annoying to constantly burn CD’s/DVD’s or mount ISO’s for all the Linux distributions that you want to deploy. Especially if you want to keep them current or customize them you’ll end up with a whole lot of discs. Booting your installations from the network, using a PXE boot server, makes life a lot easier and isn’t very hard to setup. In this post I’ll explain how to setup such a PXE boot server that is able to provide multiple Linux distribution installations for deployment over the network.

Continue reading →

Setup an FTP-server with quotas on RHEL or CentOS 6 or 7 with proftpd

Posted on by
3

Using FTP actually should be avoided whenever that’s possible but sometimes it’s just the most handy and convenient way of transferring files. In most cases, your FTP-users will be able to upload files to the FTP-server. To avoid that some users would fill up the complete machine, you can use quotas. In this post, I’ll describe how to setup a basic proftpd FTP-server with quotas on RHEL or CentOS 6 and 7.

Continue reading →

Move a guest from VMWare ESX to oVirt or RHEV

Posted on by
11

When working in an environment where you have both a VMWare hypervisor and an oVirt or RHEV hypservisor or you want to migrate away from VMWare ESX to oVirt or RHEV, you will need a way to move or copy your current virtual machines from one to another. Unfortunately this can’t be done without downtime but at least it can be done. In this post I’ll try to explain how to successfully move a VM running on ESX or vCenter to an oVirt or Red Hat Enterprise Virtualization host.

Continue reading →

Create a new virtual machine in oVirt with Python using the API

Posted on by
3

Recently, I started to work or play, it’s a matter of definition, a little more with oVirt. oVirt is less known than VMWare but it’s the upstream project for Red Hat’s Enterprise Virtualization (RHEV) and based on libvirt. In an earlier post, I explained how to create VM’s on VMWare vSphere environments using Python and the VMWare API. In this post, I’ll explain how to acomplish the same using the oVirt API. It turned out to be easier than I expected.

Continue reading →

Monitor oVirt or libvirt with SNMP and Zabbix

Posted on by
11

Somehow, I expected to have little to no work when I wanted to monitor an oVirt host over SNMP. One would expect this since oVirt is the upstream project for Red Hat’s Enterprise Virtualization (RHEV) which seems to be quite strong in the market. While it isn’t that hard to get information about using SNMP for oVirt or libvirt the outcome can be disappointing. There is some documentation about it but here you can find a more hands-on explanation.

Continue reading →

Configure two network cards in a different subnet on RHEL 6, RHEL 7, CentOS 6 and CentOS 7

Posted on by
51

When configuring a Linux host running either Red Hat Linux 6, Red Hat Linux 7, CentOS 6 or CentOS7 with two network interface cards (NIC) that each have an IP address in a different network or subnet, you could end up in a situation where one of the IP addresses isn’t reachable outside it’s own network. Both IP’s will be responding to a ping from another host in the same network as those IP addresses but only one is responding to ping from another network. On most other distributions, like Debian, this issue, which is caused by asymmetric routing, doesn’t seem to exist.

Continue reading →

Compile the Huawei ES3000 PCIe SSD driver for CentOS 7 or RHEL7

Posted on by
1

The Huawei ES3000 is an SSD accelerator card connected to the PCIe bus which delivers exceptional performance. Today, I found out that Huawei doesn’t provide any ready made drivers to use a Huawei ES3000 PCIe SSD on a system running a Linux kernel > 3.x. This means that it’s not possible to use it on RHEL 7 or CentOS 7 or any other modern Linux distribution that uses a kernel newer than version 2.6.32.63. Fortunately it’s not difficult to compile the driver, install it and start to use the SSD accelerator with a newer kernel.

Continue reading →

Expand or grow a file system on a Linux VMWare VM without downtime

Posted on by
3

When your VM’s file systems are running out of space and you want to provide more space to your VM and you can’t afford any downtime, there are basically two options. Either you delete some files on the file system or you expand your current file system. Expanding your current file system can be accomplished by growing an existing virtual disk or adding a new virtual disk. Besides cleaning up, both other solutions work fine and I’ll try to explain how to get both of them active without rebooting the system or any downtime.

Continue reading →

Create a new virtual machine in Vsphere with Python, Pysphere and the VMWare API

Posted on by
38

The VMWare API is very extensive and allows you to do almost all operations that are possible with VMWare using API calls. In order to be able to easily create and deploy new virtual machines, it can be a good idea to standardize and create VM’s using a (Python) script that calls the API. In this post, I will give some examples on how to easily create a new VM using Pysphere and the VMWare API.

Continue reading →

Clone a z/VM Linux user/guest instance from another Linux guest

Posted on by
Reply

Cloning a virtual machine or user as it is called in z/VM terms from within another guest/user running Linux can be used in a lot of different scenario’s. Especially when the people managing the z/VM platform and virtual machines are less experienced with z/VM, it could be good to let them create a clone without the need to access a 3270 terminal. This scenario can also be used to let users do some self service and provide them with new instances without intervention.

Continue reading →

Install and use the openlava job scheduler and openlava web GUI on CentOS

Posted on by
30

Scheduling on Linux mostly happens with cron or any of it’s variations. Although cron is very powerful, it lacks a few features to use it in a flexible way and especially when you want to create dependencies or “communicate” with jobs running on other hosts, it has it’s shortcomings. Cron wasn’t really designed with those features in mind. Fortunately there are a few nice schedulers out there which can be used to overcome those limitations. A few of them are SOS Jobscheduler, GNUBatch and openlava. Openlava is a limited open source fork of LSF which is now owned by IBM. Openlava doesn’t come with a GUI but there is another project, Openlava web which enables control over openlava via a web interface.

Continue reading →

Write to and manage syslog messages with logger and rsyslog

Posted on by
1

Syslog is the target where you want all log message to go on all systems that you manage. Almost all Linux distributions use a syslog implementation to gather messages. Recently, rsyslog became the most used syslog-implementation for Linux. Messages can be saved locally or sent to a remote syslog server. When creating your own applications or tools or when you want to log messages coming from processes that don’t support writing to syslog directly, you can use Logger.

Continue reading →

Use DRBD in a cluster with Corosync and Pacemaker on CentOS 7

Posted on by
32

When configuring a cluster, you want tot keep managing the server as simple as possible. Theoretically, the results given by any node in the cluster should be equal as you want the cluster to be transparent to the end-user. Part of doing this, is having the same data available on every node of the cluster when it’s active. One way to do this, is using a central file-share, for example over NFS but this also has disadvantages. Another way is to have a distributed file system that stays on the nodes itself. DRBD is one of them. This post explains how to integrate DRBD in a cluster with Corosync and Pacemaker.

Continue reading →

Forward a TCP port to another IP or port using NAT with Iptables

Posted on by
25

Besides using NAT for accessing the internet with multiple machines using a single IP address, there are many other uses of NAT. One of them is to forward all traffic that is sent to a certain TCP port to another host. In practice, this technique can be used to test a service on a new host without adjusting anything on the client. The users or the clients do not need to be pointed to a new machine in order to test it. When the test would be unsuccessful, removing the NAT-rule is all it takes to switch back.

Continue reading →

Find and recover space used by deleted files, shown by df but not by du

Posted on by
3

When you’re running low on space on a file system, that can cause various unexpected behavior of the system, depending on which file system is filling up. For me, when that happens, I usually first issue a disk free (df) to see which is the file system that is almost full. Once I know which file system, I go and search which files take up the most space in that file system and take action. Sometimes, df show that a file system is almost full while, when summing up all the space by all files doesn’t even come near that value.

Continue reading →

Install and use Debian Wheezy as Xen virtualization host

Posted on by
1

Earlier, I explained how to setup CentOS or RHEL as a KVM virtualization host. You can find that  explanation here. It also contains some basic terminology about virtualization which is also applicable for Xen. When talking about KVM, somehow, I immediately associate it with the Red Hat family just as when you talk about Xen, I associate it to Debian derivatives. So for this post, I’ll use Debian to install a host that will run Xen-VM’s by using paravirtualization.

Continue reading →

Start with a simple 2-node OpenStack setup with KVM

Posted on by
17

OpenStack is something that gets more and more in the picture and even if you’re only a little interested in the latest technologies, you must have heard from OpenStack here or there. But what is it exactly and more important, how does it work practically. The best way to figure that out is just to get going with it and try to install it and play around. Here you can find a brief explanation and a tutorial or waltrough to deploy a small OpenStack environment on top of CentOS 7 or RHEL 7.

Continue reading →

Read and write a CMS minidisk on a z/VM system from a user/guest running Linux

Posted on by
Reply

When you’re managing a z/VM host running multiple Linux images, it can be interesting to have access to the data which resides on a CMS owned minidisk. Such type of minidisk is probably used to control the z/VM related configuration of the guest itself in some way. Also, when scripting, the explained technique can be used to read or write data from and to one of the minidisks. One example could be to change the PROFILE.EXEC from a user from within the user itself.

Continue reading →

Terse, unterse and transfer datasets between z/OS and other platforms via FTP

Posted on by
12

Tersing files can be compared to tarring, zipping, rarring files on the x86 platform. It allows you to store a file or multiple files (members) in an archive. The archive is easily transferable and when the data is unpacked, it is guaranteed to be the same as the original source. Especially when transferring PDS’es and variable blocked (VB, VBS) datasets, this is important. Transferring normal datasets over FTP through other platforms from z/OS can be challenging an this article should provide some help.

Continue reading →

Execute queries on a Microsoft SQL server from the Linux CLI with ODBC and Kerberos authentication

Posted on by
11

A while ago, the Windows-world and the Linux-world were not the best friends in communicating with each other. Especially not when it came to proprietary stuff like Microsoft SQL server. in contrast to everybody’s expectations, somewhere end 2011, beginning 2012, Microsoft released an ODBC driver for SQL server for Linux. This driver allows executing queries from a Linux machine to a Microsoft SQL Server database. The driver can be used in combination with Kerberos tickets and AD authentication to execute queries.

Continue reading →

Use inotify-tools on CentOS 7 or RHEL 7 to watch files and directories for events

Posted on by
13

Today, I discovered that the package inotify-tools is nowhere to be found in standard CentOS7 or RHEL7 repositories. Alternatives like incron seem to be absent as well. The inotify-tools can be used to watch a directory or file for activity and take an action when a file is changed, added, edited or simply read. You can find a workaround (or call it solution) for the absence of the inotify-tools in the repositories here.

Continue reading →

Mount Windows (CIFS) shares on Linux with credentials in a secure way

Posted on by
17

In almost all cases, when mounting a CIFS-share on a Linux host, you will need to supply some credentials. Either you could enter the credentials by hand every time you need the share or add the credentials to /etc/fstab to automatically mount the share. Entering the password manually is secure but not comfortable, leaving the password in /etc/fstab is comfortable but not secure since the file /etc/fstab is world readable.

Continue reading →

Spacewalk 2.2 installation on CentOS 7

Posted on by
14

Spacewalk is the upstream-project for Redhat Satellite. It’s a tool which is mainly used to list, deploy and manage packages and updates to Fedora, CentOS, SLE and even Debian installations. Currently, there is no official documentation on how to run Spacewalk on a el7 based installation like CentOS 7. The installation is not a straightforward as one might think but it is possible.

Continue reading →

Install and use CentOS 7 or RHEL 7 as KVM virtualization host

Posted on by
41

When thinking about virtualization, everybody immediately thinks about VMWare. And it must be said, the product they offer is very decent but also comes with a “decent” price. As an alternative, it’s worth looking into KVM for your virtualization. As with the VMWare product range, KVM offers full virtualization and it can compete with VMWare regarding stability and performance.

Continue reading →