Posts

Showing posts from July, 2022

Environment Variables in Linux

Last week, one of my friend who's new to Linux asked me some quesitons about environment variables in Linux.  It was not too difficult for me to answer her questions, but it trigged me to think how does Linux handles environment variables.  Where does the system store the environment variables? According to the Wikipedia page , "in all Unix and Unix-like systems, as well as on Windows, each process has its own separate set of environment variables. By default, when a process is created, it inherits a duplicate run-time environment of its parent process, except for explicit changes made by the parent when it creates the child. At the API level, these changes must be done between running fork and exec." Hey, this gave us some hit that the handling of environment variables is related with function call fork and exec.  Thanks to the people who created the manual page for the exec function call, I noticed it mentioned some external variable named environ as part of the...

Fixing the Ubuntu 22.04 Boot Iussing on Dell VRTX

Image
Finding the Path to the Fix As we're able to reproduce the issue with a VM running inside the VMware ESXi environment, how can we fix the issue then? My initial thought was that since it's dropping me into the (initramfs) prompt, it might indicate either some configuration, or software program, or both were missing from the initrd.img file , which is a ram disk image that the Linux booting process uses before it can load more stuff from the root file system.  The missing piece caused my system to fail to find the root file system and has to stop with the initramfs loaded.  But what exactly was missing? So I tried to explore with the (initramfs) prompt. ( initramfs ) grep multi /proc/modules   multipath 20480 0 - Live 0xffffffffc0345000   ( initramfs ) multipath   sh: multipath: not found   It seems the kernel module for multipath had been loaded already.  But there was no multipath command available.  Could it be that the package with multipath so...

Reproducing the Multpath I/O Issue With Ubuntu 22.04

Image
Problem Statement In the previous two articles, I described the issue I was running into when installing Ubuntu 22.04 on a Dell VRTX server.  After some research, I suspect it's related to the incorrect configuration of the multipath I/O function. How to fixed it? Reproducing the Issue with VMware To reproduce and fix the multipath issue, I decided to set up a testing environment with a VM running inside VMware ESXi.  If we can create a VM with two virtual SCSI HBA cards, and connect the same virtual hard drive file to them.  The Linux VM should be able to see the same hard drive twice via two SCSI paths. Multipath I/O Config In VMware ESXi First Try With VMware ESXi, when adding a SCSI card to a VM, there is one option to configure if the SCSI bus is sharing.  The default configuration is None.  In order to share the same virtual hard drive to the VM twice via those two SCSI cards, I set them to "Physical".  Ideally, setting to "Virtual" should also work. ...

Ubuntu 22.04 Multipath I/O Problem with Dell VRTX

Image
High-Level Architecture of Dell PowerEdge VRTX In the previous article , I explain the issue I am running into when installing Ubuntu 22.04 on a Dell VRTX server.  To understand a little bit more about what went wrong and potentially fix the problem, I tried to know how the Dell VRTX server was designed. After some googling, I found the  Technical Guide from Dell.  It revealed a pretty interesting design of the system.  The Dell VRTX system is designed to be a Cost-Effective Hyperconverged server system for a small office environment or remote office environment, where there are some compute power is needed but not on a large scale.  One Dell VRTX chassis can host up to 4 server blades, up to 25 x 2.5 in. NLSAS, SAS, or SAS SSD hot-plug drives (not including the hard drive bays on the server blades), and 8 flexible PCIe slots.  All those hard drives, PCIe slots, and network ports belong to the Dell VRTX chassis.  The chassis management syste...

Dropping Into "initramfs" Prompt After Installing Ubuntu 22.04 LTS on to Dell VRTX Server

Image
Background During one of the recent projects, I want to install a Linux OS onto a Dell VRTX server. Without checking the hardware compatibility list and with limited knowledge about the Dell VRTX server platform, I started to install Ubuntu 20.04 LTS on one of the server blades in the chassis. And running into some interesting issues, which forces me to learn more about the Dell VRTX platform and Linux OS. Though spending quite a lot of effort in sorting out the issue, I am happy to share it with others who are interested to know more about it. Server blades inside the Dell VRTX chassis have no hard hard drive installed. But the chassis enclosure has 25 2.5 in SAS hard drive installed. Before starting the whole process, I picked up two of the SAS hard drive and created a RAID-1 logical drive, and assigned it to the server blade. I thought the system would be able to boot from the new logical drive. The Start of the Problem After inserting the USB disk with Ubuntu 22.04, the server blad...