[{"content":"Tek.Place has been the home of all things technology since 2017. It holds guides, stories, and notes on servers, virtualization, and Linux, written from the machines I run rather than from documentation. Longer builds are written as a series.\nThe site is written and run by zeitue, founder of Zalatek and lead engineer at Defenestra; who that is and what they\u0026rsquo;ve built is on the author page.\nSome posts contain affiliate links, which means I may earn a commission if you purchase through these links; this does not increase the amount you pay for the items.\n","date":"9 September 2026","externalUrl":null,"permalink":"/about/","section":"Tek.Place","summary":"","title":"About","type":"page"},{"content":"","date":"26 February 2024","externalUrl":null,"permalink":"/authors/","section":"Authors","summary":"","title":"Authors","type":"authors"},{"content":"","date":"26 February 2024","externalUrl":null,"permalink":"/categories/","section":"Categories","summary":"","title":"Categories","type":"categories"},{"content":"","date":"26 February 2024","externalUrl":null,"permalink":"/tags/gpu-passthrough/","section":"Tags","summary":"","title":"GPU Passthrough","type":"tags"},{"content":"","date":"26 February 2024","externalUrl":null,"permalink":"/tags/graphics/","section":"Tags","summary":"","title":"Graphics","type":"tags"},{"content":"","date":"26 February 2024","externalUrl":null,"permalink":"/categories/guide/","section":"Categories","summary":"","title":"Guide","type":"categories"},{"content":"","date":"26 February 2024","externalUrl":null,"permalink":"/tags/hardware-acceleration/","section":"Tags","summary":"","title":"Hardware Acceleration","type":"tags"},{"content":"","date":"26 February 2024","externalUrl":null,"permalink":"/tags/kubuntu/","section":"Tags","summary":"","title":"Kubuntu","type":"tags"},{"content":"","date":"26 February 2024","externalUrl":null,"permalink":"/tags/kvm/","section":"Tags","summary":"","title":"Kvm","type":"tags"},{"content":"","date":"26 February 2024","externalUrl":null,"permalink":"/tags/libvirt/","section":"Tags","summary":"","title":"Libvirt","type":"tags"},{"content":"","date":"26 February 2024","externalUrl":null,"permalink":"/tags/linux/","section":"Tags","summary":"","title":"Linux","type":"tags"},{"content":"","date":"26 February 2024","externalUrl":null,"permalink":"/tags/looking-glass/","section":"Tags","summary":"","title":"Looking Glass","type":"tags"},{"content":"","date":"26 February 2024","externalUrl":null,"permalink":"/tags/lubuntu/","section":"Tags","summary":"","title":"Lubuntu","type":"tags"},{"content":"","date":"26 February 2024","externalUrl":null,"permalink":"/tags/qemu/","section":"Tags","summary":"","title":"Qemu","type":"tags"},{"content":"","date":"26 February 2024","externalUrl":null,"permalink":"/series/","section":"Series","summary":"","title":"Series","type":"series"},{"content":"One computer, two operating systems, and full hardware acceleration\nWritten in 2024 for Ubuntu. The host I run now is an immutable Fedora system; the steps are kept as written for Ubuntu. Introduction # If you\u0026rsquo;d like to run Ubuntu or another Linux-based system OS primarily but still require a Windows-only GPU-intensive application. One way of achieving this migration to Linux is to run Windows in qemu+kvm (Quick Emulator + Kernel Virtual Machine) and pass a full GPU or a vGPU (Virtual GPU ) through to Windows.\nRequirements # To run Windows virtualized on Linux, you\u0026rsquo;ll require a suitable CPU with at least four cores, although six or more with hyperthreading is recommended. You\u0026rsquo;ll also need to ensure you have at least 16GiB of memory and that your CPU and motherboard support the virtualization technologies shown below.\nAn Intel CPU supporting VT-d An AMD CPU supporting IOMMU (AMD-Vi) For the GPU, you\u0026rsquo;ll need one of the following configurations.\nTwo dGPUs (discrete GPUs), one for Linux and one for Windows An iGPU (integrated GPU) for Linux and a dGPU for Windows An iGPU or a dGPU for Linux and a vGPU (Virtual GPU) for Windows Virtual GPU support is available in prosumer and enterprise cards such as the NVIDIA Quadro or GRIDS cards, AMD Instinct, and some Radeon Pro cards. Additionally, the 5th to 9th-generation Intel CPUs with integrated graphics support GVT-g; however, unlike the AMD or NVIDIA solutions, they are less performant. Note that some of the vGPU solutions also require a license.\nIn addition to supporting virtualization, you\u0026rsquo;ll also need to ensure your motherboard has enough PCIe lines to support the necessary GPUs. You\u0026rsquo;ll want at least x8 (8 lanes) at PCIe 3 or more speeds per dGPU; for the iGPU, it should be fine as long as your motherboard can support the correct number of video outputs. One thing to note when you\u0026rsquo;re checking this is that PCIe has both physical and electrical lanes, which can cause issues since some motherboards can have x16 PCIe physical lanes, but electrically, it may be 8x or less.\nThe last thing you\u0026rsquo;ll need is a dummy plug that emulates a physical monitor; without this, Windows will turn off the video output. Below are affiliate links for dummy plugs for various port types that I\u0026rsquo;ve tested.\nDescription Link Mini Display Port Amazon Display Port Amazon 8 x Display Port Amazon HDMI Amazon 5 x HDMI Amazon 3 x DVI Amazon This post contains affiliate links, which means I may earn a commission if you purchase through these links; this does not increase the amount you pay for the items.\nHost Setup # Now that we have the hardware requirements out the way, we\u0026rsquo;ll begin configuring the hardware, transition to the software configuration, and finally finish by setting up the virtual machine.\nBIOS Configuration # You\u0026rsquo;ll need to boot to the BIOS to enable virtualization features; this will differ depending on the maker of your computer/motherboard. See the table below to determine the proper key to hold while powering on the computer.\nMaker Key(s) ASRock F2 or Delete ASUS F2 or Delete Acer F2 or Delete Dell F2 or F12 Gigabyte F2 or DEL HP F10 Lenovo F2 or Fn + F2 MSI Delete Origin PC F2 Samsung F2 System76 F2 Toshiba F2 Zotac Delete If you have a systemd-based Linux distribution such as Ubuntu or Debian, you can use this command to reboot into the BIOS setup.\n1systemctl reboot --firmware-setup Once the BIOS is loaded, search for and enable VT-d and VT-x for Intel CPUs or IOMMU (AMD-Vi) for AMD CPUs. Note some motherboards may list VT-d as virtualization or IOMMU. While still in the BIOS, you may want to adjust your GPU settings to boot using the primary GPU for the display, and if you choose to utilize a combination with an iGPU, you should configure the graphics memory and mode for it.\nAfter changing all the required settings, save the changes and reboot.\nInstall Needed Packages # Either open your package manager and install these packages:\nvirt-manager qemu-kvm qemu-utils libvirt-daemon-system libvirt-clients bridge-utils ovmf cmake gcc g++ clang libegl-dev libgl-dev libgles-dev libfontconfig-dev libgmp-dev libspice-protocol-dev make nettle-dev pkg-config binutils-dev libx11-dev libxfixes-dev libxi-dev libxinerama-dev libxss-dev libxcursor-dev libxpresent-dev libxkbcommon-dev libwayland-bin libwayland-dev wayland-protocols libpipewire-0.3-dev libsamplerate0-dev libpulse-dev libsamplerate0-dev fonts-dejavu-core libdecor-0-dev wget Or you can run this command:\n1sudo apt install -y virt-manager qemu-kvm qemu-utils libvirt-daemon-system libvirt-clients bridge-utils ovmf cmake gcc g++ clang libegl-dev libgl-dev libgles-dev libfontconfig-dev libgmp-dev libspice-protocol-dev make nettle-dev pkg-config binutils-dev libx11-dev libxfixes-dev libxi-dev libxinerama-dev libxss-dev libxcursor-dev libxpresent-dev libxkbcommon-dev libwayland-bin libwayland-dev wayland-protocols libpipewire-0.3-dev libsamplerate0-dev libpulse-dev libsamplerate0-dev fonts-dejavu-core libdecor-0-dev wget Download the Windows ISO from here and the VirtIO Drivers to the libvirt images directory.\n1# Download and copy the virtio drivers 2wget https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/stable-virtio/virtio-win.iso 3sudo mv virtio-win.iso /var/lib/libvirt/images 4 5# move the Windows installer ISO to the libvirt images directory (you may have to change the name of the file in this command) 6sudo mv ~/Downloads/Win10_22H2_English_x64v1.iso /var/lib/libvirt/images To use virt-manager, you must be in the proper group; add yourself to the group with the following command.\n1sudo usermod -a -G libvirt $USER Isolate and Detach the GPU # To pass the second GPU to the Windows virtual machine, you must isolate and detach it from the host; this will keep the host from using the GPU.\nFirst, you\u0026rsquo;ll need to enable IOMMU support in Linux by editing the file /etc/default/grub. You can use whatever editor you\u0026rsquo;d like or edit it with nano as I\u0026rsquo;ve done below.\n1sudo nano /etc/default/grub Locate the GRUB_CMDLINE_LINUX_DEFAULT field in the file and add the proper content while keeping whatever existing parameters intact.\nFor AMD, add iommu=pt amd_iommu=on the result should look like the line below.\n1GRUB_CMDLINE_LINUX_DEFAULT=\u0026#34;quiet splash iommu=pt amd_iommu=on\u0026#34; For Intel, add iommu=pt intel_iommu=on the result should look like the line below.\n1GRUB_CMDLINE_LINUX_DEFAULT=\u0026#34;quiet splash iommu=pt intel_iommu=on\u0026#34; After you\u0026rsquo;ve made your edits, save the changes, update Grub, and reboot.\n1sudo update-grub After rebooting your machine, the next thing to do is locate the IDs of the GPU you want to passthrough; you can do this as follows.\nCreate a file called /usr/local/bin/iommu_groups\n1sudo nano /usr/local/bin/iommu_groups Fill in the following content with the below script and save it. This script is a modified version of the script from the Arch Linux wiki.\n1#!/bin/bash 2shopt -s nullglob 3for g in $(find /sys/kernel/iommu_groups/* -maxdepth 0 -type d | sort -V); do 4 echo \u0026#34;IOMMU Group ${g##*/}:\u0026#34; 5 for d in $g/devices/*; do 6 echo -e \u0026#34;\\t$(lspci -D -nns ${d##*/})\u0026#34; 7 done; 8done; Make the shell script executable\n1sudo chmod +x /usr/local/bin/iommu_groups Run the iommu_groups script and check the output. You\u0026rsquo;ll want to locate the VGA compatible controller and Audio device belonging to the GPU you wish to passthrough from your output; note that some GPUs won\u0026rsquo;t have an Audio device. Once you\u0026rsquo;ve located these, you\u0026rsquo;ll want to record the numbers at the beginning of the line. My numbers were 0000:05:00.0 for the VGA compatible controller and 0000:05:00.1 for the Audio device.\nyou\u0026rsquo;ll need to ensure the devices you\u0026rsquo;re passing through to the virtual machine are not in an IOMMU group with something that can\u0026rsquo;t be passed through, such as your ISA bridge, if they are you may want to change the PCIe location of the GPU or look at the ACS override section of this guide. Below, you can see my output has the devices from isolated in group 14, so there is no issue.\nFor those who would like to know what these numbers means. The numbers follow the pattern DDDD:BB:XX.F, where DDDD is the PCI domain, BB is the bus, XX is the device, and F is the function. So for the above number 0000 means it is part of the first CPU\u0026rsquo;s PCI domain (multi CPU systems will have different domains per CPU), it is on PCI bus 05, it is device 00 on that bus and finally its function is 0 in this case representing the graphics controller.\nExample output from the command.\n1IOMMU Group 0: 2\t0000:00:00.0 Host bridge [0600]: Intel Corporation 8th/9th Gen Core 8-core Desktop Processor Host Bridge/DRAM Registers [Coffee Lake S] [8086:3e30] (rev 0d) 3IOMMU Group 1: 4\t0000:00:01.0 PCI bridge [0604]: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 0d) 5\t0000:01:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Upstream Port of PCI Express Switch [1002:1478] (rev c7) 6\t0000:02:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Downstream Port of PCI Express Switch [1002:1479] 7\t0000:03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 23 [Radeon RX 6600/6600 XT/6600M] [1002:73ff] (rev c7) 8\t0000:03:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21 HDMI Audio [Radeon RX 6800/6800 XT / 6900 XT] [1002:ab28] 9IOMMU Group 2: 10\t0000:00:02.0 Display controller [0380]: Intel Corporation CoffeeLake-S GT2 [UHD Graphics 630] [8086:3e98] (rev 02) 11IOMMU Group 3: 12\t0000:00:08.0 System peripheral [0880]: Intel Corporation Xeon E3-1200 v5/v6 / E3-1500 v5 / 6th/7th/8th Gen Core Processor Gaussian Mixture Model [8086:1911] 13IOMMU Group 4: 14\t0000:00:12.0 Signal processing controller [1180]: Intel Corporation Cannon Lake PCH Thermal Controller [8086:a379] (rev 10) 15IOMMU Group 5: 16\t0000:00:14.0 USB controller [0c03]: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller [8086:a36d] (rev 10) 17\t0000:00:14.2 RAM memory [0500]: Intel Corporation Cannon Lake PCH Shared SRAM [8086:a36f] (rev 10) 18IOMMU Group 6: 19\t0000:00:16.0 Communication controller [0780]: Intel Corporation Cannon Lake PCH HECI Controller [8086:a360] (rev 10) 20IOMMU Group 7: 21\t0000:00:17.0 SATA controller [0106]: Intel Corporation Cannon Lake PCH SATA AHCI Controller [8086:a352] (rev 10) 22IOMMU Group 8: 23\t0000:00:1b.0 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #17 [8086:a340] (rev f0) 24IOMMU Group 9: 25\t0000:00:1b.4 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #21 [8086:a32c] (rev f0) 26IOMMU Group 10: 27\t0000:00:1c.0 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #1 [8086:a338] (rev f0) 28IOMMU Group 11: 29\t0000:00:1c.2 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #3 [8086:a33a] (rev f0) 30IOMMU Group 12: 31\t0000:00:1f.0 ISA bridge [0601]: Intel Corporation Z390 Chipset LPC/eSPI Controller [8086:a305] (rev 10) 32\t0000:00:1f.3 Audio device [0403]: Intel Corporation Cannon Lake PCH cAVS [8086:a348] (rev 10) 33\t0000:00:1f.4 SMBus [0c05]: Intel Corporation Cannon Lake PCH SMBus Controller [8086:a323] (rev 10) 34\t0000:00:1f.5 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH SPI Controller [8086:a324] (rev 10) 35\t0000:00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (7) I219-V [8086:15bc] (rev 10) 36IOMMU Group 13: 37\t0000:04:00.0 Non-Volatile memory controller [0108]: Sandisk Corp WD Black SN750 / PC SN730 NVMe SSD [15b7:5006] 38IOMMU Group 14: 39\t0000:05:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP104GL [Quadro P4000] [10de:1bb1] (rev a1) 40\t0000:05:00.1 Audio device [0403]: NVIDIA Corporation GP104 High Definition Audio Controller [10de:10f0] (rev a1) 41IOMMU Group 15: 42\t0000:07:00.0 USB controller [0c03]: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller [1912:0014] (rev 03) Create the following file /etc/initramfs-tools/scripts/init-top/bind_vfio.sh.\n1sudo nano /etc/initramfs-tools/scripts/init-top/bind_vfio.sh Add the following content to the file, while replacing the numbers in the DEVICES list with the numbers you found before.\n1#!/bin/sh 2DEVICES=\u0026#34;0000:05:00.0 0000:05:00.1\u0026#34; 3for DEVICE in $DEVICES; 4 do echo \u0026#34;vfio-pci\u0026#34; \u0026gt; /sys/bus/pci/devices/$DEVICE/driver_override 5done 6 7modprobe -i vfio-pci Save the file and set the correct permissions.\n1sudo chmod 755 /etc/initramfs-tools/scripts/init-top/bind_vfio.sh 2sudo chown root:root /etc/initramfs-tools/scripts/init-top/bind_vfio.sh We need to add some modules to the initramfs, to do this edit the file /etc/initramfs-tools/modules and add the following contents.\n1vfio-pci 2vfio 3vfio_iommu_type1 Finally update the initramfs and reboot.\n1sudo update-initramfs -u 2sudo reboot Run lspci -vn and check that the Kernel driver in use value is vfio-pci for your passthrough device, if it is, then you can continue to the next steps.\nShortened example output below.\n1 205:00.0 0300: 10de:1bb1 (rev a1) (prog-if 00 [VGA controller]) 3\tSubsystem: 1028:11a3 4\tFlags: fast devsel, IRQ 255, IOMMU group 14 5\tMemory at 70000000 (32-bit, non-prefetchable) [size=16M] 6\tMemory at 6220000000 (64-bit, prefetchable) [size=256M] 7\tMemory at 6230000000 (64-bit, prefetchable) [size=32M] 8\tI/O ports at 3000 [size=128] 9\tExpansion ROM at 71000000 [disabled] [size=512K] 10\tCapabilities: \u0026lt;access denied\u0026gt; 11\tKernel driver in use: vfio-pci 12\tKernel modules: nvidiafb, nouveau 13 1405:00.1 0403: 10de:10f0 (rev a1) 15\tSubsystem: 1028:11a3 16\tFlags: fast devsel, IRQ 255, IOMMU group 14 17\tMemory at 71080000 (32-bit, non-prefetchable) [disabled] [size=16K] 18\tCapabilities: \u0026lt;access denied\u0026gt; 19\tKernel driver in use: vfio-pci 20\tKernel modules: snd_hda_intel ACS Override (Optional) # ACS (Advanced Configuration and Status) is a PCIe mechanism that groups devices for efficient power management and resource allocation. However, this grouping can become an issue for virtualization scenarios like GPU passthrough.\nACS Override is a kernel patch that disables ACS, allowing you to isolate the GPU or other devices into their own IOMMU group and pass it through to the VM independently.\nSince this is a kernel feature, you\u0026rsquo;ll either need to compile the kernel from the source code, which is not covered in this guide, or install one that has this feature already enabled. One such choice is the XanMod kernel. To install XanMod, run the following commands.\n1# Add the public key 2wget -qO - https://dl.xanmod.org/archive.key | sudo gpg --dearmor -vo /usr/share/keyrings/xanmod-archive-keyring.gpg 3 4# Add the repository 5echo \u0026#39;deb [signed-by=/usr/share/keyrings/xanmod-archive-keyring.gpg] http://deb.xanmod.org releases main\u0026#39; | sudo tee /etc/apt/sources.list.d/xanmod-release.list 6 7# This line determines which X86-64 ABI level to use 8KERNEL=$(awk \u0026#39; 9BEGIN { 10 while (!/flags/) if (getline \u0026lt; \u0026#34;/proc/cpuinfo\u0026#34; != 1) exit 1 11 if (/lm/\u0026amp;\u0026amp;/cmov/\u0026amp;\u0026amp;/cx8/\u0026amp;\u0026amp;/fpu/\u0026amp;\u0026amp;/fxsr/\u0026amp;\u0026amp;/mmx/\u0026amp;\u0026amp;/syscall/\u0026amp;\u0026amp;/sse2/) level = 1 12 if (level == 1 \u0026amp;\u0026amp; /cx16/\u0026amp;\u0026amp;/lahf/\u0026amp;\u0026amp;/popcnt/\u0026amp;\u0026amp;/sse4_1/\u0026amp;\u0026amp;/sse4_2/\u0026amp;\u0026amp;/ssse3/) level = 2 13 if (level == 2 \u0026amp;\u0026amp; /avx/\u0026amp;\u0026amp;/avx2/\u0026amp;\u0026amp;/bmi1/\u0026amp;\u0026amp;/bmi2/\u0026amp;\u0026amp;/f16c/\u0026amp;\u0026amp;/fma/\u0026amp;\u0026amp;/abm/\u0026amp;\u0026amp;/movbe/\u0026amp;\u0026amp;/xsave/) level = 3 14 if (level == 3 \u0026amp;\u0026amp; /avx512f/\u0026amp;\u0026amp;/avx512bw/\u0026amp;\u0026amp;/avx512cd/\u0026amp;\u0026amp;/avx512dq/\u0026amp;\u0026amp;/avx512vl/) level = 4 15 if (level \u0026gt; 0) { print \u0026#34;linux-xanmod-x64v\u0026#34; level; exit level + 1 } 16 exit 1 17}\u0026#39;) 18 19# Install the kernel 20sudo apt install -y \u0026#34;${KERNEL}\u0026#34; After installing the kernel, you\u0026rsquo;ll edit the file /etc/default/grub.\n1sudo nano /etc/default/grub Locate the GRUB_CMDLINE_LINUX_DEFAULT field in the file and add the following content while keeping whatever existing parameters intact.\n1GRUB_CMDLINE_LINUX_DEFAULT=\u0026#34;pcie_acs_override=downstream,multifunction\u0026#34; After you\u0026rsquo;ve made your edits, save the changes, update Grub, and reboot.\n1sudo update-grub 2sudo reboot After rebooting, run the iommu_groups command to see if your GPU is properly isolated, if it is then continue with the guide from where you left off.\nInstall Looking Glass # The Looking Glass client does not have binaries provided by the project, so you\u0026rsquo;ll have to build them yourself, but don\u0026rsquo;t worry; it\u0026rsquo;s simple, and we have already installed all the dependencies. You can download the latest source code archive from here and extract it or use the commands below.\n1wget --content-disposition https://looking-glass.io/artifact/stable/source 2tar -xf looking-glass-B6.tar.gz You\u0026rsquo;ll navigate to the source directory, create a build directory to make the client, build it, and install it.\n1# Navigate to the source directory 2cd looking-glass-B6 3 4# Create a directory to build the client 5mkdir client/build 6 7# Navigate to the build directory 8cd client/build 9 10# Use CMake to configure the build 11cmake ../ -DENABLE_LIBDECOR=ON 12 13# Compile the code 14make -j8 15 16# Install the build 17sudo make install You can launch Looking Glass from the terminal. However, you may prefer a graphical launcher; copy the icon from the source directory and create some launchers for the virtual machine.\n1# Make sure pixmaps directory exists 2sudo mkdir -p /usr/local/share/pixmaps/ 3 4# Make sure applications directory exists 5sudo mkdir -p /usr/local/share/applications/ 6 7# Copy the icon from the source directory to pixmaps 8sudo cp ../../resources/icon-128x128.png /usr/local/share/pixmaps/looking-glass-client.png 9 10# Create fullscreen launcher 11cat \u0026lt;\u0026lt;EOF | sudo dd status=none of=\u0026#34;/usr/local/share/applications/looking-glass-fullscreen.desktop\u0026#34; 12[Desktop Entry] 13Comment= 14Exec=/usr/local/bin/looking-glass-client -F egl:doubleBuffer=no 15GenericName=Use The Looking Glass Client to Connect to Windows in Fullscreen Mode 16Icon=/usr/local/share/pixmaps/looking-glass-client.png 17Name=Looking Glass Windows (Fullscreen) 18NoDisplay=false 19Path= 20StartupNotify=true 21Terminal=false 22TerminalOptions= 23Type=Application 24Categories=Utility; 25X-KDE-SubstituteUID=false 26X-KDE-Username= 27EOF 28 29# Create windowed launcher 30cat \u0026lt;\u0026lt;EOF | sudo dd status=none of=\u0026#34;/usr/local/share/applications/looking-glass-windowed.desktop\u0026#34; 31[Desktop Entry] 32Comment= 33Exec=/usr/local/bin/looking-glass-client -T egl:doubleBuffer=no 34GenericName=Use The Looking Glass Client to Connect to Windows in Windowed Mode 35Icon=/usr/local/share/pixmaps/looking-glass-client.png 36Name=Looking Glass Windows (Windowed) 37NoDisplay=false 38Path= 39StartupNotify=true 40Terminal=false 41TerminalOptions= 42Type=Application 43Categories=Utility; 44X-KDE-SubstituteUID=false 45X-KDE-Username= 46EOF Create the shared memory file for the Looking Glass host and client to read and write.\n1cat \u0026lt;\u0026lt;EOF | sudo dd status=none of=\u0026#34;/etc/tmpfiles.d/10-looking-glass.conf\u0026#34; 2# Type Path Mode UID GID Age Argument 3f /dev/shm/looking-glass 0660 libvirt-qemu libvirt - 4EOF Edit the /etc/systemd/logind.conf file and ensure you have RemoveIPC=no; this will keep the shared memory file from being deleted randomly.\n1sudo nano /etc/systemd/logind.conf your /etc/systemd/logind.conf should look like the following.\n1[Login] 2#NAutoVTs=6 3#ReserveVT=6 4#KillUserProcesses=no 5#KillOnlyUsers= 6#KillExcludeUsers=root 7#InhibitDelayMaxSec=5 8#UserStopDelaySec=10 9#HandlePowerKey=poweroff 10#HandleSuspendKey=suspend 11#HandleHibernateKey=hibernate 12#HandleLidSwitch=suspend 13#HandleLidSwitchExternalPower=suspend 14#HandleLidSwitchDocked=ignore 15#HandleRebootKey=reboot 16#PowerKeyIgnoreInhibited=no 17#SuspendKeyIgnoreInhibited=no 18#HibernateKeyIgnoreInhibited=no 19#LidSwitchIgnoreInhibited=yes 20#RebootKeyIgnoreInhibited=no 21#HoldoffTimeoutSec=30s 22#IdleAction=ignore 23#IdleActionSec=30min 24#RuntimeDirectorySize=10% 25#RuntimeDirectoryInodesMax=400k 26RemoveIPC=no 27#InhibitorsMax=8192 28#SessionsMax=8192 Create the temporary memory file.\n1sudo systemd-tmpfiles --create Create the Virtual Machine # Open virt-manager from your menu, click Edit-\u0026gt;Preferences, then under General, checkmark Enable XML editing and click Close. Click the plus button, select Local install media (ISO images or CDROM), then click Forward. Click Browse and then select the ISO you downloaded earlier; if you don\u0026rsquo;t see it, you may have to push the refresh button for it to show up. After you select it, click Choose volume and then Forward to continue. For memory, you\u0026rsquo;ll want at least 8 GIB (8192); for the CPUs, you\u0026rsquo;ll want at least 4, but 6 or more is preferred. After you make your selection, click Forward to continue. For Windows, you\u0026rsquo;ll need to give it at least 80 GiB; I generally give each virtual machine 128 GiB. After you make your selection, click Forward to continue. Set your virtual machine\u0026rsquo;s name, then checkmark Customize configuration before install and click Finish. In the overview section, you\u0026rsquo;ll want to select UEFI x86_64: /usr/share/OVMF/OVMF_CODE_4M.ms.fd for the firmware. As you progress through the different sections, popups will ask you to apply your changes; click Yes each time. Under the CPUs section, checkmark Manually set CPU topology and set sockets to 1 and cores to the number you chose divided by 2. In this case, it\u0026rsquo;ll be 3, and finally, set threads to 2, giving the virtual machine a more common configuration. For SATA Disk 1, change the disk bus to VirtIO; note that\u0026rsquo;ll change it to VirtIO Disk 1. Click Add Hardware from the bottom of the sidebar, then click Storage and choose CDROM for the device type. Click Manage and select the virtio-win.iso downloaded earlier, then click Choose Volume and then click Finish. Click the NIC section and change the device model to virtio. Finally, start the virtual machine up by clicking Begin installation. Guest setup # Installing Windows # When the virtual machine starts booting off the CD, you may have to hit enter to get it to boot Windows. Once the installer is booted, select your region information and click Next. Click Install Now. If you have a product key, enter it now, then click Next or choose I don't have a product key. If you don\u0026rsquo;t have a product key, choose your Windows version to install; I suggest choosing Windows 10 Pro X64, then click Next. Accept the agreement with Microsoft to give away your privacy and click Next. Select Custom: Install Windows only (advanced) Click Load driver from the bottom of the dialog. Click OK Select the driver for Windows 10 from the list and click Next. Click New Click Apply Windows will now tell you that it doesn\u0026rsquo;t care what you did, and it will do what it wants with the partition table for it to work; click OK. You should now see your new partition layout created by Windows; click Next to continue. Confirm your region. Confirm your layout. Add a second layout if wanted. Click I don't have Internet. Click Continue with limited setup. Fill in your chosen username. Fill in your password. Confirm your password. Fill in whatever security questions you want; I use cat input for the security questions, so they\u0026rsquo;re unusable. Choose your privacy settings, even though it will probably ignore them. Choose what to do with Cortana. Navigate to the mounted virtio-win.iso under D:, then launch virtio-win-gt-x64.exe. Click Next. Accept the agreement and click Next. Click Next. Click Install. Click Finish. Install Looking Glass # Download the Windows Looking Glass Host binary in the virtual machine; the binary is available here. Run the downloaded Looking Glass host setup binary. Microsoft Defender will stop the execution of the binary; click More info then Run anyway. Click Next. Click Agree. Click Next. Click Install. Click Close. After installing Looking Glass, shut down Windows.\nEnable Passthrough # Open the Windows virtual machine and click the hardware information button from the top toolbar. Click Video QXL from the sidebar and change the model to None. Click Add Hardware from the bottom of the sidebar and choose `PCI Host Device from the dialog\u0026rsquo;s sidebar. Find the GPU you chose to pass through to the virtual machine from the list and click Finish. Repeat the process and find the audio controller belonging to the GPU. Click Overview, then click the XML tab, scroll to the bottom, and find the closing tag \u0026lt;/devices\u0026gt;. Above it, paste the following contents, then click Apply. Note that when you click Apply, the line will edit to include the PCI part seen in the image below.\n1\u0026lt;shmem name=\u0026#34;looking-glass\u0026#34;\u0026gt; 2\t\u0026lt;model type=\u0026#34;ivshmem-plain\u0026#34;/\u0026gt; 3\t\u0026lt;size unit=\u0026#34;M\u0026#34;\u0026gt;512\u0026lt;/size\u0026gt; 4\u0026lt;/shmem\u0026gt; After you finish changing all of the above settings, you should be able to boot the virtual machine and launch Looking Glass; you\u0026rsquo;ll want to ensure that you have your dummy plug plugged in the chosen GPU before you boot it.\n","date":"26 February 2024","externalUrl":null,"permalink":"/posts/2024/02/setting-up-gpu-passthrough-with-kvm-on-ubuntu-for-windows/","section":"Posts","summary":"One computer, two operating systems, and full hardware acceleration\n","title":"Setting up GPU passthrough with KVM on Ubuntu for Windows","type":"blog"},{"content":"","date":"26 February 2024","externalUrl":null,"permalink":"/tags/","section":"Tags","summary":"","title":"Tags","type":"tags"},{"content":"","date":"26 February 2024","externalUrl":null,"permalink":"/tags/ubuntu/","section":"Tags","summary":"","title":"Ubuntu","type":"tags"},{"content":"","date":"26 February 2024","externalUrl":null,"permalink":"/tags/ubuntu-22.04/","section":"Tags","summary":"","title":"Ubuntu 22.04","type":"tags"},{"content":"","date":"26 February 2024","externalUrl":null,"permalink":"/tags/ubuntu-24.04/","section":"Tags","summary":"","title":"Ubuntu 24.04","type":"tags"},{"content":"","date":"26 February 2024","externalUrl":null,"permalink":"/tags/virtio-win/","section":"Tags","summary":"","title":"Virtio-Win","type":"tags"},{"content":"","date":"26 February 2024","externalUrl":null,"permalink":"/series/virtualization/","section":"Series","summary":"","title":"Virtualization","type":"series"},{"content":"","date":"26 February 2024","externalUrl":null,"permalink":"/tags/windows/","section":"Tags","summary":"","title":"Windows","type":"tags"},{"content":"","date":"26 February 2024","externalUrl":null,"permalink":"/tags/xubuntu/","section":"Tags","summary":"","title":"Xubuntu","type":"tags"},{"content":"","date":"2 January 2024","externalUrl":null,"permalink":"/series/docker/","section":"Series","summary":"","title":"Docker","type":"series"},{"content":"","date":"2 January 2024","externalUrl":null,"permalink":"/tags/docker/","section":"Tags","summary":"","title":"Docker","type":"tags"},{"content":"","date":"2 January 2024","externalUrl":null,"permalink":"/tags/docker-compose/","section":"Tags","summary":"","title":"Docker Compose","type":"tags"},{"content":"","date":"2 January 2024","externalUrl":null,"permalink":"/tags/reverse-proxy/","section":"Tags","summary":"","title":"Reverse Proxy","type":"tags"},{"content":"Introduction # Traefik is a modern reverse proxy and load balancer programmed in Go. It works with Docker and Kubernetes and runs on Linux, FreeBSD, macOS, OpenBSD, and Windows. Traefik will configure all containers from labels defined in the docker-compose.yml files and auto-provision SSL certificates via a chosen provider. There are two methods for setup offered in this guide: the first is Quickstart using docker-prompt with my infrastructure git repository, and the second is that you\u0026rsquo;ll create all the configurations manually. The second method is better for learning.\nCloudflare # If you want to use Cloudflare DNS, you’ll need the email of your Cloudflare account and an API token with permission to edit DNS zones. To get an API token, navigate to dash.cloudflare.com/profile/api-tokens and click Create Token. Under API token templates, you can click on Use Template next to Edit zone DNS. Under Zone Resources, select a specific zone you want the API token for or set All zones, then click Continue to summary. Click the Create token button and copy the API token for later use.\nQuickstart # 1# Clone the git repository 2git clone https://gitlab.com/tek.place/infrastructure.git 3 4# Navigate to the Traefik docker-compose files 5cd ~/infrastructure/docker/traefik 6 7# Setup the environment with docker-prompt 8docker-prompt env.json 9 10# Create networks 11./scripts/mknet 12 13# Create directories to store data 14./scripts/mkvol 15 16# Start up traefik 17./scripts/up Manual Setup # You’ll first want to create a network to which all containers you wish to publish will connect.\n1docker network create --attachable --gateway=172.21.0.1 --subnet=172.21.0.0/16 proxy Create a network if you want to run Netcloud-AIO and still need to set it up.\n1docker network create --attachable --gateway=172.18.0.1 --subnet=172.18.0.0/24 nextcloud-aio Create a place for your configuration, docker-compose file, and volumes, then navigate there.\n1mkdir -p ~/infrastructure/docker/traefik 2cd ~/infrastructure/docker/traefik Create a directory to back the Traefik config volume; in this case, we’ll store it in the ~/infrastructure/docker/traefik directory for simplicity.\n1mkdir config Generate a hashed password to use to access Traefik\u0026rsquo;s admin dashboard. The result should start with a $ sign; insert it in the .env file instead of YOUR_HASHED_ADMIN_PASSWORD_HERE.\n1openssl passwd -apr1 Create the environment for Traefik, a file named .env containing all the environment variables. Be sure to replace name@example.com, YOUR_HASHED_ADMIN_PASSWORD_HERE, YOUR_CLOUDFLARE_API_KEY_HERE, and domain.com with the proper content.\n1# filename: .env 2# DNS Challenge provider 3ACME_CHALLENGE_PROVIDER=cloudflare 4 5# Email for Let\u0026#39;s Encrypt 6ACME_EMAIL=name@example.com 7 8# Cloudflare email address 9CF_API_EMAIL=name@example.com 10 11# Cloudflare API key 12CF_DNS_API_TOKEN=YOUR_CLOUDFLARE_API_KEY_HERE 13 14# Traefik HTTPS port 15HTTPS_PORT=443 16 17# Traefik HTTP port 18HTTP_PORT=80 19 20# Log level 21LOG_LEVEL=info 22 23# Volume used for storing config files 24TRAEFIK_CONFIG_VOL_DIR=${PWD}/config 25 26# password for HTTP Basic Auth for Traefik dashboard 27TRAEFIK_DASHBOARD_PASS=YOUR_HASHED_ADMIN_PASSWORD_HERE 28 29# username for HTTP Basic Auth for Traefik dashboard 30TRAEFIK_DASHBOARD_USER=admin 31 32# Domain to access Traefik dashboard and API 33TRAEFIK_DOMAIN=traefik.domain.com Create a file called docker-compose.yml with the following contents.\n1version: \u0026#34;3.8\u0026#34; 2 3# Use external networks 4networks: 5 proxy: 6 external: true 7 nextcloud-aio: 8 external: true 9 10# Create a data volume backed by a local directory 11volumes: 12 traefik_config: 13 name: ${PREFIX}traefik_config 14 driver: local 15 driver_opts: 16 o: bind 17 type: none 18 device: \u0026#34;${TRAEFIK_CONFIG_VOL_DIR}\u0026#34; 19 20services: 21 traefik: 22 image: \u0026#34;traefik:v2.10\u0026#34; 23 container_name: \u0026#34;${PREFIX:-}traefik\u0026#34; 24 ports: 25 - \u0026#34;${HTTP_PORT:-80}:80\u0026#34; 26 - \u0026#34;${HTTPS_PORT:-443}:443\u0026#34; 27 env_file: 28 - \u0026#34;../.env\u0026#34; 29 volumes: 30 - \u0026#34;traefik_config:/etc/traefik\u0026#34; 31 # Docker socket to be able to access containers 32 - \u0026#34;/var/run/docker.sock:/var/run/docker.sock:ro\u0026#34; 33 networks: 34 - proxy 35 - nextcloud-aio 36 labels: 37 # Configure Traefik labels for the dashboard with HTTP basic auth 38 traefik.enable: true 39 traefik.http.routers.api.entrypoints: websecure 40 traefik.http.routers.api.rule: Host(`${TRAEFIK_DOMAIN}`) 41 traefik.http.routers.api.service: api@internal 42 traefik.http.routers.api.tls.certresolver: le 43 traefik.http.routers.api.middlewares: dashboard-auth 44 traefik.http.middlewares.dashboard-auth.basicauth.users: \u0026#34;${TRAEFIK_DASHBOARD_USER}:${TRAEFIK_DASHBOARD_PASS}\u0026#34; 45 command: 46 - \u0026#34;--certificatesresolvers.le.acme.dnschallenge=true\u0026#34; 47 - \u0026#34;--certificatesresolvers.le.acme.dnschallenge.provider=${ACME_CHALLENGE_PROVIDER}\u0026#34; 48 - \u0026#34;--certificatesresolvers.le.acme.email=${ACME_EMAIL}\u0026#34; 49 - \u0026#34;--certificatesresolvers.le.acme.storage=/etc/traefik/acme.json\u0026#34; 50 - \u0026#34;--log.level=${LOG_LEVEL:-debug}\u0026#34; 51 - \u0026#34;--api.insecure=false\u0026#34; 52 - \u0026#34;--api.dashboard=${__TRAEFIK_DASHBOARD:-true}\u0026#34; 53 - \u0026#34;--providers.docker=true\u0026#34; 54 - \u0026#34;--providers.docker.exposedbydefault=false\u0026#34; 55 - \u0026#34;--entrypoints.web.address=:80\u0026#34; 56 - \u0026#34;--entrypoints.websecure.address=:443\u0026#34; 57 - \u0026#34;--entrypoints.web.http.redirections.entrypoint.to=websecure\u0026#34; 58 - \u0026#34;--entrypoints.web.http.redirections.entrypoint.scheme=https\u0026#34; Finally, start Traefik using the docker-compose command.\n1docker-compose up -d","date":"2 January 2024","externalUrl":null,"permalink":"/posts/2024/01/traefik/","section":"Posts","summary":"","title":"Traefik","type":"blog"},{"content":"","date":"2 January 2024","externalUrl":null,"permalink":"/tags/traefik/","section":"Tags","summary":"","title":"Traefik","type":"tags"},{"content":"","date":"1 January 2024","externalUrl":null,"permalink":"/tags/caddy/","section":"Tags","summary":"","title":"Caddy","type":"tags"},{"content":"Introduction # Caddy is an open-source web server written in Go. It\u0026rsquo;s fast, secure, and multiplatform, supporting Linux, Windows, macOS, FreeBSD, DragonflyBSD, and OpenBSD when writing this. Caddy will automatically obtain certificates from Let\u0026rsquo;s Encrypt to provide HTTPS for all your websites. The version of Caddy used in this guide includes several modules, such as Caddy-Docker-Proxy, that allow excellent integration with Docker by using labels similar to Traefik. There are two methods for setup offered in this guide: the first is Quickstart using docker-prompt with my infrastructure git repository, and the second is that you\u0026rsquo;ll create all the configurations manually. The second method is better for learning.\nCloudflare # If you want to use Cloudflare DNS, you’ll need the email of your Cloudflare account and an API token with permission to edit DNS zones. To get an API token, navigate to dash.cloudflare.com/profile/api-tokens and click Create Token. Under API token templates, you can click on Use Template next to Edit zone DNS. Under Zone Resources, select a specific zone you want the API token for or set All zones, then click Continue to summary. Click the Create token button and copy the API token for later use.\nQuickstart # 1# Clone the git repository 2git clone https://gitlab.com/tek.place/infrastructure.git 3 4# Navigate to the Caddy docker-compose files 5cd ~/infrastructure/docker/caddy 6 7# Setup the environment with docker-prompt 8docker-prompt env.json 9 10# Create networks 11./scripts/mknet 12 13# Increase UDP packet limit, which will take effect on the next reboot 14./scripts/incudp 15 16# Create directories to store data 17./scripts/mkvol 18 19# Start up caddy 20./scripts/up Manual Setup # You’ll first want to create a network to which all containers you wish to publish will connect.\n1docker network create --attachable --gateway=172.21.0.1 --subnet=172.21.0.0/16 proxy Create a network if you want to run Netcloud-AIO and still need to set it up.\n1docker network create --attachable --gateway=172.18.0.1 --subnet=172.18.0.0/24 nextcloud-aio Next, we’ll increase the UDP packet limit to avoid errors in Caddy’s logs. Note this will take effect after rebooting.\n1cat \u0026lt;\u0026lt;EOF | sudo dd status=none of=\u0026#34;/etc/sysctl.d/99-upd-recv-buf-size.conf\u0026#34; 2net.core.rmem_max=2500000 3EOF Create a place for your configuration, docker-compose file, and volumes, then navigate there.\n1mkdir -p ~/infrastructure/docker/caddy 2cd ~/infrastructure/docker/caddy Create a directory to back the Caddy data volume; in this case, we’ll store it in the ~/infrastructure/docker/caddy directory for simplicity.\n1mkdir data Create the environment for Caddy, a file named .env containing all the environment variables. Be sure to replace YOUR_CLOUDFLARE_API_KEY_HERE and name@example.com with your correct information.\n1# filename: .env 2 3# Volume used for storing data 4CADDY_DATA_VOL_DIR=${PWD}/data 5 6# The network name used for Caddy ingress 7# These are the networks Caddy will look at 8CADDY_INGRESS_NETWORKS=proxy,nextcloud-aio 9 10# Cloudflare API key 11CLOUDFLARE_API_KEY=YOUR_CLOUDFLARE_API_KEY_HERE 12 13# Cloudflare Email 14CLOUDFLARE_EMAIL=name@example.com Create a file called docker-compose.yml with the following contents.\n1# filename: docker-compose.yml 2version: \u0026#34;3.8\u0026#34; 3 4# Access external networks 5networks: 6 proxy: 7 external: true 8 nextcloud-aio: 9 external: true 10 11volumes: 12 data: 13 name: ${PREFIX:-}caddy_data 14 driver: local 15 driver_opts: 16 o: bind 17 type: none 18 device: \u0026#34;${CADDY_DATA_VOL_DIR}\u0026#34; 19 20services: 21 caddy: 22 image: homeall/caddy-reverse-proxy-cloudflare:latest 23 container_name: ${PREFIX:-}caddy 24 restart: unless-stopped 25 env_file: ../.env 26 ports: 27 - 80:80 28 - 443:443 29 volumes: 30 - caddy_data:/data 31 # This is included to allow Caddy to read the labels from containers 32 # connected to the proxy or nextcloud-aio networks 33 - /var/run/docker.sock:/var/run/docker.sock 34 networks: 35 - proxy 36 - nextcloud-aio 37 labels: 38 # Caddy uses the labels section to get the Cloudflare email and API key. 39 caddy.email: \u0026#34;${CLOUDFLARE_EMAIL:-}\u0026#34; 40 caddy.acme_dns: \u0026#34;cloudflare ${CLOUDFLARE_API_KEY:-}\u0026#34; Now that you have all the files created and the needed information for the environment filled out, the only thing left to do is start the containers; you can do that using the docker-compose command.\n1docker-compose up -d","date":"1 January 2024","externalUrl":null,"permalink":"/posts/2024/01/caddy-v2/","section":"Posts","summary":"","title":"Caddy v2","type":"blog"},{"content":"","date":"28 December 2023","externalUrl":null,"permalink":"/tags/amd/","section":"Tags","summary":"","title":"AMD","type":"tags"},{"content":"","date":"28 December 2023","externalUrl":null,"permalink":"/tags/asrock/","section":"Tags","summary":"","title":"ASRock","type":"tags"},{"content":"","date":"28 December 2023","externalUrl":null,"permalink":"/tags/corsair/","section":"Tags","summary":"","title":"Corsair","type":"tags"},{"content":"","date":"28 December 2023","externalUrl":null,"permalink":"/tags/cpu/","section":"Tags","summary":"","title":"CPU","type":"tags"},{"content":"Categorical list of devices and parts\nThis page is an extensive list of links to products I use in computer/server builds and other devices I\u0026rsquo;ve used or tested. This post contains affiliate links, which means I may earn a commission if you purchase through these links; this does not increase the amount you pay for the items.\nComputer Parts # Processors (CPU) # Description Link AMD EPYC 7443P Amazon AMD Ryzen 7 5700G Amazon Intel Core i9-9900K Amazon Intel Xeon E-2146G Amazon CPU Coolers # Description Link NH-U9 TR4-SP3 Amazon Cooler Master Hyper 212 EVO Amazon NZXT Kraken X73 Amazon Memory (RAM) # Description Link ASRock Rack ROMED8-2T AMD EPYC 64GB RDIMM Amazon G.Skill Ripjaws V 32 GB Amazon Mushkin Proline 32 GB Amazon Motherboards # Description Link ROMED8-T2 Amazon MSI PRO B550M-VC WiFi Amazon Gigabyte C246-WU4 Graphics Cards # Description Link NVIDIA RTX A5000 Amazon NVIDIA Tesla P40 Amazon EVGA GeForce RTX 3050 XC Gaming Amazon GeForce RTX 3080 VENTUS 3X 10G OC Amazon Hard Drives (HDD) # Description Link Western Digital Ultrastar DC HC550 18TB 7200RPM 3.5\u0026quot; Amazon Western Digital Ultrastar DC HC510 10TB 7200RPM 3.5\u0026quot; (Renewed) Amazon Western Digital Red 8 TB 3.5\u0026quot; Amazon Western Digital Red Plus 8TB Amazon NVME Drives # Description Link TEAMGROUP MP33 1TB SLC Amazon TEAMGROUP MP33 256GB SLC Amazon WD Black 1TB SN850X Amazon WD Black 1TB SN770 Amazon Explansion Cards # Description Link KONYEAD pcie4.0 x16 to m.2 m-Key nvme x 4 ssd Expansion Card Amazon Cases # Description Link anidees AI Crystal XL Amazon Corsair Carbide Series 200R iStarUSA D-400-6 iStarUSA RackChoice 3u Rackmount Amazon Drive Cages # Description Link Rosewill 3 x 5.25-Inch to 4 x 3.5-Inch Hot-swap Amazon Fans # Description Link Noctua NF-A6x25 PWM Amazon Noctua NF-A8 PWM Amazon Noctua NF-A4x10 FLX Amazon Power Supplies # Description Link Corsair RM1000e Amazon EVGA 850 GT Amazon PC Power \u0026amp; Cooling FPS1050-A5M00 Amazon ","date":"28 December 2023","externalUrl":null,"permalink":"/posts/2023/12/devices-and-parts/","section":"Posts","summary":"Categorical list of devices and parts\n","title":"Devices and Parts","type":"blog"},{"content":"","date":"28 December 2023","externalUrl":null,"permalink":"/tags/evga/","section":"Tags","summary":"","title":"EVGA","type":"tags"},{"content":"","date":"28 December 2023","externalUrl":null,"permalink":"/tags/hard-drive/","section":"Tags","summary":"","title":"Hard Drive","type":"tags"},{"content":"","date":"28 December 2023","externalUrl":null,"permalink":"/tags/hgst/","section":"Tags","summary":"","title":"HGST","type":"tags"},{"content":"","date":"28 December 2023","externalUrl":null,"permalink":"/tags/intel/","section":"Tags","summary":"","title":"Intel","type":"tags"},{"content":"","date":"28 December 2023","externalUrl":null,"permalink":"/tags/memory/","section":"Tags","summary":"","title":"Memory","type":"tags"},{"content":"","date":"28 December 2023","externalUrl":null,"permalink":"/tags/msi/","section":"Tags","summary":"","title":"MSI","type":"tags"},{"content":"","date":"28 December 2023","externalUrl":null,"permalink":"/tags/noctua/","section":"Tags","summary":"","title":"Noctua","type":"tags"},{"content":"","date":"28 December 2023","externalUrl":null,"permalink":"/tags/nvidia/","section":"Tags","summary":"","title":"NVIDIA","type":"tags"},{"content":"","date":"28 December 2023","externalUrl":null,"permalink":"/categories/resources/","section":"Categories","summary":"","title":"Resources","type":"categories"},{"content":"","date":"28 December 2023","externalUrl":null,"permalink":"/tags/solid-state-drive/","section":"Tags","summary":"","title":"Solid State Drive","type":"tags"},{"content":"","date":"28 December 2023","externalUrl":null,"permalink":"/tags/western-digital/","section":"Tags","summary":"","title":"Western Digital","type":"tags"},{"content":"","date":"24 December 2023","externalUrl":null,"permalink":"/tags/balenaetcher/","section":"Tags","summary":"","title":"BalenaEtcher","type":"tags"},{"content":"","date":"24 December 2023","externalUrl":null,"permalink":"/tags/dd/","section":"Tags","summary":"","title":"Dd","type":"tags"},{"content":"","date":"24 December 2023","externalUrl":null,"permalink":"/tags/firewall/","section":"Tags","summary":"","title":"Firewall","type":"tags"},{"content":"","date":"24 December 2023","externalUrl":null,"permalink":"/tags/installation/","section":"Tags","summary":"","title":"Installation","type":"tags"},{"content":"","date":"24 December 2023","externalUrl":null,"permalink":"/tags/networking/","section":"Tags","summary":"","title":"Networking","type":"tags"},{"content":"","date":"24 December 2023","externalUrl":null,"permalink":"/tags/self-hosted/","section":"Tags","summary":"","title":"Self-Hosted","type":"tags"},{"content":"","date":"24 December 2023","externalUrl":null,"permalink":"/series/self-hosted-infrastructure-from-homelab-to-company/","section":"Series","summary":"","title":"Self-Hosted Infrastructure From Homelab to Company","type":"series"},{"content":"Installing and Configuring Ubuntu Server\nWritten in 2023 for Ubuntu Server 22.04 with Docker Compose, Caddy, and FreeIPA. The stack I run has since moved to Fedora with Podman quadlets and Traefik; the series is kept as written. Introduction # As mentioned in the previous article, we will use the Ubuntu server for the OS; this will go through making the installation media, installing the Ubuntu, installing Docker and Docker-compose, and finally, we\u0026rsquo;ll finish by doing some configuration.\nCreating the Installation Media # You\u0026rsquo;ll need a USB flash drive or a CD (if your server has a disc drive) to install Ubuntu Server 22.04. To make a USB install disk, you\u0026rsquo;ll also need a disk writing utility like balenaEtcher, Fedora Media Writer, GNOME Disks, dd, etc. I\u0026rsquo;ll cover how to do this with balenaEtcher and dd since that covers both graphical and command line ways of making a USB install disk. The last thing you\u0026rsquo;ll need is the actual installation ISO image from Ubuntu\u0026rsquo;s website; download it from here.\nCommand line # The dd command is a program that copies and converts files and data. We\u0026rsquo;ll use it to write our ISO file to a USB flash drive. The below dd command will write the ubuntu-22.04.3-live-server-amd64.iso ISO file to our USB flash drive at /dev/sdc. Ensure you update the paths to match the location of your files accordingly (I don\u0026rsquo;t want you to erase your important data).\n1sudo dd if=~/Downloads/ubuntu-22.04.3-live-server-amd64.iso of=/dev/sdc status=progress; sync So here is the breakdown of what\u0026rsquo;s going on\nif= is assigning the input file that we are going to read from of= is the output file which we will write to, in this case, a device status=progress gives us a nice progress bar sync at the end flashes the data to the disk Graphical # You can download balenaEtcher from https://etcher.balena.io/#download-etcher. Download and install the appropriate package for your system, then Launch it.\nSelect Flash from file and open the ubuntu-22.04.3-live-server-amd64.iso ISO downloaded earlier Click Select target and select your USB flash drive If you don\u0026rsquo;t see it, you may need to click the Show hidden dropdown Click Flash and wait for it to finish Done, you can now remove the USB flash drive Installing Ubuntu Server # Insert the USB flash drive you made in the previous step into the machine in which you\u0026rsquo;ll be installing Ubuntu server 22.04. Power on the device and hold down your machine\u0026rsquo;s boot select key, commonly F11 or F12. I\u0026rsquo;ve also seen F7 or ESC used; if you can\u0026rsquo;t find the correct key, you can look it up in your motherboard\u0026rsquo;s manual. After you boot your installation media, you should see a screen like the one below; generally, you\u0026rsquo;ll choose the top option. However, if you have relatively new hardware or run into any issues, you\u0026rsquo;ll want to select Ubuntu Server with the HWE kernel; this gives you access to a more recent kernel. Language and Keyboard # Once you get to the installer, you\u0026rsquo;ll need to choose which language to use; you can use the arrow keys and enter to navigate and select. At this point, the installer might prompt you to update; it\u0026rsquo;s usually a good idea to let it update to the most recent version to avoid any issues during installation. After this, you\u0026rsquo;ll choose your keyboard layout. Installation Type # On the next screen, you\u0026rsquo;ll choose the installation type; you\u0026rsquo;ll want to leave it as Ubuntu Server; you may want to check the option Search for third-party drivers if you have any proprietary hardware like NVIDIA GPUs. Configuring Networking # In general, you\u0026rsquo;ll want to manually set up the network on your server with a static IP address and a bond interface if your server has more than one ethernet port. A static IP is an IP address that does not change and allows the server to be available at a known IP. Network bonding allows treating two or more network interfaces as one; this allows for load balancing if the router you have supports it. In a bond, if something happens to the first interface, the second could take its place without causing disruption.\nIf you have multiple ethernet cards, select Create bond and then select the devices to use with the bond; in my case, I chose all of them. For Bond mode, choose active-backup for failover or balance-rr for load balancing, then select create. To configure the interface with a static IP address, select Edit IPv4 under the interface you want to configure and then select Manual from the pop-up. If you don\u0026rsquo;t know any of the following information, you should be able to get it from either your router or your computer\u0026rsquo;s network settings.\nThe subnet must be filled using CIDR notation; this should look something like 192.168.2.0/24. The address will be the IP you choose for your server, in my case, 192.168.2.2. Check your router\u0026rsquo;s DHCP range and pick an address outside it to avoid conflicts. For the gateway, this will be the address of your router, which is 192.168.2.1 in my network. Unless you have already set up your own DNS server, you can use Cloudflare 1.1.1.1 or Google 8.8.8.8 for the Name servers. You can leave the Search domains blank unless you already have a home lab or company domain you\u0026rsquo;d like to use. Select save, and after a bit, you should see the static IP address you chose under the interface you configured. Storage # We\u0026rsquo;ll install the OS to an empty disk for this article, and other more complex parts of the configuration, such as ZFS, will happen in other articles. Select Use an entire disk. Under that, select the disk you want to use. I suggest turning off Set up this disk as an LVM group to avoid extra overhead. After you\u0026rsquo;re sure you have the correct disk selected, continue and write the disk.\nUser Account and Server Name # Fill in the information for your super user (administrator) account and choose the name for your server. If you have a lot of servers, choosing a naming convention can be helpful, such as calling them after elements, planets, symbols, or stones, or you could go with server-N, where N is a number.\nSSH Setup # You\u0026rsquo;ll want to ensure you select Install OpenSSH server; this will allow you to log in and manage the server from another computer.\nFinish and Reboot # After making your last selections, you\u0026rsquo;ll have to wait for the installation to complete, then select Reboot Now. When prompted, remove the USB and press enter. After the machine reboots, you should see some cloud-init messages about SSH keys and a login prompt when it\u0026rsquo;s finished.\nConfiguring Ubuntu Server # Now that you\u0026rsquo;ve got Ubuntu Server installed, there are a few things to configure to make the server more secure and easier to manage. Some of the following tasks will depend on whether you have particular hardware or what level of security you want to achieve on your server.\nLogging into Your Server # To manage your server, you\u0026rsquo;ll need to login to it by SSH; this is simple on Linux or macOS but requires some extra software for Windows. For Windows, I recommend getting Putty; it can be downloaded from here. To login to your server via a Linux or macOS terminal, use:\n1# Replace USERNAME with the username you set on installation 2# For HOSTNAME, you\u0026#39;ll use the IP address you set when configuring the network 3ssh USERNAME@HOSTNAME 4 5# In my case, it would be 6ssh superuser@192.168.2.2 To log in using Putty, fill in the username and IP address you used when you installed Ubuntu Server 22.04, then click Open.\nAdding SSH Keys and Block Password Login # You\u0026rsquo;ll notice from the above step that you had to log in using your password; this can get annoying and needs to be more secure if you expose your SSH port to the internet. To fix this first, we\u0026rsquo;ll need to generate an SSH key; you can do this by running the following command:\n1# Fill in everything as default for now; you can do this by pressing enter at each prompt 2ssh-keygen Now that you have an SSH key, we need to tell the server to use this key as your login credentials. You can do this by using ssh-copy-id; see below:\n1# Replace USERNAME with the username you set on installation 2# For HOSTNAME, you\u0026#39;ll use the IP address you set when configuring the network 3ssh-copy-id USERNAME@HOSTNAME 4# For mine, this would be 5ssh-copy-id superuser@192.168.2.2 Now that your server has your SSH key, you can log in using the same method as before, but now you won\u0026rsquo;t be prompted for the password. The next thing you\u0026rsquo;ll want to do is limit the ability to use a password to log in to SSH. To do this, you\u0026rsquo;ll create the file /etc/ssh/sshd_config.d/secure.conf with the following contents, but be sure to change the 192.168.0.0/16 to be your subnet:\n1# No password login allowed 2PasswordAuthentication no 3# Use publish key (SSH keys) for login 4PubkeyAuthentication yes 5# No Kerberos login 6KerberosAuthentication no 7# No GSSAPI 8GSSAPIAuthentication no 9# Use PAM as the backend for accounts 10UsePAM yes 11# Don\u0026#39;t use 2fa tokens 12ChallengeResponseAuthentication no 13# Use this command to check authorization of SSH keys 14AuthorizedKeysCommand /usr/bin/sss_ssh_authorizedkeys 15# No one is authorized 16AuthorizedKeysCommandUser nobody 17# If the address is within 192.168.0.0/16 range 18Match Address 192.168.0.0/16 19 # Allow password 20 PasswordAuthentication yes If you don\u0026rsquo;t know how to create this file, you can do so by using sudo nano /etc/ssh/sshd_config.d/secure.conf, then copy and paste the contents above into the editor using Ctrl+Shift+V, then save it by pressing Ctrl+O pressing enter, then pressing Ctrl+X to exit. If you need to learn how to use the nano text editor, you can learn more here. If you prefer a different editor, then use that one instead.\nFor the changes you just made to take effect, you\u0026rsquo;ll need to restart the SSH daemon; you can do this by running:\n1sudo systemctl restart ssh Oh-My-BASH (Optional) # The default shell that ships with Ubuntu is bash; it supports some level of auto-complete and is probably the most common shell on Linux. I usually install Oh-My-BASH as an add-on to enable better auto-complete for commands and case-insensitive auto-complete of directory paths. To add it to your server, run the following commands.\n1# Clone the Oh-My-BASH to your users home directory 2git clone https://github.com/ohmybash/oh-my-bash.git ~/.oh-my-bash 3# Backup old bash configuration 4cp ~/.bashrc ~/.bashrc.orig 5# Use the bash configuration from oh-my-bash 6cp ~/.oh-my-bash/templates/bashrc.osh-template ~/.bashrc 7 8# switch the theme to use pure instead of font (this is optional) 9sed -i -e \u0026#39;s/font/pure/g\u0026#39; ~/.bashrc 10 11# make the configuration available to all users (this is optional) 12sed -i -e \u0026#39;s/OSH=.*$/OSH=\\\u0026#34;$HOME\\/.oh-my-bash\\\u0026#34;/g\u0026#39; ~/.bashrc 13sudo cp -R ~/.oh-my-bash /etc/skel/ 14sudo cp ~/.bashrc /etc/skel/ Firewall # For firewall configuration, I\u0026rsquo;ll keep this very simple. You\u0026rsquo;ll allow the SSH port and then enable the firewall. The commands to do this are below:\n1# Allow SSH 2sudo ufw allow OpenSSH 3# Turn on the firewall 4sudo ufw enable Disable Multipathd (Optional) # If you have devices like NVMe drives in your server configured with bifurcation, then you might get issues with multipath doing some strange things to them, so to solve this use the following command to turn off multipathd:\n1sudo systemctl disable --now multipathd Increase Swapfile size (Optional) # If your server doesn\u0026rsquo;t have enough memory (RAM), it may crash when running too many services; you can improve by adding more RAM. However, if you cannot add more RAM, you can add a swap file; it\u0026rsquo;ll perform much slower than RAM and should not be considered a substitute. To resize the swap file\n1# Turn all swap off 2sudo swapoff -a 3# Move the swap file somewhere else 4sudo mkdir /var/vm 5sudo mv /swap.img /var/vm/swap.img 6 7# Increase the size of the swap file 8# Don\u0026#39;t make this file bigger than your drive 9sudo fallocate -l 64G /var/vm/swap.img 10# Format the swapfile 11sudo mkswap /var/vm/swap.img 12 13# Update the location of the swapfile 14sudo sed -i \u0026#39;s#/swap.img#/var/vm/swap.img#\u0026#39; /etc/fstab 15 16# Enable swap files 17sudo swapon -a Some Useful Tools (Optional) # I\u0026rsquo;ve collected a list of helpful tools that will aid you in administrating your server; below is a list of packages and their use.\nPackage Use fish If you\u0026rsquo;re unfamiliar with the command line, then Fish can help you by showing you the possible syntax for commands. progress The command will show you the current status of any move, copy, or deletion processes. ncdu This command is helpful to check disk usage. smartmontools These tools will allow you to check the smart status of your drives to determine if you have any issues, such as a drive failure. screen Allows you to create and detach sessions; this is useful for long-running processes or if your internet connection is unstable. zfsutils-linux ZFS support for Linux, including kernel modules and tools iotop A system monitor for I/O operations The following will install all the packages:\n1sudo apt install -y ncdu smartmontools progress fish screen zfsutils-linux iotop-c 2 3# Allow using iotop as non-root user 4sudo setcap \u0026#39;cap_net_admin+eip\u0026#39; `readlink -f $(which iotop)` Install Docker # We\u0026rsquo;ll use Docker and Docker-Compose to manage most of the infrastructure in this series of articles. Docker will be the engine running containers on the server, and docker-compose will be how we define our infrastructure for Docker, including container, network, and volumes. To install Docker and Docker-Compose, run the following commands:\n1# Get the release of Ubuntu 2# Note: only LTS releases are supported 3RELEASE=$(lsb_release -cs) 4 5# Install dependencies 6sudo apt-get install -y apt-transport-https ca-certificates \\ 7 gnupg-agent software-properties-common curl 8 9# Add repository 10wget -qO - \u0026#34;https://download.docker.com/linux/ubuntu/gpg\u0026#34; |\\ 11 gpg --dearmor |\\ 12 sudo dd status=none of=/etc/apt/trusted.gpg.d/docker.gpg 13echo \u0026#34;deb [arch=amd64] https://download.docker.com/linux/ubuntu $RELEASE stable\u0026#34; |\\ 14 sudo dd status=none of=/etc/apt/sources.list.d/docker.list 15 16# Install Docker 17sudo apt update 18sudo apt install -y docker-ce docker-ce-cli containerd.io docker-compose 19 20# Add user to group 21sudo usermod -a -G docker $USER Done # Now that you\u0026rsquo;ve finished all this, you have an Ubuntu server configured for this series\u0026rsquo;s remaining articles. The last thing you\u0026rsquo;ll need before going to the next one is to install Docker-prompt, which is a tool I wrote to make things simpler and reconfigurable; install it from here.\n","date":"24 December 2023","externalUrl":null,"permalink":"/posts/2023/12/self-hosted-infrastructure-from-homelab-to-company-part-3/","section":"Posts","summary":"Installing and Configuring Ubuntu Server\n","title":"Self-Hosted Infrastructure from Homelab to Company Part 3","type":"blog"},{"content":"","date":"24 December 2023","externalUrl":null,"permalink":"/tags/ssh/","section":"Tags","summary":"","title":"SSH","type":"tags"},{"content":"","date":"24 December 2023","externalUrl":null,"permalink":"/tags/swapfile/","section":"Tags","summary":"","title":"Swapfile","type":"tags"},{"content":"","date":"24 December 2023","externalUrl":null,"permalink":"/tags/ubuntu-server/","section":"Tags","summary":"","title":"Ubuntu Server","type":"tags"},{"content":"","date":"17 December 2023","externalUrl":null,"permalink":"/tags/bash/","section":"Tags","summary":"","title":"Bash","type":"tags"},{"content":"A simple command line tool to prompt and configure docker-compose stacks\nIntroduction # I manage a lot of small server setups, like one or two servers per location, and I\u0026rsquo;ve set up these servers using Docker and Docker-compose running on Ubuntu. I generally write scripts to handle everything from creating volumes and environment variables to starting, stopping, and updating the services. For the compose files, I typically split them into sections for different types of environments or configurations (such as FreeIPA master and replica), then use the COMPOSE_FILE variable to determine what gets included for each setup.\nProblem # This approach is very modular and allows differently configured setups to use the same compose files, but it has many drawbacks. Writing .yml files for each volume or writing complex configuration scripts is time-consuming and error-prone. Additionally, if you happened to run a configuration script more than once, it would erase the current .env file, losing all your settings. For an example, see the mkenv configuration script for Gitea below.\n1#!/bin/bash 2YOUR_BUILD=\u0026#34;compose/core.yml\u0026#34; 3echo -n \u0026#34;Enter your domain: \u0026#34; 4read YOUR_DOMAIN 5 6echo -n \u0026#34;Enter your SSH port: \u0026#34; 7read YOUR_SSH_LISTEN_PORT 8 9 10echo \u0026#34;You can use NFS, (abs)olute path or (rel)ative path\u0026#34; 11echo \u0026#34;If you want to mix them create your own compose/custom.yml file\u0026#34; 12PS3=\u0026#34;Which do you want to use?: \u0026#34; 13select choice in \u0026#34;NFS\u0026#34; \u0026#34;Absolute Path\u0026#34; \u0026#34;Relative Path\u0026#34; 14do 15 case $choice in 16 \u0026#34;NFS\u0026#34;) 17 YOUR_BUILD=\u0026#34;${YOUR_BUILD}:compose/nfs.yml\u0026#34; 18 echo -n \u0026#34;Enter NFS path to your Gitea data directory: \u0026#34; 19 read IN 20 IFS=\u0026#39;:\u0026#39; read -ra ADDR \u0026lt;\u0026lt;\u0026lt; \u0026#34;$IN\u0026#34; 21 GITEA_DATA_VOL_IP=${ADDR[0]} 22 GITEA_DATA_VOL_DIR=${ADDR[1]} 23 24 echo -n \u0026#34;Enter NFS path to your Gitea database directory: \u0026#34; 25 read IN 26 IFS=\u0026#39;:\u0026#39; read -ra ADDR \u0026lt;\u0026lt;\u0026lt; \u0026#34;$IN\u0026#34; 27 GITEA_DATABASE_VOL_IP=${ADDR[0]} 28 GITEA_DATABASE_VOL_DIR=${ADDR[1]} 29 30 break;; 31 \u0026#34;Absolute Path\u0026#34;) 32 YOUR_BUILD=\u0026#34;${YOUR_BUILD}:compose/local.yml\u0026#34; 33 echo -n \u0026#34;Enter the absolute path to your Gitea data directory: \u0026#34; 34 read GITEA_DATA_VOL_DIR 35 36 echo -n \u0026#34;Enter the absolute path to your Gitea database directory: \u0026#34; 37 read GITEA_DATABASE_VOL_DIR 38 39 break;; 40 \u0026#34;Relative Path\u0026#34;) 41 YOUR_BUILD=\u0026#34;${YOUR_BUILD}:compose/local.yml\u0026#34; 42 GITEA_DATA_VOL_DIR=\u0026#34;\\${PWD}/data\u0026#34; 43 echo \u0026#34;Enter your gitea data directory relative to \\\u0026#34;${PWD}\\\u0026#34;\u0026#34; 44 echo \u0026#34;Default location is ${GITEA_DATA_VOL_DIR}\u0026#34; 45 echo -n \u0026#34;Enter your Gitea data directory: \u0026#34; 46 read IN 47 if [ ! -z \u0026#34;$IN\u0026#34; ]; then 48 GITEA_DATA_VOL_DIR=\u0026#34;\\${PWD}/$IN\u0026#34; 49 fi 50 51 GITEA_DATABASE_VOL_DIR=\u0026#34;\\${PWD}/database\u0026#34; 52 echo \u0026#34;Enter your gitea database directory relative to \\\u0026#34;${PWD}\\\u0026#34;\u0026#34; 53 echo \u0026#34;Default location is ${GITEA_DATABASE_VOL_DIR}\u0026#34; 54 echo -n \u0026#34;Enter your Gitea database directory: \u0026#34; 55 read IN 56 if [ ! -z \u0026#34;$IN\u0026#34; ]; then 57 GITEA_DATABASE_VOL_DIR=\u0026#34;\\${PWD}/$IN\u0026#34; 58 fi 59 60 break;; 61 *) 62 echo \u0026#34;Invalid response\u0026#34;;; 63 esac 64done 65 66echo -n \u0026#34;Do you want to add your own changes in \u0026#39;compose/custom.yml\u0026#39;? (y/n): \u0026#34; 67read CUSTOM 68if [ \u0026#34;$CUSTOM\u0026#34; == \u0026#34;y\u0026#34; ] || [ \u0026#34;$CUSTOM\u0026#34; == \u0026#34;Y\u0026#34; ]; then 69 YOUR_BUILD=\u0026#34;${YOUR_BUILD}:compose/custom.yml\u0026#34; 70 touch compose/custom.yml 71fi 72 73echo -n \u0026#34;Do you want to prefix the compose setup? (y/n): \u0026#34; 74read IN 75if [ \u0026#34;$IN\u0026#34; == \u0026#34;y\u0026#34; ] || [ \u0026#34;$IN\u0026#34; == \u0026#34;Y\u0026#34; ]; then 76 echo -n \u0026#34;Enter your prefix: \u0026#34; 77 read PREFIX 78 PREFIX=\u0026#34;${PREFIX}_\u0026#34; 79fi 80 81echo \u0026#34;Generating database password\u0026#34; 82YOUR_DB_PASSWORD=$(cat /dev/urandom | tr -dc \u0026#39;a-zA-Z0-9\u0026#39; | fold -w 22 | head -n 1) 83 84echo \u0026#34;Getting UID\u0026#34; 85YOUR_USER_UID=$(id -u) 86 87echo \u0026#34;Getting GID\u0026#34; 88YOUR_USER_GID=$(id -g) 89 90sed s/YOUR_DB_PASSWORD/$YOUR_DB_PASSWORD/g example.env | \\ 91sed s/YOUR_USER_UID/\u0026#34;$YOUR_USER_UID\u0026#34;/g |\\ 92sed s/YOUR_USER_GID/\u0026#34;$YOUR_USER_GID\u0026#34;/g |\\ 93sed s/YOUR_PREFIX/$PREFIX/g |\\ 94sed s,YOUR_BUILD,$YOUR_BUILD,g |\\ 95sed s/YOUR_SSH_LISTEN_PORT/$YOUR_SSH_LISTEN_PORT/g |\\ 96sed s,YOUR_GITEA_DATA_VOL_IP,$GITEA_DATA_VOL_IP,g |\\ 97sed s,YOUR_GITEA_DATABASE_VOL_IP,$GITEA_DATABASE_VOL_IP,g |\\ 98sed s,YOUR_GITEA_DATA_VOL_DIR,$GITEA_DATA_VOL_DIR,g |\\ 99sed s,YOUR_GITEA_DATABASE_VOL_DIR,$GITEA_DATABASE_VOL_DIR,g |\\ 100sed s/YOUR_DOMAIN/$YOUR_DOMAIN/g \u0026gt; .env Solution # To avoid writing long scripts for setting up docker-compose files, I developed a simple tool to read a definition file and an optional .env file and then ask for what\u0026rsquo;s missing while still allowing the user to keep existing settings intact. The goals for this are: Don\u0026rsquo;t depend on any tools being installed on the server since many scripts require many commands to be present to work Handle already existing .env files and don\u0026rsquo;t lose settings Use a simple definition language that will take little effort to write To not depend on local tools, I implemented the tool in Go; this allows a single static binary to run everywhere. For handling existing .env files, the trick was to read the file into a string map and use these settings as the defaults for each prompt.\nCustom DSL # For the definition language, since I figured it would be a simple task, I just defined a simple line-based language as seen below:\n1DATABASE_PASSWORD=@:passgen 2PROTOCOL=@:select(tcp,udp) 3UID=@:uid 4GID=@:gid 5ADMIN_NAME=@:text_input JSON Instead # This simple Domain-specific language (DSL) quickly became a bad idea. It would require more work to implement than I wanted, so I decided to go with JSON, something familiar to most developers, and then add a few functions to generate needed information, such as passwords or user IDs. The final JSON at the time of writing this article looks as seen below:\n1{ 2 \u0026#34;version\u0026#34;: 1, 3 \u0026#34;configurations\u0026#34;: { 4 \u0026#34;.env\u0026#34;: { 5 \u0026#34;file\u0026#34;: \u0026#34;.env\u0026#34;, 6 \u0026#34;variables\u0026#34;: [ 7 { 8 \u0026#34;prompt\u0026#34;: \u0026#34;Enter prefix if wanted\u0026#34;, 9 \u0026#34;function\u0026#34;: \u0026#34;text_input\u0026#34;, 10 \u0026#34;defaults\u0026#34;: [\u0026#34;\u0026#34;], 11 \u0026#34;outputs\u0026#34;: [\u0026#34;PREFIX\u0026#34;], 12 \u0026#34;options\u0026#34;: [], 13 \u0026#34;affixes\u0026#34;: [\u0026#34;\u0026#34;, \u0026#34;_\u0026#34;] 14 }, 15 { 16 \u0026#34;function\u0026#34;: \u0026#34;bool_input\u0026#34;, 17 \u0026#34;compose_files\u0026#34;: [\u0026#34;compose/core.yml\u0026#34;], 18 \u0026#34;defaults\u0026#34;: [\u0026#34;true\u0026#34;] 19 }, 20 { 21 \u0026#34;prompt\u0026#34;: \u0026#34;would you like to add custom.yml to your compose?\u0026#34;, 22 \u0026#34;function\u0026#34;: \u0026#34;bool_input\u0026#34;, 23 \u0026#34;compose_files\u0026#34;: [\u0026#34;compose/custom.yml\u0026#34;], 24 \u0026#34;outputs\u0026#34;: [\u0026#34;CUSTOM_COMPOSE_FILE\u0026#34;], 25 \u0026#34;defaults\u0026#34;: [\u0026#34;false\u0026#34;] 26 }, 27 { 28 \u0026#34;prompt\u0026#34;: \u0026#34;Enter domain name\u0026#34;, 29 \u0026#34;function\u0026#34;: \u0026#34;text_input\u0026#34;, 30 \u0026#34;defaults\u0026#34;: [\u0026#34;\u0026#34;], 31 \u0026#34;outputs\u0026#34;: [\u0026#34;DOMAIN\u0026#34;], 32 \u0026#34;options\u0026#34;: [] 33 }, 34 { 35 \u0026#34;prompt\u0026#34;: \u0026#34;Compose project name\u0026#34;, 36 \u0026#34;function\u0026#34;: \u0026#34;text_input\u0026#34;, 37 \u0026#34;defaults\u0026#34;: [\u0026#34;${PREFIX}gitea\u0026#34;], 38 \u0026#34;outputs\u0026#34;: [\u0026#34;COMPOSE_PROJECT_NAME\u0026#34;], 39 \u0026#34;options\u0026#34;: [] 40 }, 41 { 42 \u0026#34;prompt\u0026#34;: \u0026#34;Database type\u0026#34;, 43 \u0026#34;function\u0026#34;: \u0026#34;select_input\u0026#34;, 44 \u0026#34;defaults\u0026#34;: [], 45 \u0026#34;outputs\u0026#34;: [\u0026#34;DB_TYPE\u0026#34;], 46 \u0026#34;options\u0026#34;: [\u0026#34;postgres\u0026#34;] 47 }, 48 { 49 \u0026#34;prompt\u0026#34;: \u0026#34;Database name\u0026#34;, 50 \u0026#34;function\u0026#34;: \u0026#34;text_input\u0026#34;, 51 \u0026#34;defaults\u0026#34;: [\u0026#34;gitea\u0026#34;], 52 \u0026#34;outputs\u0026#34;: [\u0026#34;POSTGRES_DB\u0026#34;, \u0026#34;DB_NAME\u0026#34;], 53 \u0026#34;options\u0026#34;: [] 54 }, 55 { 56 \u0026#34;prompt\u0026#34;: \u0026#34;PostgreSQL user\u0026#34;, 57 \u0026#34;function\u0026#34;: \u0026#34;text_input\u0026#34;, 58 \u0026#34;defaults\u0026#34;: [\u0026#34;gitea\u0026#34;], 59 \u0026#34;outputs\u0026#34;: [\u0026#34;POSTGRES_USER\u0026#34;, \u0026#34;DB_USER\u0026#34;], 60 \u0026#34;options\u0026#34;: [] 61 }, 62 { 63 \u0026#34;prompt\u0026#34;: \u0026#34;PostgreSQL password\u0026#34;, 64 \u0026#34;function\u0026#34;: \u0026#34;password_input\u0026#34;, 65 \u0026#34;defaults\u0026#34;: [\u0026#34;@passgen(30)\u0026#34;], 66 \u0026#34;outputs\u0026#34;: [\u0026#34;POSTGRES_PASSWORD\u0026#34;, \u0026#34;DB_PASSWD\u0026#34;], 67 \u0026#34;options\u0026#34;: [] 68 }, 69 { 70 \u0026#34;prompt\u0026#34;: \u0026#34;Database host\u0026#34;, 71 \u0026#34;function\u0026#34;: \u0026#34;\u0026#34;, 72 \u0026#34;defaults\u0026#34;: [\u0026#34;${PREFIX}gitea_database:5432\u0026#34;], 73 \u0026#34;outputs\u0026#34;: [\u0026#34;DB_HOST\u0026#34;], 74 \u0026#34;options\u0026#34;: [] 75 }, 76 { 77 \u0026#34;prompt\u0026#34;: \u0026#34;Landing page\u0026#34;, 78 \u0026#34;function\u0026#34;: \u0026#34;select_input\u0026#34;, 79 \u0026#34;defaults\u0026#34;: [], 80 \u0026#34;outputs\u0026#34;: [\u0026#34;LANDING_PAGE\u0026#34;], 81 \u0026#34;options\u0026#34;: [\u0026#34;explore\u0026#34;] 82 }, 83 { 84 \u0026#34;prompt\u0026#34;: \u0026#34;Disable registration\u0026#34;, 85 \u0026#34;function\u0026#34;: \u0026#34;bool_input\u0026#34;, 86 \u0026#34;comments\u0026#34;: [ 87 \u0026#34;This is used to disable registration on the landing page\u0026#34;, 88 \u0026#34;If you set it to false, you will be able to register new users\u0026#34; 89 ], 90 \u0026#34;defaults\u0026#34;: [\u0026#34;true\u0026#34;], 91 \u0026#34;outputs\u0026#34;: [\u0026#34;DISABLE_REGISTRATION\u0026#34;], 92 \u0026#34;options\u0026#34;: [] 93 }, 94 { 95 \u0026#34;prompt\u0026#34;: \u0026#34;SSH port\u0026#34;, 96 \u0026#34;function\u0026#34;: \u0026#34;int_input\u0026#34;, 97 \u0026#34;defaults\u0026#34;: [\u0026#34;22222\u0026#34;], 98 \u0026#34;outputs\u0026#34;: [ 99 \u0026#34;SSH_LISTEN_PORT\u0026#34;, 100 \u0026#34;GITEA__server__SSH_PORT\u0026#34;, 101 \u0026#34;GITEA__server__SSH_LISTEN_PORT\u0026#34; 102 ], 103 \u0026#34;options\u0026#34;: [] 104 }, 105 { 106 \u0026#34;prompt\u0026#34;: \u0026#34;Gitea domain name\u0026#34;, 107 \u0026#34;function\u0026#34;: \u0026#34;text_input\u0026#34;, 108 \u0026#34;defaults\u0026#34;: [\u0026#34;gitea.${DOMAIN}\u0026#34;], 109 \u0026#34;outputs\u0026#34;: [ 110 \u0026#34;GITEA_DOMAIN\u0026#34;, 111 \u0026#34;GITEA__server__SSH_DOMAIN\u0026#34;, 112 \u0026#34;GITEA__server__DOMAIN\u0026#34; 113 ], 114 \u0026#34;options\u0026#34;: [] 115 }, 116 { 117 \u0026#34;prompt\u0026#34;: \u0026#34;Gitea root url\u0026#34;, 118 \u0026#34;function\u0026#34;: \u0026#34;text_input\u0026#34;, 119 \u0026#34;defaults\u0026#34;: [\u0026#34;https://gitea.${DOMAIN}\u0026#34;], 120 \u0026#34;outputs\u0026#34;: [\u0026#34;GITEA__server__ROOT_URL\u0026#34;], 121 \u0026#34;options\u0026#34;: [] 122 }, 123 { 124 \u0026#34;prompt\u0026#34;: \u0026#34;User uid\u0026#34;, 125 \u0026#34;function\u0026#34;: \u0026#34;id_input\u0026#34;, 126 \u0026#34;defaults\u0026#34;: [\u0026#34;@uid\u0026#34;], 127 \u0026#34;outputs\u0026#34;: [\u0026#34;USER_UID\u0026#34;], 128 \u0026#34;options\u0026#34;: [] 129 }, 130 { 131 \u0026#34;prompt\u0026#34;: \u0026#34;User gid\u0026#34;, 132 \u0026#34;function\u0026#34;: \u0026#34;id_input\u0026#34;, 133 \u0026#34;defaults\u0026#34;: [\u0026#34;@gid\u0026#34;], 134 \u0026#34;outputs\u0026#34;: [\u0026#34;USER_GID\u0026#34;], 135 \u0026#34;options\u0026#34;: [] 136 }, 137 { 138 \u0026#34;prompt\u0026#34;: \u0026#34;Data volume\u0026#34;, 139 \u0026#34;function\u0026#34;: \u0026#34;volume_input\u0026#34;, 140 \u0026#34;defaults\u0026#34;: [\u0026#34;${PWD}/data\u0026#34;, \u0026#34;\u0026#34;], 141 \u0026#34;outputs\u0026#34;: [\u0026#34;GITEA_DATA_VOL_DIR\u0026#34;, \u0026#34;GITEA_DATA_VOL_IP\u0026#34;], 142 \u0026#34;options\u0026#34;: [], 143 \u0026#34;key\u0026#34;: \u0026#34;data\u0026#34;, 144 \u0026#34;name\u0026#34;: \u0026#34;${PREFIX}gitea_data\u0026#34; 145 }, 146 { 147 \u0026#34;prompt\u0026#34;: \u0026#34;Database volume\u0026#34;, 148 \u0026#34;function\u0026#34;: \u0026#34;volume_input\u0026#34;, 149 \u0026#34;defaults\u0026#34;: [\u0026#34;${PWD}/database\u0026#34;, \u0026#34;\u0026#34;], 150 \u0026#34;outputs\u0026#34;: [\u0026#34;GITEA_DATABASE_VOL_DIR\u0026#34;, \u0026#34;GITEA_DATABASE_VOL_IP\u0026#34;], 151 \u0026#34;options\u0026#34;: [], 152 \u0026#34;key\u0026#34;: \u0026#34;database\u0026#34;, 153 \u0026#34;name\u0026#34;: \u0026#34;${PREFIX}gitea_database\u0026#34; 154 } 155 ] 156 } 157 } 158} Done # After switching to JSON and defining how the Schema (as seen above) should look, I finished writing the initial version of docker-prompt and then tested it by porting a few of my docker-compose configurations to its JSON definition language; this worked well and achieved all the goals.\nSource code # The source for docker-prompt is available for use on Gitlab under the Mozilla Public License Version 2.0. Contributions are welcome; feel free to create a pull request.\n","date":"17 December 2023","externalUrl":null,"permalink":"/posts/2023/12/docker-prompt-a-tool-for-docker-compose-configuration/","section":"Posts","summary":"A simple command line tool to prompt and configure docker-compose stacks\n","title":"Docker Prompt: a Tool for Docker Compose Configuration","type":"blog"},{"content":"","date":"17 December 2023","externalUrl":null,"permalink":"/tags/go/","section":"Tags","summary":"","title":"Go","type":"tags"},{"content":"","date":"17 December 2023","externalUrl":null,"permalink":"/tags/json/","section":"Tags","summary":"","title":"JSON","type":"tags"},{"content":"","date":"17 December 2023","externalUrl":null,"permalink":"/tags/open-source/","section":"Tags","summary":"","title":"Open Source","type":"tags"},{"content":"","date":"17 December 2023","externalUrl":null,"permalink":"/series/programming/","section":"Series","summary":"","title":"Programming","type":"series"},{"content":"","date":"17 December 2023","externalUrl":null,"permalink":"/categories/story/","section":"Categories","summary":"","title":"Story","type":"categories"},{"content":"","date":"17 December 2023","externalUrl":null,"permalink":"/tags/yml/","section":"Tags","summary":"","title":"YML","type":"tags"},{"content":"","date":"17 October 2023","externalUrl":null,"permalink":"/tags/android/","section":"Tags","summary":"","title":"Android","type":"tags"},{"content":"","date":"17 October 2023","externalUrl":null,"permalink":"/tags/asus/","section":"Tags","summary":"","title":"ASUS","type":"tags"},{"content":"","date":"17 October 2023","externalUrl":null,"permalink":"/tags/asus-zenfone-9/","section":"Tags","summary":"","title":"ASUS Zenfone 9","type":"tags"},{"content":"How to Enable Wi-Fi Calling with T-Mobile on ASUS Zenfone 9\nIntroduction # The ASUS Zenfone 9 and other versions of the ASUS Zenfone series have issues enabling and using Wi-Fi calling when using T-Mobile as their carrier.\nSteps # To enable Wi-Fi calling, do the following:\nOpen the Phone App; it works with ASUS or Google phone apps Navigate to the dialer, where you type phone numbers in your phone app Dial this number: *#*#3642623344#*# upon entering the last symbol the number will vanish Close your phone app Open settings and navigate to Network \u0026amp; Internet -\u0026gt; Call -\u0026gt; Wi-Fi calling Toggle the setting off and wait a few seconds before turning it back on You should now see a new icon appear in the status bar, which looks like a Wi-Fi symbol coming out of an old-style phone Done You\u0026rsquo;ll have to repeat the steps in this workaround each time you reboot your phone.\n","date":"17 October 2023","externalUrl":null,"permalink":"/posts/2023/10/enabling-wi-fi-calling-with-t-mobile-on-asus-zenfone-9/","section":"Posts","summary":"How to Enable Wi-Fi Calling with T-Mobile on ASUS Zenfone 9\n","title":"Enabling Wi-Fi Calling with T-Mobile on ASUS Zenfone 9","type":"blog"},{"content":"","date":"17 October 2023","externalUrl":null,"permalink":"/series/fixes/","section":"Series","summary":"","title":"Fixes","type":"series"},{"content":"","date":"17 October 2023","externalUrl":null,"permalink":"/tags/t-mobile/","section":"Tags","summary":"","title":"T-Mobile","type":"tags"},{"content":"","date":"17 October 2023","externalUrl":null,"permalink":"/tags/wi-fi-calling/","section":"Tags","summary":"","title":"Wi-Fi Calling","type":"tags"},{"content":"","date":"15 October 2023","externalUrl":null,"permalink":"/tags/best-practices/","section":"Tags","summary":"","title":"Best Practices","type":"tags"},{"content":"","date":"15 October 2023","externalUrl":null,"permalink":"/tags/convention/","section":"Tags","summary":"","title":"Convention","type":"tags"},{"content":"","date":"15 October 2023","externalUrl":null,"permalink":"/tags/directory-layout/","section":"Tags","summary":"","title":"Directory Layout","type":"tags"},{"content":"","date":"15 October 2023","externalUrl":null,"permalink":"/tags/file-hierarchy/","section":"Tags","summary":"","title":"File Hierarchy","type":"tags"},{"content":"","date":"15 October 2023","externalUrl":null,"permalink":"/tags/file-naming/","section":"Tags","summary":"","title":"File Naming","type":"tags"},{"content":"File naming and organization for personal and team use.\nIntroduction # File naming and hierarchy are essential to finding the location of files you\u0026rsquo;ve previously worked on; this is even more important when working with a team on a project. With a good file naming convention, you know what to look for and where to look because you can infer the name and location from the pattern and logical hierarchy. So, what makes a good naming and organization convention? An excellent organizational scheme should cover all of the following:\nWell-thought-out and structured directory hierarchy A clearly defined character set, which causes no issues with any tools, operating systems, or web interfaces used A general file pattern that you can apply to all files, not just officially released ones Concise and not overly complicated rules so anyone can name or locate a file using it Proper use of file extensions for file types Carries documentation to improve clarity further I collected the information in this article over several years from studying different operating systems and company organization schemes, including what is used by Fortune 100 and 500 companies and standard large universities that published their organization methods.\nDirectory hierarchy # Directory hierarchy is fundamental to an organization scheme; it lets you know where to start looking for a file by asking yourself what it is and what it pertains to. So, what does a good directory hierarchy look like? A good directory hierarchy allows you to gain information about the files stored in it by looking at the path to the file. Some examples of this:\n/Business/Marketing/Pictures: from this path, you can tell that these images are related to business marketing. /Resources/Templates/LibreOffice: this path lets you know there are general templates for use with LibreOffice stored there. /Archives/Employees/Johnny_Blaze: this path indicates that the content is archived employee files, and the last part is each employee\u0026rsquo;s name. The hierarchy can use dates to group content taken or created on a particular day or date range, commonly done with pictures. You may further organize /Business/Marketing/Pictures from above using dates like /Business/Marketing/Pictures/YYYY/MM or /Business/Marketing/Pictures/YYYY/MM/DD. Adding the day to the path would be used in cases where a lot of content gets created each month, making the day subdivision more useful. You can use directories to help convey ownership if you\u0026rsquo;re working on content for several clients. An example would be: /Projects/ClientName/ProjectName, where the projects under the client\u0026rsquo;s name belong to the client. Further, the contents of the project belong to the project. Specific things to avoid in a directory hierarchy that can make a mess or confuse users:\nTop-level directories with similar names or which could be subdirectories; examples of this are /Business and /BusinessDevelopment, both as top-level; in this case, you\u0026rsquo;d want to move /BusinessDevelopment to /Business/Development keeping content related to advancing the business vs. general operation stuff separate but still related. You may also consider relocating /Business/Marketing/Pictures to /Business/Development/Marketing/Pictures under this new layout. Storing unrelated content under a directory that does not make sense, such as keeping general templates under /Projects instead under /Templates or /Resources/Templates/AppTheTemplateIsFor would make more sense. Duplicating names of directories in subdirectories such as /Documents/DocumentPictures in this case, you\u0026rsquo;d change this to /Documents/Pictures or relocate it to /Pictures. Using the words files, directory, or folder in conjunction with the content of the directory, examples of this would be /PictureFiles or /EmployeeFolders or ImportantDirectories in all these cases, the directory names would be better and shorter without the unneeded words added to them. File naming # Several things go into making a suitable filename, the most crucial being consistency in the names and where the files are stored, allowing users to find them easily. The following section will cover the different parts of file naming and why it should be that way.\nBe concise and avoid long names # Filenames and directory names should be concise and avoid being long or drawn out, making things more challenging to find; in addition, words like a, of, to, and the should be avoided. For example, the filename: The business meeting about making the best use of open source in production 2023 January 5th.odp is terrible. Renaming it and relocating it to /Business/Meetings/2023/01/OpenSourceForProduction.odp or if the file is not part of a collection Meeting-20230105-OpenSourceForProduction.odp both of these cases are much improved over the first.\nAvoid special characters # It\u0026rsquo;s best to avoid special characters in filenames because operating systems may interpret them as having a defined function, which can have strange adverse effects or break automation; this generally includes: `, !, @, #, $, %, ^, \u0026amp;, *, (, ), =, +, [, ], {, }, \\, |, ;, :, ', \u0026quot;, \u0026lt;, \u0026gt;, /, and ?. Windows generally will not allow the use of most of the above characters as it\u0026rsquo;s the most limited modern OS available.\nAvoid using blank spaces # Why avoid space in the names of directories or files? All major file systems support the space character ext4, xfs, btrs, hfs+, ntfs, apfs, and fat32. In addition, all modern operating systems have support for Unix (1978), macOS (1985), Linux (1992), and Windows (1995). Well, using blank spaces in filenames should be avoided for several reasons:\nSome applications don\u0026rsquo;t recognize white space properly and may treat files differently than expected; you can observe this behavior on Windows, Linux, and macOS. Some search tools don\u0026rsquo;t operate correctly on files containing spaces. Windows, Linux, and macOS command line applications require extra work to handle spaces properly. The space character is unsafe because significant spaces may disappear, and insignificant white space may get introduced when URLs are transcribed, typeset, or subjected to the treatment of word-processing programs RFC1738. Universal paths cannot contain spaces, which results in no copyable/pasteable links unless done only via the web browser, as the Windows file manager and many Linux file managers don\u0026rsquo;t correctly handle URL encoding. Spaces are less visible than other characters, resulting in common double whitespaces or blank spaces in strange places. Define general separators # A suitable file naming pattern requires two types of separators. The first is the general word separator used between words in a filename. The _ underscore is the defacto standard for word separators in most local or web-based naming conventions and is the most visually similar to a space character. In some naming schemes, the word separator does not exist; instead, filenames use title case. The second separator divides filename elements, such as document number, date, version tags, etc. The most common separator for this purpose is the hyphen/dash -.\nCase and Title case # Companies or users may prefer the look of title case over underscores in filenames because title case does not offer any adverse effects as white space does; Some use it instead of whitespace or underscores. So instead of a filename like Doc-New_Element_for_Fusion.pdf, it would be Doc-NewElementForFusion.pdf. Even if you choose not to use this option, writing each word\u0026rsquo;s first letter capitalized is still preferred vs. using fully lowercase or entirely uppercase for file names, with the latter reserved for abbreviations.\nFile extensions # Filenames for all types should include a file extension except executables on Linux and macOS, as it\u0026rsquo;s common for those types not to have a file extension.\nVersion/Revision numbers # In systems that don\u0026rsquo;t have revision control built-in, having the version of a file may be essential to distinguish a file from an older version that might have breaking changes or changes in decision. Typical schemes for version numbers include using vN, for example: Policy-v2.odt, or using a simple letter A-Z, usually capitalized, for example: Policy-A.ods. Another use of version numbers is to indicate a run of a long-running test, for example: Pressure_Test-T20.csv for the 20th run of the test.\nUsing dates # The date is often included in a filename when the file is time-sensitive or pertains to a certain period, such as taxes or event itineraries. If a file is part of a collection like pictures, it\u0026rsquo;s generally better to use the hierarchy YYYY/MM/DD and store the photos for that day in that date\u0026rsquo;s directory. In the case where the date may be in the filename, general practice is using either YYYY-MM-DD or YYYYMMDD only using as much of the date as needed and attaching it to the filename like so: Business_Taxes-2023.zip, Vacation-202305/, System_diagram-20230501.drawio. Another method used to represent dates is in quarters of a year, which often have a year followed by a quarter YYYY-qN, such as Report-2023-q2.pdf.\nUsing unique identifiers # Some companies may attach unique IDs to documents, allowing quick look-up in systems that provide search, such as Nextcloud. The unique IDs are often between 4 and 16 numeric or alphanumeric characters. The pattern or method to create these unique numbers ranges from random generators to ones based on information such as date, client, and content type. These IDs are also more commonly used for official files than individual notes or experiments.\nUse of abbreviation # Using abbreviations can significantly shorten the filename length and make files more manageable; however, overuse can make files hard to understand and locate. Use commonly known abbreviations or abbreviations within known fields of study of users. Only use obscure abbreviations if they are part of the general knowledge needed for a project.\nUsing personal names # When used in filenames, the general way is to have the last name first, the whole first name or first initial, and optionally the middle initial. An examples of this: Email_Export-Blaze_Johnny-2023-03-01.zip or Gmail_Takeout-StarkT-20231010.tar.gz\nPattern and use of elements # A file naming pattern has several required and optional elements connected by a chosen separator in a particular order. Unlike the rest of the content in this article, file naming patterns seem more diverse. Since there is no typical pattern, here is a generic one from the elements used in this article: [client-][project-][collection_type-]content_title[-person][-v(version) or date].extension. Here are some examples of files using this pattern:\nApple-iPhone16-Design-Main_CPU_Architecture-v1.pdf Datasheet-RP2040-2023.pdf Cost_Analysis-q1.ods ssh_key-Blaze_Johnny.pub G60_Sword-Review-20150608.md As you can see by these example filenames, it is easy to locate any file by simply asking yourself what it is.\nGuide # After defining a naming and organization system, it\u0026rsquo;s a good idea to document it as a guide or a cheat sheet. The whole naming and organization scheme should be compact and simple enough to fit on a single note card, making it easy for anyone to memorize quickly and use. Additionally, it should follow logical organization, so the location of files should be guessable even without a guide.\n","date":"15 October 2023","externalUrl":null,"permalink":"/posts/2023/10/file-naming-and-organization-best-practices/","section":"Posts","summary":"File naming and organization for personal and team use.\n","title":"File Naming and Organization Best Practices","type":"blog"},{"content":"","date":"15 October 2023","externalUrl":null,"permalink":"/tags/organization/","section":"Tags","summary":"","title":"Organization","type":"tags"},{"content":"","date":"15 October 2023","externalUrl":null,"permalink":"/series/storage/","section":"Series","summary":"","title":"Storage","type":"series"},{"content":"","date":"26 September 2023","externalUrl":null,"permalink":"/tags/arch-linux/","section":"Tags","summary":"","title":"Arch Linux","type":"tags"},{"content":"","date":"26 September 2023","externalUrl":null,"permalink":"/tags/debian/","section":"Tags","summary":"","title":"Debian","type":"tags"},{"content":"","date":"26 September 2023","externalUrl":null,"permalink":"/tags/fedora/","section":"Tags","summary":"","title":"Fedora","type":"tags"},{"content":"","date":"26 September 2023","externalUrl":null,"permalink":"/tags/freebsd/","section":"Tags","summary":"","title":"FreeBSD","type":"tags"},{"content":"","date":"26 September 2023","externalUrl":null,"permalink":"/tags/nixos/","section":"Tags","summary":"","title":"NixOS","type":"tags"},{"content":"","date":"26 September 2023","externalUrl":null,"permalink":"/tags/opensuse/","section":"Tags","summary":"","title":"OpenSUSE","type":"tags"},{"content":"","date":"26 September 2023","externalUrl":null,"permalink":"/tags/proxmox-ve/","section":"Tags","summary":"","title":"Proxmox VE","type":"tags"},{"content":"Choosing an operating system\nWritten in 2023 for Ubuntu Server 22.04 with Docker Compose, Caddy, and FreeIPA. The stack I run has since moved to Fedora with Podman quadlets and Traefik; the series is kept as written. Introduction # When deciding which operating system will run on your server, there are a few things to consider.\nWhat services do you intend to run, and what are their system requirements? What file system do you want to use to store your data, such as ZFS, btrfs, or something else? Familiarity and learning curve of the operating system will affect how quickly you\u0026rsquo;ll be able to get services up and running and the security of those services. Operating Systems # There are many choices for operating systems when it comes to servers. Some are for particular purposes, and others are more general. Some are stable, while others are more bleeding edge. What you choose will come down to your use case.\nLinux # The most common server operating systems are Linux-based, using either the Debian or RedHat package formats. Some Linux-based operating systems, such as NixOS, work differently and don\u0026rsquo;t follow the typical approach.\nUbuntu # Ubuntu is one of the most used Linux-based operating systems for cloud-based infrastructure. It also has one of the largest communities behind it and enterprise support from Canonical. Advantages of using Ubuntu server include:\nBuilt-in ZFS straight from the live installer. Community support, so it\u0026rsquo;s easy to find help or documentation online. Enterprise support offers 24/7 assistance, troubleshooting, and real-time security updates. Debian-based, so it\u0026rsquo;s easy to find software. Zero-cost by default, with support provided at a fee. Proxmox VE # Proxmox is a special-purpose Debian Linux-based operating system for running KVM virtual machines and LXC container workloads. Proxmox is free; however, you\u0026rsquo;ll require a subscription to get support or access to the stable enterprise repository.\nDebian # Debian is a community-driven Linux distribution available in stable and unstable branches; what Debian refers to as stable is about 1-3 years behind what Ubuntu considers stable, with Debian-unstable being more current. Debian is solely community-driven and does not offer enterprise support.\nFedora # Fedora server is a community-developed Linux distribution funded by Red Hat, Inc. It offers a Cockpit, a complete and easy-to-use web user interface, with support for KVM virtual machines and containers via Podman, an alternative to Docker. It follows a fixed-release cycle and offers more bleeding-edge software than Ubuntu but does not provide enterprise support or an LTS version.\nArch Linux # Arch Linux is a lightweight, minimalistic, rolling-release Linux distribution that focuses on keeping it simple; however, this does not mean it\u0026rsquo;s simple to use, as the user has to configure all packages since they only add what is purposely required.\nOpenSUSE # OpenSUSE is an open-source Linux distribution offered in two main variants: Leap, a stable fixed-release distro, and Tumbleweed, a more bleeding-edge rolling-release distro. OpenSUSE is backed by SUSE LLC, with enterprise support available through them. YaST stands for Yet Another Settings Tool; a significant feature only available on OpenSUSE, it offers an easy point-and-click GUI for most of your standard server configurations.\nNixOS # NixOS is a Linux distribution designed for reproducible builds with a declarative system-wide Nix configuration file; this allows all software installed and configurations to be defined. You can apply this configuration to any new installation, making it identical to the other systems with the same definition file. NixOS allows atomic updates with the ability to roll back if something goes wrong.\nFreeBSD # FreeBSD is a free and Unix-like open-source operating system with support for ZFS. Unlike Linux distributions, which are collections of software made by many different people, FreeBSD is developed by a single team, making it more coherent. FreeBSD is a rolling release system, always getting the newest software, unlike most Linux distributions, which use a fixed release cycle.\nTrueNAS # TrueNAS is a network-attached storage operating system with variants TrueNAS CORE (based on FreeBSD) and TrueNAS Scale (based Debian Linux). Both have built-in support for ZFS. The Linux-based version supports KVM virtual machines, Kubernetes, and Docker (as an add-on). The FreeBSD version supports Virtual machines with bhyve and can use jails to provide similar functionality to containers. TrueNAS includes free versions, enterprise versions, and prebuilt hardware.\nWindows # Windows is a proprietary, close source (technically, a shared source system, just not with you), heavily restricted operating system. You\u0026rsquo;ll require a license to run a Windows server, as there is no free version of this OS. In most cases, you\u0026rsquo;ll be better off with any other option on this list unless you require particular Microsoft services that cannot run on Linux or FreeBSD-based systems.\nThis series # I\u0026rsquo;ve set up and managed systems using Fedora, Ubuntu, Proxmox, and TrueNAS CORE and experimented with Debian, Windows, and OpenSUSE. We will use Ubuntu in this series because it is an excellent middle ground with community and enterprise support options.\n","date":"26 September 2023","externalUrl":null,"permalink":"/posts/2023/09/self-hosted-infrastructure-from-homelab-to-company-part-2/","section":"Posts","summary":"Choosing an operating system\n","title":"Self-Hosted Infrastructure from Homelab to Company Part 2","type":"blog"},{"content":"","date":"26 September 2023","externalUrl":null,"permalink":"/tags/server/","section":"Tags","summary":"","title":"Server","type":"tags"},{"content":"","date":"26 September 2023","externalUrl":null,"permalink":"/tags/truenas/","section":"Tags","summary":"","title":"TrueNAS","type":"tags"},{"content":"","date":"26 September 2023","externalUrl":null,"permalink":"/tags/truenas-core/","section":"Tags","summary":"","title":"TrueNAS CORE","type":"tags"},{"content":"","date":"26 September 2023","externalUrl":null,"permalink":"/tags/truenas-scale/","section":"Tags","summary":"","title":"TrueNAS Scale","type":"tags"},{"content":"","date":"17 September 2023","externalUrl":null,"permalink":"/tags/hardware/","section":"Tags","summary":"","title":"Hardware","type":"tags"},{"content":"Choosing hardware for a server\nWritten in 2023 for Ubuntu Server 22.04 with Docker Compose, Caddy, and FreeIPA. The stack I run has since moved to Fedora with Podman quadlets and Traefik; the series is kept as written. Introduction # This series will begin by identifying the appropriate hardware components for the server and assembling them. Then, set up the operating system and install and configure the desired services. Additionally, it will cover aspects related to security, data integrity, and data backup.\nTo choose the proper components, the first thing to consider is whether you\u0026rsquo;ll be using the server in a homelab setting or an enterprise one. The second is to determine the server\u0026rsquo;s primary purpose. Typical Uses for servers include virtualization, data storage (NAS), multimedia/streaming, email servers, web servers, game servers, etc. The third important thing to consider is how many users the server will serve; the more users, the more resources the server will require.\nHomelab # In a homelab setting, you\u0026rsquo;ll generally use consumer-grade hardware, such as desktop components or older server-grade parts from a few generations back. In addition, most homelabs won\u0026rsquo;t use ECC memory, a feature not found on most consumer-grade CPUs and Motherboards.\nEnterprise / Business # Enterprise and business servers are usually purchased preconfigured rather than building up the hardware from components. In cases where custom server configurations are needed, they\u0026rsquo;ll generally use workstation or server-grade motherboards and CPUs with support for ECC memory.\nProcessor # When choosing a processor, a few things to consider are the number of cores and threads, cache size, the clock speed of each CPU core, and virtualization support.\nCores and threads # The cores are the individual physical processing units within a CPU. The more compute units, the more simultaneous tasks the CPU can perform. Threads are sometimes called virtual CPUs in chips with hyperthreading support, such as the Intel Core i9-9900K. Threading, like the CPU cores, allows for better handling of more parallel tasks.\nClock speed # The clock speed of a CPU indicates how fast a single core can perform a task. Typically, there is base clock speed, which is how fast the CPU will run by default, and boost clock speed, which is the maximum speed the CPU can run.\nCache size # The cache is fast-access memory stored in the processor; typically, there are L1, L2, and L3 caches. As the cache number increases, so does the cache size; however, the memory speed decreases.\nVirtualization support # Some processors support virtualization extensions, allowing multiple operating systems to run simultaneously. Most modern CPUs, both consumer-grade and server-grade, offer virtualization extensions. Intel has VT-x, and AMD has AMD-vi. They both have extensions allowing PCIe passthrough VT-d for Intel and IOMMU for AMD. You\u0026rsquo;ll need both CPU extensions to run virtual machines with fully accelerated graphics.\nMemory # It is generally better to get error-correcting code (ECC) memory if possible, even if the data hosted on the server is not critical; however, many Homelab setups don\u0026rsquo;t use it due to the higher cost of components to support it. ZFS and other fault-tolerant technologies can help avoid and correct potential data corruption without ECC memory; however, it\u0026rsquo;s still better to have it for critical systems.\nThe memory needed will heavily depend on what services will be running, how many users will use the system, the storage configuration, and whether or not the server will employ virtualization. I recommend at least 64GiB for general use and 128GiB to 256GiB for ZFS and virtualization.\nStorage # When choosing storage, two things to consider are how much is needed and how fast it needs to be. Standard SATA or SAS hard drives generally offer high storage capacity at a lower cost but are also much slower than SATA or NVMe SSDs. You can combine multiple drives of different types using ZFS or software RAID to offer better reliability, redundancy, and speed. As seen in the figure below, one such configuration uses two mirrors, each with two drives, a read cache (l2arc), and a write cache (slog). This configuration offers some speed up on cached reads and reliability in case of power failure because of the write cache.\nflowchart TD zpool l2arc slog mirror0 mirror1 sda[(18TiB HDD)] sdb[(18TiB HDD)] sdc[(18TiB HDD)] sdd[(18TiB HDD)] nvme0[128GiB NVMe] nvme1[32GiB NVMe] zpool --\u003e mirror0 zpool --\u003e mirror1 zpool --\u003e slog zpool --\u003e l2arc mirror0 --\u003e sda mirror0 --\u003e sdb mirror1 --\u003e sdc mirror1 --\u003e sdd slog --\u003e nvme1 l2arc --\u003e nvme0 A few helpful things about storage configurations\nFor better performance under ZFS, 1GiB of memory per 1TB of usable disk space is good. For Hard drives larger than 2TiB, RAID1 (mirrors) or RAID10 is ideal due to rebuild time. For SSDs smaller than 2TiB, RAID5 or RAID6 are better due to the space available. SSDs in a mirror configuration will wear at about the same rate and likely fail around the same time. Graphics # Most servers are administrated via the command line and don\u0026rsquo;t require a graphics card to host a desktop environment. Typical uses for a graphics card in a server include\nTranscoding for media servers Supplying accelerated graphics to docker containers for 3D applications Running virtual machines with accelerated graphics For general use cases, a single consumer-grade or workstation-grade GPU will work fine; however, in cases where the number of virtual machines needing GPUs exceeds the number of PCIe slots on the server. Then, you\u0026rsquo;ll need vGPU support, which allows a single GPU to be partitioned into many virtual GPUs and individually assigned to virtual machines or docker containers. Case # Choosing a case will depend on two main things: first, how big it needs to be to fit all the components correctly, and second, where you intend to store it in a typical home without a rack, you\u0026rsquo;ll likely use a tower or desktop-style case; however, if you can rack mount your server, it\u0026rsquo;s generally the preferred choice.\nMy builds # I\u0026rsquo;ve worked on several servers which have been used both in companies as well as home settings. Most of the servers I\u0026rsquo;ve built are more general use as they can cover most of the purposes mentioned in the introduction. Below, I\u0026rsquo;ve listed the components used for a few.\nThis post contains affiliate links, which means I may earn a commission if you purchase through these links; this does not increase the amount you pay for the items.\nAMD Epyc server # Date Built 2023-07-10 Setting Company Component Selection CPU AMD EPYC 7443P CPU Cooler NH-U9 TR4-SP3 Memory 8 x ASRock Rack ROMED8-2T AMD EPYC 64GB RDIMM Motherboard ROMED8-T2 OS Drive WD Black 1TB SN850X Storage 5 x WD Ultrastar DC HC550 18TB 7200RPM 3.5\u0026quot; Storage (L2Arc) 2 x TEAMGROUP MP33 1TB SLC Storage (SLOG) 2 x TEAMGROUP MP33 256GB SLC Storage (Expansion) NVME expansion card Storage (Drive Cage) 2 x Rosewill 3 x 5.25-Inch to 4 x 3.5-Inch Hot-swap Graphics NVIDIA RTX A5000 Case iStarUSA D-400-6 Case Fans (Rear) 2 x Noctua NF-A8 PWM Power Supply Corsair RM1000e AMD Ryzen server # Date Built 2023-08-01 Setting Homelab Component Selection CPU AMD Ryzen 7 5700G CPU Cooler Included with CPU Memory 4 x G.SKILL Ripjaws V Series 32GB UDIMM Motherboard MSI PRO B550M-VC WiFi OS Drive WD Black 1TB SN850X Storage 4 x WD Red Plus 8TB Storage (Drive Cage) Rosewill 3 x 5.25-Inch to 4 x 3.5-Inch Hot-swap Graphics NVIDIA Tesla P40 Graphics Fan Custom 3D printed fan for Tesla P40 Case RackChoice 3u Rackmount Case Fans (Front) 2 x Noctua NF-A6x25 PWM Case Fans (Rear) 2 x Noctua NF-A8 PWM Power Supply EVGA 850 GT Intel Core server # Date Built 2020-09-01 Setting Homelab Component Selection CPU Intel Core i9-9900K CPU Cooler NZXT Kraken X73 Memory 4 x G.Skill Ripjaws V 32 GB Motherboard Gigabyte C246-WU4 OS Drive WD Black 1TB SN770 NVMe Storage 5 x WD Ultrastar DC HC550 18TB 7200RPM 3.5\u0026quot; Storage (Drive Cage) 3 x Rosewill 3 x 5.25-Inch to 4 x 3.5-Inch Hot-swap Graphics EVGA GeForce RTX 3050 XC Gaming Case anidees AI Crystal XL Power Supply PC Power \u0026amp; Cooling FPS1050-A5M00 Intel Xeon server # Date Built 2020-10-20 Setting Company Component Selection CPU Intel Xeon E-2146G CPU Cooler Cooler Master Hyper 212 EVO Memory 2 x Mushkin Proline 32 GB Motherboard Gigabyte C246-WU4 OS Drive WD Black 1TB SN770 NVMe Storage 4 x Western Digital Red 8 TB 3.5\u0026quot; Storage (Drive Cage) Rosewill 3 x 5.25-Inch to 4 x 3.5-Inch Hot-swap Graphics GeForce RTX 3080 VENTUS 3X 10G OC Case Corsair Carbide Series 200R Power Supply ","date":"17 September 2023","externalUrl":null,"permalink":"/posts/2023/09/self-hosted-infrastructure-from-homelab-to-company-part-1/","section":"Posts","summary":"Choosing hardware for a server\n","title":"Self-Hosted Infrastructure from Homelab to Company Part 1","type":"blog"},{"content":"","date":"10 September 2023","externalUrl":null,"permalink":"/tags/business/","section":"Tags","summary":"","title":"Business","type":"tags"},{"content":"An overview of the infrastructure in this series\nWritten in 2023 for Ubuntu Server 22.04 with Docker Compose, Caddy, and FreeIPA. The stack I run has since moved to Fedora with Podman quadlets and Traefik; the series is kept as written. Introduction # This article gives an overview of the architecture of the home lab setup used in this series. It\u0026rsquo;ll cover building everything up from the hardware to the operating system and then installing the foundation services that tie everything together; it\u0026rsquo;ll finish with several tutorials about installing individual services. Because the services tutorials are optional, you can choose what services you want in your lab. Additionally, I\u0026rsquo;ll cover information about alternative services, operating systems, and options that differ from the ones used.\nHardware # For the hardware setup, we\u0026rsquo;ll use standard consumer-grade components in a rack-mounted case with enough resources to run most services. The build will be as budget-friendly as possible and use commonly available parts.\nOperating system # The operating system will be the current LTS of Ubuntu since it has good community and enterprise support, allowing for general use in the home and enterprise settings.\nSoftware # The core software that provides the foundation for all the services will include Docker, Docker-compose, Caddy v2, Cloudflared, Pi-hole, Authentik, and FreeIPA. Docker with Docker-compose will give a simple infrastructure-as-code setup reproducible on other servers and allow for simple .env files to store configurations. Caddy V2 will provide the function of a reverse proxy, with optional Cloudflared making services accessible from outside the local network. FreeIPA and Authentik will provide login and Single Sign On (SSO). Finally, Pi-hole will provide ad blocking and DNS so you can access services with service_name.domain.com instead of an IP address with a port number.\nOutcome # After the hardware, the operating system, and the foundational software are in place, you can choose all other services depending on what you want to make available in your lab. This setup should be reliable, easy to maintain, and easy to find help online if something goes wrong.\n","date":"10 September 2023","externalUrl":null,"permalink":"/posts/2023/09/self-hosted-infrastructure-from-homelab-to-company-part-0/","section":"Posts","summary":"An overview of the infrastructure in this series\n","title":"Self-Hosted Infrastructure from Homelab to Company Part 0","type":"blog"},{"content":"","date":"10 September 2023","externalUrl":null,"permalink":"/tags/software/","section":"Tags","summary":"","title":"Software","type":"tags"},{"content":"","externalUrl":null,"permalink":"/blog/","section":"Posts","summary":"","title":"Posts","type":"blog"},{"content":"Guides and notes on servers, virtualization, and Linux, written from the machines I run. Longer builds are written as a series.\n","externalUrl":null,"permalink":"/","section":"Tek.Place","summary":"","title":"Tek.Place","type":"page"},{"content":"Taylor, known online as zeitue. I\u0026rsquo;m a software and electrical engineer at PCDworks, where I also handle IT, and the lead engineer at Defenestra, the company that spun off from it to publish Hyperpane. Zalatek is my own; I founded it to build the things that don\u0026rsquo;t fit under either of those, which currently range from a Go web toolkit to an embedded terminal to a WiFi mouse. I have two bachelor\u0026rsquo;s degrees from the University of Texas at Tyler, one in software engineering and one in electrical engineering, and I speak English and Japanese with bits of other languages.\nI started by teaching myself C in thirty days, then writing low-level operating system code, because I was using Windows at the time and it was a piece of trash, so I figured I\u0026rsquo;d try to build a better one; that was the actual beginning, and most of everything since has been picking up new things as I go. I know C, C++, Rust, Go, Java, Python, Ruby, Perl, Lua, Common Lisp, Clojure, TypeScript, JavaScript, bash, Nushell, PowerShell, and assembly for x86, x86-64, ARM64, RISC-V, and PowerPC, etc. I have experience with technologies like Zephyr, Ansible, Podman, and systemd, microcontrollers like the Nordic nRF52840 and nRF9160, Silicon Labs EFM32HG322F64, and TI CC1352P, and single-board computers like the Raspberry Pi Zero W and Raspberry Pi 4. I have built servers for companies and for my own homelab. The knowledge related to paravirtualization, mentioned in Pathway to Paravirtualization, is my own, and it is being integrated into Defenestra\u0026rsquo;s Hyperpane.\n","externalUrl":null,"permalink":"/authors/zeitue/","section":"Authors","summary":"","title":"zeitue","type":"authors"}]