Linux
Linux
u-boot kernel initramfs rootfs
https://developer.ibm.com/articles/l-linuxboot/
http://duartes.org/gustavo/blog/post/kernel-boot-process/
Kernel
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/?h=master
LTS longterm are maintaining fixes, but not features.
Distributions often pick a kernel version and maintain them with backported patches, to ensure everything that depends does not need to change.
Linux follows semantic versioning (x.y.z) semver.org to some extent, but the X jump from 2 to 3 was a real big change. The change between 6 and 7 is because the Y number grows to big. The real backwards compatibility changes are in the ABI (sysfs and symbols).
https://docs.kernel.org/admin-guide/abi.html.
Linux is not strictly following POSIX, although it is largely POSIX compatible. Windows has a minimal POSIX compatibility layer. POSIX is the formal description of how UNIX should interface.
make clean make defconfig make menuconfig make make modules
Busybox Uname
Where does uname -m find the machine type? uname finds it from a struct that is created during compile time. The struct init_uts_ns is created as __weak and can be modified later in the compile process. mkcompile_h is the last script that fills the struct. It takes the ARCH from the kernel config and copies it into UTS_MACHINE.
https://www.kernel.org/doc/html/v6.6/admin-guide/kdump/vmcoreinfo.html
init_uts_ns
https://github.com/mirror/busybox/blob/master/coreutils/uname.c .machine = UTS_MACHINE struct utsname -> Linux kernel -> init_uts_ns.name.machine -> UTS_MACHINE
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/init/version.c#n49
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/init/version-timestamp.c
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/linux/utsname.h
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/Makefile.vmlinux
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/link-vmlinux.sh
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/init/Makefile
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Makefile#n409
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Makefile#n650
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Makefile#n1470
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/mkcompile_h
grep -R UTS_MACHINE linux/arch/arm #define UTS_MACHINE "armv7l"
Influxdb / Telegraf
inputs.file files = ["/sys/class/thermal/thermal_zone0/temp"] name_override = "cpu_temperature" data_format = "value" data_type = "integer" inputs.exec commands = [ "/usr/bin/vcgencmd measure_temp" ] name_override = "gpu_temperature" data_format = "grok" grok_patterns = ["%{NUMBER:value:float}"]
eBPF
falco, tetragon, parcq, pixie, hubble, cilium prometheus, fluentd, elk, grafana (telegraf, influxdb, chrongraf), opentelemetry
BPF Brendan Gregg
https://www.youtube.com/watch?v=16slh29iN1g
https://devopsspiral.com/articles/linux/ebpf-unlock/
https://github.com/iovisor/bpftrace/blob/master/docs/tutorial_one_liners.md
https://github.com/zoidbergwill/awesome-ebpf
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/net/ethernet/eth.c?h=v6.1.44
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/net/ipv4/arp.c?h=v6.1.44
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/include/net/arp.h?h=v6.1.44
cat /proc/net/arp
bpftrace -l
bpftrace -e 'kprobe:*arp* { @[func] = count(); }'
bpftrace -e 'kretprobe:arp_hash { printf("arp_hash %x\n", retval); }'
bpftrace -e 't:syscalls:sys_enter_execve { printf("pid: %d -- comm: %s",pid,comm); }'
bpftrace -e 'k:__iwl_dbg /str(arg4) == "Rssi %d, TSF %llu\n"/ { printf("strength: %d\n", arg5); |'
bpftrace -e 'kprobe:__neigh_update { printf("%08x\n", arg1); }'
struct sk_buff *arp_create(int type, int ptype, __be32 dest_ip,
struct net_device *dev, __be32 src_ip,
const unsigned char *dest_hw,
const unsigned char *src_hw,
const unsigned char *target_hw);
bpftrace -e '
tracepoint:neigh:neigh_update
/args->family == 2 && !(args->nud_state & 0xde) && (args->new_state & 0xde)/
{
printf("%-20s %02x:%02x:%02x:%02x:%02x:%02x dev=%-10s 0x%02x->0x%02x\n",
ntop(args->primary_key4),
args->new_lladdr[0], args->new_lladdr[1], args->new_lladdr[2],
args->new_lladdr[3], args->new_lladdr[4], args->new_lladdr[5],
str(args->dev),
args->nud_state, args->new_state);
}'
objdump -tT /bin/bash
NetFilter
The firewall in the linux kernel is netfilter. There are modules that provide additional features The userland program to add, show and remove rules is iptables.
iptables
iptables -L -v -n --line-numbers iptables -P INPUT DROP iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A PREROUTING -s 185.123.21.123/32 -p tcp -j DNAT --to-destination 10.0.8.2 iptables -t nat -A POSTROUTING -s 10.0.0.0/8 -o wan -j MASQUERADE
ip6tables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT ip6tables -A INPUT -i lo+ -j ACCEPT ip6tables -A INPUT -p ipv6-icmp -j ACCEPT ip6tables -A INPUT -d fe80::/64 -p udp -m udp --dport 546 -j ACCEPT ip6tables -A INPUT -i lan -p tcp -m tcp --dport 22 -j ACCEPT
nftables
nft can deal with ipv4 and ipv6
iptab
iPXE
iPXE is the official replacement for gPXE Any feature present in gPXE is also present in iPXE, and users can seamlessly upgrade from gPXE to iPXE. (fork due to domain ownership conflict)
iPXE suppports boot over http and iscsi, alternative for tftp and nfs.
http://forum.ipxe.org/showthread.php?tid=6077
https://github.com/ipxe/ipxe/discussions
cd /usr/src/ipxe/src/ make bin/ipxe.iso apk add xorriso syslinux make -j 4 ISOLINUX_BIN=/usr/share/syslinux/isolinux.bin bin/ipxe.iso
apk add tftp-hpa service in.tftpd restart
echo "#define DOWNLOAD_PROTO_NFS" >> config/local/general.h make cp bin/undionly.kpxe /var/tftpboot/
make bin-x86_64-efi/ipxe.efi cp bin-x86_64-efi/ipxe.efi /var/tftpboot/
/jffs/configs/dnsmasq.conf.add
dhcp-match=set:ipxe,175 dhcp-boot=tag:!ipxe,undionly.kpxe,acer,10.0.0.15 dhcp-boot=http://10.0.0.15/ipxe/bootstrap.ipxe
https://askubuntu.com/questions/1029017/pxe-boot-of-18-04-iso
Raspberry Netboot
https://www.howtoraspberry.com/2022/03/how-to-pxe-boot-a-raspberry/
diskless ubuntu
ipxe/bootstrap.ipxe
#!ipxe :ubuntu sanhook --no-describe iscsi:10.0.0.15:::0:iqn.2004-04.com.qnap:ts-453a:iscsi.ubuntu.f8dd7f || goto fail sanboot --no-describe --filename \EFI\debian\grubaa64.efi || goto fail #sanboot http://10.0.0.15/ipxe/mini.iso
coreos (now part of fedora)
ipxe/bootstrap.ipxe
#!ipxe # https://coreos.com/os/docs/latest/booting-with-ipxe.html set base-url http://stable.release.core-os.net/amd64-usr/current kernel ${base-url}/coreos_production_pxe.vmlinuz initrd=coreos_production_pxe_image.cpio.gz coreos.first_boot=1 coreos.config.url=http://10.0.0.15/ipxe/pxe-config.ign root=/dev/sda initrd ${base-url}/coreos_production_pxe_image.cpio.gz boot
https://docs.fedoraproject.org/en-US/fedora-coreos/live-booting-ipxe/
#!ipxe set STREAM stable set VERSION 33.20210328.3.0 set CONFIGURL https://example.com/config.ign set BASEURL https://builds.coreos.fedoraproject.org/prod/streams/${STREAM}/builds/${VERSION}/x86_64 kernel ${BASEURL}/fedora-coreos-${VERSION}-live-kernel-x86_64 initrd=main coreos.live.rootfs_url=${BASEURL}/fedora-coreos-${VERSION}-live-rootfs.x86_64.img ignition.firstboot ignition.platform.id=metal ignition.config.url=${CONFIGURL} systemd.unified_cgroup_hierarchy=0 initrd --name main ${BASEURL}/fedora-coreos-${VERSION}-live-initramfs.x86_64.img boot
https://builds.coreos.fedoraproject.org/streams/stable.json
https://docs.fedoraproject.org/en-US/fedora-coreos/live-booting-ipxe/
curl -s https://builds.coreos.fedoraproject.org/streams/stable.json | jq '.architectures.x86_64.artifacts.metal.formats.pxe'
{
"variant": "fcos",
"version": "1.4.0",
"systemd": {
"units": [
{
"name": "hello.service",
"enabled": true,
"contents": "[Unit]\nDescription=MyApp\nAfter=network-online.target\nWants=network-online.target\n\n[Service]\nTimeoutStartSec=0\nExecStartPre=-/bin/podman kill busybox1\nExecStartPre=-/bin/podman rm busybox1\nExecStartPre=/bin/podman pull busybox\nExecStart=/bin/podman run --name busybox1 busybox /bin/sh -c \"trap 'exit 0' INT TERM; while true; do echo Hello World; sleep 1; done\"\n\n[Install]\nWantedBy=multi-user.target\n"
}
]
}
}
Fedora Updates Testing x86_64 Official
kubernetes-1.24.4-1.fc36.x86_64.rpm Container cluster management kubernetes-client-1.24.4-1.fc36.x86_64.rpm Kubernetes client tools kubernetes-kubeadm-1.24.4-1.fc36.x86_64.rpm Kubernetes tool for standing up clusters kubernetes-master-1.24.4-1.fc36.x86_64.rpm Kubernetes services for control plane host kubernetes-node-1.24.4-1.fc36.x86_64.rpm Kubernetes services for worker node host
shadow-utils containernetworking-plugins cri-tools kubernetes-node containerd or cri-o conntrack-tools kubernetes-client libc.so.6(GLIBC_2.34)(64bit) rtld(GNU_HASH) socat
sudo podman run --pull=always --privileged --rm \ -v /dev:/dev -v /run/udev:/run/udev -v .:/data -w /data \ quay.io/coreos/coreos-installer:release \ install /dev/vdb -i config.ign
Ignite
Flatcar
https://www.flatcar.org/docs/latest/installing/bare-metal/booting-with-ipxe/
wget http://boot.ipxe.org/ipxe.iso qemu-kvm -m 1024 ipxe.iso -display curses
iPXE> dhcp iPXE> chain http://${YOUR_BOOT_URL}
curl -sSL https://stable.release.flatcar-linux.net/amd64-usr/current/version.txt | grep FLATCAR_VERSION cat /etc/os-release | grep VERSION update_engine_client -update systemctl reboot diff <(curl -s https://stable.release.flatcar-linux.net/amd64-usr/current/version.txt) <(cat /var/www/ipxe/version.txt) echo $?
Kubernetes Linux Distributions
https://www.sidero.dev/ / https://www.talos.dev/
https://www.flatcar.org/ (follows CoreOS principles)
https://bottlerocket.dev/ (AWS)
https://github.com/bottlerocket-os/bottlerocket
https://vmware.github.io/photon/ (VMWARE)
u-boot
https://u-boot.readthedocs.io/en/latest/board/emulation/qemu-mips.html
https://source.denx.de/u-boot/u-boot
https://source.denx.de/u-boot/u-boot/-/tree/master/arch/mips/mach-ath79/qca953x
https://github.com/hauke/philips-hue-bsb002/tree/master/qsdk/qca/src/qca-legacy-uboot
https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=philips+hue
https://openwrt.org/docs/techref/instructionset/mips_24kc
https://www.youtube.com/watch?v=rVaiLgXccSE
apt install qemu-system-mips flex bison git clone https://source.denx.de/u-boot/u-boot.git cd u-boot git checkout v2022.01
git clean -f -x -d export TOPDIR=$(pwd) export CROSS_COMPILE=mips64-linux-gnuabi64- tar xvf ../delta.tar -C .
make distclean make bsb002_defconfig make menuconfig make
qemu-system-mips64 -cpu 24Kc -nographic -no-reboot -bios u-boot.bin /usr/share/seabios/vgabios-cirrus.bin
BIOS / UEFI
https://p.ost2.fyi/courses/course-v1:OpenSecurityTraining2+4021_Intro_UEFI+2022_v1/about
http://www.linuxfromscratch.org/blfs/view/systemd/postlfs/firmware.html
http://git.kernel.org/cgit/linux/kernel/git/firmware/linux-firmware.git/tree/
http://www.linuxfromscratch.org/hints/downloads/files/lfs-uefi.txt
You only need the efivar and efibootmgr packages to use GRUB 2.0+ in UEFI
--with-platform=efi
mount -t vfat /dev/x /boot/efi
BOOT64.EFI
grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=LFS-BOOT
https://wiki.osdev.org/UEFI
uefi-run -b /path/to/OVMF.fd -q /path/to/qemu app.efi -- <extra_qemu_args>
gdisk /path/to/uefi.img
losetup --offset 1048576 --sizelimit 46934528 /dev/loop0 /path/to/uefi.img
objdump --all-headers /path/to/main.efi
https://www.prime-expert.com/articles/a20/rebuild-efi-system-partition-from-scratch/
Commands
which ls ls -la /bin/ls ldd /bin/ls readelf -a /bin/ls objdump -x /bin/ls strace -c /bin/ls > /dev/null lsof -p 1234 gdb