Blog

Using FreshRSS's HTML + XPath with GoComics

Mon, 23 Sep 2024 00:00:00 -0400

FreshRSS supports scraping a site using XPath to generate new items for a feed. GoComics does not provide an RSS feed for comics to use but it is possible to use XPath on a comic’s page to get each day’s comic.

Set the feed URL to the comic’s page (e.g. https://www.gocomics.com/fminus for F Minus).

Under “Type of feed source”, set these options:

Type of feed source: HTML + XPath (Web scraping)

finding news items:

//div[contains(@class, "gc-deck--cta-0")]//picture[contains(@class, "gc-card__image")]

item title:
```
parent::h4
```
item content:
```
./img
```
item link (URL):
```
ancestor::a/@href
```
item thumbnail:
```
descendant::img/@src
```
item author:
```
"F Minus"
```
Custom date/time format:
```
Y.m.d
```

This is enough to regularly check the GoComics page and fetch each day’s comic. The title won’t reflect the date and instead be urn:sha1:<sha1 hash>. I haven’t figured out why that is but I also don’t care enough to fix it.

NTP using Raspberry Pi 4 with GPS

Fri, 15 Oct 2021 00:00:00 -0400

Hardware

Parts

Raspberry Pi 4. Any Raspberry Pi Model B will work. The Pi 4 moved the Ethernet to PCIe from USB so that will be less jitter for LAN traffic.
PoE HAT
Ultimate GPS HAT for Raspberry Pi
4x 15+5 mm M2.5 brass standoffs. I bought a large package of multiple sizes on Amazon. You need a 15 mm tall standoff for the stacking header below.
GPIO Stacking Header. This is the “Extra-long 2x20” version with 10 mm pins.
GPIO Female Socket Riser Header. With the GPIO pins higher up, the PoE pins also need a header to reach the PoE HAT.
SMA to uFL RF Adapter Cable
GPS Antenna

I also got a heatsink for the Raspberry Pi so the fan on the PoE HAT would not need to run as much. Once everything was assembled, there was about 2 mm of clearance between the fan and the GPS HAT.

I ordered almost everything from Adafruit because they are close by so shipping is fast.

Assembly

The Ultimate GPS HAT required soldering the included header. I have seen other GPS HATs that do not require soldering if that is a problem. I opted for this one because I had the tools, this is what Adafruit had, and rather wanted to try soldering.

Here is a component overview with everything that went into the build. The heatsink has already been applied.

Connect the stacking and 2x2 headers then attach the PoE HAT to them with the 15 mm standoffs to verify the PoE HAT works. The pins on the stacking header will go through the PoE HAT.

Use the standoffs included with the PoE HAT on top of the HAT and attach the GPS HAT on top of PoE and use the PoE HAT’s screws to secure everything together.

The PoE HAT was a bit finicky to use. The first time I connected the PoE HAT, everything was fine. I then disconnected everything for pictures, reconnected everything, soldered the header to the GPS HAT, and then the PoE HAT didn’t power on the Pi. I disconnected everything yet again, connected only the PoE HAT, and it worked again. I connected the GPS HAT a second time and it still worked so I don’t know what happened before.

Software

All of the steps assume Raspbian 10 (Buster). Adjust the packages to install to your distro of choice if you are not using Raspbian. The configuration of the software should remain very similar.

Install everything needed now at once. I opted for chrony but NTP works well and needs only small translations of the chrony options. Contrary to all other documentation I have seen on this, you do not need to compile anything from scratch.

$ sudo apt install chrony gpsd gpsd-clients pps-tools

Serial Port

The serial port is used to receive the NMEA 0183 data from the GPS device. Logging in from the serial port must be disabled but the serial hardware kept enabled for communication.

Run sudo raspi-config, choose “Interface Options”.

$ sudo raspi-config
┌─────────┤ Raspberry Pi Software Configuration Tool (raspi-config) ├──────────┐
│                                                                              │
│       1 System Options       Configure system settings                       │
│       2 Display Options      Configure display settings                      │
│       3 Interface Options    Configure connections to peripherals            │
│       4 Performance Options  Configure performance settings                  │
│       5 Localisation Options Configure language and regional settings        │
│       6 Advanced Options     Configure advanced settings                     │
│       8 Update               Update this tool to the latest version          │
│       9 About raspi-config   Information about this configuration tool       │
│                                                                              │
│                                                                              │
│                                                                              │
│                                                                              │
│                                                                              │
│                     <Select>                     <Finish>                    │
│                                                                              │
└──────────────────────────────────────────────────────────────────────────────┘

Then choose “Serial Port”.

┌─────────┤ Raspberry Pi Software Configuration Tool (raspi-config) ├──────────┐
│                                                                              │
│    P1 Camera      Enable/disable connection to the Raspberry Pi Camera       │
│    P2 SSH         Enable/disable remote command line access using SSH        │
│    P3 VNC         Enable/disable graphical remote access using RealVNC       │
│    P4 SPI         Enable/disable automatic loading of SPI kernel module      │
│    P5 I2C         Enable/disable automatic loading of I2C kernel module      │
│    P6 Serial Port Enable/disable shell messages on the serial connection     │
│    P7 1-Wire      Enable/disable one-wire interface                          │
│    P8 Remote GPIO Enable/disable remote access to GPIO pins                  │
│                                                                              │
│                                                                              │
│                                                                              │
│                                                                              │
│                                                                              │
│                     <Select>                     <Back>                      │
│                                                                              │
└──────────────────────────────────────────────────────────────────────────────┘

Select “No” to disable the login shell over serial port and then “Yes” to keep the serial port hardware enabled. Press Tab to select “Finish”. The utility will prompt you to reboot so do that.

Once the Pi comes back up, it is time to test the GPS HAT. Set the mode for /dev/serial0 to raw, 9600 baud, 8 bits, no modem control signals, and one stop bit. See the stty(1) man page for more information on everything.

$ sudo stty -F /dev/serial0 raw 9600 cs8 clocal -cstopb

Now check /dev/serial0 to verify NMEA data appears.

$ cat /dev/serial0
0.15,81.69,210821,,,A*56
$GNVTG,81.69,T,,M,0.15,N,0.28,K,A*1B
$GNGGA,154540.000,4XXX.XXXX,N,07XXX.XXXX,W,1,11,0.92,1XX.X,M,-34.2,M,,*4E
$GPGSA,A,3,02,05,20,06,19,09,,,,,,,1.25,0.92,0.84*01
$GLGSA,A,3,65,87,88,71,72,,,,,,,,1.25,0.92,0.84*10
$GNRMC,154540.000,A,4XXX.XXXX,N,07XXX.XXXX,W,0.57,81.69,210821,,,A*57

There should be output regardless if there is a fix or not. Once there is a fix, more of the GPS data will be filled in with much less “mostly comma” output. This HAT has a LED that will blink every second when it is searching and slow to blinking every 15 seconds when it has a fix.

I was able to get a fix within a few minutes while indoors, 20 feet from a window, and without attaching the external antenna.

One Pulse Per Second (1PPS)

Use the pps-tools package to monitor the 1 Pulse Per Second GPS sends to get exactly when each GPS second begins.

Edit /boot/config.txt to enable kernel support for PPS. /boot/overlays/README documents the available overlays and pps-gpio is listed in there. The Adafruit GPS HAT uses GPIO pin 4 so add that to config.txt.

dtoverlay=pps-gpio,gpiopin=4

Reboot to use the overlay.

When the Pi boots up, test the signal is received with ppstest.

$ sudo ppstest /dev/pps0
trying PPS source "/dev/pps0"
found PPS source "/dev/pps0"
ok, found 1 source(s), now start fetching data...
source 0 - assert 1629489053.004789031, sequence: 98 - clear  0.000000000, sequence: 0
source 0 - assert 1629489054.004796073, sequence: 99 - clear  0.000000000, sequence: 0
source 0 - assert 1629489055.004806575, sequence: 100 - clear  0.000000000, sequence: 0

GPSD

The gpsd package includes a systemd socket unit file, gpsd.socket, for systemd to automatically start GPSD when something connects to the default GPSD port of 2947. This is useful for regular GPS use but can be frustrating when debugging. My advice is to stop (sudo systemctl stop gpsd.socket) and disable (sudo systemctl disable gpsd.socket) the unit file if you need to debug GPSD. Leaving gpsd.socket disabled will be fine for timekeeping because gpsd.service will be used instead.

Edit /etc/default/gpsd to set DEVICES to use the serial port to receive GPS NMEA data and the pps0 device to receive the kernel’s PPS. Also set GPSD_OPTIONS to tell GPSD to immediately talk to the device and not wait for a client to connect.

DEVICES="/dev/serial0 /dev/pps0"
GPSD_OPTIONS="-n"

Restart GPSD to use those devices and enable it to start automatically at boot.

$ sudo systemctl restart gpsd.service
$ sudo systemctl enable gpsd.service

Run gpsmon or cgps -s to connect to GPSD and display the GPS information. Check for PPS data and PPS lines to appear in the packet log. Note, if the terminal is 24 lines tall, you will only see a single line of the packet log and it will be difficult to spot the PPS line.

$ gpsmon
tcp://localhost:2947          NMEA0183>
┌──────────────────────────────────────────────────────────────────────────────┐
│Time: 2021-08-22T21:53:45.000Z Lat:  XX XX' XX.XXXX0" Non:  XX XX' XX.XXXX0" W│
└───────────────────────────────── Cooked TPV ─────────────────────────────────┘
┌──────────────────────────────────────────────────────────────────────────────┐
│ GPGST GNGGA GPGSA GLGSA GNRMC GNZDA GPGRS GPGSV GLGSV                        │
└───────────────────────────────── Sentences ──────────────────────────────────┘
┌──────────────────┐┌────────────────────────────┐┌────────────────────────────┐
│Ch PRN  Az El S/N ││Time:      215345.000       ││Time:      215345.000       │
│ 0  XX  XX XX  41 ││Latitude:     4XXX.XXXX N   ││Latitude:  4XXX.XXXX        │
│ 1  XX  XX XX  37 ││Longitude:   07XXX.XXXX W   ││Longitude: 07XXX.XXXX       │
│ 2  XX XXX XX  28 ││Speed:     0.26             ││Altitude:  1XX.X            │
│ 3  XX XXX XX  36 ││Course:    94.750           ││Quality:   1   Sats: 08     │
│ 4  XX XXX XX  27 ││Status:    A       FAA: A   ││HDOP:      1.17             │
│ 5  XX  XX XX  21 ││MagVar:                     ││Geoid:     -34.2            │
│ 6  XX XXX XX   0 │└─────────── RMC ────────────┘└─────────── GGA ────────────┘
│ 7  XX XXX XX   0 │┌────────────────────────────┐┌────────────────────────────┐
│ 8  XX XXX XX   0 ││Mode: A3 Sats: 23 24 10 18  ││UTC: 215345.000RMS: 7.3     │
│ 9 XXX  XX XX   0 ││DOP: H=1.17  V=0.91  P=1.48 ││MAJ: 4.5       MIN: 4.0     │
│10  XX  XX XX  40 ││TOFF:  0.319084138          ││ORI: 164.5     LAT: 4.4     │
│11  XX 1XX XX  37 ││PPS:  0.000759500           ││LON: 4.1       ALT: 24      │
└────── GSV ───────┘└──────── GSA + PPS ─────────┘└─────────── GST ────────────┘
(72) $GPGRS,215344.000,1,-5.93,-0.14,0.83,36.9,3.35,9.75,-0.36,-2.94,,,,*72
(51) $GPGST,215344.000,7.4,4.4,3.9,150.3,4.3,4.0,23*40
------------------- PPS offset:  0.000759500 ------
(75) $GNGGA,215345.000,4XXX.XXXX,N,07XXX.XXXX,W,1,08,1.17,1XX.X,M,-34.2,M,,*4E
(52) $GPGSA,A,3,23,18,24,15,10,,,,,,,,1.48,1.17,0.91*0B
(48) $GLGSA,A,3,69,78,68,,,,,,,,,,1.48,1.17,0.91*12
(71) $GNRMC,215345.000,A,4XXX.XXXX,N,07XXX.XXXX,W,0.26,94.75,220821,,,A*5E
(35) $GNZDA,215345.000,22,08,2021,,*45
(71) $GPGRS,215345.000,1,-5.51,0.95,0.27,37.1,2.82,10.6,-2.84,-3.96,,,,*57
(51) $GPGST,215345.000,7.3,4.5,4.0,164.5,4.4,4.1,24*49
------------------- PPS offset:  0.000758989 ------

Note the “PPS offset” lines and the PPS time offset in the display. That means GPSD is receiving everything and can send the data to the desired timekeeping daemon.

Timekeeping

Before configuring chrony, run ntpshmmon as root to find the shared memory (SHM) sources that GPSD is already populating. Despite its name starting with “ntp”, ntpshmmon is part of the gpsd package.

$ sudo ntpshmmon
ntpshmmon version 1
#      Name Seen@                Clock                Real                 L Prec
sample NTP0 1629680609.372849180 1629680608.363601521 1629680608.000000000 0 -20
sample NTP2 1629680609.373066773 1629680609.041970056 1629680609.000000000 0 -30
sample NTP0 1629680609.385268612 1629680609.384725334 1629680609.000000000 0 -20
sample NTP2 1629680610.042570932 1629680610.041973357 1629680610.000000000 0 -30

The precision (Prec) field is the precision of the clock scaled with log(2). The lower the value in the precision column, the more accurate that clock is.

Usually the NMEA clock is at NTP0 and 1PPS is at NTP1. With this HAT the 1PPS appears at NTP2 (note the more accurate precision of -30 vs -20).

chrony

Edit /etc/chrony/chrony.conf to use the clocks from GPSD. I kept the pool line to allow chrony to synchronize with hosts over the internet until it can determine the accuracy of the PPS clock. The precision is because chrony cannot read the precision from the shared memory segments. The delay is because the GPS NMEA data is read over a relatively slow serial port and the larger delay allows greater range for overlap when reading the two clocks. I will adjust it later once its offset is known.

refclock SHM 0 refid GPS precision 1e-1 delay 0.1
refclock SHM 2 refid PPS precision 1e-7

Restart chrony.

$ sudo systemctl restart chrony.service

Now wait a while and check periodically to see if chrony picked the PPS clock. When it does, “PPS” clock will have the prefix be #*. Any other sources mixed in with the chosen source will have a prefix of “+” while a prefix of “-“ or “x” are excluded sources.

I added the -n argument to make chronyc not try to resolve the IPs to display the output faster.

$ chronyc -n sources
210 Number of sources = 6
MS Name/IP address         Stratum Poll Reach LastRx Last sample
===============================================================================
#x GPS                           0   4   377    17   +387ms[ +387ms] +/-  150ms
#* PPS                           0   4   377    18   -592ns[-1062ns] +/-  851ns
...

After waiting for sample times to settle, I used the offset under the “Last sample” column to set an offset for the GPS NMEA clock in the config to bring it closer to the PPS clock and prevent chrony from classifying it as a “falseticker” (the “x” in “#x”).

Edit /etc/chrony/chrony.conf to set an offset and then restart chrony.service.

refclock SHM 0 refid GPS precision 1e-1 offset 0.370

Note the addition of offset to add a fixed offset to that clock and the removal of delay because the clock is approximately correct with an offset.

Serving Time

If you are not interested in using the Raspberry Pi as a time server then you are done. If you are interested, continue onwards.

chrony

Edit /etc/chrony/chrony.conf to enable other clients to connect to chronyd.

allow 192.168.0.0/24
# If you also have IPv6 on the subnet.
allow 2600:8394:a947:bf::/64

Restart chrony to enable serving time.

$ sudo systemctl restart chrony.service

DHCP

I use pfSense as a router and first set the Raspberry Pi to have a fixed IP address and then added that IP address as an NTP server on the LAN interface. My Fedora laptop automatically picked up the new NTP server but a different Raspberry Pi running Debian and a FreeBSD server did not.

$ chronyc -n sources
MS Name/IP address         Stratum Poll Reach LastRx Last sample
===============================================================================
^? ...
^* 192.168.0.4                   1   9   377   469    -11us[ -114us] +/- 1260us

That is almost one millisecond accuracy over WiFi. The other Raspberry Pi averages 300 µs accuracy over Ethernet.

Sources

Adafruit Ultimate GPS HAT for Raspberry Pi Tutorial. This was useful for setting up the Raspberry Pi’s serial port and checking the serial device. The tutorial used /dev/ttyAMA0 but it’s /dev/ttyS0 on Raspberry Pi 3 and newer.
Steve Friedl’s Building a GPS Time Server with the Raspberry Pi 3 I found this helpful as proof that everything would work. He used a Raspberry Pi 3 but the same HAT.
GPSD Time Service HOWTO This was invaluable for learning how GPSD interacted with chrony and NTP. How to configure the two time keeping daemons was pulled from this site. Additional information on tuning performance is also available.

MPT BIOS Fault 05h encountered at adapter PCI(02h,00h,00h)

Thu, 09 Sep 2021 00:00:00 -0400

I noticed a drive failed in my NAS recently. I have a SuperMicro 826 chassis with the SAS backplane that has activity and failure lights. Rather than checking all of the drives manually, I wanted to activate the failure light for that slot and pull a single drive.

Long story short, I ended up breaking the onboard firmware plus had to pull all of the drives to check the serial manually. Check the end for the solution.

I followed Dan Langille’s guide to flashing the firmware using mpsutil. The flashing succeeded but no change in the ability to activate any failure lights. I tried again using sas2flash and it went poorly as it finished. The flashing again went okay but when sas2flash tried to reset the LSI controller, the kernel crashed. The controller was attached to the front SAS enclosure that contained all of the drives for the system, including for /. That in retrospect wasn’t a good idea.

Once the system rebooted, I was greeted with this alarming message:

Unable to load LSI Corporation MPT BIOS
MPT BIOS Fault 05h encountered at adapter PCI(02h,00h,00h)
Press any key to continue...

The megarec utility couldn’t find the controller at all so I feared I had bricked it. Turned out I was simply using the wrong tools to find and flash the controller. The Mega* utilities are for the MegaRAID firmware while the onboard controller always had the basic IT firmware. Instead I needed sas2flash the entire time.

I tried flashing with other LSI firmware files for the 2116 but they never fully succeeded. sas2flsh.exe (I was running in FreeDOS now) always ended with an error like this:

Chip is in RESET state.  Attempting Host Boot...
ERROR: Firmware Host Boot Failed !
Due to error remaining commands will not be executed.
Unable to Process Commands.
Exiting SAS2Flash.

Solution

SuperMicro’s site has the P19 firmware but someone at ServeTheHome got the P20 firmware from SuperMicro and posted it on the site. The format of the files is a bit different but the ROM files worked perfectly with sas2flsh.exe. I copied the extracted files from the zip to a FreeDOS live CD and followed the commands from the BAT file. One of the commands (sas2flsh -e 7) cleared the entire firmware, including the SAS address. I had written down the SAS address before so I could recover the original one but the erasing wasn’t necessary in the first place. I wrote the firmware then the BIOS and rebooted. The LSI BIOS loaded successfully and FreeBSD was able to use the controller once again.

Upgrade Nexus 4 to LineageOS 16.0

Thu, 15 Apr 2021 00:00:00 -0400

I recently found my Nexus 4 after the move and wanted to get it usable again. It was running the last official version of Android, 5.1.1, for the device from November 2014. LineageOS dropped support for it several years ago, the device was released in 2012, so I hoped to find either the last nightly build or an unofficial build of something more recent. I found an install guide for the device that I used when installing LineageOS.

Partitioning

First however, I needed to repartition /system because the original partition size was no longer large enough. There were scripts out there to manage the partitioning but as an experienced Linux sysadmin I opted to do it myself. I also didn’t search hard and realize the scripts existed before trying.

I installed android-tools from Fedora’s repositories, plugged in the phone, and got it work. I could adb reboot and adb shell all I wanted but fastboot and adb sideload refused to work. The install guide mentioned this so I connected through a USB 3.0 hub and that fixed all of the sideload and fastboot problems.

With fastboot working, I booted the latest TWRP image for the Nexus 4.

# adb reboot bootloader
# fastboot boot /path/to/tmp/lineageos/twrp-3.5.2_9-0-mako.img
Sending 'boot.img' (10528 KB)                      OKAY [  0.431s]
Booting                                            OKAY [  0.030s]
Finished. Total time: 0.486s

From an unofficial image thread, I knew I needed /system to be at least 1272 MB. I ran adb shell, mounted /system, and checked its size to see it was only 827 MB. Following an answer on Android StackExchange on repartitioning a different device, I set out to increase /system.

I pushed a statically linked parted (I didn’t need gdisk but pushed it anyway).

# adb push /path/to/tmp/lineageos/parted /
/path/to/tmp/lineageos/parted: 1 file pushed, 0 skipped. 591.9 MB/s (464372 bytes in 0.001s)

Now shell in, mark /parted as executable for later, and check the partition layout.

# adb shell
~ # chmod +x /parted
~ # fdisk -l /dev/block/mmcblk0
Found valid GPT with protective MBR; using GPT

Disk /dev/block/mmcblk0: 30777344 sectors, 2740M
Logical sector size: 512
Disk identifier (GUID): 98101b32-bbe2-4bf2-a06e-2bb33d000c20
Partition table holds up to 28 entries
First usable sector is 34, last usable sector is 30777310

Number  Start (sector)    End (sector)  Size       Code  Name
          1024          132095       64.0M   0700  modem
        132096          133119        512K   0700  sbl1
        133120          134143        512K   0700  sbl2
        134144          138239       2048K   0700  sbl3
        138240          139263        512K   0700  tz
        139264          184319       22.0M   0700  boot
        184320          229375       22.0M   0700  recovery
        229376          230935        780K   0700  m9kefs1
        230936          232495        780K   0700  m9kefs2
        232496          234055        780K   0700  m9kefs3
        234496          235519        512K   0700  rpm
        235520          236543        512K   0700  aboot
        236544          237567        512K   0700  sbl2b
        237568          241663       2048K   0700  sbl3b
        241664          242687        512K   0700  abootb
        242688          243711        512K   0700  rpmb
        243712          244735        512K   0700  tzb
        244736          245759        512K   0700  metadata
        245760          278527       16.0M   0700  misc
        278528          311295       16.0M   0700  persist
        311296         2031615        840M   0700  system
       2031616         3178495        560M   0700  cache
       3178496        30775295       13.1G   0700  userdata
      30775296        30776319        512K   0700  DDR
      30776320        30777310        495K   0700  grow

/system is partition 21. In order to resize it, I need to move at least the next partition. /cache is next but at 560 MB, it was not large enough and I would also need to shrink userdata (or /data).

I saved the above layout in case I needed to redo it exactly. NOTE, I already unlocked the bootloader and wiped everything but if you haven’t, this will reformat and lose everything and you must start from scratch.

Unmount partitions 21, 22, and 23 if mounted.

~ # df -h
Filesystem                Size      Used Available Use% Mounted on
tmpfs                   935.1M     20.0K    935.1M   0% /dev
tmpfs                   935.1M     16.0K    935.1M   0% /tmp
/dev/block/mmcblk0p23
                         12.9G    129.6M     12.8G   1% /data
/dev/block/mmcblk0p23
                         12.9G    129.6M     12.8G   1% /sdcard
/dev/block/mmcblk0p22
                        551.7M      9.8M    541.9M   2% /cache
~ # umount /sdcard /data /cache

Delete partitions 21, 22, and 23.

~ # /parted /dev/block/mmcblk0
GNU Parted 3.2
Using /dev/block/mmcblk0
...
(parted) print
Error: The backup GPT table is corrupt, but the primary appears OK, so that will
be used.
OK/Cancel? ok

The backup GPT being corrupt was disconcerting but I couldn’t do anything about it so accepted and continued onward. The StackExchange answer used parted’s resizepart but that only works if there’s space after the partition. There is zero free space here so must delete and recreate.

Model: MMC 016G92 (sd/mmc)
Disk /dev/block/mmcblk0: 15.8GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name      Flags
    524kB   67.6MB  67.1MB  fat16        modem     msftdata
    67.6MB  68.2MB  524kB                sbl1
    68.2MB  68.7MB  524kB                sbl2
    68.7MB  70.8MB  2097kB               sbl3
    70.8MB  71.3MB  524kB                tz
    71.3MB  94.4MB  23.1MB               boot
    94.4MB  117MB   23.1MB               recovery
    117MB   118MB   799kB                m9kefs1
    118MB   119MB   799kB                m9kefs2
    119MB   120MB   799kB                m9kefs3
    120MB   121MB   524kB                rpm
    121MB   121MB   524kB                aboot
    121MB   122MB   524kB                sbl2b
    122MB   124MB   2097kB               sbl3b
    124MB   124MB   524kB                abootb
    124MB   125MB   524kB                rpmb
    125MB   125MB   524kB                tzb
    125MB   126MB   524kB                metadata
    126MB   143MB   16.8MB               misc
    143MB   159MB   16.8MB  ext4         persist
    159MB   1040MB  881MB   ext4         system
    1040MB  1627MB  587MB   ext4         cache
    1627MB  15.8GB  14.1GB  ext4         userdata
    15.8GB  15.8GB  524kB                DDR
    15.8GB  15.8GB  507kB                grow

(parted) rm 21
(parted) rm 22
(parted) rm 23

I switched the units to sectors to get the partition math exactly correct. I did not think using MB, or anything other than only B(ytes), would be precise enough and using bytes could cause a partition to not end on a sector boundary. parted warned if I did not end on a good boundary, I think every 2 MB? When doing the sector math, I ensured the three partitions ended on a 2 MB boundary and parted was satisfied.

(parted) unit S
(parted) print
Model: MMC 016G92 (sd/mmc)
Disk /dev/block/mmcblk0: 30777344s
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start      End        Size      File system  Name      Flags
    1024s      132095s    131072s   fat16        modem     msftdata
    132096s    133119s    1024s                  sbl1
    133120s    134143s    1024s                  sbl2
    134144s    138239s    4096s                  sbl3
    138240s    139263s    1024s                  tz
    139264s    184319s    45056s                 boot
    184320s    229375s    45056s                 recovery
    229376s    230935s    1560s                  m9kefs1
    230936s    232495s    1560s                  m9kefs2
    232496s    234055s    1560s                  m9kefs3
    234496s    235519s    1024s                  rpm
    235520s    236543s    1024s                  aboot
    236544s    237567s    1024s                  sbl2b
    237568s    241663s    4096s                  sbl3b
    241664s    242687s    1024s                  abootb
    242688s    243711s    1024s                  rpmb
    243712s    244735s    1024s                  tzb
    244736s    245759s    1024s                  metadata
    245760s    278527s    32768s                 misc
    278528s    311295s    32768s    ext4         persist
    311296s    2811295s   2500000s  ext4         system
    30775296s  30776319s  1024s                  DDR
    30776320s  30777310s  991s                   grow

(parted) mkpart primary ext4 311296s 3178495s
(parted) mkpart primary ext4 3178496s 4202495s
(parted) mkpart primary ext4 4202496s 30775295s
(parted) print
Model: MMC 016G92 (sd/mmc)
Disk /dev/block/mmcblk0: 30777344s
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start      End        Size       File system  Name      Flags
    1024s      132095s    131072s    fat16        modem     msftdata
    132096s    133119s    1024s                   sbl1
    133120s    134143s    1024s                   sbl2
    134144s    138239s    4096s                   sbl3
    138240s    139263s    1024s                   tz
    139264s    184319s    45056s                  boot
    184320s    229375s    45056s                  recovery
    229376s    230935s    1560s                   m9kefs1
    230936s    232495s    1560s                   m9kefs2
    232496s    234055s    1560s                   m9kefs3
    234496s    235519s    1024s                   rpm
    235520s    236543s    1024s                   aboot
    236544s    237567s    1024s                   sbl2b
    237568s    241663s    4096s                   sbl3b
    241664s    242687s    1024s                   abootb
    242688s    243711s    1024s                   rpmb
    243712s    244735s    1024s                   tzb
    244736s    245759s    1024s                   metadata
    245760s    278527s    32768s                  misc
    278528s    311295s    32768s     ext4         persist
    311296s    3178495s   2867200s   ext4
    3178496s   4202495s   1024000s   ext4
    4202496s   30775295s  26572800s  ext4
    30775296s  30776319s  1024s                   DDR
    30776320s  30777310s  991s                    grow

(parted) name 21 system
(parted) name 22 cache
(parted) name 23 userdata
(parted) unit MB
(parted) print
Model: MMC 016G92 (sd/mmc)
Disk /dev/block/mmcblk0: 15758MB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start    End      Size     File system  Name      Flags
    0.52MB   67.6MB   67.1MB   fat16        modem     msftdata
    67.6MB   68.2MB   0.52MB                sbl1
    68.2MB   68.7MB   0.52MB                sbl2
    68.7MB   70.8MB   2.10MB                sbl3
    70.8MB   71.3MB   0.52MB                tz
    71.3MB   94.4MB   23.1MB                boot
    94.4MB   117MB    23.1MB                recovery
    117MB    118MB    0.80MB                m9kefs1
    118MB    119MB    0.80MB                m9kefs2
    119MB    120MB    0.80MB                m9kefs3
    120MB    121MB    0.52MB                rpm
    121MB    121MB    0.52MB                aboot
    121MB    122MB    0.52MB                sbl2b
    122MB    124MB    2.10MB                sbl3b
    124MB    124MB    0.52MB                abootb
    124MB    125MB    0.52MB                rpmb
    125MB    125MB    0.52MB                tzb
    125MB    126MB    0.52MB                metadata
    126MB    143MB    16.8MB                misc
    143MB    159MB    16.8MB   ext4         persist
    159MB    1627MB   1468MB   ext4         system
    1627MB   2152MB   524MB    ext4         cache
    2152MB   15757MB  13605MB  ext4         userdata
    15757MB  15757MB  0.52MB                DDR
    15757MB  15758MB  0.51MB                grow

Partitions created. I shrunk /cache from 587 MB to 524 MB and /data from 14.1 GB to 13.6 GB. /system went from 881 MB to 1468 MB.

Creating File Systems

Next re-created the ext4 file system for the three partitions.

~ # make_ext4fs /dev/block/mmcblk0p21
Creating filesystem with parameters:
    Size: 1468006400
    Block size: 4096
    Blocks per group: 32768
    Inodes per group: 8160
    Inode size: 256
    Journal blocks: 5600
    Label:
    Blocks: 358400
    Block groups: 11
    Reserved block group size: 87
Created filesystem with 11/89760 inodes and 11769/358400 blocks
~ # make_ext4fs /dev/block/mmcblk0p22
Creating filesystem with parameters:
    Size: 524288000
    Block size: 4096
    Blocks per group: 32768
    Inodes per group: 8000
    Inode size: 256
    Journal blocks: 2000
    Label:
    Blocks: 128000
    Block groups: 4
    Reserved block group size: 31
Created filesystem with 11/32000 inodes and 4110/128000 blocks
~ # make_ext4fs /dev/block/mmcblk0p23
Creating filesystem with parameters:
    Size: 13605273600
    Block size: 4096
    Blocks per group: 32768
    Inodes per group: 8144
    Inode size: 256
    Journal blocks: 32768
    Label:
    Blocks: 3321600
    Block groups: 102
    Reserved block group size: 815
Created filesystem with 11/830688 inodes and 93063/3321600 blocks
~ # e2fsck -f /dev/block/mmcblk0p21
e2fsck 1.42.9 (28-Dec-2013)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
system: 2563/53760 files (0.0% non-contiguous), 176829/215040 blocks
~ # resize2fs /dev/block/mmcblk0p21
resize2fs 1.42.9 (28-Dec-2013)
Resizing the filesystem on /dev/block/mmcblk0p21 to 358400 (4k) blocks.
The filesystem on /dev/block/mmcblk0p21 is now 358400 blocks long.

I forgot to verify if /system was created as a 1.4 GB file system or as 827 MB but resize2fs grew the file system so I assume it was too small. One last check that /etc/fstab will function

~ # cat /etc/fstab
/dev/block/mmcblk0p21 /system ext4 rw 0 0
/dev/block/mmcblk0p23 /data ext4 rw 0 0
/dev/block/mmcblk0p22 /cache ext4 rw 0 0
~ # mount /system
~ # mount /data
~ # mount /cache
~ # df -h
Filesystem                Size      Used Available Use% Mounted on
tmpfs                   935.1M     20.0K    935.1M   0% /dev
tmpfs                   935.1M     16.0K    935.1M   0% /tmp
/dev/block/mmcblk0p21
                          1.3G     26.2M      1.3G   2% /system
/dev/block/mmcblk0p23
                         12.4G    129.5M     12.3G   1% /data
/dev/block/mmcblk0p22
                        492.6M      8.6M    483.9M   2% /cache

Installing LineageOS

I tried to sideload the image now but there were errors about unable to detect filesystem for /dev/block/platform/msm_sdcc.1/by-name/system. I rebooted, skipped through the warning about no OS, and booted the recovery image again from fastboot.

I had numerous issues when trying to sideload the last nightly LineageOS 15.1 image for the Nexus 4 along with the matching Open GApps. That image resized /system back to 827 MB each time. After growing it again to sideload Open GApps, rebooting would load LineageOS only for it to reboot into the recovery image.

What did work was the unofficial image created by voron00 and the matching Open GApps for Android 9.0, micro version. The LineageOS 15.1 image also shrunk /system a small amount that I grew back to its maximum size before sideloading Open GApps.

# adb sideload /path/to/tmp/lineageos/lineage-16.0-20210311-UNOFFICIAL-mako-signed.zip
Total xfer: 1.00x
# adb shell
# ...resizing again...
# adb sideload /path/to/tmp/lineageos/open_gapps-arm-9.0-micro-20210409.zip
Total xfer: 1.07x
# adb reboot

I watched the LineageOS boot screen for a long time and then the Android 9.0 startup screen appeared. I breezed through the setup and the device was ready.

I also have the Nexus 7 from 2012 (code name grouper) that suffers from the same problem. Next up, moving that to something newer than the CyanogenMod 12.1 install.

Installing FreeBSD using libvirt and bhyve

Fri, 23 Nov 2018 08:00:00 -0500

I needed to do some testing with jails in FreeBSD and rather than enable nested jails I decided to use a FreeBSD VM. FreeBSD hosts VM images but their root disk is 30 GB when I only needed something in the 10 GB range. The official docs start with installing FreeBSD so I decided to replicate it with libvirt.

Overall it was straightforward except for getting the ISO to boot. The libvirt docs state to put the ISO first to boot from it. That did not work. The key part was discovering <boot dev='cdrom'/> in <os> to select the boot device.

After libvirt start freebsd-11 finished, running virsh console freebsd-11 brought up the boot sequence and then installer.

Working domain XML. The disk image was created using truncate.

<domain type='bhyve'>
    <name>freebsd-11</name>
    <uuid>82d9b876-d6bd-47a0-8c66-e2c8ddd967f2</uuid>
    <memory>262144</memory>
    <currentMemory>262144</currentMemory>
    <vcpu>1</vcpu>
    <os>
       <type>hvm</type>
       <boot dev='cdrom'/>
    </os>
    <features>
      <apic/>
      <acpi/>
    </features>
    <clock offset='utc'/>
    <on_poweroff>destroy</on_poweroff>
    <on_reboot>restart</on_reboot>
    <on_crash>destroy</on_crash>
    <devices>
      <disk type='file'>
        <driver name='file' type='raw'/>
        <source file='/vm/freebsd-11/freebsd-11.raw'/>
        <target dev='vda' bus='virtio'/>
      </disk>
      <disk type='file' device='cdrom'>
        <driver name='file' type='raw'/>
        <source file='/vm/freebsd-11/FreeBSD-11.2-RELEASE-amd64-bootonly.iso'/>
        <target dev='hda' bus='sata'/>
        <readonly/>
      </disk>
      <interface type='bridge'>
        <model type='virtio'/>
        <source bridge="bridge0"/>
      </interface>
      <serial type="nmdm">
        <source master="/dev/nmdm2A" slave="/dev/nmdm2B"/>
      </serial>
    </devices>
</domain>

Electricity Usage Monitoring with rtl-sdr

Sat, 27 Oct 2018 00:00:00 -0400

Inspired by a blog post, I wanted to start tracking my household’s electricity usage. A quick trip to Amazon and $30 later, I had my own rtl-sdr device. I like rrdtool’s graphs though so I used that for the data storage and graphing.

The source code is on GitHub. One interesting though about the meter is that it reports in decaWatt-hours. Mine appears to consistently update its total every five minutes. It does not happen precisely every 300 seconds but in the 300-330 second range.

Example Graph

The usage is noisier than I thought when not running a large appliance such as the oven or (electric) dryer.

Brocade ICX 6450

Sun, 21 Oct 2018 00:00:00 -0400

Thanks to a deal on ServeTheHome, I upgraded the core switch for the house from a D-Link DGS 1510-28X to a Brocade ICX 6450-48P. I wanted the 6450-24P instead but couldn’t find any good prices except for the 48P. I got very lucky in my purchase though because the switch came fully licensed.

Fans

When the switch has power, the fans start at 100% and they are very loud. Eventually FastIron starts and adjusts the fans to a lower speed. For me, even at the lower speed they are still rather loud so I’m documenting here what model they are to make it easier to find replacements in the future.

The model I saw was Nidec W40S12BGB5-52 but the closest model available are the W40S-A5. They are 40x40x28 mm, push 21.2 CFM at most, and the maximum noise is 51 dB.

I used a voltmeter to determine the fan starts at 11.9 V and then drops to 4.4 V on low. I next used a sound meter on the phone and measured around 50 dB a foot in front of the switch when the fans were on low. Even assuming inaccuracy from not using a proper meter, that was much too high and justified looking into replacements. Others on the STH thread had similar issues despite the switch supposedly being “silent” on low. Checking the fan status with show chassis indicated the fans had failed but they were still moving a lot of air.

I looked around a lot for a replacement fan and ended up picking the Sunon KDE1204PKVX. When replacing the fans, the wire arrangement of the original Nidec differed from what the Sunon used so the Sunon fans did not start with the switch. In the image below, the Nidec fan is on the left and the unmodified Sunon is on the right.

I succeeded in popping out the yellow and black wires using a heavy duty staple and then swapped them on each fan. I removed each Nidec fan, installed the Sunon fan in the appropriate orientation to keep the air blowing from front to back, and tidied the wires using the clips. The wires didn’t fully fit in the clip so none of them are latched but it is enough to keep the wires in place.

The switch no longer thinks the fans have failed, instead reporting their speed as auto. I hoped the RPMs would be reported but no luck. After leaving the switch on for a few hours, the reported temps are in the low 40s Celsius. This is higher than before, when they were around 35 °C. The switch being near silent is worth the ten degree change plus the warning temp is at 69 °C, plenty of thermal room.

Brocade VDX VCS-1002 error

Wed, 22 Aug 2018 00:00:00 -0400

I found zero hits for the VCS-1002 error for Extreme Networks (formerly Brocade) VDX line so this is my experience with it and how to fix it.

I had two VDX switches and wanted to form a VCS logical-chassis fabric. I knew their firmwares had to be the same version but was unsure how much needed to match: one switch had NOS 6.0.2b and the other 6.0.2f. I hoped that it was enough to match on the “6.0.2” part (short version: no). The switches never formed a logical-chassis fabric.

I finally realized how to check the logs and saw an error about the primary switch being unable to configure the secondary.

2018/04/30-12:55:39 [VCS-1002], 10559, SW/0 | Active, ERROR, VDX6740, Event: VCS cluster create, Coordinator IP: IP, VCS ID: 1, Status: VCS cluster failed to be created, Reason: Failed to configure all new locations.

I backed down the VCS level to fabric cluster and then a fabric formed. I still wanted the full thing however so opened a support case and after talking to the representative, discovered the versions must exactly match. I got the 6.0.2f firmware, upgraded the other switch, and logical-chassis fabric formed immediately (or as immediately as the slow switches can).

Some time later a third switch was added and this time it seamlessly joined the fabric once the VCS ID was updated.

FreeBSD devfs.rules(5)

Mon, 04 Jun 2018 00:00:00 -0400

I was experimenting with an RTL-SDR on FreeBSD. I could run rtl_tcp on Linux as a regular user but not on FreeBSD. Checking the mode and no group or world permission bits were set.

crw-------  1 root  operator  0x2b May 30 08:58 /dev/usb/0.1.0

I needed at least group permissions. Because rtl_tcp is getting data from the outside world I opted for a separate user for only that. Conveniently, an example very similar to this was in the manpage for devfs.rules(5). First I created a usb user and group and now need to allow it to read USB devices.

USB devices appear in /dev/usb so that path is what need to be used in the rules. The name can be anything while the number is the ID of the rules. Add this to /etc/devfs.rules:

[localrules=10]
add path 'usb/*' mode 0660 group usb

Those new rules have to be enabled using that name. Add this to /etc/rc.conf:

devfs_system_ruleset="localrules"

To make the new rules take effect, devfs needs to be restarted:

/etc/rc.d/devfs restart

Now, this is what the USB device looks like:

crw-rw----  1 root  usb  0x2e May 30 07:59 /dev/usb/0.1.0

bhyve, libvirt, and grub2-bhyve

Tue, 01 Aug 2017 17:59:00 -0400

At the end of my bhyve and libvirt post is note that booting with grub isn’t automatic:

Needing to manually tell grub how to boot the VM is frustrating but being able to use virtio for the disk and network is good enough.

I solved this a while ago and forgot to include how. Eventually, I noticed the Debian cloud image I used as a base used extlinux to boot instead of what I expected, grub. Grub can’t boot if there’s no grub config so I installed grub2, set the configuration, and installed it on disk. The VM booted fine after that.

Later on I tried to boot a CentOS 7 cloud image. I knew it used grub already but it wouldn’t boot when using virsh start. After several times back and forth between my Linux laptop, where I could dissect the image, and the VM host running FreeBSD, where I only had access to the serial console, I noticed the problem. The key thing is the command may be grub-bhyve but the package name is grub2-bhyve. The problem is easier to see with the --help option:

grub-bhyve --help
Usage: grub-bhyve [OPTION...] vmname
...
  -d, --directory=DIR        use GRUB files in the directory DIR
                             [default=/boot/grub]

By default it checks for the grub configuration in /boot/grub. Debian uses /boot/grub so I didn’t notice. CentOS 7 however uses /boot/grub2 (and has /boot/grub1). I added a grub symlink to point to grub2, ran virsh start and it booted fine too.

Summary: make sure /boot/grub exists and has a usable grub.cfg. Grub does not have to be installed to the disk because that bootloader isn’t run.