Ready for HF28: Both my Witness Machines Updated to 1.28.3 and a New Failover System

in Value 4 Value2 days ago (edited)

Vote for Brianoflondon's Witness KeyChain or HiveSigner
Support Proposal 342 on PeakD

This is a value for value post: see the explanation in the footer.


TL;DR: Please consider voting for my witness. I've dramatically upgraded how I roll, even as I've been falling down the rankings. I've set up a new witness machine at home using an old laptop. I've also built an automatic fail-over system so if my main witness goes down, the backup machine takes over automatically.

Update: HF28 completed

HF28

Witness Update

I wrote a few days ago how I'd commissioned my kid's old laptop to be a new machine for my brianoflondon witness.

But having now put that machine to work and thinking it would lead to me decommissioning a $50 per month Privex witness, I realised I really should run two machines and have a fail-over system.

Running a witness at home is possible, but every now and then my router does reboot and that is an issue. Just this morning we had a 2 hour power cut: my home witness would have been down for that time. So I decided to keep the Privex witness going as my primary, and use my home witness as a fail-over.

VPS proxy

There is a little wrinkle running at home: my ISP blocks all unsolicited incoming connections which breaks the P2P model of Hive. I had to set up a VPS (Virtual Private Server) and picked a local Israeli host so latency is very low. I have a basic $9 machine which relays P2P traffic to my home witness via Tailscale dodging my ISP's restrictions.

Automatic failover

As part of the re-write of @v4vapp which has been a massive amount of work, I monitor the Hive chain very closely and it just seemed natural to build a fail-over system into the backend for @v4vapp. So I've done that.

Screenshot 2025-11-17 at 12.36.18.png

And so when I upgraded my Privex node this morning (which is basically just deleting the entire machine and starting from scratch to use @privex's re-built Hive node in a box) I got to watch my failover script work perfectly. When the new machine at Privex came back on line, my monitoring system recorded it.

Github

The code for my fail-over system is on GitHub however it isn't really useable as a stand alone system yet. It is integrated into my entire @v4vapp backend.

The specific code for the fail-over system is in this file: link to Github v4vapp_backend_v2

It relies on having a config which includes this:

hive:
  hive_accs:
    brianoflondon:
      role: witness
      active_key: active-key-needed-for-switching-machines
  watch_witnesses:
    - brianoflondon
  witness_configs:
    brianoflondon:
      witness_machines:
        - name: legion-witness
          url: http://tailscale_ip_1:8091
          signing_key: STM73EW8YMXWAMrCUosEuvLEqEPALzFijghyL2KyAQnWZep3whEZD
        - name: fox-v4vapp
          url: http://tailscale_ip_2:8091
          signing_key: STM6Yvdz6HtdhyzAi6oimvm5MFevSWeThYZJvbLGSmq4UeUxAEztgd
      # Optional Kuma Monitoring
      kuma_webhook_url: https://my_kuma_machine/api/push/huSWPvpQ0L
      kuma_heartbeat_time: 60  # seconds

Behind the scenes I use Tailscale to connect all my machines and that is used to monitor them.

I also use a tool called Kuma to monitor the health of my witness nodes. If a witness node stops responding to Kuma's heartbeat pings, my fail-over system kicks in and switches the witness signing to the backup machine

Not production ready

I haven't put this in a package which is easy to use, I'm not really planning to do that separately from the entire v4v.app backend. But if you are running multiple witness nodes and want to help me get this into a stand alone production ready state, please get in touch.

Home Workstation

My home working setup

Display settings on iMac

Here's my home setup with the witness running on the laptop on the right. I'm actually not going to leave it this way, there is a fault with the CPU fan which isn't running right now and I'm waiting for a replacement to arrive from AliExpress. When that is here I'll fold it up and tuck it away (next to another of my kids' old gaming laptops). And yes the "R" key is missing, it fell off a few weeks ago. That won't matter when it is a headless server.

The iMac is a dinosaur but it is special. It is a Retina 5K iMac from 2017 and whilst I've upgraded the storage to a 2TB SSD, the CPU (3.8 GHz Quad-Core Intel Core i5) is now officially ancient. But the screen itself is stunning. It is about to fall completely out of support for macOS updates but I simply can't throw this screen away.

There are now aftermarket ways to install a dedicated driver card into this machine (and dump the actual computer) in order to use it simply as an external monitor. Lots of details of how to do that in this YouTube video.

Why no target display mode?

Interestingly the reason why Apple didn't make it easy to use this as an external monitor (they had a system called 'target display mode') was because the machine was designed and released before any display connection capable of driving a 5K display existed.

Inside the machine currently is a custom display controller which Apple designed specifically for this machine and which treats the left and right halves of the display as two separate 2560x2880 displays. Nuts. I actually use it in a 1/2 resolution mode because my 55 year old eyes can't make use of the full resolution!

Thank You!

As HF28 goes live, thank you for any and all votes and support over the years. @v4vapp is getting a fair bit of use and I'm excited about the next stage when I can use VSC Magi to keep the BTC/Hive/HBD swaps within Hive.

Vote for Brianoflondon's Witness KeyChain or HiveSigner


Value for Value

For the last few months while building @v4vapp I was generously supported by the DHF. Going forward I have a much more modest support which covers direct server costs and a little of my time.

If you appreciate the work I do on and around Hive, you can express this directly: upvoting posts on Hive is great. Also consider a direct donation (there's a Tip button on Hive or a Lightning Address) on all my posts.

hivebuzz-orca-120.png

Support Proposal 342 on PeakD
Support Proposal 342 with Hivesigner
Support Proposal 342 on Ecency
Vote for Brianoflondon's Witness KeyChain or HiveSigner


Send Lightning to Me!

Sort:  

I'm also into re-using old tech right now.

I'm also planning to convert my 2015 iMac to a screen soon. I have all the parts and just got my 2019 Ender 3 Pro 3D printer up and running to print brackets for the install based on the files from the video in your post (now handily referenced).

I can also reuse 32Gb of its 48Gb of DDR3 SODIM RAM in a 2012 Macbook Retina that I just refurbished and converted to Ubuntu for my kids. I have Linuxed 2 of 3 more than decade old Macbooks and they work really well.

The other RAM & storage will we re-used in a $55 Zimblade NAS.

I'll have to do a post on it.

Good luck with your updated.

Thanks for being on top of it! Nice super smooth transition so far. Well done!

Witness nodes can be run on much cheaper VPS these days thanks to smaller state size that can be fully stored on disk and block_log pruning.

I'm going to look into that but I'm happy to have one next to me now to really understand what I'm doing.

This is cool to be updated.

Thank you so much for doing a very great work on the database