Guillaume Matheron

Data scientist, PhD in computer science

Category: Infrastructure

  • Change ZFS volblock on a running Proxmox VM

    When we set up our infrastructure initially (Ubuntu VMs on proxmox with ZFS storage), I used the default block size, which was 8kB at the time.. Space considerations when using zvols on RAIDz Smaller block sizes are good because they limit write amplification (writing one byte in the guest require writing a full block to…

  • IP-based geolocation in Clickhouse with IPv6

    While setting up my analytics with clickhouse, I tried to map IPs to countries. I found this article that perfectly explains how to do this with IPv4, but gives no indication of how to adapt the queries to ipv6. The beginning of the process is identical: fetch IP ranges from dbip-city-ipv6 using a table with…

  • Reinstall proxmox while keeping all the configuration and VMs intact

    Let’s say you installed proxmox on a dedicated server to which you have physical access, but now you need to replace the boot disk. If you use a similar medium (replacing a SSD with a SSD or a usb key with a usb key) with a larger or equal size, then you can simply copy…

  • Taking advantage of ZFS for smarter Proxmox backups

    Let’s say we have a Proxmox cluster running ~30 VMs using ZFS as a storage backend. We want to backup each VM hourly to a remote server, and then replicate these backups to an offsite server. Proxmox Backup Server is nicely integrated into PVE’s web GUI, and can work with ZFS volumes. However, PBS is…

  • Proxmox cluster on distant bare metal servers

    I run a Proxmox cluster with three nodes that are set up on three rented bare-metal servers from OVH in different datacenters. This is a pretty unusual setup, because bare-metal rental companies to not allow bridging on their network interface. Bridging in a typical racked cluster In this context, bridging means that a single physical…

    Proxmox cluster on distant bare metal servers
  • The nightmare of Shingled Magnetic Recording for NAS drives

    IT systems are like onions, they have layers. And when the bottom-most layer fails, everything follows. This is the story of our experience with WD Red hard disk drives at Hello Watt, and how even while following good practices of system administration, deceptive advertising can cause your business to lose weeks of work.

    The nightmare of Shingled Magnetic Recording for NAS drives
  • Parcours de la DHT BitTorrent

    Le programme effectue un parcours de la DHT le plus exhaustif possible (environ un million de noeuds) en partant du bootstrap “router.bittorrent.com” de BitTorrent.Le réseau ainsi découvert est stocké au format csv pour pouvoir être ensuite étudié à l’aide d’outils tels que Octave ou Matlab.Comme la taille des données que le programme traite est conséquente,…

    Parcours de la DHT BitTorrent