basically the idea is, only worker nodes and the big astridowski (via VPN) can access the control plane and nobody else can

i wonder how hard I'm gonna lean into the kubernetes stuff. I plan on having edge nodes to do caching and I wonder if I should make them kubernetes too

to start out with, I have a single hypervisor that will run 1 control plane and 2 workers. oh no spof yea yea whatever. the second machine will become a hypervisor and then I'll have 3 control planes and 3? 4? workers. and when I'm really advanced I'll have 3 hypervisors each with its own control plane! talk about high availability amirite (they are connected to one power strip

@astrid you could totally do edge nodes as a separate node group, control plane in different subnet should be fine so long as inbound access to api server is allowed. Depending on CNI you may need to allow kube-proxy or whatever through also, some need to access the kube api server via the internal service when bootstrapping

@arichtman i think i might do that (assuming the vps can support k8s and an nginx)

but barring acts of god (oops i ran heater and toaster together) i will have uptime for dayz

@astrid Lol. Why not do vClusters??

@arichtman i like the entropy of just. throwing nodes wherever lol

@astrid fair warning that stretch clusters are highly advised against. Etcd is strongly consistent and I hear even more than a whiff of latency causes big issues.

@arichtman yeah but also, the nodes won't be more than 3-10ms apart

@arichtman they're all being switched together it's yuri

@arichtman oh yeah also the edge nodes would just be in a different zone anyways. i mean worst case they run a single-node k3s/talos cluster

@astrid @arichtman
two nodes, chilling in a cluster, 5ms away because they're not gay

@arichtman actually the biggest availability issue would be the router appliance that all my k8s nodes would be behind. i could carp it tho

@astrid very true. My virtualized router is easily one of the most critical things. Second only to the hypervisor it's on I guess

@arichtman I have several virtualized routers happening

1. inferno/Lucifer, the one hooked to the modem, it's a firewall
2. asmodeus, who will handle dn42 peering
3. charon, who will handle all packets entering my production services
4. nyaanet, the wifi router

Lucifer is opnsense and I don't like opnsense very much after almost a year of using it and I might just move the role into nyaanet.

@astrid whew that's complicated! What ticked you off about opnsense? Seems more libre than pfsense and not trash GUI like openwrt?
Also - if you really wanna have fun, you can use bgo peering for the worker nodes to advertise their ipv6 routes to your main router and skip virtual load balancing.

@arichtman doesn't allow next hop to use link local routes. maybe i should just submit a patch tho. also honestly I wanna just have a really basic front-of-house so I don't kill the Internet with my shenanigans