103 lines
3.8 KiB
Markdown
103 lines
3.8 KiB
Markdown
# `network.tjo.cloud`
|
|
|
|
Handling networking between nodes and between virtual machines.
|
|
|
|
# Architecture
|
|
|
|
<img align="left" src="docs/arhitecture.excalidraw.svg" width="40%">
|
|
|
|
__WAN interface__ either represents an actual public interface (on Hetzner) or an interface in home LAN that has port-forwarded ports to it from home router.
|
|
|
|
__LAN interface__ is an ordinary lan network.
|
|
|
|
__ingress.tjo.cloud__ has port-forwarded all public ports to it (22, 25, 80, 443, 587 etc.). No other VM is accessible from the internet.
|
|
|
|
__network.tjo.cloud__ establishes Tailscale VPN connection between other network.tjo.cloud VMs. Using subnet routing it makes it possible that each VM can connect to all other VMs on any Proxmox host.
|
|
|
|
# Subnets
|
|
We are using `10.0.0.0/16` range for IPv4 as well as `fd74:6a6f:0::/48` for IPv6.
|
|
|
|
## Designations
|
|
| Host | IPv4 | IPv6 |
|
|
|------------|---------------|-----------------------|
|
|
| _reserved_ | 10.0.0.0/20 | fd74:6a6f:0:0000::/52 |
|
|
| batuu | 10.0.16.0/20 | fd74:6a6f:0:1000::/52 |
|
|
| jakku | 10.0.32.0/20 | fd74:6a6f:0:2000::/52 |
|
|
| nevaroo | 10.0.48.0/20 | fd74:6a6f:0:3000::/52 |
|
|
| | 10.0.64.0/20 | fd74:6a6f:0:4000::/52 |
|
|
| | 10.0.80.0/20 | fd74:6a6f:0:5000::/52 |
|
|
| | 10.0.96.0/20 | fd74:6a6f:0:6000::/52 |
|
|
| | 10.0.112.0/20 | fd74:6a6f:0:7000::/52 |
|
|
| | 10.0.128.0/20 | fd74:6a6f:0:8000::/52 |
|
|
| | 10.0.144.0/20 | fd74:6a6f:0:9000::/52 |
|
|
| | 10.0.160.0/20 | fd74:6a6f:0:a000::/52 |
|
|
| | 10.0.176.0/20 | fd74:6a6f:0:b000::/52 |
|
|
| | 10.0.192.0/20 | fd74:6a6f:0:c000::/52 |
|
|
| | 10.0.208.0/20 | fd74:6a6f:0:d000::/52 |
|
|
| | 10.0.224.0/20 | fd74:6a6f:0:e000::/52 |
|
|
| | 10.0.240.0/20 | fd74:6a6f:0:f000::/52 |
|
|
|
|
Each subnet gives us 4096 IP addresses per host.
|
|
|
|
### Per host designations
|
|
|
|
| Use | IPv4 | IPv6 |
|
|
|------------------|------------------|-----------------------|
|
|
| Virtual Machines | 10.0.(y+0).0/22 | fd74:6a6f:0:y000::/54 |
|
|
| Kubernetes | 10.0.(y+4).0/22 | fd74:6a6f:0:y400::/54 |
|
|
| | 10.0.(y+8).0/22 | fd74:6a6f:0:y800::/54 |
|
|
| | 10.0.(y+12).0/22 | fd74:6a6f:0:yc00::/54 |
|
|
|
|
Each section gives us 1024 IP addresses.
|
|
|
|
# Setting up new Host
|
|
|
|
### 1. Add new device to terraform.tfvars.
|
|
|
|
### 2. Manually configure vmbr0 and use import to import it.
|
|
|
|
### 3. Deploy terraform.
|
|
|
|
### 4. Set Password (see bitwarden)
|
|
|
|
### 5. Setup Tailscale.
|
|
Ref: https://github.com/adyanth/openwrt-tailscale-enabler
|
|
|
|
```
|
|
opkg update
|
|
opkg install ca-bundle kmod-tun iptables-nft kmod-ipt-conntrack kmod-ipt-conntrack-extra kmod-ipt-conntrack-label kmod-ipt-nat kmod-nft-nat
|
|
/etc/init.d/tailscale start
|
|
/etc/init.d/tailscale enable
|
|
|
|
# Batuu
|
|
tailscale up --accept-routes --advertise-routes=10.0.16.0/20,fd74:6a6f:0:1000::/52 --accept-dns=false --ssh
|
|
# Jakku
|
|
tailscale up --accept-routes --advertise-routes=10.0.32.0/20,fd74:6a6f:0:2000::/52 --accept-dns=false --ssh
|
|
# Nevaroo
|
|
tailscale up --accept-routes --advertise-routes=10.0.48.0/20,fd74:6a6f:0:3000::/52 --accept-dns=false --ssh
|
|
```
|
|
|
|
### 6. Configure.
|
|
|
|
Once tailscale is up and manually configured (see the config files for guide).
|
|
We can use automated way of maintaining config.
|
|
|
|
```
|
|
just deploy-config batuu 10.0.16.1 fd74:6a6f:0:1000::
|
|
just deploy-config jakku 10.0.32.1 fd74:6a6f:0:2000::
|
|
just deploy-config nevaroo 10.0.48.1 fd74:6a6f:0:3000::
|
|
```
|
|
|
|
# Static Internal IP Addresses
|
|
|
|
Any VM that requires a static IP, can add it's configuration to `openwrt/etc/config/dhcp`.
|
|
|
|
# TODO
|
|
|
|
## Use gitops for tailscale ACL.
|
|
|
|
Current version is an snapshot in time, more as an example then actual version used.
|
|
|
|
## IPv6 Connectivity.
|
|
|
|
As we assign private ipv6 addresses, we would have to ise ipv6 nat to translate those to real ipv6 addresses.
|