Metallb
Jump to navigation
Jump to search
Existing bugs for md5password
TCP_MD5SIG
2023/04/25 16:21:54.905 BGP: [KTTWK-0CPJ7][EC 100663303] sockopt_tcp_signature: setsockopt(22): Protocol not available 2023/04/25 16:21:54.905 BGP: [S3P4S-FRQ8G][EC 33554494] Unable to set TCP MD5 option on socket for peer 10.x.x.x (sock=22): Invalid argument
Permissions maybe - CAP_NET_RAW to test - https://github.com/metallb/metallb/issues/762 - https://unofficial-kubernetes.readthedocs.io/en/latest/concepts/policy/container-capabilities/
other
- https://github.com/openwrt/packages/issues/16702
- https://bugs.launchpad.net/raspbian/+bug/1183546
- https://github.com/raspberrypi/linux/issues/302
This works
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.13.9/config/manifests/metallb-frr.yaml # kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/main/config/manifests/metallb-frr.yaml # kubectl apply -f metallb-frr.yaml kubectl apply -f addresspool.yaml kubectl apply -f bgppeer.yaml kubectl apply -f bgpadvertisement1.yaml
Get started with Microk8s
https://metallb.universe.tf/configuration/
If firewall/router in front you can route or dnat form public ip to metallb private address assignment
sudo ip route add 10.x.x.0/22 via 10.y.y.y (10.y.y.y is k8s instance ip address though usually you would use bgp)
- https://microk8s.io/docs/addon-metallb 10.x.x.0/22 is your ip range
- https://devopslearning.medium.com/metallb-load-balancer-for-bare-metal-kubernetes-43686aa0724f
BGP with frr password doesn't seem to work
frr
frr version 8.1 frr defaults traditional hostname hm0101 log syslog informational log file /var/log/frr/frr.log service integrated-vtysh-config ! ip router-id 10.x.x.r ! router bgp 65551 bgp log-neighbor-changes neighbor 10.x.x.x remote-as 65551 ! neighbor 10.x.x.x password metallb neighbor 10.x.x.y remote-as 65551 ! neighbor 10.x.x.y password metallb neighbor 10.x.x.z remote-as 65551 ! neighbor 10.x.x.z password metallb exit !
metallb-system configMap config config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
namespace: metallb-system
name: config
data:
config: |
peers:
- # a test below is router/fw id
peer-address: 10.x.x.r
peer-asn: 65551
my-asn: 65551
# password: "metallb"
address-pools:
- name: my-ip-space
protocol: bgp
avoid-buggy-ips: true
addresses:
- 192.168.254.0/24
From Linux Router/GW/FW
ip route | grep bgp sudo vtysh -c "show bgp sum" sudo vtysh -c "show bgp neighbor" sudo vtysh -c "show ip route" sudo tcpdump -nnpi myint port 179 sudo tail -f /var/log/frr/frr.log
kubectl logs -f -l component=speaker -n metallb-system
Pod deployment with service for entry into cluster via bgp advertised ip
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-nginx
spec:
selector:
matchLabels:
run: my-nginx
replicas: 2
template:
metadata:
labels:
run: my-nginx
spec:
containers:
- name: my-nginx
image: nginx
ports:
- containerPort: 80
tolerations:
- key: "node.kubernetes.io/unreachable"
operator: "Exists"
effect: "NoExecute"
tolerationSeconds: 10
- key: "node.kubernetes.io/not-ready"
operator: "Exists"
effect: "NoExecute"
tolerationSeconds: 10
---
apiVersion: v1
kind: Service
metadata:
name: my-nginx
labels:
run: my-nginx
spec:
ports:
- port: 80
protocol: TCP
selector:
run: my-nginx
type: LoadBalancer
Let's do a little loop and we can shutdown node it is on and watch routes change
kubectl get pods -o wide shutdown your node while true; do curl 192.168.254.4; sleep 2; done
Other options
- https://github.com/meta-magic/metallb-baremetal-example
- https://stackoverflow.com/questions/63974879/microk8s-metallb-ingress-nginx-how-to-route-external-traffic
https://v0-2-1--metallb.netlify.app/tutorial/
arp
https://link.medium.com/xgtCprRJ0lb
https://manintheit.org/en/posts/kubernetes/kubernetes-metallb-loadbalancer-with-bgp-mode/