Setting up FreeBSD and jails on Azure – part 1: networking

I set up this blog on Azure as an excuse to play with the new FreeBSD VM Depot image, learn more about jails and write the occasional blog post about random stuff. I took extensive notes while at it and I will be posting them here for future reference and to help the occasional search engine user.

I will skip all the clicking through that can easily get to a running FreeBSD VM in Azure. There is tons of FreeBSD documentation, including specific Azure tutorials that my team and others have written. I am lazy, so I will just point out specific Azure differences and how to take care of them.

A word of caution: please don’t consider what you read here to be authoritative. I’m doing this for fun and my free time is what it is, so don’t think I researched this stuff thoroughly. It worked for me and seems to be still working as I write this – that’s all I needed.

Let’s start with networking. Every public cloud has their own approach, and Azure is no different. Two things to remember about Azure IP management:

  1. Your VM will have an internal IP that will be translated to a public one. If you are puzzled about the number not being in your expected 192.168.0.0/172.16.0.0/10.0.0.0 ranges, know that you are looking at the wrong RFC: the IP will come from a “Carrier-Grade NAT” pool specified in RFC6598. And, unless you pay more set up a VNET, it will change between reboots. Being the cheapskate I am, I didn’t bother. But here be dragons, so stay tuned.
  2. Your external IP is again assigned dynamically, and dyndns’d to [yourmachine].cloudapp.net. Again, for an additional (and actually minimal) charge you can reserve a block of IPs, but given that the IP will be reserved for the lifetime of your deployment (that is, it will survive a reboot), you may well not need to pay a premium. YMMV.

With that in mind, I quickly stumbled into an issue. Jails on FreeBSD require all the services in the host machine to be bound to specific interfaces. That is you cannot bind to 0.0.0.0, *.*.*.* or any general address. Ignore this, you may end up like me – locked out of your machine as soon as you start the first jail (damn you syslogd!).

‘sockstat -4’ is your friend here as it will list which services are bound to which interfaces. On my vanilla install I had to change syslogd and sshd as they both are configured to listen to everything and the kitchen sink.

Syslogd was a piece of cake as long as I was fine with loopback only – it may become trickier if I need to get fancy and gather logs from the network. Add this in /etc/rc.conf

syslogd_flags="-s -b 127.0.0.1"

restart syslogd and you’re off to the races. Do not try ‘-s –b [yourhostname.cloudapp.net]’: intuitive as it might seem, the DNS lookup will give you the public IP address for your host, so you will be out of luck.

SSH was more of a pickle. The ListenAddress directive in sshd.conf will accept either an explicit IP address or 0.0.0.0 to listen to every IP on your machine: there is no way use an interface name such as hn0. You can specify a hostname, but then again a DNS lookup for hostname.cloudapp.net will give you the useless public IP address so you would be out of luck.

The cleanest hack I could think of was using /etc/dhclient-exit-hooks to update sshd_config with a new ListenAddress and restart sshd with every IP changes. The first version of the script looked quite brittle for such a delicate task, so I thought I’d leave it for a rainy day and moved on to something simpler based on only two moving parts:

  1. The internal IP will not change as long as the machine is up, so it’s reasonable to expect that adding a ListenAddress directive using the current IP will be quite stable. An invalid IP address will not prevent sshd from starting so I can live with it to take care of my port 22 to begin with.
  2. Belts and suspenders come handy when it comes to logging on your machine , so it occurred to me that I could use PF (yes, I like it better than ipfw) to bind SSH to the loopback address and redirecting incoming traffic from the public interface over there. This way, if the IP address changes, I can still connect using a different port (in the example below it’s 2222).

Simply enough:

add a ListenAddress line to /etc/sshd.conf:
ListenAddress 127.0.0.1

enable pf in /etc/rc.conf:

pf_enable=”YES”

add a redirection rule on /etc/pf.conf
#INTERFACES
ext_if="hn0"
rdr pass on $ext_if inet proto tcp to port 2222 -> 127.0.0.1 port ssh

start the service
service pf start

Restart sshd, cross your fingers, open a new terminal and try to ssh in again. Do NOT, for the love of what’s dearest to you, close the shell you’re in as it could be your only way back if you did something wrong. After testing that this setup works, I have closed the endpoint on the Azure firewall, but reopening it will be a matter of seconds in case things go south.

(For the record, things went south and I got locked out nevertheless. Twice. Stay tuned as I will make copious fun of myself in an upcoming post, together with providing a few survival tips to recover a deployment gone belly-up.)

The final somewhat annoying bit of having a somewhat ephemeral public IP address is that you need to be careful with naked domains. I’m not a fan of naked domains per se, but at the same time I understand there is a general expectation that http://boldlyopen.com works just as well as http://www.boldlyopen.com so I will be going with the flow.

Short of the DNS powers that be finally realizing that it’s 2015 and the limitation on CNAMEs is something they should work on, the only solution is to bite the bullet and use the public IP Azure is giving you. I could have bought a few IP addresses for a very reasonable amount well within my MSDN subscription, butI care about IPv4 space so I didn’t feel my blog should squat one of the few left. Then again, while this IP may change, it is guaranteed to remain the same throughout the lifetime of the deployment and in my case I have no plans of flip-flopping this machine at all. I made a mental note to update the DNS in case I ever start from scratch and I got on with my life.

With these quirks out of the way I was finally able to wade my way into the world of FreeBSD jails: a topic for a follow-up post.

Comments

comments

Leave a Reply

Your email address will not be published. Required fields are marked *