//
you're reading...
pfSense, SmartOS

pfSense on SmartOS KVM Zone

I just recently got pfSense running on a SmartOS KVM zone.

Here’s what I had to do:

NOTE: On SmartOS you can tab complete the Zone ID (ZID) in most the commands below, so no need to copy/paste it, just remember the first few uniq characters a zone starts with and you can tab complete the hash.

This is somewhat based on how-to-create-smartos-windows-vm

  1. Download pfSense 2.1 Live CD and ungzip it.
  2. Build the ISO
    # On OSX:
    ## Mount the ISO by double clicking on it.
    #copy /Volumes/pfSense to a dir called pfSense, location of the dir is up to you.
    rsync -avz /Volumes/pfSense/ pfSense/
    
    vi pfSense/etc/fstab
    /dev/vtbd0s1a     /          ufs          rw     1     1
    /dev/vtbd0s1b     none     swap       sw     0     0
    
    vi pfSense/boot/loader.conf 
    virtio_load="YES"
    virtio_pci_load="YES"
    if_vtnet_load="YES"
    virtio_balloon_load="YES"
    virtio_blk_load="YES"
    
    ## Create a ISO
    mkisofs -R --no-emul-boot -b boot/cdboot -V pfSense -o pfSense-2.1.vio.iso pfSense
    
    ## Copy to SmartOS server (I use /opt/Iso to store isos)
    scp pfSense-2.1.vio.iso root@smartos:/opt/Iso/
    
  3. Build the Dataset Image
    # On SmartOS
    # We build a dataset image from a KVM zone
    # I assume you have a etherstub called vswitch0
    
    mkdir /opt/DS
    
    vi /opt/DS/dspfsvm.json 
    {
      "alias": "dspfsvm",
      "brand": "kvm",
      "vcpus": 1,
      "autoboot": false,
      "ram": 1024,
      "disks": [
       {
        "boot": true,
        "model": "virtio",
        "size": 4096
       }
      ],
      "nics": [
       {
        "nic_tag": "admin",
        "model": "virtio",
        "ip": "dhcp",
        "allow_ip_spoofing": "1",
        "primary": true
       },
       {
         "nic_tag": "vswitch0",
         "model": "virtio",
        "ip": "dhcp",
         "allow_ip_spoofing": "1"
       }
      ]
    }
    
    
    vmadm create -f /opt/DS/dspfsvm.json
    
    # The Zone ID (ZID) will be shown, you will need it.
    
    # Copy Modified pfSense ISO to ZID
    cp /opt/Iso/pfSense-2.1.vio.iso /zones/<ZID>/root/
    
    # Boot from CD
    vmadm boot <ZID> order=d cdrom=/pfSense-2.1.vio.iso,ide
    
    # Get VNC Port
    vmadm info <ZID> vnc
    
    # VNC to the port number on the SmartOS server
    ## NOTE: For me using Chicken of the VNC, I had to use VNC port 0 and the port# from above as part of the IP (IP:PORT)
    ## putting the port# above as the VNC port didn't work, this is because of the way CotVNC shifts the ports
    ## other VNC clients might work differently.
    
    # At this point pfSense Live CD will be loading, when asked if you want to install hit "I"
    
    # Quick Install didn't work for me, so do the normal install, walk through the install process and when done shutdown the VM using vmadm.
    vmadm stop <ZID>
    
    # Start the Zone again from CD and select rescue mode this time
    vmadm boot <ZID> order=d cdrom=/pfSense-2.1.vio.iso,ide
    
    # If will prompt you to configure your nics, go ahead, mine had the nic name and the MAC mashed together, but the only the name was needed, vtnet0 or vtnet1
    
    # Once booted to the pfSense menu, hit 8 for shell
    
    # Goto /tmp/hdrescue/boot
    
    # Add the virtio module to the loader.conf
    vi loader.conf
    virtio_load="YES"
    virtio_pci_load="YES"
    if_vtnet_load="YES"
    virtio_balloon_load="YES"
    virtio_blk_load="YES"
    
    # Now you can exit, type 6 to halt the system, then start the zone normally
    vmadm start <ZID>
    
    # If everything looks ok, go ahead and halt the zone again from the pfSense menu.
    
  4. Build the image
    # Make sure you changed <ZID> <HID1> <HID2> and <SHA1> and your name as the creator in the following commands.
    # Also do a ls -l against the pfsense gz we made and fill in <SIZE OF GZIP> with the size in bytes
    # mine was 97937278 but yours could be slightly different. Also change the timestamps if you want.
    
    ## Make a Snapshot
    zfs snapshot zones/<ZID>-disk0@image
    
    ## Dump the Snapshot to gzip
    zfs send zones/<ZID>-disk0@image | gzip > /opt/DS/pfSense.zvol.gz
    
    ## Create a couple of Hash IDs (HID) from here http://www.famkruithof.net/uuid/uuidgen
    <HID1>
    <HID2>
    
    ## Get the SHA for the file
    shasum /opt/DS/pfSense.zvol.gz
    <SHA1>
    
    ## Create a dataset manifest
    vi /opt/DS/pfSense-2.1.vio.dsmanifest
    {
      "uuid": "<HID1>",
      "name": "pfSense",
      "version": "2.1.vio",
      "description": "pfSense 2.1 RELEASE with VirtIO drivers enabled",
    
      "os": "freebsd",
      "type": "zvol",
      "platform_type": "freebsd",
      "generate_passwords": "true",
      "users": [
        {"name": "root"}
       ],
      "creator_name": "koaps",
      "creator_uuid": "<HID2>",
      "vendor_uuid": "<HID2>",
    
      "created_at": "2013-18-11T09:00Z",
      "updated_at": "2013-18-11T09:00Z",
      "published_at": "2013-18-11T09:00Z",
    
      "disk_driver": "virtio",
      "nic_driver": "virtio",
      "files": [
       {
        "path": "/opt/DS/pfSense-2.1.vio.zvol.gz",
        "sha1": "<SHA1>",
        "size": <SIZE OF GZIP>,
        "url": "http://localhost/datasets/<HID1>/pfSense-2.1.vio.zvol.gz"
       }
      ],
      "requirements": {
      "networks": [
       {
        "name": "net0",
        "description": "admin"
       },
       {
        "name": "net1",
        "description": "vswitch0"
       }
      ]
     }
    }
    
    ## Install the image
    imgadm install -m /opt/DS/pfSense-2.1.vio.dsmanifest -f /opt/DS/pfSense-2.1.vio.zvol.gz
    
    ## Verify image is available
    imgadm list
    
  5. Create a pfSense KVM Zone
    mkdir /opt/VM
    
    vi /opt/VM/pfsvm.json
    {
      "alias": "pfsvm",
      "brand": "kvm",
      "vcpus": 1,
      "ram": 1024,
      "disks": [
        {
          "image_uuid": "<HID1>",
          "boot": true,
          "model": "virtio",
          "image_size": 4096
        }
      ],
      "nics": [
        {
          "nic_tag": "admin",
          "model": "virtio",
          "ip": "<PUBLIC IP>",
          "netmask": "<PUBLIC MASK>",
          "gateway": "<PUBLIC GATEWAY>",
          "allow_ip_spoofing": "1",
          "primary": "1"
        },
        {
          "nic_tag": "vswitch0",
          "model": "virtio",
          "ip": "<PRIVATE IP>",
          "netmask": "<PRIVATE MASK>",
          "gateway": "<PRIVATE GATEWAY>",
          "allow_ip_spoofing": "1"
        }
      ],
      "resolvers": [
        "8.8.8.8",
        "8.8.4.4"
      ]
    }
    
    vmadm create -f /opt/VM/pfsvm.json
    

That should be it, you should now have a running pfSense firewall on SmartOS.

This will work good for zones on the same SmartOS server, but if you want this to be a firewall for a bunch of SmartOS servers, you will most likely want to use VLAN’s and while that’s not super hard, it does require changing a few things, like you probably wouldn’t use a etherstub for the LAN.

Advertisements

Discussion

6 thoughts on “pfSense on SmartOS KVM Zone

  1. Hello, Thank you for pretty detailed instructions.

    I just installed and started using smartos at home. I’ve trying to install pfsense in kvm. I’m stuck at “3. Build the Dataset Image”, particularly when the pfsense is booting. It stuck at “loading /boot/default/loader.conf” for more than an hour.

    I tried several times to make iso and check the fstab, loader.conf files. Still I could not see any issue.

    Do you have any suggestions/pointers to solve my issue? Appreciate your response.

    Balaji

    Posted by Balaji | October 11, 2014, 8:05 pm
    • Hi Balaji,

      Sorry I didn’t get back to you sooner, been crazy busy with lots of projects.

      What version of the ISO are booting from? TBH I started using a zone running ipf as a firewall for some of my projects because the overhead was less and it was easier to just zlogin and make changes.

      I can try creating a image from the newest pfSense, see if I run into the booting issue.

      Posted by koaps | November 24, 2014, 2:00 pm
      • Hi, Thanks for the reply. In my setup, I want to connect WAN to my SmartOS server Nic1 and Wireless router to Nic2 (as LAN interface). Hence I can not use Zone for NATting my PC, mobile, tablet etc.

        So I want to connect my wireless router to LAN side of SmartOS server (Nic2) which will serve DHCP for my clients including router. I want to dedicate WAN from my modem to another Nic (Nic1) in SmartOS server. So all my WAN traffic will be routed through SmartOS pfSense KVM.

        Currently my pfSense installation is working except with following issues.

        1. DHCP server in pfSense does not work. But I do not have any etherstub configured for LAN side of Nic (Nic2). Is this must?

        2. Also I could not ping from pfSense interface to WAN. Again I’m not using any etherstub for this interface.

        Any help would be appreciated.

        Posted by Balaji S | November 25, 2014, 5:58 am
  2. So this is fairly similar to my home setup, except I have one thing in my mix you don’t.

    I have an old Linksys WRT54g that I put DD-WRT on that I use as a VLAN switch.

    So my DSL ( which I have static IPs for ) comes in and goes from the modem to my WRT54g. The “WAN” interface is vlan tagged ( vlan 3 ) and I have a couple of other interfaces vlan tagged ( vlan 2 ) for my wireless AP and my NAS device.

    My SmartOS server is then plugged into a port on the WRT54g that is a trunk port, and the wan and lan interfaces on my server are configured to use vlan_ids

    “Network Interfaces”: {
    “e1000g0”: {“MAC Address”: “aa:bb:cc:dd:ee:ff”, “ip4addr”: “192.168.1.254”, “Link Status”: “up”, “NIC Names”: [“wan”, “lan”, “admin”]}
    },
    “Virtual Network Interfaces”: {
    “wan0”: {“MAC Address”: “01:02:03:04:05:06”, “ip4addr”: “1.2.3.4”, “Link Status”: “up”, “Host Interface”: “e1000g0”, “VLAN”: “3”},
    “lan0”: {“MAC Address”: “07:08:09:10:11:12”, “ip4addr”: “192.168.1.1”, “Link Status”: “up”, “Host Interface”: “e1000g0”, “VLAN”: “2”}
    },

    Where 1.2.3.4 is my WAN IP, 192.168.1.1 is my LAN gateway and 192.168.1.254 is the admin interface for SmartOS.

    Both the WAN IP and the LAN gateway IP are on my IPF firewall. My Wireless clients don’t go out the IPF firewall, I have a public IP assigned to my Wireless AP, but the LAN subnet is accessible from the wireless LAN since they are all in the same VLAN.

    This works great for me, when I’m home I can ssh to various servers with no firewall in the mix, but externally my servers are behind IPF.

    If you don’t have multiple static IP’s, then what I would do is a simular setup except the DHCP on the Wireless AP would set the LAN gateway to the LAN IP of the IPF server.

    As far as your current setup, make sure you put in the “allow_ip_spoofing”: “1” for DHCP to work.

    Etherstubs aren’t needed, I use one for a internal LAN in my SmartOS server. So for instance my webserver on SmartOS, has two interfaces, one is nic_tag “vswitch0”, which is the LAN for my IPF firewall, and is the primary interface for the webserver, so outbound traffic goes through my IPF firewall. The second interface is vlan_id 2 with nic_tag “lan” which puts it on the LAN of my wireless network, so I can ssh to my webserver using a 192.168.x address.

    This allows me to see both sides of my webserver, so I can debug against 192.168.x and test against it’s external WAN address from my wireless LAN.

    For ping to work on pfSense you need to make sure the ICMP is allowed against the WAN0 interface. I think there’s a check box option you can set in the interface setup area, if not you can just create a firewall rule to allow ICMP any to wan0 interface, but you might want to lock that down later on.

    Same goes for WebUI if you want it externally accessible though that isn’t recommended.

    This is why I ended up switching to IPF, so I could modify the firewall rules externally, I use SSH with no password prompts, host keys only, makes it a pain if I don’t have my laptop with me, but i rarely make changes to my home firewalls.

    Let me know if you’re still having issues, I can email you directly and I could help you check your setup, if we can’t get anywhere, we can just jump into IRC, the smartos channel is pretty good with helping out, I’ve asked a few things there before.

    Posted by koaps | November 25, 2014, 11:31 am
  3. Hello, Thank you for the information!! Its very useful. I wonder if it is possible to set the wireless ap behind the pfsense vm and the pfsense vm as the main gateway and dhcp server?

    Posted by ManJack | December 4, 2014, 8:34 am

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s