Open Mesh and Ubiquiti Router Network 2

What:

A multi-vlan network using an Ubiquiti router and Open-Mesh WiFi access points.
This network has the following features:

• Dynamic DNS
• Individual DHCP pools for each VLAN.
• L2TP VPN
• NAT / Masquerading
• Transparent Web Proxy with individual rule sets for VLAN6 and VLAN9

Why:

I was approached by a non-profit organization who was in need of a campus wide WiFi network.

Components:

Because of price and features, the below components were used:

• Ubiquiti EdgeRouter Lite
• Open-Mesh MR1750 (2x)
  
• TP-Link 24 port Smart Switch

Firmware:

IMPORTANT: Upgrade the firmware to the latest EdgeOS. 

As of this writing, the latest is 1.7.0.

Interfaces:

I have the router interfaces setup like below:

• Ports:
  • eth0: 192.168.1.1/24
    • DHCP: 192.168.1.201-192.168.254
    
    
  • eth1: no-ip 
  • VLANs
  • 1 - 192.168.2.1/24 - Cloudtrax
  • DHCP: 192.168.2.201-254
  • 3 - 172.16.0.1/23 - OfficeStaff
  • DHCP: 172.16.1.1 - 172.16.1.254
  • 6 - 10.0.0.1/21 - Guests
  • DHCP: 10.0.2.1 - 10.0.7.254
  • 9 - 172.16.8.1/23 - ChildrensArea
  • DHCP: 172.16.9.1-172.16.9.254
  • 12 - 172.16.10.1/23 - TechStaff
  • DHCP: 172.16.11.1-172.16.11.254
  • eth2: DHCP - WAN

NAT/Masquerading:

I ran the below commands to setup masquerading:

set 'service' 'nat' 'rule' '5010' 'outbound-interface' 'eth2'
set 'service' 'nat' 'rule' '5010' 'type' 'masquerade'

I gathered some tips and ideas from the below links:
https://help.ubnt.com/hc/en-us/articles/205197660-EdgeMAX-SOHO-Example
https://community.ubnt.com/t5/EdgeMAX/Basic-SOHO-Home-Config/m-p/453453
https://community.ubnt.com/t5/EdgeMAX/Basic-SOHO-Home-Config/td-p/398055

Web Filter

The GUI has no controls for a Web Proxy Filter, but the feature is present.

I used the below tutorial to get me started.
https://community.ubnt.com/t5/EdgeMAX/Edgerouter-Lite-v1-7-0-modify-proxy-configuration-to-use/m-p/1341469

Here is an example of setting up webproxy for the "Guest" VLAN

Enable transparent proxy for the Guest DHCP pool

set service dhcp-server shared-network-name GuestDHCP subnet 10.0.0.0/21 wpad-url 10.0.0.1

Set the default proxy settings

set service webproxy cache-size 0
set service webproxy default-port 3128
set service webproxy enable-access-log
set service webproxy listen-address 172.16.6.1
set service webproxy listen-address 172.16.8.1
set service webproxy mem-cache-size 32
set service webproxy url-filtering squidguard auto-update update-hour 0
set service webproxy url-filtering squidguard block-category adv
set service webproxy url-filtering squidguard default-action allow

set service webproxy url-filtering squidguard source-group GuestProxy address 10.0.2.1-10.0.7.254 #set source group
set service webproxy url-filtering squidguard redirect-url http://google.com #blocked sites will redirect traffic to google

Filter rules:

set service webproxy url-filtering squidguard rule 20 block-category adv
set service webproxy url-filtering squidguard rule 20 block-category aggressive
...
set service webproxy url-filtering squidguard rule 20 block-category movies
set service webproxy url-filtering squidguard rule 20 block-category porn
set service webproxy url-filtering squidguard rule 20 block-category redirector
set service webproxy url-filtering squidguard rule 20 block-category ringtones
set service webproxy url-filtering squidguard rule 20 block-category spyware
set service webproxy url-filtering squidguard rule 20 block-category violence
set service webproxy url-filtering squidguard rule 20 block-category warez
set service webproxy url-filtering squidguard rule 20 block-category violence
set service webproxy url-filtering squidguard rule 20 local-block-url  youtube.com
set service webproxy url-filtering squidguard rule 20 local-block-url  m.youtube.com
set service webproxy url-filtering squidguard rule 20 local-block-url  www.youtube.com
set service webproxy url-filtering squidguard rule 20 local-block-url  googlevideo.com
set service webproxy url-filtering squidguard rule 20 local-block doubleclick.net
set service webproxy url-filtering squidguard rule 20 default-action allow
set service webproxy url-filtering squidguard rule 20 source-group GuestProxy #assign the ruleset to the source-group

L2TP/VPN

I used the below how-to for guidance on setting up L2TP

https://help.ubnt.com/hc/en-us/articles/204950294-EdgeMAX-L2TP-Server

Dynamic DNS

For Dynamic DNS, I use Duck DNS and I used the below hint to set it up.
http://community.ubnt.com/t5/EdgeMAX/Setting-up-DUCKDNS-on-Edgemax/td-p/1406473

Open Mesh and Ubiquiti Router network

What:

A multi-vlan network with an Ubiquiti router and Open-Mesh WiFi .

Why:

I was approached by a non-profit organization who was in need of a campus wide WiFi network.

Components:

Because of price and features, the below components were used:

• Ubiquiti EdgeRouter Lite
• Open-Mesh MR1750 (2x)
  
• TP-Link 24 port Smart Switch

Procedure:

I setup the WiFI mesh as below:

• SSID1  

• Guests
• Authentication via captive portal. 
• Bridged to VLAN6

• SSID2

• Office Staff
• WPA authentication. Bridged to 
• VLAN3

• SSID3

• Children's Area
• WPA authentication. 
• Bridged to VLAN9.

• SSID4
  

• Tech Staff
• WPA authentication.
  
• Bridged to VLAN12

Open-Mesh devices are controlled from the cloud with the Cloudtrax controller.

NOTE: In order for the Open-Mesh devices to work with VLAN tagging,
untagged traffic must be allowed out to the internet. I did this by setting two switch ports (21,22)
as untagged members of  VLAN1.


I setup the Switch ports as below:

Port 21 - Open-Mesh AP1

• VLAN Tagged with VLANs
  
  • VlAN1 - Cloudtrax (untagged member)
  • VlAN3 - Staff
  • VLAN6 - Guest
  • VLAN9 - Kids
  • VLAN12 - Utility

Port 22 - Open-Mesh AP2

• VLAN Tagged with VLANs
  
  • VlAN1 - Cloudtrax (untagged member)
  • VlAN3 - Staff
  • VLAN6 - Guest
  • VLAN9 - Kids
  • VLAN12 - Utility

Port 23 - to Ubiquiti router

• VLAN Tagged with VLANs
  
  • VlAN1 - Cloudtrax
  • VlAN3 - Staff
  • VLAN6 - Guest
  • VLAN9 - Kids
  • VLAN12 - Utility

Additionally, I also set VLANs (991,992,993,994) on ports 21 and 22 for Seamless Roaming.



Next: Configuring the Ubiquiti router ...

Percentage of Memory Used with Sar and Python for Linux

I needed to get the Percent Memory Used for a Linux systems using sar.
Linux grabs all the memory even though it does not use it all so it is takes a bit more work to figure out how memory is being used.
The formula is:
((kbmemused - kbbuffers - kbcached) / (kbmemfree + kbmemused)) * 100

#!/usr/bin/env python2.7
import re
import subprocess

mcmd = "sar -r -f /xactly/apps/sar/sjcxtlyapp01s/sa20 |grep Average"

sep = re.compile('[\s]+')
sar = subprocess.Popen([mcmd],
    stdin=subprocess.PIPE,
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE,
    shell=True)
string =  sep.split(str(sar.stdout.readlines()))
memfree = "%0.2f" %  (float(string[1]) / 1024)
memused = "%0.2f" %  (float(string[2]) / 1024)
buffers = "%0.2f" %  (float(string[4]) / 1024)
cached =  "%0.2f" % (float(string[5]) / 1024)
percent_used = "%0.2f" %  (((float(memused) - float(buffers) - float(cached))  / (float(memfree) + float(memused))) * 100)
print percent_used

Xubuntu MythTV build

This is my HTPC build using Xubuntu 11.04.

Hardware:
* Case:                  Silverstone ML03B
* Hard Drive:       Corsair Force Series 3 60GB SATA III (SSD)
* Motherboard:    ASUS E35M1-M PRO Fusion AMD E-350 APU (1.6GHz, Dual-Core) Micro ATX
* Memory:           Crucial 4GB (2 x 2GB) 240-Pin DDR3 SDRAM DDR3 1066
* Power Supply:  picoPSU-160 w/100 Watt Brick from http://www.logicsupply.com
* TV Tuner:         pcHDTV tuner from http://pchdtv.com/
* Remote:             Windows MCE remote (mine doesn't require lirc and uses a usb IR dongle)

 My intent was to build a totally fanless build, but after sensing the CPU getting too hot, I gave in and installed the fan that came with the motherboard. Before installing the fan, I did remove the pretty "Asus" label that sat on top of the heatsink for better airflow. Also the ML03 has a mesh opening right above the CPU heatsink.

Difficulties:
   HDMI video out was not working.
      Resolution:

1. Connected the motherboard to a monitor via VGA cable. 
2. Ran apt-get update ; apt-get upgrade  
3. Ran proprietary drivers applet from the menu (jockey-gtk from the CLI), enabled the "AMD/ FGLRX" driver. 
4. Shutdown the computer. 
5. Disconnected the VGA cable. 
6. With the HDMI cable connected to my receiver and the right input selected on my receiver, I turned on the computer, and video out worked fine. 

   HDMI audio out was not working.
(In Ubuntu 12.04 this can be enabled in the audio control panel, without the below steps.)
      Resolution:

1. Ran lspci -l 
   
   

   card 0: Generic [HD-Audio Generic], device 3: HDMI 0 [HDMI 0]
   Subdevices: 0/1
   Subdevice #0: subdevice #0
   

   
   
   Since the HDMI audio device is card 0 and device 3:
   
   
   
   1. Created a previously non-existent /etc/asound.conf
   2. Added the following:
      
      

      pcm.!default {
      type hw
      card 0
      device 3
      }
      

   3. Ran alsamixer -c 0 and pressed m to unmute S/PDIF
   4. Rebooted the computer

I now had both sound and video coming out of the HDMI port.


    MythFrontend would hang while viewing Live TV
      Resolution:

1. on the server side, I set /etc/exports to
   

   /NetworkBackup/MythTV `allowed IP`(async,rw,no_subtree_check)

2. on the client side, I set /etc/fstab to
   

   `NFS Server`:/NetworkBackup/MythTV /MythTV nfs rw,async,nfsvers=3,actimeo=0,tcp,soft  0 0

I used this link for guidance: http://www.mythtv.org/wiki/Optimizing_Performance#NFS_servers

Pictures

Running one function while waiting for another to finish

 import sys
 import threading

 #  
 # this function does a cool spinny thing  
 #  
 def draw_ascii_spinner(delay=0.2):  
  for char in '/-\|': # there should be a backslash in here.  
   sys.stdout.write(char)  
   sys.stdout.flush()  
   time.sleep(delay)  
   sys.stdout.write('\r') # this should be backslash r.  
 #  
 # the below two functions   
 # do the work of backgrounding a process  
 # so we can print a progress indicator  
 #  
 def backGround(func,args,job):  
   job.append(func(*args))  
   
 def whileWeWait(func,arg):  
  job = []  
  t = threading.Thread(target=backGround, args=(func, (arg,), job))  
  t.start()  
  while t.is_alive():  
   draw_ascii_spinner()  
   t.join(0.2)  
   
  return job[0]  

## this is how we run it ..
## assuming we have a function named findCluster that needs a companyid
print whileWeWait(findCluster, companyid)
   

Printing a Grid with Python

I'm learning python and I wanted to print a grid (table) of data on the command line from a list

and here is what I ended up with ...

def printGrid(list):
  if list:
    # the character we use to demarc columns
    splitchar = '|'
    # how much padding 
    # (spaces to add to the data in each cell)
    pad = 1 
    # 2d dictionary with each row of data
    myArray = {}
    # the array that keeps count how wide the columns should be
    myColCount = {}
    # how many rows
    rownum = 0
    # how many columns
    colnum = 0
    # how wide the column
    colwidth = 0 


    # find out how many rows we have
    rownum = len(list)
    
    # find out how many columns we have 
    for cols in list[0].split(splitchar):
      # we also initialize the values of 
      # myColCount for each column and 
      # we set it to 0
      myColCount[colnum] = 0
      # increase the colnum value by 1
      colnum += 1 
    
    # for each row ...
    for r in range(rownum):
      # we populate myArray dictionary
      # with a list created by splitting
      # on a defined character (default is |)
      myArray[r] = list[r].split('|')
    
    #loop through columns
    for c in range(colnum):
      #then loop through rows
      for r in range(rownum):
        length = len(myArray[r][c])

        #if the length of the string is bigger or equal 
        # than what is in the myColCount dictionary ...
        if length >= myColCount[c]:
          # we set the new value to length + padding
          # and that is how we populate the myColCount dictionary
          # with column width values
          myColCount[c] = (length + pad)
          colwidth = myColCount[c]

    # for each row of data ...
    for r in range(rownum):
      # we reset str and border on each iteration of this outer loop
      str = ''
      border = ''
      # for each column ...
      for c in range(colnum):
        #all our column widths are stored in the myColCount dictionary
        colwidth = myColCount[c]
        # we set the border by multiplying dashes by the column width
        border += '+' + "%-*s" % (colwidth,'-'*(colwidth))
        # we set the string with a pipe character and set it 
        # to the width specified by colwidth and we use the - to left align
        str += "|" + "%-*s" % (colwidth, myArray[r][c])

      # we add an ending mark and newline
      border += "+\n"
      # we add an ending mark and newline
      str += "|\n"
      print border
      print str
    # we print the bottom border    
    print border

Animated Gifs with Linux, Gimp, mPlayer

I have learned to create animated GIFs from YouTube clips in Linux and this is how I did it.

With FireFox I went to the desired video on YouTube and played the video I wanted.

Looked for the latest file created in /tmp
 

ls -lrt /tmp/Flash*
/tmp/FlashXXzPQB05

Copied the latest Flash* file created to my homedir

cp /tmp/FlashXXzPQB05 /my/home/dir/SomeFile.flv

I used the following command several times, each time adjusting the start time and stop time until I got the section of the video I wanted

mplayer -ao null -loop 0 -ss 00:00:03 -endpos 8 SomeFile.flv

When I got the section of video I wanted, I then converted the video to a series of JPEGs that got dumped to the AnimGifs directory

mplayer -ao null -ss 00:00:03 -endpos 8 FreshPrince.flv -vo jpeg:outdir=AnimGifs

1. Open the first image with GIMP (I'm using 2.6) 
2. Open the rest of the images by going to File->Open as Layers and selecting all the desired images (except the first one) 
3. Save the image as SomeFile.gif and choose the save as animation option and click export 

If the image is too fast .. run the whole procedure again but save the GIF with 45 to 40 MilliSecond delay.