Cisco ASA Site to Site VPN with dynamic IP addresses

Today’s problem is a new customer office opening ahead of their scheduled MPLS installation. We need to connect them back into their VPN via their existing hosted Cisco ASA. The internet connection at the new office is at this point unknown, could be a 4G dongle, could be a satellite or even a DSL connection – or a combination of the three! We have a spare ASA and we are going to create a site to site VPN, despite the fact that the new office IP is unknown or possibly dynamic.

Cisco provide a special kind of crypto map for this challenge called a dynamic crypto map and a special tunnel-group called ‘DefaultL2LGroup’ which catches L2L runnels where the peer IP address cannot be matched. There are ways of ‘steering’ dynamic L2L peers into different tunnel-groups but we only need to use the basics here.

The configuration on the ‘spoke’ end (the one with the dynamic/unknown IP address) is just a standard L2L IPSec tunnel, so we just need the Hub (Fixed IP) end:

Step 1 – Define the interesting traffic (for the NAT Exemption)

object network LOCAL_LAN
 subnet <local LAN ip range>

object network REMOTE_LAN
 subnet <remote LAN ip range>

Step 2 – Configure the NAT exemption

nat (INSIDE,OUTSIDE) source static LOCAL_LAN LOCAL_LAN destination static REMOTE_LAN REMOTE_LAN

Step 3 – Configure the phase 1 setup

crypto ikev1 enable outside
crypto ikev1 policy 10
 authentication pre-share
 encryption aes
 hash sha
 group 2
 lifetime 86400

Step 4 – Configure the phase 2 setup

crypto ipsec ikev1 transform-set MY-SET esp-aes esp-sha-hmac

Step 5 – Configure the tunnel-group

tunnel-group DefaultL2LGroup ipsec-attributes
 ikev1 pre-shared-key <secret_key>

Step 6 – Configure the Crypto maps

crypto dynamic-map DYNOVPN 10 set ikev1 transform-set MY-SET
crypto dynamic-map DYNOVPN 10 set reverse-route

crypto map VPNMAP 999 ipsec-isakmp dynamic DYNOVPN
crypto map VPNMAP interface outside
Posted in Cisco | Tagged , , , | Comments Off on Cisco ASA Site to Site VPN with dynamic IP addresses

Basic WebVPN setup on the Cisco ASA 9.x

We have resisted the change for a long time, bit its time to finally move some of our customers over to the SSL VPN who were previously using the IPSec Remote Access VPN. Windows 10 does not support the IPSec client any more, Cisco have stopped developing it and its only saving grace is that Mac seem to have no problem with the built in VPN connector.

We are moving some clients to the ASAv which I will document the installation of another time, but the software version I am using is 9.6(1).


1. Most users will be standard, tunnel-all users
2. A few users will require local LAN access for IP printers etc. These will be kept to a minimum as they pose a security risk
3. The Anyconnect software should be deployed from the ASA.
4. The users will all be stored in the ASA local database.

Stage 1 – Get a 3rd party certificate

I have a previous post on this which is still valid. I used a RapidSSL from Geotrust. The latest client has a ‘checked’ check box to disable non trusted certificates by default and could cause a lot of pain for the support guys – so do this first! make sure the time is set as per the article.

Stage 2 – Create an IP pool for the remote users

I favour using a completely separate IP range, not used anywhere else on the internal network. This saves a lot of faff with adding routes later.

ip local pool VPN-POOL

Stage 3 – Sort the NAT out

I ran into a world of pain when i did this first as the ASA started responding to ARP requests from anything on its OUTSIDE subnet. The take-home message is that avoid using ‘any’ in your NAT setup. So we want to define the POOL as an object and use that to get the NAT exemption for data leaving our ‘INSIDE’ network to the ‘OUTSIDE’ network via the VPN tunnel. Also we want traffic coming back from the client, not destined for the INSIDE network to be NATted to the internet.

object network VPN-IP-POOL
nat (OUTSIDE,OUTSIDE) dynamic interface dns

Now the NAT exemption for the INSIDE to OUTSIDE traffic. I assume there is already a LAN object defined.

nat (INSIDE,OUTSIDE) source static LAN LAN destination static VPN-IP-POOL VPN-IP-POOL

Also we’ll need to allow the OUTSIDE traffic to hairpin on the interface.

same-security-traffic permit intra-interface

Stage 4 – Add the webvpn config

Here we need to upload the pkg files which can be downloaded from into the flash of the ASA. they are then referenced in the config.

enable OUTSIDE
anyconnect image disk0:/anyconnect-win-4.2.05015-k9.pkg 1
anyconnect image disk0:/anyconnect-macosx-i386-4.2.05015-k9.pkg 2
anyconnect image disk0:/anyconnect-linux-64-4.2.05015-k9.pkg 3
anyconnect enable
tunnel-group-list enable

Note the pkg references have an index number to permit multiple files to be uploaded.

Stage 5 – Group Policy

We’ll create a Group Policy to set the parameters for the users. Its best to create a new policy rather than edit the default. This is our ‘tunnel-all’ policy which will be referenced by the tunnel group as the default policy.

group-policy CUSTOMER-POLICY internal
group-policy CUSTOMER-POLICY attributes
dns-server value
vpn-tunnel-protocol ssl-client
split-tunnel-policy tunnelall

Stage 6 – The Tunnel Group

Here a tunnel group is created which pulls it all together

tunnel-group CUSTOMER type remote-access
tunnel-group CUSTOMER general-attributes
address-pool VPN-POOL
default-group-policy CUSTOMER-POLICY
tunnel-group CUSTOMER webvpn-attributes
group-alias CUSTOMER-LOGIN enable

The group aliases appear in the dropdown when the user logs in.

Stage 7 – The Users

The users are all using the default group policy of ‘CUSTOMER-POLICY’ unless we specify differently.

username user_name password pass_word
username user_name attributes
group-lock value CUSTOMER
service-type remote-access

I’ve also locked the user into the correct group to be secure.

This is enough to get up and running – there is loads more to do with customisation, additional security and the like, but for now the customer needs to get online.

Posted in Cisco | Tagged , , , | Comments Off on Basic WebVPN setup on the Cisco ASA 9.x

Upgrading MySQL on CentOS 6

I’ve been migrating a web site from a community web server to a dedicated 30:30 server running CentOS, Apache, MySQL and PHP. The issue is that CentOS stopped updating MySQL at version 5.1 in the CentOS 6 track and the ‘source’ database is version 5.5. After the first MySQLdump would not import into the 5.1 version I decided to update the 30:30 server to a more modern version of MySQL – 5.7!

Firstly, backup the databases using mysqldump.

Stop the server.

# service mysqld stop

Remove the old software.

# yum remove mysql mysql-server

At this point I found it was a good idea to completely remove the mysql directory.

# cd /etc/lib/
# rm -f mysql/

Fetch the new software.

# wget

Install the code.

# yum localinstall mysql57-community-release-el6-7.noarch.rpm

Install the new software.

yum install mysql mysql-server

Start the service.

# service mysqld start

Now here’s the next snag – normally I’d run the secure install programme, which I did, but I could not get in with the usual blank password, which I thought was odd! Also the old root password failed the password strength requirement MySQL had assigned itself! I wanted to maintain the old password for the time being so that issue needed resolving.

First things first, MySQL 5.7 assigns itself a temporary root password which is logs in the MySQL log file. You will need to find this.

# grep "temporary password" /var/log/mysqld.log

Now you can run:

# mysql_secure_installation

Lastly, if you want to remove the password enforcement policy then add this to your .cnf file


So that’s it for this task, now I can get on with the site migration.

Posted in CentOS, MySQL | Leave a comment

Retro fitting httpd-itk to a CentOS Apache setup

I have a few CentOS servers which despite being dedicated server are being managed by web developers who want to do the whole thing with an FTP client. This is fine, however I needed to install and configure httpd-itk so that the whole apache process is taken care of by the FTP user in question.

1. Install wget

 yum install wget 

2. Install epel repositary

rpm -ivh epel-release-6-8.noarch.rpm

3. Install httpd-itk

yum install httpd-itk

4. Edit /etc/sysconfig/httpd and add


5. Set the default itk user by editing /etc/httpd/conf.d/itk.conf

AssignUserId apache apache

6. Modify /etc/httpd/conf.d/php.conf to accommodate mod_php

<IfModule itk.c>
   LoadModule php5_module modules/

7. Reset all the file ownerships on the web space

chown -R ftp_user:ftp_user /var/www/home_directory/*

8. Specify the ftp user inside the VirtualHost block

AssignUserId ftp_user ftp_user

9. Restart the Apache daemon

service httpd restart

All done!

Posted in Apache, CentOS | Tagged , | Leave a comment

Cisco periodic reboot using perl

A request to reboot a Cisco ASA every week came in today. Under Cisco IOS there is a kron which would accommodate us here, however there appears to be no such thing under the ASA OS (is it still called FOS?). The next best thing is to run a perl script which utilises the Net::SSH::Expect module to negotiate the SSH login.

As we are using an ASA without any kind of AAA usernames we also need to get past uplifting our privileges using en.

I’m on a FreeBSD server so i need to install the correct perl module via the ports first:

# cd /usr/ports/net/p5-Net-SSH-Expect/
# make install

Now we can write and test the script:

#! /usr/bin/perl -w

use strict;
use Net::SSH::Expect;

my $host_ip = "";
my $login_name = "xxxx";
my $login_password = "yyyy";
my $en_password = "zzzz";

my $ssh = Net::SSH::Expect->new (
            host => $host_ip,
            password=> $login_password,
            user => $login_name,
            raw_pty => 1

my $login_output = $ssh->login();
if ($login_output !~ /Type help/) {
        die "Login has failed. Login output was $login_output";

$ssh->waitfor('Password:', 1) or die "prompt 'Password:' not found after 1 second";
$ssh->exec("reload noconfirm");

Add the script to the crontab and we’re away!

Posted in Cisco, Perl | Tagged , | Leave a comment

Using apachetop with a Community Server

Having just installed apachetop on a community server to get an overview of what sites are busy on the server, I needed to create a single log file to read. Adding a line into each Apache stanza was simple but tiresome, I simply added this line:

        CustomLog /data/logs/all.log combined

This gave me a central log file which has all of the sites in it, then I can call the apachetop program with he following format:

apachetop -f /data/logs/all.log

This worked really well with one problem. Th server has a lot of sites running WordPress and the like and apachetop was not differentiating in its output. The solution was to use Apache and create a new logging format. I wanted to shim the ‘ServerName’ into the request so I could see which site’s wp-login.php (for example) page it actually was showing. My new format is:

    LogFormat "%h %l %u %t \"%m %v%U%q %H\" %>s %B" central

Its called ‘central’ so I needed to update the apache config accordingly which sed took care of it quickly and painlessly. Now I get output like:

  304 10.13 879.0 29.3*
   22  0.81   0.0  0.0
    4  0.13 222.5  7.4
    3  0.12  15.5  0.6
    2  0.07 112.9  3.8
    1  0.03   0.7  0.0
Posted in Apache | Tagged , | Leave a comment

VMware VDP Appliances – they don’t write, they don’t call

It seems that since the summertime changeover, none of our VDP appliances have been reporting home with an email each day. I let this go for a bit as I was busy with other things, but after a while you start to wonder… After first running some basic diagnostics like:

  1. check spam filters – nothing in spam bin
  2. check mail relay servers- nothing in logs
  3. Perform a test email from the control panel – all good
  4. check logs in control panel and found:
Send email summary report error: The email summary report timer is set to the incorrect hour. It does not match the vdrdb value.

A bit of Googling suggests that the email function needs resetting. Our reports are set to come out at 0800 so the clock is clearly skewy. I logged in and clicked the Edit button and then re-saved the settings without making any changes.

I’ll update the post when i can confirm whether this worked or not.

==== UPDATE ====

OK, 0800 has been and gone and I have all the expected email reports in my inbox. I’ll chalk that up as a success and I look forward to the next time change back to GMT!

Posted in VMware | Tagged , , | Leave a comment

MySQL ZFS snapshot restoration

Since yesterdays test on ZFS snapshots, we now have 25 snapshots in the snapshot archive which can be seen below.

# cd /DATA/mysql/.zfs/snapshot/
# ls -lah
total 36
dr-xr-xr-x  21 root   wheel    21B Oct 16 21:00 .
dr-xr-xr-x   4 root   wheel     4B Sep 24 20:23 ..
drwxr-xr-x   5 mysql  mysql    16B Oct 15 21:50 2013101522
drwxr-xr-x   5 mysql  mysql    16B Oct 15 21:50 2013101523
== snip ==
drwxr-xr-x   5 mysql  mysql    16B Oct 15 21:50 2013101620
drwxr-xr-x   5 mysql  mysql    16B Oct 15 21:50 2013101621

These were generated by the ‘cronned’ script which is making the hourly snapshot and naming it accordingly. The next test is to perform a realistic restoration of a specific record. Its unlikely that we would just want to roll back to a previous time, but more likely a bit of data has been lost and we need it back to re-insert into the current data. The solution proposed here is to copy and rename the archived database and then do the data transfer to the live database.

In this test I chose the snapshot from 11:00 today and am restoring the ‘extranet’ database to a database called ‘restored’.

# cd 2013101611
# cp -R extranet/ /DATA/mysql/restored
# chown -R mysql /DATA/mysql/restored/

Now we should be able to open the database up and see the results:

mysql> show databases;
| Database           |
| information_schema |
| extranet           |
| mysql              |
| restored           |
| test               |

mysql> use restored;
Database changed
mysql> show tables;
| Tables_in_restored    |
| table1                |
| table2                |
== snip ==
nn rows in set (0.00 sec)

All looks fine, but the search goes on….

Posted in FreeBSD Administration, MySQL, ZFS | Tagged , , , | Leave a comment

Using ZFS snapshots to keep MySQL backups

Whilst at a customer site the other day the question was raised over how they could keep a fair volume of backups of a MySQL database off site, while minimising bandwidth and disk space. So the usual ‘moon on a stick’ enquiry. My thought was to use MySQL replication combined with ZFS snapshotting. This is my test setup to ensure it all works nicely.

Stage 1 – Replication server setup

I’m using FreeBSD 9.1 (how retro!) with a UFS boot drive and a second ZFS data drive. I created the main /DATA zpool and a sub directory (dataset) called mysql:

# zfs list
DATA        34.9M  19.5G    32K  /DATA
DATA/mysql  34.7M  19.5G  32.1M  /DATA/mysql

Stage 2 – MySQL setup

Now we must install MySQL server and ensure the data is stored in the non default directory of /DATA/mysql. To do this, add the following line to the /etc/rc.conf file:


Stage 3 – MySQL Replication

The next stage is to get the data replicating into the database replication server. I use the standard procedure documented on the MySQL website and ensure I have a dedicated ‘Replication’ user. When its all done, complete the ‘show slave status’ to ensure its all running correctly:

show slave status\G
Slave_IO_Running: Yes
Slave_SQL_Running: Yes

Stage 4 – Set a cron task to snapshot each hour

I wrote a little script to take the snapshot and set it to run every hour:

#! /bin/sh
DATESTRING=`date +'%Y%m%d%H'`
zfs snapshot DATA/mysql@$DATESTRING

The snapshots can be seen in ‘very’ hidden directory:

ls -lah /DATA/mysql/.zfs/snapshot/
total 2
dr-xr-xr-x  3 root   wheel     3B Oct 15 22:25 .
dr-xr-xr-x  4 root   wheel     4B Sep 24 20:23 ..
drwxr-xr-x  5 mysql  mysql    16B Oct 15 21:50 2013101522

So far, thats it! I’ll let it build for while and then try using de-dupe and compression to see what works best.

Posted in FreeBSD Administration, MySQL, ZFS | Tagged , , , | Leave a comment

Converting Cisco 877 from ADSL to FTTC

Normally when we deploy FTTC to a client in an MPLS VPN we use the Cisco 887 and bypass the BT installed VDSL modem. On this occasion the client is already fully kitted out with Cisco 877 routers and is not minded to replace them all straight away so we need to check out the config needed. In short we need to stop using the ATM interface and hook up an ethernet interface into the VDSL modem supplied by BT.

Stage 1 – Disable the ATM port

int ATM0

Stage 2 – Create the new vlan and and assign to a switch port

vlan 100

int E3
 switchport access vlan 100

Stage 3 – Create the BVI and enable PPPoE

interface Vlan100
 description *** FTTC ***
 no ip address
 pppoe enable group local
 pppoe-client dial-pool-number 1

Stage 4 – Enable VPDN

vpdn enable
vpdn-group 1

Stage 5 – Set up the dialer interface

interface Dialer0
 mtu 1492
 ip address negotiated
 ip virtual-reassembly
 encapsulation ppp
 dialer pool 1
 ppp chap hostname XXXXXXX
 ppp chap password XXXXXXX

Stage 6 – Add a default route

ip route Dialer0

Thats the lot, it should all work now – Good Luck!

Posted in Cisco | Tagged , , | Leave a comment