Telenet Networks

China Mountain Rescue

2011-02-06T13:19:00+08:00

On the eve of December 23 2010 a good friend and I went hiking in the mountains in Northwest China. I had been planning this trip for over two months and after speaking to a few locals (both 中国人 and expats) no-one was willing to come hiking with me in winter.

The average temparature for december is about -12 degrees celcius so I wasn’t at all surprised at the lack of enthusiasm.

The first hurdle was that I didn’t have a tent or sleeping bag but I focused most effort on finding a sleeping bag as I am 6”7 and I don’t fit into most bags. Much to the amusement of my wife and friends I spent about 2 months searching the web and the local camping stores (which were very few and far inbetween) for a winter down bag.

Eventually I purchased a Mountain Hardware sleeping bag from Paddy Palins in Australia (where Kevin lives) and he brought it across to China for me. Unfortunately I had to pay a premium for the bag because I couldn’t a 7” bag anywhere else on the planet!

Anyway Kevin arrived in China late December and we got some last odds and ends as we prepared to head into the mountains. The elevation here is about 1100 meters and at the top of the mountains it is about 3000 meters.

I had climbed up various parts of the before in preparation for the ‘big’ hikeand wanted to be as conservative as possible considering the risk of freezing / injury, etc. Therefore in theory I was prepared to camp at ‘basecamp’ even though it was pretty low and boring BUT I was also open to ascending further depending on conditions.

As we were entering the park the cashier at the ticket office asked us if we were going to stay overnight - I told her that we would stay and mentioned that we had prepared well, or at least so I thought. The words “我们准备了好!” now ring in my mind.

We started the climb and got to basecamp within an hour or so and had a break - basecamp is about 1600m elevation. Kevin had a little trouble with his breathing on the way up but it subsided after we got to basecamp.

We decided to ascend higher and headed westward to the next peak. In retrospect it would probably have been better to stay at basecamp… To be continued!

Securing Your Splunk

2011-01-15T18:30:00+08:00

Splunk is a great tool to monitor systems and networks. Although it does take while to learn how to use Splunk properly (things I have learnt so far is extracting fields, NOT running splunk as root, configuring how long splunk keeps data for, etc) it is worth the effort required to become proficient using this tool.

Over the last few days I have been working on migrating away from the built-in CA (It should have been done ages ago!) to my own CA and enforcing mutual authentication of both the indexer and forwarder. Below are the steps necessary to achieve the above mentioned:

On the Splunk indexer

Make another copy of /opt/splunk/bin/genRootCA.sh, substituting appropriate values for organization, location, etc. Delete all the old certificates and keys. I also deleted the stuff in /opt/splunk/etc/auth/audit because I don’t trust those certs either. Run the script and follow the prompts. Keep the files safe - they are your new CA! For each forwarder create / sign a certificate *Edit /opt/splunk/etc/system/local/inputs.conf and add the following

requireClientCert=true

This will force the indexer to only accept connections from forwarders using certificates signed by your CA!

On the Splunk forwarder

Copy the cacert.pem and the appropriate certificate from the indexer to the forwarder. Edit the /opt/splunk/etc/system/local/outputs.conf and add the following options :

sslVerifyServerCert=true
      sslCommonNameToCheck=indexer.example.com
      altCommonNameToCheck=indexer

These options will force the forwarder to check that the indexer it connects to is using a cert that was signed by your CA and that the common name of the cert is as specified. Your Splunk installation will now be in a much better position to keep the bad guys out!

There is a long version of this here from answers.splunk.com.

Wikid Token & OpenDNS

2010-12-09T20:44:00+08:00

There is a lot to like about OpenDNS, my favourite features being protection from malware and being able to effectively and easily control what content my users / devices can access simply via DNS.

One of the things I dislike is they return their own IP (67.215.65.132) or hit-nxdomain.opendns.com. if the host or record you are looking up doesn’t exist. In this particular case using OpenDNS prevents Wikid 2-factor authentication tokens (only iPhone tokens) from being able to obtain a one time passcode.

Over the past year I have been unable to use my iPhone Wikid token when using wifi, which was quite odd. In October I switched to China Unicom as my mobile / 3G provider, since China Mobile’s 3G doesn’t work and ever since that I haven’t been able to use the token on wifi or 3G, which was making my work impossible!

If you attempt to use the token client on an iPhone that is using Opendns as a resolver you will get an error :

Error Unable to fetch passcode

I ran a packet capture of the iPhone talking to the wikid server and discovered that the iPhone token client uses DNS to resolve the IP of the Wikid Server (IP addresses obfuscated) :

proto: UDP (17), length: 75) xxx.xx.88.126.59389 > 208.67.222.222.53: 
      [udp sum ok]  52734+ A? 146175076181.wikidsystems.net. (47)

So looking at the capture above the iPhone token looks for a 146175076181.wikidsystems.net. host which doesn’t exist and Opendns returns a bogus IP, 67.215.65.132 which the token then tries to get a OTP from and fails!

Digging a little deeper I found 888888888888.wikidsystems.net. does exist and is an alias for beta.wikidsystems.com. I summize that this a beta feature that only found it’s way into the iPhone client (fortunately).

This is a disappointing design decision because :

It is intimated that the Wikid domain, which is a zero padded representation of the public IP of the Wikid server would preclude the need for any reliance on DNS, which is inherently insecure.
Secondly it introduces another dependency or level of complexity and in a security context this is always a bad idea.
The benefit of mutual authentication is weakend if a denial of service can performed simply by forging a dns reply

I hope that this can be fixed in the next revision of the iPhone token client.

Great Firewall Slowdown

2010-11-04T22:21:00+08:00

Since moving to China I have noticed more and more sites referencing twitter.com or facebook.com, which is good and all except if you live in China (where both of those services are blocked).

This probably isn’t a problem for the majority of sites out there but if you have an audience in China then you would certainly be concerned about adding 30+ seconds to your page load time (for the TCP connection to terminate - not sure if this timing is accurate).

It would be cool if Pingdom Pageload Tools could give an indication of load times when accessing a site from a country that blocks major services to give customers an indication of the time penalty when adding twitter and facebook connect widgets.

As more and more sites connect to twitter and facebook is this going to be the Achillies heal of the “Great Firewall?”

Authentication in the Cloud

2010-11-04T19:32:00+08:00

During my n00b days I thought ssh-keys were the bees knees; they seemed more ‘advanced’ than passwords because you had a file placed in a hidden (.) directory and the file had this gobble-dee-goop in it that somehow allowed me to get into my linux boxes securely.

Back then it was even cooler to use linux ‘cause all my colleagues were stoked about Windows NT 4.0, which then was a newcomer to the whole multi-user scene andwasn’t very stable nor secure. I eventually started to use a passphrase for my ssh-key as I figured out if someone hacked my Windows 95 PC that putty-agent would to all to keen to hand out my beloved ssh-key without a passphrase, and even with a passphrase if they got deep enough.

I now use two-factor authentication in my cloud based services. I am yet undecided as to whether it is actually more secure since my servers are running on a system on which I don’t have any physical control of the system or network.

I am holding out for Cohesive FT to bring out a t1.micro version of their VPN-Cubed appliance so that I don’t have to add another $35 per month just for a network concentrator.

One of my reservations is that I like to have a consistent authentication mechanism (ie: 2fa) for interactive authentication and it wouldn’t suit the use case of the VPN-Cubed appliance. Therefore the weakest link from a security persepective would be the VPN-Cubed appliance and ultimately would it then even be worth using 2fa if access to a critical system could be gained without 2fa authentication?

Generating A Certificate Signing Request With Openssl

2010-05-15T00:00:00+08:00

I have an apache server that hosts two sites requiring SSL; in order to generate a second certificate you need to use the existing servers’ private key and hence don’t need to generate a second one (I guess you could but it won’t provide additional security per se). If you want to start from scratch, that is without any existing SSL cert, Ubuntu has a good tutorial at Ubuntu server guide.

I used the following command :

‘openssl req -new -key /etc/ssl/private/server.key -out /etc/ssl/certs/newssl.csr’

Openssl will then jump into interactive mode and ask you a few questions about the site you want to host and it will then create the certificate signing request file; newssl.csr. You can then paste this request file into your CA’s site and order or process the certificate!

O’Reilly has a good book on OpenSSL if you want to go pro!

Openvpn And Openssl Certificate Types

2010-04-11T00:00:00+08:00

If you want Openvpn client to verify that the server or vpn endpoint that it is actually connecting to is the server that you have in your config then you will need to perform one of the steps in the howto, however if you want to use the ‘remote-cert-tls server’ method you will have to had planned this before putting the server into production, or you will have to generate a new certificate for the server.

If you want to check what attributes your server certificate has you can use the OpenSSL command :

openssl x509 -in server.crt -text

It will then spit out info about your OpenVPN server certificate which you can use to determine if your certificate was built with the nsCertType=server attribute. Look for something like this :

Netscape Cert Type: SSL Server

If it doesn’t have that or :

X509v3 Extended Key Usage: TLS Web Server Authentication

Then you will have to use a different method or re-generate the certificate.

Quick And Dirty Sshd Radius

2010-03-04T00:00:00+08:00

This was tested using Ubuntu 8.04.

aptitude install libpam-radius-auth

sudo vi /etc/pam_radius_auth.conf

delete the spurious 127.0.0.1 line (unless your radius server is @localhost)

add your radius server IP address, shared secret and timeout value

sudo vi /etc/pam.d/sshd and add ‘auth sufficient /lib/security/pam_radius_auth.so’ at the top of the config.

also make sure you have set :

‘PasswordAuthentication yes’ and ‘PubkeyAuthentication no’ in /etc/ssh/sshd_config

then run :

‘sudo /etc/init.d/ssh restart’

If you are using Firewall Builder you will need to add your own custom radius service as the default radius service object in fwbuilder is on UDP/1645 instead of UDP/1812 - that little lesson lost me 1.5hrs!

Eventid 4515

2009-01-31T00:00:00+08:00

I have fixed my first event 4515 problem duplicate dns zone problem. This came about when adding another Active Directory site to my client’s multi-site network.

When I added the site the connectivity between the new site wasn’t that greatand therefore I did not see the DNS records immediately and I panicked! Thereafter I re-installed the DNS service and must have set the option to replicate DNS records to all DNS servers instead of to all domain controllers in the domain, thereby overwriting (or duplicating) the existing zones!

This can be fixed by deleting the duplicate zone as per the instructions on technet above. It is however quite scary deleting things using Adsiedit as the potential for disaster is high if the wrong partition is deleted. Even though I was using Windows Server 2003 R2, Active Directory was using the Windows 2000 compatible partition scheme (replicate dns info to all domain controllers - Option 3).

Once I worked out the sequence of events I knew which partition to delete and restarted the dns service and ever since then event 4515 has disappeared!

Pki And The Poor Helpdesk Lady

2009-01-29T00:00:00+08:00

For security reasons I store my ATO Business portal certificate on a Rainbow ikey 3000, so when I attempted to renew my certificate ATO’s website gave me a ‘certificate is not a valid production Tax office certificate’ error.

I called ATO tech support and the helpdesk lady gets me to go through the usual gauntlet of help desk blind-led questions:

What operating system are you using? “Windows XP”, I replied (I didn’t want to tell her I was running it as a virtual machine !) “Home or Professional” she asked. “Um..Professional, ahem Professional”
What is your Service pack level? “3”, I said.
Are you using a network? “Um…Yes. Just a home network”
What firewall are you using sir? “Monowall”

I told her the error message I was getting and she instructs me to go into the CSI certificate management utility. We do the shuffle again…Start Menu…All Programs - CSI blah blah.

I tell her that my cert is stored on the smartcard and not the usual CSI store but she responds that the cert isn’t actually on the smartcard but is rather on the computer and then gets me to deselect the Microsoft CAPI store (aka on the smartcard, where my cert actually resides).

Naturally I enthusiastically obey and the certificate disappears from view! I told her and there is a long pause. After about 30 seconds she asks if it is OK to put me on hold.

After a while she says “your certificate is not a valid ATO certificate”. Now prior to calling I had just logged onto the Business Portal so I know for certain that the cert is valid and working.

So I thought…”I can successfully login to the ATO with a invalid certificate - REALLY! Then the Australian government’s ATO PKI infrastructure has been compromised!!!”

Anyway she was very nice about it and advised me that I would receive a call from an escalations team in a couple of days. I felt quite sorry for her as my configuration might have been a tad over what she would have encountered on an everyday basis.

In the end they told me that storing the certificate on a smartcard was not supported so I am now back to storing the cert in a java certificate store. So much for better security!

Monowall And Wikid

2008-12-09T00:00:00+08:00

I found out about Monowall about a year and a half ago and really love it. Some of it’s best features are:

Easy to deploy (via PC / Embedded or even as a VM)
Easy to Manage (web based management interface second to none)
Powerful. It is easy to create a good strong firewall rule set. Traffic Shaper, Captive Portal, PPTP and IPSec VPNs, User manager, Single config file (easy backup of configs) and the list goes on.
Reliable ; Good community support available

In the last month I bought a 2-Factor Authentication system from Wikid and have set it up as my authentication mechanism for my monowall-terminated PPTP VPN (doesn’t work with IPsec at time of publication). The steps to set this up are:

Install Wikid Auth server via the ISO or use the RPMs (I found the ISO easier and just upgraded the RPMs after install)

Enable the Radius Protocol modules for your wikid domain (I have the IP Address for the Radius server set as 127.0.0.1 – not sure why that works but it obviously still spawns a radius daemon on the wikid auth server’s real interface) I also had the “Multihomed” setting set to ‘on’ (the default).

Setup a network client for your monowall using radius and a shared secret (I have the network client pointing to the interface address of my vlan, not the LAN interface address)

Setup a Token Client and ensure that you can authenticate to the wikid auth server.

On the monowall:

Set the PPTP VPN settings to use Radius authentication. Set the IP address of the radius server to the IP of the Wikid Auth server and set the shared secret that you specified on the Wikid Auth Server.

Reboot the Monowall – I spent a day trying to figure out why this was not working. I setup a Iptables firewall rule to log all traffic and could not see any traffic coming from the monowall while trying to authenticate to the VPN. Eventually out of frustration I rebooted the firewall and viola it worked !!

Since then it has been working pretty well, except that the iPhone token doesn’t work on some 3G and Wifi networks - it gives the message “unable to get passcode”.

Other than that issue the solution works well and I like the Wikid 2-Factor authentication system. Another cool feature is you can have wikid “domains” which allow you to have different zones of authentication while using only one token, for example I can use a domain to authenticate to my VPN and another (same token, protected by a different PIN) to authenticate to my linux servers.

Keylogger Paranoia

2008-04-20T00:00:00+08:00

I want a network that is reasonably secure.For the past few years my requirement has been for access to my systems to be resistant to keyloggers and brute force attacks. My second requirement is that this needs to be available when travelling, ie in hostile computing environments.

The Majority of the systems I run are Linux based systems so for the most part I have been trying to secure SSH sessions, therefore I purchased a Rainbow I-key 3000 hoping that I could authenticate my ssh sessions using a ‘keylogger proof’ smart card.

I spent ages trying to get ssh-agent to use a smartcard, actually I think there is still a bug in openssh’s bugtrac for that , to no avail. I even tried recompiling openssh but did not have any success. I gave up on that and now just use a standard ssh key.

Last year I stumbled upon Wikid but since I want to secure two hosts on insecure networks I will need to create a vpn between the two hosts, which creates some other challenges.

As of 2007 I have outsourced a lot of our systems to SAAS providers, for example Harvest for our time tracking / billing, therefore the shape of the systems I protect has changed somewhat, namely I don’t need to worry about the underlying system nor the application, just access to the data itself, which is serious enough. With that in consideration I use Openid to authenticate to these services, using a SSL certificate, which works well except it does not travel well (or at all!).

To sum it all up after 5 years I have not acheived any of my goals and I don’t really have too many options either, which is quite disappointing! Either I implement 2 factor authentication system like wikid or RSA securid or review my requirements.