I have worked with SIP for several years and one of the things I like about SIP is the simplicity when analyzing SIP messages. H323 on the other hand is quite different and I find it frustrating that analyzing H323 is more cumbersome than SIP. Although H323 has been around a long time it’s not going to disappear anytime soon. It’s used quite heavily in both the Service Provider and Enterprise environments.

router# debug gatekeeper main 10

router# debug gatekeeper ?
gup Gatekeeper GUP messages
load Gatekeeper load-balancing messages
servers Gatekeeper Servers

As you can see above, the command I am referring to is undocumented. The great thing about this command is the simplicity in the debug output.

Click the screenshot and see for yourself.

This has nothing to do with VoIP but may be useful for some folks. There are times when Leopard or Snow Leopard will not let you empty the Trash due to a file being in use. Even when holding down Option and emptying the Trash you may still encounter an error. To get around this, open Terminal (Applications > Utilities > Terminal) and enter the following command.

sudo rm -Rf ~/.Trash/*

CSIM is an undocumented IOS command.  When configuring a router to act as a voice gateway it is possible to test outbound calls by originating calls from the router.  If the router is configured correctly then the far end number will ring and may be answered. Use the csim start dialstring hidden command to initiate simulated calls to whichever real-world E.164 number is desired. This allows you to determine whether you can properly go offhook from the router to the PSTN, send digits, and complete a call to the destination phone. You can modify the POTS dial-peer appropriately to account for long-distance access codes and other prefixed digits as necessary. In the example below, the POTS dial-peer can match on any string of digits starting with “9”, and all digits that follow the “9” are played out voice-port X/Y/Z.

dial-peer voice 1 pots
destination-pattern 9T
port 1/0:0

r1# csim start 918005551212

csim: called number = 18005551212, loop count = 1 ping count = 0

csim err csimDisconnected recvd DISC cid(28)

csim: loop = 1, failed = 1   csim: call attempted = 1, setup failed = 1, tone failed = 0

Despite the call working in this case CSIM will always display failed=1.  There is no real explanation for this.  CSIM can also be used with PRI, CAS, and if calls are not completing it is recommended to begin analyzing the call flow using debug commands. For example debug isdn q931 will show the Tx and Rx messages (including ISDN Cause Codes) between the local router and the far end TDM equipment.

Here are other examples of POTS dial-peers which reflect different ways to pass digits to the TDM network.

Any of these now sends the entire string “12345678” to the PBX:

!
dial-peer voice X pots
 destination-pattern 1234....
 port 1/0:0
 forward-digits all
!

or:

!
dial-peer voice X pots
 destination-pattern 1234....
 port 1/0:0
 no digit-strip
!

or:

!
dial-peer voice X pots
 destination-pattern 1234....
 port 1/0:0
 prefix 1234
!

One of the biggest complaints I hear from people is how horrific Cisco’s own search engine is on cisco.com.  I use Google instead.  It’s spot-on every time and never lets me down!

I acquired a used CUE module running CUE 2.1.2 and wanted to proceed upgrading the module to CUE 7.0.3.  The only way to upgrade CUE is through FTP.  Here is the procedure.

First thing I needed to do was install vsftpd on my RHEL 5 Client Workstation.  If you are running CentOS or Fedora you may already have vsftpd installed or you can easily install it using YUM.  The YUM repository for RHEL5 Client Workstation does not include the option to install vsftpd through YUM but you can login to your Red Hat account on their web site and search ‘vsftpd’ and there is an RPM you may download and install (with no dependency problems).

# rpm -ivh vsftpd-2.0.5-16.el5.x86_64.rpm to install vsftpd

# man vsftp to read the very short manual

# chkconfig vstpd on to start vsftpd on bootup

# service vsftpd stop|start|restart to reload the service at any time

By default the ftp root directory is /var/ftp

# cd /etc/vsftpd

# vi vsftpd.conf and allow anonymous users (not required but makes things a bit easier)

From Cisco’s CCO page navigate to Cisco Unity Express 7.0.3 and download cue-vm-k9.nm-aim.7.0.3.zip. This file is for NM-CUE and AIM modules and there is a different zip file for NME-CUE. You also must download the appropriate license files for the number of mailboxes you are using and the IVR ports if applicable. I also had to include the specific language file cue-vm-en_US-langpack.nm-aim.7.0.3.prt1 in my FTP folder or the installation would fail regardless of the language pack being present. Extract all the zip files into the /var/ftp folder on your FTP server.

Log in to your CME router populated with the CUE module.  You will need to open a session through enable mode (not config mode). You should already have your service-engine and service-module parameters configured. IP 177.3.11.1 is the default gateway on my router and 177.3.11.254 is the IP I have assigned to my CME service module.

interface Service-Engine1/0
ip unnumbered Vlan11
service-module ip address 177.3.11.254 255.255.255.0
service-module ip default-gateway 177.3.11.1

router# service-module service-engine 0/1 session
Trying… Open

cue# show software version
Cisco Unity Express version 2.1.2

Be sure you can ping your FTP server form the CUE command prompt.

cue# ping 177.3.11.2
PING 177.3.11.2 (177.3.11.2) 56(84) bytes of data.
64 bytes from 177.3.11.2: icmp_seq=1 ttl=255 time=0.506 ms
64 bytes from 177.3.11.2: icmp_seq=2 ttl=255 time=0.287 ms
64 bytes from 177.3.11.2: icmp_seq=3 ttl=255 time=0.252 ms
64 bytes from 177.3.11.2: icmp_seq=4 ttl=255 time=0.257 ms
64 bytes from 177.3.11.2: icmp_seq=5 ttl=255 time=0.272 ms

— 177.3.11.2 ping statistics —
5 packets transmitted, 5 received, 0% packet loss, time 2ms
rtt min/avg/max/mdev = 0.252/0.314/0.506/0.098 ms, ipg/ewma 0.551/0.407 ms

In this particular scenario I am performing a clean install. CUE offers the option of doing an upgrade instead of a clean install but you must refer to the Cisco Unity Express 7.0 Installation and Upgrade Guide to make sure the the existing version of CUE is at the correct version.

cue# software install clean url ftp://177.3.11.2/cue-vm-k9.nm-aim.7.0.3.pkg username anonymous password anonymous@

The file cue-vm-k9.nm-aim.7.0.3.pkg is downloaded by CUE which then proceeds to download the remaining files from the CUE zip files that was extracted in to the FTP folder. You will be prompted to choose a language and this is where you must have the specific language pack in your FTP folder (referred to as language payload by CUE). CUE will run through a series of steps to install CUE 7.0.3 and prompt you to reload the module.

cue# show software version
Cisco Unity Express version (7.0.3)
Technical Support: http://www.cisco.com/techsupport Copyright (c) 1986-2008 by Cisco Systems, Inc.

Components:

- CUE Voicemail Language Support version 7.0.3

The license files must now be loaded. Be sure you downloaded the correct ones as one is for CCM and the other CME. Once you install one license type you cannot change the system to a different license unless you perform a clean install.

cue# software install clean url ftp://177.3.11.2/cue-vm-license_12mbx_cme_7.0.3.pkg username anonymous password anonymous@

cue# software install clean url ftp://177.3.11.2/cue-vm-license_4port_ivr_7.0.3.pkg username anonymous password anonymous@

cue# reload

cue# show software licenses
Installed license files:
- voicemail_lic.sig : 50 MAILBOX LICENSE
- ivr_lic.sig : 4 PORT IVR BASE LICENSE

Core:
- Application mode: CCME
- Total usable system ports: 8

Voicemail/Auto Attendant:
- Max system mailbox capacity time: 6000
- Default # of general delivery mailboxes: 15
- Default # of personal mailboxes: 50

- Max # of configurable mailboxes: 65

Interactive Voice Response:
- Max # of IVR sessions: 4

Languages:
- Max installed languages: 5
- Max enabled languages: 5

As I go through my CCIE Voice tasks I frequently need to wipe the CUE configuration and start with a fresh CUE 7 initialization:

cue# offline
cue# restore factory default

To wipe the config and reboot into a fresh system prompt with the initialization questions do the following:

cue# erase startup-config

The first time you log into the web interface of CUE you will need to associate the administrator credentials of the router with CUE so CUE may access CME.

r3-br2(config-telephony)#web admin system name admin password cisco

r3-br2(config)#ip http path flash:/gui

If you forget the last statement containing ip http path flash:/gui then CUE will fail to validate the admin/cisco credentials when entered in the CUE web interface.  CUE requires administrator access to CME.

The following is an example of using a Cisco 2811 slotted with an NM-16A ASYNC module to manage other Cisco devices as if you were connected to them locally with a laptop and a serial connection. The ASYNC modules are available in different port densities. There is also an HWIC version.

router# show hardware

Cisco 2811 (revision 53.50) with 772096K/14336K bytes of memory.

Processor board ID FTX1124A3VQ
2 FastEthernet interfaces
15 Serial interfaces
4 Low-speed serial(sync/async) interfaces
16 terminal lines
2 Channelized/Clear E1/PRI ports
4 Channelized/Clear T1/PRI ports
1 Virtual Private Network (VPN) Module

!
line con 0
exec-timeout 0 0
privilege level 15
line aux 0
line 1/0 1/15
session-timeout 5
transport input all
autohangup
stopbits 1
line vty 0 4
login
!
router#show line

Tty Line Typ Tx/Rx A Modem Roty AccO AccI Uses Noise Overruns Int
0 0 CTY – – – – – 0 0 0/0 -
1 1 AUX 9600/9600 – – – – – 0 0 0/0 -
1/0 66 TTY 9600/9600 – – – – – 0 0 0/0 -
1/1 67 TTY 9600/9600 – – – – – 0 0 0/0 -
1/2 68 TTY 9600/9600 – – – – – 0 0 0/0 -
1/3 69 TTY 9600/9600 – – – – – 0 0 0/0 -
1/4 70 TTY 9600/9600 – – – – – 0 0 0/0 -
1/5 71 TTY 9600/9600 – – – – – 0 0 0/0 -
1/6 72 TTY 9600/9600 – – – – – 0 0 0/0 -
1/7 73 TTY 9600/9600 – – – – – 0 0 0/0 -
1/8 74 TTY 9600/9600 – – – – – 0 0 0/0 -
1/9 75 TTY 9600/9600 – – – – – 0 0 0/0 -
1/10 76 TTY 9600/9600 – – – – – 0 0 0/0 -
1/11 77 TTY 9600/9600 – – – – – 0 0 0/0 -
1/12 78 TTY 9600/9600 – – – – – 0 0 0/0 -
1/13 79 TTY 9600/9600 – – – – – 0 0 0/0 -
1/14 80 TTY 9600/9600 – – – – – 0 0 0/0 -
1/15 81 TTY 9600/9600 – – – – – 0 0 0/0 -

The ASYNC module is in slot 1 and there are 16 available ASYNC ports labeled 1/0 – 1/15.

router# show diag

Slot 1:
Async Port adapter, 16 ports
Port adapter is analyzed
Port adapter insertion time 00:15:02 ago
EEPROM contents at hardware discovery:
Hardware revision 0.1 Board revision B0
Serial number 15674198 Part number 800-02244-05
FRU Part Number NM-16A=

Configuration example

Best-practice is to create a loopback interface. They are never ‘down’ unless they are administratively shutdown.

router(config)# interface Loopback0
router(config-if)# ip address 177.1.254.254 255.255.255.255

Use the ‘ip host’ command to assign a hostname to your reverse telnet session. 20xx port number should match the intended physical connection from the rear of the NM-16A as seen in ’show line’ output.

router(config)# ip host hq 2066 177.1.254.254
router(config)# ip host sw 2067 177.1.254.254
router(config)# ip host br1 2068 177.1.254.254
router(config)# ip host br2 2069 177.1.254.254

Breakdown

• Router starts all reverse telnet ports with 20xx
• The lowest xx value is 66 and the highest is 81 (review ’show line’ output)
• Possible assignable ranges are 2066 – 2081 (which correspond with Line 1/0 – 1/15)
• Rear of NM-16A module uses two Octal cables labeled Port 0-7 and Port 8-15
• Port/Cable 0 = Line 1/0; Line 1/0 = 2066
• Port/Cable 1 = Line 1/1; Line 1/1 = 2067
• Port/Cable 2 = Line 1/2; Line 1/2 = 2068
• Port/Cable 15 = Line 1/15; Line 1/15 = 2081

To verify this configuration is working to br2 type the host name in enable mode

router# br2
Translating “br2″

Trying br2 (177.1.254.254, 2069)… Open

r3-br2#

Press and hold CTRL SHIFT 6 (in that order) then let go and press X to exit the console session and return back to the original router# acting as the Access Server.

CTRL+SHIFT+6 X does not permanently end the session, it only jumps the session back to the Access Server.  Knowing that the connection to br2 is still ‘open’ it is still possible to console to yet another device without ending the original session to br2.  If I want to connect to br1 from the router# I simply type br1. Pressing CTRL+SHIFT+6 X will escape the br1 session and return the session to the Access Server.

It is important to keep in mind the router still has two active console connections which include br2 (the original session) and br1. Entering ‘br2′ will not allow a connection to the br2 router because a session is already in open. Instead, enter ‘resume br2′ which uses the existing session to resume the console connection from the Access Server. When it is time to permanently end a console session enter ‘disconnect br2′ at the Access Server router prompt and the console session will end.

Recently I needed to change the firmware on some Cisco 7965 phones from SCCP to SIP. By far the simplest method is loading the COP file on UCM and letting the phone upgrade on its own.  In my case, this upgrade was being done without using UCM.  The Cisco read-me doc for the SIP firmware covers the COP upgrade procedure only.  It tells you that you may unzip the files on a TFTP server but there is no procedure which explains what else you must do to load the SIP firmware.

In this example I am upgrading Cisco 7965 phones to SIP firmware 8.5.  Once you have downloaded the zipped version of the SIP firmware from CCO place the unzipped files in your TFTP servers root directory.  Modify your XMLDefaults.cnf.xml file so the load information matches your firmware.

<loadInformation8 model=”Cisco 7965″>SIP45.8-5-3TH1</loadInformation8>

You should connect your IP phone to LAN where DHCP provides the IP, subnet, and TFTP server IP.  Make sure your phone has DHCP enabled = YES. Your DHCP server needs to support DHCP Options.  TFTP option 66 is required for Cisco phones running SIP.  Option 66 can be used to provide an IP address (recommended) but can also support a DNS names (assuming you are also providing at least one DNS server IP via DHCP).  Option 150 only supports IP addresses and is required for SCCP firmware.  You can safely configure your DHCP to issue both TFTP options.

Next pull the power from your phone and plug it back in.  Hold down # until the line keys start to blink and press 123456789*0# and your phone should reset.  Your phone should display “Upgrading” on the screen.  If you are using a Unix based tftp server you can execute tcpdump port 69 and you should see your phone requesting the files.  Your phone should display the progress of the SIP firmware upgrade and eventually reboot.  After it reboots you can press Settings > Model Information and scroll down until you see the Call Control Protocol = SIP.

If you performed a factory reset and did not have DHCP enabled then your phone is most likely stuck at the Upgrading screen. Pressing keys on the phone will not change the status. At this point you should pull the power, plug it back in, hold # and then enter the keys 3491672850*# to factory reset the phone.  This allows the phone to clear its flash and still download new firmware.  Your screen is going to be totally black and it will appear as if your phone is not functional, but the phone is really sending a DHCP request and waiting for an IP, subnet, and TFTP IP assignment before proceeding to download the firmware.  All of this is happening while the phone’s screen is black. If you want to read the official word on this, Cisco has a field notice on their web site.  Monitoring tcpdump on the TFTP server is useful in this case because you know the phone is doing something.  Also, you can view the DHCP bindings to verify your phone successfully acquired an IP address.

Since my post in January 2009 where I explained how to install various Unified 7.0 applications in VMWare, Cisco has updated the latest installation media to higher versions. UCM 7.0(1) is now UCM 7.1(3) and CUPS 7.0(1) is now CUPS 7.0(5).  Below are the latest VMWare requirements to get the applications working properly.  Note that I am using VMWare Workstation 7.0 on Red Hat Enterprise Linux 5.4.  Nothing has changed for UCCX.

UCM7.1(3) now needs to use a SCSI controller instead of IDE.  Installation will fail the hardware check if you are using IDE.  Now you must use an external NTP server and you can no longer use the internal clock.  The installation will not proceed unless a valid external NTP source is successfully connected.  If you don’t have a Linux machine handy running NTP but your VMWare Workstation can reach the internet, you may either use pool.ntp.org or do an nslookup on pool.ntp.org and use any of the IP addresses returned.

Unity Connections 7.1(3) – Same specs as UCM7.1(3)

CUPS 7.0(5) installs the same as 7.0(1) using an 80GB hard drive and IDE controller, but 7.0(5) will display a warning every time you boot the VM telling you that VMWare is unsupported.  You must agree before the boot continues.  7.0(1) does not do this.

By default Cisco locks down the UC Linux appliances so no one can access its underlying Linux operating system and obtain root access.  There are known methods to bypass this.  If you have already found a way then you may configure the virtual machine such as CUPS 7.0(5) so it does not force you to Agree to using the software on an unsupported platform every time you reboot.  This normally is not an issue if you are sitting in front of the VM while it is loading, but if you are rebooting remotely and don’t have access to the console then this is a problem.  Luckily the workaround is simple.

vi /usr/local/bin/base_scripts/hardware_check.sh and change the following:

if [ "$hwmodel" = "vmware" ];

to

if [ "$hwmodel" = "appliance" ];

As the database grows on the Broadworks Application and Network servers there will be a need to change the memory allocation for the TimesTen datastore. The Maintenance Guide does not contain all the required steps. The rule of thumb is the allocated “perm” size should not exceed more than 25% of total system memory and the “temp” size should be equal to 25% of the perm size.

The following example assumes 8GB of memory on both AS1 and AS2.

1. SSH to AS1 as bwadmin
2. stopbw
3. repctl stop
4. su as root
5. cd /usr/local/broadworks/bw_base/bin
6. timesten.pl unload
7. ./resizeDSN (perm=2048; temp=512)
8. exit (return to bwadmin)
9. repctl start
10. startbw

– Wait 10 minutes for buffered replication changes from AS2 –

1. SSH to AS2 as bwadmin
2. stopbw
3. repctl stop
4. su as root
5. cd /usr/local/broadworks/bw_base/bin
6. timesten.pl unload
7. ./resizeDSN (perm=2047; temp=512)
8. exit (return to bwadmin)
9. importdb.pl AppServer as1 AppServer (replace as1 with your primary AS hostname or IP)
10. repctl start
11. startbw

If everything went smoothly you should be able to run sychcheck_basic.pl -a on AS2 and the database should show synchronized. If the importdb.pl command in step 9 was unable to import the database, you will need to manually perform the backup and restore procedure.

1. On AS1: bwBackup.pl AppServer dbBackup.db
2. scp the file to AS2: scp dbBackup.db bwadmin@as2:dbBackup.db
3. On AS2: stopbw
4. repctl stop
5. bwRestore.pl AppServer dbBackup.db
6. repctl start
7. startbw

On one other occasion AS1 would not start replication after resizing the DSN due to an error which stated AS2 was on a different patch version than AS1. The two nodes were patched identical, but the patch tool was not responding on AS2 and therefore AS1 could not verify appropriately thus reporting the error. The solution was as simple as restarting the patch tool. However, the Maintenance Guide does not explain how to do this so I spent more time trying to find the procedure than it actually took to execute the commands.

as2$ stoppt.pl
as2$ startpt.pl

Something commonly overlooked is how to store incremental backups of your Cisco router configurations. Some people use Rancid or Kiwi Cat Tools and they are great, but what most people do not realize is IOS 12.3(4)T and higher has an archive command. Each time you perform a write mem or copy run start the router will save an archived version to the path you specify.

In this example, we chose to save the archive configuration files on the flash card in slot0; however, you can also store the configuration files remotely using such protocols as TFTP.

To create an archive of old configuration files, use the following set of commands:

Router1#configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
Router1(config)#archive
Router1(config-archive)#path slot0:/configs/$h
Router1(config-archive)#write-memory
Router1(config-archive)#time-period 1440
Router1(config-archive)#end
Router1#