2012-08-13 Update: simple monitoring with Cacti.
The goal
I want to have a clean Wowza setup for a live streaming platform. The access to the streaming server will be open but every other service (management interface, ssh, web server, etc) will be firewalled. There will be usage reporting through awstats, monitoring through SNMP (performance) and Nagios NRPE (availability).
Ubuntu
Ubuntu installation
I’ve done a standard installation, nothing fancy. Partitionning is done like this:
- /boot 1GB ext2
- / 10GB ext4
- swap 2G
- /usr/local 10G ext4
- /var 5G ext4
- /home * ext4 .
The /home partition will hold the content files, if any. All partitions are mounted with the noatime flag and have a label defined to their mounting point. I just installed OpenSSH server at installation, I will install the rest manually later.
Ubuntu configuration
Install all the required & useful packages:
apt-get install screen tcpdump vim mc ngrep ifstat tofrodos
sed -i \
-e 's/#startup_message off/startup_message off/' \
-e 's/vbell on/vbell off/' \
-e 's/defscrollback 1024/defscrollback 10000/' \
/etc/screenrc
Put a basic iptables ruleset in /etc/iptables and add “cat /etc/iptables | iptables-restore” in /etc/rc.local . Reboot to test if it works :)
Wowza Media Server
Wowza Media Server installation
Download Java JRE (currently 1.7 version) from http://www.java.com then install it:
mkdir /opt/java && cd /opt/java
wget -O - "http://javadl.sun.com/webapps/download/AutoDL?BundleId=63204" | tar -xzvf -
update-alternatives --install /usr/bin/java java /opt/java/jre1.7.0_04/bin/java 1
Download the installation package (Linux DEB): http://www.wowza.com/pricing/installer and install it:
wget http://www.wowza.com/downloads/WowzaMediaServer-3-1-1/WowzaMediaServer-3.1.1.deb.bin
chmod +x WowzaMediaServer-3.1.1.deb.bin
./WowzaMediaServer-3.1.1.deb.bin
Buy a license or request a trial one and copy/paste it in /usr/local/WowzaMediaServer/conf/Server.license Open the required ports for Wowza (see http://www.wowza.com/forums/content.php?217#openPorts). Start Wowza:
/etc/init.d/WowzaMediaServer start
Try to open http://:1935 with your browser. If you see something about Wowza, then it’s ok.
Wowza Media Server configuration
Change the default admin password:
mcedit /usr/local/WowzaMediaServer/conf/admin.password
I create an application for this streaming:
mkdir /usr/local/WowzaMediaServer/applications/axislive
mkdir /usr/local/WowzaMediaServer/conf/axislive
cp /usr/local/WowzaMediaServer/conf/Application.xml /usr/local/WowzaMediaServer/conf/axislive/
In that new axislive/Application.xml file, I need to change some parameters for live streaming (see http://www.wowza.com/forums/content.php?39). First, change Streams/StreamType from “default” to “live” . Change Streams/LiveStreamPacketizers from empty to “cupertinostreamingpacketizer, smoothstreamingpacketizer, sanjosestreamingpacketizer” . Change Streams/Authentication/PlayMethod> to none . Then I want to open some streams at Wowza start, so I use the StartupStreams.xml file. In there, you to define which application will start which stream. The stream is defined in a .stream file which contains an URL. For me, the URL looks like this:
rtsp://login:password@1.2.3.4:1234/axis-media/media.amp
Restart Wowza and it’s going to receive video data from the cameras. At this point, you should be able to see the camera videos with VLC or a flash player. I need to limit the concurrent connections on my applications, there’s a module for that (http://www.wowza.com/forums/content.php?152-Module-to-limit-the-number-of-connections-to-an-application)). To install it, do the following:
wget http://www.wowzamedia.com/downloads/forums/collection/wms-plugin-collection.zip
unzip wms-plugin-collection.zip
cp wms-plugin-collection/wms-plugin-collection/lib/wms-plugin-collection.jar /usr/local/WowzaMediaServer/lib
Then add the following to your application modules:
<Module>
<Name>ModuleLimitStreamViewers</Name>
<Description>Limit connects to an Application</Description>
<Class>com.wowza.wms.plugin.collection.module.ModuleLimitConnectionsToApplication</Class>
</Module>
Then add the following property to your application properties:
<Property>
<Name>maxApplicationConnections</Name>
<Value>200</Value>
<Type>Integer</Type>
</Property>
Restart Wowza and tada!
Configuration and tuning
Now that we can stream what we want, we need to have a bit of reporting.
Usage reporting with awstats
apt-get install nginx fcgiwrap awstats libgeo-ip-perl
Change the default port for nginx to something like 8080, in /etc/nginx/sites-enabled/default and restart nginx. Follow these instructions (http://www.howtoforge.com/serving-cgi-scripts-with-nginx-on-debian-squeeze-ubuntu-11.04-p3) to enable CGI on nginx. We also have to tell nginx where to find awstats icons:
location /awstatsicons {
alias /usr/share/awstats/icon;
}
We need to patch /usr/lib/cgi-bin/awstats.pl a little so that it handles the protocols of Wowza. Here’s the patch:
--- awstats.pl.org 2012-05-16 16:24:17.312306052 +0200
+++ awstats.pl 2012-05-16 16:26:05.808310486 +0200
-17755,7 +17755,11 @@
|| uc($field\[$pos_method\]) eq 'MMS'
|| uc($field\[$pos_method\]) eq 'RTSP'
|| uc($field\[$pos_method\]) eq 'HTTP'
- || uc($field\[$pos_method\]) eq 'RTP' )
+ || uc($field\[$pos_method\]) eq 'RTP'
+ || uc($field\[$pos_method\]) eq 'RTMP'
+ || uc($field\[$pos_method\]) eq 'RTMPE'.
+ || uc($field\[$pos_method\]) eq 'RTMPT (HTTP-1.1)' )
+ )
)
{
We customize a bit our awstats configuration to take Wowza output (or you can configure Wowza to change its output to match awstats input). So the file is /etc/awstats/awstats.axislive.conf and we put the following lines in it:
LogFile="awk 'BEGIN{FS="\t"; OFS="\t";} {if ($4 == "destroy") print $1, $2, $18, $17, $19, $7, $37, $23, $13, $28, $27, $26, $8, $18, $12;}' /usr/local/WowzaMediaServer/logs/wowzamediaserver_access.log |"
LogType=S
LogFormat="date time %method c-ip %referer %code %url sc-bytes filelength x-sname sc-stream-bytes cs-stream-bytes cs-uri-stem %extra1 %virtualname"
LogSeparator="\t"
SiteDomain="honeywell"
HostAliases="_definst_"
DNSLookup=1
DirData="/usr/local/WowzaMediaServer/awstats-wmc/data"
DirCgi="/"
DirIcons="/awstatsicons"
AllowToUpdateStatsFromBrowser=0
AllowFullYearView=2
You can tweak it more if you want, I don’t need to. Create the required directory:
mkdir /usr/local/WowzaMediaServer/awstats-wmc/data
Execute awstats a first time to see if it’s working:
/usr/lib/cgi-bin/awstats.pl -update -config=axislive -showdropped
If so (it should be), put it in crontab. Restart nginx, check if it’s working: http://1.2.3.4:8880/cgi-bin/awstats.pl?config=axislive .
Monitoring performance with SNMP
Monitoring of standard things.
apt-get install snmpd
Edit /etc/snmp/snmpd.conf to your needs (version, community, login/password, restrictions, etc). Edit /etc/default/snmpd and remove “-Lsd” fromS SNMPDOPTS so that it’s doesn’t flood your logs with connections. Restart snmpd.
Monitoring of Wowza viewers.
I chose to monitor to connection count of each type (flash, rtsp, etc) and not bother with connection count by application or stream name (just because I don’t know how to update dynamically Cacti graphs). So this is the easiest way I think. How it is done: a script query in Cacti that gets Wowza stats with the HTTP interface. It’s parsed and the totals is printed back, then Cacti graphs this result. The script is here, rename it to CactiWowzaDIM.php . Here’s the XML templates, import it to Cacti through the web interface. Bug: there’s a problem with this script and Cacti and I don’t get why it’s there. Although the script will print a value for the key sessionsCupertino , Cacti keeps it at “-nan”. If someone finds the reason and the fix, it would be great!
Monitoring availability with Nagios
Streaming server side
apt-get install nagios-nrpe-server
Edit /etc/nagios/nrpe_local.cfg and change any parameters you see fit (allowed_hosts for example).
Nagios server side
You need to tell Nagios which services you want to monitor. I will check load average, free disk space, zombie processes, SSH (TCP/22), Wowza RTMP (TCP/1935), Nginx (TCP/8080) . The configuration file for Nagios looks like this: Later :)