2012-08-13 Update: simple monitoring with Cacti.
The goal
I want to have a clean Wowza setup for a live streaming platform. The access to the streaming server will be open but every other service (management interface, ssh, web server, etc) will be firewalled.
There will be usage reporting through awstats, monitoring through SNMP (performance) and Nagios NRPE (availability).
Ubuntu
Ubuntu installation
I’ve done a standard installation, nothing fancy.
Partitionning is done like this:
- /boot 1GB ext2
- / 10GB ext4
- swap 2G
- /usr/local 10G ext4
- /var 5G ext4
- /home * ext4 .
The /home partition will hold the content files, if any. All partitions are mounted with the noatime flag and have a label defined to their mounting point.
I just installed OpenSSH server at installation, I will install the rest manually later.
Ubuntu configuration
Install all the required & useful packages:
apt-get install screen tcpdump vim mc ngrep ifstat tofrodos sed -i \ -e 's/#startup_message off/startup_message off/' \ -e 's/vbell on/vbell off/' \ -e 's/defscrollback 1024/defscrollback 10000/' \ /etc/screenrc
Put a basic iptables ruleset in /etc/iptables and add “cat /etc/iptables | iptables-restore” in /etc/rc.local .
Reboot to test if it works :)
Wowza Media Server
Wowza Media Server installation
Download Java JRE (currently 1.7 version) from www.java.com then install it:
mkdir /opt/java && cd /opt/java wget -O - "http://javadl.sun.com/webapps/download/AutoDL?BundleId=63204" | tar -xzvf - update-alternatives --install /usr/bin/java java /opt/java/jre1.7.0_04/bin/java 1
Download the installation package (Linux DEB): www.wowza.com/pricing/installer and install it:
wget http://www.wowza.com/downloads/WowzaMediaServer-3-1-1/WowzaMediaServer-3.1.1.deb.bin chmod +x WowzaMediaServer-3.1.1.deb.bin ./WowzaMediaServer-3.1.1.deb.bin
Buy a license or request a trial one and copy/paste it in /usr/local/WowzaMediaServer/conf/Server.license
Open the required ports for Wowza (see www.wowza.com/forums/content.php?217#openPorts).
Start Wowza:
/etc/init.d/WowzaMediaServer start
Try to open <yourserveripaddress>:1935 with your browser. If you see something about Wowza, then it’s ok.
Wowza Media Server configuration
Change the default admin password:
mcedit /usr/local/WowzaMediaServer/conf/admin.password
I create an application for this streaming:
mkdir /usr/local/WowzaMediaServer/applications/axislive mkdir /usr/local/WowzaMediaServer/conf/axislive cp /usr/local/WowzaMediaServer/conf/Application.xml /usr/local/WowzaMediaServer/conf/axislive/
In that new axislive/Application.xml file, I need to change some parameters for live streaming (see www.wowza.com/forums/content.php?39).
First, change Streams/StreamType from “default” to “live” . Change Streams/LiveStreamPacketizers from empty to “cupertinostreamingpacketizer, smoothstreamingpacketizer, sanjosestreamingpacketizer” . Change Streams/Authentication/PlayMethod> to none .
Then I want to open some streams at Wowza start, so I use the StartupStreams.xml file. In there, you to define which application will start which stream. The stream is defined in a .stream file which contains an URL. For me, the URL looks like this:
rtsp://login:password@1.2.3.4:1234/axis-media/media.amp
Restart Wowza and it’s going to receive video data from the cameras.
At this point, you should be able to see the camera videos with VLC or a flash player.
I need to limit the concurrent connections on my applications, there’s a module for that (www.wowza.com/forums/content.php?152-Module-to-limit-the-number-of-connections-to-an-application). To install it, do the following:
wget http://www.wowzamedia.com/downloads/forums/collection/wms-plugin-collection.zip unzip wms-plugin-collection.zip cp wms-plugin-collection/wms-plugin-collection/lib/wms-plugin-collection.jar /usr/local/WowzaMediaServer/lib
Then add the following to your application modules:
<Module> <Name>ModuleLimitStreamViewers</Name> <Description>Limit connects to an Application</Description> <Class>com.wowza.wms.plugin.collection.module.ModuleLimitConnectionsToApplication</Class> </Module>
Then add the following property to your application properties:
<Property> <Name>maxApplicationConnections</Name> <Value>200</Value> <Type>Integer</Type> </Property>
Restart Wowza and tada!
Configuration and tuning
Now that we can stream what we want, we need to have a bit of reporting.
Usage reporting with awstats
apt-get install nginx fcgiwrap awstats libgeo-ip-perl
Change the default port for nginx to something like 8080, in /etc/nginx/sites-enabled/default and restart nginx.
Follow these instructions (www.howtoforge.com/serving-cgi-scripts-with-nginx-on-debian-squeeze-ubuntu-11.04-p3) to enable CGI on nginx.
We also have to tell nginx where to find awstats icons:
location /awstatsicons {
alias /usr/share/awstats/icon;
}
We need to patch /usr/lib/cgi-bin/awstats.pl a little so that it handles the protocols of Wowza. Here’s the patch:
--- awstats.pl.org 2012-05-16 16:24:17.312306052 +0200
+++ awstats.pl 2012-05-16 16:26:05.808310486 +0200
-17755,7 +17755,11 @@
|| uc($field[$pos_method]) eq 'MMS'
|| uc($field[$pos_method]) eq 'RTSP'
|| uc($field[$pos_method]) eq 'HTTP'
- || uc($field[$pos_method]) eq 'RTP' )
+ || uc($field[$pos_method]) eq 'RTP'
+ || uc($field[$pos_method]) eq 'RTMP'
+ || uc($field[$pos_method]) eq 'RTMPE'.
+ || uc($field[$pos_method]) eq 'RTMPT (HTTP-1.1)' )
+ )
)
{
We customize a bit our awstats configuration to take Wowza output (or you can configure Wowza to change its output to match awstats input). So the file is /etc/awstats/awstats.axislive.conf and we put the following lines in it:
LogFile="awk 'BEGIN{FS="\t"; OFS="\t";} {if ($4 == "destroy") print $1, $2, $18, $17, $19, $7, $37, $23, $13, $28, $27, $26, $8, $18, $12;}' /usr/local/WowzaMediaServer/logs/wowzamediaserver_access.log |"
LogType=S
LogFormat="date time %method c-ip %referer %code %url sc-bytes filelength x-sname sc-stream-bytes cs-stream-bytes cs-uri-stem %extra1 %virtualname"
LogSeparator="\t"
SiteDomain="honeywell"
HostAliases="_definst_"
DNSLookup=1
DirData="/usr/local/WowzaMediaServer/awstats-wmc/data"
DirCgi="/"
DirIcons="/awstatsicons"
AllowToUpdateStatsFromBrowser=0
AllowFullYearView=2
You can tweak it more if you want, I don’t need to.
Create the required directory:
mkdir /usr/local/WowzaMediaServer/awstats-wmc/data
Execute awstats a first time to see if it’s working:
/usr/lib/cgi-bin/awstats.pl -update -config=axislive -showdropped
If so (it should be), put it in crontab.
Restart nginx, check if it’s working: 1.2.3.4:8880/cgi-bin/awstats.pl?config=axislive .
Monitoring performance with SNMP
Monitoring of standard things.
apt-get install snmpd
Edit /etc/snmp/snmpd.conf to your needs (version, community, login/password, restrictions, etc). Edit /etc/default/snmpd and remove “-Lsd” fromS SNMPDOPTS so that it’s doesn’t flood your logs with connections. Restart snmpd.
Monitoring of Wowza viewers.
I chose to monitor to connection count of each type (flash, rtsp, etc) and not bother with connection count by application or stream name (just because I don’t know how to update dynamically Cacti graphs). So this is the easiest way I think.
How it is done: a script query in Cacti that gets Wowza stats with the HTTP interface. It’s parsed and the totals is printed back, then Cacti graphs this result.
The script is here, rename it to CactiWowzaDIM.php . Here’s the XML templates, import it to Cacti through the web interface.
Bug: there’s a problem with this script and Cacti and I don’t get why it’s there. Although the script will print a value for the key sessionsCupertino , Cacti keeps it at “-nan”. If someone finds the reason and the fix, it would be great!
Monitoring availability with Nagios
Streaming server side
apt-get install nagios-nrpe-server
Edit /etc/nagios/nrpe_local.cfg and change any parameters you see fit (allowed_hosts for example).
Nagios server side
You need to tell Nagios which services you want to monitor. I will check load average, free disk space, zombie processes, SSH (TCP/22), Wowza RTMP (TCP/1935), Nginx (TCP/8080) . The configuration file for Nagios looks like this:
Later :)
Thanks a lot for the great tutorial. Let me try the monitoring setup with wowza
Sathish
Hi man,
Great article and great scripts. It seems that the bug you mention about the sessionsCupertino field is related to the name. RRDTool limits data_names to 19 characters and that one is bigger, that’s why it’s failing.
Hope it helps. :)
Thank you for sharing this.
I’ve changed wowza output to match awstats input. This way you don’t have to rewrite like you did with awk in the LogFile option.
Configure wowza logging by editing log4j.properties and set:
log4j.appender.serverAccess.layout.Fields=date,time,c-proto,c-ip,c-referrer,x-status,cs-uri-stem,sc-bytes,x-duration,x-sname,sc-stream-bytes,cs-stream-bytes,x-ctx,c-proto,x-appinst
log4j.appender.serverAccess.layout.EventInclude=destroy
Using this configuration only ‘destroy’ events are logged and wowza output matches awstats input.
Ah yes, that’s another approach I didn’t think of. I will do this for my next setup, I seems more logical/easy.
I am trying to use the scripts you provided and somehow i am stuck with the following error. When i comment out Line 19 it goes to the next line which has writeToLog function.
This is the error i get when i run from shell
PHP Notice: Undefined variable: writeToLog in /usr/share/cacti/resource/script_queries/CactiWowzaDIM.php on line 19
This is the error from cacti
12/06/2015 02:16:02 PM – CMDPHP: Poller[0] Host[9] DS[58] WARNING: Result from CMD not valid. Partial Result: U
Hello, you just need to change line 17 (
global $logfile;) with the following:global $logfile, $writeToLog;The variable $writeToLog is not in the function scope, so you need to explicitely look for it in the main scope.