ISP monitoring

 

This script will probably need updating to use mysqli instead of mysql.

This is a variation on my hosting monitor scripts.  I produced it when I found my 3G connection was poor after I returned to my holiday caravan from a week at home. Following two calls to Three they admitted the problems were due to congestion and as it wouldn’t be fixed for a least three months they gave me a 50% discount. Using this script I can keep an eye on what is happening and hopefully when are the best times to use the connection.

Sample Output
Monitoring Report
Samples: 109
Fastest load time: 0.151 2015-07-08 19:07:46
Slowest load time: 3.223 2015-07-08 12:40:47
Average load time: 1.346
Load time > 2 sec: 42 (38.5%)
Checks : 139 Good : 109 Fails : 30 Fail% : 21.6%
Last 24hrs downloads : 35200 bytes

ISP monitoring chart

Latest Errors Report
2015-07-08 18:13:44 CURL: Resolving timed out after 7516 milliseconds
2015-07-08 18:07:32 CURL: Could not resolve host: www.bbc.co.uk
2015-07-08 17:07:27 CURL: Resolving timed out after 7516 milliseconds
2015-07-08 17:01:15 CURL: Could not resolve host: www.bbc.co.uk
2015-07-08 16:55:15 CURL: Resolving timed out after 7687 milliseconds
2015-07-08 16:49:04 CURL: Resolving timed out after 7641 milliseconds
2015-07-08 16:18:46 CURL: Resolving timed out after 7594 milliseconds
2015-07-08 16:06:35 CURL: Could not resolve host: www.bbc.co.uk
2015-07-08 16:00:35 CURL: Resolving timed out after 7609 milliseconds
2015-07-08 15:48:21 CURL: Resolving timed out after 7610 milliseconds
2015-07-08 15:39:00 CURL: Resolving timed out after 7625 milliseconds
2015-07-08 15:00:07 CURL: Could not resolve host: www.bbc.co.uk
2015-07-08 13:59:53 CURL: Resolving timed out after 7657 milliseconds
2015-07-08 13:35:34 CURL: Resolving timed out after 7688 milliseconds
2015-07-08 13:23:20 CURL: Resolving timed out after 7672 milliseconds
2015-07-08 12:53:00 CURL: Resolving timed out after 7656 milliseconds
2015-07-08 12:16:37 CURL: Resolving timed out after 7656 milliseconds
2015-07-08 11:34:12 CURL: Resolving timed out after 7656 milliseconds
2015-07-08 10:57:55 CURL: Could not resolve host: www.bbc.co.uk
2015-07-08 10:39:54 CURL: Could not resolve host: www.bbc.co.uk
2015-07-08 10:39:34 CURL: Could not resolve host: www.bbc.co.uk
2015-07-08 10:15:48 CURL: Could not resolve host: www.bbc.co.uk
2015-07-08 10:09:48 CURL: Resolving timed out after 7656 milliseconds
2015-07-08 10:03:37 CURL: Resolving timed out after 7672 milliseconds
2015-07-08 09:45:23 CURL: Resolving timed out after 7672 milliseconds
2015-07-07 22:47:28 CURL: Resolving timed out after 7516 milliseconds
2015-07-07 22:12:16 CURL: Resolving timed out after 7578 milliseconds
2015-07-07 21:52:04 CURL: Resolving timed out after 7657 milliseconds
2015-07-07 21:21:52 CURL: Resolving timed out after 7672 milliseconds
2015-07-07 21:01:39 CURL: Resolving timed out after 7688 milliseconds



To run this project you’ll need to install a Wampserver to run the scripts. I found it was necessary to install  Microsoft Visual C++ redistibutable  for my Windows 8.1 64bit system. The Wampserver install then prompted me to allow access through the Windows firewall. Without the firewall permission the server failed to work. I uninstalled Wampserver and reinstall after installing the redistributable dll to get it working and that is when I saw the firewall prompt.

You’ll also need to download JpGraph to produce the bar chart. This isn’t necessarily the best php graphing tool it was just the first one I found.

Once Wampserver is in place you’ll need to download my project files and copy the folder into the www directory setup by Wampserver. Next copy the JpGraph src folder under that (I renamed the src folder to jpgraph and my scripts depend on that name).

With Wampserver running you can click the phpmyadmin link and set up a database and user for this project then create this table. You’ll need to edit the scripts with your chosen dbname, user and password.

CREATE TABLE IF NOT EXISTS `monitoring` (
`recno` int(10) unsigned zerofill NOT NULL AUTO_INCREMENT,
`timestamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`url` varchar(100) CHARACTER SET latin1 COLLATE latin1_general_ci NOT NULL,
`load_time` decimal(4,3) NOT NULL,
`curl` varchar(240) CHARACTER SET latin1 COLLATE latin1_general_ci DEFAULT NULL,
`http` varchar(6) CHARACTER SET latin1 COLLATE latin1_general_ci NOT NULL DEFAULT '0',
`datasize` mediumint(9) NOT NULL DEFAULT '0',
PRIMARY KEY (`recno`),
KEY `timestamp` (`timestamp`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=1 ;

The monitoring script as provided refreshes every 6 minutes and only stores data from 08:00 through to midnight. I run it under IE which I don’t usually use for my browsing. Here is the source..

<html>
<head>
<meta http-equiv='refresh' content='360'>
</head>
<body>
<?php
// connect to MySQL
$link = @mysql_connect('localhost','[db-user]','[password]');
mysql_select_db('[database]');

function Visit($url,$down,$timeout)
 {
 global $down;
 $down = "$url";
 echo "Checking $url Timeout: $timeout ms<br>";
 $agent = "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)";
 $ch=curl_init();
 curl_setopt($ch,CURLOPT_URL,$url );
 curl_setopt($ch,CURLOPT_USERAGENT, $agent);

 curl_setopt($ch,CURLOPT_HEADER, true);
 curl_setopt($ch,CURLOPT_NOBODY, true);

 curl_setopt($ch,CURLOPT_RETURNTRANSFER, 1);
 curl_setopt($ch,CURLOPT_VERBOSE,false);
 curl_setopt($ch,CURLOPT_ENCODING,"");
 curl_setopt($ch,CURLOPT_TIMEOUT_MS, $timeout);
 curl_setopt($ch,CURLOPT_SSL_VERIFYPEER, FALSE);
 curl_setopt($ch,CURLOPT_SSLVERSION,3);
 curl_setopt($ch,CURLOPT_SSL_VERIFYHOST, FALSE);

 $time = microtime();
 $time = explode(' ', $time);
 $time = $time[1] + $time[0];
 $start = $time;

 $page=curl_exec($ch);

 $time = microtime();
 $time = explode(' ', $time);
 $time = $time[1] + $time[0];
 $finish = $time;
 $load_time = round(($finish - $start), 3);

 $page = nl2br($page);

 if (curl_error($ch))
 {
 $curl = curl_error($ch);
 $down .= " CURL: $curl";
 $httpcode = "";
 $datasize = 0;
 curl_close($ch);
 $add_query = "INSERT INTO monitoring (url,load_time,curl,http) VALUES('$url','$load_time','$curl','$httpcode')";
 $result = mysql_query($add_query);
 echo "Request time:$load_time seconds CURL:$curl<br><br>";
 return;
 }
 else
 {
 $curl = "";
 $info = curl_getinfo($ch);
 $httpcode = $info['http_code'];
 $datasize = strlen($page);
 echo "<br>$page";
 }

 curl_close($ch);

 $add_query = "INSERT INTO monitoring (url,load_time,curl,http,datasize) VALUES('$url','$load_time','$curl','$httpcode','$datasize')";
 $result = mysql_query($add_query);

 echo "Request time: $load_time seconds HTTP code :$httpcode<br>Length:".strlen($page)."<br>";
 return;
 }


date_default_timezone_set('Europe/London');
$date = date('H:i:s');
$hour = date('H');
echo "Running SiteChk $date<br>";

if ($hour > '07' )
 {
 Visit("http://www.bbc.co.uk","","7000");
 $maint_query = "DELETE FROM monitoring WHERE timestamp < ADDDATE( NOW( ) , -1 )";
 $result = mysql_query($maint_query);
 }
else echo "Skipping check";

echo "<br>Script completed";
?>
</body>
</html> 

Here is the script that produces the report…

<?php
// connect to MySQL
$link = @mysql_connect('localhost','[db-user]','[password]');
mysql_select_db('[database]');
?>
<html>
<head>
 <title>Local URL Report</title>
 <meta http-equiv="imagetoolbar" content="no">
 <meta http-equiv='refresh' content='180'>
 <link rel="stylesheet" type="text/css" href="monitoring.css"/>
</head>
<body>
<div id="container">
<?php
echo "<div id=\"top_box\">";
echo "<h2>Monitoring Report</h2>\n";
$over_time = 2; // seconds
$my_query = "SELECT load_time,timestamp FROM monitoring where http = 200 ORDER BY load_time ASC limit 0,1;";
$my_result = mysql_query($my_query);
$my_row = mysql_fetch_array($my_result);
$fastest = $my_row ['load_time'];
$faststamp = $my_row ['timestamp'];

$my_query = "SELECT load_time,timestamp FROM monitoring where http = 200 ORDER BY load_time DESC limit 0,1;";
$my_result = mysql_query($my_query);
$my_row = mysql_fetch_array($my_result);
$slowest = $my_row ['load_time'];
$slowstamp = $my_row ['timestamp'];

$my_query = "SELECT AVG(load_time) AS average, COUNT(recno) AS samples FROM monitoring where http = 200 ;";
$my_result = mysql_query($my_query);
$my_row = mysql_fetch_array($my_result);
$average = round(($my_row ['average']),3);
$samples = $my_row ['samples'];

$my_query = "SELECT COUNT(recno) AS overs FROM monitoring where load_time > '$over_time';";
$my_result = mysql_query($my_query);
$my_row = mysql_fetch_array($my_result);
$overs = $my_row ['overs'];

$overpc = round (($overs * 100 / $samples),1);

$my_query = "SELECT COUNT(recno) AS checks FROM monitoring";
$my_result = mysql_query($my_query);
$my_row = mysql_fetch_array($my_result);
$checks = $my_row ['checks'];

$my_query = "SELECT COUNT(recno) AS successes FROM monitoring where http = 200";
$my_result = mysql_query($my_query);
$my_row = mysql_fetch_array($my_result);
$successes = $my_row ['successes'];

$bad = $checks - $successes;
$badpc = ($bad * 100) / $checks;
$badpc = round($badpc,1);

$my_query = "SELECT sum(datasize) FROM monitoring where http = 200";
$my_result = mysql_query($my_query);
$my_row = mysql_fetch_array($my_result);
$tot_down = $my_row ['0'];

echo "Samples: $samples<br>\n";
echo "Fastest load time: $fastest &nbsp; &nbsp; $faststamp<br>\n";
echo "Slowest load time: $slowest &nbsp; &nbsp; $slowstamp<br>\n";
echo "Average load time: $average<br>\n";
echo "Load time > $over_time sec: $overs ($overpc%)<br>\n";
echo "Checks : $checks Good : $successes Fails : $bad &nbsp; &nbsp; &nbsp; &nbsp; Fail% : $badpc%<br>";
echo "Last 24hrs downloads : $tot_down bytes";

echo "<br>\n";
echo "</div>";
echo "<div id=\"top_box\">";
echo "<img style=\"display:block; margin-left:auto;margin-right:auto; max-width:100%; height:auto;\"src=\"chart.php\">";
echo "</div><br>";

$http = "";
$curl = "";
$hdr = 0;
$my_query = "SELECT * FROM monitoring WHERE http <> 200 order by recno DESC LIMIT 0,30";
$my_result = mysql_query($my_query);
while ($my_row = mysql_fetch_array($my_result))
 {
 $timestamp = $my_row ['timestamp'];
 $curl = $my_row ['curl'];
 $http = $my_row ['http'];
 $datasize = $my_row ['datasize'];
 if (!$hdr)
 {
 echo "<div id=\"top_box\">";
 echo "<h2>Latest Errors Report</h2>\n";
 $hdr++;
 }
 if ($http == 200) echo "$timestamp HTTP code: $http Datasize: $datasize<br>\n";
 else if ($curl) echo "$timestamp CURL: $curl<br>\n";
 }
echo "<br>\n";
echo "</div>\n";
?>
</div>
</body>
</html>

Finally the script which produces a chart of response times over the previous 24 hours. Full pink bars with crosshatching show the checks that timed out.

<?php
require_once ('jpgraph/jpgraph.php');
require_once ('jpgraph/jpgraph_line.php');
require_once ('jpgraph/jpgraph_bar.php');

// connect to MySQL
$link = @mysql_connect('localhost','user','password');
@mysql_select_db('dbname');
$my_query = "SELECT * FROM monitoring order by recno";
$my_result = mysql_query($my_query);

$i = 0;
while ($my_row = mysql_fetch_array($my_result))
{
$timestamp[$i] = substr($my_row['timestamp'],11,5);
$http = $my_row['http'];
if ($http == 200)
{
$response[$i] = $my_row['load_time'];
$fail[$i] = "";
}
else
{
$response[$i] = "";
$fail[$i] = "5";
}

$i++;
}
// Width and height of the graph
$width = 960; $height = 320;

// Create a graph instance
$graph = new Graph($width,$height);

// Specify what scale we want to use,
$graph->SetScale('textlin');

// Setup a title for the graph
$graph->title->Set('Response times');

// Setup titles and X-axis labels
# $graph->xaxis->title->Set('(Timestamp)');
$graph->xaxis->SetTickLabels($timestamp);
$graph->xaxis->SetLabelAngle(-90);
$graph->xaxis->SetLabelAlign('center');
$graph->xaxis->SetFont(FF_ARIAL,FS_NORMAL,5);

// Setup Y-axis title
# $graph->yaxis->title->Set('(Response time)');

// Create the plot
$fail=new BarPlot($fail);
$resp=new BarPlot($response);
$fail->SetWidth(1.0);
$fail->SetNoFill();
$fail->SetPattern('5','black');

$resp->SetWidth(1.0);
// Add the plot to the graph
$graph->Add($resp);
$graph->Add($fail);
// Display the graph
$graph->Stroke();
?>


============oOo Update oOo============

Well I’m not quite sure what has happened but the chart is now showing a fantastic performance increase. I’m hoping it is going to last – especially as my Three account has already been credited with the full three months of discount.

Improved connection

I do know that many phone masts only have limited back haul. Given the rural location the mast I’m using may only have the equivalent of a home broadband ADSL connection so a single user may well have been able to saturate it enough to cause the issues I was getting. Maybe they were running torrents as although the current monitoring doesn’t cover 00:00 – 07:59 it did originally before I added the chart and there were problems all around the clock. Maybe they, or more likely their parents, got the bill and the activity has stopped permanently. I do so hope it was an outrageous one.



 Leave a Reply

(required)

(required)