BikeBiz Bicycle and cycling retail news
What follows is a long – very, very long – set of arguments and counter-arguments by some techie posters to the BikeBiz bulletin board.
The whole topic was pulled because it got too long and boring for non-techie readers, and was taking up a lot of space on the BBS.
However, if you’re want to know the ins and outs of static/fluid IP addresses, this is the article for you.
PS
Once the IP stuff is finished there’s another pulled topic for your entertainment. It’s all about guessing quotes from films, very off-subject.
Unique Users – Define Please
Author: singletrack (—.vip.uk.com)
Date: 04-02-02 15:09
Carlton,
Your stats are very impressive but they don’t add up.
You can’t take a daily count of Unique Users and multiply it by 31 to get a monthly total. What you are assuming by doing that is that those 650 users only log in once a month with a different 650 logging in each day.
At Singletrackworld we get an average of 1500 unique users a day but that equates to just 20,000 unique users a month. We know this because our stats monitoring software logs all IP addresses of our visitors over many months and compares this log with our visitors every day, thus elliminating any duplicate visitors.
It may not seem a big deal to most readers of this site but when we are trying to market advertising on our site many potential advertisers may start to wonder why BikeBiz seemingly gets more visitors per month.
Using your method we could claim nearly 50,000 visitors a month, but we don’t because this is a misleading figure.
Unless you track IP addresses over a period of at least a month you can’t make any claims to a monthly unique user count.
Reply To This Message
Re: Unique Users – Define Please
Author: singletrack (—.vip.uk.com)
Date: 04-02-02 15:39
Just noticed that you do say that not all your monthly figures add up to individuals but this still goes to show that there needs to be correct and standard use website statistical jargon.
Your monthly figure would be more correctly described as a ‘visit’ although that is still not totally correct since it is a figure born of simply multiplying a daily average. isn’t it?
I don’t have all the strict definitions to hand but perhaps Si can come in on that front and lay down some definitions and explanations of terminolgy.
When sites like ourselves are trying to sell advertising by using statistics like these it is pretty important that everyone is singing from the same hymn sheet and we all know what each figure actually means.
To us the only important figures are Unique Users, which equate as closely as possibly to actual readers – and Page Requests, since this can show how many times an advertisers banner or button is displayed.
Hands up who gets confused by website statistics?
For reference, here’s Singletrackworld’s stats for January…
Unique Users – 20,000* (Daily average 1500)
Page Requests – 419,000
File requests – 1,594734
*A glitch means we can’t be more accurate than 500 or so this month.
A Page Request is counted when one complete page of a site is ‘viewed’. A page can be made up of many files, hence the File Request figure is so high. File Requests are often refered to as ‘Hits’. A ‘hit’ does NOT correspond to a single visit by a reader as is a popular misconception. Be very dubious of any site that quotes its stats in terms of ‘Hits’.
Reply To This Message
Re: Unique Users – Define Please
Author: mj (—.innotts.co.uk)
Date: 04-02-02 16:21
Just to add to the confusion…
We run a web-enabled pc in the shop. Each day 3 or 4 different members of staff will log on to BikeBiz, from the same computer but at different times.
Will we be accessing through different (dynamic) IP addresses? Will we therefore be 4 unique users, or just 1 heavy user?
MJ
Reply To This Message
Re: Unique Users
Author: A. gent (—.server.ntl.com)
Date: 04-02-02 16:48
And don’t forget that most ISP’s give you a different IP address every time you log on.
Reply To This Message
Re: Unique Users – Define Please
Author: Roger (—.server.ntl.com)
Date: 04-02-02 17:13
Access logs only tell you what the server experienced and do not tell you much about visitor numbers.
Almost all ISP’s now have their own cache. This means that if I try to look at one of your pages and anyone else from the same ISP has looked at that page recently, the cache will have saved it, and will give it out to me without ever telling you about it. (This applies whatever my browser settings.) So hundreds of people could read your pages, even though you’d only sent it out once.
Let’s suppose a service provider introduces a better cache. Then access by their customers is on average faster, so users are more likely to stay around, and read more of your pages: but your server statistics are likely to fall, because more of their accesses are coming out of the cache.
Now let’s suppose in the interest of getting "better statistics", you make your pages uncacheable. So, access becomes
sluggish, users get fed up and go somewhere else. But because every access it being logged, your statistics go up although in effect your user activity is in fact lower.
The same goes for people who put a hit counters on their website. Waste of time. If you want to impress people then get a random number generator to pop a suitable figure onto your page.
So if someone is asking for your user visits then they have shown themselves to be rather clueless so you can tell them anything you want knowing full well that they cannot disprove you.
—
Roger
WP
Reply To This Message
Re: Unique Users
Author: singletrack (—.vip.uk.com)
Date: 04-02-02 17:19
Actually most don’t.
It’s about 25% so I have been told.
But yes, the whole website stats issue is a huge headache for businesses like us. The most we can push for is that everyone sings from the same hymn sheet as I have already said.
The most useful stat for us and probably any other website that seeks advertisers, is the ‘Unique User’ which is defined as a unique IP address (from an individual computer). Unfortunately dynamic IP addresses (Ones that change each time the computer connects to the internet) really screw things up and prevent us from being able to claim that a unique user corresponds exactly to an individual human being (reader). However, this is offset to a certain degree by the example given above where 4 users access a site from the same computer. A great deal of our traffic at Singletrackworld comes from users accessing the site from work. Many of these are accessing via their company network which results in lots of readers but a single IP address. We work on the basis that one generally counters the other and that our unique user figures correspond to individual readers. It’s just a rule of thumb though.
But even this figure can be manipulated, which is why I brought up the whole topic in response to Carlton’s news story. I’m not accusing Carlton of ‘manipulation’ (back pedalling like mad here but I think he has made an inaccurate extrapolation.
Unless you know to a good degree of accuracy how many times an average user logs on to the site in a month you can’t extrapolate a monthly Unique User figure from a daily one.
If daily figures are all a site has then that is all it should be quoting. Anything more is tantamount to guesswork.
Reply To This Message
Re: Unique Users – Define Please
Author: siwatts (—.cableinet.co.uk)
Date: 04-02-02 17:30
This is going to take some time to compile, so I’ll post on the issue later.
Sorry to disappoint for now!
Reply To This Message
Re: Unique Users – Define Please
Author: siwatts (—.cableinet.co.uk)
Date: 04-02-02 17:36
Very true for the average site Roger, but the majority of websites like this where the output is server generated are not cached by the majority of local ISP’s. The Proxy Cache’s employed by most ISP’s only cache pages with htm or html file extensions and image file types, stylesheets and other static files. Server parsed pages just don’t get cached, and if they do then the ISP is actually decreasing the efficiency of the web.
Reply To This Message
Re: Unique Users – Define Please
Author: carlton (—.in-addr.btopenworld.com)
Date: 04-02-02 17:45
I didn’t extrapolate.
Webalizer gives me Unique Visits per month, and average visitors per day.
Naturally, I will bow to anyone with greater knowledge of the Webalizer method of counting.
Reply To This Message
Re: Unique Users – Define Please
Author: Roger (—.server.ntl.com)
Date: 04-02-02 17:59
File extension plays no part in how the web pages are cached. ISP’s only cache what you tell them (or what they guess themselves). Caching is determined by the web author and server settings and ensures that complete cache control is achievable. If you want your server generated php pages to be cached then it’s a trivial task. People who disregard cache effects, or choose not to make their pages cache friendly only make their websites a pain to browse. By using correct cache control you make the web much *more* efficient and pleasurable to use.
Reply To This Message
Re: Unique Users – Define Please
Author: Roger (—.server.ntl.com)
Date: 04-02-02 18:01
Webalizer just tells you what the server pumped out.
Reply To This Message
Re: Unique Users – Define Please
Author: carlton (—.in-addr.btopenworld.com)
Date: 04-02-02 18:09
Having now had a chat with Si Watts, I have been pointed to which bit of Webalizer I should have been looking at.
So, BikeBiz.co.uk got 9573 visitors in January ie different IP addresses. And even this stat isn’t perfect. Bummer.
Anyway, story pulled.
Reply To This Message
How was january?
Author: sara (213.78.66.—)
Date: 04-02-02 18:19
We have just had the best January ever by a long way. Is this just a fluke or has the trade been extra good. If it has been good has anyone any theories as to why?
Reply To This Message
Re: Unique Users – Define Please
Author: siwatts (—.cableinet.co.uk)
Date: 04-02-02 19:35
Point taken on board after a little research into the subject (obviously caching isn’t on my list of required knowledge and I bow to you superior intellect)
http://www.oreilly.com/…/ch05.html
Reply To This Message
Re: Unique Users – Define Please
Author: siwatts (—.cableinet.co.uk)
Date: 04-02-02 19:41
OOOH! I’ve been given cart blanche to do a long post! yippee!. If only I had time to do it!
Pauls server (where BikeBiz is hosted) runs, to my knowledge, Webalizer which is the same stats package I operate on my server.
Some limitations of Webalizer. It is not user specific, ie it does not use cookies to track visitors (by parking a small text file called a cookie on the visitors machine). All it does is to take the RAW SERVER LOGS and run a set of pre-defined queries on them.
All web servers, regardless of whether they are UNIX or WINDOWS based keep logs of access to their file system. Every time a computer anywhere on the internet asks for content located on that particular web server, there is a transaction which takes place between those 2 computers. The easiest way to try and describe the transaction is to imagine it as a conversation;
Hi I’m PC. have you got FOOBAR
Hi I’m SERVER. Yes I’ve got FOOBAR but I need you to give me your credentials.
Here are my credentials, can I now have FOOBAR
Thank you for your credentials, here is FOOBAR
In this conversation, FOOBAR is a single file, generally in the form of a web page or image etc. Credentials which get returnmed to the server are; the current IP address of the requesting computer, the language (protocol) with which the transfer needs to take place, the browser type and the operating system. There are also some other more detailed entries but they’re not really relevant this thread. Here’s a sample output from a raw log file;
20.138.1.143 – – [27/Jan/2002:04:15:07 +0000] "GET HTTP/1.0" 304 – "-" "Mozilla/3.01 (compatible;)"
20.138.1.143 – – [27/Jan/2002:04:22:18 +0000] "GET /nav.js HTTP/1.0" 304 – "-" "Mozilla/3.01 (compatible;)"
12.72.25.153 – – [27/Jan/2002:06:11:45 +0000] "GET /whatbike.htm HTTP/1.1" 200 15425 "http://google.yahoo.com/…/query?p=motor+bikes+for+commuting&hc=0&hs=0" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT; AT&T CSM6.0; AT&T CSM 6.0)"
12.72.25.153 – – [27/Jan/2002:06:11:46 +0000] "GET /style.css HTTP/1.1" 200 2921 "http://www.whycycle.co.uk/whatbike.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT; AT&T CSM6.0; AT&T CSM 6.0)"
12.72.25.153 – – [27/Jan/2002:06:11:48 +0000] "GET /images/banner.jpg HTTP/1.1" 200 19327 "http://www.whycycle.co.uk/whatbike.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT; AT&T CSM6.0; AT&T CSM 6.0)"
12.72.25.153 – – [27/Jan/2002:06:11:48 +0000] "GET HTTP/1.1" 200 43 "http://www.whycycle.co.uk/whatbike.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT; AT&T CSM6.0; AT&T CSM 6.0)"
Once the browser has requested the page, it subsequently goes back and requests all the other files it needs to complete that page. Each one of these transactions is called a "HIT"
The web server records all of these transactions in it’s log files, along with the time at which the request was made and the other pieces of proprietary information it can collect.
Stage one complete.
Now deciphering the raw weblog is a mammoth task to undertake by hand, so most of us who have servers have server programs to take care of the analysis for us, and in the case of my server (and as far as I know Paul’s) that program is Webalizer. (come on Roger, you know about this stuff too!)
Each one of the sites hosted on my server has access to the graphical output that webalizer generates (it happens once a day at 4am) via a normal web browser. BUT that output is based ONLY on the content of the server logs.
When they log into stats they are greeted by an overview screen which has a simple bar chart and the months listed with each of the following "entities" as a numeric figure;
Daily Avg
Hits
Files
Pages
Visits
Monthly Totals
Sites
KBytes
Visits
Pages
Files
Hits
My best interpretations, and those which I give to my clients.
In both the monthly and daily average totals, the term HITS refers ONLY to how many items were requested from the server, regardless of what type they were.
FILES refers to the number of Files served, again regardless of type, whereas PAGES refers specifically to the number of hypertext protocol documents, be they html, php, asp, jsp etc etc. VISITS relates to the IP address of any PC requesting content, and is probably the best determination on a daily level for the number of unique visitors a site is receiving PER DAY. But on a monthly level, this is wholly innacurate as a means to determing unique users, since the VISITS relates to different IP addresses in a given time scale. In the case of my server, it terminates a VISIT once there has been a period of inactivity from the IP address greater than 30 minutes, thus the same person could theoretically be counted twice if they left more than half an hour between their last request of the first visit and the first request of the last visit. SITES would be the only real indication on a monthly basis of how many unique users the site has received, since it is the number of different IP addresses requesting content from that site. In terms of the monthly totals though, VISITS does give us a reasonable indication of how many visits, whether unique or repeat that a site received. KBYTES is simply the amount of data transferred.
The limiting factors in all of this are;
Dynamic IP addresses assigned by ISP’s. Unless you are using an ADSL or Cable Modem connection (ie you use a dial-in modem, either ISDN or 56K) then each time you make a connection to the internet through your ISP you are assigned a dynamic IP address. Every user on the internet has to be unique, and this is based upon the IP number protocol, a series of 4 octets of numbers, eg 218.356.689.101. Because there are a finite number of these IP numbers, ISP’s retireve the IP number from you when you log off and put it back into a pool to be handed out to the next user when required. As a result, it is more likely for you to have a totally different IP address each time you use the internet than it is for you to have the same one, thus when you revisit a server it cannot differentiate between you as a repeat visitor with a new IP address and as a new visitor, you are simply classed as a new visitor.
Web Caching by ISP’s. As mentioned by Roger, the majority of the worlds ISP’s each make use of various Web Caches located at strategic positions within the internet infrastructure around the world. The idea of the cache is that once a request for a web page has been made by a PC, a copy of it is stored within the cache such that future requests for that page can be served by the cache instead of by the actual server. In most cases these caches are cleared every 24 to 48 hours (and in some cases more frequently) but there is the possibility that your server doesn’t actually serve out the web pages to all the people who see them. The idea generally is to speed up web responses by shortening the routes the data has to transferred over.
So what’s the Crux?
Whilst Webalizer is suitable to give you an indication of what is happening within any site, it should by no means be taken as being authoratative. To gain the kind of "real statistics" you’d need to base advertising pitches on, you’d need to look to something like WebTrends statistical reporting which uses cookies.
The following information is taken from http://www.geocities.com/…/unixlinux.html and is NOT my wordings, but is taken from an authoratative site relating to Webalizer.
Hits
Any request made to the server which is logged, is considered a ‘hit’. The requests can be for anything… html pages, graphic images, audio files, cgi scripts, etc… Each valid line in the server log is counted as a hit. This number represents the total number of requests that were made to the server during the specified report period.
Files
Some requests made to the server, require that the server then send something back to the requesting client, such as a html page or graphic image. When this happens, it is considered a ‘file’ and the files total is incremented. The relationship between ‘hits’ and ‘files’ can be thought of as ‘incoming requests’ and ‘outgoing responses’.
Pages
Pages are, well, pages! Generally, any HTML document, or anything that generates an HTML document, would be considered a page. This does not include the other stuff that goes into a document, such as graphic images, audio clips, etc… This number represents the number of ‘pages’ requested only, and does not include the other ‘stuff’ that is in the page. What actually constitutes a ‘page’ can vary from server to server. The default action is to treat anything with the extension ‘.htm’, ‘.html’ or ‘.cgi’ as a page. A lot of sites will probably define other extensions, such as ‘.phtml’, ‘.php3’ and ‘.pl’ as pages as well. Some people consider this number as the number of ‘pure’ hits… I’m not sure if I totaly agree with that viewpoint. Some other programs (and people 🙂 refer to this as ‘Pageviews’.
Visits
Whenever a request is made to the server from a given IP address (site), the amount of time since a previous request by the address is calculated (if any). If the time difference is greater than a pre-configured ‘visit timeout’ value (or has never made a request before), it is considered a ‘new visit’, and this total is incremented (both for the site, and the IP address). The default timeout value is 30 minutes (can be changed), so if a user visits your site at 1:00 in the afternoon, and then returns at 3:00, two visits would be registered. Note: in the ‘Top Sites’ table, the visits total should be discounted on ‘Grouped’ records, and thought of as the "Minimum number of visits" that came from that grouping instead. Note: Visits only occur on PageType requests, that is, for any request whose URL is one of the ‘page’ types defined with the PageType option. Due to the limitation of the HTTP protocol, log rotations and other factors, this number should not be taken as absolutely accurate, rather, it should be considered a pretty close "guess".
Sites
Each request made to the server comes from a unique ‘site’, which can be referenced by a name or ultimately, an IP address. The ‘sites’ number shows how many unique IP addresses made requests to the server during the reporting time period. This DOES NOT mean the number of unique individual users (real people) that visited, which is impossible to determine using just logs and the HTTP protocol (however, this number might be about as close as you will get).
KBytes
The KBytes (kilobytes) value shows the amount of data, in KB, which was sent out by the server during the specified reporting period. This value is generated directly from the log file, so it is up to the web server to produce accurate numbers in the logs (some web servers do stupid things when it comes to reporting the number of bytes). In general, this should be a fairly accurate representation of the amount of outgoing traffic the server had, regardless of the web servers reporting quirks. (Note: A kilobyte is 1024 bytes, not 1000 bytes)
Reply To This Message
Re: Unique Users – Define Please
Author: scmurray (—.dial.btopenworld.com)
Date: 04-02-02 21:37
Actually, I stick cache invalidation code in to each page too so that dynamically generated pages don’t become ‘old’ and the user gets an old page from an ISP’s cache.
Reply To This Message
Re: Unique Users – Define Please
Author: scmurray (—.dial.btopenworld.com)
Date: 04-02-02 21:45
The only problem with cookies being that some people really object to them on privacy grounds. ie. They don’t like being counted as a marketing statistic as they show just exactly when they they connected, for how long, who they were, what they did…..
Reply To This Message
Re: Unique Users – Define Please
Author: Roger (—.server.ntl.com)
Date: 04-02-02 22:27
Better still for dynamic news sites is to set the expiry date to say, 5 or 10 minutes from the current time. This makes flicking forward and back through a website much faster and saves a trip back to the originating server each time I hit the forward/back button, or revisit a previously seen page.
On the WP site I expire the pages at the end of the current day. I should really expire the fairly static ones a lot longer and keep the newsy ones a bit shorter.
HTTP Headers from the wp site …
Date: Mon, 04 Feb 2002 21:23:54 GMT
Server: Apache/1.3.19 (Unix) PHP/4.0.6 FrontPage/5.0.2.2510
Content-Type: text/html
Expires: Mon, 04 Feb 2002 23:59:59 GMT
X-Powered-By: PHP/4.0.6
Reply To This Message
Re: Unique Users – Define Please
Author: siwatts (—.cableinet.co.uk)
Date: 04-02-02 22:58
In total agreement Shaun, I hate sites that serve cookies up to me particularly from 3rd party adverts etc, they simply get rejected by default.
What I was, however, trying to say was that despite the fact that programs which analyze raw server logs give an indication of what is happening visitor wise for a site, they can never be judged as being authoratative,unlike systems which use cookies (which have a far higher accuracy level). I don’t use cookies for that specific reason, I don’t see the need for any of my sites to use cookies for that reason, and I similarly would not like to be tracked via that method, but still the vast majority of users haven’t even a clue what a cookie is, what it does etc etc etc and as such run around the internet accepting cookies from any and every website they visit.
Reply To This Message
Re: Unique Users – Define Please
Author: scmurray (—.btinternet.com)
Date: 04-02-02 23:26
The time set alters depending on the page type in the caching code but anyway, I’m sure we’re getting WAY off topic and too boring now.
Reply To This Message
Re: Unique Users – Define Please
Author: scmurray (—.btinternet.com)
Date: 04-02-02 23:30
Especially the ones that come from advertising servers or multiple sites which share cookies across sites. Yuk.
I think basing the number of visitors on IP addresses is a good enough rule of thumb and is equally less invasive to the user.
Reply To This Message
Re: Unique Users – Define Please
Author: Roger (—.server.ntl.com)
Date: 04-02-02 23:44
…providing the IP does not resolve to webcacheb05a.cache.pol.co.uk or similar which could represent between 1 and 3 million Freeserve users.
Reply To This Message
Re: Unique Users – Define Please
Author: siwatts (—.cableinet.co.uk)
Date: 05-02-02 00:05
Snap. Shaun don’t tell me we actually agree about something???
I happen to also believe that the reports generated by Webalizer and other such Raw server log analysis programs are sufficient in the majority of cases. BUT where a site is largely dependent upon as near accurate stats reporting as possible because its existence depends upon Ad revenue based on quoted visitor figures, there is sufficient inaccuracy in webalizer et al to require a more accurate (possibly cookie driven) solution. Having said that, it is likely that overall Webalizer et al will be reporting the statistics short rather than exagerating them. The only true statistic it is impossible to record currently being unique visitors.
As was said by Mark at the beginning of this tech-head thread, Hits aren’t visitors, but even visits aren’t necessarily unique, only the IP address used to create that visit.
lets imagine 1 user, who logs on to bikebiz 5 times a day, 9am, 11am, 1pm, 3pm and 5pm via dial up modem. As far as the server is concerned, that user is 5 different unique sites (since the chances are that each connection will have resulted in a different IP address) assuming that webalizer is being run in it’s default settings, ie session length of 30 mins. This user would create, in the stats, 5 visits and 5 sites. Now look at me, using a cable modem connection (ie permanently connected and therefore a permanent IP address) making the same log-ins to bike biz. I would create 5 visits, but only 1 site, but my "site" would be hgher up in the stats since my site has all the pages etc logged against it for every day of the month ad finitum (well for as long as it takes blueyonder to reposses my current IP and re-issue me with a new one, which as it’s 11 months and it hasn’t changed yet seems unlikely), whereas the dial in users pages etc are split against his 5 visits. This is why Carlton sees me in his top 30 "sites". I pop in to bikebiz almost every hour on virtually every day of the week, primarily because I’m sat behind this thing far too long each day . What I’m trying to say is that even the number of IP addresses (sites) could be a little way off mark since the one user mentioned above could in theory generate 30 x 5, ie 150, "sites" in the period of 1 month. In the case of most retail type sites etc where a daily visit or multiple daily visits is not the norm, then yes the stats are pretty accurate, but in the case of bikebiz they could be quite dramatically off-mark.
I’ll make no comments as to the validity or otherwise of Mark’s quoted STW stats as I don’t know what server stats analysis package they are employing and unless it is webalizer I have no knowledge of it anyway.
Reply To This Message
Re: Unique Users – Define Please
Author: siwatts (—.cableinet.co.uk)
Date: 05-02-02 00:39
Mark, just to quantify your stats, these are genuine figures lifted from what is the busiest of my sites;
Monthly totals for Jan 2002.
Sites: 2738
KBytes: 913793
Visits: 5916
Pages: 32565
Files: 181781
Hits: 244912
And from this I would assume the following;
The site received visits from 2738 unique IP addresses in January (possibly equivalent to 2738 unique visitors although not definitely)
The server sent out 0.87GB of data on behalf of this site.
A total of 5916 distinct visits took place, from those 2738 sites implying that each visitor on average made 2.16 visits
A total of 32565 pages were served out, implying that each visitor on average viewed 12 pages within the site
Files and Hits are all but irrelevant.
By my reckoning (quick) that makes STW 10 times busier than the site for which I quoted figures (I’ll not say which site without the owners permission)
Reply To This Message
Re: Unique Users – Define Please
Author: franky (—.server.ntl.com)
Date: 05-02-02 00:45
Bloody generated sites! HAND CODERS FOREVER!!!
Reply To This Message
Re: Unique Users – Define Please
Author: Roger (—.server.ntl.com)
Date: 05-02-02 01:13
Remember there are 2 types of IP address, the client IP (my computer on the Net) and the host IP. Apache server logs use host IP which is something along the lines inktomi2-man.server.ntl.com which does not represent a local dialup user, rather the host that they are connected to.
Every time I connect to the net via a modem I am seen as inktomi2-man.server.ntl.com
If you see a numeric IP then it’s still the hostname (DNSlookup could be off or not available).
Reply To This Message
Re: Unique Users – Define Please
Author: singletrack (—.vip.uk.com)
Date: 05-02-02 01:21
We use Analog 5.03 apparently.
Reply To This Message
Re: Unique Users – Define Please
Author: siwatts (—.cableinet.co.uk)
Date: 05-02-02 01:36
OK so we’re getting pedantic and a little beyond the scope of what was originally requested but here’s the full info for your viewing pleasure, and of course, as ever Roger IS correct.
Taken from http://httpd.apache.org/…/logs.html
standard log entry format in CLF (common on virtiually all apache webservers.
127.0.0.1 – frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb. gif HTTP/1.0" 200 2326
127.0.0.1 (%h)
This is the IP address of the client (remote host) which made the request to the server. If HostnameLookups is set to On, then the server will try to determine the hostname and log it in place of the IP address. However, this configuration is not recommended since it can significantly slow the server. Instead, it is best to use a log post-processor such as logresolve to determine the hostnames. The IP address reported here is not necessarily the address of the machine at which the user is sitting. If a proxy server exists between the user and the server, this address will be the address of the proxy, rather than the originating machine.
Reply To This Message
Re: Unique Users – Define Please
Author: darth vader (—.btinternet.com)
Date: 05-02-02 09:16
Wonder whose site that is? (Si check Febs, yes i know its only five days but…)
Reply To This Message
Re: Unique Users – Define Please
Author: mike poyzer (—.server.ntl.com)
Date: 05-02-02 09:21
As a mere bicycle dealer I have to say that this thread has broken some records in the field of boredom. The main question I have is has Si got a secretary or is he just a fast typer. His posting no. 20 which I am saving for my next holiday read was only posted six minutes after his previous one.
Reply To This Message
Re: Unique Users – Define Please
Author: franky (—.server.ntl.com)
Date: 05-02-02 09:43
Yes, Si’s always had supple wrists! For some reason even before he was in the IT industry.
Reply To This Message
Re: Unique Users – Define Please
Author: steve b (—.btinternet.com)
Date: 05-02-02 09:50
Roger
I know @!#$ about the web so this may be a very stupid question if this was the case how come each time I go to STW the forum page updates? If your theory is correct surely I would just get the same page without new posts.
I hit the forum page of STW about 5 times a day and it is always different.
Steve
Reply To This Message
Re: Unique Users – Define Please
Author: siwatts (—.cableinet.co.uk)
Date: 05-02-02 10:04
LOL!
Mike, for your information, I typed the "Long One" into notepad and then cut and pasted it into the window once completed. Whilst I can type pretty quickly, that would, even for me be just a little byond the scope of reality!
Reply To This Message
Re: Unique Users – Define Please
Author: siwatts (—.cableinet.co.uk)
Date: 05-02-02 10:26
I did however forget to include the following gem of information, with regards ISP proxies. For all my sites where the owner is active on their own site, I see a common pattern that begins to emerge. I can SEE my own IP address quite distinctly within the stats, although it is the IP address of the proxy my connection is made through, and yet there is no similar one recurring IP address. But what can be seen, particularly within sites where I have been doing work and the owner is taking more than usual looks, is the existence of a large number of "similar" IP addresses, eg
194.117.133.84
213.122.161.101
213.122.136.73
213.122.82.4
213.122.124.49
213.122.144.129
213.122.75.78
213.122.87.218
213.122.154.179
213.122.107.147
68.10.57.163
213.122.7.43
172.188.111.108
213.122.106.63
213.1.183.40
172.188.153.86
62.254.0.7
12.4.195.34
193.131.85.1
62.7.69.103
67.201.35.75
213.122.16.67
213.123.193.4
62.255.0.4
153.103.2.1
213.122.1.154
193.113.57.163
212.187.157.202
64.69.187.210
62.255.16.84
213.122.0.0 – 213.122.255.255 is operated as a netblock by BT which corresponds to the connection used by the "owner" of this site. hense there are a large numer of entries starting 213.122.X.X. The top listed IP is mine and exists in all my sites.
So what am I trying to say? Well, although there is truth in the fact that the IP address quoted in raw weblogs cannot be directly attributed to an absolute unique user, they are about as close an indication as it possible to get. The proxies that actually do the requesting still have to use an IP address and as there are so many unique IP’s in each netblock it is quite possible, and pretty likely that each request from the proxy will come with a different IP.
Reply To This Message
Re: Unique Users – Define Please
Author: roger (—.server.ntl.com)
Date: 05-02-02 10:48
%h in the log format records the users host machine and not their personal IP. The host IP can be numeric, or more commonly converted to something more readable by use of a dns lookup. For example, below is a wedge from my own log. It shows the host of the originating requester and this can either be a host within a company, a users ISP, or a proxy server acting as a host. The numeric address seen within the log is because a lookup failed. A dns lookup adds hardly any extra processing time yet yields more valuable information.
In your examle above, 127.0.0.1 is refered to as "local host" and can equally be replaced with http://localhost/
55 trek5.sv.av.com
54 web5.th.netweaver.net
53 193.113.48.17
52 webcacheb05a.cache.pol.co.uk
52 inktomi4-lee.server.ntl.com
39 193.131.85.1
39 webcachew02b.cache.pol.co.uk
38 213.38.201.226
31 www.bigorangebob.com
30 212.124.239.62
25 host217-34-77-150.in-addr.btopenworld.com
23 ldsas19-83-134-37.cw-visp.com
22 proxy3.bournemouth.ac.uk
21 smh-renal5.smh.bris.ac.uk
19 public1-with2-4-cust66.manc.broadband.ntl.com
I invite Paul Rundle into this discussion. Paul is a webmaster and also a server administrator. It’s important for everyone to know the valu of web access logs and what information can be deduced.
Reply To This Message
Re: Unique Users – Define Please
Author: roger (—.server.ntl.com)
Date: 05-02-02 10:54
I should add that the numbers adjacent to the hosts were the number of readable web pages my server sent. I wrote my own log analysing software.
Reply To This Message
Re: Unique Users – Define Please
Author: roger (—.server.ntl.com)
Date: 05-02-02 10:59
The author of the site has decided that the pages should not be cacheable and hence the pages are not stored in your local cache (your PC) and neither in any proxy cache. So you are obliged to retrieve a brand new page everytime you visit the site. This ensures that each page is upto date, the downside being that browsing the site is often much slower.
Reply To This Message
Re: Unique Users – Define Please
Author: steve b (—.btinternet.com)
Date: 05-02-02 11:00
Nah, he’s covering up, he is just a really big wanchor!
Steve
Reply To This Message
Re: Unique Users – Define Please
Author: steve b (—.btinternet.com)
Date: 05-02-02 11:01
So I’m told!
Steve
Reply To This Message
Re: Unique Users – Define Please
Author: siwatts (—.cableinet.co.uk)
Date: 05-02-02 11:29
Roger I’m fully aware that 127.0.0.1 is localhost. The example output line was not lifted from any of my site stats, simply from the paache example pages themselves, as is the associated text, a fact which I clearly stated.
In reality the lookup to a hostname adds additional overheads to the server when logs are rotated and as such is, in my opinion, an unsuitable usage of my servers processing resources and as such has been disabled. Being able to see what host a visitor comes from is to me pretty much irrelevant because it still goes no further toward uniquely identifying an individual. It is a waste of system resources in much the same way that building static pages (ie those that don’t use database fed content) and forcing the server to parse them because they’ve been given php or other server parsed file extensions is. Similarly, forcing pages to not be cached when they are unlikely to change more than once in any 48 hour period is a waste of internet resources in the whole, in my opinion.
Reply To This Message
Re: Unique Users – Define Please
Author: siwatts (—.cableinet.co.uk)
Date: 05-02-02 11:56
These are from a different site (which I hadn’t checked on yesterday and would subsequently appear to be the busiest site) on the same server;
Monthly totals for Jan 2002.
Sites: 3693
KBytes: 991146
Visits: 7498
Pages: 38047
Files: 193302
Hits: 245306
And from this I would assume the following;
The site received visits from 3693 unique IP addresses in January (possibly equivalent to 3693 unique visitors although not definitely)
The server sent out 0.945GB of data on behalf of this site.
A total of 7498 distinct visits took place, from those 3693 sites implying that each visitor on average made 2.03 visits
A total of 38047 pages were served out, implying that each visitor on average viewed 10 pages within the site.
Once again, due to the limitations of raw server logs and the programs that analyse them, these are all simple estimates, NOT hard fact relating to unique users/visits etc.
Reply To This Message
Re: Unique Users – Define Please
Author: roger (—.server.ntl.com)
Date: 05-02-02 12:08
Server logs are used to glean information about who/what is requesting pages and one important statistic for me is which organisations are taking an interest.
Rather than see something like 123.456.789.000 it is much better to see something like bath.futurenet.co.uk which tells me that one or more of the staff from probably mbuk/WhatMTB/C+ have been taking an interest. Similarly it gives you an idea of how many accademic sites are requesting pages and which uni/college/school they are comming from, and what private companies are looking at your site, and not forgetting the proxies (which tell me little).
The server overhead is negligiable and all hosting companies I have dealt with provide the dns lookup as standard.
Anyway, we have obviously chosen different strategies for using the data in our server logs and of setting rules for contolling cache. Just bear in mind that trying to determine the count of individual users is at best a guess.
Reply To This Message
Re: Unique Users – Define Please
Author: Ben (—.btinternet.com)
Date: 05-02-02 12:52
I have a choice of Analog or Webalizer. Both produce some very pretty results, but for the same period they never agree! Therefore I deduce that:
1. Analysing logs is a bit hit-and-miss and very open to interpretation. Lies, damned lies and weblogs.
2. Unless the numbers change drastically, I don’t really care. What matters is how many of these real or theoretical visitors actually buy anything…
Reply To This Message
Re: Unique Users – Define Please
Author: franky (—.server.ntl.com)
Date: 05-02-02 12:54
EVERYONE STOP!!!!!!!!!
Let Roger have the last word he so desperatly wants.
I’m in the IT industry and even I’m bored of this. The thread is in danger of taking up the whole front page of the bbs! (1024×768 and no I dont want a debate about average screen sizes!)
Go out ride a bike (or your wife! or someone elses wife!) anything but typing frigging IP address’s!
Reply To This Message
Re: How was january?
Author: franky (—.server.ntl.com)
Date: 05-02-02 13:14
I’d like to reply to your thread as it’s burried in the middle of the most boring conversation ever and I felt sorry for it.
I’m not an IBD but I do know from talking to our clients (I cant name anyone without their permission) that a lot of them have had a fantastic January too.
One shop took about 5 £3k bike orders over the phone in the 2 days I was in and their not even a specialist mail order firm
Another who’s just a normal IBD too (very good one tho) reckons they’ve had an amazing month too although it’s just started to die off a little.
I dont know why its like this, but I guess it cant be bad for business (it is for us I suppose tho as we’re only busy when you shops are dead!)
Franky
Reply To This Message
Re: How was january?
Author: darth vader (—.dial.btopenworld.com)
Date: 05-02-02 13:25
What a good idea! (stopping or ‘riding’ i will leave to your imagination)
Reply To This Message
Re: How was january?
Author: darth vader (—.dial.btopenworld.com)
Date: 05-02-02 13:27
that was meant to be on the end of Franky’s message but its not?
Reply To This Message
Re: Unique Users – Define Please
Author: roger (—.server.ntl.com)
Date: 05-02-02 13:52
I don’t want the last word. Any posts were made to correct errors and misconceptions. The thread was opened up to discuss the definition of unique users. If you are not interested in this topic the don’t read the thread.
I have no more to say on the http protocol.
Take what you want and leave the rest. Choice is yours, choose whatever version of truth you want.
PS. I don’t ride bikes and do not possess one.
Cheers
Reply To This Message
Re: Unique Users – Define Please
Author: franky (—.server.ntl.com)
Date: 05-02-02 14:54
WORD!
(ps: if you dont own a bike, I did suggest 2 other alternatives!)
FILM TITLE QUIZ:
New Topic | Go to Top | Go to Topic | Threaded View | Search
Newer Topic | Older Topic
Your only supposed to blow the bloody doors off! (trade only version!)
Author: steve b (—.dial.btopenworld.com)
Date: 05-02-02 13:26
This is a thread that has run a few times on STW so figured it was time to try a "trade only" version!
Post up movie one liners and see who get’s it.
I’ll start with title of the post (a real easy one!).
Steve
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: UFO (—.core.plus.net)
Date: 05-02-02 13:29
"what we got here is a failure to communicate"
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: siwatts (—.cableinet.co.uk)
Date: 05-02-02 13:30
Is it from Gone with the Wind?
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: steve b (—.dial.btopenworld.com)
Date: 05-02-02 13:30
UFO
Thats not a line from a movie but a strapline to the bike industry init?
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: jjweiss (159.50.96.—)
Date: 05-02-02 13:31
is business that quiet ???
Ans : italian job.
How about ……. Snakes, why does it always have to be snakes ?
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: siwatts (—.cableinet.co.uk)
Date: 05-02-02 13:32
Indiana Jone, dunno quite which one though
Next;
Ah so you’re the famous pussy galore
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: darth vader (—.dial.btopenworld.com)
Date: 05-02-02 13:33
not that every body is sat in front of this bbs
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: jjweiss (159.50.96.—)
Date: 05-02-02 13:35
Bond, who know’s which one, there have been about 278 of them.
show me the money.
the truth, you can’t handle the truth.
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: siwatts (—.cableinet.co.uk)
Date: 05-02-02 13:35
"He’s not coming out, he’s been a very naughty boy"
or
"Half a Dinari for my life story… Blummin cheapskate"
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: jjweiss (159.50.96.—)
Date: 05-02-02 13:36
the best film ever made ….. Life of Brian.
and another from the same stable … We are the Knights who say Ni !
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: siwatts (—.cableinet.co.uk)
Date: 05-02-02 13:37
That’ll be the Holy Grail then!
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: steve b (—.dial.btopenworld.com)
Date: 05-02-02 13:37
Si,
Toooooooo easy.
Life.
"Your a big man, but with me it’s a job".
Has a connection with the title line.
Steve
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: siwatts (—.cableinet.co.uk)
Date: 05-02-02 13:38
Your mother was a hamster and your father smelled of elderberries.
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: siwatts (—.cableinet.co.uk)
Date: 05-02-02 13:40
Dunno.
"Whatever happens, just remember Don’t lose your head"
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: darth vader (—.dial.btopenworld.com)
Date: 05-02-02 13:43
Try and type the answers and then theres 5 more messages up whats going on?
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: jjweiss (159.50.96.—)
Date: 05-02-02 13:44
the … what we have here is a failure to communicate …
was from Cool Hand Luke, and it wasnt Paul Newman.
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: jjweiss (159.50.96.—)
Date: 05-02-02 13:52
jack carter … Get carter ….
and he actually said ….
"You’re a big man, but you’re out of shape. With me it’s a full time job."
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: steve b (—.dial.btopenworld.com)
Date: 05-02-02 13:58
JJwiess
Ya ya, I was typing the quick version!
how about-
"Our moto is more human than human"
I put this one on the recent STW and it was got real quick but I rekon it was a fluke.
Steve
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: jjweiss (159.50.96.—)
Date: 05-02-02 14:06
Blade Runner , Tyrell …..
Next.
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: steve b (—.dial.btopenworld.com)
Date: 05-02-02 14:08
JJwiess
well done that man!
Right then…..
Not a line from a movie but a strap line-
"They came, they saw, they did a little shopping!"
Steve
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: jjweiss (159.50.96.—)
Date: 05-02-02 14:10
whilst I ponder on that strap line (what is a strap line ??) here’s one for you …
Let me put it this way, Mr. Amer. The 9000 series is the most reliable computer ever made. No 9000 computer has ever made a mistake or distorted information.
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: steve b (—.dial.btopenworld.com)
Date: 05-02-02 14:13
Hummm,
Thinking.
Back soon.
Steve
BTW- a strap line is the bit they put under the film title in the trailers.
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: jjweiss (159.50.96.—)
Date: 05-02-02 14:18
havent a clue …..
but to guess .. something like Mars Attacks ? or
one of the National Lampoon vacation films.
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: steve b (—.dial.btopenworld.com)
Date: 05-02-02 14:28
Morons from outer space.
Still thinking on yours.
Steve
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: Ben (—.btinternet.com)
Date: 05-02-02 14:31
2001 – you’re a bit late
How about "Would you like to play a game?"
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: jjweiss (159.50.96.—)
Date: 05-02-02 14:35
while youre thinking on that one , try this easy , both from the same film …
Damn FBI don’t respect nothin’
Leave the gun. Take the cannoli
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: jjweiss (159.50.96.—)
Date: 05-02-02 14:37
damn, I know this ….. but ….
this must appear in a million films ….
Jumanji.
Excorcist.
Bill & Ted
The Game (!!)
Chuckie films .
Am I close ???
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: Ben (—.btinternet.com)
Date: 05-02-02 14:38
It’s spoken by a computer – does that help?
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: jjweiss (159.50.96.—)
Date: 05-02-02 14:41
is this 2001 also ?
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: Ben (—.btinternet.com)
Date: 05-02-02 14:45
Nope. Another clue? "Global Thermonuclear War"
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: siwatts (—.cableinet.co.uk)
Date: 05-02-02 14:46
War Games with Matthew Broderick the computer being the WOPR
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: jjweiss (159.50.96.—)
Date: 05-02-02 14:47
bummer, I had that.
bit too obscure for me without the clues.
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: siwatts (—.cableinet.co.uk)
Date: 05-02-02 14:49
"How F***cking hard can he be to kill, he’s only a frigging cook"
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: Ben (—.btinternet.com)
Date: 05-02-02 14:51
Under Siege. How about: "Hey Marvin, wanna earn a buck?"
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: siwatts (—.cableinet.co.uk)
Date: 05-02-02 14:56
"Hold the line, stay with me, and if you find yourself riding in green fields with the sun on your back, do not be troubled"
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: jjweiss (159.50.96.—)
Date: 05-02-02 14:56
Who knows ?
At least we seem to have killed off that coma inducing thread on ‘hit’ rates etc.
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: jjweiss (159.50.96.—)
Date: 05-02-02 14:58
Biggus Dickus, or was it Maximus Tothelimitus in Gladiator.
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: Ben (—.btinternet.com)
Date: 05-02-02 15:00
Though we’re probably doing Carlton’s hit rates a world of good!
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: jjweiss (159.50.96.—)
Date: 05-02-02 15:02
your last message has acheived the award of pushing everything else off the page.
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: Ben (—.btinternet.com)
Date: 05-02-02 15:05
Has it? I can still see all the way down to the statistics posts!
If that one was too hard, how about "In space, no-one can hear you scream."
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: jjweiss (159.50.96.—)
Date: 05-02-02 15:19
I have a small screen I got it wrong, so can I.
Alien.
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: franky (—.server.ntl.com)
Date: 05-02-02 16:52
Try this one.
"Let him fire me, If he does, I’ll thank him for it. Who ever she was would have…….."
cant finish it off as it would make it too easy!
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: steve b (—.dial.btopenworld.com)
Date: 05-02-02 17:07
Jeees,
I go away for a few hours and look what happens!
JJweiss- Give up and that earlier one.
Ben- Hitchhikers and Aliens
Steve
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: franky (—.server.ntl.com)
Date: 05-02-02 17:24
"Show me the money" = Jerry Maguire – Tom Cruise
"The Truth. You cant handle the truth" – forgotton the film name but it was said by Jack Nicholson to Tom Cruise regarding "the code red"
Obviously a Tom cruise fan in the ranks!
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: franky (—.server.ntl.com)
Date: 05-02-02 17:27
oh and the Bond film was Goldfinger (Pussy Galore – played by Honour Blackmen AKA the first Avengers girl prior to Dianna Rigg who preceeded Joanna Lumly)
Christ I’m sad.
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: franky (—.server.ntl.com)
Date: 05-02-02 17:28
I thought it was "Your a big man but your out of shape" said by Michal cane in Get carter thus the connection to the itallian jobs "Your only supposed to blow the bloody doors off"
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: franky (—.server.ntl.com)
Date: 05-02-02 17:29
oops sorry J, didnt see your reply when I just posted the one above!
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: franky (—.server.ntl.com)
Date: 05-02-02 17:30
Is it BiCentenial man?
Havnt seen the film but I think it might be appropriate
Reply To This Message
Re: Your only supposed to blow the bloody doors off! (trade only version!)
Author: franky (—.server.ntl.com)
Date: 05-02-02 17:31
is it "SHOPPING" that dodgy C4 film
