Current User: Guest Login
Please consider registering


Lost Your Password?

Search Forums:


 






Minimum search word length is 4 characters – Maximum search word length is 84 characters
Wildcard Usage:
*  matches any number of characters    %  matches exactly one character

How to create a custom report element within an awstat report?

Reply to Post Add a New Topic
UserPost

3:37 pm
January 28, 2009


albert_newton

Member

posts 30

The logs that I am getting ( to use as a source for awstat) do not have the url field in the logs. The fields that it logs ( relating to the url) are:

cs-uri-scheme cs-host cs-uri-path cs-uri-query

Basically I wanted to create a top url element within the report:

1. So I guess there is not really a way for awstats to generate the top urls report since there is no field named 'url' within the log file ( or is there a way to do so?).

2. Thus another alternative I thought of was to create a new table which has the top uri-path and the host columns next to each other. So, basically the table would be sorted by the column 'uri-paths', and the column prepending that would be the 'host' for that uri-path, so that this will at least give a basic idea of the top url paths.

It would be good if I could do number 1, that is somehow create a url report since 

 url = cs-uri-scheme + cs-host + cs-uri-path + cs-uri-query

I do not really need the cs-uri-query but all other information will be useful.

Or if that is not possible maybe I can do number 2 to achieve my mission? If so how can I get this statistic to show up in my report? I guess I would have to make some changes in the EXTRA SECTIONS to be able to do number 2?

5:29 am
January 29, 2009


Jean-Luc

Admin

posts 1125

URL's in AWStats usually do not include the sheme and the host name. I am not sure this format is acceptable for AWStats.

Assuming it is acceptable and as the url field must be cs-uri-scheme + cs-host + cs-uri-path , I would preprocess the log file before I pass it to AWStats. The preprocessing would concatenate these three fields in one and I would call this field %url  in the AWStats LogFormat directive.

9:32 am
January 29, 2009


albert_newton

Member

posts 30

Oh ok… Thanks for the info. So I guess that awstat will not be able to use the log file as is at present, to generate the url… I will have to think about the preprocessing option (since that will all have to be automated, it might be a pain). In the meanwhile, I was thinking if option 2 was feasible since that will work for my purposes too.  Right now awstat is giving the report for top hosts and top pages-url as different statistics, and I am using the %url in the Awstats LogFormat directive ( which corresponds to cs-uri-path from my log file), so that it shows the top uri-path in that report instead of the top urls. If I can somehow add a column for the host before this top cs-uri-path column, it will be almost as if I got the top urls report, ( since then I will be getting host + uri-path, and I don't really worry about scheme, and the uri-query). So I was wondering if there is a way to add the host column in the Pages-URL (Top 10) report.

10:24 am
January 29, 2009


Jean-Luc

Admin

posts 1125

Let us look at a tricky solution.Wink

AWStats is able to combine a %url  field with a %query  field ant to process the combined "URL with query" as a URL field. So if you use %url  for cs-host  and %query  for cs-uri-path , the resulting "URL with query" should be what you are looking for. You will need to set URLWithQuery  in the AWStats config file to make this work:

URLWithQuery=1

10:33 am
January 29, 2009


albert_newton

Member

posts 30

That's very creative…unfortunately it is a requirement that we do not want to show the query string within the url for security (confidential) purposes… that's a whole another headacheFrown… or else your idea was perfect… So I do not mind getting everything from the url except the query string.

10:47 am
January 29, 2009


Jean-Luc

Admin

posts 1125

If you follow my suggestion, the cs-uri-query  field of the log file will be ignored by AWStats. The intention is to fool AWStats by calling %query  the field that is called cs-uri-path in your log file and by calling %url  the cs-host  field. Do you see what I mean ?

10:52 am
January 29, 2009


albert_newton

Member

posts 30

Damn.. that's good… I was doing the same thing before for %url where I was using the cs-uri-path in my log file for the awstat directive %url… but I missed what you meant right now… Let me try what happens.

11:46 am
January 29, 2009


albert_newton

Member

posts 30

That thing works like a charm… except for one thing… it appends a '?' at the end of the host, ahhh:

This is because the config file says:

# Possible values:

# 0 – URLs are cleaned from the query string (ie: "/mypage.html")

# 1 – Full URL with query string is used     (ie: "/mypage.html?p=x&q=y")

So when I do URLWithQuery=1, it appends that question mark after the host but before the cs-uri-path. This happens even though I changed the URLQuerySeparators="?;" to URLQuerySeparators="/" within the config file.

so instead of showing: mysite.com/path_within_site, it shows:

mysite.com?/path_within_site

1:48 pm
February 4, 2009


albert_newton

Member

posts 30

Now, I am trying to filter the statistics to just show the results for *mysite.com*. Since the hosts in the report are showing the ip addresses and not the domain name, I cannot filter by host ( for *mysite.com*)… The only place I can see my domain name ( mysite.com) appearing is in the Pages-URL (Top 10) report. Is there a way I can set a filter to just show the statistics for *mysite* on the url or any other field? Looks like awstat config file only has the SkipHosts option to filter out such a statistic.

4:06 pm
February 4, 2009


Jean-Luc

Admin

posts 1125

Go in the "Pages/URL" section and select "Full list", then enter "mysite.com" in the filter box and press "OK".

It is uncommon to put the domain name in the "URL" as you do. A more usual way to handle multiple domains in a log file is to generate a report per domain using the %virtualname  field (called cs-host  by IIS) and to specify the appropriate host name in the SiteDomain  and HostAliases  definitions in the AWStats config file.

8:48 pm
February 4, 2009


albert_newton

Member

posts 30

The report that I have getting generated is a static html file in my hard drive. So, it does not allow me to click on the "Full list" option within the Pages-Url section to set a filter on it… Clicking on the "Full list" option within the Pages-Url section, it gives me the error: 'Cannot find 'file:///C:/Documents%20and%20Settings/username/Desktop/Desktop/AwstatScripts/awstats-6.5/wwwroot/cgi-bin/awstats.www.mysite.urldetail.html'. Make sure the path or Internet address is correct.'

1:10 am
February 5, 2009


Jean-Luc

Admin

posts 1125

It seems that you only generated the main page of the static report. The AWStats documentation (III. Run reports: Building and reading reports) explains how to generate all the static reports you need, including URL details with filter.

9:47 pm
February 8, 2009


albert_newton

Member

posts 30

Yes, that was exactly the reason why I could not click on the link and open the individual reports… Now I am able to click on the "Full list" for the "Pages/Url" section to open up that list, but it does not show me any option or text box to put the filter in. Maybe there is something that is missing? Also, is it possible to just add the filter somehow, so that the main report itself will be filtered for that url, ( so that all statistics in the main report that display are only for that url) rather than filtering on the individual url report? Since the main report is the one that we will be sending out to the client, we only want the urls pertaining to their site being displayed to them ( without the query string in it).

4:46 am
February 9, 2009


Jean-Luc

Admin

posts 1125

The AWStats documentation provides this example:

perl awstats.pl -config=mysite -output=urldetail:/news -staticlinks > awstats.mysite.urldetailwithfilter.html

where you have to replace "/news " by your filter (the name of the chosen site).

As I wrote earlier, it is uncommon to put the domain name in the “URL” as you do. A more usual way to handle multiple domains in a log file is to generate a report per domain using the %virtualname  field (called cs-host  by IIS) and to specify the appropriate host name in the SiteDomain  and HostAliases  definitions in the AWStats config file. Even, if for some reasons, you need to have a global set of reports, you can generate other complete sets of report for every site.

Reply to Post

Reply to Topic:
How to create a custom report element within an awstat report?

Guest Name (Required):

Guest Email (Required):

NOTE: New Posts are subject to administrator approval before being displayed

Smileys
Confused Cool Cry Embarassed Frown Kiss Laugh Smile Surprised Wink Yell
Post New Reply

Guest URL (required)

Math Required!
What is the sum of:
5 + 11
   


About the InternetOfficer.com Forum

Forum Timezone: UTC 1

Most Users Ever Online: 302

Currently Online:
10 Guests

Currently Browsing this Topic:
1 Guest

Forum Stats:

Groups: 2
Forums: 9
Topics: 638
Posts: 2709

Membership:

There are 257 Members
There have been 302 Guests

There is 1 Admin
There is 1 Moderator

Top Posters:

cssfsu – 55
deepakgupta – 34
albert_newton – 30
cosminpana – 20
DTNMike – 19
ahtshun83 – 17

Recent New Members: raju, todd2taylor, sbdcunha, mansigill1987, ThomasDuh, ThomasKic

Administrators: Jean-Luc (1125 Posts)

Moderators: Jean-Luc (1125 Posts)