Nothing beats a full stats package, but with a little grepping and a logfile and we can figure out some quick statistics.
Get only hits coming from Google.com/search:
grep 'google.com/search' access_log | head -1
Extract only the referrer field:
grep 'google.com/search' access_log | head -1 | awk '{print $11}'
Get the search terms only:
grep 'google.com/search' access_log | \
head -1 | \
awk '{print $11}' | \
cut -d\? -f2 | cut -d\& -f1 | \
sed 's/+/ /g;s/%22/"/g;s/q=//'
head -1 | \
awk '{print $11}' | \
cut -d\? -f2 | cut -d\& -f1 | \
sed 's/+/ /g;s/%22/"/g;s/q=//'
Here’s a little script that puts it all together. It finds the unique search terms, cleans out the trash, sorts them and list the top twenty five in order of descending hit count.
#!/bin/sh
LOG="/var/logs/httpd/access_log"
grep 'google.com/search' $LOG | \
awk '{print $11}' | \
cut -d\? -f2 | cut -d\& -f1 | \
sed 's/+/ /g;s/%22/"/g;s/q=//' | \
sed 's/%[0-9a-fA-F][0-9a-fA-F]/ /g;s/"//g' | \
grep -v '=' | sort | uniq -c | sort -rn | head -25
LOG="/var/logs/httpd/access_log"
grep 'google.com/search' $LOG | \
awk '{print $11}' | \
cut -d\? -f2 | cut -d\& -f1 | \
sed 's/+/ /g;s/%22/"/g;s/q=//' | \
sed 's/%[0-9a-fA-F][0-9a-fA-F]/ /g;s/"//g' | \
grep -v '=' | sort | uniq -c | sort -rn | head -25
To see them all remove the | head -25 statement at the end of the command.
Thanks to an article at Linux Journal for the examples.
Popularity: 1% [?]
this works very nice! I’m a shell scripting noob, it helped me much.