Deconstructing Google url search parameters
[Quick note: this post isn't about the Google search url that's created when you use Google to search the web. If you're interested in those "request parameters", you can't do better than Google's own resource: Search Protocol Reference.]
As a UX designer, I use web stats a lot. Typically Google’s Analytics product is the go-to source with more site analytics than you can shake a stick at.
However I also have a fondness for StatCounter, especially after reading their Open Letter to Roger Capriotti of Microsoft, rubbishing his understanding of browser usage share data. I’ve been using StatCounter’s free service for years now and often recommend them to smaller companies because of the simplicity of their results and the ease with which novices can start to get some real insight on their sites.
For my own blog stats for this site, my favourite StatCounter page is undoubtedly “Recent Pageload Activity”. I have a modest readership and from this page I can easily see exactly where people are coming from (previous website & geographical location), as well as where they go next, where they exit, what browser they use and their ISP.
Increasingly, I’m seeing a Referring Link (the page the visitor was on before they hit your site) of the following form:
Clicking on this link takes me from StatCounter directly to my own website. That almost suggests that my site is self-referring visitors back to my site, or that people are searching for my exact url? Both situation are obviously very unlikely. For some time I assumed these visitors were coming from another Google property such as Reader.
What’s actually happening is that Google is hiding the referrer data through a redirect on the SERP. The links you see in the result listing look like they go straight to the website you’re interested in, but actually, if you inspect the link you’ll find a url of the type seen above. This allows Google to hide the q value from the referring string – used by so many stats packages.
Personally, I believe that as a user, that you should be able to hide your Google searches by using Google’s secure search (https). However, as Alex Wall says:
Paid advertisers continued to receive that data, however, a blow to integrity for Google, who claimed the switch was the sake of privacy. Sure, it was for the sake of privacy unless there was a chance that they could sell that data for money. Okay.
So what do all these url parameters actually mean? After some research I’ve tracked down the meaning of a few of these parameters and with your help, we might be able to track down the rest!
So that url again, broken down to it’s constituent parameters:
url is pretty easy. Google explains it themselves here:
The key difference between the [the old style url and the new] is that instead of “/search?” the URL contains a “/url?”. If you run your own analyses, be sure that you do not depend on the “/search?” portion of the URL to determine if a visit started with an organic search click. Google Analytics does not depend on the “/search?” string in the referrer, so users of Google Analytics will not notice a difference in their reports, but other analytics packages may need to adapt to this change in our referrer string to maintain accurate reports.
Rank Panel have an useful table that lists a large number of Google’s parameters and certainly, if they had all of the parameter’s I’m seeing there wouldn’t be any need for this article! I’d urge you to read through it.
From that table, they define “sa” as a “User search behavior parameter” (where “sa=N”: User searched and “sa=X”: User clicked on related searches in the SERP).
But what about “sa=t”; t = true?
I can’t find any information on rct. Interestingly, “j” is the only value I ever get. rct=receipt?
This is meat of the query. Frustratingly this is absent more often than present. Whether you believe this is to protect the privacy of Google’s users or an example of anti-competitive practice is up to you. Read a little more about it at SearchEngineLand.
Try searching by http (as opposed to https) and you’ll see all the q values are populated, reinforcing the idea that it’s the new secure search that hides your keywords.
It’s possible that esrc is linked to that blank query string. See http://www.wordstream.com/blog/ws/2012/02/24/keyword-not-provided-esrc for more details. More conjecture from DC Storm:
Also included in the SSL search is an additional URL parameter, “ESRC”. This could be the indicator of the SSL search and also could be an acronym for “Encrypted Search”. However this is pure conjecture, but would be an opportunity for tool vendors to determine if the unencrypted or use SSL search was used. The accuracy of this remains to be seen.
And indeed, if I search Google via https I see esrc in the link parameters, if I search by http, esrc is absent. (Interestingly, when I search by http, all the q values (search query keywords) are populated, reinforcing the idea that it’s the new secure search that hides your keywords.)
Doesn’t take a genius to work this one out. But what would the other options be? Again, all my “source” values = web. There used to be a sourceid that coded for natural web searches, Google Tool bar searches, Firefox Tool bar etc. Ideas welcomed!
This seems to indicate the ranking position of your link in the SERP. See SeoMoz for more.
There has been some suggestion that the ved parameter refers to browser type. However, looking through my results, I see very different ved values for the same browser (browser as reported by Statcounter).
Update: www.seomanontroppo.com has some great insights into the codes used with ved, particularly with regard to News results.
This is the link to my site with various character encoded: e.g. %3A=”:” , %2F= “/” etc. Decoded to: http://www.t75.org/2011/08/handy-finder-shortcut/
Obviously something that needs to be encoded. There’s some suggestion that this shows if the visitor has navigated the Google search pages by clicking “next”. But if “cd” really does mean I was ranked third, then the visitor wouldn’t have needed to click on the next link? Unless I was the third result on the second, third, etc. page?
Again, something too secret to share. USaGe? USerGoogle?
Can you help?