Deconstructing Google url search parameters
[Quick note: this post isn't about the Google search url that's created when you use Google to search the web. If you're interested in those "request parameters", you can't do better than Google's own resource: Search Protocol Reference.]
As a UX designer, I use web stats a lot. Typically Google’s Analytics product is the go-to source with more site analytics than you can shake a stick at.
However I also have a fondness for StatCounter, especially after reading their Open Letter to Roger Capriotti of Microsoft, rubbishing his understanding of browser usage share data. I’ve been using StatCounter’s free service for years now and often recommend them to smaller companies because of the simplicity of their results and the ease with which novices can start to get some real insight on their sites.
For my own blog stats for this site, my favourite StatCounter page is undoubtedly “Recent Pageload Activity”. I have a modest readership and from this page I can easily see exactly where people are coming from (previous website & geographical location), as well as where they go next, where they exit, what browser they use and their ISP.
Increasingly, I’m seeing a Referring Link (the page the visitor was on before they hit your site) of the following form:
www.google.com.ua/url?sa=t&rct=j&q=&esrc=s&source=web&cd=3&ved=0CGMQFjAC&url=
http%3A%2F%2Fwww.t75.org%2F2011%2F08%2Fhandy-finder-shortcut%2F&ei=ELLhT5SeLcXu-gavpbmdAw&usg=AFQjCNGqhzD-SwHsPkbxvXmjiAr_iM5n8g&sig2=vgJOia59rlFfYmszSkddzw
Clicking on this link takes me from StatCounter directly to my own website. That almost suggests that my site is self-referring visitors back to my site, or that people are searching for my exact url? Both situation are obviously very unlikely. For some time I assumed these visitors were coming from another Google property such as Reader.
What’s actually happening is that Google is hiding the referrer data through a redirect on the SERP. The links you see in the result listing look like they go straight to the website you’re interested in, but actually, if you inspect the link you’ll find a url of the type seen above. This allows Google to hide the q value from the referring string – used by so many stats packages.
Personally, I believe that as a user, that you should be able to hide your Google searches by using Google’s secure search (https). However, as Alex Wall says:
Paid advertisers continued to receive that data, however, a blow to integrity for Google, who claimed the switch was the sake of privacy. Sure, it was for the sake of privacy unless there was a chance that they could sell that data for money. Okay.
Read more
So what do all these url parameters actually mean? After some research I’ve tracked down the meaning of a few of these parameters and with your help, we might be able to track down the rest!
So that url again, broken down to it’s constituent parameters:
- url?
- sa=t
- rct=j
- q=
- esrc=s
- source=web
- cd=3
- ved=0CGMQFjAC
- url=http%3A%2F%2Fwww.t75.org%2F2011%2F08%2Fhandy-finder-shortcut%2F
- ei=ELLhT5SeLcXu-gavpbmdAw
- usg=AFQjCNGqhzD-SwHsPkbxvXmjiAr_iM5n8g
- sig2=vgJOia59rlFfYmszSkddzw
1. url?
url is pretty easy. Google explains it themselves here:
The key difference between the [the old style url and the new] is that instead of “/search?” the URL contains a “/url?”. If you run your own analyses, be sure that you do not depend on the “/search?” portion of the URL to determine if a visit started with an organic search click. Google Analytics does not depend on the “/search?” string in the referrer, so users of Google Analytics will not notice a difference in their reports, but other analytics packages may need to adapt to this change in our referrer string to maintain accurate reports.
2. sa=t
Rank Panel have an useful table that lists a large number of Google’s parameters and certainly, if they had all of the parameter’s I’m seeing there wouldn’t be any need for this article! I’d urge you to read through it.
From that table, they define “sa” as a “User search behavior parameter” (where “sa=N”: User searched and “sa=X”: User clicked on related searches in the SERP).
But what about “sa=t”; t = true?
3. rct=j
I can’t find any information on rct. Interestingly, “j” is the only value I ever get. rct=receipt?
4. q=
This is meat of the query. Frustratingly this is absent more often than present. Whether you believe this is to protect the privacy of Google’s users or an example of anti-competitive practice is up to you. Read a little more about it at SearchEngineLand.
Try searching by http (as opposed to https) and you’ll see all the q values are populated, reinforcing the idea that it’s the new secure search that hides your keywords.
5. esrc=s
It’s possible that esrc is linked to that blank query string. See http://www.wordstream.com/blog/ws/2012/02/24/keyword-not-provided-esrc for more details. More conjecture from DC Storm:
Also included in the SSL search is an additional URL parameter, “ESRC”. This could be the indicator of the SSL search and also could be an acronym for “Encrypted Search”. However this is pure conjecture, but would be an opportunity for tool vendors to determine if the unencrypted or use SSL search was used. The accuracy of this remains to be seen.
And indeed, if I search Google via https I see esrc in the link parameters, if I search by http, esrc is absent. (Interestingly, when I search by http, all the q values (search query keywords) are populated, reinforcing the idea that it’s the new secure search that hides your keywords.)
6. source=web
Doesn’t take a genius to work this one out. But what would the other options be? Again, all my “source” values = web. There used to be a sourceid that coded for natural web searches, Google Tool bar searches, Firefox Tool bar etc. Ideas welcomed!
7. cd=3
This seems to indicate the ranking position of your link in the SERP. See SeoMoz for more.
8. ved=0CGMQFjAC
There has been some suggestion that the ved parameter refers to browser type. However, looking through my results, I see very different ved values for the same browser (browser as reported by Statcounter).
Update: www.seomanontroppo.com has some great insights into the codes used with ved, particularly with regard to News results.
9. url=http%3A%2F%2Fwww.t75.org%2F2011%2F08%2Fhandy-finder-shortcut%2F
This is the link to my site with various character encoded: e.g. %3A=”:” , %2F= “/” etc. Decoded to: http://www.t75.org/2011/08/handy-finder-shortcut/
10. ei=ELLhT5SeLcXu-gavpbmdAw
Obviously something that needs to be encoded. There’s some suggestion that this shows if the visitor has navigated the Google search pages by clicking “next”. But if “cd” really does mean I was ranked third, then the visitor wouldn’t have needed to click on the next link? Unless I was the third result on the second, third, etc. page?
11. usg=AFQjCNGqhzD-SwHsPkbxvXmjiAr_iM5n8g
Again, something too secret to share. USaGe? USerGoogle?
12. sig2=vgJOia59rlFfYmszSkddzw
Nada.
Can you help?
Interesting article, many thanks for this :)))
The encoded parameters are probably base64 encoded, but I haven’t yet tried to decode them.
We have an active AdWords account, so all referrer strings have the ‘q’ param (as do those from other search engines – I wasn’t aware that if you didn’t Google would conceal the query string – eww), except when from https.
Carlos Ramírez asked: “Can you differentiate between a natural and a paid search link?”
This is a very good question and I arrived here looking for a definitive answer. As far as I can tell, if they clicked on an ad, there’ll either be an ‘adurl’ parameter or the path will include ‘aclk’ (probably always both). But this jury is still out. If I find time to trawl through the logs and gather more useful evidence, I’ll post again…
Keep up the good work,
Jake
Hey! I’m not quite sure what you mean, but I can’t differentiate between the two with my limited results as I’ve never bought search placement. Certainly GA gives you good visibility on your paid/natural results.
Hey! Can you differentiate between a natural and a paid search link?
[...] http://www.t75.org/2012/06/deconstructing-googles-url-search-parameters/ [...]
Thanks.
What about the heirloom-serp or heirloom parameter?
For example in http://www.zoklet.net/bbs/archive/index.php/t-257440.html
Hyperlinks in a Google Docs text document are encoded in a similar way as described in the present article for search parameters, and the encoding may survive export of the Docs text document to pdf, odt, docx or rtf format or publication to the web. I describe the situation in a sequence of postings [1] on the Google Docs product forum.
I wonder if Tim or someone else here can shed some light on the parameters that are used for hyperlinks in Google Docs text documents. The general form is as follows:
http://www.google.com/url?q=URL&sa=D&sntz=1&usg=CODE
where URL denotes the proper target of the hyperlink and CODE denotes some string of about 34 characters, perhaps a hash of some kind. As an example, if the Google Docs text document contains a link to http://www.example.com and the link address has been edited (as described under [1]) then the hyperlink in the exported or published document will go to
http://www.google.com/url?q=http%3A%2F%2Fwww.example.com&sa=D&sntz=1&usg=AFQjCNEFCU3RF7exbH39Ef7Szy9s-UX8WA
As explained under [1], I don’t really have a clue what the Google Docs team has in mind with this encoding, and in fact I suspect that it just reflects a software error; the encoding is (I suspect) a remnant of something else. However, I do not know this.
[1] (2011-09-14 and later) External hyperlinks may again be redirected through http://www.google.com
https://productforums.google.com/d/topic/docs/v2TqlvBWfoI/
Here you have the results of a little research I did about ved parameter, http://www.seomanontroppo.com/google-news-analytics-stuff/
the sig2 parameter seems to be present only when the user is logged in with its Google account.
hope this helps !
Many thanks! I certainly hope to keep it up to date – judging by the traffic a lot of people are finding it useful. And thanks for the thoughts on VED, I’ll definitely have a closer look at that.
love this post and hope you keep adding more info as you get it. VED, i think contains some very useful info about how/and where a user clicked, but it’s difficult to deconstruct all of it. One thing I’m fairly confident about is that any ved= string that includes the 3 letter series “Qlw” means the user click came from a news callout within the SERPS. I think this value can also indicate image, video, local or other blended search, but i’ve not fully figured out all the details.
[...] More information on this from designer Tim Minor in his blog post: Deconstructing Google url search parameters . [...]
[...] come si deduce dal nome ha come valore la lingua e il secondo, usg, di cui non si sa nulla. In questo approfondito articolo sui singoli parametri utilizzati da Google nell’indicizzazione dei siti usg rimane avvolto [...]
Last night on my iPhone I clicked a link with those parameters. From an Atomic web browser toolbar search (first page, 4th link) the site I tried to access instead threw me back at the google page using these parameters:
sa=t source=web cd=3 ved=0CFwQFjAC
I actually had first performed the google search from an ipod (first page, 2nd link) but Atomic browser popped up, “URL blocked by Ad Block filter”, Atomic web’s first site block and pop up in the year I’ve used the browser. I never saw the url but the pop up was weird.
That’s when I tried it with my phone where Atomic browser let me click. The browser fleetingly stopped at a white page with something in the top left corner about java, then it landed at a blank, unsearchable google site.
I only had this problem with iOS. Hope it helps.