| Author |
Google in python? |
techb
Member
Posts: 384
Location:
Joined: 15.02.09
Rank: Hacker Level 2 |
|
I know the debate on what googles crawler is written, and I haven't looked into what it uses today but in the early days of google stanford says it was first implemented in python:
"""...In order to scale to hundreds of millions of web pages, Google has a fast distributed crawling system. A single URLserver serves lists of URLs to a number of crawlers (we typically ran about 3). Both the URLserver and the crawlers are implemented in Python."""
link to paper:
http://www.infolab.stanford.edu/~backrub/google.html
*the url tag wouldn't work with the link. IDk why...
True? I don't know, this could be a fake. But then again why would some one take that much effort into writing the paper?
|
 
|
| Author |
RE: Google in python? |
goluhaque
Member
Posts: 194
Location: India
Joined: 17.02.10
Rank: Hacker Level 1
Warn Level: 30
|
|
Broken Link. 
That applause I receive from y'all on posting this post would have gotten me drunk on power if I hadn't already been high on life. |
|
|
| Author |
RE: Google in python? |
techb
Member
Posts: 384
Location:
Joined: 15.02.09
Rank: Hacker Level 2 |
|
I don't know why it doesn't work...
google search = stanford python google
it should be the first link titled: "The Anatomy of a Search Engine"
I discovered it while working on timed6. Which is complete now if any one cares. Found a nifty google module to pares(spelling?) searches.
Edited by techb on 26-04-10 07:15 |
|
|
| Author |
RE: Google in python? |
goluhaque
Member
Posts: 194
Location: India
Joined: 17.02.10
Rank: Hacker Level 1
Warn Level: 30
|
|
The www shouldn't be there---
http://infolab.stanford.edu/~backrub/google.html
That applause I receive from y'all on posting this post would have gotten me drunk on power if I hadn't already been high on life. |
|
|
| Author |
RE: Google in python? |
techb
Member
Posts: 384
Location:
Joined: 15.02.09
Rank: Hacker Level 2 |
|
a url tag didn't work with that either
[url=http://infolab.stanford.edu/~backrub/google.html]test[/url]
|
|
|
| Author |
RE: Google in python? |
fuser
Member
Posts: 959
Location: in front of a computer (duh)
Joined: 05.04.07
Rank: HBH Guru |
|
|
techb wrote:
a url tag didn't work with that either
[url=http://infolab.stanford.edu/~backrub/google.html]test[/url]
you didn't close the tags properly.
[url]http://infolab.stanford.edu/~backrub/google.html[/url]
huh, doesn't work either.
Telling modern Internet users to stop whining is like telling them to stop breathing — it seems unrealistic and inhumane. Paul Lutus
Edited by fuser on 26-04-10 07:46 |
|
| Author |
RE: Google in python? |
techb
Member
Posts: 384
Location:
Joined: 15.02.09
Rank: Hacker Level 2 |
|
i was going for something a link like this...
|
|
|
| Author |
RE: Google in python? |
stealth-
Member
Posts: 998
Location: Eh?
Joined: 10.04.09
Rank: God |
|
I could see python being used as the original web crawler, it tends to be used for applications of that type. However, with the massive amount of crawling/searching google has to do since they've became the size they are now, I highly doubt they would still rely on such a slow language. It's just really inefficient to have to purchase more/faster hardware when you could simply redesign the code once and not have to worry as much about increasing processing power.
The irony of man's condition is that the deepest need is to be free of the anxiety of death and annihilation; but it is life itself which awakens it, and so we must shrink from being fully alive.
http://www.stealth-x.com |
|
|
| Author |
RE: Google in python? |
techb
Member
Posts: 384
Location:
Joined: 15.02.09
Rank: Hacker Level 2 |
|
I've heard it is written in ASM now. I could be wrong though.
|
|
|
| Author |
RE: Google in python? |
fuser
Member
Posts: 959
Location: in front of a computer (duh)
Joined: 05.04.07
Rank: HBH Guru |
|
ASM sounds impractical for Google to use, since it's platform specific, and we're talking the world leader in Internet services, so this is a major hindrance.
If they said they now use Ruby, well, I might believe it. Where did you hear about it.
Telling modern Internet users to stop whining is like telling them to stop breathing — it seems unrealistic and inhumane. Paul Lutus
|
|
|
| Author |
RE: Google in python? |
techb
Member
Posts: 384
Location:
Joined: 15.02.09
Rank: Hacker Level 2 |
|
It was here on HBH, I don't remember who said it though.
|
|
|
| Author |
RE: Google in python? |
fuser
Member
Posts: 959
Location: in front of a computer (duh)
Joined: 05.04.07
Rank: HBH Guru |
|
huh, whoever said it is either pulling your leg or is plain clueless on web applications.
Telling modern Internet users to stop whining is like telling them to stop breathing — it seems unrealistic and inhumane. Paul Lutus
|
|
|
| Author |
RE: Google in python? |
techb
Member
Posts: 384
Location:
Joined: 15.02.09
Rank: Hacker Level 2 |
|
eh... It doesn't matter.
I don't really care what its written in. Sure it would be interesting to know, but not useful.
I thought it was interesting because I code in python.
I thought it was coded in C/++.
|
|
|
| Author |
RE: Google in python? |
fuser
Member
Posts: 959
Location: in front of a computer (duh)
Joined: 05.04.07
Rank: HBH Guru |
|
you're not the first one. When I first heard about it, I assumed that it can only be done in C/C++ or some other language. But then I learned about scblockedripting languages, and while the original crawler might have been done in Python, I'll bet it's now written in Ruby.
Telling modern Internet users to stop whining is like telling them to stop breathing — it seems unrealistic and inhumane. Paul Lutus
|
|
|
| Author |
RE: Google in python? |
wolfmankurd
Member
Posts: 1519
Location: UK
Joined: 30.05.05
Rank: God |
|
Pretty sure it's true google always goes on about python being at the heart of it some shit.
I don't knwo about noadays though.
BY READING MY POST, YOU ACCEPT IT AS IS AND AGREE TO MY DISCLAIMER OF ALL WARRANTIES, EXPRESS OR IMPLIED, AS WELL AS DISCLAIMERS OF ALL LIABILITY, DIRECT, INDIRECT, CONSEQUENTIAL OR INCIDENTAL, THAT MAY ARISE FROM THE USE OF THIS (MIS)INFORMATION.
|
|
|
| Author |
RE: Google in python? |
techb
Member
Posts: 384
Location:
Joined: 15.02.09
Rank: Hacker Level 2 |
|
|
fuser wrote:
you're not the first one. When I first heard about it, I assumed that it can only be done in C/C++ or some other language. But then I learned about scblockedripting languages, and while the original crawler might have been done in Python, I'll bet it's now written in Ruby.
Why Ruby over Python?
|
|
|
| Author |
RE: Google in python? |
fuser
Member
Posts: 959
Location: in front of a computer (duh)
Joined: 05.04.07
Rank: HBH Guru |
|
after a brief search, I discovered that it is possible that python is just part of it. There are parts of it written in C++, and they recruit people who use other languages as well.
And this link seems to be the most conclusive:
http://stackoverflow.com/questions/754301/what-programming-language-is-google-written-in
post #17 of this thread gives the complete list:
http://forums.digitalpoint.com/showthread.php?t=673630
aaanndd here's from one of the google's devs themselves:
http://steve-yegge.blogspot.com/2007/06/rhino-on-rails.html
edit: techb, the reason why I think they might be using Ruby because I heard it's faster than Python, but I'm just learning, so I might be wrong until someone posts a benchmark between several different languages.
Telling modern Internet users to stop whining is like telling them to stop breathing — it seems unrealistic and inhumane. Paul Lutus
Edited by fuser on 28-04-10 11:32 |
|
|
| Author |
RE: Google in python? |
wolfmankurd
Member
Posts: 1519
Location: UK
Joined: 30.05.05
Rank: God |
|
they love python and c++. Go is pretty much c++ getting python drunk the reusltant one night stand.
BY READING MY POST, YOU ACCEPT IT AS IS AND AGREE TO MY DISCLAIMER OF ALL WARRANTIES, EXPRESS OR IMPLIED, AS WELL AS DISCLAIMERS OF ALL LIABILITY, DIRECT, INDIRECT, CONSEQUENTIAL OR INCIDENTAL, THAT MAY ARISE FROM THE USE OF THIS (MIS)INFORMATION.
|
|
|
| Author |
RE: Google in python? |
techb
Member
Posts: 384
Location:
Joined: 15.02.09
Rank: Hacker Level 2 |
|
I've looked a little into GO and wasn't really impressed. I've heard a lot of bad things about it also.
|
|
|
| Author |
RE: Google in python? |
wolfmankurd
Member
Posts: 1519
Location: UK
Joined: 30.05.05
Rank: God |
|
|
techb wrote:
I've looked a little into GO and wasn't really impressed. I've heard a lot of bad things about it also.
Go has a lot of good features that were always around just never picked up by the mainstream imo
BY READING MY POST, YOU ACCEPT IT AS IS AND AGREE TO MY DISCLAIMER OF ALL WARRANTIES, EXPRESS OR IMPLIED, AS WELL AS DISCLAIMERS OF ALL LIABILITY, DIRECT, INDIRECT, CONSEQUENTIAL OR INCIDENTAL, THAT MAY ARISE FROM THE USE OF THIS (MIS)INFORMATION.
|
|
|
Main:
