Follow us on Twitter!
The important thing is not to stop questioning. - Albert Einstein
Thursday, April 17, 2014
Navigation
Home
HellBoundHackers Main:
HellBoundHackers Find:
HellBoundHackers Information:
Learn
Communicate
Submit
Shop
Challenges
HellBoundHackers Exploit:
HellBoundHackers Programming:
HellBoundHackers Think:
HellBoundHackers Track:
HellBoundHackers Patch:
HellBoundHackers Other:
HellBoundHackers Need Help?
Other
Members Online
Total Online: 21
Guests Online: 20
Members Online: 1

Registered Members: 82818
Newest Member: Ahmed
Latest Articles
View Thread

HellBound Hackers | Computer General | Programming

Page 1 of 2 1 2 >
Author

Google in python?

techb
Member



Posts: 384
Location:
Joined: 15.02.09
Rank:
Moderate
Posted on 26-04-10 05:04
I know the debate on what googles crawler is written, and I haven't looked into what it uses today but in the early days of google stanford says it was first implemented in python:

"""...In order to scale to hundreds of millions of web pages, Google has a fast distributed crawling system. A single URLserver serves lists of URLs to a number of crawlers (we typically ran about 3). Both the URLserver and the crawlers are implemented in Python."""

link to paper:
http://www.infolab.stanford.edu/~backrub/google.html
*the url tag wouldn't work with the link. IDk why...

True? I don't know, this could be a fake. But then again why would some one take that much effort into writing the paper?



www.userbars.com/74460/665255/337-7865-ubda3219.gif
kbcarte.wordpress.com
Author

RE: Google in python?

goluhaque
Member



Posts: 197
Location: India
Joined: 17.02.10
Rank:
Apprentice
Warn Level: 30
Posted on 26-04-10 07:10
Broken Link. Sad


That applause I receive from y'all on posting this post would have gotten me drunk on power if I hadn't already been high on life.
Author

RE: Google in python?

techb
Member



Posts: 384
Location:
Joined: 15.02.09
Rank:
Moderate
Posted on 26-04-10 07:12
I don't know why it doesn't work...

google search = stanford python google

it should be the first link titled: "The Anatomy of a Search Engine"

I discovered it while working on timed6. Which is complete now if any one cares. Found a nifty google module to pares(spelling?) searches.



www.userbars.com/74460/665255/337-7865-ubda3219.gif


Edited by techb on 26-04-10 07:15
kbcarte.wordpress.com
Author

RE: Google in python?

goluhaque
Member



Posts: 197
Location: India
Joined: 17.02.10
Rank:
Apprentice
Warn Level: 30
Posted on 26-04-10 07:32
The www shouldn't be there---
http://infolab.stanford.edu/~backrub/google.html


That applause I receive from y'all on posting this post would have gotten me drunk on power if I hadn't already been high on life.
Author

RE: Google in python?

techb
Member



Posts: 384
Location:
Joined: 15.02.09
Rank:
Moderate
Posted on 26-04-10 07:34
a url tag didn't work with that either
test


www.userbars.com/74460/665255/337-7865-ubda3219.gif
kbcarte.wordpress.com
Author

RE: Google in python?

fuser
Member



Posts: 960
Location: in front of a computer (duh)
Joined: 05.04.07
Rank:
Mad User
Posted on 26-04-10 07:39
techb wrote:
a url tag didn't work with that either
test


you didn't close the tags properly.

http://infolab.st. . .oogle.html

huh, doesn't work either.


img.userbarz.com/51/10006.png
img.userbarz.com/146/29144.gif
img.userbarz.com/99/19602.jpg
img.userbarz.com/4/600.png
img.userbarz.com/45/8814.gif
img360.imageshack.us/img360/9231/bfbarlr0.jpg
[url=http://userbarz.com/][img]ht

Edited by fuser on 26-04-10 07:46
catinthecpu@hotmail.com
Author

RE: Google in python?

techb
Member



Posts: 384
Location:
Joined: 15.02.09
Rank:
Moderate
Posted on 26-04-10 07:48
i was going for something a link like this...





www.userbars.com/74460/665255/337-7865-ubda3219.gif
kbcarte.wordpress.com
Author

RE: Google in python?

stealth-
Member



Posts: 1003
Location: Eh?
Joined: 10.04.09
Rank:
Mad User
Posted on 28-04-10 04:50
I could see python being used as the original web crawler, it tends to be used for applications of that type. However, with the massive amount of crawling/searching google has to do since they've became the size they are now, I highly doubt they would still rely on such a slow language. It's just really inefficient to have to purchase more/faster hardware when you could simply redesign the code once and not have to worry as much about increasing processing power.


The irony of man's condition is that the deepest need is to be free of the anxiety of death and annihilation; but it is life itself which awakens it, and so we must shrink from being fully alive.
http://www.stealt. . .
http://www.stealth-x.com
Author

RE: Google in python?

techb
Member



Posts: 384
Location:
Joined: 15.02.09
Rank:
Moderate
Posted on 28-04-10 06:38
I've heard it is written in ASM now. I could be wrong though.


www.userbars.com/74460/665255/337-7865-ubda3219.gif
kbcarte.wordpress.com
Author

RE: Google in python?

fuser
Member



Posts: 960
Location: in front of a computer (duh)
Joined: 05.04.07
Rank:
Mad User
Posted on 28-04-10 10:39
ASM sounds impractical for Google to use, since it's platform specific, and we're talking the world leader in Internet services, so this is a major hindrance.

If they said they now use Ruby, well, I might believe it. Where did you hear about it.


img.userbarz.com/51/10006.png
img.userbarz.com/146/29144.gif
img.userbarz.com/99/19602.jpg
img.userbarz.com/4/600.png
img.userbarz.com/45/8814.gif
img360.imageshack.us/img360/9231/bfbarlr0.jpg
[url=http://userbarz.com/][img]ht
catinthecpu@hotmail.com
Author

RE: Google in python?

techb
Member



Posts: 384
Location:
Joined: 15.02.09
Rank:
Moderate
Posted on 28-04-10 10:50
It was here on HBH, I don't remember who said it though.


www.userbars.com/74460/665255/337-7865-ubda3219.gif
kbcarte.wordpress.com
Author

RE: Google in python?

fuser
Member



Posts: 960
Location: in front of a computer (duh)
Joined: 05.04.07
Rank:
Mad User
Posted on 28-04-10 10:52
huh, whoever said it is either pulling your leg or is plain clueless on web applications.


img.userbarz.com/51/10006.png
img.userbarz.com/146/29144.gif
img.userbarz.com/99/19602.jpg
img.userbarz.com/4/600.png
img.userbarz.com/45/8814.gif
img360.imageshack.us/img360/9231/bfbarlr0.jpg
[url=http://userbarz.com/][img]ht
catinthecpu@hotmail.com
Author

RE: Google in python?

techb
Member



Posts: 384
Location:
Joined: 15.02.09
Rank:
Moderate
Posted on 28-04-10 10:56
eh... It doesn't matter.

I don't really care what its written in. Sure it would be interesting to know, but not useful.

I thought it was interesting because I code in python.

I thought it was coded in C/++.


www.userbars.com/74460/665255/337-7865-ubda3219.gif
kbcarte.wordpress.com
Author

RE: Google in python?

fuser
Member



Posts: 960
Location: in front of a computer (duh)
Joined: 05.04.07
Rank:
Mad User
Posted on 28-04-10 11:18
you're not the first one. When I first heard about it, I assumed that it can only be done in C/C++ or some other language. But then I learned about scripting languages, and while the original crawler might have been done in Python, I'll bet it's now written in Ruby.


img.userbarz.com/51/10006.png
img.userbarz.com/146/29144.gif
img.userbarz.com/99/19602.jpg
img.userbarz.com/4/600.png
img.userbarz.com/45/8814.gif
img360.imageshack.us/img360/9231/bfbarlr0.jpg
[url=http://userbarz.com/][img]ht
catinthecpu@hotmail.com
Author

RE: Google in python?


Member

Your avatar

Posts:
Location:
Joined: 01.01.70
Rank:
Guest
Posted on 28-04-10 11:21
Pretty sure it's true google always goes on about python being at the heart of it some shit.
I don't knwo about noadays though.


Author

RE: Google in python?

techb
Member



Posts: 384
Location:
Joined: 15.02.09
Rank:
Moderate
Posted on 28-04-10 11:27
fuser wrote:
you're not the first one. When I first heard about it, I assumed that it can only be done in C/C++ or some other language. But then I learned about scripting languages, and while the original crawler might have been done in Python, I'll bet it's now written in Ruby.


Why Ruby over Python?


www.userbars.com/74460/665255/337-7865-ubda3219.gif
kbcarte.wordpress.com
Author

RE: Google in python?

fuser
Member



Posts: 960
Location: in front of a computer (duh)
Joined: 05.04.07
Rank:
Mad User
Posted on 28-04-10 11:31
after a brief search, I discovered that it is possible that python is just part of it. There are parts of it written in C++, and they recruit people who use other languages as well.

And this link seems to be the most conclusive:

http://stackoverf. . .written-in

post #17 of this thread gives the complete list:

http://forums.dig. . .p?t=673630

aaanndd here's from one of the google's devs themselves:

http://steve-yegg. . .rails.html

edit: techb, the reason why I think they might be using Ruby because I heard it's faster than Python, but I'm just learning, so I might be wrong until someone posts a benchmark between several different languages.


img.userbarz.com/51/10006.png
img.userbarz.com/146/29144.gif
img.userbarz.com/99/19602.jpg
img.userbarz.com/4/600.png
img.userbarz.com/45/8814.gif
img360.imageshack.us/img360/9231/bfbarlr0.jpg
[url=http://userbarz.com/][img]ht

Edited by fuser on 28-04-10 11:32
catinthecpu@hotmail.com
Author

RE: Google in python?


Member

Your avatar

Posts:
Location:
Joined: 01.01.70
Rank:
Guest
Posted on 28-04-10 15:51
they love python and c++. Go is pretty much c++ getting python drunk the reusltant one night stand.


Author

RE: Google in python?

techb
Member



Posts: 384
Location:
Joined: 15.02.09
Rank:
Moderate
Posted on 28-04-10 15:53
I've looked a little into GO and wasn't really impressed. I've heard a lot of bad things about it also.


www.userbars.com/74460/665255/337-7865-ubda3219.gif
kbcarte.wordpress.com
Author

RE: Google in python?


Member

Your avatar

Posts:
Location:
Joined: 01.01.70
Rank:
Guest
Posted on 28-04-10 15:57
techb wrote:
I've looked a little into GO and wasn't really impressed. I've heard a lot of bad things about it also.

Go has a lot of good features that were always around just never picked up by the mainstream imo


Page 1 of 2 1 2 >