Would It Be Possible?

[100%]

Just got the this program for Firefox called Spiderzilla
with you can grab an entire website

So if one set things right it should be possible to "Grab" the Entire Internet
How big would it be ???????
To make a "Back Up" of the internet?
Where to Start?

[h1]

:huh: :blink: :frusty: You're shitting me, right? :ph34r: :ghostface: :ninja:

[100%]

This is just speculation
to "grab" it as it is *now *12:34:00:00 am
how big is it?
how to do it?¨
Is it possible?

[balamm]

Sure ya can!! I'm almost there :lol: :lol: :lol:

[Robert00000]

Originally posted by Zedaxax@3 March 2004 - 00:00
Just got the this program for Firefox called Spiderzilla
with you can grab an entire website

So if one set things right it should be possible to "Grab" the Entire Internet
How big would it be ???????
To make a "Back Up" of the internet?
Where to Start?
Backup the internet? :clown:


:lol: :lol:

[100%]

found this on google

* AltaVista 1.689 billion pages.
* FAST 2.106 billion pages.
* Google 3.083 billion pages.
* Inktomi 1.600 billion pages.
* Wisenut 1.453 billion pages.

Image Resized
[img]http://www.people.virginia.edu/~mmh9a/pics/3d/internet.gif' width='200' height='120' border='0' alt='click for full size view'> ('http://www.people.virginia.edu/~mmh9a/pics/3d/internet.gif')

[NightStalker]

Your kidding right? :rolleyes:

If your not, then no you can't download all of the internet, thousands of new pages are being added every minute, and I doubt you have enough HDD space to store the entire "internet" on. :ghostface:

[muchspl2]

http://members.cox.net/ot_space/TheInternet.gif

[100%]

muchspl2
need to Format my floppy in A: \

wait...................

no

wait.......................................


ok

done


muchspl2 - click - ok

[muchspl2]

OK®

[100%]

seriously.....


any answers, would like to know?

[muchspl2]

http://members.cox.net/dodger1954/slap.gif no
so stop being a troll

[TRshady]

Well .. how big is your hard drive?

A site like download.com is at least 100gb alone .. and thats one site as an example mind ... so ... nope... theres no way it can ever be done, unless ...

how many 99minute cd-rs do you have?

[NightStalker]

Originally posted by muchspl2@2 March 2004 - 20:00
http://members.cox.net/ot_space/TheInternet.gif
:lol: Classic! :lol:

[ghoulardi]

This may help you with your Idea.




http://www.numion.com/Calculators/Bytes.html


http://www.numion.com/Calculators/Time.html


:lol: :lol: :lol:

[h1]

:P Enjoy your several hundred petabyte download...

[SeK612]

http://www.archive.org/ (The internet archive) :)

It is possible in theory to back up the internet but with todays technology you would need a very large period of time and a massive storage capacity.

[4th gen]

If you had a T3 connection (which is feasible) then there's still gonna be more data added every second than you can download every second. Thus, unless people stopped putting info up, you'd not be able to download it, unless you could get a faster connection.

[Illuminati]

At the time of this post, Google can search 4,285,199,774 web pages.

Imagine 1 byte of each web page - that's 4,285,199,774 bytes
= 4,712,109 kb
= 4,601 Mb
= 4.5 Gb
That means that to store one simple byte of every web page searched by Google, 4.5Gb is needed.

Now most sites are easily above 100kb, others use 100kb per image so can go up to 10Mb at least.

100kb * 4.5Gb/avg. byte = 4,500 Terabytes

Good luck :rolleyes:

[sharedholder]

I am the INTERNET :lol: :lol: :lol:

[100%]

I was just thinking
In relation to some towns in the world which will
not have world wide internet accesss for many years to come

it could be quite possible to do a "partial" internet which runs only in that town
and then the people there could learn from it and build up there own web.

Every now and then their web gets upgraded from the www
and their enlargements get submitted to the www.

[4th gen]

Originally posted by Zedaxax@3 March 2004 - 12:48
I was just thinking
In relation to some towns in the world which will
not have world wide internet accesss for many years to come

it could be quite possible to do a "partial" internet which runs only in that town
and then the people there could learn from it and build up there own web.

Every now and then their web gets upgraded from the www
and their enlargements get submitted to the www.
They could call it an Intranet maybe? ;)

[100%]

They could call it an Intranet maybe?  ;)
Thats what that is....
and there i was thinking i came up with a geniuos idea.. :frusty:

[Neo 721]

This doesnt realy need much thought, if you mean every HTP site then it will mearly reqire a link to every global web server and a vast amount of storage. Such connection and storage is probably not yet possible but the word future will always come in to it.

[Manny Roscoe]

Why do you need to D/L the ENTIRE Internet for anyway....to set a record or something??

Whats a "Petabyte" anyway? :huh:

[Ariel_001]

You could try using HTTrack (http://www.httrack.com/) on google (http://directory.google.com/ (http://directory.google.com/) would be better) and set it to go anywere. If you do that tell me how far you gone before you ran out of HD space. :P

[vivitron 15]

if the internoet stopped being added to, then it certainly would be possible, but you wouldnt be able to get it all instantaneously: so someone would add to it before you got finished.


having said that, think of Google - it works by caching every webpage it comes to (look below the result, theres a cached link - often differs from the main page) so i guess thats about as near as you will get.

[h1]

Originally posted by sharedholder@3 March 2004 - 08:16
I am the INTERNET :lol: :lol: :lol:
The man's right... :P

And a petabyte is 1,024 exabytes, which is 1,024 terabytes, which is 1,024 gigabytes... :lol:

[Triadcool]

You could try using HTTrack on google (http://directory.google.com/ would be better) and set it to go anywere. If you do that tell me how far you gone before you ran out of HD space. 


I am trying that right now, its been going for about an hour and has downloaded over 4,000 sites and files, im going to let it run all night and tomorrow and see if it fills up my 25gb partition.

[h1]

:ninja: I knew someone would pull crazy shit like that. :ghostface:

Someone's going way over their bandwidth limit... :lol:

[Manny Roscoe]

Holy shit.....they're actuall trying it..... :blink:

What kind of hard drives you guys have?!

I only got a measley 20 gb hd..... <_<

[100%]

Originally posted by Triadcool@4 March 2004 - 06:48

You could try using HTTrack on google (http://directory.google.com/ would be better) and set it to go anywere. If you do that tell me how far you gone before you ran out of HD space.


I am trying that right now, its been going for about an hour and has downloaded over 4,000 sites and files, im going to let it run all night and tomorrow and see if it fills up my 25gb partition.


What site did you start with?
How many did you get?
You could sell it in an auction house later titled "a bit of the internet as it was on 4 of March 2004"

[hooked]

THIS THREAD NEEDS MAJOR








STFU

[Ariel_001]

Originally posted by Zedaxax+4 March 2004 - 11:04--></div><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td>QUOTE (Zedaxax @ 4 March 2004 - 11:04)</td></tr><tr><td id='QUOTE'> <!--QuoteBegin-Triadcool@4 March 2004 - 06:48

You could try using HTTrack on google (http://directory.google.com/ would be better) and set it to go anywere. If you do that tell me how far you gone before you ran out of HD space.


I am trying that right now, its been going for about an hour and has downloaded over 4,000 sites and files, im going to let it run all night and tomorrow and see if it fills up my 25gb partition.


What site did you start with?
How many did you get?
You could sell it in an auction house later titled "a bit of the internet as it was on 4 of March 2004" [/b][/quote]
How far have you gone so far? Am interested to see how far someone can go.
did you change the HTTP user-agent


:ninja: I knew someone would pull crazy shit like that. :ghostface:

Someone&#39;s going way over their bandwidth limit... :lol:

You should make sure you have unlimited bandwidth (or one that is higher that then your drive) :P :lol:

[djweiser]

NOPE IT WOULD NOT&#33;&#33;

1) That would be like trying to download 100000&#39;s of servers worth of data - not possible.
2) New websites come and go everyday, far faster than you could ever download them.
3) How would you ever find the "entire" internet? Many websites may not be listed anywhere.

Also why the heck would you wanna download the entire internet?

:rolleyes:

[Triadcool]

Ok guys I need some help here, I got a couple gigs downloaded and I got THIS (http://www.hostultra.com/~triadcool/untitled.JPG) error and tells me to use the #L option to download more links or something? How do I do this?

[Ariel_001]

Originally posted by Triadcool@4 March 2004 - 22:57
Ok guys I need some help here, I got a couple gigs downloaded and I got THIS (http://www.hostultra.com/~triadcool/untitled.JPG) error and tells me to use the #L option to download more links or something? How do I do this?
Image Resized
[img]http://www.chinkii.com/uploads/album/misc/ht.jpg' width='200' height='120' border='0' alt='click for full size view'> ('http://www.chinkii.com/uploads/album/misc/ht.jpg')


Maximum number of links
Maximum number of links that can be analyzed, that is, either downloaded, or not downloaded. Do not set a too low limit for that, because once the limit is reached, the engine will stop immediately.
Do not set a too high limit, too, because it will take some memory.. 100,000 links (default) is generally enough.


Set this value higher. I think this will work. :D

[Triadcool]

Ok I changed that setting and I started downloading again. So far I have ran it for about half an hour and ive downloaded 644mb 17,979 Files and 19,027 Folders. Im going to let it run while I sleep and hopefully everything will go well and itll still be going when I wake up. Ill tell you how far it is in the morning.

[Ariel_001]

Originally posted by Triadcool@5 March 2004 - 04:16
Ok I changed that setting and I started downloading again. So far I have ran it for about half an hour and ive downloaded 644mb 17,979 Files and 19,027 Folders. Im going to let it run while I sleep and hopefully everything will go well and itll still be going when I wake up. Ill tell you how far it is in the morning.
thats cool. I know this a bit off topic but what do you use to make your desktop look like that? I wanna to try it. :D :P :lol:

[delphin460]

i downloaded the entire internet once but i found it was currupt, out of date and mostly irrelevant lol, so i deleted it and read a book instead

[toco004]

IT IS NOT POSSIBAL&#33;&#33;&#33; unless you have a fast internet conecttion with
a rack of Hard Drives


http://www.cains-cottage.co.uk/smileys/toot.gif :lol:

[Cheese]

If you take all the porn then there probably isn&#39;t much left...

[h1]

Originally posted by toco004@4 March 2004 - 23:37
IT IS NOT POSSIBAL&#33;&#33;&#33; unless you have a fast internet conecttion with
a rack of Hard Drives


http://www.cains-cottage.co.uk/smileys/toot.gif :lol:
It&#39;s not possible at all... I don&#39;t know why some people are trying it. :ninja:

[Cheese]

Serious question though, how much of the internet is porn or porn related in percentage terms...?

[muchspl2]

Originally posted by haxor41789+5 March 2004 - 00:48--></div><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td>QUOTE (haxor41789 @ 5 March 2004 - 00:48)</td></tr><tr><td id='QUOTE'> <!--QuoteBegin-toco004@4 March 2004 - 23:37
IT IS NOT POSSIBAL&#33;&#33;&#33; unless you have a fast internet conecttion with
a rack of Hard Drives


http://www.cains-cottage.co.uk/smileys/toot.gif :lol:
It&#39;s not possible at all... I don&#39;t know why some people are trying it. :ninja: [/b][/quote]
I can only think they are kidding how stupid can someone be to really think they could even see 1:100,000,000 in their lifetime not even talking about downloading any
oh well good luck

[toco004]

why do you even want to back up the net :lol:

[vivitron 15]

Originally posted by Withcheese@5 March 2004 - 04:38
If you take all the porn then there probably isn&#39;t much left...
Quote from ER (i think)

If you took all the porn sites off the internet, there&#39;d only be one site left - it would be a blank page with the words "where did all the porn go?" on it

[Triadcool]

QUOTE (Triadcool @ 5 March 2004 - 04:16)
Ok I changed that setting and I started downloading again. So far I have ran it for about half an hour and ive downloaded 644mb 17,979 Files and 19,027 Folders. Im going to let it run while I sleep and hopefully everything will go well and itll still be going when I wake up. Ill tell you how far it is in the morning.&nbsp;


thats cool. I know this a bit off topic but what do you use to make your desktop look like that? I wanna to try it.&nbsp; &nbsp;


Go HERE (http://www.themexp.org) , thats were I got mine. ;)

[Triadcool]

It&#39;s not possible at all... I don&#39;t know why some people are trying it.



I know its not possible, im not trying to download the whole internet im just seeing how many website and files and folder will download untill I fill up my 25 gb partition.

[ghoulardi]

STOP&#33;&#33;&#33;

Before you try to download the ENTIRE internet you should turn it off first, like others have said, It will be impossible with the rate that new pages are added&#33;&#33;&#33;&#33;&#33;&#33;

If you would like to turn it off then go here

http&#58;//www.turnofftheinternet.com

But try to make it fast because I have to finish downloading every file on kazaa.
:01:


STAYSICK&#33;
GHOULARDI

[Triadcool]

STOP&#33;&#33;&#33;

Before you try to download the ENTIRE internet you should turn it off first, like others have said, It will be impossible with the rate that new pages are added&#33;&#33;&#33;&#33;&#33;&#33;

If you would like to turn it off then go here

CODE&nbsp;
http://www.turnofftheinternet.com



But try to make it fast because I have to finish downloading every file on kazaa.



STAYSICK&#33;
GHOULARDI


HAHA, read above post, I dont need to turn it off.

[Triadcool]

Ive had it running all night and so far ive got 4.40gb downloaded with 173,933 Folders and 170,136 Files . Thats a shit load of files and folders :blink: . I wonder how many there will be by the time the 25gb is full. Well I got to go, ill be back in about 10 hours.

[muchspl2]

Image Resized
[img]http://members.cox.net/xcuse/hahainternet.jpg' width='200' height='120' border='0' alt='click for full size view'> ('http://members.cox.net/xcuse/hahainternet.jpg')

haha I&#39;m download stuff I don&#39;t need and fragmenting my HD&#33;&#33;&#33;&#33;1

[Triadcool]

haha I&#39;m download stuff I don&#39;t need and fragmenting my HD&#33;&#33;&#33;&#33;1

Nothing that windows defrager cant fix. :D

[Triadcool]

I gave up about an hour ago, I needed the bandwidth for other things, HAHA.

[h1]

Well, it had to happen...

[100%]

Conclusion:

after many reptative statements of
"it cant be done"
Triadcool honorably did it
but got bored, so
so yes it is possible to downlowd it.(please refer to "save as" and size of hard disk plus speed)

As to the actual size -
its bigger than the mind can grasp -petagbs

Thank you for the information

the internet is yours

[h1]

You know what&#39;s really hard to grasp? A googoolplex.

How did Google get its name? It was from a number called a googool, which is 10^100 (1 followed by 100 zeroes). Well, a googoolplex is 1 followed by a googool zeroes, which means the number would take forever to write... even in scientific notation.

You can cheat by doing this though...

10^(10^100)

Just sit back and think about it...

It is estimated there are not that many electrons in the universe. :ninja:

[Neo 721]

Originally posted by haxor41789@6 March 2004 - 06:10
You know what&#39;s really hard to grasp? A googoolplex.

How did Google get its name? It was from a number called a googool, which is 10^100 (1 followed by 100 zeroes). Well, a googoolplex is 1 followed by a googool zeroes, which means the number would take forever to write... even in scientific notation.

You can cheat by doing this though...

10^(10^100)

Just sit back and think about it...

It is estimated there are not that many electrons in the universe. :ninja:
Your steping into unknown ground there. But when we say download the internet, what is the internet , images, strings of code, text. No that is the content or data on the internet, so the question should realy be "would it possible to download everything off the internet". In essance the internet is a matrix that is mearly a link to data around the world, that realy was just glued together through means of changes in technology and cannot be classed a material, but and array of fibre optics and copper wires in which data can be passed more easily.