SC Nibbits Map Collection (Topic)

Staredit Network > Forums > General StarCraft > Topic: SC Nibbits Map Collection

SC Nibbits Map Collection

Sep 17 2019, 2:02 am
By: Cerorec

Pages: 1

Sep 17 2019, 2:02 am Cerorec Post #1

Here is all of the SC1 maps on SC Nibbits I could find, including hidden ones and "removed" ones!
Site claims to have: 15,804 maps. (As of 9/16/19)
Total maps found: 15,823 maps!
Size: 2.27 GB (.7z file)
Size: 3.82 GB

(UPDATED 6/24/2020)
Download Here -> https://mega.nz/folder/q2gDgKgD#L6QX4JBcBtYo7ozgAEvKFw
Old Links...
.7z Map Pack PART 1 https://mega.nz/#!RoRR3IDT!XVv_fid3D8b9XW-pSGuBi3Ux_aTx0_uILfccB_rGijQ
.7z Map Pack PART 2 https://mega.nz/file/kpBXEQ6R#sX8ZzeQlVXbSXV4psb-9I50_06apCStO7GcVftaG7lU

I tried contacting the site owner via email about a "Download All" button, but the email doesn't work and returns an error. I can't find any other way to contact the site owner and I might as well just release this full map pack.

How I did it: On http://sc.nibbits.com/maps/browse/?sort=date&order=desc#filters=0 , go to "Show all maps" which includes duplicated maps. Each page has 25 links, except for the last one. Go to the last page which is page 633 and download every single map individually until you come back to page 1. Now, I know what you are thinking. This is very tedious and long process. I used tools to help me out like a script I made in Java and URLOpener.com that opens multiple pages at once to quicken this process. With these tools, and how the site SC Nibbits is built, I could download around 1500 maps in about an hour! The hidden maps weren't on the pages, but were hidden within the "gaps". For ex. on page 1, sc.nibbits.com/maps/get/140996/ is Cat n Mouse Highway V2.4.scx (number being 140996) and sc.nibbits.com/maps/get/140995/ is Arbiter Bound 2(P).scx (number being 140995). Now, Cat n Mouse Highway V2.4.scx is 5th from the line and Arbiter Bound 2(P).scx is 6th from the line, and so there is no number gap between these two maps. However, there are tons of number gaps throughout the whole site, and some gaps range to the 1000s which I skipped. Within these gaps, there are hidden maps, like ones whose pages were deleted by the admin because of its sexual content, like Kerrigan Has Sex With Duke.scm and Strip ashley massaro 2.2.scx. With my script, I obtained every number from this map on this page to that map on that page and used URLOpener.com to open them all at once, downloading all from the working links and obtaining many hidden maps.

My story: When I started this project, I started out slow. I wanted to archive.org every single map and download them all too. I didn't have the tools with me yet, but each day I got a little better at my process. Then I calculated it would take me around 1-2 months to finish this (250 - 500 maps per day). I wanted to finish this fast and 1-2 months was too long for me, so like several days later I found shortcuts and tools to help quicken the process. So basically I archived maps from pages 633 to 281. On the day when I was at page 280, archive.org knew of my activity and implemented, "429 Too Many Requests. You have sent too many requests in a given amount of time." so I couldn't mass archive links anymore. Well, that was a bummer. Afterwards, I just said screw it and just downloaded the maps straight from the site itself, SC Nibbits with my tools. You can archive those download links, but you can't use URLOpener.com or similar sites to archive multiple pages at once anymore... because of me.

Or maybe it only affects me who knows.

The math: The site claims to have 15,804 maps. Ok, let see if that's correct. Each page shows 25 links. The last page (633) shows 4 links. 632 x 25 + 4 = 15,804 maps, which is how the site got its number. However, some of these links direct to maps that are undownloadable, which numbers to 49. On top of that, many links aren't maps at all, but other things like txt and ico files, which numbers to 6. So, 15804 - 49 - 6 = 15749 maps. In other words, this site has 15749 downloadable maps on its pages. Now, the number of hidden maps I've found is 84. If you add it up then 15749 + 84 = 15833 maps. BUT, in my folder I have 15823 maps. Wut... So I must have miscalculated somewhere as the difference is 10. If I did miss a map from the pages, it is probably around 1-4 maps, tops. No more than 4, but I'm confident I have all of the maps from the pages, so "10" has to be from miscalculations from elsewhere, maybe hidden maps, broken maps, or non maps. With the number of 15823, which is higher than 15804, I'm confident to say that I have at least 99% of all of the SC1 maps from SC Nibbits, in my map pack.

Now, there may be more hidden maps on this site. There are gaps numbering in the 1000s which I skipped, so there may have been hidden maps within these gaps. I found around 84 hidden or extra maps, and the number could be higher. I used the link sc.nibbits.com/maps/get/x/ where x is a number, to find maps. This is just a redirect link. For ex. sc.nibbits.com/maps/get/140996/ redirects to sc.nibbits.com/media/maps/140/996/Cat n Mouse Highway V2.4.scx where the map is stored. It may be possible that there are maps that don't have redirect get links, so who knows how many hidden maps this site has.

Well, I think that covers it. In the map pack there are two txt files. I took notes while I was archiving and downloading the maps. I'll post them here too.

Left Off Page & Math.txt

download
http://sc.nibbits.com/maps/browse/?sort=date&order=desc&av=1&p=1

web archive
http://sc.nibbits.com/maps/browse/?sort=date&order=desc&av=1&p=280

https://web.archive.org/save/ (DOESN'T WORK 7STAR BROWSER WITH URL OPENER.COM!!!)
(DOESN'T WORK WITH VIVALDI!, FIREFOX SUCKS! AND DOESN'T WORK! WITH URL OPENER.COM)
(WOW BRAVE BROWSER WORKS!)

633 pages, 1 page has 4 maps
632 x 25 = 15800, 15800 + 4 = 15804
site claims 15,804 maps, which matches, however, dozens of map links are broken
14,830 + 1000 = 15,830, 26+ extra maps?

TOTAL: 15,823

15,823 - 15,804 = 19 more maps

49 broken maps

15804 - 49 = 15755

10+1+1+1+2+1+1+3+4+3+7+7+1+10+24+1+1+5+1 - Extra maps = 84

15755 + 84 = 15839
15839 - 6 = 15833 (6 non maps)
Difference: 15833 - 15823 = 10

WARCRAFT
2463 pages, last page is 25 maps
2463x25=61575

STARCRAFT 2
118 pages, last page is 10 maps
117x25+10=2935

Yep, Warcraft 3 files is over 60000! Idk if I want to take the job of downloading them all. I'll consider it, and hopefully Nibbits.com stays online for a long time hosting these maps. They Must Implement Download All!

Attachments:

Left Off Page & Math.txt
Hits: 0 Size: 0.89kb

Broken File Links.txt
Hits: 0 Size: 7.84kb

Post has been edited 3 time(s), last time on Jun 25 2020, 4:26 am by Cerorec.

None.

Sep 17 2019, 5:07 am

NudeRaider Post #2

We can't explain the universe, just describe it; and we don't know whether our theories are true, we just know they're not wrong. >Harald Lesch

Nice!

Is there a reason you skipped the larger gaps? Should be no problem with your tools if thousands of pages turn out empty, right?

Sep 19 2019, 6:49 am Cerorec Post #3

Quote from

NudeRaider

Nice!

Is there a reason you skipped the larger gaps? Should be no problem with your tools if thousands of pages turn out empty, right?

Yea... I should since I now I have tools. At first I didn't have the tools, and I was using archive.org, so I thought it would take too much time going through each and every number. Archive.org is quite slow to use too... due to a lot of things.

At first, 20 pages = 500 maps = 1 hour of my time each day = about 600 links visited, but now with my tools and techniques it was like 1500 maps per hour, going through like 3000+ links. What I'll probably do next is go through the entire numbers, from 1 to 141003 the latest map. But, the first map uploaded starts at number 14995, which is odd, so like http://sc.nibbits.com/maps/download/14995/ or http://sc.nibbits.com/maps/get/14995/.

None.

Sep 19 2019, 7:27 am Cerorec Post #4

I have a feeling this is going to take a very long time... I just opened up like
http://sc.nibbits.com/maps/get/1001
http://sc.nibbits.com/maps/get/1002
.
.
.
http://sc.nibbits.com/maps/get/1999
http://sc.nibbits.com/maps/get/2000
999 links at once, took around 4-5 minutes to load all pages completely using Brave Browser, and they are all "Page Not Found" links.

There may be other complications along the way. Stay tuned!

To add, probably the biggest reason I skipped the gaps because I assumed they were empty like "Page Not Found" links and to save time. That was before I had the tools, and I was starting out slow and tedious. It was not until like midway I discovered there are extra maps not found on those pages. So yea... sorry it took me awhile to refresh my memory.

Post has been edited 1 time(s), last time on Sep 19 2019, 7:49 am by Cerorec.

None.

Options

Pages: 1

Back to forum
Please log in to reply to this topic or to report it.
Members in this topic: None.

Global Shoutbox

[01:39 am]

Ultraviolet -- no u elky skeleton guy, I'll use em better

[10:50 pm]

Vrael -- Ultraviolet

Ultraviolet shouted: How about you all send me your minerals instead of washing them into the gambling void? I'm saving up for a new name color and/or glow

hey cut it out I'm getting all the minerals

[10:11 pm]

Ultraviolet --

[10:11 pm]

Ultraviolet -- How about you all send me your minerals instead of washing them into the gambling void? I'm saving up for a new name color and/or glow

[2024-4-17. : 11:50 pm]

O)FaRTy1billion[MM] -- nice, now i have more than enough

[2024-4-17. : 11:49 pm]

O)FaRTy1billion[MM] -- if i don't gamble them away first

[2024-4-17. : 11:49 pm]

O)FaRTy1billion[MM] -- o, due to a donation i now have enough minerals to send you minerals

[2024-4-17. : 3:26 am]

O)FaRTy1billion[MM] -- i have to ask for minerals first tho cuz i don't have enough to send

[2024-4-17. : 1:53 am]

Vrael -- bet u'll ask for my minerals first and then just send me some lousy vespene gas instead

[2024-4-17. : 1:52 am]

Vrael -- hah do you think I was born yesterday?

Please log in to shout.

Members Online: Mysylia52,

Roy, jun3hong