Staredit Network > Forums > Technology & Computers > Topic: Cross-OS MPQ handling
Cross-OS MPQ handling
Apr 16 2008, 3:29 pm
By: fatimid08  

Apr 16 2008, 3:29 pm fatimid08 Post #1



I have been working on MPQ code which can run on Mono (hence the cross-OS) for the past few months, and I am to the point where I can extract files using zlib or pklib compression properly on my Windows computer. I have no way of testing on a Mac, and the only Linux boxes I have access to are webservers or dns servers, so I don't want to test there.

So, I humbly ask any Mac and Linux user willing to help me out to run the test program attached to this post, which is a simple GUI which shows files based on the listfile in the archive and allows to extract 1 file at a time (so no stardat.mpq unless yours has a listfile in it). Also, I don't think it's going to work on a big-endian machine. And don't try to extract sounds under penalty of application crash (huffman isn't quite working yet).

According to the mono migration analyzer, there is no code in there which shouldn't run on version 1.2.5, 1.2.6, and 1.9.

If the attachment doesn't work for some obscure reason, here is a download link.

Once I get all of decompression working properly, I'll post the source and then move on to adding files to archives.
It'll all probably be GPLed, I think StormLib is, and this library is heavily based on it (I hope I'm not infringing on it by posting a build without the source right now, if I am, I'll post it).

Update:
This is now fully in C#, so there is only 1 dll + the tester application. No more mono-basic dependency in case that was annoying. Above post, attachment, and download link updated.

Attachments:
MpqTester.zip
Hits: 2 Size: 65.54kb

Post has been edited 1 time(s), last time on Jun 5 2008, 10:32 pm by fatimid08.



None.

Apr 16 2008, 5:38 pm Twitch Post #2



Not to bust a bubble but there a modding tool for every OS now.
Thanks to shadowflare and the maker of the python tools.



None.

Apr 16 2008, 5:47 pm fatimid08 Post #3



Lol, I check the site after an hour or so, and the post is like 15 seconds old...

I know, but using SC is my personal way of getting better at programming, so that's that. Plus combining hobbies is great no? I like programming and SC, so 2 in 1 combo. And programming scales better with my schedule than actually playing SC, so programming for SC even if it's already been done is more of a selfish thing on my part than a service I want to make to the mapping/modding community.



None.

Apr 16 2008, 9:49 pm Syphon Post #4



Quote from Twitch
Not to bust a bubble but there a modding tool for every OS now.
Thanks to shadowflare and the maker of the python tools.

Mac has no mem editor.



None.

Apr 17 2008, 3:22 am Praetor Post #5

layin' in the cut

Macs cannot run EXE or DLL files. Those are Windows-only.



this too shall pass

Apr 17 2008, 3:34 am Nai Post #6



Mem editor will be a summer project for me.



None.

Apr 17 2008, 6:52 am ShadowFlare Post #7



Quote from fatimid08
Also, I don't think it's going to work on a big-endian machine.
If that doesn't work, then it is a bug in Mono. (assuming there exists a version of Mono that runs on a big-endian architecture)



None.

Apr 17 2008, 9:54 am fatimid08 Post #8



Quote from Praetor
Macs cannot run EXE or DLL files. Those are Windows-only.
Macs can run exes built using the .Net Framework compilers if you use Mono. for more info, refer to www.mono-project.com



None.

Apr 17 2008, 10:26 pm Syphon Post #9



Quote from Praetor
Macs cannot run EXE or DLL files. Those are Windows-only.

What the fuck do I care, and what does that have to do with it not having a mem editor? Just because SC isn't an .exe doesn't mean SC isn't loaded into the memory.



None.

Apr 17 2008, 10:34 pm Forsaken Archer Post #10



Can it run from command line?
Can I extract specific files via file name?

Currently, SEN uses mpq-tools (https://babelize.org/mpq-tools.php) to get .chks out of maps by making an educated guess at which file it is. Although it seems to work 99% of the time, I would like to make it simpler.
I do notice that mpq-tools actually updated from .3 to .4. I thought it was a dead project ^^. I'll have to look at that later.



None.

Apr 17 2008, 10:37 pm fatimid08 Post #11



Euh Syphon, I don't think he was referring to you. I think it's because my attachement contains an exe and 2 dlls, and he thought they wouldn't run on Mac, but those are specially built to run with Mono on any platform it supports (of which Mac is one).

Quote
If that doesn't work, then it is a bug in Mono. (assuming there exists a version of Mono that runs on a big-endian architecture)

Forgot to answer that. MPQs are made little-endian even on big-endian systems I believe, and the BitConverter class will read as big-endian if I understand its behaviour correctly. That could make for interesting results.

Quote
Can it run from command line?
Can I extract specific files via file name?
And no, it doesn't run from command line, that's just a test app to make you people test if it works on Mac and Linux. If it does, I'll have to make the map screenshot app Yoshi used on Maplantis for a time use that, and then that app does work from the command line.

It does extract via name, it's the only option right now as a matter of fact. Send name and locale, and it extracts for you.



None.

Apr 17 2008, 11:22 pm Praetor Post #12

layin' in the cut

Quote from fatimid08
Euh Syphon, I don't think he was referring to you. I think it's because my attachement contains an exe and 2 dlls, and he thought they wouldn't run on Mac, but those are specially built to run with Mono on any platform it supports (of which Mac is one).
That is correct.



this too shall pass

Apr 18 2008, 9:45 am ShadowFlare Post #13



Quote
Forgot to answer that. MPQs are made little-endian even on big-endian systems I believe, and the BitConverter class will read as big-endian if I understand its behaviour correctly. That could make for interesting results.
If Mono does not do the conversion, then that would very severely break things, and the purpose of Mono is compatibility, so I see no reason for them to make it so programs running in Mono need to care about big endian on the target architecture.

Think of it this way. Programs that were only developed on Windows for the .NET Framework, which by design Mono is intended to be able to run, would not run properly (if at all) on a big endian system even in Mono if Mono was designed in the way you say. That's kind of like saying that a port of an x86 emulator to a big endian architecture should run the emulated x86 cpu in big endian mode instead of what it is supposed to be -- it just doesn't make sense.

Of course many parts would use big endian internally, but to the program it is running, it must work as if it is little endian, as the program expects.

Post has been edited 1 time(s), last time on Apr 18 2008, 9:52 am by ShadowFlare.



None.

Apr 18 2008, 2:18 pm fatimid08 Post #14



Your argument makes sense, but then what's the use of the IsLittleEndian field? Maybe the BitConverter class is architecture dependent, and for independence, you would need to use the BinaryReader or something.

Also, has anyone been able to run the test app on Mac or Linux successfully?



None.

Apr 18 2008, 8:39 pm Praetor Post #15

layin' in the cut

I would, but at the current time I cannot install mono because it requires a password (as of now I don't know it).



this too shall pass

Apr 20 2008, 10:54 am ShadowFlare Post #16



Quote from fatimid08
Your argument makes sense, but then what's the use of the IsLittleEndian field? Maybe the BitConverter class is architecture dependent, and for independence, you would need to use the BinaryReader or something.

Also, has anyone been able to run the test app on Mac or Linux successfully?
Heh, I'm sure Microsoft has put in some weird things here and there in the .NET Framework classes that aren't particularly useful. Maybe if Microsoft made a version for some big endian architecture they would do that, but people who want to make things that are actually compatible aren't likely to implement something in a way that would be incompatible just because Microsoft intended to have it that way. ;)



None.

Apr 23 2008, 3:16 am Laser Dude Post #17



I like your map editor, so I gave this a shot. I installed Mono, and then tried to run your program, to no success. My OS is Ubuntu 7.10 Gutsy

Code
laserdude@laserdude-desktop:~/MonoMPQ$ mono MpqTester.exe

** (MpqTester.exe:6863): WARNING **: The following assembly referenced from /home/laserdude/MonoMPQ/MpqTester.exe could not be loaded:
    Assembly:   Microsoft.VisualBasic    (assemblyref_index=1)
    Version:    8.0.0.0
    Public Key: b03f5f7f11d50a3a
The assembly was not found in the Global Assembly Cache, a path listed in the MONO_PATH environment variable, or in the location of the executing assembly (/home/laserdude/MonoMPQ/).


** (MpqTester.exe:6863): WARNING **: Could not load file or assembly 'Microsoft.VisualBasic, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
The entry point method could not be loaded
laserdude@laserdude-desktop:~/MonoMPQ$


It appears I don't have VB installed. I'd say it's pretty backwards for something 'cross-platform' to require something that's not cross platform.



None.

Apr 23 2008, 10:03 am fatimid08 Post #18



I thought the VB runtime came with Mono, wtf...

Found a few things or their page:

Quote
Starting with Mono 1.2.3, packages of the Mono Visual Basic Compiler and Visual Basic Runtime are available at our Downloads page.
and also
Quote
includes the Visual Basic 8 compiler written in Visual Basic 8, and Mainsoft's Visual Basic runtime for 2.0
and
Quote
VB.NET compiler and runtime were released.
for the current 1.9 version

That's why I decided to do it on VB.Net, because they said they packaged it with Mono. If you can't find it, I'll start porting it all to C# next week (I didn't use it in the first place because it takes me more time to work with C-like syntax compared to VB syntax).

Also, the MoMA tool which analyzes wether or not everything I call is supported by Mono tells me it should all work, so I really don't know.



None.

Apr 23 2008, 9:33 pm Cole Post #19



This might be useful..not sure
http://rpmfind.net/linux/rpm2html/search.php?query=mono(Microsoft.VisualBasic)

Simply various Microsoft.VisualBasic runtimes for mono based on the OS you're running.



None.

Apr 27 2008, 9:36 am ShadowFlare Post #20



I could collaborate with you on this project if you want, including helping with the VB to C# conversion, if you decide to do so.

I also started an mpq library for .NET (but in C#) a while back. I think I had it coded to the point of being able to extract uncompressed files, but I haven't continued from there yet.



None.

Options
  Back to forum
Please log in to reply to this topic or to report it.
Members in this topic: None.
[01:19 pm]
Vrael -- IM GONNA MANUFACTURE SOME SPORTBALL EQUIPMENT WHERE THE SUN DONT SHINE BOY
[01:35 am]
Ultraviolet -- Vrael
Vrael shouted: NEED SOME SPORTBALL> WE GOT YOUR SPORTBALL EQUIPMENT MANUFACTURING
Gonna put deez sportballs in your mouth
[2024-5-01. : 1:24 pm]
Vrael -- NEED SOME SPORTBALL> WE GOT YOUR SPORTBALL EQUIPMENT MANUFACTURING
[2024-4-30. : 5:08 pm]
Oh_Man -- https://youtu.be/lGxUOgfmUCQ
[2024-4-30. : 7:43 am]
NudeRaider -- Vrael
Vrael shouted: if you're gonna link that shit at least link some quality shit: https://www.youtube.com/watch?v=uUV3KvnvT-w
Yeah I'm not a big fan of Westernhagen either, Fanta vier much better! But they didn't drop the lyrics that fit the situation. Farty: Ich bin wieder hier; nobody: in meinem Revier; Me: war nie wirklich weg
[2024-4-29. : 6:36 pm]
RIVE -- Nah, I'm still on Orange Box.
[2024-4-29. : 4:36 pm]
Oh_Man -- anyone play Outside the Box yet? it was a fun time
[2024-4-29. : 12:52 pm]
Vrael -- if you're gonna link that shit at least link some quality shit: https://www.youtube.com/watch?v=uUV3KvnvT-w
[2024-4-29. : 11:17 am]
Zycorax -- :wob:
[2024-4-27. : 9:38 pm]
NudeRaider -- Ultraviolet
Ultraviolet shouted: NudeRaider sing it brother
trust me, you don't wanna hear that. I defer that to the pros.
Please log in to shout.


Members Online: Roy