GPTP Integration! 🥳

GPTP DatExtend Repo

PyMS: https://github.com/poiuyqwert/PyMS
Right click list in PyDAT and add entries.

More details in this post: http://www.staredit.net/377086/

---

Previous post:

Update:
Uploaded a modified DatCC so you can decompile dat files to ini and recompile them to have more entries with a -n command-line option. Also updated the plugin, at clokr's request, to have sprites.dat have every value for every entry. No real progress on the plugin itself, but at least now it is usable!

* Fixed DatCC so that images.dat's blank entry won't crash when loading SC. It's not a huge issue, unless you don't use the images you add.

---

Here is a demo of the dat extender plugin. The plugin lets you specify how many entries are in each dat file. It just creates some new memory and replaces all references to the existing dat arrays to the new memory, and replaces a few other things to make it work.

The only dats I have working currently are units.dat, flingy.dat, sprites.dat and images.dat.

There are some thing about Units.dat, though:
- Every property in the extended units.dat is available to every entry. DatEdit won't show them except for the ones they are normally limited to.
- The flingy.dat ID (graphics) is normally 1-byte, so there is a patch that expands it to 2 bytes so you can have more than 256 flingies. This probably should just always be done, but right now it's an option.
- Due to some of SC's code that reads several entries at a time (for some reason), units.dat must be a multiple of 12.
- IDs 228 to 250 are used for special cases (as seen in FireGraft), so instead of finding and changing all of them these IDs have been reserved as special use and should not be used.
- Because of how FireGraft works, I had to copy the button tables to some unused memory (unit count tables, kill count table, and death table) which gives a maximum limit of 912 units. I could probably find new memory if this is ever a problem, but I don't imagine it will be necessary.

Some of the other dats I haven't done yet might have similar things, but I haven't looked in to them fully yet.


Using the plugin currently is a bit awkward. There's not really an easy way to add entries to a dat file (if only there were a dat editor that would let you add entries...). DatEdit is pretty easy to change to read an expanded dat file, other than the previously mentioned problem with units.dat, by just changing the .def files, add entries to the data lists and add them to the origins lists. FireGraft is a bit tricky though. Increasing the unit counts and adding to the unit lists is easy enough (found in fgd file and unitlist.txt in the data.mpq), except FG reads data directly from the EXE so I had to make a modified EXE with the data copied to the corresponding location so it wouldn't break.

Once you actually have the .dat files, the plugin is pretty straightforward. Just add it to MPQDraft or FireGraft and hit Configure and it will let you enter the new entry counts (or save/load them from a config file). I would have just automatically counted them from the filesizes, except the plugin runs before the MPQs are loaded.. I could probably add a bunch of code in to SC to do it, but this is easier.

I attached a demo I've been using to test. It has 252 units (2 usable ones), +10 flingies, +1 sprite, and +1 image. The two new units are buildable at a supply depot. They're just random units I created to be usable and test it out.
If anybody wants, either just play around with it in games to see if anything crashes or if anyone can think of places unit counts or IDs are that I missed. One obvious one visible is the extended units don't have names. Other ones is I haven't modified triggers in any way to read them. Or tell me which files I should focus on next.

Attachments:

Post has been edited 12 time(s), last time on Oct 26 2022, 6:03 am by FaRTy1billion.

You're a real mother fucker, Farty. Thanks a ton. Keep us all updated on the improved usability and such.

Played with it a bit, interceptors aren't working properly. (Or was it just due to this test mod?)
One array whose size depends on unit entry count and hasn't been handled is the ai strength array at 006BB210. Could that be redirected next? =)

Edit: Also 006902D8 for ai build limits.

Post has been edited 1 time(s), last time on May 22 2016, 4:10 pm by Neiv.

What!

Hope something like this could be done to increase the iscript entries.

Nice work!

Now this is getting interesting.

Let's see what we can do with this bad boy.

oshit, that might be necessary.

I'll look at interceptors, they may just be some random thing I missed. And thanks for the addresses! I'll look at those.

Do you know how big these are, or their formats? I see their indexes multiplied by weird numbers, so I don't know what they mean. 006BB210 seems to make sense (int thing[228]), except somewhere the index is multiplied by 912 (sizeof(int) * 228) which makes me think there is more after it, or more than one table here? 006902D8 is [something * 1256 + something + 006902D8], and I don't know what 1256 means.

EDIT:
After looking at 0x006BB210 + 912 (which shares similar functions to 0x006BB210) it looks like there's just the two arrays, especially since 0x006BB210 + 912*2 is a single value. I still don't know what 1256 is.

Post has been edited 2 time(s), last time on May 26 2016, 5:35 am by FaRTy1billion.

Oh, right. 006BB210 should be 2 * 228 32-bit ints, one for unit's strength vs. air and one for vs. ground.

006902D8 is a part of a larger structure. Guess it causes complications?
This structure starts at 0068FEE8, has 8 entries (1 for each player), and is 0x4e8 bytes large.
(So the unit build limit array is at offset 0x3f0 from the struct's beginning)

So there's 157 entries per player? Or each entry is 0x4E8? I'm just not sure how 0x4E8 relates to the unit count, or how much I need to add to it.

EDIT:
So would it be 0x4E8 for each player, and then each unit has 1 byte, so I'd just add as many bytes as units added * 8, and then just adjust the size of the struct (the all of the *1256)? It looks like if this is true then there are 20 bytes at the end, but that shouldn't be a problem because those unit IDs shouldn't be used.

There is a structure which is would be something like this:


That makes it total 0x4e8 bytes per struct, and there are 8 of them.

I don't think you should resize the entire struct, but figure a way to get the build limits to a separate location? Or if the entire struct is resized, you'd have to redirect everything that accesses something else from that struct.

Ok, that's what I (eventually) thought.

Whichever works. The only problem with moving the part of it is if something references it from the struct's base address, such as when zeroing it. But that shouldn't be difficult.

Hm, good point. Zeroing should be simple, but the struct does get saved, so moving it partially requires more fixes in the function which saves it.

It shouldn't be too hard to just resize and move the whole struct. All of the copy tables in the plugin code are pretty much designed exactly for that purpose.

EDIT:
I only found 487 references to the table, so it shouldn't be that bad.


EDIT2:
Ok, I think got those two tables. I don't really know how to test it (the game doesn't crash in the 30 seconds I tried it, and the AI was doing stuff).

There were a bunch of constants that might be significant that I kept seeing but didn't know what to do with, so I ignored them.
Ones I wrote down some addresses to were 0x2710 (remarkably similar to 0x4E8*8, which is 0x2740) and 0x26A0 (which is (0x4E8 - 20) * 8).


I didn't look in to the interceptor/scarab thing (other than it appears that they require a second order in order to acknowledge the order -- attacking/whatever once will launch them, but you must attack again for the launched interceptors to attack). I'll either do that next, or work on more .dat files. Hopefully I have some time this weekend.

Attachments:

Post has been edited 3 time(s), last time on Jun 4 2016, 11:50 pm by FaRTy1billion.

That's Great to know, Also as a suggestion, i guess add more units with their own grps? They might maybe share iscript entries for some of them? Just to start testing sprite data?

Post has been edited 1 time(s), last time on May 27 2016, 7:50 am by BeatMeistro.

I haven't added GRPs, but you should just be able to add lines to images.tbl. I tested sprites/images by making the added marine hallucinated.

Yeah, adding entries to images.tbl has always worked, as has adding new iscript.bin entries. You'll just have to type the numbers by hand in iamges.dat, or modify the dat editors's default data files. And if the .dat editors don't find .grps for their previews, they spit out annoying error messages (but work properly anyways)

WOW! Very cool!!! By the way, It is compatible with GPTP ?

Probably not as it is. I was looking at GPTP to see if maybe I could make a custom header or something to make it work, and it is possible but I'll have to play with it later.

Uploaded a modified DatCC so you can decompile dat files to ini and recompile them to have more entries with a -n command-line option. Also updated the plugin, at clokr's request, to have sprites.dat have every value for every entry. No real progress on the plugin itself, but at least now it is usable!

DatCC download and stuff is in the first post.



An Example for expanding units.dat:

Decompile default units.dat extended to 264 entries.

Re-compile units.dat to 264 entries and use 2-byte Flingy IDs.

See the readme or --help for the other dat files.

Note that the file loaded entries, the base dat file entries, the decompile -n value, and the compile -n value can all be totally different. Any extra entries are ignored, any missing entries are filled in with blank entries.


Also tell me if there are any bugs with the datcc. I didn't really test it.

Post has been edited 5 time(s), last time on Jun 5 2016, 12:02 am by FaRTy1billion.

I tried but maybe my mind its a bit little to understand; how exactly it works? I execute the DatCC.exe file and it opens and instantly closes. Perhaps the problem are my poor abilities programming or is there something wrong with another topic about the program?