Staredit Network > Forums > Modding Discussion > Topic: Many Manly Manmade Modern Modding... Programs
Many Manly Manmade Modern Modding... Programs
Jul 14 2017, 8:45 pm
By: Pr0nogo  

Oct 16 2018, 10:12 pm Lagi Post #21



hi,

when i use PY or dataEdit to change some value (like SCV set HP to 10 f.ex), i can not make .mpq which would be applied in .exe created by FireGraf 0,94 (where can i change the tech-tree, f.ex. allow to build Academy without Barrack).

If i use dataEdit to make .exe all change done in dataEdit work.
if i use firegraf to make .exe all change done in firegraf work.

but i cannot combine both of them.

any quick tutor?

edit: SOLVED

first do stuff in dataEDIT => export (both mpq & exe will work with FireGraft)
second do stuff in FireGraf => save as .exe
will pop up ("Would you like to copy an archive into the saved file?") select YES : select your file made in dataEdit

Post has been edited 1 time(s), last time on Oct 17 2018, 6:02 am by Lagi.



None.

Oct 17 2018, 8:06 am Pr0nogo Post #22



You really shouldn't be using DatEdit and you shouldn't be saving your .dat changes to its own exe. Instead use PyMPQ to create a new mpq archive, add files to that, then add that to your firegraft exe using the third step you wrote.




Jan 9 2019, 12:19 pm IlyaSnopchenko Post #23

The Curious

One more question: does the PYMS (or, to be precise, PYDAT) not support the Remastered .BIN format? Or am I doing something wrong again? :)



Who am I, and why should I care?

Jan 9 2019, 6:27 pm Pr0nogo Post #24



I'm almost certain that no tools currently work with the SCR bin format, maybe animosity can change the webms they use for animations but actually editing the UI's functions and button placements hasn't been attempted yet as far as I'm aware.




Jan 9 2019, 8:42 pm poiuy_qwert Post #25

PyMS and ProTRG developer

Quote from IlyaSnopchenko
One more question: does the PYMS (or, to be precise, PYDAT) not support the Remastered .BIN format? Or am I doing something wrong again? :)
I assume you meant PyBIN for editing the Dialog/UI BIN format? As Pr0nogo mentioned PyMS does not currently support that, or really any SCR stuff (there is a branch which has some preliminary support for reading CASC's, but thats it).

This seems to be the most requested feature recently, so when I get back into updating PyMS (life is chilling out right now, and just got a new laptop and building a new windows PC, so hopefully I'll have time soon!) it will probably be the highest priority new feature. I think FaRTy already has the specs so I would just need to get that from him, and I don't think its changed very much, so should be pretty easy to support (though probably without webm previewing).




Jan 10 2019, 4:33 am Pr0nogo Post #26



The highest priority features are definitely related to PyAI and PyTILE :)




Jan 10 2019, 4:55 am poiuy_qwert Post #27

PyMS and ProTRG developer

Quote from Pr0nogo
The highest priority features are definitely related to PyAI and PyTILE :)
I don't think there are any high priority new features for PyAI and PyTILE right now? Or did you mean bug fixes? Bug fixes are of course higher priority than new features.

Edit: I just noticed that this thread is linking to Neivv's forked version of PyMS on github. I don't think that is the correct place to link to for a few reasons:
- It does not contain all the fixes and changes I make to PyMS on the main repository
- The version checks in PyMS are checking the main PyMS repository
- If users create github issues, I would like to have them on the main PyMS repository so it can be fixed there
- The main changes that Neivv is doing on his fork is adding custom AI opcodes. This is great, one of the things I've wanted with it being open source is people being able to do custom things they may need, and I wish it happened more. But these custom opcodes are not really available to anyone (at least at the moment), so these additions can only confuse people. If Neivv ends up releasing a plugin with custom opcodes, then I would hope he would create a pull request and his additions would be merged into the main PyMS repository.
- Neivv has submitted fixes to the main PyMS repository, and I hope he continues to do so (another great part of being open source), so you shouldn't be missing out on any fixes he implements.
Please consider changing the link to my PyMS repository: https://github.com/poiuyqwert/PyMS

Post has been edited 1 time(s), last time on Jan 10 2019, 5:44 am by poiuy_qwert.




Jan 10 2019, 5:57 am Pr0nogo Post #28



The AI opcodes are fully available to everyone, I have links in many places and especially in my discord server where troubleshooting and help are provided in real time. I'll add them here.

I've added a link to the main PyMS, but if it's possible to merge neiv's work it'll reduce clutter and confusion. Thanks for the updates.

P.S. please open a dedicated PyMS maintenance/update thread similar to SCMDraft's.

Post has been edited 1 time(s), last time on Jan 10 2019, 6:15 am by Pr0nogo.




Jan 10 2019, 8:25 am IlyaSnopchenko Post #29

The Curious

Quote from poiuy_qwert
Quote from IlyaSnopchenko
One more question: does the PYMS (or, to be precise, PYDAT) not support the Remastered .BIN format? Or am I doing something wrong again? :)
I assume you meant PyBIN for editing the Dialog/UI BIN format? As Pr0nogo mentioned PyMS does not currently support that, or really any SCR stuff (there is a branch which has some preliminary support for reading CASC's, but thats it).

This seems to be the most requested feature recently, so when I get back into updating PyMS (life is chilling out right now, and just got a new laptop and building a new windows PC, so hopefully I'll have time soon!) it will probably be the highest priority new feature. I think FaRTy already has the specs so I would just need to get that from him, and I don't think its changed very much, so should be pretty easy to support (though probably without webm previewing).
Gah! Sure, I mean PYBIN, got a brain bug after a work day. It's not something that is a deal breaker for me of course, just been curious.

Might I suggest another feature to implement in PYDAT though? Back in the day (15+ years ago) I was editing - say - Portdata.dat, adding links to nonexistent-by-default SMK dir strings (that I created in Portdata.tbl, of course) to make use of some portraits that by default point to something else (say, all these beta portrait entries like Merc Gunship or Peter, which by default point to marine portrait). I used them for custom units when I was making a partial conversion back in 2003ish and it worked. My point is, PYDAT by default reverts these "extra" values to Flag12 (last entry in Portdata.tbl) so is it possible to enable using "non-standard" values for SMK dir? (I presume there's stuff like this that can be added to other similar lists - say, images.dat - but this is the one that came to mind first). Maybe it could be an option. Or can I just edit Portdata.tbl in the PYMS folder (under Libs\MPQ\Arr) and it'll work?

Of course I'm not sure that creating "new" entries (at least as far as file paths are concerned) will work in Remastered. Also, does the engine still use .tbl files internally for the ARR\ stuff? Just want to be sure.

P.S. Another thing regarding PYICE (and presumably the other utilities where you have to write pseudocode): can you add the feature to toggle INSERT/OVERWRITE in text boxes (obviously by pressing INS)?

P.P.S. I think the few unknowns in Units.dat > Advanced had been decoded a long time ago? They were labelled properly in Arsenal III but for some reason not DatEdit. I'm referring to the 0x01 ... 0x80 flags. They're supposed to be the movement properties: 0x01 is non-creep (unchecked for larvae/eggs), 0x04 is flying, 0x40 is ground, 0x80 is hovering (as in vulture / archon / peon...) Or has that been disproved?

All in all, thanks for the great modding kit. I have most of the old stuff on my hard drive (gathering dust there for about 15 years or more) but many programs no longer work due to missing runtime libraries - although I had a few set stashed away too - and just some general weirdness. For example I miss the old MPQ Viewer with built-in WAV player. Nice to have all of this now in a single, uniform suite.

Post has been edited 1 time(s), last time on Jan 10 2019, 8:32 am by IlyaSnopchenko.



Who am I, and why should I care?

Jan 10 2019, 8:15 pm poiuy_qwert Post #30

PyMS and ProTRG developer

Quote from Pr0nogo
The AI opcodes are fully available to everyone, I have links in many places and especially in my discord server where troubleshooting and help are provided in real time. I'll add them here.
There still an issue of people having to know about it. If users try to make AI's and see these commands they won't necessarily understand that they are not built in and require a plugin to use. It has to be handled gracefully, similar to the way that TrigPlug is supported by PyTRG.


Quote from Pr0nogo
I've added a link to the main PyMS, but if it's possible to merge neiv's work it'll reduce clutter and confusion. Thanks for the updates.
I would love to. I had no idea that his work was as complete as it is, and with a public plugin. I don't have the time to follow all the things like this, I rely on people submitting bug reports and feature requests to drive development.


Quote from Pr0nogo
P.S. please open a dedicated PyMS maintenance/update thread similar to SCMDraft's.
There was an official thread but it got pushed off the first page, and then I essentially commandeered the "Problems with Current Modding Tools" thread since there are not really many other tools under active development. A new thread sounds like a good idea. Edit: Created a new thread: http://www.staredit.net/topic/17719/


Quote from IlyaSnopchenko
Might I suggest another feature to implement in PYDAT though? Back in the day (15+ years ago) I was editing - say - Portdata.dat, adding links to nonexistent-by-default SMK dir strings (that I created in Portdata.tbl, of course) to make use of some portraits that by default point to something else (say, all these beta portrait entries like Merc Gunship or Peter, which by default point to marine portrait). I used them for custom units when I was making a partial conversion back in 2003ish and it worked. My point is, PYDAT by default reverts these "extra" values to Flag12 (last entry in Portdata.tbl) so is it possible to enable using "non-standard" values for SMK dir? (I presume there's stuff like this that can be added to other similar lists - say, images.dat - but this is the one that came to mind first). Maybe it could be an option. Or can I just edit Portdata.tbl in the PYMS folder (under Libs\MPQ\Arr) and it'll work?
Most if not all PyMS programs (including PyDAT) attempt to ensure that what you are doing is correct, so you don't run into crashes where you have no idea what is going on. For most programs to do that, they load many additional files internally to check for consistency (for example PyDAT making sure the portraits you reference are actually in the portdata.tbl). As a convenience, the programs come setup to load their default files, so you can get to work right away. But as you work on your mod and things change, you need the programs to load your modded files. All programs have settings where you can choose the MPQ's to load, as well as override which additional files are loaded (either from MPQ's or from a folder). You should never update the files in `Libs\MPQ\Arr`, you should just have the programs load your own files. I can provide more details/steps/screenshots when I get home if you need it.


Quote from IlyaSnopchenko
Of course I'm not sure that creating "new" entries (at least as far as file paths are concerned) will work in Remastered. Also, does the engine still use .tbl files internally for the ARR\ stuff? Just want to be sure.
I'm not really sure how SCR works for this, sorry.


Quote from IlyaSnopchenko
P.S. Another thing regarding PYICE (and presumably the other utilities where you have to write pseudocode): can you add the feature to toggle INSERT/OVERWRITE in text boxes (obviously by pressing INS)?
I'm not sure if the UI framework I am using supports that. I can take a look though.


Quote from IlyaSnopchenko
P.P.S. I think the few unknowns in Units.dat > Advanced had been decoded a long time ago? They were labelled properly in Arsenal III but for some reason not DatEdit. I'm referring to the 0x01 ... 0x80 flags. They're supposed to be the movement properties: 0x01 is non-creep (unchecked for larvae/eggs), 0x04 is flying, 0x40 is ground, 0x80 is hovering (as in vulture / archon / peon...) Or has that been disproved?
I don't really remember exactly, but I thought most of the stuff in DatEdit/PyDAT had been confirmed. Other people probably know better at this point, so if there are any changes I hope its reported.


Quote from IlyaSnopchenko
All in all, thanks for the great modding kit. I have most of the old stuff on my hard drive (gathering dust there for about 15 years or more) but many programs no longer work due to missing runtime libraries - although I had a few set stashed away too - and just some general weirdness. For example I miss the old MPQ Viewer with built-in WAV player. Nice to have all of this now in a single, uniform suite.
I'm glad you like it. It only gets better as more people use it and report bugs and feature requests, so don't be shy!

Post has been edited 1 time(s), last time on Jan 10 2019, 9:09 pm by poiuy_qwert.




Jan 14 2019, 8:28 am IlyaSnopchenko Post #31

The Curious

Thanks. I'll try the TBL files when it becomes relevant (so far, there's a dearth of SCR portraits anyway). My project has no MPQ per se - just a folder structure I pack into a SAMASE exec when I want to test - so I'll see what I can try.

Any bugs / issues that I may find, I'll report in the new PYMS thread.



Who am I, and why should I care?

Jan 15 2019, 12:32 am poiuy_qwert Post #32

PyMS and ProTRG developer

Quote from IlyaSnopchenko
Thanks. I'll try the TBL files when it becomes relevant (so far, there's a dearth of SCR portraits anyway). My project has no MPQ per se - just a folder structure I pack into a SAMASE exec when I want to test - so I'll see what I can try.
You don't need an MPQ for this, you can load your custom files from folder. Using MPQ's can be more convenient in some cases, but is not required.

Quote from IlyaSnopchenko
Any bugs / issues that I may find, I'll report in the new PYMS thread.
Thanks




Jan 23 2019, 6:07 pm Pr0nogo Post #33



Updated to include mtl and chainwmode. Merged them with IsDebug to create the utilities section.




Aug 28 2019, 5:34 am Pr0nogo Post #34



I've recently encountered a bug associated with the latest version of Firegraft (0.94) that causes immense pathfinding glitches. When compared with identical mods created with 0.93, the bug does not occur. To this end I've changed the version of Firegraft linked in this thread. I can't guarantee that all the glitches have been resolved, but testing has so far yielded a much smoother playing experience. If GAG Mania or someone else can investigate the issues with 0.94 and fix them, I'll post their work.




Dec 18 2019, 10:08 am UndeadStar Post #35



Hi,

I'm a bit frustrated over the fact that exe made by Firegraft prevent from starting Starcraft non-Broodwar campaigns (even without any change, plugin or anything).

Since that's what I need most: what alternatives are there to create an exe containing a patch made from GPTP?

edit:
Tried MPQDraft, fought a lot to get a .exe, but same result, the .exe, even without plugin file, won't load starcraft non-broodwar campaigns.
On the other end, loading the plugin through MPQDraft without creating a self-executable work (saw the GPTP text message on 1st Terran mission of 1st campaign)

Is the very method universally used to make self-executable mods broken? :(

Post has been edited 1 time(s), last time on Dec 18 2019, 9:02 pm by UndeadStar.



None.

Dec 19 2019, 5:20 pm Neiv Post #36



You could try using a plugin to fix the issue.

The two points that are broken are at 004DB5D5 and 004DA5AF, where BW tries to check if a specific mpq (Broodwar.mpq / Starcraft.mpq / Install.exe) contains rez\glucmpgn.bin when loading vanilla SC or rez\gluexpcmpgn.bin with BW.

At least Firegraft always just returns the .bin file from any loaded mpq instead of the specific mpq that BW asked, I don't know about how MPQDraft handles this.

So you'd have to make these two specific calls to bypass file reading hooks that exist on SFileOpenFileEx and call the unhooked storm function directly. Or maybe not call SFileOpenFileEx at all and use some completely different logic there. I'm not sure what's the simplest way to fix this, but maybe you can figure out something.



None.

Dec 23 2019, 3:44 pm UndeadStar Post #37



Quote from Neiv
You could try using a plugin to fix the issue.

The two points that are broken are at 004DB5D5 and 004DA5AF, where BW tries to check if a specific mpq (Broodwar.mpq / Starcraft.mpq / Install.exe) contains rez\glucmpgn.bin when loading vanilla SC or rez\gluexpcmpgn.bin with BW.

At least Firegraft always just returns the .bin file from any loaded mpq instead of the specific mpq that BW asked, I don't know about how MPQDraft handles this.

So you'd have to make these two specific calls to bypass file reading hooks that exist on SFileOpenFileEx and call the unhooked storm function directly. Or maybe not call SFileOpenFileEx at all and use some completely different logic there. I'm not sure what's the simplest way to fix this, but maybe you can figure out something.
Thanks for the clues.
I used WinMerge to compare 3 versions of SC memory (without mod, MPQDraft direct load that allow campaign + gptp base version, autoexec of gptp not allowing campaign), but as I don't know how to optimize/automatize the concept "difference is a problem is first 2 are identical and 3rd is different), your information helped me a lot.
Now my issue is that while patching something with CheatEngine seemed to work, patching from GPTP doesn't, probably attempting to patch too early in Starcraft loading.
Still working on it.

Just in case, what I did with Cheat Engine was to copy 4 values at 4 addresses, those:
Code
address: value to put there

0x004FE440: D0820115
0x004FE538: D07D0115
0x004FE570: 60790115
0x004FE578: 804A0115

Maybe it's too much, maybe it's not enough, and maybe it would cause trouble in cases I haven't explored, but if somebody else want to play with fire :D

edit:

Ok, found a way to fix things through GPTP code, didn't see the Starcraft campaign crash after 1st mission, nor did the Brood War one crash, nor did BW content fail to load in BW campaign (most likely issues I expected :P )
This is what I added in initialize.cpp:
Code
    jmpPatch((void*)0x150182D0, 0x004101AE);
    jmpPatch((void*)0x15017DD0, 0x004100B2);
    jmpPatch((void*)0x15017960, 0x004100C4);
    jmpPatch((void*)0x15014A80, 0x004100BE);

Thanks again Neiv :cool2:

Post has been edited 1 time(s), last time on Dec 24 2019, 12:49 pm by UndeadStar.



None.

Jan 10 2020, 9:18 pm UndeadStar Post #38



I had changed the version of Firegraft I use before because it got detected as malware, and it seemed serious.
But recently, Windows Defender suddenly tried to quarantine the version I use, with a detection seeming less
serious.
I tried to get it whitelisted officially at https://www.microsoft.com/en-us/wdsi/submission/897a1b3d-2ba8-4e00-ae11-18055bc1c2d2
but based on the "Analyst comment" that got added today, I probably will have to keep the whitelisting on my side :bored:

edit: to be accurate, what make me not optimistic is the Cloud detection detecting the same thing as Client detection (what I
reported), even if "Final determination: Not malware" is somewhat a good thing (not sure it would affect Windows Defender though)



None.

Jan 11 2020, 3:35 pm IlyaSnopchenko Post #39

The Curious

The antivirus software (like the Norton suite I'm using) keeps attacking the modding utilities, and I have to set up exclusions for each new version of SAMASE, for example. So I guess this is just something we'll have to live with.



Who am I, and why should I care?

Options
  Back to forum
Please log in to reply to this topic or to report it.
Members in this topic: None.
[02:00 pm]
NudeRaider -- You are undead, why wouldn't you and your creations (GPTP) exist as (in) graves? I don't get why this needs fixing.
[01:58 pm]
UndeadStar -- I don't know in how many people imagination me and/or my GPTP existed as graves, but this will finally be fixed now that I've made the online code available
[10:26 am]
NudeRaider -- I can't say but it sounds plausible. You'd still have the problem with the palette.
[10:25 am]
Oh_Man -- it's not like the pixels themselves are inherit to one tileset or another.
[10:24 am]
Oh_Man -- Couldn't you redefine all the tiles from the other tilesets to be read as part of just one tileset instead of multiple perhaps
[10:16 am]
NudeRaider -- *make new tiles
[10:15 am]
NudeRaider -- The map structure only ever recognizes one tileset at a time. So you'd have to define your own map format then make an .exe that can load such files. I don't think it's ever been done; it sounds like a lot of work. What's commonly done with modding is creating a new tileset. Then you decide which tiles go in there. You can also make new ones. You will probably also have to create a new palette as the different tilesets share only part of their colors. Finally entering speculation territory you could maybe use EUDs to dynamically change the graphics of certain tiles. Not sure if it's possible, but if it is, you better make a topic for it.
[04:11 am]
O)FaRTy1billion[MM] -- no, that's not a thing you can do without modding the game
[03:56 am]
youarenotworthy -- I forget, did starforge ever allow you to mix tilesets?
[03:44 am]
O)FaRTy1billion[MM] -- use scmd or NudeRaider
NudeRaider shouted: You are right, however, in that the Remastered *graphics* are just a skin are interchangeable, even midgame. Skinability being another new functionality. Anyways, for tools that rely on the original mpqs you will have to obtain a legacy version of sc.
Please log in to shout.


Members Online: Roy, memory218638