Staredit Network > Forums > SC1 Mapping Tools > Topic: Chkdraft Project
Chkdraft Project
Oct 9 2012, 12:42 am
By: jjf28
Pages: < 1 12 13 14
 

Oct 30 2018, 11:45 am jjf28 Post #261

Cartography Artisan

Quote from Pr0nogo
I think an open source mapping tool is paramount to high end development and at this point in BW's age as a content platform it's pretty depressing we don't already have one. SCMDraft's problem is that it's not open source and thus resistant to new features, and Chkdraft's problem is that it isn't complete. Would you be willing to collaborate with other developers to remedy the latter?

Other developers are always welcome to take a fork (or request access to make feature branches) and raise a pull request to development; I tend to process these in under a week, so far others have only committed some small things https://github.com/jjf28/Chkdraft/graphs/contributors , except Fary who decompiled a ton of graphics https://github.com/jjf28/Chkdraft/tree/graphics-beta (though I didn't think it stable, so I haven't merged it yet).

On the knowledge side of things, SI, Fary, and Heinermann have helped a ton when I've needed it (though SI still owes me some ISOM code :P ).

I should note that it would complicate things if someone were to work a ton of features while I'm completely changing the base code around https://github.com/jjf28/Chkdraft/tree/feature/fundamental-refactoring , but if anyone's particularly eager to contribute I'll make it a priority to have that stable by this weekend.



Rs_yes-im4real - Clan Aura - jjf28.net84.net

Reached the top of StarCraft theory crafting 2:12 AM CST, August 2nd, 2014.

Oct 31 2018, 11:51 pm Pr0nogo Post #262



I'm just weighing the options right now but I'm glad to know you're open to it. If you can post when you're done with that fundamental refactoring, I'd be much obliged.




Nov 19 2018, 4:36 am jjf28 Post #263

Cartography Artisan

Mostly a note to self: not going to support preserving jump section information between saves (this violates my change-nothing-not-touched policy, and every exception needs some justification) - jump sections are a section with a negative size such that they jump backwards in the chk file to a section header embedded within some other section. Jump sections can be used both for compression and for protection - protection in that they're harder to write a parser for, so programs are more likely to fail against them; and if you build large enough compressed section data you could crash most editors/run them out of memory.

If a chk file contained jump sections that increased the scenario file size - it should be considered protected.

If a chk file contained jump sections that decreased the scenario file size - it should be considered compressed, if changed, original jump sections should be discarded, and upon saving (or upon releasing, if there's a substantial runtime impact to doing compression operations on every save) jump section compression should be reattempted. Compression is much less important in remastered since blizzard has greatly improved download times, so losing this info entirely may be acceptable.

Jump sections present numerous challenges trying to preserve - having to setup alarms on every data change potentially affecting the header, the data inbetween, or the size between (including changing one sections size, or adding or deleting sections), and because we're talking raw data monitoring it would serve as a great complication to any object-based abstraction - requiring something like holding the raw data form and object form and keeping them in sync at all times, or rapidly generating the raw data form (and risking out-of-memory errors for otherwise low-impact operations), notifying the exact changes to the jump section for handling, and in the handler recalculating the position of the jump sections (which may not always meet user expectations, as they might not want a jump section header in a action in something like a trigger they just copied).

Because there's no actual point to jump sections except compression, the compression can be recalculated at save time, and preserving the information would mean tons of work, overhead/resources, and imperfect user-experiences, maps with jump section compression will lose it (have the sections recompiled into one chunk) and potentially have it recalculated during save.



Rs_yes-im4real - Clan Aura - jjf28.net84.net

Reached the top of StarCraft theory crafting 2:12 AM CST, August 2nd, 2014.

Jan 26 2019, 8:47 pm Wormer Post #264



Quote from jjf28
Mostly a note to self
There might not be much attention around, but hey! :w00t: I've just learned something new and beautiful at technical part of things! Thank you for sharing that bit! :cool2: Any updates coming soon?



Some.

Mar 12 2019, 1:24 am sethmachine Post #265



Not sure if this is done, but if you could abstract the CHK parsing code from the GUI into its own library, e.g. think a CHK to JSON and JSON to CHK, that would be infinitely helpful. Ideally written in Python...




Mar 12 2019, 1:35 am jjf28 Post #266

Cartography Artisan

CHK to Json is one of the things I want to do, but that's not in my short term plans (think 6 months from now). A library in python is not something I'd be personally interested in unless python became super relevant career wise - you'd have to use a LIB or DLL or other inter-language feature of python to interface with libraries from Chkdraft, or write your own port.



Rs_yes-im4real - Clan Aura - jjf28.net84.net

Reached the top of StarCraft theory crafting 2:12 AM CST, August 2nd, 2014.

Mar 12 2019, 8:56 am Suicidal Insanity Post #267

I see you !

You can also write a C++ pymodule that compiles natively for python. But speaking from experience that is a pain




Mar 12 2019, 7:41 pm sethmachine Post #268



Quote from jjf28
CHK to Json is one of the things I want to do, but that's not in my short term plans (think 6 months from now). A library in python is not something I'd be personally interested in unless python became super relevant career wise - you'd have to use a LIB or DLL or other inter-language feature of python to interface with libraries from Chkdraft, or write your own port.

It's not clear how to use any of the Chkdraft code to do what I want--interpret triggers from TrigEdit or similar style to the TRG binary format and back. If you had an example that would help. For example even StormLib had examples in C++ and I could reuse this in Python by using the DLL as you mentioned, which is how I wrapped it into my Python MPQ library.

I'm curious why Python is not relevant to your career? In fact I'd think the opposite about C++/C. Everyone uses Python or Java in my circles and even Java is considered cumbersome, let alone C++! It's the defacto scripting language and has one of the lowest barrier to entry, along with an easy way to learn AI/machine learning/data science. I have been coding for almost 10 years and work full time as a SE but I wouldn't touch C++ with a 100 foot poll. I guess you work on hardcore game development, embedded systems, or optimization?

I'm not sure why the community is resistant to this idea. I mean if I did any real modding project, the first thing I'd do is write a library or module that does one thing good and one thing only which would be to read CHK to and from a human readable and friendly format like JSON. I'd build all my projects on top of that dependency. And then anyone else in the community can use it, so we don't end up fragmented and write 12 different parsers for CHK format.

Parsing the format seems tedious, but it's not innovative nor where any new projects should spend their time. Progress is hard to make when we can't re-use existing work.

This includes you too, Suicidal Insanity. Could you release whatever code interprets TrigEdit into the TRG format at least? I am about to write my own interpreter and would rather avoid repeating that work.



None.

Mar 12 2019, 8:33 pm Pr0nogo Post #269



Unfortunately most tool devs have a strong aversion to releasing their source. pq and jj are anomolies compared to the insane number of devs who have chewed me out for even asking. It seems foolish but most of them don't want their code looked at, in my experience.

C++ is also used for bw plugins fyi, you will need to get a 101 foot pole or write an alternative library if you want to get involved with those. There is a lesser known library in rust but it's not very fleshed out yet iirc.




Mar 12 2019, 9:40 pm jjf28 Post #270

Cartography Artisan

Quote
It's not clear how to use any of the Chkdraft code to do what I want--interpret triggers from TrigEdit or similar style to the TRG binary format and back. If you had an example that would help. For example even StormLib had examples in C++ and I could reuse this in Python by using the DLL as you mentioned, which is how I wrapped it into my Python MPQ library.

As a project yet to complete the basic features I don't regularly package libraries, but I'll get a DLL for you here and some example code in the near future.

Quote
I'm curious why Python is not relevant to your career? In fact I'd think the opposite about C++/C. Everyone uses Python or Java in my circles and even Java is considered cumbersome, let alone C++! It's the defacto scripting language and has one of the lowest barrier to entry, along with an easy way to learn AI/machine learning/data science. I have been coding for almost 10 years and work full time as a SE but I wouldn't touch C++ with a 100 foot poll. I guess you work on hardcore game development, embedded systems, or optimization?

I write backend stuff, maybe that'll be python someday but at present I don't need it, it hasn't shown up big on job requirements nor in my job training (it has shown up a little, which is why I know a little python, but not enough to justify spending weeks or months on it).

Quote
I'm not sure why the community is resistant to this idea. I mean if I did any real modding project, the first thing I'd do is write a library or module that does one thing good and one thing only which would be to read CHK to and from a human readable and friendly format like JSON. I'd build all my projects on top of that dependency. And then anyone else in the community can use it, so we don't end up fragmented and write 12 different parsers for CHK format.

Not at all resistant to the idea, there's a reason Chkdraft is open source and a reason I carefully isolated all of the mapping code (and am even redoubling isolation efforts in my current refactor) - cause that part of my project is intended as reusable CHK handling code, with serializations to JSON and binary and all that jazz. That said Chkdraft isn't intended to do that one thing and that one thing only, there are several goals, and I chase down whatever I most need or most fancy at the time ;)

Post has been edited 1 time(s), last time on Mar 12 2019, 9:47 pm by jjf28.



Rs_yes-im4real - Clan Aura - jjf28.net84.net

Reached the top of StarCraft theory crafting 2:12 AM CST, August 2nd, 2014.

Mar 12 2019, 9:53 pm Suicidal Insanity Post #271

I see you !

Quote from sethmachine
This includes you too, Suicidal Insanity. Could you release whatever code interprets TrigEdit into the TRG format at least? I am about to write my own interpreter and would rather avoid repeating that work.

I just maintain the text trigger code, I'm not the original author of that plugin.




Mar 14 2019, 12:14 am sethmachine Post #272



Quote
Not at all resistant to the idea, there's a reason Chkdraft is open source and a reason I carefully isolated all of the mapping code (and am even redoubling isolation efforts in my current refactor) - cause that part of my project is intended as reusable CHK handling code, with serializations to JSON and binary and all that jazz. That said Chkdraft isn't intended to do that one thing and that one thing only, there are several goals, and I chase down whatever I most need or most fancy at the time

Understood and now it makes sense from your point of view. If I did work with C++ I could probably reuse your work (but I may still reference it) but I need everything in Python.

I went ahead and started my own implementation of the CHK format into JSON/Python objects called chkjson. It is rudimentary right now, but if you look at the organization and type annotations, you can get an idea for the standards I expect from a library that would parse CHK. I only do STR section now (and it's far from complete) but also intend to add tests for the compile/decompile and every other function, like adding a new string to the STR section. I don't plan on adding every section (but community members could make pull requests once I finish the code for the sections I care about, like TRG and MRGN.

See: https://github.com/sethmachine/chkjson




Options
Pages: < 1 12 13 14
  Back to forum
Please log in to reply to this topic or to report it.
Members in this topic: None.
[09:57 am]
Zycorax -- :wob:
[01:24 am]
razorback9423 -- u
[12:29 am]
lil-Inferno -- :wob:
[11:31 pm]
MTiger156 -- :wob:
[11:17 pm]
Wing Zero -- :wob:
[03:17 pm]
Pr0nogo -- u
[02:35 pm]
lil-Inferno -- u
[02:02 pm]
Corbo -- No
[01:28 pm]
razorback9423 -- Um, is this shoutbox active?
[2019-9-14. : 11:18 am]
razorback9423 -- C.I.S. Conversion Mod will have a campaign, and the first part is coming this September.
Please log in to shout.


Members Online: UEDCommander, Roy, verctirappthink45, ecixi, efykosydy