Staredit Network > Forums > Modding Discussion > Topic: .dat file unknowns
.dat file unknowns
May 11 2016, 7:44 pm
By: Clokr_  

May 11 2016, 7:44 pm Clokr_ Post #1



I'm rewriting some .dat loading code and I noticed that even from the latest DatEdit there are still a lot of unknown fields. I thought we could try to see what some of them might do. I got the following ones:

Flingy.dat:
  • Move control: partially mobile / weapon. I did some tests and I'm pretty sure this is flingy.dat control but without deccelerating. Makes sense bullets would use that. I assume the halt distance field was automatically calculated by some app and that's why it is not zero for these units.
  • Halt distance: this was automatically computed for every unit that moves as (top speed^2)/(2*acceleration). Only used if unit goes at top speed, as otherwise SC does the computation with the current speed.
  • Turn radius: this is really a turn speed, probably measured in deg/frame, where each degree is 1/256th of the full circumference. It is used even if control is set to iscript.bin.

Sprites.dat:
  • Unknown: this is unused by both SC and StarEdit.

Techdata.dat:
  • Race: This is not used by SC. Used by StarEdit to classify them in folders. Also "ALL" means "UNUSED", so that StarEdit will hide it.
  • Unused: This is actually a 'researched by default' flag used by StarEdit to determine whether it should allow the user edit the mineral, gas and time costs of the tech or not.

Upgrades.dat:
  • Race: same as techdata.dat.

I will keep editing this post with any new info I come up with or anyone else does.

Post has been edited 4 time(s), last time on May 30 2016, 11:51 pm by Clokr_.



?????

May 12 2016, 12:52 am O)FaRTy1billion[MM] Post #2

👻 👾 👽 💪

Quote from Clokr_
Race: Is this used by SC at all?
It is not.



TinyMap2 - Latest in map compression! ( 7/09/14 - New build! )
EUD Action Enabler - Lightweight EUD/EPD support! (ChaosLauncher/MPQDraft support!)
EUDDB - topic - Help out by adding your EUDs! Or Submit reference files in the References tab!
MapSketch - New image->map generator!
EUDTrig - topic - Quickly and easily convert offsets to EUDs! (extended players supported)
SC2 Map Texture Mask Importer/Exporter - Edit texture placement in an image editor!
\:farty\: This page has been viewed [img]http://farty1billion.dyndns.org/Clicky.php?img.gif[/img] times!

May 12 2016, 4:21 am Neiv Post #3



Oh, the partially mobile / weapon behaviour is nice to know :O

The halt distance specified in flingy.dat is used only when unit is moving at max speed, otherwise bw calculates it as "(speed^2) / (acceleration * 2)".



None.

May 12 2016, 9:36 am Clokr_ Post #4



Quote from Neiv
The halt distance specified in flingy.dat is used only when unit is moving at max speed, otherwise bw calculates it as "(speed^2) / (acceleration * 2)".

Mmm, what sense does that make? I thought the halt distance was stored to avoid time-consuming calculations at run-time. Maybe they assumed that units halting while not going at full speed were rare events? All the flingy.dat halt distances seem to be calculated that way though, which means that indeed they were automatically computed.

Btw, if anyone wants to know where that formula comes from:
Assuming a unit is going at speed v and has flingy acceleration a and that SC subtracts a from v each frame and adds v to the current position each frame, the total distance it needs to completely halt is
v + (v-a) + (v-2a) + ... + (v-na)
where n = v/a. This sum roughly evaluates to vn - an^2/2 = v^2/(2*a). (Actually if one assumes time is not discrete, then you get an integral which exactly evaluates to v^2/(2*a)).

Does anyone have any idea about what the sprites.dat unknown can be for? It is set only for men, men death sprites, building sprites and building rubble sprites...
EDIT: I tried inverting its value for every entry in sprites.dat and found no in-game difference. Either it is unused, some debug flag, used by Staredit (unlikely) or refers to something related to the inner machinery of SC.

Post has been edited 3 time(s), last time on May 12 2016, 11:23 pm by Clokr_.



?????

May 13 2016, 12:56 am O)FaRTy1billion[MM] Post #5

👻 👾 👽 💪

Unknown in sprites.dat is not used by SC.



TinyMap2 - Latest in map compression! ( 7/09/14 - New build! )
EUD Action Enabler - Lightweight EUD/EPD support! (ChaosLauncher/MPQDraft support!)
EUDDB - topic - Help out by adding your EUDs! Or Submit reference files in the References tab!
MapSketch - New image->map generator!
EUDTrig - topic - Quickly and easily convert offsets to EUDs! (extended players supported)
SC2 Map Texture Mask Importer/Exporter - Edit texture placement in an image editor!
\:farty\: This page has been viewed [img]http://farty1billion.dyndns.org/Clicky.php?img.gif[/img] times!

May 18 2016, 5:40 am Heinermann Post #6

SDE, BWAPI owner, hacker.

I posted a lot of the unknowns a while back. I will try to dig it up.

EDIT: The last couple years of broodwarai.com were wiped. I've written what I know from the top of my head below.
EDIT2: I found a modified version of PyMS in my public dropbox folder which has some updated fields. Adding below.

The entries that are all 0xFFFF are the requirement IDs for their respective file.

units.dat (Advanced -> Unknown): unit build requirements ID
upgrades.dat (Unknown): upgrade requirements ID
techdata.dat (Unknown): tech research requirements ID, and tech use requirements ID

These are remapped after they are loaded to the internal use requirements (typically changed by firegraft or whatever the kids use these days). But I think they are only remapped if they are set to 0xFFFF (but I could be wrong).

Other stuff
units.dat
AI Actions -> This is a bitmask. 0x01 = Don't use on Strategic Suicide Missions, 0x02 = Don't use as a guard

Advanced -> Advanced Properties: https://github.com/bwapi/bwapi/blob/develop/bwapi/BWAPI/Source/BW/UnitPrototypeFlags.h

I'm pretty sure I figured out the other unknowns here, however I don't remember what they were off the top of my head.

images.dat
Drawing properties -> Function: https://github.com/bwapi/BroodWarLib/blob/master/BWLib/RLEType.h

Sounds (?)
I figured a lot of these out before, but again they were lost AFAIK.

Unknown1 has a value between 0 and 3 inclusive.
Unknown2 are flags
Unknown3 is the race for the building placement sfx.
Unknown4 is a volume percentage (0-100), possibly the minimum volume that the mute trigger applies to it?

techdata.dat
The unused checkbox is a "researched" flag. (even though some things are unused does not mean we can't know what they would be used for otherwise)

Post has been edited 7 time(s), last time on May 18 2016, 7:22 am by Heinermann.




May 19 2016, 6:40 pm Clokr_ Post #7



Some of the ones you posted are already in the first topic. DatEdit also has some of them updated. Still thanks for the reply! I will sure check the sfxdata unknowns at some point :) By the way, I'm pretty sure the requirements ID is always overwritten, as I have tried randomly changing it and I saw no side effects in-game.

I have done some testing on the portdata.dat fields, which DatEdit says they are a probability controlling how often the portrait track changes and some unknown, but I have seen absolutely no difference in-game. Has anyone tested these before? Are they even used?



?????

May 20 2016, 12:53 am O)FaRTy1billion[MM] Post #8

👻 👾 👽 💪

Quote from Clokr_
I have done some testing on the portdata.dat fields, which DatEdit says they are a probability controlling how often the portrait track changes and some unknown, but I have seen absolutely no difference in-game. Has anyone tested these before? Are they even used?
The memory is referenced, but I didn't look at what it did.



TinyMap2 - Latest in map compression! ( 7/09/14 - New build! )
EUD Action Enabler - Lightweight EUD/EPD support! (ChaosLauncher/MPQDraft support!)
EUDDB - topic - Help out by adding your EUDs! Or Submit reference files in the References tab!
MapSketch - New image->map generator!
EUDTrig - topic - Quickly and easily convert offsets to EUDs! (extended players supported)
SC2 Map Texture Mask Importer/Exporter - Edit texture placement in an image editor!
\:farty\: This page has been viewed [img]http://farty1billion.dyndns.org/Clicky.php?img.gif[/img] times!

May 20 2016, 3:40 am Heinermann Post #9

SDE, BWAPI owner, hacker.

Quote from Clokr_
Some of the ones you posted are already in the first topic. DatEdit also has some of them updated. Still thanks for the reply! I will sure check the sfxdata unknowns at some point :) By the way, I'm pretty sure the requirements ID is always overwritten, as I have tried randomly changing it and I saw no side effects in-game.

First topic? What first topic?

The latest version of DatEdit that I obtained was from the downloads section on this site just yesterday, and all of those were labelled unknown/unused. Is there a newer version I don't know about?




May 20 2016, 8:56 am Clokr_ Post #10



Quote from Heinermann

First topic? What first topic?

The latest version of DatEdit that I obtained was from the downloads section on this site just yesterday, and all of those were labelled unknown/unused. Is there a newer version I don't know about?

First post*. I think you can download the latest version from stormcoast fortress. Some of the labelled as unknowns do have the right descriptions qhen clicked upon, as is the case with the requeriment id fields.



?????

May 23 2016, 4:24 pm Neiv Post #11



Quote from Heinermann
Sounds (?)
I figured a lot of these out before, but again they were lost AFAIK.

Unknown1 has a value between 0 and 3 inclusive.
Unknown2 are flags
Unknown3 is the race for the building placement sfx.
Unknown4 is a volume percentage (0-100), possibly the minimum volume that the mute trigger applies to it?

Unknown 4 is the minimum volume the sound plays at. It causes explosion sounds to be heard regardless of how far screen is from the explosion.

Unknown 3 can be used to modify the duration of talking portrait animation (Milliseconds, can be negative) when the related sound is played as an unit response. I have no explanation why the building placement sounds have values that match the sounds race...

Edit: Unknown 1 is the priority. Sounds with higher priority are allowed to override lower priority ones that are already playing if bw doesn't have enough sound channels available.

Post has been edited 1 time(s), last time on May 30 2016, 9:35 pm by Neiv.



None.

Jul 26 2018, 4:08 pm Neiv Post #12



The new SCR patch moved from sfxdata.dat to a JSON file defining the sound settings, seemingly to make it easier for Blizz to add new ones when they need any.

But the interesting part is that JSON has names for sfxdata unknowns. While they were mostly described above, Unk2 flags weren't.

Unknown 1: priority
Unknown 2: Various flags:
- 1 preload
- 2 unitSpeech
- 16 oneAtTime
- 32 neverPreempt
Unknown 3: lengthAdjust
Unknown 4: minVolume

Afaik the other flags than those (4 and 8 at least) get set/reset runtime by the sound caching code, and setting them in .dat will just cause problems.



None.

Options
  Back to forum
Please log in to reply to this topic or to report it.
Members in this topic: None.
[01:06 am]
SiberianTiger -- anyone wanna play footmen frenzy with me on us west?
[12:50 am]
O)FaRTy1billion[MM] -- NudeRaider
NudeRaider shouted: Lanthanide ya, I was pretty sure this must be the case, but I was referring to FaRTy1billion
i was mostly joking. I haven't seen any specs for it or anything other than vague comments
[10:02 pm]
Suicidal Insanity -- jjf28
jjf28 shouted: if you do parse those correctly I'll probably turn on sub-string recycling by default since it makes smaller map sizes, else I'll prompt the user before turning it on
I won't parse those correctly - I do not remember why, maybe it was the embedded nulls in stat_txt.tbl, but I clip strings at the next offset in the table header and don't search for a nul byte
[04:26 pm]
Dem0n -- ya he's in there
[03:52 pm]
NudeRaider -- Lanthanide
Lanthanide shouted: i don't know, but Armo does, so he can be asked
whcih discord? SEN?
[03:51 pm]
NudeRaider -- Lanthanide
Lanthanide shouted: and also there are tools right now that let you use it, so to say "no one can use it" is wrong
ya, I was pretty sure this must be the case, but I was referring to FaRTy1billion
FaRTy1billion shouted: nobody knows
[02:02 pm]
jjf28 -- if you do parse those correctly I'll probably turn on sub-string recycling by default since it makes smaller map sizes, else I'll prompt the user before turning it on
[02:01 pm]
jjf28 -- e.g. "asdf<NUL>" and "df<NUL" occupy the same space in character data but have two different offsets
[02:00 pm]
jjf28 -- hey SI, are you parsing STR sections with sub-string recycling?
[12:52 pm]
Moose -- spicy memes
Please log in to shout.


Members Online: Roy, oxysuner, addisonevans