I want to redraw black fog of war in maps, so I asked this question in reddit

https://www.reddit.com/r/broodwar/comments/cfv8wh/redraw_black_fog_of_war_in_scenario/

and a really helpful user tried to help me, but it seems I'm not getting the grasp of memory adressess and dereferencing
Through that, I discovered EUD. Now, I've tried applying this

http://www.staredit.net/topic/15793/

but I already crashed Starcraft Remastered too many times with Cheat Engine, so it seems I'm not looking in the right direction. Now, what I'm actually trying to do is implement the green-over-black campaign outros in UMS, so redrawing fog of war would be as useful as changing every tile to a black tile. So, could any of you guys point me in the right direction?

There's no need to use an elaborate solution for this. Why not fill a corner of the map with null tiles and "center view" there for all players? If there isn't enough space for this already, just expand the map size.

Edit: Probably better to expand anyway (vertical or horizontal), make said expansion null terrain, and use triggers to make it a "no fly zone".

The trickest thing there is to dereference a pointer @0x6D1260. Check out the current player trick.

I'll try to explain the problem. You can access any static address @0xABCDEF with action (or condition) of the like MemoryAddr(0xABCDEF, SetTo/Exactly, Value). This applies only if you know @0xABCDEF. What if the actual address you need to access is written as an integer number somewhere, for instance @0x6D1260 ? That variable @0x6D1260 is called a pointer.

[Note]
StarCraft uses 32-bit address space that means all pointers are represented as 32-bit or 4-byte unsigned integer variables (uint, unsigned, or just u32 for brevity), which are subject to 32-bit arithmetic. This also means that hex number 0xABCDEF is an abbreviation for 0x00ABCDEF with omitted leading zeros.

To understand an answer you need to understand the internal workings of MemoryAddr which is basically the core of what EUD/EPD are all about. The following is an explanation for newbies that you may skip all the way down to the current player trick if you already know this.

[TL;DR]
MemoryAddr is not a basic action/condition at all, it is getting translated either into action SetDeaths(Player, Unit, Modifier, Value) or condition Deaths(Player, Unit, Comparison, Value). Let's focus on action for a moment, which would be write access, because read access is all the same. SetDeaths is originally intended to modify a unit and a player specific u32 variable which holds deaths of the particular unit. These variables are stored as an array (table) which starts in memory @0x58A364. The array is organized as follows:

There are 228 concrete units in SC, which are listed in the wiki along with their IDs. In short we write this memory layout as a two-dimensional array of u32 integers:

In 'SetDeaths(P, U, M, V)' P is the number between 0 and 11, which is the affected player ID; U is the number between 0 and 227, which is the desired unit ID; M is one of the numbers 7, 8, or 9 which are just codes for operations SetTo, Add, and Subtract respectively; V is an u32 integer. Machine instruction level workings are as follows: apply the action with op-code M to the variable stored @(0x58A364 + (U*12 + P) * 4) [formula 1], which accordingly to the table above is exactly the address of the variable that holds deaths of player's P unit U. The multiplication by 4 is there because we treat those variables in memory as 4-byte integers and must specify the first byte of the variable for the operation. Keep in mind that on machine level every byte can be addressible in principle (or without too much details otherwise accessible), but the instruction that is used in SetDeaths treats argument as 4-byte integer and we have to live with it.

How all this makes possible to access arbitrary addresses? The fact is SC doesn't do validation check on arguments P and U of SetDeaths. When those supposed to be player ID and unit ID numbers, it turns out P and U are treated just like any other u32 and u16 integer numbers respectively (where u16 stands for 16-bit or 2-byte unsigned integer). Given formula [1] we construct such values for P and U that make SetDeaths modify address of our like. The inherent limitation is we can only address, as programmers say, 4-bytes memory aligned variables. When we make use of extra values for U only along with the normal range (0-11) for values for P the technique is called Extended Unit Deaths (EUD), when we use extra values for P as well this is called Extended Player Deaths (EPD). In essence these two are the same, but it was first discovered for units overflow, when players overflow is just easier to use due to the fact that P is u32 integer. Please note, that we can get back in address space as well because in arithmetic modulo Q (which 32-bit arithmetic is a kind of, 'Q = 2^32') 'A minus B' can be expressed as 'A plus (Q-B)', where (Q-B) is positive.

[Current player trick explained]
The last bit to the puzzle comes with the current player trick. In fact, I have omitted a special case where P can be 13. That code is used to represent Current Player in trigger action or condition. The value of Current Player is stored in an u32 variable @0x6509B0 that is getting substituted in place of P in formula [1]. Fortunately, address @0x6509B0 is open for write access. We utilize that to our advantage to indirectly access pointer @0x6D1260: do a non-destructive copy of the value from @0x6D1260 to @0x6509B0 using binary countofs. According to formula [1] that value must be divided by 4 in the process. After that execute SetDeaths(Current Player, ... ), which will utilize our crafted value of the current player variable in formula [1] to access what we want.

This is the description of technique you use to access memory directly (EUD/EPD) or through pointers (CP-trick). I hope you get the basic understanding of what is happening under-the-hood. If this is understood the rest you can easily do by yourself.

[P.S.]
This said, MTiger156 has the point. Why use cannon for a bird?

[P.P.S.]
I am probably not right about EPD being as simple as using arbitrary P for overflow. For what exactly happens see this presentation page 28 I found on google. I have no idea how long the link might be valid. I am particularly suspicious about the check 'if(dwPlayer >= NET_MAX_NODES) return 0;'. The good thing what I am saying is enough to understand things, and all the rest is abstracted in condition/action MemoryAddr(...).

Post has been edited 11 time(s), last time on Aug 7 2019, 3:59 pm by Wormer.

So, let's see if I understood.
I need to select U and P so (0x58A364 + (U*12 + P) * 4)=0x006D1260
P is current player, so with that, I can get U

Now, the value of 0x006D1260 is an array X of size Width*Height. In this case, X[0][0] is the tile (0,0), X[0][1] is the tile (0,1) and so on. So, I access 0x006D1260, I copy it's value/4 to 0x6509B0 and done?



Now, for the doubts
1. Why can't I say value at 0x006D1260=some_value?
2. If X is an array, how should I access it's values? In C, it's *(pointer+offset), It's the same here?
3. Suppose I get those values, now, the flags, how those should be accessed?
4. Now that Starcraft will be 64 bits, would pointers be 64-bit variables?


My brain is getting slow, so forgive me if those questions are too obvious.


And regarding what MTiger156 said, I tried doing that, but I want the maps to be a little more professional.

DISCLAIMER:
I'm a theoretician, don't blame me if anything doesn't work! I encourage you to go to the editor and cook up a test map that will put my words to a test.

To the questions.
Because this wouldn't change the location of tiles array itself, but only the pointer value. In worst case this will corrupt the game, because the pointer will point to memory that isn't the tiles array anymore and the game still assumes to find a tiles array at that place. In best case scenario the game will restore the pointer after trigger cycle finishes like nothing happened (it is probably what 'baked by code' means).

Yes, it's the same. The integer value written @0x6D1260 is a pointer to the start of the tiles array, which is tiles[0][0]. If you want to access tiles[x][y] then you need to add the offset which probably is (x + y*MAP_WIDTH). Every tile is already 4-byte integer hence you don't need to divide offset by 4 when adding it to CP.

That needs further investigation. What are your ideas on this one?

Internally, yes. But for us everything is emulated like it's good old 32-bit address space. In fact, SC worked like explained in 1.16.1, but since Remastered everything isn't like it was anymore -- it is being emulated for us to believe that SC works like described. You don't want to know how SC really works now and I can't explain that. See the presentation linked above for all details. This is some kick-ass stuff worth of good year of work for exceptional specialist in the field.

Bottom line for us: it's safe to assume that we're working on 32-bit address space even if the client is 64-bit now.

For a set of actions that will copy u32 value from @0x6D1260 to @0x6509B0 in essense you first select such U1 and P1 that (0x58A364 + (U1*12 + P1) * 4) = 0x6D1260, then select U2 and P2 that (0x58A364 + (U2*12 + P2) * 4) = 0x6509B0. One can note that we can use U1 and U2 equal to 0 and just supply sufficient P1 and P2. Then you don't even need to do that nowadays, since SmdDraft2 supports action and condition MemoryAddr(A, ... ) where you just supply the required address as the first argument (it selects U and/or P under-the-hood for you).

I should ask now: do you know what is a binary countoff, and how to utilize it to make a destructive and undestructive copy of a death counter. Let's say: how to copy deaths of Terran Marine into deaths of Zerg Zergling for current player?

Post has been edited 6 time(s), last time on Aug 7 2019, 11:22 pm by Wormer.

I was starting to read about binary countoffs. Basically, raise a number from zero, until another one reaches zero.
http://www.staredit.net/wiki/index.php?title=Binary_Countoffs

So, in SCMDraft, I should do, for n=0,1,2,3,4,5,...
Condition
Memory at Death Table 0x6D1260 at least 2^n, using bitmask 0xFFFFFFFF
Action
Modify memory at Death Table 0x6D1260: substract 2^n, using bitmask 0xFFFFFFFF
Modify memory at Death Table 0x6509B0: add 2^n, using bitmask 0xFFFFFFFF



Then, for 0x6D1260+k, for k=0,..,length*width
Condition
Memory at Death Table 0x6D1260+k at least 2^n, using bitmask 0xFFFFFFFF
Action
Modify memory at Death Table 0x6D1260+k: substract 2^n, using bitmask 0xFFFFFFFF
Modify memory at Death Table 0x6509B0: add 2^n, using bitmask 0xFFFFFFFF


and so on
Everytime 0x6D1260+k reaches zero, we have discovered the original value of 0x6D1260+k.


That should be a destructive copy, right? A non destructive copy would be one in which we don't lower the value at 0x6D1260+k, wouldn't it?



And for accessing that value, maybe we should just add the value of the @flag to the value of the tile? That would mean that it's not actually 0x6D1260+k what we should do, but 0x6D1260+k+c*m, k a counter for flags, c the amount of flags and m the tile

You can always copy into your target value, then copy back into the source to be non-destructive; or if you need to keep the newly copied target (assuming it's not something you can process immediately/perform the check as you copy) you can copy into two values and copy one of them back

Alright! There is some meta-code here. Let's do it step by step.

First things first.


Tiles table isn't @(0x6D1260+k) at all, this would be an error! The pointer value isn't it's address. You must dereference the pointer, i.e. take the value written at memory location @0x6D1260 and substitute that value as an argument to the condition! This is the reason to use clever CP-trick. Before you grasp the concept of dereferencing pointers, let's fix more basic stuff.

Let's look at 'using bitmask 0xFFFFFFFF' part. This is something new in SCMD2 and would be of great help to do the flags manupulation part on separate tiles, given it's a working feature (yet I'm not sure). For simplicity let's put bitmask aside and use simple test/set memory conditions.

Minor things about decor, look at this:

Think about what additional information gives the underlined part. Try not to write redundant (meta)code.

After that, the code above doesn't define n, did you actually mean to write:
Please, be verbose in everything, because it's the source of all kind of misunderstandings and errors.

I'm quitting being a jerk now. Let's tear down the first block.

The devil is in details. The code above doesn't ideologically match construct from Binary Countoffs. The order of triggers in the sequence is different. You should do it 'for n = ..., 5, 4, 3, 2, 1, 0', see the difference? From higher to lower. Why is it important? How high do you need to get with n? These are kind of questions you should ask yourself if you fully understand what is going on.

I try to help you providing a binary interpretation of the process. Let's consider a copy from @Source to @Target, where those entities might be regular death counters, variables at memory, or score/resource amounts, all the same. Let's also assume those are 8-bit integers. Let @Source be 0b00101101 in binary (what number is that?) and @Target is 0b00000000 (zero). The test of a kind

is equivalent to the question

Where 'higher', means 'at left' and positions are enumerated from right to left starting with position #0 (as it usually happens with numbers). This makes more sense if you note that 2^n is exactly a number that has a binary representation with singular '1' at position #n.

Binary Countoff does deconstruction of the @Source from higher bits into lower bits, and constructs the same value at @Target in parallel. The nature of test [*] requires us to deconstruct @Source, because the condition we would like to know is in fact different:

When we are sure the number is "deconstructed" towards bit #n, we use [*] interchangeably in place of [**]. There is no direct way to ask for exactly [**] in triggers.

Binary Countoff does a bit more than copying, it does the assignment of the kind '@Target = @Target + @Source', where @Source equals 0 afterwards. When @Target equals 0 before we start the countoff it is an assignment '@Target = @Source' with destruction of @Source in the process, i.e. destructive copy.

We can adopt binary countoff to do more general assignment all in one go:

This is done with triggers:

Given that we are able to make a non-destructive copy using 2 binary countoffs and a temporary location @Temp:


The last but not the least, you do a primitive, which I call a mutative assignment, i.e. divide value @Source by constant 2^k in the process:

Like this:

After these triggers fire @Target is enlarged by the quotient of the @Source divided by 2^k, and remainder of the division resides in @Source.

About flags.
Nope. Keep in mind, we are addressing chunks of memory in 4-bytes, not singular bits. If you add c*m then you are c*m*4 bytes ahead and that's just another tile.

The classic method to test for bit #s in @Source would be to deconstruct the @Source all the way towards the bit #s with a binary countoff, perform a test, and reconstruct the value using what's being copied to @Target.

If the bitmask feature really works in SCMD2, then it's as trivial as applying the appropriate mask to expose the bit in interest and check for value of @Source is at least 1. I suppose.

Post has been edited 5 time(s), last time on Aug 8 2019, 8:45 pm by Wormer.

So, after reading thislink you posted like 10 times, I get how to dereference.

Now,
, for k=0,...,length*width is the address of every tile, so all of those values should be dereferenced. The same as C, as we said above, right? And, when I get that value I should modify it to anything I want. I could do unbuildable zones, or maybe make Korhal tar zones walkable or whatever. But's that after I've accessed the respective flags. And that's what's next.
My guess is the following:
if what I want is redo fog of war for player 7, what I should do is use bitmask 0x00000006 (if player count starts at 0).

So, what I should do is
Modify value pointed by 0x6D1260+k: 0 using bitmask 0x00000006

if visibility flag=0 is not visible. If visibility flag=1 is not visible, then I should set that value to 1

There are a lot of flags, but not so much that testing is impossible
First two are actually 24, for 12 players, so that give us 40 bitmasks to test. Which I fond odd, because if every tile is separated by 4-bytes, and every byte is 8 bits, then we'd have more bits than 4-bytes


Please do. I'm learning a lot of useful information here, as I'm in informatics. I actually learnt to code doing AI scripts for Warcraft III

Yes, it starts at 0, but it's not as simple as 0x6. Bit-mask does bitwise logic 'and' on binary numbers representation. You need to mask out the bit #6 (zero-based, big-endian) in the last byte means that you need to do bitwise 'and' with 0b01000000 that is decimal 64 or hexadecimal 0x40. If you want to clear the explored bit #14 then mask out 0x4000, and if you want to expose both you apply mask 0x4040.

Terminological note: My use of big-endian here means that we start numbering of (bits) digits in (binary) numbers from the right end.

Given that bit-mask works with CP-trick, the code more or less be the following:

If explained:
My use of if construct in this meta-code means that on each iteration of the enclosing for-cycle you choose only one alternative for trigger action, which will be 'Modify memory at Death Table @0x6509B0: add 1' most of the time but the last iteration that will contain 'Modify memory at Death Table @0x6509B0: set to 6' instead. Code above is equivalent to:

However I don't know how to make MemoryAddr with mask use CP trick [*], and if it is possible at all. Anybody comment on this?

If it's not possible we have got more work to do for manual manipulation of bits:


Post has been edited 18 time(s), last time on Aug 9 2019, 9:22 am by Wormer.

There is something wrong above that I want to correct.



I think (got too lazy to check anything nowadays ) that direct player overflows have never been possible, and NET_MAX_NODES is probably defined to 28. That way, to address memory at 4-bytes aligned location @A via deaths you need to solve an equation '(0x58A364 + (U*12 + P) * 4) = A' against U and P, where P is a value between 0 and 27. Even probably P is the value between 0 and 11, which is sufficient because what we see is 'A/4 - 0x1628D9‬ = U*12 + P' that in turn is a simple integer division of the left side value by 12 with remainder P and quotient U. This way of thought is usually referred to as EUD. Because U is only u16, not all memory can be addressed this way.

Later, CP-trick was discovered, which allows to address the whole memory, because P is u32, and which is simpler in some way becasuse you just put U = 0, P = A/4 - 0x1628D9‬ in the above. This is referred to EPD.

If I'm still wrong on terminology EUD/EPD, people who know more correct me, please.

ADDED:
And now in SC:R they probably just retrospectively added support for direct player overflows, because that piece of code 'if(dwPlayer >= NET_MAX_NODES) return 0;' is gone as seen on slide 57 of that presentation I referred to above.

Post has been edited 5 time(s), last time on Aug 9 2019, 11:46 am by Wormer.

I'm not sure player can't be overflowed - I think the check that triggers the EUD emulator is player out of bounds or unit out of bounds, and when scmdraft solves for optimal P and U there are no constraints besides P > 28 && U > 232. But I don't have source access so I can't check

From what I see looking at this old code, I realize the CP-trick was never possible for set deaths before. Just look at slide 28: app_assert in the first line guards for straightly ill formed values, and the check 'if(dwPlayer >= NET_MAX_NODES) return 0;' guards against the CP-trick, unless NET_MAX_NODES is a huge number. Maybe NET_MAX_NODES is just 8 and that is the reason we can't set deaths of units for extended players (right?). If NET_MAX_NODES is 8 then it also means not even all addresses had been accessible in the old days.

On the other hand test deaths (condition) doesn't guard against the CP-trick in the old code (but still guards with app_assert against ill-formed values).

If the remastered code still resembles to something similar shown us at slides 57-58 then both restrictions on unit and on player are gone now.

Remember, the 1.16 patch prevented EUDs, and they were re-enabled by third party launchers.

Indeed. That demonstration code can't come from patched version, but it might as well be just an internal build, where players overflow is patched and units overflow is left for demo.

I come to conclusion that CP trick is a fairly recent discovery, and EUD/EPD refer to old style direct overflows on units and players respectively that were possible pre-1.16. Terminology explained here: click!-click!.

In other things,
this is sufficient but not necessary. Isn't it just easier to assume U = 0?

Post has been edited 1 time(s), last time on Aug 9 2019, 5:32 pm by Wormer.

Bottom line here. The first approach from there is possible, but SCMD doesn't support CP-trick masked actions yet, so you have to use third party tools to make those triggers. As shown here this approach, given 128x128 map, is easily reduced to 512 triggers, and with more effort boils down to 263 triggers.

The second approach from there works without masked memory access but is of a theoretical interest because it requires thousands triggers (~819000 triggers for a 128x128 map).

What kind of third party tools?
I've found this
http://www.staredit.net/topic/10525/
but it seems I can't make it work. Any folder Iplace it, it doesn't find StarCraft.exe


Another question. I'm reading MTiger suggestion and it got me thinking...what would happen if I centered view in an outside bounds location?

Nothing thrilling at all, it is easy to try. Hello map borders.

Not this kind. In SC:R EUD doesn't need to be enabled to use them. You just need a way to produce triggers. Look at something like TrigEdit++, EUDDraft, eudplib, ProTRG, TriGen, or this. Some of them are outdated, or have not been updated for a while, some are more advanced to use and set up than others.

There is an obvious problem in my code:
CP value must be assigned a EPD for the address stored @0x6D1260, not the value itself, hence:

Thanks SI for the support of masked CP memory access in the SCMD (which is in it's alpha stage yet). Using that I was able to cook up the test map.

Unfortunatelly, it doesn't work . For some reason . Any ideas?

TestVision_v00.scx

My countoffs for memory copy above are designed to copy 32-bit value, which according to EUDDB appears to be the case. But I'm hearing from more experienced people out there that sometimes 64-bit countoffs are needed for addresses manipulation. Is it the problem at hand?

Post has been edited 2 time(s), last time on Aug 13 2019, 11:55 am by Wormer.

Okay, I took the time and created a code that can reset FOW. I used remastered bitmasks (which is now supported in 1.16.1 using the new enabler). You can see the source codes here. Attached is a map with 2 areas. Step on the light one to remove FOW, on the dark one to add FOW (to restore using default FOW you must edit the source code yourself - change one "false" to "true", not that hard to do). Funny thing is, we cannot manipulate tiles in remastered, not even those flags. It's possible to rework the copiers to counters, but it's a waste of time and space.

Edit: it's possible to erase FOW permanently by slight modification in the code. Example in the "_insta.scx" map. Move around and notice that there is no explored map no matter where you go to.
See video

Also, I used TRIG section triggers (enumerated all of them for P1). For performance reasons it would be reasonable to use STR and iterator triggers (~5 triggers instead of tens of thousands).

Attachments:

Post has been edited 3 time(s), last time on Aug 27 2019, 7:16 pm by T-warp.