Bug Report: "Copy as group search" assumes values are consecutive when used with utf-8 text and utf-16 text options

[HorridModz]

@EnybyHello, I am reporting a behavior that appears to be a bug in gameguardian.

It seems that the "Copy as group search", when used with "UTF-8" and "UTF-16", assumes that the values are consecutive, i.e. one byte apart. However, this is not always the case, causing group searches to sometimes fail.

This is best illustrated by examples: Let's search the string "abc" in UTF-8 and save the first 3 results, but leave out the second one:

gg.clearResults()
gg.setRanges(gg.REGION_ANONYMOUS)
gg.searchNumber(":abc", gg.TYPE_BYTE)
gg.addListItems(gg.getResults(1)) -- first result - "a"
gg.addListItems(gg.getResults(1, 2)) --third result - "c"

Now, we have two results: 97 byte, representing the character "a", and 99, representing the character "c". These results are a byte apart - so they do not directly form the string "ac" and would not be searchable as ":ac"; rather, they would be searchable by a group search that accounts for the byte in between: 97B;0~~0B;99B::3. However, the Copy as Group Search function, when I select UTF-8, gives ":ac". This is not the right group search and seems to be a bug.

In order to show why this is wrong, here is the situation where I discovered the bug. I had a script that writes a string to memory, like this:
 

function Write_String(mystring)
    --[[ Allocate memory in Anonymous region and write string to allocated memory ]]--
    address = gg.allocatePage(gg.PROT_READ | gg.PROT_WRITE, gg.REGION_ANONYMOUS)
    values = {}
    for i = 0, #mystring do
        values[#values + 1] = {address = address + i, value = ":" .. mystring:sub(i, i), flags = gg.TYPE_BYTE}
    end
    gg.loadResults(values) -- must load results before we can edit with setvalues
    gg.setValues(values)
end

Write_String("Here is my special string!")

If I run this script, I get the string written in consecutive byte values. I use Copy as Group Search for UTF-8, and it gives me the correct search: ":Here is my special string!".

But now let's revert what we just did by refreshing the game, change the address spacing to every 2 bytes, so the string is no longer consecutive, and copy it as a group search again:

function Write_String(mystring)
    --[[ Allocate memory in Anonymous region and write string to allocated memory ]]--
    address = gg.allocatePage(gg.PROT_READ | gg.PROT_WRITE, gg.REGION_ANONYMOUS)
    values = {}
    for i = 0, #mystring do
        values[#values + 1] = {address = address + i*2, value = ":" .. mystring:sub(i, i), flags = gg.TYPE_BYTE}
    end
    gg.loadResults(values) -- must load results before we can edit with setvalues
    gg.setValues(values)
end

Write_String("Here is my special string!")

We copy it as group search again, and get the exact same thing:  ":Here is my special string!". However, now that the addresses are 2 bytes apart, this is wrong and will not work. If we try to search it, nothing comes up.


So, this is definitely not working right. I'm pretty sure that it's a bug. My suggestion for fixing this would simply be displaying an error message if the addresses were not consecutive, like this: "UTF-8 [or UTF-16] group search only works for consecutive addresses. Please use a different type of group search."

If there's any other information anyone would like, feel free to ask.

Edited April 21 by HorridModz

[APEXggV2]

2 hours ago, HorridModz said:

@EnybyHello, I am reporting a behavior that appears to be a bug in gameguardian.

It seems that the "Copy as group search", when used with "UTF-8" and "UTF-16", assumes that the values are consecutive, i.e. one byte apart. However, this is not always the case, causing group searches to sometimes fail.

This is best illustrated by examples: Let's search the string "abc" in UTF-8 and save the first 3 results, but leave out the second one:

gg.clearResults()
gg.setRanges(gg.REGION_ANONYMOUS)
gg.searchNumber(":abc", gg.TYPE_BYTE)
gg.addListItems(gg.getResults(1)) -- first result - "a"
gg.addListItems(gg.getResults(1, 2)) --third result - "c"

Now, we have two results: 97 byte, representing the character "a", and 99, representing the character "c". These results are a byte apart - so they do not directly form the string "ac" and would not be searchable as ":ac"; rather, they would be searchable by a group search that accounts for the byte in between: 97B;0~~0B;99B::3. However, the Copy as Group Search function, when I select UTF-8, gives ":ac". This is not the right group search and seems to be a bug.

In order to show why this is wrong, here is the situation where I discovered the bug. I had a script that writes a string to memory, like this:
 

function Write_String(mystring)
    --[[ Allocate memory in Anonymous region and write string to allocated memory ]]--
    address = gg.allocatePage(gg.PROT_READ | gg.PROT_WRITE, gg.REGION_ANONYMOUS)
    values = {}
    for i = 0, #mystring do
        values[#values + 1] = {address = address + i, value = ":" .. mystring:sub(i, i), flags = gg.TYPE_BYTE}
    end
    gg.loadResults(values) -- must load results before we can edit with setvalues
    gg.setValues(values)
end

Write_String("Here is my special string!")

If I run this script, I get the string written in consecutive byte values. I use Copy as Group Search for UTF-8, and it gives me the correct search: ":Here is my special string!".

But now let's revert what we just did by refreshing the game, change the address spacing to every 2 bytes, so the string is no longer consecutive, and copy it as a group search again:

function Write_String(mystring)
    --[[ Allocate memory in Anonymous region and write string to allocated memory ]]--
    address = gg.allocatePage(gg.PROT_READ | gg.PROT_WRITE, gg.REGION_ANONYMOUS)
    values = {}
    for i = 0, #mystring do
        values[#values + 1] = {address = address + i*2, value = ":" .. mystring:sub(i, i), flags = gg.TYPE_BYTE}
    end
    gg.loadResults(values) -- must load results before we can edit with setvalues
    gg.setValues(values)
end

Write_String("Here is my special string!")

We copy it as group search again, and get the exact same thing:  ":Here is my special string!". However, now that the addresses are 2 bytes apart, this is wrong and will not work. If we try to search it, nothing comes up.


So, this is definitely not working right. I'm pretty sure that it's a bug. My suggestion for fixing this would simply be displaying an error message if the addresses were not consecutive, like this: "UTF-8 [or UTF-16] group search only works for consecutive addresses. Please use a different type of group search."

If there's any other information anyone would like, feel free to ask.

this isn't a bug..  copy group size will copy byte values with proper distance..( 97;99::3 )

if you want to merge the characters together to make a consecutive string,  then select utf8 or utf16.. (:ac) this is useful if you have half a string ("Hello") starting at address 0xABC and also need another half ("World") at address 0xCBA,  Now you can merge and search. ":Hello World"  and not do a group byte search for values that are ::270 , this would be drastic!!  

Edited April 22 by APEXggV2

[NoFear]

3 hours ago, HorridModz said:

@EnybyHello, I am reporting a behavior that appears to be a bug in gameguardian.

It seems that the "Copy as group search", when used with "UTF-8" and "UTF-16", assumes that the values are consecutive, i.e. one byte apart. However, this is not always the case, causing group searches to sometimes fail.

This is best illustrated by examples: Let's search the string "abc" in UTF-8 and save the first 3 results, but leave out the second one:

gg.clearResults()
gg.setRanges(gg.REGION_ANONYMOUS)
gg.searchNumber(":abc", gg.TYPE_BYTE)
gg.addListItems(gg.getResults(1)) -- first result - "a"
gg.addListItems(gg.getResults(1, 2)) --third result - "c"

Now, we have two results: 97 byte, representing the character "a", and 99, representing the character "c". These results are a byte apart - so they do not directly form the string "ac" and would not be searchable as ":ac"; rather, they would be searchable by a group search that accounts for the byte in between: 97B;0~~0B;99B::3. However, the Copy as Group Search function, when I select UTF-8, gives ":ac". This is not the right group search and seems to be a bug.

In order to show why this is wrong, here is the situation where I discovered the bug. I had a script that writes a string to memory, like this:
 

function Write_String(mystring)
    --[[ Allocate memory in Anonymous region and write string to allocated memory ]]--
    address = gg.allocatePage(gg.PROT_READ | gg.PROT_WRITE, gg.REGION_ANONYMOUS)
    values = {}
    for i = 0, #mystring do
        values[#values + 1] = {address = address + i, value = ":" .. mystring:sub(i, i), flags = gg.TYPE_BYTE}
    end
    gg.loadResults(values) -- must load results before we can edit with setvalues
    gg.setValues(values)
end

Write_String("Here is my special string!")

If I run this script, I get the string written in consecutive byte values. I use Copy as Group Search for UTF-8, and it gives me the correct search: ":Here is my special string!".

But now let's revert what we just did by refreshing the game, change the address spacing to every 2 bytes, so the string is no longer consecutive, and copy it as a group search again:

function Write_String(mystring)
    --[[ Allocate memory in Anonymous region and write string to allocated memory ]]--
    address = gg.allocatePage(gg.PROT_READ | gg.PROT_WRITE, gg.REGION_ANONYMOUS)
    values = {}
    for i = 0, #mystring do
        values[#values + 1] = {address = address + i*2, value = ":" .. mystring:sub(i, i), flags = gg.TYPE_BYTE}
    end
    gg.loadResults(values) -- must load results before we can edit with setvalues
    gg.setValues(values)
end

Write_String("Here is my special string!")

We copy it as group search again, and get the exact same thing:  ":Here is my special string!". However, now that the addresses are 2 bytes apart, this is wrong and will not work. If we try to search it, nothing comes up.


So, this is definitely not working right. I'm pretty sure that it's a bug. My suggestion for fixing this would simply be displaying an error message if the addresses were not consecutive, like this: "UTF-8 [or UTF-16] group search only works for consecutive addresses. Please use a different type of group search."

If there's any other information anyone would like, feel free to ask.

I believe the utf-8 and utf-16 is assumed consecutive characters, so will always be treated that way.  Not really a bug. Can't search utf with "wildcard"/missing character. Byte search would be best approach.