Autoit Bejeweled bot tutorial part 3: reading the gem colors

AutoIt Bejeweled bot tutorial part 3This article makes part of the Bejeweled bot tutorial series.

Today it's time to create our main loop and integrate a function to read the gem colors. We already located the gem play field and we know each gem its position. With this information we should be fine to complete today's part.

Reading the gem colors

The difficulties

In the previous article we calculated the center of the three gems on the corners. The easiest way to continue, would be to read the pixel color at each gem center. Unfortunately that's not sufficient as the gem center we have calculated is not always the exact gem center. When writing the bot code originally, I found out that Bejeweled often seemed to randomly add an additional horizontal and vertical row of pixels to the gem field. If the gem field would contain only yellow gems for instance, our pixel reading function would not always return exactly the same color. Sometimes the function would return the color of the pixel beside the actual center of the gem, because of the additional row or column somewhere in the field.

The possible solutions

I found two potential solutions for this problem:

  1. Reading the color code at each calculated gem center and applying a margin to the color code when determining whether gems are equal afterwards
  2. Reading the gem color on locations close to the center and using exact color codes to determine equal gems afterwards
I chose the second solution because I don't like to apply margins to the gem colors. Even the lowest margin can lead to different gem colors being associated, as some gem colors have color codes which also exist at other gems with different colors.

The pixel reading offset

Now, a few years later, I discovered the real problem of the additional rows and columns. Bejeweled namely draws one additional horizontal and vertical row in the center of the gem field. Next image shows this row and column in white. Because of these lines, our gem center calculations are not correct anymore in part 2, 3 and 4

bejeweled bot gem field

With this new information I can update the pixel reading mechanism by calculating the exact center for part 2, 3 and 4 separately, but I won't change it at this moment. The pixel reading offset functionality which I implemented as workaround covers this case and still proves to work fine anyway.

The pixel reading offset or pixel shift method lets the bot read pixel colors on locations other than the calculated center. Each gem type (color) has an area with multiple pixels of the same color which we could call the safe area. If we read the pixel color within the borders of that area, we always get back the same color code for that gem, even if the gem shifts 1 pixel horizontally or vertically.

Following images show the safe zone for the white and purple gem. They both benefit from the pixel shift set -4 (x coordinate), -2 (y coordinate). The dot in black is the pixel of which the color code is read.

bejeweled bot center gems

After having investigated all gems, I found out that three pixel shift sets would be sufficient to cover all gem types: 10,10 / 5,5 and -4,-2

Note that the pixel shift set values relate somehow to the bejeweled game proportions. I configured the pixel shift set values when playing on If a site other than displays the bejeweled game with a different size, these pixel shift sets need to be revised. For instance, if the bejeweled game has only half the size of the one on, then the pixel shift sets need to divided by two as well.

Implementing the pixel reading offset

First add following key to the configuration file:

Next add this code to the declaration section:

$pixelShift=IniRead ( $iniFile, "Common" , "pixelShift", 0 )
The pixel shift sets need to be split into array for easier processing. Add this code to the main section:

;Read pixel shift sets into an array
Global $pixelShiftSetCounter = 0
Global $pixelShiftSet = StringSplit($PixelShift,"/")

;For each pixel shift set, create an array storing the x and y coordinates
for $i = 1 to Ubound($pixelShiftSet) - 1
$pixelShiftSet[$i] = stringSplit($pixelShiftSet[$i],",")
In order to loop through the different pixel shift sets when reading the gem field, add following code to the function section:

Func _getNextPixelShiftSet()
if $PixelShiftSetcounter < (Ubound($PixelShiftSet) - 1) then $PixelShiftSetcounter = $PixelShiftSetcounter + 1 else $PixelShiftSetcounter = 1 endif _log("Using Pixelshiftset with number " & $PixelShiftSetcounter & " and content " & _ArrayToString($PixelShiftSet[$PixelShiftSetcounter],",",1)) return $PixelShiftSet[$PixelShiftSetcounter] EndFunc
Once we have the pixel shift sets ready, we can add the pixel reading function. Add following code to the function section:

Func _readPixels()
_log("Start reading pixels")
$pixelShift = _getNextPixelShiftSet()
$shiftHorizontal = $pixelShift[1]
$shiftVertical = $pixelShift[2]

ToolTip('Reading pixels playfield', 0, 0)

dim $Field[8][8]

for $j = $field1[1] to $field3[1] step $blockWidth
for $i = $field1[0] to $field2[0] step $blockWidth
$field[$k][$l] = pixelgetColor($i + $shiftHorizontal,$j + $shiftVertical)

;mousemove($i + $shiftHorizontal,$j + $shiftVertical,40)
$k = $k + 1
$l = $l + 1
$k = 0

return $field
Reading the pixels is one thing, but being able to see which colors were recorded is even better. Add following code to the function section:

Func _logpixels()
$i = 0
$j = 0
for $j = 0 to 7
$line = ""
for $i = 0 to 7
$temp = $field[$i][$j]
$line = $line & " " & hex($temp)

The main loop

Now all pixel reading functions have been implemented, we can set up the main loop of our bot. This loop will read the color code of each gem's center, log the color codes which have been recorded, and later on implement the logic to decide upon which move to make.

Add following code to the main section:

while 1
$switched = false
$field = _ReadPixels()

ToolTip('Start checking if gems can be moved', 0, 0)

;PLACEHOLDER for the movement of gems

ToolTip('Stop checking if gems can be moved', 0, 0)

if $switched = true then
_log("a move was performed")
_log("checked whole field, no moves were done")
All above function calls and variables which the loop is referencing to should exist, except for the delay of searching new gems after having performed a move. Add following key to the configuration file:
First add following key to the configuration file:

And add following code to the declaration section:

$searchDelay=IniRead ( $iniFile, "Common" , "searchDelay", 1200 )

Proof running the code

At this point you should be able to run you script and see more results. Save your script and configuration file, and run the bot script by pressing F5 in your ScITE editor. If you correctly followed all steps, the bejeweled bot should be performing all steps from previous parts, and the bot should read the pixel colors of the gems. You can verify this by uncommenting the 2 comment lines in the _readPixels function. This will move the mouse to each gem and will give you an idea of what is happening exactly.

If you encounter difficulties with the execution of these steps and you don't succeed in fixing the problem yourself, then leave a message in the comment section. Don't forget to include the error message shown in the ScITE output window. I will answer to your question as soon as possible.

Continue to the fourth part of the bejeweled bot tutorial: switching the gems

Related Posts by Categories


24 Responses to "Autoit Bejeweled bot tutorial part 3: reading the gem colors"

Justin said... April 13, 2009 at 3:57 AM

It works well, however when im trying to fix it for Facebook's Bejeweled blitz, I cannot seem to get a match for the colour across the 3 different offsets. I have tried various offsets to no success. I also had to fix the coordinate for the game screen as the background changes every game. This i have been able to sort out. I am using Mozilla Firefox and I maximize the window. My resolution is 1440x900. Could you please take a look and see what my fixed coordinates for the game area should be for each corner and the pixel offsets cause I got no hope of figuring this out lol.

Steven Machtelinckx said... April 13, 2009 at 8:13 PM

I would work with fixed corner coordinates, or with variable corner coordinates starting from the blue bullets on the four corners. If the blue bullets don't exactly form an even and perfect square to create the gem field from, you can always add offsets to the corners as well, as mentioned in a comment on the previous article:
(my comment from April 2, 2009 8:03 AM)

In order to find how much offset you need for all corners, I recommend to make a screenshot and to work with that file, for instance in paint or photoshop if possible.

When having a look at a screenshot I made from the gem play field and zooming in, I notice that they applied a lot of gradients to the gem colors :-) This means that you need to form a correct square around the gems which, when divided into gem squares, need to have exactly the same size. Using an offset to get in a "safe zone" won't help that much apparently as I don't see safe zones :-)

Possibly you might have to introduce a small margin when comparing the color codes afterwards.

When I find some more time, I will investigate this further.

Justin said... April 14, 2009 at 3:39 AM

Yeah what I thought was to break down the red,green and blue colur numbers of each gem and determining with if statements what each gem would be based on that. If I went by the blue bullets, would it mean that a negative offset would not be used as it would go off the game area?

Steven Machtelinckx said... April 14, 2009 at 7:34 AM

Breaking down the red, green and blue values can be a solution; I don't have much experience with it.

Regarding the blue bullets: it can be necessary to apply a negative offset to the lower right blue bullet in order to get closer to the imaginary perfect square around the gem play field.

Anonymous said... May 17, 2009 at 8:56 PM

Hey, what is ETA for part 4? Waiting eagerly :D

Steven Machtelinckx said... May 18, 2009 at 7:47 AM

That's a very good question :-) I will try to have it ready at the beginning of next week!

Anonymous said... May 18, 2009 at 9:14 PM

Awesome, thanks, Can't wait :-)

Anonymous said... June 23, 2009 at 12:40 PM

Where is the 4th part with logic etc?!

Steven Machtelinckx said... June 24, 2009 at 7:29 AM

The 4th part is still to come. I've been very busy the last months. But at this moment it's hard to give an ETA.

Anonymous said... July 23, 2009 at 4:53 PM

is it possible to see a source code of at least how the logic works so that we can try to figure it out ourselves in the mean time? I mean if you have followed up to this point, chances are you can understand the logic behind the code if you see it.

Anonymous said... September 8, 2009 at 7:49 AM

Still waiting for the 4th part ... sept already.

Steven Machtelinckx said... September 11, 2009 at 7:18 AM

One of the reasons of postponing the article is the fact that I would like to rewrite the last part before publishing it. The code works fine, but I find that it's not well structured.

It will take some time but before the end of the month I will publish the new article. Next week I'm on holiday so I'm not able to start with it yet.

Anonymous said... September 11, 2009 at 5:44 PM

Enjoy a good holiday!
We all look forward to your part 4 to complete the tutorial. Thanks in advance.

Anonymous said... November 3, 2009 at 11:03 PM

im having trubles with the # include file thingy i tried adding in the file name and i dont know what i am doing wrong but i keep on gettign the same thing my msn is if any one can help

Jason Palchak said... June 7, 2010 at 3:27 AM

Hello again. In my TH bot there is no piece offset. The pieces are the same EVERY time. So what would I do to make them "readable"

How would I do the image recognition?

Anonymous said... August 3, 2010 at 2:00 PM

I am getting the errors below when trying to compile the script so far...
Sorry I am new to programming so might be noob problem!

C:\BOT\V1.au3(197,115) : ERROR: syntax error
if $PixelShiftSetcounter < (Ubound($PixelShiftSet) - 1) then $PixelShiftSetcounter = $PixelShiftSetcounter + 1 else
C:\BOT\V1.au3(197,301) : ERROR: syntax error
if $PixelShiftSetcounter < (Ubound($PixelShiftSet) - 1) then $PixelShiftSetcounter = $PixelShiftSetcounter + 1 else $PixelShiftSetcounter = 1 endif _log("Using Pixelshiftset with number " & $PixelShiftSetcounter & " and content " & _ArrayToString($PixelShiftSet[$PixelShiftSetcounter],",",1))
C:\BOT\V1.au3(200,1) : ERROR: syntax error
C:\BOT\V1.au3(134,22) : ERROR: _ReadPixels(): undefined function.
$field = _ReadPixels()
C:\BOT\V1.au3(135,12) : ERROR: _LogPixels(): undefined function.

Any help appreciated!

Steven Machtelinckx said... August 3, 2010 at 3:55 PM


The error in your post references to the function _ReadPixels() which cannot be found. Do you have it in your script file?

Anonymous said... September 16, 2010 at 6:35 AM

i do
but it says
1. Func _logpixels()
2. $i = 0
3. $j = 0
4. for $j = 0 to 7
5. $line = ""
6. for $i = 0 to 7
7. $temp = $field[$i][$j]
8. $line = $line & " " & hex($temp)
9. next
10. _Log($line)
11. next
12. _Log("")
13. EndFunc

it says no matching end func for log pixels is found?????

Anonymous said... January 5, 2011 at 4:31 AM

Trying to create a bot for Bejeweled Blitz on facebook, bot works great except that the program takes 2 seconds to read the playing field. This causes the bot to run slow because it basically can't execute more than about 25 moves per game and I can't sustain the speed bonus. What can I do to improve the speed of the loop? This is the current loop that I have.

Func _readPixels()
_log("Start reading pixels")
ToolTip('Reading pixels playfield', 0, 0)

dim $Field[8][8]
;MsgBox(0, "Field", $field1[1] & "," & $field3[1])
for $j = $field1[1] to $field3[1] step $blockWidth
for $i = $field1[0] to $field2[0] step $blockWidth
$field[$k][$l] = pixelgetColor($i,$j)

;mousemove($i + $shiftHorizontal,$j + $shiftVertical,40)
$k = $k + 1
$l = $l + 1
$k = 0

return $field

Loïc said... May 19, 2011 at 3:11 AM

>"D:\Program Files (x86)\AutoIt3\SciTE\..\autoit3.exe" /ErrorStdOut "F:\BEJEWELED BOT\BJBot.au3"
F:\BEJEWELED BOT\BJBot.au3 (352) : ==> "Func" statement has no matching "EndFunc".:
Func _readPixels()

>Exit code: 1 Time: 0.272


You had make a nice tutorial.
Thanks you so much. I'll start to learn a few code doing bots. I carefully follow the steps, but I still getting this error, I don't understand why.

Have a nice day.

anoniem said... July 10, 2011 at 10:59 AM

Hi, I keep getting: >"C:\Program Files\AutoIt3\SciTE\..\autoit3.exe" /ErrorStdOut "C:\Users\Mike \Documents\autoit bestanden\autoit bejeweled bot.au3"
C:\Users\Mike \Documents\autoit bestanden\autoit bejeweled bot.au3 (338) : ==> "Func" statement has no matching "EndFunc".:
Func _readPixels()

>Exit code: 1 Time: 0.229

when I check it, it says:

>C:\Program Files\AutoIt3\SciTE\..\au3check.exe "C:\Users\Mike \Documents\autoit bestanden\autoit bejeweled bot.au3"
AutoIt3 Syntax Checker v1.54.8 Copyright (c) Tylo 2007

C:\Users\Mike \Documents\autoit bestanden\autoit bejeweled bot.au3(336,115) : ERROR: syntax error
if $PixelShiftSetcounter < (Ubound($PixelShiftSet) - 1) then $PixelShiftSetcounter = $PixelShiftSetcounter + 1 else
C:\Users\Mike \Documents\autoit bestanden\autoit bejeweled bot.au3(336,301) : ERROR: syntax error
if $PixelShiftSetcounter < (Ubound($PixelShiftSet) - 1) then $PixelShiftSetcounter = $PixelShiftSetcounter + 1 else $PixelShiftSetcounter = 1 endif _log("Using Pixelshiftset with number " & $PixelShiftSetcounter & " and content " & _ArrayToString($PixelShiftSet[$PixelShiftSetcounter],",",1))
C:\Users\Mike \Documents\autoit bestanden\autoit bejeweled bot.au3(101,22) : ERROR: _ReadPixels(): undefined function.
$field = _ReadPixels()
C:\Users\Mike \Documents\autoit bestanden\autoit bejeweled bot.au3(102,12) : ERROR: _LogPixels(): undefined function.
C:\Users\Mike \Documents\autoit bestanden\autoit bejeweled bot.au3(132,54) : ERROR: _ColorsMatch(): undefined function.
if _ColorsMatch($ColorGem1, $ColorGem2, $ColorGem4)
C:\Users\Mike \Documents\autoit bestanden\autoit bejeweled bot.au3(137,46) : ERROR: _Switch(): undefined function.
$switched = _Switch($firstGem, $secondGem)
C:\Users\Mike \Documents\autoit bestanden\autoit bejeweled bot.au3 - 6 error(s), 0 warning(s)
>Exit code: 2 Time: 0.339

Jason Palchak said... August 24, 2011 at 6:30 PM


I have been modifying this bot to work on this puzzle:

Which is VERY similar to bejewled.

I'm having issues with this part (reading the gem colors)

How would I modify this since I don't need the offset you have since the puzzle screen is the same size all the time?

The mediafire link is my code + images for you to look at. Any help is GREATLY appreciated.

Software Testing Class said... June 19, 2012 at 8:50 PM

Indeed, those closing tags for the array and file include should not be there. The code for matter is adding them for an unknown reason. You can completely delete them.
You can get info on The Software Testing Class as well with some guidelines with different way of thinking.

TestWithUs said... May 7, 2013 at 5:57 AM

SWIFT Interview questions on

For selenium solution visit

For QTP interview questions

Post a Comment

Recent Articles

Top Commenters

Recent Comments