Auto-challenge-checkers

What is it

The Auto-challenge-checkers is one of the many paid membership features at Project-GC. If you are a paid member, Project-GC automatically runs Challenge checkers for you in the background so that when you view the Challenges you will know beforehand if you fulfill them or not. This will for example be shown on the maps where the cache icon will get a challenge status indicator in the lower left corner. The checker image in the cache descriptions can also show the status. The status shown is a cached result, not a real-time result.

Background processing

Since there are tens of thousands of Challenge checkers and also thousands of paying members we can't run them all for every account on an hourly basis, or even daily. Some Challenge checkers require about a second to run, while others require 30 seconds. Due to this we have an algorithm that prioritizes what we expect to be most relevant.

Basically there are ten processes running 24/7, only storing results from the Challenge checkers. The process is the following:

• Go through all paying members and order them by the date when they were last processed.
• Find the 500 most relevant Challenge checkers for each user. These 500 are chosen by going through a few steps. As soon as 500 is reached, the rest of the steps are skipped.

1. Challenges with a note from the user, that haven't been run for a week.
2. Challenges not logged by the user, not tested the last month, and within 500 km from home.
3. Challenges that haven't been tested the last two months, within 500 km from home.
4. Challenges not logged by the user and not tested the last two months, in the same country as the user. If country is United States, also requires the home state.
5. Challenges not tested the last three months, in the same country as the user. If country is United States, also requires the home state.
6. Challenges not logged by the user and not tested in three months, within 1000 km from home.
7. Challenges not tested within four months and within 1000 km from home.
8. Challenges not logged by the user, not tested in four months, from the whole world, in a random order.
9. Challenges not tested for six months, from the whole world, in random order.

• Run those (up to) 500 challenge checkers.
• Start over, with the next user in the queue.

Steps depending on home coordinates are skipped if they are not known to Project-GC. Step 1-7 are ordered by distance from home, closest to home is run first.

Exceptions

Some Challenge checkers are never run automatically. Basically there are two variants:

1. Some of the Challenge checkers at Project-GC are written in alternative ways and are not supported by the Auto-challenge-checkers. It's challenge checkers written by employees at Project-GC and they aren't compatible.
2. Challenge checkers that often require more than 30 seconds to run are excluded from the automation. This to leave room for multiple others to be run instead.

Hardware used

As mentioned there are ten processes handling this. They are run in a Virtual machine with 12 hardware threads and RAM enough to handle the maximum amount of memory allowed for each process. Depending on the challenge, and its script, the database cluster could be the bottleneck, or the LUA scripts. The LUA scripts are created by the community, and while some are very optimized, others are less so. The database cluster is shared with other services doing data harvesting. It consists of 8 servers, with 16 hardware threads each, and more memory than they need.

Future

We plan to provide a front-end configuration tool where the users can move the focus. For example, if a user plans to travel to another country, they might want to ask the system to run the Challenge checkers in that country instead. The current idea is to allow up to five circles of various radiuses on a map, which the users can move themselves.

Such page would probably also include some analyzing numbers, like which Challenge checkers were last run, when they were run, how many have been run and such.