Difference between revisions of "Auto-challenge-checkers"

From Project-GC
Jump to: navigation, search
m (grammar)
(Various copy edits.)
Line 1: Line 1:
 
== What is it ==
 
== What is it ==
The Auto-challenge-checkers is one of the many [[paid membership]] features at Project-GC. As a paid member, Project-GC is automatically running [[Challenge checkers]] in the background, so that when you view the Challenges, you will know beforehand if you fulfill them or not. This will for example be shown on the Maps where the cache icon will get a challenge-status-indicator in the lower left corner. The checker image in the cache descriptions can also show the status. The status shown is a cached result, not a real-time result.
+
The Auto-challenge-checkers is one of the many [[paid membership]] features at Project-GC. If you are a paid member, Project-GC automatically runs [[Challenge checkers]] for you in the background so that when you view the Challenges you will know beforehand if you fulfill them or not. This will for example be shown on the maps where the cache icon will get a challenge status indicator in the lower left corner. The checker image in the cache descriptions can also show the status. The status shown is a cached result, not a real-time result.
  
 
== Background processing ==
 
== Background processing ==
Since there are tens of thousands of [[Challenge checkers]] and also thousands of paying members we can't run them all for every account on an hourly basis, not even daily. Some [[Challenge checkers]] requires about a second to run, while others requires 30 seconds. Due to this we have an algorithm in hand that prioritizes what we expect to be most relevant.
+
Since there are tens of thousands of [[Challenge checkers]] and also thousands of paying members we can't run them all for every account on an hourly basis, or even daily. Some [[Challenge checkers]] require about a second to run, while others require 30 seconds. Due to this we have an algorithm that prioritizes what we expect to be most relevant.
  
 
Basically there are ten processes running 24/7, only storing results from the [[Challenge checkers]]. The process is the following:
 
Basically there are ten processes running 24/7, only storing results from the [[Challenge checkers]]. The process is the following:
* Go through all paying members, order them on the date they were last processed.
+
* Go through all paying members and order them by the date when they were last processed.
* Find the 500 most relevant [[Challenge checkers]] for each user. Those 500 are chosen by going through a few steps. As soon as 500 is reached, the rest of the steps are skipped.
+
* Find the 500 most relevant [[Challenge checkers]] for each user. These 500 are chosen by going through a few steps. As soon as 500 is reached, the rest of the steps are skipped.
# Challenges with a note from the user, that hasn't been run for a week.
+
# Challenges with a note from the user, that haven't been run for a week.
 
# Challenges not logged by the user, not tested the last month, and within 500 km from home.
 
# Challenges not logged by the user, not tested the last month, and within 500 km from home.
# Challenges that haven't been tested last two months, within 500 km from home.
+
# Challenges that haven't been tested the last two months, within 500 km from home.
 
# Challenges not logged by the user and not tested the last two months, in the same country as the user. If country is United States, also requires the home state.
 
# Challenges not logged by the user and not tested the last two months, in the same country as the user. If country is United States, also requires the home state.
# Challenges not tested the last 3 months, in the same country as the user. If country is United States, also requires the home state.
+
# Challenges not tested the last three months, in the same country as the user. If country is United States, also requires the home state.
 
# Challenges not logged by the user and not tested in three months, within 1000 km from home.
 
# Challenges not logged by the user and not tested in three months, within 1000 km from home.
# Challenges not tested within 4 months and within 1000 km from home.
+
# Challenges not tested within four months and within 1000 km from home.
# Challenges not logged by the user, not tested for four months, from the whole world, in a random order.
+
# Challenges not logged by the user, not tested in four months, from the whole world, in a random order.
 
# Challenges not tested for six months, from the whole world, in random order.
 
# Challenges not tested for six months, from the whole world, in random order.
 
* Run those (up to) 500 challenge checkers.
 
* Run those (up to) 500 challenge checkers.
* Start over, with the next user in queue.
+
* Start over, with the next user in the queue.
Steps depending on home coordinates are skipped if not known to [[Project-GC]].
+
Steps depending on home coordinates are skipped if they are not known to [[Project-GC]].
 
Step 1-7 are ordered by distance from home, closest to home is run first.
 
Step 1-7 are ordered by distance from home, closest to home is run first.
  
 
== Exceptions ==
 
== Exceptions ==
Some Challenge checkers are never run automatically. Basically there are two variants.
+
Some Challenge checkers are never run automatically. Basically there are two variants:
 
# Some of the Challenge checkers at Project-GC are written in alternative ways and are not supported by the Auto-challenge-checkers. It's challenge checkers written by employees at Project-GC and they aren't compatible.
 
# Some of the Challenge checkers at Project-GC are written in alternative ways and are not supported by the Auto-challenge-checkers. It's challenge checkers written by employees at Project-GC and they aren't compatible.
# Challenge checkers that often requires more than 30 seconds to run are excluded from the automation. This to leave room for multiple others to be run instead.
+
# Challenge checkers that often require more than 30 seconds to run are excluded from the automation. This to leave room for multiple others to be run instead.
  
 
== Hardware used ==
 
== Hardware used ==
As mentioned there are ten processes handling this. The server they are running on is not the bottle neck. The bottle neck is a database cluster, which is shared with other data harvesting processes. It consists of 8 servers, with 16 hardware threads each, and more memory than they need.
+
As mentioned there are ten processes handling this. The server they are running on is not the bottleneck. The bottleneck is a database cluster, which is shared with other data harvesting processes. It consists of 8 servers, with 16 hardware threads each, and more memory than they need.
  
 
== Future ==
 
== Future ==
 
We plan to provide a front-end configuration tool where the users can move the focus. For example, if a user plans to travel to another country, they might want to ask the system to run the Challenge checkers in that country instead. The current idea is to allow up to five circles of various radiuses on a map, which the users can move themselves.
 
We plan to provide a front-end configuration tool where the users can move the focus. For example, if a user plans to travel to another country, they might want to ask the system to run the Challenge checkers in that country instead. The current idea is to allow up to five circles of various radiuses on a map, which the users can move themselves.
  
Such page would probably also include some analyzing numbers, like which Challenge checkers was last run, when they were run, how many has been run and such.
+
Such page would probably also include some analyzing numbers, like which Challenge checkers were last run, when they were run, how many have been run and such.

Revision as of 12:35, 16 June 2020

What is it

The Auto-challenge-checkers is one of the many paid membership features at Project-GC. If you are a paid member, Project-GC automatically runs Challenge checkers for you in the background so that when you view the Challenges you will know beforehand if you fulfill them or not. This will for example be shown on the maps where the cache icon will get a challenge status indicator in the lower left corner. The checker image in the cache descriptions can also show the status. The status shown is a cached result, not a real-time result.

Background processing

Since there are tens of thousands of Challenge checkers and also thousands of paying members we can't run them all for every account on an hourly basis, or even daily. Some Challenge checkers require about a second to run, while others require 30 seconds. Due to this we have an algorithm that prioritizes what we expect to be most relevant.

Basically there are ten processes running 24/7, only storing results from the Challenge checkers. The process is the following:

  • Go through all paying members and order them by the date when they were last processed.
  • Find the 500 most relevant Challenge checkers for each user. These 500 are chosen by going through a few steps. As soon as 500 is reached, the rest of the steps are skipped.
  1. Challenges with a note from the user, that haven't been run for a week.
  2. Challenges not logged by the user, not tested the last month, and within 500 km from home.
  3. Challenges that haven't been tested the last two months, within 500 km from home.
  4. Challenges not logged by the user and not tested the last two months, in the same country as the user. If country is United States, also requires the home state.
  5. Challenges not tested the last three months, in the same country as the user. If country is United States, also requires the home state.
  6. Challenges not logged by the user and not tested in three months, within 1000 km from home.
  7. Challenges not tested within four months and within 1000 km from home.
  8. Challenges not logged by the user, not tested in four months, from the whole world, in a random order.
  9. Challenges not tested for six months, from the whole world, in random order.
  • Run those (up to) 500 challenge checkers.
  • Start over, with the next user in the queue.

Steps depending on home coordinates are skipped if they are not known to Project-GC. Step 1-7 are ordered by distance from home, closest to home is run first.

Exceptions

Some Challenge checkers are never run automatically. Basically there are two variants:

  1. Some of the Challenge checkers at Project-GC are written in alternative ways and are not supported by the Auto-challenge-checkers. It's challenge checkers written by employees at Project-GC and they aren't compatible.
  2. Challenge checkers that often require more than 30 seconds to run are excluded from the automation. This to leave room for multiple others to be run instead.

Hardware used

As mentioned there are ten processes handling this. The server they are running on is not the bottleneck. The bottleneck is a database cluster, which is shared with other data harvesting processes. It consists of 8 servers, with 16 hardware threads each, and more memory than they need.

Future

We plan to provide a front-end configuration tool where the users can move the focus. For example, if a user plans to travel to another country, they might want to ask the system to run the Challenge checkers in that country instead. The current idea is to allow up to five circles of various radiuses on a map, which the users can move themselves.

Such page would probably also include some analyzing numbers, like which Challenge checkers were last run, when they were run, how many have been run and such.