Mods Request for GetFavorites

The GetFavorites API call that we use can only return 100 caches. This means that any disabled or archived caches get filtered out for printing the table. However we need them in the list to see if a finder has found them. At this time there is no way of making multiple calls to build a list larger than 100 in the API call because there is no max favorite points option to get the second batch of 100.

An example of how this is impacting things is in http://project-gc.com/forum/read?8,9938

There are two actions that can solve this problem.

The first is increase the max of limit to something larger. 1000 comes to mind from other calls.

This would only be temporary until the number of archived and disabled plus the number of caches is more than 1000.
To solve this it would be nice to have a maximum number of FP/Wilson/... option as well. That way one could get to any number larger than limit through multiple calls.

Thanks.

The reason for the fairly low limits is to protect the data of Geocaching.com. Avoiding data harvesting that is, or at least making it harder.

If I understand your issue correctly, the best solution would be to return top 100 active geocaches and then also add all archived and disabled on top of that?

In programming it would be more like, get the number of favorite points of the cache with the 100th most. Then get all caches which has that amount of FP or more.

Does this sound like a solution?

It would check the box and let this checker through the gates. However once someone requests a 100 of the top 250 we are back in the same boat.

However it seems to have some other issues from my perspective. To minimize confusion among developers, it seems that there should be as much parallelism between GetFavorites and GetOldestHidden calls. 1000 is ok for getting oldest but not GetFavorites?

Regarding the data harvesting, it appears the horse is already out of the barn. The GetOldestHidden call already allows folks to pull in all the data By repeated calls and editing the minHiddenDate between calls one can step through the data and harvest it all. (Atleast until there is more than limit data in the smallest query available for a day.)

If data harvesting is the real concern, then there should be some API changes but expect some unhappiness from the users and cache owners. Instead of returning lists, there should be GetOldest and GetFavorite calls that return just a date or a FP value that can be used to threshold the finds. As an example, to be a top 1000 Favorite Cache in the US, a cache must have 873 favorite points. (Likewise to be older than the 100th the cache had to be place before April 1, 1976) Examples are completely fictious but I hope illustrative. However that would eliminate the ability to provides lists of candidate caches to find to complete the challenge in the output.

Another possibility that maintains some parallelism with GetOldestHidden is to add to flags to GetFavorites.

dontCountArchivedTowardsLimit: (bool) defaults to true
dontCountDisabledTowardsLimit: (bool) defaults to false

These were added to GetOldestHidden in this thread.

As an aside, from my naive perspective, I would have expected the GetFavorites, and GetOldestHidden to be almost identical code except for the actual SQL query which could be passed in as a string. Really the only change I would expect is the sort order. Having more parallelism would hopefully give you less code to maintain.

Are there issues still to discuss?

There is a checker waiting for the 100 oldest so the stopgaps would be appreciated and we can deal with the bigger issues later.

sloth96 Wrote:
-------------------------------------------------------
> Are there issues still to discuss?
>
> There is a checker waiting for the 100 oldest so
> the stopgaps would be appreciated and we can deal
> with the bigger issues later.

I had partly forgotten it. Not completely, but it fell deeper down in the TODO-list. The initial issue was that I went away for a week just after we spoke last.

The goal for today was to attend this and two other tasks. While one of those tasks went smoothly, the other didn't, therefore I am not sure I will manage this today. But right now tomorrow has an empty schedule, so hoping to attack it then.

I will start by re-reading it, if I have any questions I will ask them straight away. I am sorry for the wait.

Thank you for reminding me how PGC_GetOldestCaches() was solved in the end. I used a similar approach to solve the above. dontCountArchivedTowardsLimit and dontCountDisabledTowardsLimit was added.

Just as with GetOldestCaches it will figure out which is the minimum amount of FP to fit into the <limit> top list, and then return all that has at least that amount. Pretty similar to what I suggested above, but with the same approach that we ended up with last time, after several iterations. The side effect is that limit 1000 can actually return more than 1000 geocaches, if #1000 and #1000 has an equal amount of FP.

The limit has also been increased to 1000, though I am not sure if I see the need for such large selection. Maybe log top 100/1000 favorites in Germany is reasonable.

Hopefully nothing is broken, I did some testing and it has passed my tests. I will release it within an hour from now.