Re: Lab caches

This isn't really a new method, but close enough.

I wanted to inform you all that we are working with Geocaching HQ to add Lab caches into the checkers. Once we have the data in Project-GC I see a few ways of exporting it into the checkers.

1. Injecting it into the finds-data.
I believe this is a bad idea. Since most detailed data about the finds won't exist, many checkers will break due to finding type = nil and such. Also, some checkers could potentially consider it as a new type, counting one more type found than the user actually has.

2. Adding another input method
We could easily just add a GetLabCaches() -method. It's then up to the checker to use that method and data as well, if it needs to. It would be a complement to GetFinds().
At least all checkers related to milestones and streaks should use it. Number of types in a day is discussable and probably pretty much depending on the Challenge caches. It's probably wise for the scripts to support a config option like useLabCaches: true/false.

3. Add option to GetFinds
We could add a parameter to GetFinds() to include the lab caches. The LabCaches would then be injected into the returned result, with null in all the fields where labs doesn't have data (type, size, difficulty, terrain...)


Feedback is welcome. As I see it right now, I think both #2 and #3 could be implemented. I think #1 is crap.

Replies wasn't allowed, so it was hard to write feedback. It should be fixed now.

EDIT: Replies are only allowed by script writers now, since they are the ones working with the methods.



Edited 1 time(s). Last edit at 05/09/2016 03:12PM by ganja1447. (view changes)

yes #1 will break many checkers
#3 is the most useful one when is is easy to integrate in old scripts.
#2 would be nice to have and easy to do when it is #3 with only labs and only existing fields set.
Just adding #2 will likely slowdown addition of labs to checkers when #3 cant be that hard to code amd the same merge code will likely be needed when profile stats are generated.
Handling null values in the data field is easy. You have to do it today with latitude, longitude for non premium menbers

One thing that has to be specified is the order of the lab caches because it is unknown relativ to the other caches. Adding them to the end of the day would be my idea.

When i looked at the GetFinds data field and the lab gpx i guess the following fields can be included

cache_id,gccode (calculated from cache_id It will be longer then the usual code on i have logged is id 1575770191 and that is gccode GC1R1NZ1D)
cache_name
visitdate
type (Lab Cache)
latitude, longitude, elevation,country, region, county,
hidden, last_publish_date (both the same as the publish time)
num_finds (if available)
cache_description

I noticed that labs have D and T and both are 0 the same for owner id. The field are not required in the xsd but a comment says "Difficulty and terrain are on a 1 to 5 scale. 1 being easiest. Can be incremented by .5".
I guess that developers of handheld GPS units and mobile apps have been sloppy

I am unsure of how Geocaching.com handles the order. Since it affects their milestones it's relevant, but this far I have no idea how they merge them. It could very well be that they use a full timestamp for both finds and lab caches, data we don't have as it is today (we only have dates). The exact time exists in the API, but I don't know how it's used. It's all logical if I post a log on todays date, and now. But if I post a log 5 days ago, does it use NOW()-5*86400 seconds then?

Don't expect any of the above mentioned fields to exist for lab caches. It's likely to pretty much only be a lab-cache-id that says nothing, and a visitdate. Note that we won't even have access to the name of the lab cache.

The information i mentioned above exist for lab caches.
I looked at the GPX file you download for lab caches.
The log time with hundreds of a second are in the XML file with stats you can download if you own lab caches. It also has the number of logs or TotalCompletions as it is called.
But what you get is depending on what groundspeak does.

Latitude and longitude is almost necessary when many challenges have geographical requirements and for stats.
And name to get a resonable output. Is is needed if they are used in milestones.
It would be nice if you generated a GC code from the cache_id. I often uses gccode ad the unike key for caches and they are the same value in another number format

What order groundspeek uses is a good question. The usual sort by log id is likely not used. There is no log id in the stats XML. I would as them when you talk about the api extention and use the same on pgc.
My guess is that logs are first sorted by logdate and the date the log was posted. ie the same as log id for regular cache
It looks like that when i look whew you can set milstones in gc.com stats. Bur i might be mistaken because i looged the labs the same night and the other caches later.
I also noticed that the caches in the milestones settings are in reverse order for each day. The fist logged cache is listed last

Do you know if logging a Lab cache affects geographical stats at Geocaching.com? If it doesn't then lat/lon/country isn't really relevant. I do not know myself.

They are not in the gc.com stats. I have 44 cache missing from my countries. 41 are labs and 3 are locationless virtuals.
There is no county information in the lab GPX and you dont set it in the labs builder.
But it can be created from the coordinates and the regions polygons you uses.

But they are included in findstatgen3 if you add them to the gsak database and set country.

Thank you for investigating. I have also sent the question to Geocaching HQ, together with a few others.

Since it's not affecting their statistics, we probably do not want them to affect the data for the checkers either, at least not as a normal case. I think it would be safe to assume that the Challenge Cache Owner wants Geocaching.com compliance if nothing else is specified. However, if the challenge says "Log 10 lab caches in Germany" it's another thing. (not saying that will or will not be allowed)

Good discussion.

I think #2 is best in the long term, but maybe #1 for a few weeks first so we can get a feel for the data that is coming out.

For getFinds(), I think the flag should have three states:
* Get everything.
* Get Just Lab Caches.
* Get just "regular" cache data

This would also be usable in the future if a 4th state or more becomes desirable.

I would think including lab caches *might* make a challenge easier to complete. Requiring ONLY lab caches would probably be harder and if he getFinds is flaged as such, the PGC's Challenge Difficulty rating should be bumped up a bit.

-TG

Regarding:

1. Injecting it into the finds-data.
I believe this is a bad idea. Since most detailed data about the finds won't exist, many checkers will break due to finding type = nil and such. Also, some checkers could potentially consider it as a new type, counting one more type found than the user actually has.

There are alos challenges with number of types per day where Lab-caches are accepted as a cache type.. So it would be a good idea to be able to select them in the checkers.

In many cases, including GC's statistics page, Lab Caches appears as just another cache type. Accordingly I would prefer to use option 3 - have an option on the getfinds. Lab caches can be identified by their missing information, so we can easily use the same getfinds method to deal with lab cache only challenges, or challenges that require different cache types that allow lab caches as one of those types (I have seen a number of these).

In fact, as GC shows lab caches as another cache type, they should be treated as a cache type. I would like them included in getfinds by default. However I suspect that may break some scripts that may not handle D/T/size = null.

With the release of the latest on the Geocaching Blog, https://www.geocaching.com/blog/2016/05/return-of-challenge-caches/, lab caches may not be used for Challenge caches. I agree with their reasoning, that they are too rare. Thus the entire discussion above is moot. However I would still like to see PGC importing details on lab caches if only to add their stats - another cache type, and for those who strictly keep to one find per cache, getting the find counts to match that shown on GC's site. Secondly, although they are not permitted, some old challenge caches do allow them, and it would be nice to still provide for them in challenge checkers.

You are actually missing a part, which is the very reason for this topic.

Quoting from https://www.geocaching.com/blog/2016/05/return-of-challenge-caches/

Quote
We are not permitting Lab Cache challenges because Lab Caches are temporary, are generally only available to those who attend Mega- or Giga-Events, are not associated with Found It logs, and are not completely integrated into Geocaching.com stats. However, since they are included in user profile stats for Total Finds, Longest Streak, and Finds for Each Day of the Year grid, we are making an exception to permit Lab Caches to be used as qualifiers for challenges related only to those metrics.

As you know, the checkers currently don't do what the quoted text says, but we are working on a solution regarding Lab caches together with Groundspeak.

Prior to this, we could not really say what we wanted to say, but It's mostly a matter of time and prioritizing to create the methods above. It is very likely that both #2 and #3 will be made.

Any progress on that topic? Are lab caches implemented yet?

Frohike. Wrote:
-------------------------------------------------------
> Any progress on that topic? Are lab caches
> implemented yet?


It's not implemented. By every day that passes I guess it gets less interesting since no new challenges can include lab caches. But the main reason that it's not implemented is because the original question in this thread isn't really answered. We would need to know what use cases there are, and how the methods are expected to work.

> answered. We would need to know what use cases
> there are, and how the methods are expected to
> work.

In my case the reason to have labs was to get Groud Speak compatible total number of finds which was needed to the challenge calculations. In that case a separate list of labs would be enough and easy to use. I guess that only the number of labs totally or per day is the most interesting factor.

The use case for lab caches has already been set - by geocaching.com. GC treats lab caches as "just another cache" as far as its statistics go, and that is how project-gc should treat them. Lab caches should be considered any time the type of cache is not relevant, such as streaks, caches per country, top finds per day, distance from home, longest slump etc, as this is exactly how GC treats them. Would also need to update cache types per day, as GC treats lab caches as just another type of cache (refer https://www.geocaching.com/profile/?guid=40804577-ce13-4d32-a27a-fae4778181d1 for an example).

However I agree there are problems. I see two solutions - one is to do a union set, PGC would do this for its statistics and challenge writers might do this for their challenges (if using option 2 or 3 above (Option 3 is my preference of these two)). But a union set would mean rewriting the vast majority of queries used to generate PGC's statistics.

I wonder if a better solution would be to roll the lab caches into the main cache data set. A dummy GC code would have to be added, a code that could never occur in GC - possibly using a special character. How about "#LCxxxx"? These GC codes are sometimes used in PGC as links back to GC, so maybe point them to a dummy page or simply to not hyperlink them. D/T = 0 or null, requires a rewrite of the DT based queries D/T = 0 would cause less breakages, but still lead to erroneous output. Owner is already defined, as is lat/long. Set Placed_By = Owner.

This will break some challenge checkers, but probably very few - perhaps only those involving D/T. A managed call to challenge writers to ensure that their challenges handle the D/T (by detecting D/T = 1 rather than assuming the bottom bound).

I am certainly keen to to see PGC treat lab caches the way geocaching.com does - as just another cache that gets factored into all statistics (where appropriate).

I think it would cause a lot less breakup of statistics if you would use D/T 1/1 for labcaches.
Btw programs like gsak already use 1/1 as standard for labcaches and ratings at Geocaching.com can never be lower then 1/1

For the average D/T rating setting it to 0/0 would give to much weight on lab. Caches and would place them outside the matrix.

Yes, using 1/1 would be better than 0/0 and more consistent.

Yes! Set them to 1/1! Why didn't I think of that?

Quote

The use case for lab caches has already been set - by geocaching.com. GC treats lab caches as "just another cache" as far as its statistics go, and that is how project-gc should treat them. Lab caches should be considered any time the type of cache is not relevant, such as streaks, caches per country, top finds per day, distance from home, longest slump etc, as this is exactly how GC treats them.

Guidelines reguires that "Challenge cache criteria must be based upon caches with the seeker’s logs: Found it!, Attended, Webcam Photo Taken." There is no visible logs available for labs but the number of finds are visible in the official statistics as you stated.

It depends on how the challenge is formulated. If the statistics are referred then labs should be used to make it compatible with the used and accepted criteria but there should not be a case where you have to list log entries for labs because they are not listed allowed log types.

Practically this means that only the visitdate information is meaningless for the challenge. If the challenge criteria has filters for any attributes they do not have, challenges should not pass this filter. For example terrain less than 2 is not verifiable therefore 0 or 1 is wrong default value.

includelabs = true with nil values for unknown attributes should work best and not break anything.

I agree with you. Setting difficulty/terrain to 0 or 1 wouldn't be correct. Setting it to NULL is something I would consider correct and the checker script would then have to handle it in the way the challenge expects it. Adding NULL values like that is likely to break existing scripts, or at least give unexpected results.

On the other hand, I also agree with what you are quoting.

Quote

Challenge cache criteria must be based upon caches with the seeker’s logs: Found it!, Attended, Webcam Photo Taken.

(from Geocaching.com)

I can only interpret that as finds of Lab caches should not be considered at all. At least not for challenges published under the latest guidelines. It's however not clear if the log type of a Lab cache is "Found it!" or not.

There is another interesting quote from Source of criteria, listed under the "no"-column.

Quote

Trackable, Benchmarking, Waymarking logs, or specifying Lab Cache finds. (new 2016)

This, in my opinion, says that challenges should consider lab cache finds, but the challenge can't require it to be such log. As I see it, one can require a streak where a lab cache find holds one of the dates. Or 10 cache types per day could be 9 real cache types plus a lab cache.

Another issue we see is that, found our analyzes, about 50% of the lab caches has been logged on the wrong date. This makes them useless for calculating streaks, at least in a correct way. It actually only makes them useful for number of finds challenges.

Then to be honest, something that just hit me. Making the lab cache data available to Project-GC was one of the things we wanted to work out before removing the moratorium, which pretty much says that Geocaching HQ wanted them to be considered as finds.


As a side note. From a technical viewpoint I can inform you that Lab caches are stored and handled in a separate infrastructure system at Geocaching HQ. They are not mixed into the same database tables as real geocaches. Not even in the same database as I understand it.


Any more opinion/interpretations? Otherwise we can make a write-up and ask Geocaching HQ what they actually want.