Challenge Exceptions is back (soon)

The Challenge Exceptions page was removed when the new attributes arrived. Two primary reasons.
1) The code needed to be re-adjusted to be based on the attribute instead of type+name.
2) We were hoping that HQ would handle the attribute well enough to stick to only that.

Regarding #2, it doesn't seem like it's enough and we are bring back the exception list. This has taken some time since HQ didn't add the Challenge attribute straight away, but it seems to have been done now.

We have also cleansed the list, a LOT. From 2554 listed geocaches there are now only 93. I am sure vogelbird will add a handful more very soon though, I know he has a list. We have removed all exceptions that didn't feel necessary anymore. Typically two cases:
1) Has up-votes enough to be counted as a Challenge, and the geocache has the challenge-attribute.
2) Has down-votes enough to NOT be a challenge, and doesn't have the attribute either.

We have then added a few others to the list, which we (by code, not manual review) assumed are challenges, but is missing the attribute. These have received an up-vote by the user "Project-GC.com". Hopefully that's correct, otherwise, just keep down-voting them until they are gone.

If you believe we have removed exceptions that shouldn't have been removed. Please inform me in this thread. I will then look at what happened and why. All old votes are stored in another place, so if you say that GCxxxx should have been kept on the list, I can bring back all old votes for that geocache. Maybe even for others if we can realize why it was removed when it shouldn't have been.

A list of all gc-codes that were voted before this can be found here: https://pastebin.com/bJa4134M (the url expires in 1 year)

I have some more code to go through, but I expect the system to go live in an hour.

There are still two scheduled jobs that are disabled, one of them is the one sending PMs to users to vote. I am waiting a few days to see if I get any negative feedback on my purges/cleansing.

The other job disables checker tags that it believes shouldn't exist. Right now it's matching a lot of tags. I need to figure out why.
EDIT: One reason is that those exceptions I have added myself today don't have enough votes yet.



Edited 2 time(s). Last edit at 09/11/2020 11:43AM by magma1447. (view changes)

Due to some really great users there are already zero exceptions that needs voting. So I will enable the scheduled job sending PMs when it's needed.

Also, now when the exceptions are in place I could test the scheduled job that automatically disables faulty checker tags. It found one tag, it was a crap tag on a traditional. So that seems correct.

Upon next release I will enable both scheduled jobs again.

And the link has been changed from https://project-gc.com/Tools/Challenges?moderate to https://project-gc.com/Tools/Challenges?challengeExceptions

Not sure if this is relevant to this sub-forum, but given that you spoke about automated jobs: I noticed during voting that some caches have multiple checkers. Maybe there should be a job that checks for caches with multiple checkers, since surely one or more of these checkers are actually wrong for the challenge? There could be many reasons: user cloned it themselves and/or it was rejected for publication and/or user requested a checker from forum 8 and we didn't notice there was an incorrect checker already. Not sure what should be done with the output of such a job though and/or who should be notified about this? I know there's a job that automatically disables a duplicate checker (same script and config) but maybe this needs a human touch instead?

What's done regarding multiple checkers is that the newest gets removed if they are identical. Identical means same script and a config that's identical after doing json_encode(json_decode(config)). So, just switching order of the options will make them different.

It would be hard to automatically determine which is best if there is more than one. There can also be reasons for having more than one. Maybe one has better feedback to the user when it's not fulfilled, maybe the other has a better example log. Hopefully both of them agree whether the user fulfills it or not.

I also see it like this:
* The CO has approved one checker. That shouldn't be removed without the CO agreeing to, if it's not faulty.
* Adding a better/more helpful one as an additional is okay in my opinion. But there is always a risk there is a minor error in a checker that the CO haven't had the chance to review.

In an ideal world, maybe in the future, we might only allow one. But then we need to be able to make anyone improve any checker. Think more like open source on github.

I'm not suggesting that one checker is automatically chosen and/or disabled. I'm merely suggesting that you add a job to check for multiple checkers on the same GC code and maybe notify the taggers that they exist. We could then maybe investigate as to why there are multiple checkers (sometimes it would be obvious that one checker is completely wrong if the text doesn't match the checker) and in conjunction with the CO either remove or disable the incorrect checkers, or even provide them with a better checker.

Sounds reasonable, I'll keep it in mind and see where we end up.

Just as a remark, On the old challenges there are a lot which have multiple checkers because the early scripts are not wrong but new checkers are added because of better outputs or avoiding time out problems

As example https://project-gc.com/forum/read?8,43543

I can see a lot of bonus caches and lab bonuses in the new list, is this due to the COs adding the challenge attribute and so need voting on ?

If they have the challenge attribute that's probably why. There seems to be a lot of COs adding the Challenge attribute when they shouldn't. I assume because they like to mess things up, but in some cases it might be because they don't understand better.

I am waiting on HQs answer how that should be handled. I expect them to say NM, which won't help, because the CO won't care. I am hoping they say a NA should be posted to alert the reviewers.

Yes, thank you, I have just had another look and that it is due to the presence of the attribute.
Seems a bit harsh to put a NA on a cache just because it has the wrong attribute, and it was inevitable that COs added it incorrectly, so a difficult one to answer.
The number of small (12 caches) trails I have seen now with the power trail attribute is laughable