×

To be able to write in the forum you need to authenticate. Meanwhile it's read-only.

Re: My trail / Cache chain

My trail / Cache chain
January 10, 2018 12:49AM
I updated the script to a new version which is using considerably less memory than earlier version. I think that this version is good enough for real use. It is now capable to find the longest cache chain distance for Alamogul in 15 seconds. (Over 700km long and 78794 caches) and it is now using Vincenty for the final result.

Today I got only few error codes. After a long run I got
Unknown error (script error 5, 5a554e833cb6e4.33027560)
Then I changed to another profile name and immediatelly got the same error!
Then tried again and it worked with the other username with less finds.

Earlier today I got constantly good results when testing Alamogul but then it started to give these errors and now I get only red "Error" sign and no number or error codes at all. All field are empty.

I started to play with configurable time-out settings and there may be something. At first, when I used default 28 sec time-out value, the script gave only this red Error sign. When I adjusted time-out shorter, the script exits earlier and I found that the script thinks that it used only 15 seconds but on screen there is "Script executed in 29 seconds. Peak memory usage: 329,472 kB". I think that the time-out in my script is not working properly. Earlier I saw it trying to find the shortest route up to 29 seconds and doing a controllled exit but now it crash with 28 second setting.

Now I tried to set it to 20 seconds and it did not crash.
Area scan ready 15.21993s
The time limit has been reached!
Final 20.212645s
Script executed in 23 seconds. Peak memory usage: 329,472 kB

When trying again I managed to get results with 28 second timeout again.
Area scan ready 15.771814s
The time limit has been reached!
Final 28.364944s
Script executed in 29 seconds. Peak memory usage: 329,472 kB

It seems that when the sandbox time-out, then there is only the red Error sign and nothing else. And for some reason, sometimes it time-out too early after 20 seconds according to the OsClock.



Edited 1 time(s). Last edit at 01/11/2018 09:40AM by ganja1447. (view changes)
Re: Most impressive challenge idea?
January 10, 2018 12:57AM
I did double the RAM for the VM a few hours ago.

I got nothing interesting from that dump. I would need to add some additional logging to track it.

I will however forget it for now, I will try to implement a queue system instead so that I know how many checkers are running simultaneous. I should also make sure that one user can only run one checker at a time, as a DoS protection, that is not the case now.

I have been thinking about how to implement it properly this evening, hopefully I can fix it later this week. I have been thinking about it so much that I can't sleep. Just left the bed to do the book keeping instead of laying sleepless. Meeting with my book keeper tomorrow anyway, need to prepare some papers.
My trail / Cache chain
December 28, 2017 10:41AM
I actually have an idea that I believe is rather new. I have had the idea for 1-2 years and was working on a technical solution with Target a while back. It turned out to be quite hard to solve though, technically. It requires quite a lot of calculations and scales very badly with the number of finds. The easy implementation is to calculate the distance between all finds one has. Some optimizations to that can be made though.

I might look into that idea at some point again. I am however afraid of two things.
1) I won't be able to create it generic enough to make it work over the whole world. Instead I might need to pre-calculate a lot of data, which I will then only do for the area where it's relevant for my own challenge.
2) I won't be able to find anyone else who has completed it. However, that might be negotiable with the reviewers if I can show that a decent amount of people can complete it very easily.

In short it's about building your own trail from finds in the world (or defined area). The idea is to have two parameters, max distance between finds and minimum total distance to complete the challenge (between those that are furthest away from each other).

A theoretical example: Build a trail through Europe that's 1000 km long with max 10 km between each canister. To be clear, it's not about cache-to-cache-distance. It's about distance for those furthest from each other.



Edited 1 time(s). Last edit at 01/11/2018 09:37AM by ganja1447. (view changes)
Re: Most impressive challenge idea?
December 30, 2017 01:27AM
I have made something like this. Not excatly the same idea at all, but this also needs calculating distances between all finds.

Example: https://project-gc.com/Challenges/GC7FEN8/30511

You can define minimum and maximum distances between finds and the script scans the whole world to find an area where is enough suitable finds. Now the script counts only number of finds to qualify but anyway it can locate any chain of caches which is distributed according to those two distance paratemeters plus one unique attribute like county if only one cache per county is allowed.
Re: Most impressive challenge idea?
December 30, 2017 11:16AM
arisoft Wrote:
-------------------------------------------------------
> I have made something like this. Not excatly the
> same idea at all, but this also needs calculating
> distances between all finds.
>
> Example:
> https://project-gc.com/Challenges/GC7FEN8/30511

I like that one actually. Maybe I will use it for something in the future. The problem is that it can't manage my 17000 finds, without a filter. The FP-filter you have in that configuration seems to be enough. As a test I created a new one with 0 fp required, I got a timeout back.


For the challenge I wanted, I wanted to filter on my region. That would set me back at ~7000 finds. Though I am quite sure others have over 10k finds in that region. In 2 years from now, maybe 13k.

There are some optimizations that can be made, but we never managed to get the execution time fast enough. We (mostly Target.) spent quite a lot of time on it. His scripts still exists somewhere, not sure if I have the links/IDs still.

I remember two optimizations from the top of my head.
* We didn't calculate Haversine directly, but first a very fast approximation. If abs(latA-latB) > X and abs(lonA-lonB) > Y then continue;
* I know Target. was building clusters as well. If cache A was close enough to B, we only needed to check the distance to one of those to C for example.
The problem, in my opinion, was that the code was quite full of complex optimizations, so it was hard to read in the end. At least for me, who didn't write it.

PS! I will bring up my (non LUA) code and have a look at what I did myself.
Re: Most impressive challenge idea?
December 30, 2017 11:26AM
My script is quite slow (seconds in first column). Not sure what it does and how much I can optimize it. Here is the output:
0.000 Getting finds from the database
0.261 Number of finds: 6692
0.262 Calculate distances
306.946 Building areas
307.388 Filtering areas
Number of areas before: 359
Number of areas after: 22
307.392 Finding an area with large distance enough
Found new best distance, 0.00 km
Found new best distance, 61,515.31 km
Found new best distance, 63,695.55 km
Found new best distance, 72,244.22 km
Found new best distance, 73,245.14 km
Found new best distance, 74,219.21 km
Found new best distance, 75,285.26 km
Found new best distance, 77,840.08 km
Found new best distance, 78,179.38 km
Found new best distance, 78,254.50 km
Found new best distance, 78,287.04 km
Found new best distance, 78,406.92 km
Found new best distance, 78,476.53 km
Found new best distance, 80,714.87 km
Found new best distance, 81,061.76 km
312.124 Longest distance is 81.19 km from GC1FMF6 to GC1QPA4
312.126 Figuring out which path is the shortest
Preparing
Calculating
313.647 Shortest path has 77 jumps
313.648 Storing in VGPS
Script peaked at 3,278,848 kB memory

The script exports my trail into a Virtual GPS, here is a screenshot of a map of that VGPS.
https://1447.se/shutter/Screenshot_20171230_12%3A24%3A25_afa.png
Re: Most impressive challenge idea?
December 30, 2017 02:08PM
The link to the image doesn't work but I think that the idea and the problem is interesting. The main problem is that if it really needs calculating all distances between all caches, the number of calculations grows exponentially and finally it always exceeds the allowed time when there is enough finds, regardless of the method of calculating the distances.

In my own distribution script I started by calculating distances to a one fixed point and then used this distance to exclude caches which are too far away from each other. This is quite fast process and the runtime is mostly consumed to select a set of caches which meets the criteria.
Re: Most impressive challenge idea?
December 30, 2017 10:49PM
I tried, how fast it is possible to search caches in given range. I made a test script which tries to find all finds inside 1000m range from every find. This is something like calculating all distances between all finds but optimized to search only finds inside given radius.

https://project-gc.com/Challenges//30999

Well..

Got profile name, Alamogul
Found 2650628 caches inside 1000 m range for 177095 finds
Runtime 7.602095s

It seems to work ;-)

The next step could be using some advanced polygon geometry to find border of the largest polygon which is possible to build.
Re: Most impressive challenge idea?
December 31, 2017 09:14PM
I have studied this problem a little more. Do you mean something what is displayed in this image?
Blue lines are connecting dots within limited distance. Red line is the longest distance between connected dots.
This kind of checker will be pretty fast and can handle large amount of data.

Re: Most impressive challenge idea?
January 01, 2018 02:55PM
From studying the screenshot, it seems to be exactly what I am talking about. Love the illustration of data by the way.

What my own script does, that yours doesn't seem to do, is that it calculates distances between all nodes. The reason it does it, is because it ends with running a "shortest path" algorithm between the two nodes you have the red line between. To give some cool output in the example log and map. This is of course not necessary, and I guess it's worth letting that go if one can actually make this work with decent resources.

The output you have created can probably be created with the checkers as well actually. Not sure if there is any "line" support. If there isn't, it's easy to implement. Also, if it's not documented, it could also be that the documentation is missing, but the implementation exists. It's using the same code as PGC uses for generating maps itself if I recall correctly. So there are support for some weird stuff in some cases.


I would be amazed if "I" could finally get a challenge like this to work. If you are interested in some "real" data, my two variants I have been playing with have both had these filters:
Country: Sweden
Region: Skåne

Then I have two different setups for numbers.
1)
Max length between caches: 5 km
Min length between furthest nodes: 150 km
2)
Max length between caches: 1 mile
Min length between furthest nodes: 30 miles


I fulfill the second one myself (at least I did, some geocache might have been moved, though not likely).
Last time I checked I did not fulfill the first one. I have injected additional finds when playing around with it myself so that it was completed for me. Geocaches I would then have to log. From looking at my code I think those were: GC3WBWZ, GC5198W, GC67RWM

PS! I fixed my screenshot link. My screenshot tool that has a button to send it to a web server needed to be restarted. It failed to upload the screenshot and there is no error handling on that part. It's a bit of a hack.
Re: Most impressive challenge idea?
January 02, 2018 02:51PM
Quote

The output you have created can probably be created with the checkers as well actually. Not sure if there is any "line" support. If there isn't, it's easy to implement. Also, if it's not documented, it could also be that the documentation is missing, but the implementation exists. It's using the same code as PGC uses for generating maps itself if I recall correctly. So there are support for some weird stuff in some cases.

At this moment I don't know any working method to create graphical output or display a map from LUA checker, but this could be a good reason to implement one more output table for LUA script results.

{ ok = ok, log = log, html = html, map = list_of_caches }

The map could be subset of the original finds from PGC_GetFinds() or just a list of gccodes. If this list exists on return, the checker platform could display a map of qualified caches. That could be a much better way to display results when the output is a long list of caches. It is also easy to implement afterwards to exsiting checkers when only a list of caches is needed.

I have now a working (perl) model on my own computer. It seems to be fast enough but I will test it more using your "real" data before porting it to LUA.
Re: Most impressive challenge idea?
January 02, 2018 04:34PM
The map thingy works almost as you suggest. I thought it was documented, but after looking through the documentation I realize it's not.

The reason is that I created it as a test for the script "Generic polygon checker" by lillfiluren. By that time, it was considered beta and not only was it undocumented, it was also hard coded to only work with his script-id. Both because it was very untested, and also because it was sloppy written and didn't check for any exploit attempts, or similar. Since then, the code has been cleaned up, reviewed, and accepted by all script-ids. But never documented.

To use the map, one returns a key named map just as you suggested, though it's not only a list of caches, but an associative array of different options. The simplest example would be:
var map = { caches: [] }
{ ok = ok, log = log, html = html, map = map }

If I recall correctly, the cache list can be either gccodes or cache-ids (the ints). It can not be a combination of them, the first item in the list is checked if it's numeric or starts with GC to determine the rest of the list.

The map return also handles a few other options. Looking at the code I see it handles:
caches, outboundBox, polygons, polygonLines and circles.

The different keys:
* caches is already described above.
* outboundBox is optional. If not set, it will be determined from the caches. If defined, it's a list of [minLat, minLon, maxLat, maxLon]
* polygons looks like this (a list of polygons):
var map = { caches: [], polygons: [ {polygon: [ [lat, lon],[lat, lon],[lat, lon], ... , color: '#f30', weight: 3, opacity: 0.4, fillOpacity: 0.1, url: 'https://example.com';, 'urlText': 'foo' ] }
I have used the default values as examples, except for the polygon-lat/lon which needs to be defined. Each polygon must have at least 3 vertices.
* polygonLines
A list of lat,lon pairs
var map = caches: [], polygonLines: [ polygonLine: [ [lat, lon], [lat, lon], [lat, lon], [lat, lon] ], polygonLine: [ [lat, lon], [lat, lon] ] ]
A polygonLine can have from two vertices. No upper limit in the code, but with thousands one can expect the map to be extremely slow.
* circles
var map = { caches: [], circles: [ {latitude: 12.3456, longitude: 123.4567, color: , weight: , opacity: , fillOpacity: , centerDot true/false}, tooltip: 'text/html-string', ... ] }

Now, the above isn't really documented anywhere, so I wrote it while reading the code. High risks of not being exactly like I say, or that I messed up with {} and [].

One can also look at the source of any script with polygon in its name I guess, to get some details. Anyway. If/when you are that far, I would be happy to assist to find out why things ain't working.


In case it makes your life easier. I have put my gccodes, lat and lon on pastebin. Should be easy enough to parse with perl.
https://pastebin.com/gqGgHve9
Re: Most impressive challenge idea?
January 02, 2018 04:39PM
I know from experience that the map is getting real slow with 5000 geocaches. Even an excellent workstation has a bit of a pain in those cases. I would like to rewrite our map code and try to make it faster, but I haven't prioritized it yet.

This also tells me that it might be a pain with thousands of lines. I would suggest testing the map return with a sliced result, especially if your web browser "stops responding".

In Linux Firefox seems to be faster than Chrome these days (on PGC maps). That wasn't the case 2-3 years ago, but that's the case now. I bet it can differ for different platforms though.
Re: Most impressive challenge idea?
January 05, 2018 04:45AM
For some reason circles did not work and there is no tooltip for polygons to add more information. I used polygon url to display some info but the "one line format" is not very informative and the map disappears if one click the url. :)

Anyway... I just enabled a test script: https://project-gc.com/Challenges//31104

During testing I faced out of memory situation (> 1GB) when handling large number of finds. I managed to shrink the memory consumption a lot but I think that It may crash without any warning if tested with huge amount of finds. The problem is that there is no way to check how much memory is left and stop the script before it runs out. In the worst case it is possible to make two tags. One for normal players and an another for prefessionals with limited functionality.
Re: Most impressive challenge idea?
January 06, 2018 09:21PM
Now the checker for the longest chain https://project-gc.com/Challenges//31104 is almost ready.

There is some random phenomena which makes it crash without error messages if used with tens of thousands finds. Sometimes it works with more than 100,000 finds and then, when tried again, it crash without reason. Actually this is not a new feature. It has happened also with other checkers for a while. It relates somehow to memory consumption.
Re: Most impressive challenge idea?
January 08, 2018 03:50PM
I must say that I am (very) impressed.

I was thinking about digging into this deeper today, but as usual something else came up or something else took more time than expected. I have been a bit off a few days, extending the Christmas holidays, but now I am back to 100% again. Just have some stuff to catch up on.

What I didn't realize at first is that you did not filter on country/region. I notice the code supports it, it's more that I am surprised it was fast enough even without that. While my own trail in Skåne is quite boring, my partner's (mimsa) is much more interesting, because it's not just a straight line. More S-shaped, it's lovely to see, those are the effects I was hoping for.

The crashes you see, are they immediate, or after a while? I am interested in knowing if it even starts processing, or it fails to contact some back-end service.

I can look into tool-tips for polygons, I assume it shouldn't be that hard to add.

I can also look into circles, maybe I sent you faulty instructions for them. Circles are supported, and here is a proof of concept: https://project-gc.com/Challenges//20717 . A challenge I am not allowed to publish with the new guidelines. I had it planned before they were published. The tag is mine, but I haven't written the script.
Re: Most impressive challenge idea?
January 09, 2018 06:20AM
I have also an another version which can process all finds from Alamogul in 15 seconds, but it does not find the shortest route yet. (The current version stops before all finds are checked) I think I can include the shortest route using an another algorithm, which is not using so much memory to speed up the process, as the current version does.

>The crashes you see, are they immediate, or after a while?
>I am interested in knowing if it even starts processing, or it fails to contact some back-end service.

This seems to be quite important issue to fix. I just finished a new script which took quite long time to build because so many test runs ended to Error 5 without an understandable reason. Seems like number of finds is the major factor. 30k finds works better than 40k finds. And 170k finds works only randomly. Sometimes it happens more frequently and later everything seems to work perfectly.

The problem which I faced today relates to time out system. The script tries to detect when it should time-out gracefully by checking OsClock difference from the beginning one. (For some reason the starting value is not near zero as it is documented.) When I tested a simple loop, which does no database functions at all, the test run seems to exit just before the sandbox times out in 30 seconds. But when getting some data first, then the results were not uniform. Error 5 happens quite often with large number of finds. In most cases this happens after long time. Sometimes I have to reload the checker page again until it runs without problems. It is possible that changing the script just before run is an important factor. Anyway, what I found was that script took very long time and then gave Error 5. Next run was succesfull in couple of seconds. Maybe it is just the database which took over 30 seconds to get any data?

I was writing this checker: https://project-gc.com/Challenges/GC7GY7A/31203
Just now it seems to work.
1. run: Script executed in 16 seconds. Peak memory usage: 534,016 kB
2. run: Script executed in 3.6 seconds. Peak memory usage: 533,760 kB

My intuition says that explanation may relate to the free memory. The limit is 1GB but sometimes all scripts behaves like the memory limit is much lower. I remember situation when already tested script started to crash and removing some fields from GetFinds or trying other users with less finds helped a lot.

Is there any possibility to get litte more information than Error 5? It would be important to separate time-out and memory-out from other type of errors.
Re: Most impressive challenge idea?
January 09, 2018 02:17PM
I don't know the exact reason for that error, it could be several cases.

I will rewrite it a bit today, so it might return something else, or it will return:
Unknown error (script error 5, {$dumpId})
That dump-id will then give me a back-trace which might help me finding the reason. Maybe not, but at least I can narrow it a bit.

I tried running your https://project-gc.com/Challenges/GC7GY7A/31203 ~30 times. I did not get a single "error 5". Once it failed to run completely though, with an instant fail.

I am of course interested to find out why things fails sometimes. It would be easier with a good reproduction case though.

It could be that the database queries takes more than 30 seconds in some cases. It shouldn't, but that's a decent theory at least.

Reloading the page should never change anything as long as there haven't been any JavaScript errors. It's just an ajax call and is not dependent on what's already loaded, as long as the user is authenticated.

I don't think it's memory related, but it could be. All user triggered (in difference to the auto-challenge-checkers running in the background) is executed in it's own virtual machine. It does not have RAM enough for a lot of simultaneous checkers. I will see if I can reproduce the error by trying to DOS it, by running many checkers at the same time.
Re: Most impressive challenge idea?
January 09, 2018 02:34PM
Quote

I don't think it's memory related, but it could be. All user triggered (in difference to the auto-challenge-checkers running in the background) is executed in it's own virtual machine. It does not have RAM enough for a lot of simultaneous checkers. I will see if I can reproduce the error by trying to DOS it, by running many checkers at the same time.

I have a feeling that in most cases just waiting a while and then trying again later helps a lot. That makes it difficult to reproduce later and also supports the idea that the number of simultaneous checkers may be one possible reason. At least it changes time to time.
Re: Most impressive challenge idea?
January 09, 2018 03:29PM
There definitely are memory issues. I will look into if I can increase the RAM available tomorrow (if the host still has more ram, or I need to move it).
Re: Most impressive challenge idea?
January 10, 2018 11:16AM
It's looking really good.

I've been thinking about this style of challenge and I'd suggest supporting a variation of 'Cross [county/region/country] type challenges.

So for instance require a East/West chain across the county of Cambridgeshire (UK) would have the requirements:
a) the first cache in the chain is in either the county of Bedfordshire or Northamptonshire
b) the last cache in the chain must be in either the county of Surfolk or Norfolk
c) the other caches in the chain must be in the county of Cambridgeshire and most be within x km of the previous and next cache in the chain
d) the chain must be at least y km long

I don't think this would fall foul of the restrictions as it isn't based on your defined regions, and I think it would allow for quite a few nice variations of the concept which aren't only based on total length.

If they do become popular it might be useful for live map (map compare etc) to have the option to plot circles round cache with arbitrary sized circles to make the planning for such trips much simpler
Re: Most impressive challenge idea?
January 10, 2018 11:28AM
mole125 Wrote:
-------------------------------------------------------
> I've been thinking about this style of challenge
> and I'd suggest supporting a variation of 'Cross
> [county/region/country] type challenges.

I like the idea. It should only make the processing easier as well I assume. A bit more code though.

I also like how one can play around with the concept to have a long distance connected between the caches, and not just a straight line between those furthest away. I chose the concept of flight distance between those furthest away to not make a power trail the easiest solution.

Regarding additional tools. A start could be that each geocache's tool-tip could link to the LiveMap with coordinates as a center for the geocache one clicked on, with a high zoom. As you know, checkers themselves can't get geocache data that one hasn't logged as it is today. Currently there are no plans to changing that. It could be solved more beautiful in that way though.
Re: Most impressive challenge idea?
January 10, 2018 11:58AM
mole125 Wrote:
> I've been thinking about this style of challenge
> and I'd suggest supporting a variation of 'Cross
> [county/region/country] type challenges.

I have also some variations in my mind. One possibility is the area covered which needs more chains to cover larger area but your idea seems to be more interesting.

> So for instance require a East/West chain across
> the county of Cambridgeshire (UK) would have the
> requirements:
> a) the first cache in the chain is in either the
> county of Bedfordshire or Northamptonshire
> b) the last cache in the chain must be in either
> the county of Surfolk or Norfolk
> c) the other caches in the chain must be in the
> county of Cambridgeshire and most be within x km
> of the previous and next cache in the chain
> d) the chain must be at least y km long

This seems quite funny idea. Limiting the starting area makes the searching faster and also may allow more stipulations to add.

> I don't think this would fall foul of the
> restrictions as it isn't based on your defined
> regions, and I think it would allow for quite a
> few nice variations of the concept which aren't
> only based on total length.

If you have too many limiting restrictions it may seem a "polygon". As counties are allowed, it is safe to say that the chain starts from county A and ends to county B. If there is no other restrictions and there is many ways to build the route, it may be accepted. One can also make a list of allowed counties between endpoints.

> If they do become popular it might be useful for
> live map (map compare etc) to have the option to
> plot circles round cache with arbitrary sized
> circles to make the planning for such trips much
> simpler

Yes. That is why I tried to add a circle to show how far is the nearest goal. The map could display nearest "fingers" from both directions to help finding the nearest route to fill the gap.

The current script is like a proof of concept that this kind of checkers are possible to get working using the current LUA platform. The map display was a nice surprise for me and almost must for this kind of challenge. Now we have only to wait until someone requests this kind of checker for a challenge cache. :D
Re: Most impressive challenge idea?
January 10, 2018 01:55PM
arisoft Wrote:
-------------------------------------------------------
> Now we have only to wait
> until someone requests this kind of checker for a
> challenge cache. :D

I already have two concepts planned which works with the current code. I could create those geocaches and add them to the queue at any time. I even have local code to find geocachers who complete them. I don't remember if it's 10+ though. But I hope at least one of my concept has that.

Right now I am rewriting the process of executing checkers to a queue system, I will try to complete that before I spend time on creating challenge descriptions.

PS! If you missed it, I linked a checker that uses circles, so they are possible to add. It's likely I didn't give you the correct instructions from the start.
Re: Most impressive challenge idea?
January 10, 2018 07:03PM
I have also two ready for submission to my local reviewer so I can you keep them as they are now plaese
Re: Most impressive challenge idea?
January 10, 2018 07:16PM
vogelbird Wrote:
-------------------------------------------------------
> I have also two ready for submission to my local
> reviewer so I can you keep them as they are now
> plaese

My intention is to keep all current features as they are now. I may try to add a circle to visualize the radius of the required span, but I will keep this script enabled and use another script for testing. This script is so tuned for mass calculations that similar checkers with a different goal is best to separate to an another script anyway.
Re: My trail / Cache chain
January 11, 2018 09:40AM
I split the original thread. By accident two of the posts which wasn't correctly threaded ended up first instead of last here. I'll let that be.
Re: My trail / Cache chain
January 11, 2018 10:07AM
"My trail" seems to be a catchy name for the script :)
Re: My trail / Cache chain
January 11, 2018 10:09AM
arisoft Wrote:
-------------------------------------------------------
> "My trail" seems to be a catchy name for the
> script :)

That was my original concept name. I never mentioned it because I didn't really love it. I was hoping for someone else to
find an alternative name.

Though I do think "My trail" is suitable for the name of a challenge, I think (geo)cache chain is a much better name for the script itself. All in all I don't care much about the name to be honest. My trail was a desperate attempt from my own part when I named my original script.

PS! I hope the checkers are working. I was going to make a release, but it failed. The JavaScript is now written with ES6, which the minifier wasn't supporting, so there was a partial release. I am working on a solution.
Re: My trail / Cache chain
January 11, 2018 12:05PM
You may be right that "The longest cache chain" is better name for the script because it tells what it is doing. :)

> PS! I hope the checkers are working.

Checker is working and I just adjusted https://project-gc.com/Challenges//30999 to the new 60 sec time-out for testing.

With the additional run-time it can now show the Alamogul's trail instead of the maximum distance only.

Quote

The distance between the farthest linked caches from GC4XARD to GC33K3Q is 717.029 km long.
The shortest route between the endpoints contains 238 caches and it is 891.546 km long.
Script executed in 34 seconds. Peak memory usage: 329,472 kB

It took over 17 seconds to find the shortest route from 78807 caches but the additional 4 seconds was enough B)
Re: My trail / Cache chain
January 11, 2018 12:18PM
I still find your script amazing.

Should I start preparing real challenges for it? I collect finds for almost 20 calendars (T2 on every date, 1.5/1.5 on every date, wheelchair on every date, and so on), but today and tomorrow i have ZERO empty spots in those calendars, so I can use my evenings to hide canisters instead. :)

PS! Queue system updated, post incoming in the proper thread.
Re: My trail / Cache chain
January 11, 2018 12:42PM
ganja1447 Wrote:
-------------------------------------------------------
> Should I start preparing real challenges for it?

Go for it!

I will try to add the needed radius circle today but it is only cosmetic change. There is no plans to change any significant part in the script which would render it useless.
Re: My trail / Cache chain
January 11, 2018 12:49PM
arisoft Wrote:
-------------------------------------------------------
> ganja1447 Wrote:
> --------------------------------------------------
> -----
> > Should I start preparing real challenges for
> it?
>
> Go for it!
>
> I will try to add the needed radius circle today
> but it is only cosmetic change. There is no plans
> to change any significant part in the script which
> would render it useless.

Tell me if you can't figure out how it works. If this concept will go "viral", I wouldn't mind spending some time to make the best of it.

We spoke about implementing a tool-tip somewhere, I should take a look at that.
Re: My trail / Cache chain
January 11, 2018 01:10PM
polygons now supports "textTooltip" as complement to url and urlText.

textTooltip has precedence. When provided, url and urlText will be ignored.
url and urlText was used to inject into a prewritten <a href="></a>.
textTooltip can only contain text, it's html-escaped, just as urlText was/is.

If you need line-breaks, I will either have to look into a special case for that, or implement html tool-tips.
The problem with html is that I don't want html from "untrusted" sources, due to XSS issues. I will then have to run it through purification code, and that can be quite expensive.

The workarounds I can think of if it's just line-breaks that is needed are:
  1. Support \n by replacing it with <br> server side.
  2. Allow (not require) textTooltip to be an array, where every elements ends up as its own <p>.
  3. Implement htmlTooltip.
Re: My trail / Cache chain
January 11, 2018 01:20PM
> The workarounds I can think of if it's just
> line-breaks that is needed are:
>
    >
  1. Support \n by replacing it with <br> server
    > side.
    >
  2. Allow (not require) textTooltip to be an
    > array, where every elements ends up as its own
    > <p>.
    >
  3. Implement htmlTooltip.
    >

To be universal I suggest to....

- Add new "popup" field which displays same popup as urlText without any link
- Allow \n for line break in tooltip, urlText and popup fields.
Re: My trail / Cache chain
January 11, 2018 01:11PM
ganja1447 Wrote:
> Tell me if you can't figure out how it works. If
> this concept will go "viral", I wouldn't mind
> spending some time to make the best of it.

The code is currently commented out in the script.

-- TableInsert(map.circles,{ latitude = item.latirude, longitude = item.longitude, radius = 10, color = "red" })

If I enable this line the result is "Error Failed to execute script 3" and nothing else. Earlier I got an empty map when tried.

> We spoke about implementing a tool-tip somewhere,
> I should take a look at that.

Currently polygons show more information when clicked. I am not sure if a tooltip is better but the main problem is that the urlText is not formatted and the useless link goes to nowhere. Line breaks "\n" and <br/> are not working and everything must be in a single line. Without the url field the urlText box is not visible at all. Formatted tooltip may be the answer if it happens not to be too agressive on the map.
Re: My trail / Cache chain
January 11, 2018 01:16PM
arisoft Wrote:
-------------------------------------------------------
> If I enable this line the result is "Error Failed
> to execute script 3" and nothing else. Earlier I
> got an empty map when tried.

"Failed to execute script 3" currently means the token (from the queue) didn't work. It shouldn't be related to your code. You probably need to reload the page after I made a release, your currently loaded JS is outdated.
Re: My trail / Cache chain
January 11, 2018 01:26PM
After reloadign it shows an empty map. You can see the results here:

https://project-gc.com/Challenges/GCZZZZZ/28776
Re: My trail / Cache chain
January 11, 2018 01:29PM
arisoft Wrote:
-------------------------------------------------------
> After reloadign it shows an empty map. You can see
> the results here:
>
> https://project-gc.com/Challenges/GCZZZZZ/28776

I ran it four times with 0 issues. You should try shift-reloading your browser. It SHOULD not be needed though.
Re: My trail / Cache chain
January 11, 2018 01:31PM
arisoft Wrote:
-------------------------------------------------------
> After reloadign it shows an empty map. You can see
> the results here:
>
> https://project-gc.com/Challenges/GCZZZZZ/28776

Ah, I misunderstood you I think. You are getting cache icons in the map and such, but no tiles from Google maps?
Re: My trail / Cache chain
January 11, 2018 01:34PM
> Ah, I misunderstood you I think. You are getting
> cache icons in the map and such, but no tiles from
> Google maps?

Yes... no red circles and no map(tiles). Can you find the reason?
Re: My trail / Cache chain
January 11, 2018 01:35PM
arisoft Wrote:
-------------------------------------------------------
> > Ah, I misunderstood you I think. You are
> getting
> > cache icons in the map and such, but no tiles
> from
> > Google maps?
>
> Yes... no red circles and no map(tiles). Can you
> find the reason?

I will try my best. Is it after adding circles? I will try to look at your LUA to see the source of the issue. The web console says "cannot read property lat of null", so I assume the circles don't have coordinates. At least not in the way the code expects it. :)
Re: My trail / Cache chain
January 11, 2018 01:37PM
I would guess the error is here:
latitude = item.latirude
latiTude, not latiRude. :)

I haven't tested it since I can't edit your source.
Re: My trail / Cache chain
January 11, 2018 01:27PM
I have implemented a htmlTooltip as well. In that you should be able to add any html that will fit the tooltip created by Leaflet.

While testing the code I have had strange issues with the checkers twice. Most likely related to the queue. I don't know why yet.
Re: My trail / Cache chain
January 11, 2018 02:09PM
ganja1447 Wrote:
-------------------------------------------------------
> I have implemented a htmlTooltip as well. In that
> you should be able to add any html that will fit
> the tooltip created by Leaflet.

Now it is working. Circles and multiline tooltips. You may consider to use the same fieldname "tooltip" for both cases (polygons and circles) as they both are using HTML and the function is the same. I think that is all what I need to tune it up later today.
Re: My trail / Cache chain
January 11, 2018 02:17PM
arisoft Wrote:
-------------------------------------------------------
> ganja1447 Wrote:
> --------------------------------------------------
> -----
> > I have implemented a htmlTooltip as well. In
> that
> > you should be able to add any html that will
> fit
> > the tooltip created by Leaflet.
>
> Now it is working. Circles and multiline tooltips.
> You may consider to use the same fieldname
> "tooltip" for both cases (polygons and circles) as
> they both are using HTML and the function is the
> same. I think that is all what I need to tune it
> up later today.

I would then want to change the circles. The problem is that it's used by code I don't control, and rather not edit (though I could).

I guess it's not thought through enough before I named it. The main reason for that is that it didn't matter if it was html or not since it was only controlled code that used it. Then I exposed it to the checker system when it had already existed for a few years. I will let it be as is for now.
Re: My trail / Cache chain
January 11, 2018 03:10PM
Idea from my girlfriend.

You earlier discussed having trails from region A to region B, or county A to county B.

Would it be feasible (in the long run) to make trail challenges that goes from A to B to C to D and so forth? Either with the the correct order required, or not.

See it as brainstorming of ideas.
Re: My trail / Cache chain
January 11, 2018 03:25PM
I think I would be interested in one that requires caches found in Germany, Denmark and Sweden with <= 10 km in between. Not sure I fulfill it myself though, high risk that I don't.

Another concept I would like is a trail between the three biggest cities (has to be counties) of Sweden. I guess it's the same code as above, but with counties instead of countries.
Re: My trail / Cache chain
January 12, 2018 01:32AM
I added the reference circle and polygon corner tooltip circles to this test script
https://project-gc.com/Challenges/GCZZZZZ/28776
You can see the red reference circle only if user has not qualified, like me :)

The major problem is that the red circle covers all tooltips on the map. You can remove the circle by unselecting "Area" from layers to get access to other tooltips. Is there some way to select correct Z-order? For example circle with no tooltip should go to the bottom. Now the circle behaves like it has tooltip even there is none.
Re: My trail / Cache chain
January 12, 2018 07:39AM
arisoft Wrote:
-------------------------------------------------------
> I added the reference circle and polygon corner
> tooltip circles to this test script
> https://project-gc.com/Challenges/GCZZZZZ/28776
> You can see the red reference circle only if user
> has not qualified, like me :)
>
> The major problem is that the red circle covers
> all tooltips on the map. You can remove the circle
> by unselecting "Area" from layers to get access to
> other tooltips. Is there some way to select
> correct Z-order? For example circle with no
> tooltip should go to the bottom. Now the circle
> behaves like it has tooltip even there is none.

Did you change the behavior? I can't manage to click the red circle at all. What does the red circle really represent? It's quite far away from the green polygon for most of the arc. I was expecting circles in the corners of the convex hull (where you have green ones).

I can't click the green circles either. Shouldn't they have the radius of separation*unitsize unit?
Re: My trail / Cache chain
January 12, 2018 07:48AM
That is the problem. The mouse cursor changes to finger when you put it inside the red circle. If you open layers selector and unselect the "Area" then you can use polygon and corner circles as usually.

The red circle visualize the 150km distance from the center of the red line. It is like the the goal, how large one needs to enlarge the trail. I was going to add tooltip to the red circle but removed it while trying to put it background.



Edited 1 time(s). Last edit at 01/12/2018 07:50AM by arisoft. (view changes)
Re: My trail / Cache chain
January 12, 2018 08:23AM
I found a solution to the primary problem. I can select the Z-order of circles. Only the tooltip of the green polygon is covered by the red circle but I can put the same tooltip text to the red circle so it is not a real problem.
Re: My trail / Cache chain
January 12, 2018 08:53AM
arisoft Wrote:
-------------------------------------------------------
> I found a solution to the primary problem. I can
> select the Z-order of circles. Only the tooltip of
> the green polygon is covered by the red circle but
> I can put the same tooltip text to the red circle
> so it is not a real problem.

Ah, I misunderstood. I thought the red circle should have a tooltip.

The problem is that they have to be plotted in the correct order. My code really doesn't support that, it always plots polygons first.

Do you need a tooltip for the red circle? Maybe the best thing would be if the red circle wasn't clickable at all? I assume it won't be clickable if a tooltip isn't provided. But if you have some useful information for it, I understand that you want it there.
Re: My trail / Cache chain
January 12, 2018 08:57AM
> Do you need a tooltip for the red circle? Maybe
> the best thing would be if the red circle wasn't
> clickable at all? I assume it won't be clickable
> if a tooltip isn't provided. But if you have some
> useful information for it, I understand that you
> want it there.

If you can disable the tooltip feature when there is no text, it would be the best option.
Re: My trail / Cache chain
January 12, 2018 09:00AM
arisoft Wrote:
-------------------------------------------------------
> > Do you need a tooltip for the red circle? Maybe
> > the best thing would be if the red circle
> wasn't
> > clickable at all? I assume it won't be
> clickable
> > if a tooltip isn't provided. But if you have
> some
> > useful information for it, I understand that
> you
> > want it there.
>
> If you can disable the tooltip feature when there
> is no text, it would be the best option.

Doesn't it disappear if you don't set tooltip for the red circle? (or set it to false, which should be default in my end).
Re: My trail / Cache chain
January 12, 2018 08:57AM
There is a potential for two red circles to overlap, isn't there? Therefore it might be better to only have the polygons clickable. Since the circle has the same information anyway (now), it's no loss.
Re: My trail / Cache chain
January 12, 2018 09:53AM
In this case there will be only one red circle when ok = false. Overlaying circles are not a problem here but anyway the "feature" is unnecessary in general.

I set tooltip = false in the current
https://project-gc.com/Challenges/GCZZZZZ/28776
It has no effect. If you can fix it easily I would like to leave it this way.

Then an another user experience problem. Links to cachepages are using target="_blank" but links from tooltips are not and the target is removed from the link if applied. I added "Show the nearest caches" to each corners of polygon and you may guess what happens :X Would it be practical to add target to every link to avoid unintented checker reruns?
Re: My trail / Cache chain
January 12, 2018 10:18AM
I will look into both matters. What I have so far...

None-clickable circles, I am not sure it's possible (or easy enough). I will read up on Leaflet which is the map engine to see what I can find. This far I haven't found any examples which aren't clickable. It seems very weird to me though. I would take it for granted that it works. I haven't spent more than a few minutes though.

Links, it's the HTML purification that removes the target. I have added a rule to allow it (target="_blank") in my development environment, but your source code doesn't have target set anymore from what I can see, I am playing with tag 28776, so I couldn't test it. I assume my patch will work though.

PS! I have added a My-trail-challenge to the queue. I wouldn't be surprised if it takes a few days to review since it's a new concept. I doubt a single reviewer will take responsibility alone, I would expect that a reviewer wants input from two others so that they agree with each other. But personally, I can't see anything that's even close to breaking the rules. In the cache description I added five screenshots of completed trails, from two completely different areas.
Re: My trail / Cache chain
January 12, 2018 10:20AM
Found a solution to make the circles not clickable. You should set fill = false. The server side code doesn't support it yet, but I will implement it now and make a release. Expect it to work within 30 minutes from now.
Re: My trail / Cache chain
January 12, 2018 11:46AM
Target attribute works now and the new script version is almost ready to replace the older one.

I tried: fillOpacity = 0.0, fill = false, tooltip = false in https://project-gc.com/Challenges/GCZZZZZ/28776
but the red circle is still stealing the polygon tooltip. I will try later today again.

I know how difficult it is to get any new concept accepted. In most cases there is some guideline which can be interpreted many ways and sometimes it may be cancelled just for some subjective reasons. I don't think also that there is anything so suspicious that it would cause problems.
Re: My trail / Cache chain
January 12, 2018 11:48AM
arisoft Wrote:
-------------------------------------------------------
> I tried: fillOpacity = 0.0, fill = false,
> tooltip = false
in
> https://project-gc.com/Challenges/GCZZZZZ/28776
> but the red circle is still stealing the polygon
> tooltip. I will try later today again.

I just refreshed your test script and noticed it was enabled again. I will use that to debug why the fill: false isn't working. Could be that I lose that option somewhere on the way. I'll get back to you.
Re: My trail / Cache chain
January 12, 2018 11:52AM
Silly code error. Since it's a mix of languages (PHP and JS) it's hard to read sometimes.

I first have a boolean in PHP which is true/false. I then need to convert that into the string 'true'/'false' to add it in the inline JS. But by accident, I added quotes in JS so that it ended up as:
fill: 'false',
instead of
fill: false

It's working in the development environment now, I will push yet another release.
Re: My trail / Cache chain
January 12, 2018 01:39PM
About the "Show the nearest caches" link. Maybe it's a difference of opinion (I am quite colored in the case), but as a geocacher I would prefer that the link was
https://project-gc.com/LiveMap/#c=60.445117,25.82675;z=15
instead of
https://www.geocaching.com/play/search/@60.445117,25.82675?origin=60.445117,25.82675

The z-value could be tweaked depending on the separation value as well, but that's just a bonus.
Re: My trail / Cache chain
January 13, 2018 02:43AM
ganja1447 Wrote:
-------------------------------------------------------
> About the "Show the nearest caches" link. Maybe
> it's a difference of opinion (I am quite colored
> in the case), but as a geocacher I would prefer
> that the link was
> https://project-gc.com/LiveMap/#c=60.445117,25.826
> 75;z=15
> instead of
> https://www.geocaching.com/play/search/@60.445117,
> 25.82675?origin=60.445117,25.82675
>
> The z-value could be tweaked depending on the
> separation value as well, but that's just a bonus.

I have updated the production script with some cosmetic changes as circles are working now perfectly.

Now there are map links both at corners and polygons. The later was not compatible enough with PGC map so I decided to add links to both maps with a tweaked z-value as you suggested.

The longest (final) route is always displayed regardles of smallest visible polygon size parameter, if there is reasonable amount of caches in this chain, to be useful for shorter chains of unqualified players. Polygons can also be disabled by using a huge limit for them.

The script also supports the new cli parameter which will have great effect to runtime. (tu)
Re: My trail / Cache chain
January 13, 2018 08:58AM
My first "My trail" has been published. https://www.geocaching.com/geocache/GC7H2BE_challenge-my-trail-skane-301-mi Cache description is in English.
Re: My trail / Cache chain
January 13, 2018 09:34AM
Got a bug report from a user. Maybe it's a new issue from the latest changes? The user is OlaHansson.

His log on the matter: http://coord.info/GLTXQP54, in Swedish, but it includes some data.

There was a dump ID, and I looked up the LUA error in it:
(
    [lineNo] => 310
    [error] => bad argument #1 to 'TableInsert' (table expected, got nil)
)
Re: My trail / Cache chain
January 13, 2018 10:34AM
It is now fixed. The bug was there from the beginning. When two calculated total distances were exactly the same, then the sort order gone wrong way. simple ">=" to ">" corrected it. I have tested this part many times with asserts, because it is known to be the most difficult programming task in the universe, but apparently never got two same distances at the same time before this.



Edited 1 time(s). Last edit at 01/13/2018 10:35AM by arisoft. (view changes)
Re: My trail / Cache chain
January 13, 2018 10:36AM
Thanks. You are super awesome. My new second favorite person in the world (only my girlfriend beats you). :)
Re: My trail / Cache chain
January 13, 2018 04:02PM
Thank you for the compliment ;)

The challenge seems to have received quite a good reception. (tu)
Re: My trail / Cache chain
January 14, 2018 07:33PM
I started planning a challenge for mystery caches. I found one a little bit dangerous feature. The map shows caches at the corrected coordinates and if someone takes a screenshot, it may spoil where caches are located. I think that it would be safest if the default setting for checker map would show caches at the posted coordinates.



Edited 1 time(s). Last edit at 01/14/2018 07:35PM by arisoft. (view changes)
Re: My trail / Cache chain
January 16, 2018 09:53AM
arisoft Wrote:
-------------------------------------------------------
> I started planning a challenge for mystery caches.
> I found one a little bit dangerous feature. The
> map shows caches at the corrected coordinates and
> if someone takes a screenshot, it may spoil where
> caches are located. I think that it would be
> safest if the default setting for checker map
> would show caches at the posted coordinates.

I had already realized this issue. But I saw it as a none issue. As long as the trail is long enough, the zoom will be so far that you can't really see where it is anyway. It's also likely you don't know which mystery it is.

But I will start to implement something against it in a few minutes.
Re: My trail / Cache chain
January 16, 2018 10:08AM
You should set map.correctedCoordinatesInUse = false.

It won't work yet, I have some implementations to do before it gets passed along properly. It will change the default setting of using corrected coordinates to off. It will also make the trail maps look much better.



Edited 1 time(s). Last edit at 01/16/2018 10:08AM by ganja1447. (view changes)
Re: My trail / Cache chain
January 16, 2018 03:46PM
I added the setting to the script but I am not sure is it working yet.

The map shows caches in the posted coordinates IF there is no coordinates in user notes. Some caches, which has coordinates in my user notes (green check mark on the left corner) , stays in those user coordinates until I click the corrected coordinates option twice. Caches with only the pencil symbol will stay in the posted coordinates all the time.
Re: My trail / Cache chain
January 16, 2018 05:27PM
I haven't released the patch yet, but it's ready for next release. I tested it now with your changes, seems to work fine, hope I didn't break anything else. :)

Leaving home in 20 minutes, but I'll try to get a release before that.
Re: My trail / Cache chain
January 25, 2018 11:55AM
I now have 3 challenges named "My trail" running. The last one isn't even FTFd yet.
  1. Challenge - My trail, Skåne 30:1 mi
  2. Challenge - My trail, Skåne 150:5 km
  3. Challenge - My trail, 500:20 km

I still love the concept and feel that I could make an endless of variants. I don't want to wash it out though. It shouldn't be too much.

What I would like, is two variations that isn't supported. I also completely understand if it's not arisoft's highest priority in life. I am super grateful what he managed this far. But I will at least present my ideas.

The first idea we have already talked about. I would like to be able to require the trail to pass through defined areas. For example a trail that includes (country) Sweden, Denmark and Germany. Or a trail that includes (county) Stockholm, Gothenburg, Malmö (Sweden's three largest cities).

My other idea is to measure the area of the green polygons instead of the distance between two points. I assume this is a fairly large rewrite since the presentation would have to be very different as well. It should probably show all the caches building the convex hull and also how they are connected. I do think this would be an interesting challenge though, in the end one could make a challenge to "connect 50% of the area of a country" for example.


From what I can see I am the only one who has published these challenges this far. If I recall correctly vogelbird had 1-2 in queue. I will ask him what happened to those.



Edited 1 time(s). Last edit at 01/25/2018 11:57AM by ganja1447. (view changes)
Re: My trail / Cache chain
January 25, 2018 12:11PM
> My other idea is to measure the area of the green polygons instead of the distance between two points.

This one is particulary in my mind too. It is easy to add depending on how exact the measurement should be. The size of the polygon is possible to add also to the current version as additional information. It would be some kind of territory challenge and gives many possibilities to enlarge the area.

The easiest way to determine the area is just using plane coordinates forgetting that the area is on the spheric surface. Results are slightly too small but the calculation is super fast.

This is already on my todo list.

The another idea of selected routes is also possible but it needs a new script and a good plan what features it should have. The easiest case is to find a route from one site to an another. Maybe all configurations could be achieved by using this primary function once or multiple times?
Re: My trail / Cache chain
January 26, 2018 02:42PM
This is starting to get similar to the other thread for 10% of the surface of the globe.

https://project-gc.com/forum/read?4,15439
Re: My trail / Cache chain
January 26, 2018 09:07PM
Indeed, that was on my mind. It is much easier to estimate area on small polygons than very large ones. Of course there are tools to calculate area correctly in both cases but the math library is quite complex. For example this javascript library can do it: https://geographiclib.sourceforge.io/scripts/geographiclib.min.js
Re: My trail / Cache chain
January 28, 2018 09:27PM
I have updated the scipt. Now it can calculate the area and perimeter of each green polygon. Area is calculated using Vincenty's and Heron's formulas. The result is good enough for comparison of reasonable size areas but gives noticeably wrong results when used with huge areas like 10% from surface of the earth. There is a better formula available but the length of the formula is longer than the script itself >:D<

Example checker is available here https://project-gc.com/Challenges//30999



Edited 1 time(s). Last edit at 01/28/2018 09:56PM by arisoft. (view changes)
Re: My trail / Cache chain
January 29, 2018 08:11AM
arisoft Wrote:
-------------------------------------------------------
> I have updated the scipt. Now it can calculate the
> area and perimeter of each green polygon. Area is
> calculated using Vincenty's and Heron's formulas.
Wow, you are awesome, to say the least.

> The result is good enough for comparison of
> reasonable size areas but gives noticeably wrong
> results when used with huge areas like 10% from
> surface of the earth.
Understandable, and in my opinion a good compromise. I don't need it, but in case you haven't tested, you should know if it works between longitude -180 and +180. My guess is that it will be an issue. On the other hand, the case is probably so rare, and hard to solve, that it should be considered a compromise.

> There is a better formula
> available but the length of the formula is longer
> than the script itself >:D<
LOL! :)

PS! My biggest area was surprisingly small with your test-tag.



Edited 1 time(s). Last edit at 01/29/2018 08:12AM by ganja1447. (view changes)
Re: My trail / Cache chain
January 29, 2018 09:41AM
>Understandable, and in my opinion a good compromise. I don't need it, but in case you haven't tested, you should know if it
>works between longitude -180 and +180. My guess is that it will be an issue. On the other hand, the case is probably so
>rare, and hard to solve, that it should be considered a compromise.

I know this limit. The same limit may be in Ground Speak's way to display farthest caches in west and east. (I saw someone to ask about this problem lately) The algorithm which is searching the nearest caches could be easily converted to skip over the 180 degrees but the convex polygon search would be hard or impossible, because in that case the convex polygon goes always around the globe. This was discussed also in the another thread, how to calculate area covered by the farthest finds. Fortunately the 180 degrees meridian line is located in Pacific Ocean and the cache saturation in that area will be minimal :)



Edited 2 time(s). Last edit at 01/29/2018 11:11AM by arisoft. (view changes)
Re: My trail / Cache chain
January 29, 2018 02:10PM
Found another issue. It's something I didn't think about myself. Maybe you did.

I created a tag for logging ~90% of the area of Skåne (Sweden). Skåne is 10939 km2 according to Wikipedia, so I rounded it 10000 km2 needed.

The result was this image:

As you can see I have logged ~13500 km2 (of 11000 possible). It didn't take long time to realize it's because the sea is included, and also some of the neighboring regions.

What we could do, if we want to solve this, is to build a new API method to which you send your polygons. The API could then make an intersect between the polygon and the defined area and return the amount of km2 back to you. Probably a more exact number as well, and probably faster to calculate (PostGIS) since it's more optimized, and in C/C++.

In theory I could return the polygon as well, but it's likely to have a million vertices, so it's probably not worth it, would be to expensive to plot it. Any thoughts? Worth spending time on?
Re: My trail / Cache chain
January 29, 2018 02:35PM
Det finns för övrigt redan PGC_GetSimplifiedRegionPolygons och PGC_GetSimplifiedCountyPolygons om man vill rita ungefärliga polygoner. Osäker på hur tunga de är dock, de är "simplified", dvs har färre vertices än vad PGC normalt hanterar.
Re: My trail / Cache chain
January 29, 2018 03:08PM
ganja1447 Wrote:
-------------------------------------------------------
> Found another issue. It's something I didn't think
> about myself. Maybe you did.

I have not thought about limiting the area with other means than the chain itself. What I have though is a method, how to add some concave parts to the polygon, but here things are more or less about pattern recognition and goes too complicated.

> In theory I could return the polygon as well, but
> it's likely to have a million vertices, so it's
> probably not worth it, would be to expensive to
> plot it. Any thoughts? Worth spending time on?

In this particular case it could be used, but generally, to qualify to a challenge, you practically need an arbitrary limit. If it happens that the limit is too low, as it is this case, you just rise the limit until it is appropriate.



Edited 1 time(s). Last edit at 01/29/2018 03:10PM by arisoft. (view changes)
Re: My trail / Cache chain
January 29, 2018 03:32PM
For example, if you calculate the convex area of Skåne and use 90% of this calculated area as the required area, then it is relatively 90% of the total area and can go up to 100% if every corner of Skåne is found.
Re: My trail / Cache chain
January 29, 2018 03:36PM
Yes, I understand the issues with the convex hull. That I get more area than I should. But it felt wrong to get areas that actually were outside Skåne itself, when it was defined in the conf.

It would of course be even better if something better than a convex hull was used. Due to the lack of university education I am not familiar with any formulas for it either.
Re: My trail / Cache chain
January 29, 2018 03:42PM
In theory I guess we could use PostGIS's Concave Hull for the purpose. As long as you have calculated the cache chains, we could through the waypoints at PostGIS to get a hull back. With the risk of it getting quite complex.
Re: My trail / Cache chain
January 29, 2018 03:58PM
Using a concave hull would also give a nicer presentation. I find the current area presentation quite dull actually. It's hard to see how it's connected. No offense meant, I am still impressed. I am just trying to be constructive and see it from the geocachers perspective.

I don't have much more computer time right now, otherwise I would be interested in implementing something that took a few thousand waypoints and delivered a concave hull back. Then the script would have to look up which geocaches matches those coordinates to plot them, and for drawing circles at them.

It sounds like it's starting to make the script glide further away from the original, and maybe a fork should be considered?

I am just presenting my thoughts, it's all up to you. :)
Re: My trail / Cache chain
January 29, 2018 04:11PM
ganja1447 Wrote:
-------------------------------------------------------
> Using a concave hull would also give a nicer
> presentation. I find the current area presentation
> quite dull actually. It's hard to see how it's
> connected. No offense meant, I am still impressed.
> I am just trying to be constructive and see it
> from the geocachers perspective.

Indeed. Using convex is just the choise for the original problem.

Convex polygon is good for area challenges when the idea is to "conquer" the area using orthogonal chains. Maximum area with minimum caches.

Concave polygon is basically not an unique property. As you see from the link you offered earlier, you have to adjust the process to create the best looking result. I could add a method which draws the concave polygon using the original separation limit. In that case the polygon follows the outer edge of the chain itself. In that case the area of the polygon will be quite small, unless you have some large loops which may be even empty inside. I have no idea how this will look like so it may be interesting to try (:P)
Re: My trail / Cache chain
January 29, 2018 04:16PM
I think I expected a polygon along the cache edges, and the holes in the middle to be ignored (considered to be included in the area), which was also the default of the PostGIS function.

I bet it's much harder to fill something like 90% of a region that way, but it would also make it more interesting.

Another up-side is that one probably don't need to intersect polygons for a defined area. Using a concave hull will make it stay within the defined area, or at least close enough for the extra area stolen from the outside to be insignificant.

I would love to see such version. Hopefully it would give a result looking like I expect, but there is a risk that it would feel unusable as well. I will try to look at my own finds in Skåne just to imagine how the result might look.
Re: My trail / Cache chain
January 29, 2018 04:24PM
I plotted all my finds in Skåne, and added a 10 km measured line on the right side (in the sea, close to the mouse cursor) as reference.


It seems to me like I could get fairly close to filling most of Skåne with 10 km seperation. At least I have most of the coast covered. I am not all the way up to the northern borders though.
Re: My trail / Cache chain
January 29, 2018 04:32PM
ganja1447 Wrote:
-------------------------------------------------------
> I plotted all my finds in Skåne, and added a 10
> km measured line on the right side (in the sea,
> close to the mouse cursor) as reference.
>
>
> It seems to me like I could get fairly close to
> filling most of Skåne with 10 km seperation. At
> least I have most of the coast covered. I am not
> all the way up to the northern borders though.

Looking at your map it seems that this may work. At least it is a good challenge to fill all gaps :)
Re: My trail / Cache chain
January 29, 2018 03:34PM
Meanwhile I added an API method that can be used if the area is filtered by the conf.

I added this snippet into your script:
if conf.filter.country ~= nil then
          if conf.filter.county ~= nil then
            area = PGC_GetIntersectedPolygonSize(polygon, {country=conf.filter.country,county=conf.filter.county})
          elseif conf.filter.region ~= nil then
            area = PGC_GetIntersectedPolygonSize(polygon, {country=conf.filter.country,region=conf.filter.region})
          end
        end
Just before
local tip = "This chain contains "..#points.." caches in "..measure(area,2).." area<br/>There are "..#hull.." caches around "..measure(perimeter).." long perimeter<br/>Maximum distance is "..measure(maxdistance).." long<br/>"
around line 500.
Re: My trail / Cache chain
February 02, 2018 06:24PM
I have been working on the concave polygon problem. It is "little bit" harder than the convex one but now it seems to be working as my local test script shows correct results.

I have planned to replace or make an alternative for the previous green convex polygon with a concave type polygon. The polygon follows the chain around the group just the same way as the shortest route does thru the area and it does not have large extra areas between corners as the convex have. It will take for a while to port it to LUA but I think it can be made.
Re: My trail / Cache chain
February 02, 2018 06:45PM
arisoft Wrote:
-------------------------------------------------------
> I have been working on the concave polygon
> problem. It is "little bit" harder than the convex
> one but now it seems to be working as my local
> test script shows correct results.
>
> I have planned to replace or make an alternative
> for the previous green convex polygon with a
> concave type polygon. The polygon follows the
> chain around the group just the same way as the
> shortest route does thru the area and it does not
> have large extra areas between corners as the
> convex have. It will take for a while to port it
> to LUA but I think it can be made.

Sounds very promising, I hardly can't wait. :)
Which language do you use for your initial hacks/testing?
Re: My trail / Cache chain
February 02, 2018 06:51PM
I am using Perl for testing ideas. Porting to LUA is quite easy. Just remove all punctuation characters :P
Re: My trail / Cache chain
February 04, 2018 01:42AM
Re: My trail / Cache chain
February 04, 2018 06:40PM
arisoft Wrote:
-------------------------------------------------------
> Sneak preview
> https://project-gc.com/Challenges//32341

I love it. Played with it this morning, then I had an event I was lecturing at. Got home again a few hours ago, have eaten dinner and played with it even more.

Anything specific that makes it not production ready yet? Any known issues? Looking forward to adding a challenge to the queue. I might have to hide more canisters in my challenge woods though. Running out of unused canisters out there. :)
Re: My trail / Cache chain
February 04, 2018 07:18PM
ganja1447 Wrote:
-------------------------------------------------------
> arisoft Wrote:
> --------------------------------------------------
> Anything specific that makes it not production
> ready yet? Any known issues? Looking forward to
> adding a challenge to the queue. I might have to
> hide more canisters in my challenge woods though.
> Running out of unused canisters out there. :)

There may be some issues left with the new feature depending on conditions (user-id, number of finds) but I updated the production script and made this https://project-gc.com/Challenges//31104 tag for evaluation.

New feature in the tag is just the parameter: "polygon":"concave"





Edited 1 time(s). Last edit at 02/04/2018 07:28PM by arisoft. (view changes)
Re: My trail / Cache chain
February 04, 2018 09:55PM
Would you find it reasonable, or would you say that it might be a bug, that the following tag runs in 4 seconds:
{ "unit": "km", "unitsize": 1000, "separation": 3, "required": 1000, "displayed": 1000, "property": "area", "polygon": "concave" }
But the following doesn't finish in 60 seconds?
{ "unit": "km", "unitsize": 1000, "separation": 2, "required": 1000, "displayed": 1000, "property": "area", "polygon": "concave" }

The above was tested several times on my own username.


Since I was curious if it was a scaling issue I wanted to test with 2.5 km and so on. I then realized that I actually can.
The following tag runs in 4 seconds as well:
{ "unit": "m", "unitsize": 1, "separation": 2400, "required": 1000000000, "displayed": 1000000000, "property": "area", "polygon": "concave" }
And this one doesn't complete in 60 seconds:
{ "unit": "m", "unitsize": 1, "separation": 2380, "required": 1000000000, "displayed": 1000000000, "property": "area", "polygon": "concave" }
This however completed in 42 seconds:
{ "unit": "m", "unitsize": 1, "separation": 2390, "required": 1000000000, "displayed": 1000000000, "property": "area", "polygon": "concave" }

Further separation:2389 doesn't complete, while 2391 finishes in 4 seconds.

It's hard to make an assumption in my opinion. It all ends up with 2389 doesn't work, 2390 is slow, 2391+ is fast. It's an extreme difference for a very small distance difference. My impulsive guess is that it wouldn't be a bug, but I have a hard time to realize why it would be such huge difference in processing.

EDIT: Realized now that I could use floats for separation and didn't need to change unit size so that it became unreadable. :)



Edited 1 time(s). Last edit at 02/04/2018 09:59PM by ganja1447. (view changes)
Re: My trail / Cache chain
February 04, 2018 10:05PM
> But the following doesn't finish in 60 seconds?

This is the problem I found also myself. There may be possibility that the program is in endless loop or there is just too many caches. Result is, at least, depending on the user-id.
Re: My trail / Cache chain
February 06, 2018 07:03PM
There were two similar cases where the concave algorithm went to a endless loop due to numerical instability issues. I was fearing something like this but did not know what kind of situation it will come up with. In this case a straight line of caches could trigger this. In your example, mystery caches in Gullringsbo put it to endless loop.

I tweaked the algorithm to avoid these situations but there is some more. Now all of your example tags seems to run in couple of seconds when using your id but my own id goes to the loop this time >:D<

While debugging this I really missed the debug log output. Is there any possibility to get the output saved?

PS. Now it is working also with my ID. At this moment I do not know any open issues.



Edited 1 time(s). Last edit at 02/06/2018 07:48PM by arisoft. (view changes)
Re: My trail / Cache chain
February 07, 2018 06:13PM
I have now located and fixed some more concave polygon issues by using the new log tool but still trying to validate all possible situations.

When everything seems to be in good order I got an unexpected error message:

Error message Unknown error (script error 6, 5a7b3b5086e9a3.81741977)
Using token: dc3a24f7d9e8f5b97b3d6b45a378d378

Fortunately I had the "nonvolatile" logging on and I got this real-time log!
https://project-gc.com/Challenges?viewDebugLog=dc3a24f7d9e8f5b97b3d6b45a378d378

The log shows that script was ready in 32 seconds. The last text is just before the exit.

Without any modifications I tried again and then it worked:
Script executed in 34 seconds. Peak memory usage: 374,528 kB

This is a good example how the new log helps a lot.
Re: My trail / Cache chain
February 07, 2018 06:37PM
I'll release an update of the web in a minute. There are two new changes that are relevant.
1) A fix for PHP warnings for the new Log() function. I tested it in DEV and it seems to work. But in case it stops working the coming minutes, you know who to blame. If so, tell me.
2) More debug regarding the error 6. There are pretty much 6 different if-cases which can end with that error. I have added some additional data to the dump-id so that I can see where it comes from. So the next time you get an error 6 and don't feel you know why, please inform me of that dump-id so that I can resolve it into data.

The code to return errors from LUA is a bit messy. The problem is that errors can come from several different sources and the code might have been modified a bit too many times without a proper cleanup.
Re: My trail / Cache chain
February 07, 2018 06:48PM
The log is now more messy than before. (:P)

https://project-gc.com/Challenges?viewDebugLog=d9df8925486b38190fc47ffd179337e0

Newlines and also some data is missing.
It is supposed to look like this:

Got profile name, seping
Discarding chains with less than 40 caches
Excluding, GC0
Considering, 103948 caches
Database ready 2.513s
0 / 37.486984s
1000 / 37.457135s
2000 / 37.436533s
3000 / 37.387234s
4000 / 37.360422s
5000 / 37.346204s
6000 / 37.334395s
7000 / 37.322321s
8000 / 37.297005s
9000 / 37.258232s
10000 / 37.22764s
11000 / 37.20764s
12000 / 37.170672s
13000 / 37.134552s
14000 / 37.104205s
15000 / 37.094758s
16000 / 37.084733s
17000 / 37.074554s
18000 / 37.063865s
19000 / 37.046595s
20000 / 37.019358s
21000 / 37.007005s
22000 / 36.998408s
23000 / 36.980046s
Number of chains located 329
Tree scan ready 4.529s
GC1CXEF/6748 4.529s
GC1CB9/6346 5.060s
GC4AY57/5498 6.992s
GC2TMWB/5409 7.038s
GC14XZV/4171 8.211s
GCPYR0/3497 8.417s
GC152DH/3160 9.033s
GC5M1DC/1638 9.181s
GC3VN01/1315 9.491s
GC60HVG/1303 9.760s
GC4RA2M/1069 10.027s
GCWFDF/1026 10.061s
GC1A1NF/986 10.113s
GC1F8D1/977 10.144s
GCKZXE/964 10.383s
GC1C99V/947 10.429s
GC6338/934 10.807s
GC44A2P/843 10.841s
GC3XGK9/763 10.995s
GCQCNC/760 11.058s
GC3WEGG/732 11.087s
GC2PV9C/688 11.224s
GCC94A/638 11.382s
GC17KMN/629 11.410s
GC1CQA0/621 11.484s
GC6XP2Y/603 11.514s
GC52V57/598 11.549s
GC2T8MM/597 12.065s
GC3RZRA/585 12.140s
GC3W151/563 12.250s
GC350B1/555 12.305s
GC4CW9V/548 12.338s
GC69E2G/500 12.436s
GC3YY1T/472 12.459s
GC56AAP/465 12.551s
GC6XHWC/453 12.573s
GC5JF9D/447 12.639s
GC11RE2/442 12.801s
GC3DGH6/422 12.833s
GC1K1Y6/421 12.882s
GC34MAJ/419 12.910s
GC58JHE/416 13.037s
GC1H5MZ/400 13.092s
GC5026Z/397 13.104s
GC1JJ21/396 13.128s
GC2DEJ3/396 13.153s
GC1YWPK/372 13.209s
GC5Y229/372 13.289s
GC5853F/346 13.317s
GCHRVV/345 13.327s
GC2PJVZ/330 13.372s
GC5F54Z/325 13.456s
GC6558W/325 13.485s
GC24FAK/312 13.548s
GC4KGRP/300 13.563s
GC11H0K/297 13.590s
GC1GM9A/292 13.606s
GC6Q5H5/292 13.625s
GC4HEZK/292 13.638s
GC3Q7M2/287 13.661s
GC2M52T/273 13.695s
GC1K13T/271 13.768s
GC2DX37/264 13.792s
GC2W8YB/260 13.883s
GC4RB92/259 13.905s
GC2A5CV/244 13.931s
GC2J6EW/235 13.947s
GC3J4RD/235 13.966s
GC3HHHE/233 13.999s
GC2AYA1/229 14.018s
GC2TVGF/225 14.144s
GC4QTEY/221 14.161s
GC1NJBF/219 14.211s
GC17K6X/218 14.225s
GC47XDB/218 14.235s
GC4BWMP/214 14.275s
GC27T0Z/212 14.294s
GC2T67Q/212 14.327s
GC5JQM6/207 14.338s
GC179RB/205 14.389s
GCQXCR/205 14.402s
GC3WFBN/203 14.429s
GC4JBM0/202 14.470s
GC5K8C5/200 14.484s
GC3680T/199 14.508s
GC6BFC3/196 14.531s
GC2KCMG/196 14.551s
GC13FE1/193 14.571s
GC36FJH/184 14.604s
GC3Z3ZN/181 14.652s
GC3FZTF/176 14.688s
GC1WDFK/175 14.721s
GC2JDMK/174 14.779s
GC10WEH/174 14.836s
GC63DB/172 14.857s
GC35AXD/167 14.868s
GC180V4/165 14.874s
GC20GR8/161 14.887s
GC6KFMD/159 14.913s
GC44CGR/158 14.933s
GC58ZEY/157 14.966s
GCN2GF/150 15.035s
GC2J3BR/150 15.071s
GC3K601/150 15.100s
GC579XQ/149 15.135s
GC23J8K/148 15.164s
GC69W16/148 15.182s
GC470VY/147 15.195s
GC1W3WA/146 15.208s
GC1GX1D/143 15.225s
GC8C38/141 15.244s
GC11H6C/140 15.260s
GC293JC/138 15.278s
GC2J3YP/137 15.289s
GC5NQPE/137 15.300s
GC272Q3/136 15.310s
GC43QRE/136 15.331s
GC3XPNY/134 15.354s
GCPK35/134 15.368s
GC6MH84/133 15.393s
GC71VTD/133 15.401s
GC483W7/131 15.453s
GC28GNZ/131 15.518s
GC10YME/131 15.535s
GC1YVR5/130 15.557s
GC4C0Q7/129 15.564s
GC39157/128 15.597s
GC3VAXB/127 15.637s
GC3BQ6Y/122 15.649s
GC37FM8/122 15.669s
GC1DNJV/121 15.679s
GC1M76Z/121 15.691s
GC37H1A/120 15.695s
GC3XNCW/117 15.707s
GC3N6D7/114 15.720s
GC6XB7M/113 15.735s
GC3PAJ2/112 15.746s
GC1YJC0/112 15.764s
GC2MWP5/112 15.776s
GC266XH/111 15.804s
GC18AMD/111 15.813s
GC3BD16/111 15.835s
GC5F9PR/110 15.850s
GCB0EB/109 15.863s
GC6VGHZ/108 15.873s
GC63J2Y/107 15.885s
GC3KWH1/107 15.899s
GC3P7J9/106 15.911s
GC1WDCW/106 15.924s
GC5DBXQ/106 15.939s
GC373X8/105 15.947s
GC2RYGA/105 15.964s
GC2JRGY/104 15.977s
GC1NC67/104 15.985s
GC2BMTG/104 15.990s
GC3XG6W/102 16.006s
GC2ZCVB/102 16.016s
GC3M2ZA/101 16.030s
GCZ43V/101 16.038s
GC5Q1CE/100 16.054s
GC28CHE/100 16.065s
GC5GA4J/100 16.075s
GC4T56W/99 16.086s
GCQZ2W/98 16.094s
GC4JMXJ/98 16.102s
GC1AQR2/97 16.112s
GC1M15R/96 16.127s
GC3Y44T/96 16.147s
GC1DF60/96 16.163s
GC2QE6Q/95 16.182s
GC1PNG8/95 16.195s
GC47V1P/95 16.204s
GCP0AB/94 16.213s
GC3T2E3/94 16.223s
GC3E6YY/93 16.237s
GCHCE0/92 16.255s
GC16B3A/92 16.260s
GC2TEPN/91 16.270s
GC51PK1/91 16.279s
GC4CTHM/88 16.292s
GCA8C4/88 16.301s
GC38ZFA/88 16.315s
GC3C1ZQ/88 16.331s
GC2HDQV/87 16.339s
GC3APC3/86 16.347s
GC2Y1HW/86 16.356s
GC3591C/85 16.368s
GC46JZ5/85 16.375s
GC3E50W/84 16.380s
GC888F/84 16.398s
GC2AQEX/84 16.413s
GC60D3M/83 16.437s
GC4DJ4T/83 16.443s
GC2C6Q8/82 16.455s
GC4A6Q6/81 16.467s
GC33FKQ/81 16.473s
GC2KDVK/81 16.479s
GC50J22/80 16.485s
GC2G175/80 16.488s
GC1FTFW/79 16.502s
GC8FDD/79 16.513s
GC1YEQ2/79 16.522s
GC4BPEK/78 16.538s
GC2J2V2/78 16.558s
GCR2TE/77 16.569s
GC3NTWV/77 16.575s
GC1E5DA/75 16.585s
GC4D1B9/75 16.594s
GC4H1XP/73 16.600s
GC3W1HW/73 16.622s
GC2AAG4/72 16.625s
GC27Q0C/72 16.630s
GC4MK8J/72 16.640s
GC1HERB/72 16.652s
GC4EZYW/71 16.657s
GC4HWZ3/71 16.667s
GC1P80Q/71 16.675s
GC21GP2/71 16.682s
GC4D9JF/71 16.695s
GC6A8YN/70 16.703s
GC6QRDZ/70 16.710s
GC252VB/69 16.719s
GC3H25Z/68 16.726s
GC4YGE1/68 16.733s
GC20ZFJ/68 16.743s
GC2R2M0/68 16.748s
GC5JQRF/67 16.761s
GC355QA/67 16.774s
GC4WTD0/67 16.779s
GC5MCBC/67 16.786s
GC2ZDPC/66 16.790s
GC44A7N/66 16.804s
GC27VY9/65 16.813s
GC19J3W/65 16.821s
GC3GXNY/65 16.827s
GC2HDMX/65 16.838s
GC36JFW/64 16.844s
GC57V0R/63 16.849s
GC1HQZF/63 16.854s
GC1YKG7/63 16.862s
GC18AMJ/62 16.868s
GC4YH81/61 16.880s
GC26G74/60 16.884s
GC3F438/60 16.896s
GC54J8K/60 16.904s
GC5XB1C/60 16.911s
GC50DWB/59 16.920s
GC3Q9H8/58 16.925s
GC556PC/58 16.932s
GC3G52D/58 16.937s
GC1HQ6G/58 16.944s
GC4VMTZ/58 16.950s
GC1Q824/58 16.957s
GC17E49/57 16.963s
GC66Y8K/56 16.972s
GC3YPJV/56 16.976s
GCVYZR/56 16.984s
GC1N9TD/56 16.989s
GC5T340/55 17.002s
GC6J2VX/55 17.006s
GC24VZX/55 17.014s
GC1TG1C/55 17.033s
GC2TP1G/54 17.041s
GCKD4A/54 17.065s
GC1R9K2/53 17.070s
GC2J0R6/53 17.078s
GCW4RX/52 17.092s
GC5QMCD/52 17.097s
GC3NBZK/52 17.106s
GC3EQY6/51 17.114s
GC3X13K/51 17.122s
GC26EAC/51 17.135s
GC5VBG5/51 17.148s
GC6X5GX/51 17.167s
GC2A6A3/50 17.180s
GC18T1F/50 17.193s
GC3TV5P/50 17.203s
GC4PD0N/50 17.215s
GC482RM/50 17.227s
GC2RK6Z/50 17.239s
GC4973H/50 17.251s
GC3K4YP/50 17.257s
GC1TN08/49 17.270s
GC2ACAA/49 17.278s
GC3T64J/49 17.286s
GC5CCH1/49 17.292s
GC28NPF/49 17.300s
GC57PF8/49 17.311s
GC3JNJB/48 17.320s
GC22HM2/47 17.326s
GC5HEDP/47 17.336s
GC1ZRJX/47 17.344s
GC3DN7K/46 17.359s
GC3W5C6/46 17.377s
GC4259P/46 17.387s
GCV71M/46 17.405s
GC1NERE/46 17.415s
GCY1GN/46 17.419s
GC46QTR/46 17.425s
GC3NH2Z/45 17.431s
GC100CZ/45 17.433s
GC5NCE6/45 17.436s
GC1AFDM/45 17.443s
GC62264/45 17.450s
GC17JR1/45 17.455s
GC234P9/45 17.463s
GC15N80/45 17.467s
GC28923/45 17.473s
GC195NP/45 17.481s
GC27GTK/44 17.486s
GC14G2G/44 17.493s
GC1D2KD/43 17.502s
GC36V37/43 17.507s
GC3YXNM/43 17.514s
GC1BR6P/43 17.525s
GC2MJA9/43 17.534s
GCG9BR/43 17.552s
GC1PW0M/43 17.557s
GC547E0/42 17.565s
GC2R80C/42 17.570s
GC1HQ61/42 17.575s
GC57GC9/41 17.579s
GC42K6T/41 17.588s
GC1PC4K/41 17.595s
GC1T6V9/41 17.600s
GC12B84/40 17.603s
GC4TPF1/40 17.607s
GC2QEQX/40 17.612s
GC1REWN/40 17.617s
Polygons ready 17.624s
Routing ready 17.780s



Edited 1 time(s). Last edit at 02/07/2018 06:51PM by arisoft. (view changes)
Re: My trail / Cache chain
February 07, 2018 06:52PM
arisoft Wrote:
-------------------------------------------------------
> The log is now more messy than before. (:P)
>
> https://project-gc.com/Challenges?viewDebugLog=d9d
> f8925486b38190fc47ffd179337e0
>
> Newlines and also some data is missing.

True that now when I think again. I'll revert the change until I have time to write a better patch for it. It seemed on the warning I got like one of the arguments to PGC_Log() was a list/array, it only handles strings properly. And PHP will most likely auto typecast ints.

I have already released a reverted updated.
Re: My trail / Cache chain
February 07, 2018 06:59PM
> True that now when I think again. I'll revert the
> change until I have time to write a better patch
> for it. It seemed on the warning I got like one of
> the arguments to PGC_Log() was a list/array, it
> only handles strings properly. And PHP will most
> likely auto typecast ints.

I tried to Log (args[1].environmentSettings) ( =env ) today and it crashed without explanation in the end. This is example when the parameter is a table. Sometimes it is very useful.
Re: My trail / Cache chain
February 07, 2018 07:07PM
I will release an update that has two code paths for PGC_Log()

IF the first argument is an array. It will return that argument only as a text blob, using PHP's print_r().
ELSE, it will do as it did. Concatenate all arguments, into a string, assuming all arguments already are strings.

Example LUA code:
PGC_Log("string1\n")
PGC_Log("string2\n")
PGC_Log("string3.1","string3.2","\n")
PGC_Log(args[1])

Log result: https://project-gc.com/Challenges?viewDebugLog=6076bc23c56b2cbd341f081c32e606fd

Release live in 1-2 minutes.
Re: My trail / Cache chain
February 07, 2018 07:06PM
New Error 6 situation occurred while testing concave hulls

Error message Unknown error (script error 6, 5a7b3b5086e9a3.81741977)
Using token: a492a818e719f287266cf3e78cec2e02
https://project-gc.com/Challenges?viewDebugLog=a492a818e719f287266cf3e78cec2e02

It took long time before error. Script seems to be ok because the log is correct.
Re: My trail / Cache chain
February 07, 2018 07:22PM
It seems like it died because PGC_Log() received a list/array as an argument.
[errno] => 8
[errstr] => Array to string conversion
[errfile] => /opt/project-gc/project-gc/htdocs/include/lua_callbacks.php
[errline] => 68
This resulted in a PHP warning which should then be returned to the client. When I try to reproduce it, it looks like this:
https://1447.se/shutter/Screenshot_20180207_20%3A20%3A04_5a6.png


My added debug code doesn't work either, and I am very surprised about it. Right now I really can't see why. There is definitely something weird happening that I really can't see the reason for.

What I do notice though, is that the token you mention is not the same as in the dump. I can't explain that either right now.

I feel like I am not much of a help on this topic. :(
Re: My trail / Cache chain
February 07, 2018 07:30PM
> My added debug code doesn't work either, and I am
> very surprised about it. Right now I really can't
> see why. There is definitely something weird
> happening that I really can't see the reason for.

New try worked again.

> What I do notice though, is that the token you
> mention is not the same as in the dump. I can't
> explain that either right now.

I haven't ever mentioned that when this kind of errors happen the debug log on the page usually shows some very old output. Wrong username etc. displayed on the output. May this explain why there is no connection between results and debug logs?

I verified that the user id was "Dave w/o id" in both the log and the checker selector box. Did your dump show a different user?
Re: My trail / Cache chain
February 07, 2018 07:40PM
Dump 5a7b3b5086e9a3.81741977 has your username
So does 5a7b3b5086e9a3.81741977.

I will start with releasing an update that hides the old result when pushing the Run checker button. Maybe that will give us some clarity in the long run. At least it will be more correct and user friendly.
Re: My trail / Cache chain
February 07, 2018 07:47PM
Now I got an immediate error 6. The debug log on web page displays result from my old PGC_Log(env) testing
Got profile name, arisoft
Array
(
[cli] =>
[maxExecutionTime] => 60
[token] => 02e3ddc0c354bd4ac9f93f770135abde
[maxMemoryUsage] => 128M
)

The real user was "Baad Daata"
Error message Unknown error (script error 6, 5a7b3b5086e9a3.81741977)
Using token: 9b93b291988f5510da73a52bc572a9c0

This time the nonvolatile log was empty. Something happened before it started at all.
Re: My trail / Cache chain
February 07, 2018 07:51PM
So maybe it's as simple as that it never starts. I shall review the code to see if I can seperate connection errors from the rest.
Re: My trail / Cache chain
February 07, 2018 08:11PM
User "Aldokyl" Failed to execute script 4
This is a permanent error only with this user. Maybe some issue with the account itself.
Re: My trail / Cache chain
February 07, 2018 08:16PM
Aldokyl isn't supposed to work. I will update that error to be more clear for future use.

I won't release the change right now since it's not urgent. But the message will be updated to:
Failed to execute script (user opt-out)

The user has decided to totally opt-out from the site. He can not use the site and the name can't be used in input-boxes. Sadly HQ forced us to accept these demands from a handful of users. (~10 of over 100k users).
Re: My trail / Cache chain
February 07, 2018 08:13PM
I have made a patch that should return a specific error if the client for some reason aborts the ajax call, and another error in case of (connection) timeouts.

I have a hard time testing it though. The only way I can think of is to bring the frontend web-server server down, which will affect the Live site as well.

If the code works, Execution status should either be "Connection timeout" or "Connection aborted" in these cases.
Re: My trail / Cache chain
February 08, 2018 03:30AM
Now I have two similar tags. Only difference is cache separation 2km and 3km. When checking Alamogul, the 2km version will give Error 6 every time and 3km version works ok. The log file shows that both checkers terminated gracefully before time out. Actual runtimes are 48.791s and 32.007s. With mondou2 both of them crash without error number but correct debug logs are available. Maybe there is some (time out) problem with the map generation.

https://project-gc.com/Challenges//28776
https://project-gc.com/Challenges//32341
Re: My trail / Cache chain
February 08, 2018 04:36PM
Actually, when running it now, both tags worked "fine". Got no error 6 at all. But I noticed that they finish in ~60 seconds, which made me look in the right place, maybe.

I am quite sure I have managed to replace the faulty error 6 messages with proper timeouts now. In the development environment, no release yet. It seems to be due to timeouts though.

Even though the LUA code is finished, enough time has passed for the process to not be likely to finish, so it makes sure it returns a proper error (well it didn't, but now it should).

There will be an issue as long as the LUA script takes 60-65 seconds. It will sometimes finish, but it won't return as it should. This is more or less by design. Ultimately it should be killed at exactly 60 seconds, but that isn't how PHP works, from what I have learned. The max-execution-time is a bit flaky.

I will try to make two additional changes to see the effects of it. The first change is to try to add more allowed execution time to the process if it's only <= 5 seconds above the allowed limit. Resetting the allowed time to see if it can finish.

The next problem will be the map generation, since that might take some time. A while back, I moved the HTML purification to a separate AJAX call. First I just store the HTML from the LUA script, then the browser fetches it in a purified version. I will try to do the same change to Maps. That will make the AJAX call which called the LUA code return quite fast after the LUA is done. Then it will have to fetch any additional data afterwards, as it already does with the HTML.
Re: My trail / Cache chain
February 08, 2018 05:06PM
So, the map-code is now handled by a separate ajax call. The dinner was on the table so I didn't have the chance to make a nice looking "loading". I just added a <p>Loading map</p> meanwhile.
Re: My trail / Cache chain
February 08, 2018 05:43PM
Quote

I will try to do the same change to Maps. That will make the AJAX call which called the LUA code return quite fast after the LUA is done. Then it will have to fetch any additional data afterwards, as it already does with the HTML.

This explains a lot about the behaviour. I was trying to determine when the script shoud terminate but the extra time needed for the map generation was not constant. Now the script gives the result very fast.

I will test again all 20 users with most finds. If all of them pass the test, I will update the script.
Re: My trail / Cache chain
February 08, 2018 06:59PM
I have finally decided for 2 configurations that I want to use for future challenges. It requires that I log a few geocaches to fulfill them myself though.

Meanwhile I have been running the script a lot. If you later on need a list of users failing with certain configurations, I can probably provide that. I have a small script that runs a checker-tag on top 1000 finders in Sweden. The current version seems to fail ~30/1000. Maybe it's the cases you are already working on though.
Re: My trail / Cache chain
February 08, 2018 07:07PM
At the beginning, most problems related to straight lines of caches.
Now I am solving, many caches in the same coordinate, issue.

If you have a tool which can run many configurations in a batch, this can be very useful, after I have first solved all of these 20 users.
Re: My trail / Cache chain
February 08, 2018 09:39PM
All users I tried, passed now the test without problems and timeout timer works also pretty well now.
I updated the production script with the current version.

Example checker is here https://project-gc.com/Challenges//31104
Re: My trail / Cache chain
February 08, 2018 10:37PM
This config
{
"unit": "km",
"unitsize": 1000,
"separation": 3,
"required": 3333,
"displayed": 100,
"property": "area",
"polygon": "concave"
}

Seems to fail on 2/1000 users now.
Krösa
Robin84

I didn't look any closer,besides running the first manually to confirm it still failed. I am actually in bed. Sick with the flu this week. On the other hand, I am still thrilled about this whole thing.
Re: My trail / Cache chain
February 09, 2018 04:23PM
There was the same problematic pattern in both cases. Many caches in straight lines near Uspastorp.



It happened that 1 ≠ 1 :S and the algorithm followed a random path. Fixing this may cause a new error somewhere else.
Re: My trail / Cache chain
February 09, 2018 08:42AM
The other tag I had got 0 fails, so it seems like you have found most issues by now.
Re: My trail / Cache chain
February 09, 2018 04:52PM
That last issue was something I had already been waiting for because it is well known that comparing floats sometimes does not work how they should work.

Running test batches with different separations is very effective check. Practically only the separation value is significat factor to the algorithm because it has so great effect on resulting patterns. Too long skips and too low does not reach the problematic spot at all.
Re: My trail / Cache chain
February 10, 2018 03:31AM
There was a funny coincidence. In the previous version I changed the time-out function in the script. It happened that it can kill also the endless loop so it is no longer so endless as before. 0 fails may indicate that the time-out code works.

I found this while optimizing the code faster and it has no effect with mondou2, because the script exits every time after 50 seconds. For verifying, it is important to add parameters "scantime": 99, "routetime": 99 into the tag to detect errors.

All latest changes are now updated to the production version.
Re: My trail / Cache chain
February 13, 2018 12:24PM
The flu caught up with me in a way that all my energy was lost. But I am almost back in business now again.

I added the parameters to one of my tags and re-run it. Only 1 error in 1000 users, and that wasn't the checkers fault. For some reason one of the users has lost its username in the database, hard to give it as input to the checker then. In other words, it seems to work great.
Re: My trail / Cache chain
February 13, 2018 05:28PM
Maybe it is good time to go forward to the next idea. Chains between named areas. For example from county A to county B. Defining more complex route could be sorted out by filtering allowed counties forehand. Should the checker be more complex than a single route from A to B?
Re: My trail / Cache chain
February 26, 2018 01:24PM
I have been away for a week, in Germany, geocaching. Before I left home I added a concave-area-version to the publish queue. It was denied by the Swedish reviewers.

There denial was very vague (in my opinion), not really saying what it was against, but instead saying things like "similar challenges has been denied before" and "it's similar to defining an area".

Since I disagreed with all their arguments I tried an appeal. It failed. I am starting to agree with whoever said that it is hard to create new and innovative challenges. HQ's argument was:

Quote
HQ
First, like your reviewer, we do have some concern about whether this merits a geocaching achievement. But our greater concern is that, when our team and reviewers considered the challenge, there was near unanimous agreement that most cachers wouldn't know how to go about qualifying for the challenge. A person can use the checker to see if they've qualified. But if they don't qualify, it's very difficult to know how to advance towards qualification. A challenge \"should be simple, and easy to explain, follow and document,\" and we don't feel it's easy to follow.

I could try to argue with them saying that the checker is also a tool, but I doubt it will lead anywhere. The fact is that I increased my area from 2376.8 km2 to >3333 km2 after spending 20-30 minutes with the checker (and then a few hours in the field of course).

My challenge was named "My territory 3333:3 km" and required >=3333 km2 with a <=3 km separation. Included in the description were 8 screenshots from different areas of Sweden where it had been completed.

Right now I feel kinda hopeless to be honest. I also don't see how this challenge would be harder to understand or follow-up on than the "My trail". On the other hand, "my trail" wasn't appealed to HQ. If it was, maybe that one would have been denied as well.
Re: My trail / Cache chain
February 26, 2018 08:41PM
Interesting that HQ overruled the reviewer with totally different argument. Maybe you could try with the reviewer again because the reviewer may think that the challenge is easy enough to follow and there is just a misunderstanding because the concept is not seen before.

One explanation can be that there is expectation that there is at least theoretical possibility to verify challenge without special checker and if not, it is doomed to be too difficult to understand.
Re: My trail / Cache chain
February 27, 2018 09:47AM
The Swedish reviewer(s), they always come to a conclusion together in these cases, listed "too administrative" as well. I just didn't quote them correctly. Their list was a bit long and vague. One couldn't tell what they actually meant, more like "we don't want this".

I know the Swedish reviewers fairly well, and if HQ said no, I won't be able to push them to something. In my opinion, the Swedish reviewers usually rather says no than yes if it's even close to a gray zone.

HQ also said "when our team and reviewers considered the challenge", so I assume there has been more than the Swedish reviewers involved in discussing this.
Re: My trail / Cache chain
February 27, 2018 10:53AM
I see. When I tried earlier my "Last to find" challenge idea I got similar response. Vague statements based on first impressions not by only one reviewer but from many international ones. Sometimes it is hard to tell whether they do not like the idea or just didn't catch the idea right. However, it is unnecessary to fight windmills.
Re: My trail / Cache chain
February 27, 2018 02:33PM
Listening to a recent HQ podcast, it sounds like there is a regular meeting where a number of people discuss and consider appeals.
Re: My trail / Cache chain
March 08, 2018 07:59PM
Quote
ganja1447
I patched https://dev-01.project-gc.com/Challenges//30511 (script 2514) so that it works in the new Sandbox. What surprised me though is that it takes 8 seconds on live servers and 48 seconds on dev servers. I can not explain why.

Thanks! I just opened the dev server and tried myself. First I tried my own simple test challenge which runs under 1 second on both systems. Then I tried your https://project-gc.com/Challenges/GC7HAHJ/31360

Live:
Script executed in 14 seconds. Peak memory usage: 38,656 kB

Dev:
First run: error
Second run: Script executed in 12 seconds. Peak memory usage: 28,453 kB

Which tag and user you tried when you got 48 seconds compared to 8 seconds in live server?
Re: My trail / Cache chain
March 08, 2018 08:05PM
arisoft Wrote:
-------------------------------------------------------
>
Quote
ganja1447
> I patched
> https://dev-01.project-gc.com/Challenges//30511
> (script 2514) so that it works in the new Sandbox.
> What surprised me though is that it takes 8
> seconds on live servers and 48 seconds on dev
> servers. I can not explain why.
>
>
> Thanks! I just opened the dev server and tried
> myself. First I tried my own simple test challenge
> which runs under 1 second on both systems. Then I
> tried your
> https://project-gc.com/Challenges/GC7HAHJ/31360
>
> Live:
> Script executed in 14 seconds. Peak memory usage:
> 38,656 kB
>
> Dev:
> First run: error
> Second run: Script executed in 12 seconds. Peak
> memory usage: 28,453 kB
>
> Which tag and user you tried when you got 48
> seconds compared to 8 seconds in live server?


I tested on myself only if I recall correctly. I tried it three times with consistent results.

You can probably ignore the first error. It's an issue with the dev environment when it's not used. Some persistent connection to something that's lost.
Re: My trail / Cache chain
March 08, 2018 08:12PM
ganja1447 Wrote:
-------------------------------------------------------
> I tested on myself only if I recall correctly. I
> tried it three times with consistent results.

If you can reproduce the problem you can look at the debug output. There is detailed timing for each part of the script. It could help to locate the problem if there is only one part which took longer than normally. For example at the beginning is line:

Database ready 0.371s

Which tells how long it took to collect your finds etc.
Re: My trail / Cache chain
March 08, 2018 08:20PM
Hm, the tag I had issues to have nothing to do with the cache-chain checker.
https://dev-01.project-gc.com/Challenges//30511

It's "Script name: Cache distribution checker (by arisoft)". That's also why I didn't post it in this thread. :)

PS! Took me 52 seconds today on first try.



Edited 1 time(s). Last edit at 03/08/2018 08:21PM by ganja1447. (view changes)
Re: My trail / Cache chain
March 08, 2018 08:27PM
OK, my mistake because I have tried to fix this one earlier and this was also fixed ;)
Sorry, you do not have permission to post/reply in this forum.