News

We strongly recommend that you follow our Facebook page where we post about our changes more frequently. Most of the information end up here as well at a later point though.

Support

2020-01-15

 - 

A recap of 2019

Intro

As you might have noticed we didn't compile many news entry during 2019. In fact, we only wrote one. We have now been going through our logs to see what we actually did during 2019, and will try to make a short recap here.

For us it feels that we didn't make much progress to the site, compared to before. But by looking in our revision history, we can see that we did do quite a lot. The issue is that most of it was quite boring work that didn't give the end users much, besides keeping the site running.

In 2019 we had 889 commits into our version control system. In 2018 we only had 692 commits. This has been done with the same human resources, or maybe even less.

Rewrote framework code

As mentioned, most of the work was actually behind the scenes code, or background code that shouldn't be noticable.

In the beginning of 2019 (January to February) we rewrote the very core code of our framework. In the end, the visible result was similar to before. There were however two primary goals which were fulfilled.

  1. Easier future development, and more compatible code.
  2. The core code became a bit faster, requiring less resources.

New API

Geocaching HQ also had prepared a new version of their API. The API is used to communicate with their servers and fetch Geocaching data. For us, this was an enormous amount of work. It required both planning and a lot of new code. Not the least, the new data we had access to wasn't 100% compatible with our old data, so we had a lot of adjustments and compromises to do.

The fact is that all Partners to HQ had a quite long period to implement the API. It was released long before the deadline to switch to it (the shutdown of the old API). The problem was that no partner really rushed into it, leaving it quite untested.

There was also some functionality missing. Some parts were required for us to be able to keep existing, other partners had other needs. We can not speak for others, but HQ was very accommodating to us, listening to our needs and solving them.

Most of our time was spent on adapting the site towards the new API between March and June 2019. That says a bit of how much work it was for a small group as us. Added to that there has of course been many bugs that we had to solve after actually switching to the new API. In the beginning, quite a few, then fewer and fewer. The latest "new" API related bug we fixed was about two months ago.

In general we are happy with the new API though. It's more thought through, more consistent, and not the least, more stable.

Personal health

After working a lot during the spring, definitely more than full-time, we decided to take it a bit chill during the summer. That's when it hit me (magma1447, owner and main developer of Project-GC). I got a thrombus in my right eye.

Besides the fact that I lost more or less all my eye sight in that eye, I also had quite a lot of pain in the eye, which made me constantly tired. The eye had huge issues focusing since it was all blurred by blood. This resulted in a very tired eye, and therefore the pain. The best solution was to not use the eye more than neccessary. As self-employed it's not always too easy to just call yourself stick and skip work for a month or two.

At least I could work as little as possible and only do neccessary maintenance, bookkeeping, bug fixing and such. I am also happy that I worked so much during the spring, so that it still felt fair.

I have then gradually been working more and more, and since 2-3 months back I am back at full-time, and also full of energy. Losing site in one eye doesn't really affect my work anymore, the human eyes and brain have amazing abilities to compensate, even for someone who is 40+.

GDPR

Another thing that has taken more time than we would have hoped for is GDPR (General Data Privacy Regulations). At first, it was mostly about documenting, rewriting licensing, going through data and comparing it to needs.

Geocaching HQ however had to take a lot more actions. Their actions however help us in a way. If they remove unwanted data, it's removed from us as well. The problem here is that we have been going through multiple variants and implementations. Each of them costing us time for implementing, workarounds, bugfixes and so forth. As we see it, we are up to 4-5 different solutions by now. Each one of them have created weird bugs which has been hard to track down.

This has been an ongoing work since 2016, and we had expected it to be a one-time thing. But it has really followed us and created new work every year. The latest is that Californa has implemented their version (CCPA), which also caused a few new changes in the API.

It's not that we are against the privacy laws, not at all. But it's an issue for a small company when it takes up even as little as 10% of the resources you have for a longer time period.

Outro

All in all, we just wanted to tell you that we actually have been working a lot during 2019, even though it might not seem so.

Next time we will provide some more user-visible news. Both from 2019 and from the beginning of 2020.


2019-11-22

 - 

Early Black Friday/Cyber Monday!

Black Friday Campaign

We are starting this year's Black Friday campaign early. It started today, the 22nd November 12:00 UTC. The honest reason that we are starting the campaign this early is that we are in need of US Dollars. We have a big invoice to pay, and we do not have enough money in USD. Using other capital (EUR/SEK) will cost us a few extra percent in exchange fees.

If you pay for a Project-GC membership during this campaign with US Dollars, you will receive 25% extra membership time. Paying in EUR or SEK will grant you 10% extra membership time. The extra time from the campaign will be added a day or two after your purchase.

Please help us get some US Dollars into our bank account. This is a perfect time for an early renewal, or to start paying for that service that you use regulary.

Visit our Membership page to take an advantage of this deal before it ends.

The site's economy

While we don't feel that the site is threatend by lack of income, we currently aren't bringing in as much money as we need. This is mainly holding back on further development on the site since we can't afford hiring developers/artists. We do have many ideas and features that we would like to implement, but right now it's going slow.

This has led us to change our strategy regarding ads and free use of the site. As before, the site is 100% ad free for our paying members, and we have no plans to change that. But since ~90% of our users are no-paying, we feel that we have the need to try to convert more of them or at least get some more income from ads. Therefore, the site has been going through a few changes on that topic lately.

First off we have moved the ads around, and without doubt increased the amount of ads. We will get back to this to make sure that we feel that there is a reasonable level of ads, and that they aren't too intrusive. Related to this, we have also improved our anti-Adblock implementations. If you do not wish for a paid membership, we at least hope that you can accept seeing ads on the site.

We have also added more reminders in different forms, asking frequent users to upgrade to a paid membership. We will try our best to not nag those who rarely uses the site.

Google Maps

We have recently added Google Maps as tilelayers to our maps again. They are only available for our paying members though. There are a few extra clicks to enable them, and we are sorry about that. The fact is that it would just cost way too much to have them loaded on each map load. We estimate that the cost would be 10-20% of our revenue.

While many users are satisfied with other alternatives to the Google Road map layer, Google's Satellite images are hard to replace. Therefore we are happy that we finally have managed to provide a decent solution for this.

Free membership time

Project-GC is translated into several languages. This is all done by the community and we are extremely grateful for that. We hope those users using the site in another language than English are as well. To reciprocate those translating the site we will start granting free membership time to them. The amount of words/strings translated will be converted into a number of days of free membership time.

In the beginning we handled this manually, but with a growing amount of translators, it just wasn't managable anymore. Now we have been working on a more automatic process. The plan is to grant free membership time retroactive for the work during 2018 and 2019. After that we will grant it on a monthly basis. See this FAQ entry for some more details.

Hopefully we can find similar solutions for those helping out in the QA forum and creating Challenge Checkers as well.

The lack of news

As the observant may have noticed we haven't posted any news here in over a year. There is no real reason for that, besides having the wrong priorities. We will therefore post a few entries the coming week, telling about some of the work that has been done the last year.

We do however recommend you to follow us on Facebook. Besides posting about site issues and such, we are better at posting about smaller changes there. It also happens that we just post some interesting data for people to enjoy.

2018-05-09

 - 

Happy birthday!

Birthdays

In May 2000 Geocaching was born on the 3rd of May, this date the first documented Geocache was hidden. That means that Geocaching is now 18 years old, and still thriving!

Project-GC also celebrates its birthday in May, we had our official launch May 9th 2012. We therefore celebrate 6 years of active service today. Project-GC has existed a bit longer than that though, but in the beginning it was quite unofficial and most didn't know of it. It has been six exciting years, and it still feels like we are just getting started. We have so many more things that we would like to create and present to you all.

To celebrate Geocaching's and most of all, Project-GC's birthday, we will be running a discount campaign on our paid memberships. More of that later in this newsletter, under Discount campaign.

Geographic updates

We have been updating some geographic data lately, as usual. This time it was a bit more complicated for us since HQ has updated some of their data and we of course needed to adjust to this as well. The biggest change is that Czech Republic has been renamed to Czechia in the systems. Then there has been some minor changes to UK, Canada and various small islands as well. This was only step one of HQs updates that we will have to adjust to, we are waiting for the next batch.

We have also been updating polygon data for Slovenia the last month.

System upgrades

As mentioned in the last newsletter, we have been updating some software in the system. We went through with the upgrade from PHP 5.6 to 7.0. Even though we had prepared for it for a long time, it was still plenty of work and we had several minor issues while launching it. Plenty of other systems have also been upgraded, but we still have a few more to go through.

Most importantly we need to bring our database servers up to more recent versions. We also have some operating systems that we would like to upgrade to the latest stable releases. This will be an ongoing project the coming months. Upgrading the database servers will probably require the longest downtime, maybe even a full day. We will get back to you with information regarding this ones it's scheduled.

Discount campaign

Starting Wednesday this week (May 9th, UTC) we will grant extra membership time for every sale we make. For the first days, we will grant 30% extra membership time, and after that it's reduced to 20%.

Starting Wednesday this week, until the end of Sunday (9th - 13th), everyone who buys a membership will get 30% additional membership time added to their account. Take your chance and get 4 years of membership while only paying for 3 years. We have also just introduced the "5 years" level, so that you can buy 5 years at a time. If you wish, you can even buy 5 years twice, and get a total of 13 years of membership time added to your account.

For the rest of the month, from Monday 14th May until the end of May, we will grant 20% extra membership time to everyone who pays for a membership. If you already have an active membership, additional bought time will be added upon your current time. No paid membership time will be lost.

You can verify that you got your additional membership time added by visiting the Membership page and clicking Payment history. Please allow a few minutes for the payment to go through. This is also where you can see when your current membership expires.

All dates/times mentioned are defined for timezone UTC. This offer can not be combined with gifting, referrals, or any other offers.

Free membership time

Besides the campaign we are running above, we are also giving away 100 months of free membership. If you click the below link, you can activate a 30 day free membership. If you already have an active membership, your current one will be extended. But, the below coupon code only has 100 charges. The first 100 to click it and follow the instructions will get their free 30 days.

http://project-gc.com/c/HANJW8

2018-03-13

 - 

Downtime scheduled on Thursday the 15th

Scheduled Downtime

The last week we have been working with major software upgrades on our servers. We have now scheduled an update of the Live servers for Thursday the 15th of March at 9 am UTC (10 am CET). Expected downtime is ~6 hours.

During the downtime a status page will be shown where we will try our best to post updates about how it's going, and if we are on schedule or not.

The primary task is to upgrade PHP from 5.6 to 7.0, something we have been working towards for well over a year. With this we have also completely rewritten our Sandbox environment for the Challenge checkers. Related to that, there have been quite a few updates and changes for the developers of challenge checkers, it has been a fairly big task overall.

New firewall and routing equipment

Last week we switched our old firewall and router to new hardware. The new equipment is more modern and allows us a more automatic failover when there are Internet issues on our primary connection.

During this time we had some downtime as well. The first downtime period lasted for about an hour, but there was some configuration issues and we reverted. A few hours later we brought the systems down again. That time the maintenance window lasted for ~40 minutes, and the switch of hardware was successful.

We are very satisfied with that result, considering we had written over 2500 lines of configuration for the hardware, with minimal testing.

More downtimes

Earlier, we have been holding off on maintentance that has required downtime of the site. We have really wanted to keep the site up. That might not have been very smart since we now have plenty of tasks that requires downtime. We will try to change our point of view here and actually do it when we feel that it's neccessary. Therefore, there might be a few more maintenance windows the coming months. We have at least two more major upgrades that we would like to execute this spring.

When we are starting to plan these, we will first announce them on our Facebook page. They are announced there when we have an idea of which day it might happen. When we have more detailed information we will try to post it on the site as well. Either as an alert message on top, or in a news entry like this one.

Challenge tab in Profile Stats

For our paying members we have added a new tab in the Profile stats, a Challenge tab. Besides listing all challenges that you have logged, there is also a seperate section listing challenges that has been noted. On those noted challenges it will also show a cached result from the corresponding Challenge checker. This way you can see if you have any signed challenges that you have now fulfilled.

Proper list of challenges

While working on the Challenge tab for Profile stats we also added an interface to handle a proper list of challenges. Some geocaches has the word challenge in the title even though they are not challenges, and also the other way around. Project-GC now has a proper way to maintain a list of which geocaches actually is a challenge cache, many thanks to those in the community who has helped out building that list.

Polygon updates

As usual we have been going through some polygon updates. Here is a list of countries that have been updated since we last posted a list back in April (last year). Some of them have been updated more than once during the period.

  • Netherlands
  • Brazil
  • Austria
  • Spain
  • Bulgaria
  • Switzerland
  • Luxembourg
  • Chile
  • Finland
  • New Zealand
  • Australia
  • Norway
  • Denmark
  • Malaysia
  • Indonesia


2017-11-23

 - 

Black Friday / Cyber Monday

Black Friday (and Cyber Monday)

UPDATE: Campaign extended over Cyber Monday as well. Last chance!

It's that time of the year, Black Friday. Black Friday is a tradition originating in the United States. It's an informal name of the day following Thanksgiving. The concept is that most retailers open up very early and offer promotional sales.

Since we are in the online business we are already open 24/7, but we will tag along by offering cheaper prices. Not only during Black Friday either, we'll keep going the whole weekend. As long as it's the 24th to the 26th November 2017 anywhere in the world we will offer the following promotional sales.

Membership

  • When buying membership time for Project-GC you'll get more added time for the same price!
    Pay for 1 year and get that 1 year plus an additional 4 months (20 EUR / 24 USD / 180 SEK).
    Pay for 2 years and get those 2 years plus an additional 5 months (36 EUR / 44 USD / 320 SEK).
    Pay for 3 years and get those 3 years plus an additional 6 months (48 EUR / 59 USD / 440 SEK).
    This offer can not be combined with other offers, like for example referral kickbacks.

Memberships: https://project-gc.com/Home/Membership

Geocoins

  • A 20% discount when buying the 3-set Project-GC Geocoins. Use coupon code BF2017 when checking out to use the promotion.
    Normal price, 41.30 EUR plus any applicable vat, with the 20% discount that's as cheap as 33.04 plus vat.
    This offer is available to the 100 first customers buying the 3-set Geocoins.

Shop: https://project-gc.com/shop/


Don't wait for too long or the offers might expire.


Lab caches update

We are currently working on two known issues with the Lab caches. It might take a few days for us to fix the issues, meanwhile we wanted to let you all know about them.

  1. Some users are seeing duplicate logs of labcaches at Project-GC, making the total number of finds mismatch between Project-GC.com and Geocaching.com.
  2. Some Lab cache finds are on the wrong date. This is likely to be a timezone issue and therefore more common the further from UTC you live.

We have high hopes to have these issues resolved in a week.

2017-11-21

 - 

Lab caches in Profile stats, and more...

It has already been a month since we last sent a newsletter. Time just flies by!

While we do post about updates on Facebook a bit now and then, we also want to tell you about the most important changes here. We do however recommend that you like/follow our Facebook page if you want to keep up with the latest changes.

Lab caches in Profile stats

We expect this to be the most appreciated change we have made the last few weeks. Quite a while back we added the number of lab cache finds to the header of the Profile stats. For our paying members we have now also added the lab cache data into the statistics themselves, for example for Most cache types in a day. In short, they have been added to all modules of the Profile stats where they fit in (remember that lab caches don't have attributes like size, difficulty, terrain and so forth).

It didn't come as a surprise that when released last week we had a few bugs included with it. Those bugs have now been corrected and we currently don't know about any related issues.

If you don't want the lab caches included, they can easily be excluded by turning them off in the Profile stats settings.

TB Rescue - Closed missions

TB Rescue is a system that's in place so that you can add watch zones and get notified if someone adds a rescue mission for their trackable. The concept was originally created by another site, but when they closed their service down it was appreciated by both that site owner and the community that we did our best to fill the gap.

For some unknown reason, the system closed ~250 rescue missions about 3 weeks back. We still haven't been able to find the reason behind it, but at least we have been able to restore the rescue missions so that they are active again.

CloudFlare

A while back we switched Content delivery network from MaxCDN to KeyCDN. We have now made yet another move, this time to CloudFlare. CloudFlare isn't a CDN per se but a combination of many services. We plan to stick around with CloudFlare which is now acting as a frontend to Project-GC. It helps us serving static content and protecting our services against various attacks. The plan is to create more redundancy with the help of this service.

Elevation

Several of our users have noticed that we have "lost" elevation data for many geocaches. The data haven't really been lost, but a new service we added has returned unexpected data. Many geocaches existing on the top of mountains was updated to 0 meters above sea level for example.

Yesterday we released new code for handling elevation data. The workflow is now much simpler than before and it has also been described in our FAQ. Now we will have to recalculate elevation data for all 5.6 million geocaches though, we expect that to take a few days, maybe even over a week.

Web browser extension

We have a Web browser extension (userscript) that integrates Project-GC into Geocaching.com. Other improvements has also been added. To use it one needs for example the Greasemonkey or Tampermonkey plugins installed in the web browser. Firefox has recently moved on to version 4 of Greasemonkey which isn't compatible with old userscripts, our script was affected as well.

The userscript is however open-source and just hours after we added an issue about it on Github a pull request was submitted by a frequent user of Project-GC. Much appreciated!

2017-10-24

 - 

Caught up on support tickets

Support queue

After years of having hundreds of unanswered support tickets we have finally managed to close almost every single ticket in the system. At least we have been down to as low as three open tickets.

Related to this we would like to remind our users about our open Community support system. We strongly suggest that you try to find help there before contacting us. The response times are likely to be faster, and with a big crowd reading your questions you might actually get much better help here.

However, if your question is of a private nature, for example regarding payments, you should definitely contact us directly. Preferably by using one of the Support buttons on the web.

FAD 2017

The first weekend of November we will be visiting Mega Sweden FAD 2017 in Uppsala, Sweden. If you are interested in learning more about Project-GC, Hans who answers most of our support will hold a one hour long lecture there, in English.

We will try to record this lecture as well, and maybe even live stream it. Last time we had huge issues with the audio and we do not know the hall's capabilities yet, so we do not know if it will be better.

Profile stats

In the Hides tab of your Profile stats we have added a new module "Finders by country", feel free to check it out.

We have also added more Continent maps in the Maps tab. Formerly we only had continent maps for Europe and Asia. Now we have added North America, South America, Africa and Oceania to the mix.

Polygon data

As usual we have also been updating some polygon data. The biggest change was to Luxembourg which finally got rid of the definition that has been obsolete for several years. For Switzerland we removed a county that had been merged with another. We added Taiwan which we didn't have polygon data for earlier. Then we added some minor fixes for Egypt and Oman as well.

Challenge checkers

We have reworked the challenge checkers a bit. In some cases, they are now much faster to execute. We should now also show a proper text for when they get timeouts.

For paying members we have a system that we call Auto-challenge-checkers in place. The concept is simple, Project-GC runs checkers all over the world in your name on different challenges so that the system knows beforehand if you fulfill the requirement or not. Whenever you see the cache icon on a map, it will then get either a green or red marker depending on your status.

The Auto-challenge-checkers system has also gotten speed improvements, quite a lot actually. We have also redesigned how the system chooses which challenges to prioritize, hopefully to the better.

2017-10-10

 - 

New cache notifiers

New cache notifiers

If you are a paying member of Project-GC you are hopefully aware of the fact that we have several email notifiers in place. Some of those are:

  • FP notifiers - Daily summary of added/removed favorite points on your geocaches
  • Event notifiers - New events in the world, based on rules that you set up yourself, including number of attendees and distance from home.
  • Edited logs notifiers - If someone updates their log text on one of your geocaches, you should know about it, right?
  • Log images notifiers - If someone uploads a photo with their log, we bet you are interested. If it's geotagged with coordinates, we'll tell you!

Now we have added a new form of notifiers, New cache notifiers. They are quite similar to Geocaching.com's instant notification service, therefore our notifiers also requires a premium membership with Geocaching.com. We do not even see our New cache notifiers as a substitute to theirs, more as a complement. The biggest difference is that our notifiers aren't bound to an 80 km circle. Since it's Project-GC, we of course allow plenty of filtering. You can for example be notified about new Virtual caches all over the world.

The Virtual caches is also exactly why we rushed into releasing this new system. Since Geocaching HQ announced that there might be up to 4000 new Virtuals published in the world, we wanted to help the geocachers to get to know about the publishings. We were also in a bit of hurry to get it out there as fast as possible when this was happening, therefore everything wasn't really in place yet.

The last two weeks we have made a couple of major changes to it. First off, we added region and county filtering as well. Since it was partly included on the page, we assume most felt it was coming soon. This week we have also added Digests, you can now get a daily report of all new geocaches published that match your filters. All current New cache notifiers that didn't have the Follow-up checkbox checked have been converted into digests.


Infrastructure changes

On the technical aspects we have made some changes as well. The site is now fully SSL compatible and we have switched certificate provider, now using Let's encrypt. Associated with this we also had to switch to a new CDN provider, now using KeyCDN instead.

Finally we have switched to http/2 which should help us improve the response times of the web.

2017-08-30

 - 

Virtuals, Cache notifiers, Czech Republic

Virtuals

As you might have heard, Geocaching HQ has rewarded some geocachers by allowing them to publish one Virtual cache each. Geocaches of the type Virtual is something that has been grandfathered since 2005, but now it looks like we might get up to 4000 new ones.

Geocaching HQ's announcement is available at https://www.geocaching.com/blog/2017/08/virtual-rewards/.

If you want to see all newly published Virtual caches, Project-GC can of course help you out with that. We find it's easiest to use Map compare (without comparing) as a tool to solve it. An exampe is this configuration.

Some numbers regarding Virtuals:

  • There has been 7451 Virtual caches published before these news.
  • 4582 of those are still active.
  • There has been 208 new published this month.
  • There has been a total of almost 4 million found logs on Virtual caches.
  • The Virtual with the most found logs is GC45CC in United Kingdom. It has 24,825 found logs and is now archived. It's a bit special though and some geocachers have logged it several times.
  • The Virtual with the second most found logs is GCF496 in Czech Republic. It has 23,553 found logs and is still active. It's also the geocache with 5th most logs in the world, 4th if only counting active ones.
  • United States has the highest number of active Virtuals in the world, 3776 of them. United Kingdom is second with 195.

New cache notifiers at Project-GC

On the same day that this was announced we also launched a new beta function at Project-GC. Notifiers for new caches. It's supposed to be a complement to the Instant Notification Service provided by Geocaching HQ. We provide some advantages, but also have some disadvantages. We believe ours are much easier to set up, has less restrictions, but at the same time they also allow more filtering.

You can for example easily set up notifiers for Virtual caches in the whole world. Visit the page, click add, click Virtual, click save, and you are done. Assuming you already have your email address added at Project-GC that is.

If you are a paying member at Project-GC you can check it out at https://project-gc.com/User/NewCacheNotifiers. For the notifiers to work you also need to be a premium member with Geocaching.com.

This was first announced on Facebook. This is often where we announce things first, to get them tested by a smaller crowd. If you always want the latest information, you should like and follow our Facebook page.

We just released our second public version of the notifiers.

Czech Republic

This weekend we will travel to Great Moravia 2017 in Czech Republic. If you see us at the event site, or anywhere else, feel free to say hello. On Saturday at 13:00 we will have a 1-1.5 hours long lecture about Project-GC. The primary focus will be Planning geocaching trips with Project-GC. But we will also mention some of the other premium features of the site.

We will also sell our Project-GC Geocoins while we are here. They will either be sold by ourselves, or the official Great Moravia Souvenirs stand. This is something we will work out on Friday together with the helpful crew.

If you take a walk in the GPS Maze, which you should, you will also find some data provided by us in there.

2017-08-15

 - 

A system administrator's report

A system administrator's report

We have recently been rebuilding our infrastructure more or less from scratch. This has caused us a few hours of downtime for which we would like to apologize. There are many reasons behind this, but mainly we needed to cleanup the environment. We have been growing the site every day since we launched and always have had a bit too much to do. In many cases we therefore create emergency solutions and at some point it's getting a bit too much of a mess. The goal has been to create a cleaner and more secure hosting environment.


Some of the changes that have been made are:

  • New development environment
    • This is now better separated from the live servers but at the same time also more identical in workflow. We hope that this will give the site more stability and also make it easier to detect and fix different issues.
  • Network
    • Every single network cable has been removed and connected to a new core switch. The environment is now properly configured with multiple VLANs with different purposes.
  • Our old IP networks are now gone and we have a completely new IP plan
  • New VPN system
    • All servers which support it now have out of bounds management properly configured and connected (iLO, iDRAC, IPMI). This VLAN is only accessable from a specific VPN and the administration VLAN.
  • New virtual machines
    • We have installed several new virtual machines. We had two servers which did a bit of everything in the environment. This has now been properly cleaned up so that each server has its own purpose.
  • New firewall
    • The routing between the VLANs is now done in our firewall, a Linux server using netfilter. It currently has 3 Gbps throughput between the VLANs which is much more than needed this far. For a while we were using layer 3 routing in the core switch, but we decided to revert that.
  • Updated documentation
    • Some documentation has lagged behind a bit. We weren't 100% aware of this, but started to notice when installing new servers for example. We wouldn't be surprised if we find the same issue in a few years again, but at least it's not 10 years behind then.
  • Backups
    • We now have a new backup solution, based on Borgbackup (https://github.com/borgbackup/borg). On top of that we have homebrewn scripts handling databases and KVM guests (libvirt).


In the middle of all this we have also been upgrading some of our software to newer major versions.

  • PostGIS
    • We installed a new PostGIS server in parallell with the old one, once that worked as expected we made the switch. It was quite cumbersome for us, but there were no known issues that could be seen by an end user.
  • InfiniDB / MariaDB Column Store
    • For some of our database queries we are using a column-based DBMS, until recently we have been using InfiniDB Community. We did however not have the most recent version and there was also the fact that the project was abandoned.
      MariaDB had forked InfiniDB and we therefore decided to move to their version (MariaDB Column Store). It wasn't a hard decision since our main database cluster is based on MariaDB.
      This transition has caused us some issues though. Probably both because it's a new product, and the fact that we were using an older major version number. One of the callbacks used by some of the Challenge Checkers were affected for example. We also recently found out that Top Archived Loggers (http://project-gc.com/Statistics/TopArchived) didn't work properly due to this. Even more recently we discovered that the connection pool got full from time to time. All these issues has been solved by now though.
  • Monitoring
    • Our old monitoring system was also quite outdated and has received an upgrade. Besides being outdated in software version we haven't really had the time to maintain it and add new servers and services either. We have now spent a few days on this, but we are still not where we need to be. At least the alert system is working again and some of the checks for functionality. In other words, we should now get notified about server issues.
      We also have an external service (Pingdom) telling us when the site is down, but our internal system can provide us with much more detailed information.
      We will keep working on this topic to improve the site stability, with the goal find issues before our users do.
  • Diagnoses tool
    • An offline tool has been written that tests many parts of the site individually. In case something is broken, we can hopefully find where it's broken much faster now.
  • Survey system
    • We have added a survey system where we can setup a questionaire and ask our community how they want things. This far we have had three of them. We also have the option to thank those filling out the form by granting them additional membership time. This was done with one of those surveys.

Disclaimer: This report was written three months ago and parts of it is now old news.