If you have read the topic Big boring changes incoming (PHP7) you know that we are up for some changes. I'll try to make it short to not repeat myself too much. Read the linked topic for more details.
We are upgrading to PHP 7 (from 5.6). The old LUA Sandbox implementation doesn't work anymore, we have added a new one. It affects quite a few scripts. It also affects the syntax that will be used in the future. Also, we are adding all of the LUA scripts to GitLab.
Due to current incompatibility issues between our Development environment and the Live servers we have disabled all editing of LUA source code and creating of LUA scripts on the Live servers. Until further notice, they have to be edited via a development server that will be used as a staging server. More information later in this post.
All scripts in THIS THREAD are broken with the new LUA sandbox. For most of them, there are notes about why. I will try to fix a few of them myself as well, as soon as I get the time. Most of them are very easy to fix.
As mentioned, to edit LUA scripts you now need to access a development server. Every user who has access to create LUA scripts now has will get a new menu item at Project-GC. Admin. In this menu you will find one item only, use it to grant yourself access to the development server. While on that, you can edit your own scripts, just as usual.
We have also pushed are also pushing all LUA scripts to GitLab.com, including all historical versions (with the wrong time-stamps though). When a script is edited at Project-GC.com it will automatically be stored both in our database and at GitLab.com. If edited at GitLab.com, it will be pushed to Project-GC.com as well. This push normally takes 1-2 seconds.
With these changes there are now three ways to edit LUA scripts.
At Project-GC.com, just as before.
Online at GitLab.com, via their online editors.
Cloning the repository, editing, pushing back to origin.
The repositories at GitLab.com are private. They are not open for the public (though we could very well make them readable in the future). If you want access there, register an account and reply to this post with your GitLab username, we will then grant you access to the repository, if you have access to writing LUA scripts at Project-GC.com. When editing scripts at GitLab.com you can edit ANY script, not just your own. This will hopefully help us emptying the list of broken scripts.
While fixing scripts it's important that we keep them compatible with the Live servers as well, which is using the old Sandbox. We therefore added a new parameter with a sandbox version that the scripts can access.
args.environmentSettings.sandboxVersion will equal 2 when in the new sandbox. In the old sandbox args.environmentSettings.sandboxVersion does not exist.
With this you could potentially have different code paths if needed.
All LUA functions have been renamed with the new Sandbox. For example, TableSort will become table.sort. Until later on you should not care about that. To keep it compatible with the Live servers we need to stick to the old syntax until the Live servers has been updated. We will then update the documentation as well.
There are 66000 versions of the scripts to push to GitLab.com. We are currently pushing one version per second (due to rate limits). Therefore it will take almost a day until they are all pushed. Once they are all pushed, we will add the menu item to access the development environment, and we will also start granting access to GitLab.com. Feel free to register there and give us your usernames beforehand.
PS! It's not impossible that we will use different branches on GitLab.com for keeping development versions of scripts in the future. This is step one towards endless of possibilities.
Edited 2 time(s). Last edit at 03/02/2018 10:54PM by ganja1447.
> Once you have access to the development server,
> feel free to test such things as well, including
> trying to produce errors and see if you get the
> error texts back.
I see error messages but they are pretty useless to locate the error. For example "Error 8, challenges.php:1590 Undefined offset: 0" means that there is no return statement in the end of the script and "Error 4096, challenges.php:1522 Object of class LuaSandboxFunction could not be converted to string" means that I forgot empty parentheses from a function call.
Is this just temporary or something which should be addressed?
> ganja1447 Wrote:
> > Once you have access to the development server,
> > feel free to test such things as well,
> > trying to produce errors and see if you get the
> > error texts back.
> I see error messages but they are pretty useless
> to locate the error. For example "Error 8,
> challenges.php:1590 Undefined offset: 0" means
> that there is no return statement in the end of
> the script and "Error 4096, challenges.php:1522
> Object of class LuaSandboxFunction could not be
> converted to string" means that I forgot empty
> parentheses from a function call.
> Is this just temporary or something which should
> be addressed?
I am not sure how I could generally give better errors when you are using callback functions in an unexpected way.
The first error sure, a better error message could be given after checking if the return value is valid or not. But for the second error, that would mean a lot of error handling for every callback there is. And this specific error probably wouldn't be covered even then. I might be able to catch the exception and rephrase it though.
My point when asking if you got decent error messages actually was, "where you got it before". It's supposed to be at least as good as before. My focus was that the LUA errors are still getting back to the developer. But I do hear you. Some decent error handling could very well be implemented, like checking that the parameters to functions exists and are of the correct type.
I will definitely keep this in mind. I am playing with the idea to have user contributed modules for Profile stats as well, and then things like this should be kept in mind.
I was expecting the upload to GitLab to be done late Saturday, or at least Sunday morning. It hasn't.
Sometimes the script was aborted due to errors, and it had to be manually restarted with some adjustments, it wasn't built for knowing where it was. It's supposed to be a one time thing and I didn't want to spend too much time on the script.
At first I thought it was errors due to rate limiting. I knew there was rate limiting, but couldn't find any exact numbers. I then added a sleep, and it seemed to work better, but not good enough. The problem was that it was interrupted during night time when I didn't restart it.
I then added code to read the rate limit ratio. I found out that it was impossible to hit the rate limit with a single thread. So the errors I got seem to be random HTTP 500. Since it has been weekend I have had some other stuff to do as well and have just restarted the script, a lot. This evening I did get the time to rewrite some code though. It now does retries on HTTP 500 and seems to work much better.
The retry on HTTP 500 has only been implemented in this one-time script now, but I will change that for the "real" code tomorrow. It should be a very easy change.
The upload itself could have been much faster by cloning the repository and adding/committing files locally, and then push to origin. But I wanted to test the new code, to find issues like this. I actually found another unrelated big issue with PHP 7.x and the database servers as well while at it. It has been a great extra test of the development new environment.
The current status is that ~1100 of ~2700 scripts has been uploaded. But from now on, I am not expecting any interrupts, and all sleeps has been removed. It seems to be ~28500 commits left to do. I expect them to take 1-2 seconds each, so 8-16 hours. With some luck, it's all ready tomorrow before lunch!
NOW, the development environment and the new code is available!
I gave up on having GitLab answer today. I through away the broken project/repository and created a new one. I clone the repository, filled it locally and pushed the data. It was 10-20 times larger than I expected. Lots of LUA-code (with historical versions).
Everyone with LUA-access should now have an Admin menu item, from where they can access the development environment. From here you should be able to edit your scripts (or at GitLab). I have invited everyone who wrote their GitLab account (if I didn't miss anyone).
I'll take a short break, then I will start working on adjusting the scripts that uses the variable name table. Remember, if you fix any scripts, test run them afterwards, especially if it's not your own scripts!
Wow, great work! Can't wait to help with correction of codes. GitLab access works great, just a small question, where should we look for admin menu - maybe a small glitch, but I've looked around and can't find it. Thanks.
> Wow, great work! Can't wait to help with
> correction of codes. GitLab access works great,
> just a small question, where should we look for
> admin menu - maybe a small glitch, but I've looked
> around and can't find it. Thanks.
If it doesn't, I have failed in some way. I looked at the code, and it seems like I needed to add permissions both to the menu item and the menu itself. I had only added to the menu item. Try now and see if it works better.
I will try to fix a few more before that though. But I will probably skip those that requires me to really understand the code. Those where I believe I know how to fix it already, I will try to fix. It's also easier to fix them when they work on the Live servers since the result can then be compared.
I am rushing getting the latest changes onto the Live servers. It's such big changes that I prefer to do it while I still remember most of what I went through upgrading DEV. Also, it's a bit cumbersome having the Dev-environment completely different from the Live servers.