How to check submission error #176
Replies: 3 comments 7 replies
-
|
I have a guess of the cause of this issue, although I do not have the server logs to confirm it. When starting the bot, the mjai.app/python/mjai/http_server/server.py Lines 7 to 17 in dc9dbfb However, the player client waits only at most 10 seconds: mjai.app/python/mjai/player.py Lines 137 to 140 in dc9dbfb So if the decompression does not complete in 10 seconds, due to either limited CPU or disk I/O speed, the matches cannot start. It can explain why there were no stdout or stderr log files. (They were missing instead of empty, because empty logs would have the You can test it out by uploading a If my hypothesis is true, it can be the root cause for this similar issue too: #139 |
Beta Was this translation helpful? Give feedback.
-
|
I think your focus is a little off from the root cause of the issue. For the 10-second and 2-second timeouts, they are for the responses of actions to the corresponding lists of events. These apply after the matches already start, and you will get log outputs anyway in this case (which can be empty but not missing). The root cause should be the timeout of starting the docker container for the player client and waiting for the HTTP server inside the container to be ready. These happen before matches start. Since the decompression of About the things you proposed, delaying the running of The solution should be to increase the 12 seconds timeout, because the author mentions that the maximum upload size is 1GB, so the timeout should always allow such big archives to decompress. Otherwise, the upload size limit is meaningless because the actual upper bound is decided by CPU and disk I/O speed. I also think that the timeout should not be removed or put off after decompression, because you always need to prevent some unexpected situations that the decompression never ends or takes too long, blocking the matching process. So increasing the timeout to 2 times the common time to decompress an archive containing 1GB data on the cloud VM instance (plus the time to launch the docker container) should be a good enough solution. (Another concern is that in a real match, the player clients will start in turn, and some clients will need to load a model or other data from disk after starting the bot process, which can occupy disk I/O throughput, causing decompression of other bots to be slower if happening at the same time. Concurrent disk I/O is hard to avoid because there is no reliable way to tell whether the bots have completed loading, and programs may choose to read from files progressively that spans a long time. So the timeout needs to be abundant.) |
Beta Was this translation helpful? Give feedback.
-
|
No, this judging system should refer to existing OnlineJudge platforms. In these platforms, the judging process is actually divided into two steps:
Typically, the execution time limit for a problem is around 1–2 seconds. In mjai’s current execution model, decompressing is analogous to compiling. So, do you think the online judge’s time limit should equal the compile time plus the execution time? Clearly, this is unreasonable because in many cases the compilation and linking process itself can take over 1 second. This would effectively compress the actual execution time to only 0–1 second. Moreover, in mjai, the maximum allowed compressed package size is 1GB. A 1GB compressed package can contain the source code for Python libraries, potentially including tens or even hundreds of thousands of files. Fully decompressing such a package might already consume up to 10 seconds, leaving only 2 seconds for the program startup and model loading. If the model file is large, it is simply not feasible for the program to start within just 2 seconds. |
Beta Was this translation helpful? Give feedback.
-
|
So, I believe that the program's initialization time must be separated from the decompression time. It is very unreasonable to mark a user's code as timed out simply because the decompression takes too long. In your proposal, you only increased the time limit to 15 seconds. While I agree that, under most circumstances, 15 seconds is enough for most AI code to be decompressed and executed, the system should still allow for some margin. We should not disregard the remaining 5% of bots whose initialization time exceeds 15 seconds just because 15 seconds suffices for over 95% of cases. |
Beta Was this translation helpful? Give feedback.
-
Regarding your point on responses, I believe that once decompression is completed, the server should be able to start immediately and respond with "OK". At the same time, it should launch another bot process. The player will then send a "start_game" message to the server and wait for a response from the bot process. Since the bot process is already running, data can immediately be written to its standard input, and the bot should reply with a response like |
Beta Was this translation helpful? Give feedback.
-
|
I think you still confused what I mean. I said that please do not refer to what I said in the issue #139, because the issue encountered was completely unrelated to what we discuss here. I said here that we should "increase the timeout to 2 times the common time to decompress an archive containing 1GB data on the cloud VM instance." I did not mention 15 seconds anywhere here. You are correct that 1GB archives can take vastly differing time to decompress depending on the compression levels, but we have to set some timeout anyway. I pointed out that the issue is likely in About your analogy between mjai.app and online judge systems, I think what I said has no problems at all. The decompression of A side note for my original posts in the issue #139 (because you are always confusing it with the current issue): my previous versions of bots take more than 10 seconds to initialize half of the time, but the |
Beta Was this translation helpful? Give feedback.
-
|
After checking, I found that the issue is most likely caused by the zip file decompression leading to a timeout. The My bot.zip file is 575 MB. Considering that mjai.app limits the maximum file size to 1GB, I believe the httpserver should consider increasing the timeout limit for unzip. Alternatively, consider using Python's multiprocessing to decompress the archive, thereby fully utilizing multi-core CPU performance. |
Beta Was this translation helpful? Give feedback.
-
|
update: Since we identified that the timeout was caused by the unzip operation, I performed a |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
I tried to submit, but it always returns a failed status, and stdout and stderr are empty.
So I pulled the image smly/mjai-client:v3 and submitted it using mjai.Simulator; the execution results indicate that everything is running normally. But it still returns failed.
Beta Was this translation helpful? Give feedback.
All reactions