# go-judge
[](https://pkg.go.dev/github.com/criyle/go-judge) [](https://goreportcard.com/report/github.com/criyle/go-judge) [](https://github.com/criyle/go-judge/releases/latest) 
[中文文档](README.cn.md)
## Executor Service
Fast, Simple, Secure
### Prerequisite
- Linux Kernel Version >= 3.10
- Cgroup file system mounted at /sys/fs/cgroup. Usually done by systemd
### Architecture
```text
+----------------------------------------------------------------------------------+
| Transport Layer (HTTP / WebSocket / FFI / ...) |
+----------------------------------------------------------------------------------+
| Executor Worker (Environment Pool w/ Environment Builder ) |
+-----------------------------------------------------------+----------------------+
| EnvExec | File Store |
+--------------------+----------------+---------------------+---------------+------+
| Linux (go-sandbox) | Windows (winc) | macOS (app sandbox) | Shared Memory | Disk |
+--------------------+----------------+---------------------+---------------+------+
```
### REST API
A REST service to run program in restricted environment and it is basically a wrapper for `envexec` to run single / multiple programs.
- /run POST execute program in the restricted environment (examples below)
- /file GET list all cached file id to original name map
- /file POST prepare a file in the executor service (in memory), returns fileId (can be referenced in /run parameter)
- /file/:fileId GET downloads file from executor service (in memory), returns file content
- /file/:fileId DELETE delete file specified by fileId
- /ws WebSocket for /run
- /version gets build git version (e.g. `v1.4.0`) together with runtime information (go version, os, platform)
- /config gets some configuration (e.g. `fileStorePath`) together with some supported features
Monitor HTTP endpoint (default `:5052`, specified by `-monitor-addr`)
- /metrics prometheus metrics (specifies `ES_ENABLE_METRICS=1` environment variable to enable metrics)
- /debug (specifies `ES_ENABLE_DEBUG=1` environment variable to enable go runtime debug endpoint)
### Command Line Arguments
Server:
- The default binding address for the executor server is `:5050`. Can be specified with `-http-addr` flag.
- By default gRPC endpoint is disabled, to enable gRPC endpoint, add `-enable-grpc` flag.
- The default binding address for the gRPC executor server is `:5051`. Can be specified with `-grpc-addr` flag.
- The default log level is info, use `-silent` to disable logs or use `-release` to enable release logger (auto turn on if in docker).
- `-auth-token` to add token-based authentication to REST / gRPC
- By default, the GO debug endpoints are disabled, to enable, specifies `-enable-debug`, and it also enables debug log
- By default, the prometheus metrics endpoints are disabled, to enable, specifies `-enable-metrics`
- Monitoring HTTP endpoint is enabled if metrics / debug is enabled, the default addr is `:5052` and can be specified by `-monitor-addr`
Sandbox:
- The default concurrency equal to number of CPU, Can be specified with `-parallelism` flag.
- The default file store is in memory, local cache can be specified with `-dir` flag.
- The default CGroup prefix is `executor_server`, Can be specified with `-cgroup-prefix` flag.
- `-src-prefix` to restrict `src` copyIn path (need to be absolute path)
- `-time-limit-checker-interval` specifies time limit checker interval (default 100ms) (valid value: \[1ms, 1s\])
- `-output-limit` specifies size limit of POSIX rlimit of output (default 256MiB)
- `-extra-memory-limit` specifies the additional memory limit to check memory limit exceeded (default 16KiB)
- `-copy-out-limit` specifies the default file copy out max (default 64MiB)
- `-open-file-limit` specifies the max number of open files (default 256)
- `-cpuset` specifies `cpuset.cpus` cgroup for each container (Linux only)
- `-container-cred-start` specifies container `setuid` / `setgid` credential start point (default: 10000) (Linux only)
- for example, by default container 0 will run with 10001 uid & gid and container 1 will run with 10002 uid & gid...
- `-enable-cpu-rate` enabled `cpu` cgroup to control cpu rate using cfs_quota & cfs_period control (Linux only)
- `-cpu-cfs-period` specifies cfs_period if cpu rate is enabled (default 100ms) (valid value: \[1ms, 1s\])
- `-seccomp-conf` specifies `seecomp` filter setting to load when running program (need build tag `seccomp`) (Linux only)
- for example, by `strace -c prog` to get all `syscall` needed and restrict to that sub set
- however, the `syscall` count in one platform(e.g. x86_64) is not suitable for all platform, so this option is not recommended
- the program killed by seccomp filter will have status `Dangerous Syscall`
- `-pre-fork` specifies number of container to create when server starts
- `-tmp-fs-param` specifies the tmpfs parameter for `/w` and `/tmp` when using default mounting (Linux only)
- `-file-timeout` specifies maximum TTL for file created in file store (e.g. `30m`)
- `-mount-conf` specifies detailed mount configuration, please refer `mount.yaml` as a reference (Linux only)
- `-container-init-path` specifies path to `cinit` (do not use, debug only) (Linux only)
### Environment Variables
Environment variable will be override by command line arguments if they both present and all command line arguments have its correspond environment variable (e.g. `ES_HTTP_ADDR`). Run `executorserver --help` to see all the environment variable configurations.
### Install & Run
Download compiled executable from [Release](https://github.com/criyle/go-judge/releases) and run.
Or, by docker
```bash
docker run -it --rm --privileged --shm-size=256m -p 5050:5050 criyle/executorserver
```
#### Build Executor Server
Build by your own `docker build -t executorserver -f Dockerfile.exec .`
The `executorserver` need root privilege to create `cgroup`. Either creates sub-directory `/sys/fs/cgroup/cpuacct/executor_server`, `/sys/fs/cgroup/memory/executor_server`, `/sys/fs/cgroup/pids/executor_server` and make execution user readable or use `sudo` to run it.
#### Build Shared object
Build container init `cinit`:
`go build -o cinit ./cmd/cinit`
Build `executor_server.so`:
`go build -buildmode=c-shared -o executor_server.so ./cmd/ffi/`
For example, in JavaScript, run with `ffi-napi` (seems node 14 is not supported yet):
### Build Executor Proxy
Build `go build ./cmd/executorproxy`
Run `./executorproxy`, connect to gRPC endpoint expose as a REST endpoint.
### Build Executor Shell
Build `go build ./cmd/executorshell`
Run `./executorshell`, connect to gRPC endpoint with interactive shell.
### Return Status
- Accepted: Program exited with status code 0 within time & memory limits
- Memory Limit Exceeded: Program uses more memory than memory limits
- Time Limit Exceeded:
- Program uses more CPU time than cpuLimit
- Or, program uses more clock time than clockLimit
- Output Limit Exceeded:
- Program output more than pipeCollector limits
- Or, program output more than output-limit
- File Error:
- CopyIn file is not existed
- Or, CopyIn file too large for container file system
- Or, CopyOut file is not existed after program exited
- Non Zero Exit Status: Program exited with non 0 status code within time & memory limits
- Signalled: Program exited with signal (e.g. SIGSEGV)
- Dangerous Syscall: Program killed by seccomp filter
- Internal Error:
- Program is not exist
- Or, container create not successful (e.g. not privileged docker)
- Or, other errors
### Container Root File