Packaging (docker, ci, variants)

How big?

GreyCat is a standalone executable with minimal dependencies

otool -L greycat # or ldd on linux
ls -alh greycat

You will obtain something like:

greycat:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.100.5)

-rwxr-xr-x  1 user  staff   3.5M Sep 18 09:26 /Users/user/.greycat/bin/greycat

As we said in the intro GreyCat can handle terabytes of data but can also run a tiny ARM computers.

---

GreyCat variants: arch and os

we distribute GreyCat as a standalone executable with no dependencies, it is distributed with the following variants

• x86_64-linux-gnu
• x86_64-linux-none
• x86_64_v3-linux-gnu
• x86_64_v3-linux-none
• aarch64-linux-gnu
• aarch64-linux-none
• windows and mac are supported since version 6

x86_64 is a generic build for intel or amd powered servers

x86_64_v3 is reserved for server with AVX2 instructions ability

aarch64 is for ARM64 bits server like EC2 Gravitron or RPI3+

gnu is for libc powered linux distribution (debian, ubuntu, redhat)

none is compiled statically and do not depends on any distribution, good for scratch

---

CI and build process

a “classic” CI workflow for a GreyCat project is the following

greycat test
greycat build
npm run build
docker build -t project:version .
docker push

(optional) npm run build is relevant in case of more complex web development (not necessary for the demo)

greycat test and greycat build command can also be executed within a multi stage docker

---

Docker

Dockerfile scratch version

• no webroot, script mode

FROM scratch
ADD project.gcp /project.gcp
ADD greycat /bin/greycat
VOLUME /gcdata
CMD ["/bin/greycat","run"]

• with webroot, server mode

FROM scratch
ADD build /webroot
ADD project.gcp /project.gcp
ADD greycat /bin/greycat
VOLUME /gcdata
CMD ["/bin/greycat","serve","--port=8080", "--webroot=/webroot"]

scratch images only contains GreyCat executable and related files, no update process is necessary (no apt or yum)

---

Dockerfile GNU version

FROM busybox:1.36.1
ADD build /webroot
ADD project.gcp /project.gcp
ADD greycat /bin/greycat
VOLUME /gcdata
CMD ["/bin/greycat","serve","--port=8080", "--webroot=/webroot"]

or

FROM oci.datathings.com/greycat/greycat/busybox:${GREYCAT_VERSION}
ADD build /webroot
ADD project.gcp /project.gcp
VOLUME /gcdata
CMD ["/bin/greycat","serve","--port=8080", "--webroot=/webroot"]

The GNU version is fully compatible with LD_PRELOAD utility such as malloc or other specialized monitoring utility

---

Multi-stages and minimal GNU

considering that a ldd command on GNU version of GreyCat would print the following

ldd ./greycat
        linux-vdso.so.1 (0x0000ffffa32e0000)
        libm.so.6 => /lib64/libm.so.6 (0x0000ffffa2cb0000)
        libc.so.6 => /lib64/libc.so.6 (0x0000ffffa2ae0000)
        /lib/ld-linux.so.1 (0x0000ffffa3293000)

we can build a minimal scratch GNU yet with the libc version according to corporate rules

FROM ubi8-minimal:8.8 as base
ADD greycat /bin/greycat
RUN greycat build
RUN greycat test
RUN yum -y nodejs && npm i && npm run build

FROM scratch
COPY --from=base ["/lib/ld-linux.so.1", "/lib64/libc.so.6", "/lib64/libm.so.6", "/bin/greycat"]
COPY --from=base /root/build /webroot
COPY --from=base /root/project.gcp /project.gcp
VOLUME /gcdata
CMD ["/bin/greycat","serve","--port=8080", "--webroot=/webroot"]

this version has a minimal attack surface while being on a fixed libc version

---

Baremetal or Virtual Machine Installation

In case of virtual machine we recommand to use a systemd minimal base operating system such as : https://fedoraproject.org/cloud/download.

Then create a systemd file /etc/systemd/system/greycat.service with the following content:

[Unit]
Description=greycat
After=network.target

[Service]
Type=simple
ExecStart=/data/bin/greycat serve --unsecure --webroot=webroot
Restart=on-failure
WorkingDirectory=/data

[Install]
WantedBy=multi-user.target

Create a directory /data

Inside copy the greycat executable under the directory /data/bin

Please verify that the executable run properly. Especially take care that you are on the x86_64_v3 processor!

Then copy the webroot, .env, project.gcp into the /data directory.

Once you’re done with the files, enable the service:

systemctl enable greycat.service

It will start automatically on next boot. You could even enable and start the service in one go with

systemctl enable --now greycat.service

---

GreyCat license file

• GreyCat license file is delivered by DataThings

• this file is required to unlock full computation power of Greycat

• in case of unreadable file, GreyCat runtime/server fail-back to freeware limitations

  • this can block the write processes in case the storage is limited

license file can copied in two potential places

• inside the docker build as an additional instruction, copied aside project.gcp

ADD license license

• or inside the gcdata if manage only by IT Team

gcdata
└── license

---

External scripts

warning: GreyCat allows developer to call an external script

System::exec("python main.py");

this can be limited by administrator by removing the right from the roles

however, in this case the Dockerfile should reflect the environment required by the script

calling sub script is potentially cost in case of cold VM start, take care of call granularity

---

Schema evolution

Whenever GreyCat start from a project.gcp it compare it with the previous version

In case an evolution of type is detected, GreyCat trigger an update procedure

This procedure verifies that

• type do have additional nullable field
• recompute all index to keep data store compatible
• avoid former type removal (instead they become anonymous types)

It is always easier to rebuild the data from scratch from the raw data

but when it is not possible it is important to code a check routine inside the main function at boot

it is essential to add all new field as nullable otherwise GreyCat can reject the upgrade