7.0.1685-testing
In this page
  1. 7.0
  2. 6.10
  3. 5.x to 6.x

Core Changelog

7.0

Anonymous Objects

The following is no longer allowed, a lang fix is available that will generate the type automatically for you.

var foo = { bar: "hello };

removed columns from CsvFormat

CsvColumn and all it’s associated types have been removed in favor of the new typed reader.

CsvFormat has new attribute called format and tz to pass the date custom format and timezone if the csv doesn’t have an iso8061 date

In case you have multiple dates in different formats you can specify each format individually

type CsvLine {
  date1: time; // will parse as iso8601
  @format("d/m/Y")
  date2: time; // will parse using the custom format
}

Permissions

The new default permissions are

Permission Description
public Default, associated with anonymous users.
admin Allows to administrate anything on the server.
api Allows access to exposed functions.
debug Allows access to low-level graph manipulation functions.
files Allows access to files under /files/ & /webroot/ according to ACL.

The new default roles are

Role Permissions
public public
admin public, admin, api, debug, files
user public, api, files

To create a new Permissions

@permission("foo", "The description for my custom permission");

To create a new role

// The first value is the name of the role, the rest are its permissions
@role("custom", "foo", "api");

To edit an existing role, you can only add new permissions to a default role

@role("user", "debug");

Change to module variables

Non **node** module variables are not supported anymore

The advantage of this no need to optionally create a nodeIndex / nodeTime / nodeList / nodeGeo anymore with “?=”, they will be automatically created for you, this is only valid for the specified node indexes in module variables (outside functions) and not valid for node

var s: Map?; // wrong
var s2: int // wrong
var s3: nodeIndex<String, String>; // correct
var s4: node<int?>; // correct
fn main(){
  s3.set("hello", 42); // s3 already exists
}

multi write

Greycat can now write in parallel, this comes with a few limitation.

The same node can’t be edited in two separate jobs running in parallel this will cause the final commit to fail saying that it failed to merge.

You have to either in a pre or post step update your global indexes.

@write deprecated

There are no more distinction between read and write function, every function can be both, the write check is made at the commit phase meaning when the function ends.

If you relied on the @write to make write permission checks, it’s better to now add @permission(“graph.write”) to error out early, instead of error at the end the function.

New perf log (replacing debug)

//TODO give an example

perf level enable any part of the code to emit a performance related log to trace user defined KPI

Use cases

  • line per seconds
  • bytes per seconds

New typed Readers

type CsvReader<T> {
  fn read(): T;
}

type CsvLine {
  a: String;
  b: String;
}

fn main(){
  var reader = CsvReader<CsvLine>{path : "file.csv"};
  reader.read(); // return now type CsvLine! both in lang and runtime
}

Every GreyCat reader is an iterator, calling read() on it will yield a record and advance the internal cursor pos.

while (reader.can_read()) {
  var record = reader.read();
  // do something with the validated record
}

For more information read the dedicated pages for each reader in the io section

Protected field name

We now support complex field name to match JSON capability

type A {
    "my field &": String;
}

fn main(){
  var a = A {
     "my field &": "hello"
  };
  pprint(a."my field &"); // pretty print
}

New @volatile pragma

To facilitate the new Type everything philosophy and alleviate upgrades a new pragma for types has been introduced

This is primarily intended to only be used for frontend GUI, intermediate parsers (Csv, Json, Text), anything that is not to be stored on the graph.

Do not Store a volatile type in the graph 
@volatile
type Foo {
  bar: String;
}

@volatile
type Foo {
  /* ... */
  baz: String; // new non null field, No need to remove the state anymore
}

New auto generated directories

There a few new directories generated with greycat now along side /gcdata.

As previously mentioned , /bin and /lib , as well as a /files directory will be auto generated.

The files directory contains a global log file (all logs will now be printed inside here even logs created from inside tasks) and all tasks result will be located here.

Tasks

v7 comes with several changes on how tasks are used, a new Task API and 2 new concepts: Job, await

Firstly, the keyword task has been removed; now every function can be spawned as a task!

On v6

fn run_task() { // cannot be an entrypoint
  Task::spawn(my_task, my_args); // spawns and forget
}

on v7

fn main() {
  var jobs = [Job { function: my_task }]; // describes a list of jobs
  await(jobs); // actively awaits jobs' completion
  Assert::equals(jobs[0].result(), 42); // lazily evaluates a Job's result
}

fn my_task() {
  return 42;
}

Functions (here, main) are now always run in a task context, eg. an asynchronous context

Asynchronicity allows us to enable await, which will pause the current task while awaiting for its jobs to complete

A Job also provides a way to retrieve the result of its execution (its returned value) via job.result()

await will throw if at least one job has raised an exception, you can try/catch it and iterate over the job’s result to find the error

Creating a Job does nothing on its own, it is just a GreyCat object, await-ing jobs schedules them for completion (which, if the queue is full, might be long)

Tuples short notation

Tuple now can be instantiated with a short notation such as the following:

var t = ("my_name", {v: "content"});

Meta Programming

The v7 introduces as full new meta programming API. Type pointers can now be used to compare and introspect values for fields or enumerations.

Core module method sameType(x,y) is replaced by type::of(x) == type::of(y)

Similarly, the method valueEnum is replaced by type::enum_value. The signature:

native static fn enum_value<T>(enum_type: typeof T, offset: int): T;

Illustrates the new usage of generic type to keep typing til the result of the meta fetch methods.

Enumerations can also be introspected using the following method to get the number of available values or directly values.

core::DurationUnit.enum_values();
core::DurationUnit.nb_enum_values();

Same approach exists for field in object.

type type {
    /// Return the number of field of this type
    native fn nb_fields(): int;
    /// Return all field descriptor from this type
    native fn fields(): Array<TypeField>;
    /// Return field offset or null from this type and given a field name
    native fn field_by_name(name: String): int?;
}
type Sensor {
  name: String;
  value: float;
}
fn main(){
  project::Sensor.fields();
  var offset_name = project::Sensor.field_by_name("value"); // return 1
  var sensor = Sensor {"sensor_1", 42.5};
  println(type::field_get(sensor, 0)); // print sensor_1
  println(type::field_set(sensor, 1, 142.5));
}

Type pointer usage for dynamic mapper

Newly introduced type pointer can now be used within Map together with function pointer to create dynamic mapper. Please find below an example of usage:

type MyType {}
fn main(){
    var m = Map<type, function>{};
    m.set(project::MyType, fn(t: MyType){  println(t); });
    var obj = MyType{};
    var f = m.get(type::of(obj));
    info(f(obj));
}

New keyword: typeof

Function with generic parameter can now map generic parameter with the type passed as parameter. This enables the use of dynamic type mapper such as in parser for instance.

native fn parse<T>(str: String, s: typeof T) : T;

Generic function

Function can now have a generic parameter allowing to specify return type according to parameter type for instance.

native fn clone<T>(v: T): T;

Type pointer

Any GCL type can now be reference by a qualified name

/// project.gcl
type Sensor {

}
fn main(){
  var sensor_type: type = project::Hello;
  println(sensor_type);
}

Precision Decorator

Float values can now be decorated by an annotation @precision(0.01). The annotation value defined the expected precision kept by GreyCat by the dev. This simple annotation can compress by an order of magnitude the size of GreyCat store.

Private type attribute

Keyword private is now available to protect type attributes from external mutations. The following example illustrates the semantics associated to the keyword.

Essentially, only local methods are allowed to mutate private attributes.

type Foo {
  private x: int;

  fn set(x: int) {
    this.x = x; // correct
  }
}

fn main() {
  var foo = Foo { x: 5 };
  foo.set(100);
  Assert::equals(foo.x, 100); // allowed to read private attribute

  foo.x = 42; // this will raise an error
}

New command: install

A new command install is available in the greycat CLI

This command will download the required libraries and greycat version defined in the project.gcl:

@library("std", "7.0.1304-dev"); // greycat core version
@library("useragent", "7.0.22-dev");  // user agent parser library

Before the first greycat run (or serve), user should run:

greycat install

It should output something like:

Download: https://get.greycat.io/files/core/dev/7.0/x64-linux/7.0.1304-dev.zip
Extracted: /home/max/Projects/analytics/bin/greycat
Extracted: /home/max/Projects/analytics/lib/std/core.gcl
Extracted: /home/max/Projects/analytics/lib/std/io.gcl
Extracted: /home/max/Projects/analytics/lib/std/runtime.gcl
Extracted: /home/max/Projects/analytics/lib/std/util.gcl
Extracted: /home/max/Projects/analytics/lib/std/include/greycat.h
Extracted: /home/max/Projects/analytics/lib/std/include/greycat.def
Download: https://get.greycat.io/files/useragent/dev/7.0/x64-linux/7.0.22-dev.zip
Extracted: /home/max/Projects/analytics/lib/std/core.gcl
Extracted: /home/max/Projects/analytics/lib/std/io.gcl
Extracted: /home/max/Projects/analytics/lib/std/runtime.gcl
Extracted: /home/max/Projects/analytics/lib/std/util.gcl
Extracted: /home/max/Projects/analytics/lib/std/include/greycat.h
Extracted: /home/max/Projects/analytics/lib/std/include/greycat.def
Extracted: /home/max/Projects/analytics/lib/useragent/useragent.gcl
Extracted: /home/max/Projects/analytics/lib/useragent/useragent.gclib

As you can see the greycat executable is now located locally in the bin directory,

All previous scripts using the global greycat path should instead use the local one

The libraries are located in a local lib directory

New command: print

A new command print is available in the greycat CLI

This command prints to stdout the content of the given GreyCat binary file (.gcb)

greycat print result.gcb

A --format=json flag can be specified to have it output JSON instead:

greycat print --format=json result.gcb

Removed module: math

Module math as been removed from the std library.

All math functions are now directly accessible from the core module.

The core module being always available in scope, it does not require any additional use statement now to use the old math symbols.

Updated types: Date, time

Date is no longer a native type, it is now designed for human presentation, while time is designed for calculations.
This means that most of the v6 Date API has been moved to time.

API Breaking changes

v6 v7
Date::new(): Date Date {year: , month: , ...} normal object creation
Date::fromEpoch(): Date Date::fromTime(time, TimeZone)
Date::toTime(): time Date::toTime(TimeZone): time
Date::add() removed, time::calendar_add(value, CalendarUnit, TimeZone)
Date::substract() removed, time::calendar_add(value, CalendarUnit, TimeZone) negative value
Date::floor() removed, time::calendar_floor(CalendarUnit, TimeZone): time
Date::ceiling() removed, time::calendar_ceiling(CalendarUnit, TimeZone): time
Date::startOfWeek() removed, time::startOfWeek(TimeZone): time
Date::dayOfYear() removed, time::dayOfYear(TimeZone)
Date::dayOfWeek() removed, time::dayOfWeek(TimeZone)
Date::weekOfYear() removed, time::weekOfYear()
Date::leapYear() removed
Date::isLeap() removed, time::isLeap(year: int)
Date::daysInYear() removed
Date::daysInMonth() removed
Date::totalDaysInYear() removed, use time::totalDaysInYear(year)
Date::totalDaysInMonth() removed, use time::totalDaysInMonth(month, year)
Date::startOfWeek() removed, use time::startOfWeek(Timezone):time
Date::endOfWeek() removed, use time::endOfWeek(Timezone):time
Date::set() removed, either set field, or use time methods (safer)
Date::setTimeZone() removed, timezone is not stored
Date::getTimeZone() removed, timezone is not stored
Date::shiftTimeZone() removed, timezone is not stored
Date::clone() removed
Date::toString() removed
Date::get(DatePart::year),… removed, use Date.year, Date.month, etc.
Date::year(), … removed, use Date.year, Date.month, etc.
Date::diff() removed, use an operation eg. date1.year - date2.year, or use times
Date::equals() removed, use an operation eg. date1.year == date2.year, or use times
Date::min(), max() removed, use time:min(), time::max()
time::toDate(): Date removed, Date::fromTime(time, TimeZone): Date
time::toDateUTC(): Date removed, Date::fromTime(time, TimeZone): Date
Duration::years removed,
Duration::months removed,

The new CalendarUnit replaces DatePart:

enum CalendarUnit {
  year(0);
  month(1);
  day(2);
  hour(3);
  minute(4);
  second(5);
  microsecond(6);
}

Updated type: Quantizer

Quantizer is no longer a native type, its configuration is set at object creation.

The configuration per dimension is set with the appropriate type: QuantizerSparseDim and QuantizerDenseDim. Their use is unchanged from v6.

API Breaking changes

v6 v7
Quantizer::configure(dim: Array<any>): Quantizer see below
Quantizer::new(): Quantizer Quantizer {dimensions: Array<QuantizerDim>} normal object creation
Quantizer::dimensions(): int removed, available from dimensions field

6.10

  • migrate mbedtls to 3.6.0
  • add support for Google OAuth2 SSO
  • add protection and raise error for modification of nodeX while iterating over it
  • add support for Key rotation of SSO such as Azure AD
  • add support for Linux AArch64 platform and glibc binaries
  • add executable signature for Apple related binaries
  • add executable signature for Windows related binaries
  • add skeleton for Apple .app
  • cleanup STD GCL files according to newly introduced LSP server with better nullable checkes
  • enable support for weakly formed CSV with nullable empty fields
  • fix http get support for server that do not add binary len property
  • fix ABi protocol for broken Array message
  • fix codegen for SDK Java to allows unaligned version of ABI server and generated code
  • fix support for breaking statement within for-in loops
  • fix rangeSize in nodeTime and nodeList
  • fix CSV support for trailing null values

5.x to 6.x

  • Installation Install GreyCat v6 by visiting here, then:
  • Rename: ~/.greycat/licence to ~/.greycat/license (notice that ‘c’ becomes ‘s’)

API breaking changes

v5 v6
Env::get System::getEnv()
Table::new(2, true) Table::new(2)
Table<T>::new(8, true) Table<T>::new(8)
CSVColumnString CsvColumnString
CSVColumnFloat CsvColumnFloat
CSVColumnInteger CsvColumnInteger
CSVColumnTime CsvColumnTime
CSVColumn CsvColumn
CSVFormat::new() CsvFormat {}
CSVFormat CsvFormat
BoxPlotF64 BoxPlotFloat
PCA::new() PCA {}
GaussianND::new() GaussianND {}
Tensor::new() Tensor {}
Gaussian::new() Gaussian {}
HistogramF64::new() HistogramFloat {}
HistogramI64::new() HistogramInt {}
gaussian.min() gaussian.min
ProgressTracker::new(); p.start(); ProgressTracker { start: time::now() };
p.steps() p.counter
p.throughput() * 1000000 p.speed (speed now is counter / s before it was counter / us)
p.duration() p.duration
.bin extension renamed to .gcb
@library("network"); no longer needed, lib network has been merged in std::io
Directory::new File::mkdir
Directory::open FileWalker::new
FileWriter::new("${filename}.json", false) JsonWriter::new("${filename}.json")
FileWriter::new("${filename}.bin", false) GcbWriter::new("${filename}.gcb")
fw.writeBin(metaTypeIndex) fw.write(metaTypeIndex);
File::open has different semantic now, it returns a file descriptor and not a reader. To read we need a specific type based on the format (eg. JsonReader, CsvReader, etc.)
File::open("${filename}.bin"); GcbReader::new("${filename}.gcb")
file.readBin() file.read()
file.close() file = null;
File::open(csvPath) csvFile = CsvReader::new(csvPath, csvFormat)
csvFile.read(csvFormat) csvFile.read() Ideally replace to csvFile.readTo(obj)
csvFile.name() doesn't exist anymore
csvFile.path() path
use http;, use smtp; use io;
JSON::parseFile(file) JsonReader::new(file.path).read()
gaussian.clear() gaussian = Gaussian{}
var clonedGaussian = gaussian.clone() var clonedGaussian = clone(gaussian)
Task::spawn("module.function", [params]); Task::spawn(module::function, [params]);
SmtpAuthMode SmtpAuth
SmtpConfig {...}; Smtp {...};
Smtp::send(...); var smtp = Smtp {...}; smtp.send(...);
Email {..., contentType: EmailContentType::html, ...} Email {..., body_is_html: true, ...}
@public @permission("public")

  • File iterator

    • Old v5 way:
    for (i, entry in dir) {
  if (entry is Directory) {
    // entry is dir
  } else if (entry is File) {
    // entry is file
  }
}
    • New v6 way:
    var fw = FileWalker::new(path);
if (fw != null) {
  while (!fw.isEmpty()) {
    var entry = fw.next();
    if (entry != null) {
      if (entry.isDir()) {
        // entry is dir
      } else {
        // entry is file
      }
    }
  }
}

  • Update webpack proxy middleware

module.exports = (app) => {
  app.use(
    // ensures all non-GET requests are sent to GreyCat instead of Webpack-dev-server
    createProxyMiddleware((path, req) => req.method !== 'GET' || path.match('^/files'), {
      // your GreyCat endpoint URL
      target: greycatProxy,
    }),
  );
};

  • Get files (JavaScript) See source

  • Upload files (JavaScript) See source

  • Download files in the browser (JavaScript)

const link = document.createElement('a');
link.download = 'myFile.csv'; // the name of the downloaded file in the browser
link.href = '/files/path/to/myFile.csv'; // the actual path to download the file on GreyCat
document.body.appendChild(link);
link.click(); // triggers the download
document.body.removeChild(link);
  • Download file from task result
export const downloadResource = async (task: runtime.Task) => {
  fetch(`/files/${task.user_id}/tasks/${task.task_id}/result.gcb?json`)
    .then((res) => {
      return res.text();
    })
    .then((data) => {
      const fileName = data.trim();
      const link = document.createElement('a');
      link.download = fileName;
      const url = new URL(
        `/files/${task.user_id}/tasks/${task.task_id}/${fileName}`,
        window.location.origin,
      );
      link.href = url.toString();
      document.body.appendChild(link);
      link.click();
      document.body.removeChild(link);
    });
};
  • Task API
const task = await myModule.myTask();
const handler = new TaskHandler(task);
const info = await handler.start(pollingDelayInMs, (info) => {
  // if you want to show progress somewhere
});
// getting files is using the standard "files" API
const result = await greycat.default.getFile(`${task.user_id}/tasks/${task.task_id}/result.gcb`);
const args = await greycat.default.getFile(`${task.user_id}/tasks/${task.task_id}/arguments.gcb`);
  • New Enums

Get an Enum by accessor method:

<module>.<enum>.<enumAccessor>()

e.g.:

const role = tenant.TenantRole.superadmin();

Get an Enum by string:

<module>.<enum>[<string> as <module>.<enum>.Field]()

e.g.:

const field = "superadmin"
const role = tenant.TenantRole[field as tenant.TenantRole.Field]();

Compare Enums Just compare them directly, no need to use value nor field (now called key).

  • New concepts
  • File now is just the file descriptor (name, path, size etc)
  • For every specific format, there is dedicated class for example: JsonReader, CsvReader etc.
  • You can get specific file paths by calling these static methods:
    • File::baseDir() translates to "./gcdata/files/"
    • File::userDir() translates to "./gcdata/files/<user_id>/"
    • File::taskDir() translates to "./gcdata/files/<user_id>/tasks/<task_id>"

where <user_id> and <task_id> are contextual.

  • Code generation

    v5 v6
    greycat-lang gen -o ./src/greycat greycat codegen project.gcl ./src/greycat

  • Styling if using MUI

@import '@greycat/web/css/greycat.base.css';

else

@import '@greycat/web/css/greycat.css';
  • Breaking changes

Can not spawn Tasks from a spawned Task , only one level possible.

Progress tracker in Task is disabled

All usage of structuredClone need to be changed.