Extend ^{(applications…)}

• Generic
• Email
• Sites Folder
• Scribe
• Kit
• Geo
• Manager
• Tenant
• People
• Match Domains
• Openresty
• Tarantool
• Teller

Generic

Provides default generic behaviours, including authentication and session handling. Unlike other applications, it is always loaded for all sites.

Views

• generic/not-found.html
  The 404 error page, shown when any user-requested page cannot be found. The not-found view must specify page.error=false to prevent fallback to the error view.
• generic/unknown.html
  Shown when a domain does not match any site. Displays the node.provider.name and
• generic/error.html
  Shown when any internal error occurs, such as an error in the code syntax of a view, or a database error.
• node.provider.domain if defined.

• generic/signin.html
  Shown when users who are not signed-in, attempt to access a locked (restricted) page without permission.
• generic/unauthorised.html
  Shown when users who are signed-in, attempt to access a locked page without permission.
• generic/account.html
  Default page for signed-in users.
• generic/verify.html
  Shown after a login reminder is requested.

Behaviours

• Creates a localhost site on which it enables itself.
• If used to authenticate a user with one of its authentication functions, will add the user.sessions table, maintain user.last with the timestamp of their last session, and also populate request.client.session.

Functionality

• on addresses: authenticator = "generic.Authenticator"
  Accepts either a built in token cookie to initiate a database query and establish a user session, or a cookie named preferences formatted as language|locale|timezone|… to populate client; lacking these will populate client using the browser requested language, or GeoIP if enabled. Furthermore enables sign-in and sign-out on any page address using a form posted with a field named action having either the value SignIn or SignOut
• action.Signin ()
  Accepts a form with email (containing an valid identification, including telephone or userid) and password, if valid, establishes a user session, otherwise displays the generic/signin page. Also provided at the /Signin address which accepts an email or id parameter as a default value.
• generic.Password ("password")
  Returns a secure, salted and hashed version of the password string for storage in a user record. This is required for use with the Signin method. Note that the salt used is node.secret therefore if this ever changes, all user passwords will be invalidated.
• user.session
user.session.visits=count
request.client.keychain.User
  These are defined for any sucessfully signed-in user
• generic.Session (token)
  Creates a persistent cookie-based sign-in session for the current user using the specified token, the current user.token, or assigns a new one. See people.Adopt and people.Session for assigning sessions to other users.
• generic.Signout ()
  Terminates the user session. Also provided at the /Signout address.
• generic.Email { to="recipient@example.com", cc="recipient@example.com", bcc="recipient@example.com", subject="Test", body="content", server="smtp.example.com", username="login", password="secret", from="sender@example.com", headers={} }
  Sends an email. Defaults for all values except to, cc & bcc, may be specified as per settings below. Recipient addresses may be specified as arrays of addresses, and each address may be specified using the complete [["Name" <address>]] format. Note that this function blocks the scribe from processing other requests whilst communicating with the SMTP server, and should therefore ideally be run as a task, else response to the user will be delayed.

Settings

• node.localhost.applications = { "application", }
  A list of applications to load for the localhost (node default) site.

Email

Provides an abstraction for sending messages, using transports known as a 'courier', and includes a default SMTP courier capable of sending messages without a relay server.†

Features

• Can transcribe messages using template .eml files placed in a bundle's emails/ directory.
• In async servers, maintains a persistent message queue to retry messages (with backoff) when a server is unavailable. Not capable of handling significant backlogs especially of HTML messages or those with attachments due to queue serialisation.

Behaviours

• Message body may utilise macros as expression tags for variables.
• Addresses are specified as strings, either "mailbox@example.com" or "Display Name <mailbox@example.com>", and optionally as an array of these.
• Attributes may be specified in the message, site settings, or Node settings, where a prior takes precedence over a latter.
• Can add a default node.email.message.bcc to all outgoing messages for monitoring, and in dev mode node.email.message.to will capture all messages.

Functionality

A single function wraps underlying behaviours, see in-code comments for further details.

• email.Send
  Send a message by email. Defaults for all values except to‡, cc & bcc, may be specified as per settings. Note that in a synchronous server this function will block processing whilst communicating with the SMTP server which can be slow, and should therefore ideally be run as a task.
  {
  • to="recipient@example.com",
  • cc="recipient@example.com",
  • bcc="recipient@example.com",
  • subject="Test",
  • body="content",
    the following are use-case specific and may also include courier attributes
  • transcribe="appname/eml-name", the message body will be the named .eml file functioning similar to a view, but providing a complete text, html or multipart message body; it should use a code block to set message.headers['Content-Type'] = 'multipart/alternative; boundary="-------abcdefghij"' as appropriate (not required for plain text), and may modify any other message attributes such as message.subject using localised values
  • fields={name="value",…} expression tags in the message body will be replaced with these values
  • defer=false, in async servers this causes a wait upon the dispatch result, in sync servers this may invoke a task (refer to the server documentation)
  • fail="appname.Function", passed the message on failure
  • sent="appname.Function", passed the message on success
  • page="name", render a page as the body using scribe.Page, you may for example place all these views in emails/ directory and thus set the value to "emails/view-name"; may optionally specify site=id to render the view from
    the following are typically set as defaults for the Node and/or site
  • courier="name", e.g. "smtp"
  • from="sender@example.com",
  • headers={},
  }
  

Settings

• site.from = "Display Name <mailbox@address>"
  
• node.email = "courier" e.g. "smtp" but more typically specified as a table of attributes as follow
  
• node.email = { courier="courier" }
  
• node.email.message = default attributes for all messages and couriers
  {
  • to="mailbox@example.com", ‡in dev mode this will capture all outgoing messages and send them to this address instead of any specified recipients
  • … any message attributes
  }
  
• node.email.courier =
{
  • … any message or courier attributes
  }

SMTP Courier

After dispatch message.status will provide the SMTP status code, and message.error where applicable.

• server="domain", optional, e.g. "localhost" or "example.net"; may be set to false on messages to override a node default and use direct dispatch
• sender="mailer-daemon@example.net", return address, defaults to moonstalk@hostname; should be a mailbox that can receive bounces
• username="login", optional
• password="secret", optional
• bind_ip="127.0.0.1", optional, the outgoing IP address for use when connecting to relay or recipient servers

†When no server is specified the message will be dispatched directly to recipient mailservers. This is likely to result in the message being marked as forged, unless you provide a valid sender with reverse DNS for the dispatching server, plus configure SPF as as required.

Direct dispatch has the advantage of providing immediate delivery with confirmation of address validity, rather than being delayed through an intermediary that returns failed messages by email. A successful status with direct delivery can thus be considered as having placed the message in the recipient's mailbox, however most receiving systems use filters and may themselves relay to final mailbox servers so this cannot be assumed as immediate, but is nonetheless faster and more programmatic than using an SMTP relay server.

Sites Folder

Loads bundles as sites from the sites folder, and optional sites folders within applications.

Behaviours

• The bundle name defines site.domain (the primary domain).
• If the primary domain is not prefixed with www an additional domain with the prefix is added, or if prefixed, a domain is added without it. Thus both are always handled.
• Views are automatically mapped to addresses using their filename (without extension), see settings below to disable.

Settings

Any attributes of a site bundle may be specified in the site's settings.lua file, however the following are specific to the settings file, providing simplified syntax. All attributes and their values defined in the site settings file (including your own) are accessible in the site table with each request.

• domains = { "example.net", "wibble.example.com", "example.org" }
  A list of alternate domains that should serve the same content as your primary domain.
• applications = { "manager", }
  If you wish for applications to be enabled in this site, you must specify a list of their names. Order is unimportant unless more than one application attempts to define the same resource (i.e. address) in which case the first specified will be used. Dependancies will be resolved and those applications also enabled.
• autoaddresses = true
  Addresses are automatically added for your views, if you wish to specify addresses manually you must set this to false.
• redirect = true
  The default behaviour with multiple domains is to redirect requests for alternate domains (as listed in domains, or for the 'www' variant of your primary domain if none) — to your primary domain. I.e. if your primary domain is example.com then www.example.com will be redirected to that, or vice-versa. If you specify false, domain redirection is disabled and all domains function as aliases for the same site content. A canonical link is also added to pages on any non-primary domain, referencing your primary domain.
• When using multiple domains with redirect = false, references to assets must currently use absolute URLs to a site's primary domain only. (Use absolute references, or enable the built-in CDN remapping.)

Scribe

This is the core application which provides handling for the page generation cycle, including outside a request-response flow.

Behaviours

• The last-modified header is set to the modification date of the first rendered view, whether as defined by an address, or called using scribe.Page/View but does not apply when a user is authenticated. If multiple views are rendered, page.modified should be set explicitly, such as to an appropriate site.views['name'].modified or the last-modified time of a core dataset being presented.

Views

An interesting side effect of compiling views as first-class Lua functions is that you can declare blocks of HTML as local functions themselves, essentially acting as embedded content block templates. Thus each time you call the function it will render the same layout with different values, however you reference them (arguments, upvalues, or globals). The follow example demonstrates.

table.insert(moonstalk.readers, function(view) end) -- declare a function that may conditionally parse a static view returning a modified content block e.g. for markdown or other formats, or simply to modify all content blocks, such as an alternate templating system; all readers run sequentially in an order determined only by how they're declared >

<html>

1

2

3

</html>

Kit

Provides helper functionality for working with HTML pages, loading and running JavaScript, and facilitating the use of third-party integrations such as Google Analytics.

Kit utilises an editor() and thus its page modifcation functions are retroactive, and may be called at any time (e.g. before any view has even provided page content to modify).

Behaviours

• The javascript-dominoes library (5k) is loaded with every page to allow asynchronous loading of any additional script files.
• A kit and vocabulary library is loaded with every page.
• A language metatag is added with the value of page.language (if any).
• Inserts site.css and page.css into every request.
• In views, all src attributes are made absolute. The root may be specified (such as to use a CDN) with node.base, or this may be set to false to disable this behaviour entirely. This feature enables you to develop views locally with relative asset references, and they will be changed automatically upon deployment.

Functionality

Javascript

• page.javascript.varName = value
  Assign a value to a variable in the client-side JavaScript environment.
• kit.Script ( script )
  Asynchronously load and run a script. The script value may be either a path to an external script file, e.g. script.Load "/assets/script.js", an assignment e.g. script.Load "myvar = 'foo'", or a function to be called after loading e.g. script.Load "myfunc()". You may delcare dependencies to be loaded simultaneously as supplementary parameters, e.g. script.Load ("/moonstalk.kit/jquery.js", "/assets/script.js", "myfunc()") or sequential dependencies as a single string using greater than as a seperator, e.g. script.Load ("/moonstalk.kit/jquery.js > /assets/script.js", "myfunc()"). Expressions should not be terminated with semi-colon.
• site.services.analytics = "id"
  Loads the Google Analytics code with every HTML page when logging is below 4.
• site.services.reinvigorate = "id"
  Loads the Reinvigorate tracker code with every HTML page.

Page

• kit.Head ( tag )
  Add a tag to the head element.
• nocache = true
  Adds an HTTP cache-control header with the value no-cache. May be specified in address or page.
• robots = "none"
  Adds an HTML robots meta tag with the specified value. May be specified in address or page.

Assets

Kit includes a number of useful JavaScript libraries.

• kit.Script "/moonstalk.kit/jquery.js"
• kit.Script "/moonstalk.kit/jquery-ui.js"
• kit.Script "/moonstalk.kit/jquery-hoverIntent.js"
• kit.Script "/moonstalk.kit/jquery-format.js"

Forms

For any key in the page.flags table corresponding to a form input name created using a tag function, it's class is set to error, and if the value of the key is a string an error message is displayed in a span with the id error. The tag functions should be used in server tags not expression tags.

• tag.select ( "name", list, selected, localised )
  Creates an HTML form select input with the specified name, and an option corresponding to each item in the list—having integer values and display names from the list item value, either as a localised string name, or (if localised is false) the value itself. Specify selected using or syntax to provide a default e.g. request.form.name or 2.
• tag.checkbox ( "name",selected )
  Creates an HTML form checkbox input with the specified name, and checked as per selected, which may be specified using or syntax to provide a default e.g. request.form.name or true.
• tag.input ( "name", value ) or tag.text { name="name", value=value, attribute="value", ... }
  Creates an HTML form text input with the specified name and value, or the attributes and their values specified in a table parameter. Specify value using or syntax to provide a default e.g. request.form.name or "value". If no value parameter is specified it will default to request.form[name]
• page.focusfield = "name"
  Sets the browser focus to the form input field with the specified name.
• tag.Error ("name", "message")
  Sets the browser focus to the form input field with the specified name, but preserving any prior focus, and assigns an optional message to be displayed for the corresponding input created with one of the above tag functions.

Formatting

• format.Date ( datetime, format, toggle )
  Format a date using a current locale's named date format, or an arbitrary format string. The date may be provided as a date table (e.g. from localtime), or as a reftime to be converted to the site's localtime. Named formats are "numeric", "long", "short", "abbreviated", and "mini", however not all locales differentiate these. Except for "numeric" (the default if unspecified), dates will be returned as semi-relative dates (weekday and year added/omitted according to age) unless the toggle parameter specifies one of "complete", "year" or "weekday". The associated format.ReferenceDate may be used to output a reftime without conversion to site localtime.
• format.Time ( datetime, format )
  Format a time using the current locale's time format, datetime is handled in the same was as in format.Date. Similarly the associated format.ReferenceTime may be used to output a reftime without conversion.
• tag.time ( datetime )
  Outputs an HTML 5 time tag that will be refreshed client-side with a relative local date (e.g. "just now", "1 week ago", "13th Sept."). This is recommended as it provides a reliable and continuous adjustment using the browser's current timezone.
• format.Number ( number, decimals, locale )
  Format a number or 0, as HTML with the specified decimal places or 2, using the specified locale or user.locale.
• format.Money ( number, decimals, locale )
  Format a monetary value as HTML using the specified locale or user.locale. If no number is provided, displays n/a.
• format.TelPrefix ( number )
  Adds an international prefix or localises the prefix of an internationally-formatted telephone number (with or without punctuation) to the user.locale.

CAPTCHA

• captcha.Generate ()
  Renders a simple math-based captcha question into a request.form. Includes the text and input field. Should be used in a server tag.
• captcha.Validate ()
  Call this in your form validation, if there is an error the form behaviours above will apply.

Geo

The geo application provides geographic reference and manipulation functionality, including RAM cached (<1ms) GeoIP user location lookups, using free country and city level databases. In future this application will also provide GeoNames data.

Configuration

Node

Providing the data files are available, the application is enabled by default to provide a country for the user locale in cases where a request language is not country-specific. To disable specify geo=false in the Node settings, to force lookups for all requests, specify geo=true or geo={lookup=true}.

You may enable country-level resolution for all requests by specifying geo={countries={'cc'}}, and/or city-level resolution by specifying geo={cities={'cc'}} in the same manner, an empty table will load all available data. At the time of writing per-scribe RAM use is 10MB for country level data for ambigious languages, 40MB for UK city-level data, and 250MB RAM for the US.

Sites

• geo = nil
  Lookup for ambigious languages only.
• geo = false
  Never lookup.
• geo = true
  Always lookup.

Data

The following unpacked third-party files must be placed in the data/library/geoip folder.

• GeoIPCountryWhois.csv (~12MB)
  Available from MaxMind, © LGPL. Required for country-level resolution.
• IpToCountry.csv (~8MB)
  Available from Web77. Required for country-level resolution.
• GeoLiteCity-Blocks.csv (~70MB)
  Available from MaxMind, © LGPL. Required for city-level resolution.
• GeoLiteCity-Location.csv (~20MB)
  Available from MaxMind, © LGPL. Required for city-level resolution.

Behaviours

• Requests not having a user-defined language or locale, invoke a GeoIP lookup to define it.
• For lookups resolving to city-level, the request.client.place table is defined.

Functions

• geo.LocateIp ('ip.ad.dr.es')
Perform a lookup for the specified IP address string. Also defines the user.place table.
• geo.Hash (latitude,longitude)
Create a geohash from the defined coordinate numbers or strings.
• geo.Coordinates ('geohash')
Returns latitude, longitude numbers for the specified geohash string.

Tables

• user.place = {city="name", iso3166_2="cc", region="name", geohash="geohash"}

Manager

The manager application provides an adminstration interface for the Moonstalk servers, extensible by other applications. Requires the User application.

Behaviours

• Enables itself on the localhost site.node.secretoperator
• Creates a default operator user if none exists, with a default password of the and a key named .
• Access is possible by any registered user, allowing applications that extend the Manager to restrict access to their pages as appropriate. The Manager's own functions are restricted to users with the operator key.

Content

Enables the retrieval of localisable pages (content objects) from a database (the Teller), and extends the Manager application with functionality to manage these pages. Requires the Tenants application.

Page tables are composed of keys referred to as pieces, each having any valid value(s), including localised tables. A localised table is a table containing a key for each language identifier and a table or string value, but should also contain _localised=true. When a piece is requested corresponding to a localised table, only the value matching the user's preferred language or site language, is returned. If no language matched we also set _localised=false in the response. If the localised table does not contain a _localised flag the corresponding piece name should be specified as "name[localised]".

Configuration

For site folders only, to avoid disassociation of data in the case of a folder name change, it is desirable to provide an ID that is unique and permanent. Define one to the site settings.

• id = guuid

Behaviours

• page.pieces = {"piece"}
  Specify the peices to be retrieved with a specific page. Must be specified before collators are run; from an address would also require collators={content.PageCollator} to be specified.
• page.content_urn = "pattern"
  Specify a match pattern that will be used by the content.PageCollator in place of page.address (the URN) to retrieve a corresponding page.

Scribe Functions

• Pieces ("urn", {"piece"})
  Retrieves the localised named pieces of a page having the given urn. The default pieces are {"meta", "vocabulary", "title", "body"}, however you may specify true to retrieve all (i.e. the same as the page table itself, but without any redundant languages). The following attributes are always returned: created, modified, controller, view, template, css.

TODO: address.authenticator = false or 'application' -- defaults to 'generic'; specifies the application name whose Authenticator function will be used to lookup session data with the request.client.id (the function is not given any parameters) TODO: site.authenticator = false or 'application' -- used by all addresses without an authenticator

Delegates

• NewPageDelegate (page, page_id, site, site_id)

Tenant

This application virtualises sites, storing their settings and content in the Teller database and extends the Manager application, where settings may be managed.

Primarily intended to facilitate developing SaaS applications where users have individual domains, it also enables support for the Content application (or any other per-site/per-user content in a database) with disk-based sites.

Multi-tenanted SaaS

Tenant (virtual) sites have similar functionality as disk-based site folders. They can have per-site application settings and content (using the Content application) but they cannot define addresses (beyond those supported by the Content application), controllers, views, or databases. Nor may they use variables in content, or disable/enable applications.

All virtual sites use a foundation site (node.sites.tenants) with which a virtual site is merged, enabling a configured set of applications for every tenant site. All CoreAPI site attributes are valid within a virtual site and will replace those defined in the foundation site, and it is therefore necessary to ensure arbitrary attributes cannot be specified for virtual sites from any management interface exposed to a tenant.

Generic foundation

Specify the following in Node.lua to customise the generic/unknown view.

provider = { domain="example.com", name="Example SaaS", }

Custom foundation

Specify a site (loaded elsewhere, e.g. from disk using Sites Folder) to be used instead using tenant in Node.lua.

tenant = { applications = {}, template = "name", subdomain = "", }

Features

• Front pages
  Sites can specify their own front pages, either as views or redirects, and may randomise amongst several.
• Caching
  Employs very short-term virtual site caching (1-3 seconds) to optimise for heavy traffic, reducing database queries. Requests under load are thus typically fractionally faster than when under light load. This ensures the ability to respond to requests with fresh data is never compromised.

Behaviours

• Page changes
  As with sites, but the modification time is the time you edit the content of a page in the Manager. This cannot be manipulated.
• Curator
  The tenant application's curator always returns a default site when a site doesn't match, it should therefore always be run last, after any other curators that may provide sites (e.g. the Sites Folder or Match Domains applications. The curator order may be set using node.curators={"sitesfolder","matchdomains","tenant"}.

Teller Events

• event.sites (site, site_id)
• event.domains (domain, site, site_id)

People

Provides storage of user records in the Teller database, with handling for multi-tenant request.client.keychain.

Functionality

• people.DisplayName (user)
Concatenates the first and lastname for output, either for the specified user, or the current user if none.
• people.SplitName ("name")
Returns firstname, lastname for a given user-input name string where the latter may be nil.
• people.New {name=name, email="address", telephone="number", }
Returns a new user object with ID. Accepts any valid keys from the user table, in addition to the above convience keys.
• people.Adopt ()
  Same as New but also updates the current user.

Teller Events

• event.user (user, user_id)
• event.email (email, user, user_id)

Match Domains

Adds support to sites for wildcarded domains, such as to accept requests for sites on a multi-tenanted platform hosted with subdomains. To enable add the following to node/Host.lua.

• node.curators = {"matchdomains",}

For a domain in the domains list of settings.lua, specify their names with the * wildcard character (asterisk) in one of the following manners. Matching is not supported for primary domains (site folder name).

Subdomain

Simply specify a period as the first character of the name.

• domains = {"*.example.com",}
  Catches all subdomains of example.com, but not example.com itself.
• domains = {"prefix.*.example.com",}
  Catches subdomains of example.com having a prefix.

Notes

With OpenResty, any domain containing a wildcard (as opposed starting with) is not declared in NGINX with its own server block, therefore cannot be used to access files, only Moonstalk pages. Its assets should be hosted by an application or another domain.

This application functions simply by parsing all site domains upon startup for a valid pattern, building an array of these domains, and providing a curator function which is utilised to return a corresponding site (if any) for each request.

The Tenants application already supports multi-tenant sub-domains for users via its database, however both applications may still be used together but node.curators should be specified such that matchdomains appears before tenant e.g. node.curators = {"matchdomains", "tenant"}.

OpenResty

Manages the Nginx webserver under the OpenResty framework enabling Lua functions through its lua-nginx module that provides native and asynchronous Lua execution using the LuaJIT interpreter and Lua coroutines hooked into the Nginx event architecture.

Async functions must be called with the Moonstalk coroutine handler moonstalk.Resume(ngx.req.read_body) else the request table can become invalid following the async call. Where provided, instead use Moonstalk's abstractions such as openresty.GetPost(). -- TODO: wrap all the async functions with resume by default?

Not compatible with the official NGINX package and must use the patched version provided with the OpenResty package. If you have the official version you must rename or remove it.

Features

• Transparency
  Automatically configures itself for your sites and applications using best-practices.
• SSL
  Add your certificates in the private/ssl director as public.pem and private.pem.
• Let's Encrypt free certificates. cd /usr/local/moonstalk; certbot certonly --webroot -w sites/example.com/ -d example.com (for as many sites as needed) and make sure to add a cron job for renewals certbot renew --deploy-hook "/usr/local/elevator restart web" so when the certificates renew there will be no downtime; note that the primary site.domain must match the first domain for the certificate.

Files

• data/configuration/nginx.conf any Nginx directives valid in the root may be declared in this file and will override Moonstalk values -- TODO: support other blocks such as http and
• sites/*/private/nginx.conf any Nginx directives valid in a server may override Moonstalk values for the site.
• The following defaults will not be applied and must be declared explictly if desired:
  • client_max_body_size 32k;
• applications/*/private/nginx-servers.conf any Nginx directives valid in a server may define values for each site's server block.
• applications/*/private/nginx-http.conf any Nginx directives valid in the http block such as to define sites.

Functions

• util.Shell() is asynchronous.
• all resty component and module functions are available

Behaviours

• openresty.multipart
  Takes options for multipart handling when permitted by the post attribute on an address.
• request.query
  By default the first 10 values are always parsed into this table. To get them all use request.query = ngx.req.get_uri_args() which defaults to 100. Note however that this function parses arguments without a value, e.g. wibble in the following ?wibble&wobble=wubble as booleans e.g. request.query.wibble==true. However the Moonstalk app, also parses the first as request.query[1]=="wibble" but does not support more than one. It's worth remembering that some platforms where referring traffic append ?clid=token to URLs, therefore if the URL was originally ?wibble+wubble it would become ?wibble+wubble&clid=token

Databin

Must never use the same file with multiple instances or servers. This mechanism is for exclusive use by a single process, per file and does not provide a shared database.

Provides simple in-memory table persistence suitable for low-risk or flow-frequency updates. Uses the efficient Luabins binary serialisation library.

Database tables may be specified in an application's schema.lua and these are then available as db.name.

Not suitable for very large datasets due to save being a blocking process that will take longer the larger the dataset, thus generally unsuitable for on-demand use. With the primary save only upon shutdown there is a risk of data loss unless carried out explicitly and bearing in mind its cost.

Functions

• databin.Load("file") reads the named file, returning its deserialised table
• databin.Reoad("file") reloads data for the named database table
• databin.Save("file", table) saves the named file with the given table serialised, or if no table is provided saves the corresponding database table; fails if no data is provided
• databin.Cache{…}; returns a cache table serialised to disk; provides a convenience wrapper for occasionally generated in-memory caches; see functions.lua comments for usage

Schema table attributes

• system = "databin"
• autoload = {server=true, …)
  by default tables are loaded in every server instance; specifying this will only load it in the named servers
• autosave = "server"
  automated saving (upon shutdown) is restricted to a single authoratative server and only if it is running a single instance
• autoreload = {server=false, …}
  when any server saves the table, all other servers will reload it unless explictly disabled, this includes when the authoratative autosave server shuts down
• loaded = "bundle.function"
  a named function run when the table is loaded or reloaded

Tarantool

Manages database processes for the Tarantool system, through the standard moonstalk schema.lua configuration file, with automatic support for roles and table replication across hosts having the same roles enabled.

If not running Tarantool as root, .

Always shutdown the Tarantool server after the Scribe backends to allow application writes to complete, or use a pool of servers. You will require bespoke orchestration logic with multiple servers.

Features

• Transparency
  Built-in support for connections to the Tarantool database processes, on localhost or other hosts. No configuration or connection management is required.
• Model
  Use Tarantool tuples as Lua tables containing named values.
• Functions
  Define and call Lua functions to run on the database process, as if they were inside your application. Selected attributes of the Scribe environment (e.g. user, site) are available to these functions.
• Lua data types
  Store tables, strings and number. Note that are some behavioural differences for tables, notably that sparse arrays use msgpack.NULL for empty positions such as when deleted, instead of nil.
• Indexes
  You can declare indexes in the schema for each table just as you would in Tarantool.

Configuration

To enable a node as a Tarantool client you must add node.tarantool.role_name = {host="address",port=number,password="secret"} must be configured; password is optional when the server is on the same node and configured by Moonstalk, and port is not required if a unix domain socket path (absolute) which is the default on the same node.

To enable the Tarantool server you must add "tarantool" to node.servers and optionally tarantool={role="role_name"}. Its password will be the node secret with username moonstalk. The server will handle only tables having a corresponding role_name, by default "world".

Databases

To define a new database table (as opposed extend another application's tables, such as users or tenants), declare it in an application's schema.lua file.

The recommended naming is a plural for a table and the singular form for individual record models e.g. users = { record="user", …}.

table_name = {
• system="tarantool",
• [1]="first_field", [2]="second_field", … declare the fieldnames with their Tarantool tuple positions as keys
• indexes = { {name="id", type="hash", parts={1,"unsigned"}}, …}, these declaration use native Tarantool values and reference the fields by their above declared tuple positions
• record = "record_name", declaring a record name allows use of the model functionality
• role = "role_name", when used in clusters, tables will correspond to the corresponding node
}

Tarantool stores records as tuples (arrays) of field values, therefore the order (position) of fields cannot be changed once initialised nor can fields be removed, new fields may be added to make the tuple longer. Field names may be changed (considering that existing values may need manually removing or normalising), and must be changed everywhere referenced.

Tables

the model table provides enumeration of fields

• model.record.field gives its tuple index
• model.record[index] gives its field name (table key)

In Tarantool you can reference fields by name e.g. tuple.field however this has a cost and in high-use scenarios it is cheaper to use tuple[model.record.field].

Scribe Functions

These are called in your pages (views, controllers) to access the database from Scribe backends.

• tarantool.default("appname.FunctionName", argument, …) call a function in the Tarantool instance named default. Functions are declared in application/tarantool.lua
• db.record[index_key] fetch a record
• db.record[index_key] = {…} fetch value as either a Tarantool tuple query result (Lua array), or a Lua table in record form to convert between them
• or local record = db.record[index_key]; record:Save()
• application.Procedure() native call executing the named function in the Tarantool database process; these functions are declared in the file tarantool.lua
• model.record(value) converts between a Lua table and Tarantool tuple matching the model's fields; handles nil fields

Records are Lua tables with schema field names. Null values from Tarantool are nil. When retrieved from db.record they can be modified, and even have non-schema keys added for ephemeral use, as when saved they will be discarded.

Records

Application Functions

Applications may define database functions that run in the Teller (procedures), and thus have unrestricted access to the database. This provides the ability to work with records, such as to iterate over all records (e.g. for search) without the overhead of fetching [all] values into a page first. You call and define these functions as you would a standard function, but you create them in the include/tarantool.lua file.

• application_name.Procedure_name() native call executing the function in the database; returns nil,err on database and connection failures, therefore ensure internal error states return false,err if differentiation is required; must return true if no other value

Database functions are not suitable for long-running complex queries as they block other queries unless you use the Taranool yield behaviour.

Behaviours

Working with hierarchical data tables (nested hashmaps and arrays) contrasts with column and row-based data tables. Consider this when designing your data structures, and optimise for retrieving tables and subtables, instead of individual values.

• Persisted (saved) values must be strings, numbers, booleans or tables. Cyclic references and pointers in persisted values are treated as unique key-value combinations and thus will be duplicated, loosing their prior associations. Keys must be numbers or strings.

Advanced Use

Scribe

The core web server framework, handling the request-response flow.

Developing applications that modify the Scribe can have unintended consequences

TODO: flow diagram with hooks

Features

Functions

• scribe.AddLoader(myapp, "type", function(data) return data end) myapp is the bundle (global namespace), type is the file extension ("lua" for controller, "html" for views) and function is a pointer to a function that will receive the file data, and optionally modify and return it before it is compiled, or set as content

Application Functions

• application.Site(site) called with a site table when a new one is added, permitting the application to configure it or derive data from it; generally only applies with site bundles
• application.Curator()

Behaviours

• Query strings containing an argument named ≈ (almost equals character) treat its value as a token, e.g. /path/?≈=abD34Hj45ghHy343J7K12 (this also works in a mixed attribute query string), invoking an authenticator thus permitting sign-in links to function with any URL. The authenticator is responsible for identifying the type of token, determining its privileges and running corresponding mechanisms.

Advanced Use

Tips and tricks

• for a directory of static assets shared across all domains, simply create an application with a public folder, for example assets/public and all your domains can reference it's contents with src="/assets/public/file.ext"