Extend (applications…)

Generic

Provides default generic behaviours, including authentication and session handling. Unlike other applications, it is always loaded for all sites.

Views

Behaviours

Functionality

Settings

Email

Provides an abstraction for sending messages, using transports known as a 'courier', and includes a default SMTP courier capable of sending messages without a relay server.†

Features

Behaviours

Functionality

A single function wraps underlying behaviours, see in-code comments for further details.

Settings

SMTP Courier

After dispatch message.status will provide the SMTP status code, and message.error where applicable.

†When no server is specified the message will be dispatched directly to recipient mailservers. This is likely to result in the message being marked as forged, unless you provide a valid sender with reverse DNS for the dispatching server, plus configure SPF as as required.

Direct dispatch has the advantage of providing immediate delivery with confirmation of address validity, rather than being delayed through an intermediary that returns failed messages by email. A successful status with direct delivery can thus be considered as having placed the message in the recipient's mailbox, however most receiving systems use filters and may themselves relay to final mailbox servers so this cannot be assumed as immediate, but is nonetheless faster and more programmatic than using an SMTP relay server.

Sites Folder

Loads bundles as sites from the sites folder, and optional sites folders within applications.

Behaviours

Settings

Any attributes of a site bundle may be specified in the site's settings.lua file, however the following are specific to the settings file, providing simplified syntax. All attributes and their values defined in the site settings file (including your own) are accessible in the site table with each request.

Scribe

This is the core application which provides handling for the page generation cycle, including outside a request-response flow.

Behaviours

Views

An interesting side effect of compiling views as first-class Lua functions is that you can declare blocks of HTML as local functions themselves, essentially acting as embedded content block templates. Thus each time you call the function it will render the same layout with different values, however you reference them (arguments, upvalues, or globals). The follow example demonstrates.

table.insert(moonstalk.readers, function(view) end) -- declare a function that may conditionally parse a static view returning a modified content block e.g. for markdown or other formats, or simply to modify all content blocks, such as an alternate templating system; all readers run sequentially in an order determined only by how they're declared >
<html>

1


2


3

</html>

Kit

Provides helper functionality for working with HTML pages, loading and running JavaScript, and facilitating the use of third-party integrations such as Google Analytics.

Kit utilises an editor() and thus its page modifcation functions are retroactive, and may be called at any time (e.g. before any view has even provided page content to modify).

Behaviours

Functionality

Javascript

Page

Assets

Kit includes a number of useful JavaScript libraries.

Forms

For any key in the page.flags table corresponding to a form input name created using a tag function, it's class is set to error, and if the value of the key is a string an error message is displayed in a span with the id error. The tag functions should be used in server tags not expression tags.

Formatting

CAPTCHA

Geo

The geo application provides geographic reference and manipulation functionality, including RAM cached (<1ms) GeoIP user location lookups, using free country and city level databases. In future this application will also provide GeoNames data.

Configuration

Node

Providing the data files are available, the application is enabled by default to provide a country for the user locale in cases where a request language is not country-specific. To disable specify geo=false in the Node settings, to force lookups for all requests, specify geo=true or geo={lookup=true}.

You may enable country-level resolution for all requests by specifying geo={countries={'cc'}}, and/or city-level resolution by specifying geo={cities={'cc'}} in the same manner, an empty table will load all available data. At the time of writing per-scribe RAM use is 10MB for country level data for ambigious languages, 40MB for UK city-level data, and 250MB RAM for the US.

Sites

Data

The following unpacked third-party files must be placed in the data/library/geoip folder.

Behaviours

Functions

Tables

Manager

The manager application provides an adminstration interface for the Moonstalk servers, extensible by other applications. Requires the User application.

Behaviours

Content

Enables the retrieval of localisable pages (content objects) from a database (the Teller), and extends the Manager application with functionality to manage these pages. Requires the Tenants application.

Page tables are composed of keys referred to as pieces, each having any valid value(s), including localised tables. A localised table is a table containing a key for each language identifier and a table or string value, but should also contain _localised=true. When a piece is requested corresponding to a localised table, only the value matching the user's preferred language or site language, is returned. If no language matched we also set _localised=false in the response. If the localised table does not contain a _localised flag the corresponding piece name should be specified as "name[localised]".

Configuration

For site folders only, to avoid disassociation of data in the case of a folder name change, it is desirable to provide an ID that is unique and permanent. Define one to the site settings.

Behaviours

Scribe Functions

TODO: address.authenticator = false or 'application' -- defaults to 'generic'; specifies the application name whose Authenticator function will be used to lookup session data with the request.client.id (the function is not given any parameters) TODO: site.authenticator = false or 'application' -- used by all addresses without an authenticator

Delegates

Tenant

This application virtualises sites, storing their settings and content in the Teller database and extends the Manager application, where settings may be managed.

Primarily intended to facilitate developing SaaS applications where users have individual domains, it also enables support for the Content application (or any other per-site/per-user content in a database) with disk-based sites.

Multi-tenanted SaaS

Tenant (virtual) sites have similar functionality as disk-based site folders. They can have per-site application settings and content (using the Content application) but they cannot define addresses (beyond those supported by the Content application), controllers, views, or databases. Nor may they use variables in content, or disable/enable applications.

All virtual sites use a foundation site (node.sites.tenants) with which a virtual site is merged, enabling a configured set of applications for every tenant site. All CoreAPI site attributes are valid within a virtual site and will replace those defined in the foundation site, and it is therefore necessary to ensure arbitrary attributes cannot be specified for virtual sites from any management interface exposed to a tenant.

Generic foundation

Specify the following in Node.lua to customise the generic/unknown view.

provider = { domain="example.com", name="Example SaaS", }

Custom foundation

Specify a site (loaded elsewhere, e.g. from disk using Sites Folder) to be used instead using tenant in Node.lua.

tenant = { applications = {}, template = "name", subdomain = "", }

Features

Behaviours

Teller Events

People

Provides storage of user records in the Teller database, with handling for multi-tenant request.client.keychain.

Functionality

Teller Events

Match Domains

Adds support to sites for wildcarded domains, such as to accept requests for sites on a multi-tenanted platform hosted with subdomains. To enable add the following to node/Host.lua.

For a domain in the domains list of settings.lua, specify their names with the * wildcard character (asterisk) in one of the following manners. Matching is not supported for primary domains (site folder name).

Subdomain

Simply specify a period as the first character of the name.

Notes

With OpenResty, any domain containing a wildcard (as opposed starting with) is not declared in NGINX with its own server block, therefore cannot be used to access files, only Moonstalk pages. Its assets should be hosted by an application or another domain.

This application functions simply by parsing all site domains upon startup for a valid pattern, building an array of these domains, and providing a curator function which is utilised to return a corresponding site (if any) for each request.

The Tenants application already supports multi-tenant sub-domains for users via its database, however both applications may still be used together but node.curators should be specified such that matchdomains appears before tenant e.g. node.curators = {"matchdomains", "tenant"}.

OpenResty

Manages the Nginx webserver under the OpenResty framework enabling Lua functions through its lua-nginx module that provides native and asynchronous Lua execution using the LuaJIT interpreter and Lua coroutines hooked into the Nginx event architecture.

Async functions must be called with the Moonstalk coroutine handler moonstalk.Resume(ngx.req.read_body) else the request table can become invalid following the async call. Where provided, instead use Moonstalk's abstractions such as openresty.GetPost(). -- TODO: wrap all the async functions with resume by default?

Not compatible with the official NGINX package and must use the patched version provided with the OpenResty package. If you have the official version you must rename or remove it.

Features

Files

Functions

Behaviours

Databin

Must never use the same file with multiple instances or servers. This mechanism is for exclusive use by a single process, per file and does not provide a shared database.

Provides simple in-memory table persistence suitable for low-risk or flow-frequency updates. Uses the efficient Luabins binary serialisation library.

Database tables may be specified in an application's schema.lua and these are then available as db.name.

Not suitable for very large datasets due to save being a blocking process that will take longer the larger the dataset, thus generally unsuitable for on-demand use. With the primary save only upon shutdown there is a risk of data loss unless carried out explicitly and bearing in mind its cost.

Functions

Schema table attributes

Tarantool

Manages database processes for the Tarantool system, through the standard moonstalk schema.lua configuration file, with automatic support for roles and table replication across hosts having the same roles enabled.

If not running Tarantool as root, .

Always shutdown the Tarantool server after the Scribe backends to allow application writes to complete, or use a pool of servers. You will require bespoke orchestration logic with multiple servers.

Features

Configuration

To enable a node as a Tarantool client you must add node.tarantool.role_name = {host="address",port=number,password="secret"} must be configured; password is optional when the server is on the same node and configured by Moonstalk, and port is not required if a unix domain socket path (absolute) which is the default on the same node.

To enable the Tarantool server you must add "tarantool" to node.servers and optionally tarantool={role="role_name"}. Its password will be the node secret with username moonstalk. The server will handle only tables having a corresponding role_name, by default "world".

Databases

To define a new database table (as opposed extend another application's tables, such as users or tenants), declare it in an application's schema.lua file.

The recommended naming is a plural for a table and the singular form for individual record models e.g. users = { record="user", …}.

Tarantool stores records as tuples (arrays) of field values, therefore the order (position) of fields cannot be changed once initialised nor can fields be removed, new fields may be added to make the tuple longer. Field names may be changed (considering that existing values may need manually removing or normalising), and must be changed everywhere referenced.

Tables

Scribe Functions

These are called in your pages (views, controllers) to access the database from Scribe backends.

Records are Lua tables with schema field names. Null values from Tarantool are nil. When retrieved from db.record they can be modified, and even have non-schema keys added for ephemeral use, as when saved they will be discarded.

Records

Application Functions

Applications may define database functions that run in the Teller (procedures), and thus have unrestricted access to the database. This provides the ability to work with records, such as to iterate over all records (e.g. for search) without the overhead of fetching [all] values into a page first. You call and define these functions as you would a standard function, but you create them in the include/tarantool.lua file.

Database functions are not suitable for long-running complex queries as they block other queries unless you use the Taranool yield behaviour.

Behaviours

Working with hierarchical data tables (nested hashmaps and arrays) contrasts with column and row-based data tables. Consider this when designing your data structures, and optimise for retrieving tables and subtables, instead of individual values.

Advanced Use

Scribe

The core web server framework, handling the request-response flow.

Developing applications that modify the Scribe can have unintended consequences

TODO: flow diagram with hooks

Features

Functions

Application Functions

Behaviours

Advanced Use

Tips and tricks