Extend (applications…)

Generic

Provides default generic behaviours, including authentication and session handling. Unlike other applications, it is always loaded for all sites.

Views

Behaviours

Functionality

Settings

Email

Provides an abstraction for sending messages, using transports known as a 'courier', and includes a default SMTP courier capable of sending messages without a relay server.†

Behaviours

Functionality

A single function wraps underlying behaviours, see in-code comments for further details.

Settings

SMTP Courier

After dispatch message.status will provide the SMTP status code, and message.error where applicable.

†When no server is specified the message will be dispatched directly to recipient mailservers. This is likely to result in the message being marked as forged, unless you provide a valid sender with reverse DNS for the dispatching server, plus configure SPF as as required.

Direct dispatch has the advantage of providing immediate delivery with confirmation of address validity, rather than being delayed through an intermediary that returns failed messages by email. A successful status with direct delivery can thus be considered as having placed the message in the recipient's mailbox, however most receiving systems use filters and may themselves relay to final mailbox servers so this cannot be assumed as immediate, but is nonetheless faster and more programmatic than using an SMTP relay server.

Sites Folder

Loads bundles in the sites folder, as sites.

Behaviours

Settings

Any attributes of a site bundle may be specified in the site's settings.lua file, however the following are specific to the settings file, providing simplified syntax. All attributes and their values defined in the site settings file (including your own) are accessible in the site table with each request.

Scribe

This is the core application which provides handling for the page generation cycle, including outside a request-response flow.

Behaviours

Views

An interesting side effect of compiling views as first-class Lua functions is that you can declare blocks of HTML as local functions themselves, essentially acting as embedded content block templates. Thus each time you call the function it will render the same layout with different values, however you reference them (arguments, upvalues, or globals). The follow example demonstrates.

table.insert(moonstalk.readers, function(view) end) -- declare a function that may conditionally parse a static view returning a modified content block e.g. for markdown or other formats, or simply to modify all content blocks, such as an alternate templating system; all readers run sequentially in an order determined only by how they're declared >
<html>

1


2


3

</html>

Kit

Provides helper functionality for working with HTML pages, loading and running JavaScript, and facilitating the use of third-party integrations such as Google Analytics.

Kit utilises an editor() and thus its page modifcation functions are retroactive, and may be called at any time (e.g. before any view has even provided page content to modify).

Behaviours

Functionality

Javascript

Page

Assets

Kit includes a number of useful JavaScript libraries.

Forms

For any key in the page.flags table corresponding to a form input name created using a tag function, it's class is set to error, and if the value of the key is a string an error message is displayed in a span with the id error. The tag functions should be used in server tags not expression tags.

Formatting

CAPTCHA

Geo

The geo application provides geographic reference and manipulation functionality, including RAM cached (<1ms) GeoIP user location lookups, using free country and city level databases. In future this application will also provide GeoNames data.

Configuration

Node

Providing the data files are available, the application is enabled by default to provide a country for the user locale in cases where a request language is not country-specific. To disable specify geo=false in the Node settings, to force lookups for all requests, specify geo=true or geo={lookup=true}.

You may enable country-level resolution for all requests by specifying geo={countries={'cc'}}, and/or city-level resolution by specifying geo={cities={'cc'}} in the same manner, an empty table will load all available data. At the time of writing per-scribe RAM use is 10MB for country level data for ambigious languages, 40MB for UK city-level data, and 250MB RAM for the US.

Sites

Data

The following unpacked third-party files must be placed in the data/library/geoip folder.

Behaviours

Functions

Tables

Manager

The manager application provides an adminstration interface for the Moonstalk servers, extensible by other applications. Requires the User application.

Behaviours

Content

Enables the retrieval of localisable pages (content objects) from a database (the Teller), and extends the Manager application with functionality to manage these pages. Requires the Tenants application.

Page tables are composed of keys referred to as pieces, each having any valid value(s), including localised tables. A localised table is a table containing a key for each language identifier and a table or string value, but should also contain _localised=true. When a piece is requested corresponding to a localised table, only the value matching the user's preferred language or site language, is returned. If no language matched we also set _localised=false in the response. If the localised table does not contain a _localised flag the corresponding piece name should be specified as "name[localised]".

Configuration

For site folders only, to avoid disassociation of data in the case of a folder name change, it is desirable to provide an ID that is unique and permanent. Define one to the site settings.

Behaviours

Scribe Functions

TODO: address.authenticator = false or 'application' -- defaults to 'generic'; specifies the application name whose Authenticator function will be used to lookup session data with the client.id (the function is not given any parameters) TODO: site.authenticator = false or 'application' -- used by all addresses without an authenticator

Delegates

Tenant

This application virtualises sites, storing their settings and content in the Teller database and extends the Manager application, where settings may be managed.

Primarily intended to facilitate developing SaaS applications where users have individual domains, it also enables support for the Content application (or any other per-site/per-user content in a database) with disk-based sites.

Multi-tenanted SaaS

Tenant (virtual) sites have similar functionality as disk-based site folders. They can have per-site application settings and content (using the Content application) but they cannot define addresses (beyond those supported by the Content application), controllers, views, or databases. Nor may they use variables in content, or disable/enable applications.

All virtual sites use a foundation site (node.sites.tenants) with which a virtual site is merged, enabling a configured set of applications for every tenant site. All CoreAPI site attributes are valid within a virtual site and will replace those defined in the foundation site, and it is therefore necessary to ensure arbitrary attributes cannot be specified for virtual sites from any management interface exposed to a tenant.

Generic foundation

Specify the following in Node.lua to customise the generic/unknown view.

provider = { domain="example.com", name="Example SaaS", }

Custom foundation

Specify a site (loaded elsewhere, e.g. from disk using Sites Folder) to be used instead using tenant in Node.lua.

tenant = { applications = {}, template = "name", subdomain = "", }

Features

Behaviours

Teller Events

People

Provides storage of user records in the Teller database, with handling for multi-tenant client.keychain.

Functionality

Teller Events

Match Domains

Adds support to sites for wildcard sub-domains (i.e. DNS CNAME) and pattern-matched domains. To enable add the following to settings.lua.

To enable matches for a domain in the domains list of settings.lua, specify their names in one of the following manners. Matching is not supported on primary domains (site folder name).

Subdomain

Simply specify a period as the first character of the name.

This example would catch all subdomains of example.com, but not example.com itself unless specified as the primary domain / site folder name.

Pattern

Specify the name using Lua pattern matching syntax. Be sure to escape the periods in the name itself.

This example is the equivalent pattern syntax of the prior.

Notes

This application functions simply by parsing all site domains upon startup for a valid pattern, bulding an array of these domains, and providing a curator function which is utilised to return a corresponding site (if any) for each request.

The Tenants application already supports multi-tenant sub-domains for users via its database, however both applications may still be used together but node.curators should be specified such that matchdomains appears before tenant e.g. node.curators = {"matchdomains", "tenant"}.

Openresty

Manages the Nginx webserver under the OpenResty framework enabling Lua functions through its lua-nginx module that provides native and asynchronous Lua execution using the LuaJIT interpreter and Lua coroutines hooked into the Nginx event architecture.

Not compatible with the official NGINX package and must use the patched version provided with the OpenResty package. If you have the official version you must rename or remove it.

Features

Files

Functions

Databin

Provides simple in-memory table persistence suitable for low-risk updates, but only occasional saves as each serialises the entire table.

Not suitable for very large datasets due to save being a blocking process that will take longer the larger the dataset, thus generally unsuitable for on-demand use. With the primary save only upon shutdown there is a risk of data loss unless carried out explicitly and bearing in mind its cost.

Functions

Tarantool

Manages database processes for the Tarantool system, through the standard moonstalk schema.lua configuration file, with automatic support for roles and table replication across hosts having the same roles enabled.

.

Always shutdown the Tarantool server after the Scribe backends to allow application writes to complete, or use a pool of servers. You will require bespoke orchestration logic with multiple servers.

Features

Tables

To define a new database table (as opposed extend another application's tables, such as users or tenants), declare it in an application's schema.lua file.

Tarantool stores records as tuples (arrays) of field values, therefore the order (position) of fields cannot be changed once initialised nor can fields be removed, however field names may be changed (considering that existing values may need manually removing or normalising), and new fields may be added to make the tuple longer.

The recommended naming is a plural for a table and the singular form for individual record models e.g. users = { record="user", …}.

Scribe Functions

These are called in your pages (views, controllers) to access the database from Scribe backends.

Records are Lua tables with schema field names. Null values from Tarantool are nil. When retrieved from db.record they can be modified, and even have non-schema keys added for ephemeral use, as when saved they will be discarded.

Records

Application Functions

Applications may define database functions that run in the Teller (procedures), and thus have unrestricted access to the database. This provides the ability to work with records, such as to iterate over all records (e.g. for search) without the overhead of fetching [all] values into a page first. You call and define these functions as you would a standard function, but you create them in the include/tarantool.lua file.

Database functions are not suitable for long-running complex queries as they block other queries unless you use the Taranool yield behaviour.

Behaviours

Working with hierarchical data tables (nested hashmaps and arrays) contrasts with column and row-based data tables. Consider this when designing your data structures, and optimise for retrieving tables and subtables, instead of individual values.

Advanced Use

Teller

Runs a new server process providing data persistence (storage) and queries (retrieval). For general usage see the tutorial.

This application supports and is required by many of the bundled Moonstalk applications. Use of an alternative database system with Teller-compatible applications would require an application implementing the Teller database interface functions.

Always shutdown the Teller process after the Scribe backends and only using signal 15 (use ./elevator stop teller) otherwise current queries may not be able to complete and data could be lost before final persistence and replication is carried out.

Features

Tables

To define a new database table (as opposed extend another application's tables, such as users or tenants), declare it in an application's settings file.

Teller database tables are global keys (names) and therefore a key for any database value will always begin with a database table name, and which cannot conflict with any other global name (i.e. application names, and Moonstalk tables or functions).

Scribe Functions

These are called in your pages (views, controllers) to access the database from Scribe backends.

Application Functions

Applications may define database functions that run in the Teller (procedures), and thus have unrestricted access to the database. This provides the ability to work with records, such as to iterate over all records (e.g. for search) without the overhead of fetching [all] values into a page first. You call and define these functions as you would a standard function, but you create them in the database.lua file.

Database functions accept a maximum of 3 parameters, and may return a single value only (use of a single table is recommended in both cases). Returning nil,"message" invokes the error view with the provided message.

Database functions are not suitable for long-running complex queries as they block queries from other backends. You may however run procedures in separate threads (see Tasks application), or you may periodically release the CPU (such as with every x iterations) using coroutine.yield() to allow the database to interleave the servicing of other requests with your procedure.

Teller Functions

These are used with your database functions to work directly in the database. Keys must be specified as serialised Lua strings (using the varkey function as necessary).

Delegation

Provides event-type behaviour such that when operations take place that other applications may be dependant upon, any application may catch the event notification by simply defining a function with the corresponding name. The parameters are dictated by the application that instigates the delegation and carrying out the operation. The name of the delegate functions may be the same as the instigating function (e.g. DeleteUser), or different to better distinguish the purpose (e.g. DeleteUserDelegate).

The primary use of delegation is to maintain indexes.

It is essential to note that any indexed table (i.e. a 'record' table that is pointed to from other 'index' tables), must not have any new new table assignment made to it, e.g. don't use index[1] = users[1] and then save("users[1]",{…}) as the index will still point to the old table unless the delegate mechanisms update it accordingly (thus in this example one must also call teller.Delegate("IndexUser",id) to create pointers to this new table that replaced the old. This also applies to any subtable. To update values within such a table without replacing, you must use the teller.Update function.

Behaviours

Working with hierarchical data tables (nested hashmaps and arrays) contrasts with column and row-based data tables. Consider this when designing your data structures, and optimise for retrieving tables and subtables, instead of individual values.

Advanced Use

Database table keys are specified using native Lua table syntax, however this cannot be used to retrieve the entire database table or a root key (e.g. fetch(database)). However the key may be specified instead as a string, e.g. fetch "database", and for which hierarchical keys incorporating variables may use the following function.