The nity-gritty ^(develop…)

This manual is a pre-release preview. Please follow Moonstalk on Twitter or Facebook to be notified of a public release.

• Begin
• Learn
• Develop
  • Tables
    • request
    • form
    • page
    • user
    • response
  • Functions
  • Components
• Applications
• Architecture
• Credits

Tables

These data-structures are universal providing access to the Moonstalk environment, the request, the response, and in many cases also database queries and their results (see next). You may check and change the values in these tables to determine how to handle a request—and to generate a page for response.

All tables exist in either a global or ephemeral (per-request) scope. The latter are discarded after each request is completed and may be modified freely, whilst the prior exist across all requests and should not be modified. Additional reference (pointer) tables are provided in the ephemeral scope for convenient access to some global tables.

Global tables should not be changed during a request, and are intended to only be modified by application load-time functions. Failure to adhere to this will result in random results when using more than a single backend instance. Whilst reference tables change with each request, they are dynamic pointers, or aliases, to the contents of global tables and therefore should also not be changed. No tables are persisted (use a database such as the Teller).

Typically if a controller has many references to, or a view consumes many instances of values in these tables, it is advisable for both cleaner code and better performance to assign them to a local at the top of these files. e.g. local data = page.data local form = request.form

request.

This predominantly contains details of the HTTP headers, with parsing into a more Lua-friendly format where appropriate. Use of this table is discouraged as its format and structure is dependant upon the web server API. See form and page instead.

request.get and request.post are not conditional, instead use request.method=="GET" (case sensisitive).

Ephemeral table

• time
• method = "GET" or "POST" or "HEAD" …
• get = true -- if method=="GET"
• post = true -- if method=="POST"
• head = true -- if method=="HEAD"
• url
• secure
• protocol
• domain
• path = "/My/Account"
• query = {attribute,key=value,…} -- conditional, only present if attributes are from ?attribute+attribute whilst ?key=value& and both may exist simultaneously
• browser
  HTTP User-Agent header.
• client = {ip="address", token=, id=, timezone=, languages=, locale=}
  Originating IP address. Use of user.ip is advised instead.
• referrer
• cookies = {name="value"} -- in openresty this is a metatable
• language
• languages
• locales
• client = {token="cookie-id", ip=ip_address, language=code, timezone=zone_id, locale=locale_id, keychain={},}

request.client.

Contains attributes that lead to authentication, and some present irrespective, providing a place to find both browser and user dervived values.

Ephemeral table

• token = "cookie-id"
• ip = "ip_address"
• language = "code"
• timezone = "zone_id"
• locale = "locale_id"
• keychain = {Key={…},}

request.form.

Contains values encoded in the request.body such as in an HTML form; is always present regardless of method, in OpenResty when not urlencoded scribe.GetPost() must be called to populate it before attempting to consume its values.

Ephemeral table

• name = "value" or {name="name", size=bytes, ['content-type']="mime/type", contents="data"}
  For every input sent as either a GET or POST. If the value is a file the value is an upload table.
• files = { {name="name", size=bytes, ['content-type']="mime/type", content="data"}, }
If a multi-part POST this will contain an array of the parts, this is primarily useful for multi-file uploads in particular using an HTML5 file input with the multiple attribute.

site.

Reference table

• redirect = false or url; default behaviour is to redirect all non-primary domains to the primary, false will disable this, or if set to a URL any domain without a redirect will then use it; the requested path will be appended to the URL (if undesired, terminate the URL with a question mark to transform the path in query string).
  
• See components

page.

Defines the page to be rendered as a response.

Ephemeral table

Do not set new keys in this table — use only the data key.

• address = "my/account"
  This is the normalised request.path used for matching addresses. (Lowercased and transliterated for Latin accents, with leading and trailing slashes removed.)
• transliterated = true
  Indicates if transliteration of the address occurred.
• paths = { my=true, account=true }
  A keyed table for easy lookup of path components (split with forward slash).
• authenticator = "generic.Authenticator", }
  Enables an authenticator function which is run after an address is found (or allocated by a Binder) and before the page is rendered. These may perform any actions but typically authenticate credentials to retrieve data and populate client/user, either simply from cookies such as for language preferences, or from a database for full session management. May be set to a named application's function, true to inherit the site.authenticator or node.authenticator, or false to disable.
• locks = { "name", }
  Typically inherited from an address, but may be set by a curator.
• locale = "us"
  The locale the page is using as derived from user.locales and site.locale.
• language = "en"
  The language the page is using as derived from user.languages and site.vocabulary. Defines the Language header.
• localise = true
  Must be set to enable use of language vocabularies and l.term localisation. May also be set on addresses, bundles (applications and sites) and for node, having corresponding inheritances. Cannot be set in controllers or views, therefore must be enabled for the site if most pages require it, and optionally disabled on pages that do not require it.
• controller = "name"
• view = "name"
• template = "name"
• editors = { function() end, myapp.editor, }
  Array of editor functions to be run after the view and template.
• modified = 1290172118
  Epoch date for the Last-Modified header.
• type
  Constant for the Content-Type header.
• error
  Detail of any page-processing errors. Use scribe.Error() rather than modifying directly.
• headers = { ['Header-Name']="value", }
• status = 200

page.data

Reserved name

• This is reserved for your use to temporarily store values during a request such as upon retrieval and manipulation in a controller, and subsequent consumption in views. Perform assignments, page.data.foo = bar or replace with a database query page.data = db.name.field. Where any contents of the table will be referenced repetitively, create local data = page.data to use them, likewise subtables.

Always declare variables as local else use page.data.

moonstalk.

Global table

• applications

node.

The global Moonstalk environment for each server (node). Settings are configured in the data/configuration/Host.lua file, and may also be changed at launch time by the .Elevator

Settings

• servers = {"application"}
The server ids that need to be running. This defines the webserver (e.g. "openresty" for nginx or "lighttpd") and databases (e.g. "teller", "tarantool"). If not specified these are automatically detected, and will also be added from the database configuration for which "teller" is the default.

• roles = {"name"}
When running multiple nodes, allows an individual node to only handle specific roles, such as specific servers and only the correspondingly associaated database tables.

• scribe = {server="application"}
As above if not specified this will be detected or defaulted.

• curators = {"application", …}
The curators and order in which they should be run for each request.

• applications = {"application", …}
Default applications that must be enabled for all sites.

• logging = 2
  The log level.
• Using logging=4 (detailed information) generally results in a request capacity decrease of 20% compared to using logging=1 (minimal warnings).
• geoip = false
  May be used to enable lookup of a user's locale to supplement the default.

Table (global)

• All unspecified values from the Host.lua file
• geoip
• logging
• sites

scribe. ^(global)

• hits

Databases

Each database application provides its own interface and behaviours. The OpenResty application bundles drivers for MySQL, Redis, and Memcached. Some Moonstalk database applications (e.g. Databin and Tarantool) offer a standardised interface and configuration using a schema.lua file in the application as follows.

schema files can also define enum.name = keyed{…} constants the same as in settings files.

Tables

To define a new database table (as opposed extend another application's tables, e.g. users or tenants), declare it in an application's schema.

The recommended naming is a plural for a table e.g. users = {…}.

The following keys in each database table declaration are supported for each table. See the database application documentation for additional configuration such as field declarations.

• system = "tarantool"
  This declares the application that will manage the table. It should not be changed once declared and initialised by a system.
• server = {…}
  If specified this takes precedence over the below role handling. Parameters are defined by the system application, typically host and password.
• role = "name"
  This declares a generic name used to associate multiple tables with servers, specified as node.databases.role and providing the server parameters. If node.roles contains the role name and the system is specified in node.servers then the server is assumed to be local. See Node.

Functions

For application-specific functions see the applications page.

Content

write

• (expression)
• [[string]]

Appends the passed value to the current output section. Values may only be strings, numbers or nil (which is ignored).

Expressions evaluated by write or ?(expression) tags may only provide string or number values, nil and false values are ignored.

A table or function value will invoke an error as they cannot be represented as text in a response, for debugging you may however output them with tostring().

In views, do not call write instead use an ?(expression) tag.

l.

• key

Returns the given language key's value from a translated vocabulary, in its original case. Use L.key for an initial capital.

This is a proxy table—a function with table.key syntax. (Achieved using a metatable __call overload.)

macro

• { [[text]], key="value", }

Replace defined ?(key) macros in the specified text, with their correspondingly specified values.

Whilst macros in views (expression tags) accept expressions (thus, function calls) for replacement, text macros do not, and where no corresponding key is specified a macro is not replaced.

Text macros are typically used for customising strings, such as translated language keys from vocabularies, where pre-defined grammar is customised using embedded variables.

Page

scribe.Section

• "name"

Changes the current output to a new container, as named. (The default page response output is named content.) See also Cut.

scribe.Mark

• "name"

Marks the current position in a view to later Insert a section or other value.

scribe.Insert

• "section_name"
• "mark_name"

Inserts a previously defined section into the current view/section. Takes an optional mark_name parameter to insert at the specified Mark instead of the current location. See also Paste.

scribe.Page

• "name"

Run a specified controller and/or associated view and append it's output to the current page. Name is normalised (lower-case).

scribe.View

• "name"

Run a specified view only and append it's output to the page. If the view provides versions for different cultures a matching translation/localisation will be served. Name is normalised (lower-case).

scribe.Controller

• "name"

Run a specified controller only. Name is normalised (lower-case).

scribe.Cookie

• ("name", value)
• {name="", value="", expires=, httpOnly=true, path="", domain=""}

Sets a cookie in the user's browser.

Abandonment

The following functions abandon page processing in some way, to display an alternate view or response.

The current function from which these functions are called, and any chaining from it, will continue to run but their output will be surpressed. No further functions will run except for the default template (if any). To prevent the current function from continuing, follow the function call with a return termination.

scribe.Redirect

• "address"

Abandons the page processing to instead returns a 302 redirect response to the user's browser. The address may be a path or URL.

scribe.Authorised

• { "name", }

Checks that the current user has one of the specified keys, otherwise abandons the page processing. If not signed-in the generic/signin view is displayed, or if signed-in but without a specified key, the generic/unauthorised view is displayed.

For a conditional termination use if not scribe.Authorised{"name"} then return end. For conditional content check user.keychain directly.

scribe.Error

• "synopsis"
• {title="synopsis", detail="explanation", realm="page"}

Abandons the page processing with the generic/error view using the specified details.

scribe.NotFound

• ()
• ("name")

Abandons the page processing with the generic/not-found view. Accepts an optional user-friendly name to be displayed instead of the page.address.

Utility

The following functions abandon page processing in some way, to display an alternate view or response.

Lua Modules

All standard Lua libraries are available, with the following, and some other less commonly used libraries that are documented on the credits page.

• JSON4Lua — JSON serialisation and derserialisation
• LuaCrypto — MD5 hashing
• MIME — encoding and decoding
• LFS — filesystem access

You may use load other libraries (.lua or .so) saved in the same folder as a view or controller, or within an application folder by using the full path.

• require "libraryfile"
• require "appname/libraryfile"

Moonstalk uses its own package.path and cpath, therefore if you wish to use other libraries elsewhere on your system (such as from LuaRocks), you must append those paths to Moonstalk's values. Applications may do this in the settings or functions files.

• package.path = package.path .. ";/my/lib/dir/?.lua"

Components

Web server

Moonstalk uses the data/configuration/lighttpd.conf as its base configuration, with which its own configuration is combined at launch-time, and then saved to the temporary/lighttpd.conf file to launch Lighttpd.

Behaviours

• No address may end in .lua.
• Large uploads are rejected if no token cookie is set.
• All addresses on any domain, starting application.id/ serve the corresponding static file (if any) from an application's assets folder.
• Any other address is handled by the Moonstalk pages backend unless an additional application specific configuration applies:
  • For folder sites, on the primary domain (not tenant sites) any address (except as above) containing a period but not having a query string, serves the corresponding static file (if any) directly from disk without any processing by Moonstalk. This includes the source of .html views. If a static file needs to be served from the primary domain with a query string it may be placed in an application, or the web server configuration supplemented.
  • For tenant sites, any address starting tenants/ serves the corresponding static file (if any) from data/tenants/.

Bundles

Behaviours

Views

• If the view defined in an address contains no dynamic markup, then the page.modified value (and thus Last-modified header) is automatically set using the last modification date of the corresponding view's file.false
• If a default template exists it is applied to all addresses, to prevent this explicitly set address template=false or to the name of another template.
• If a view contains an <html> tag, page.template=false. If you wish to run a template controller with such a view, specify the template using an address, or use a custom Collator.
• HTML, Javascript and CSS comments are stripped out when rendered, line numbers preserved.
• The source code as saved for a view can be accessed by anyone simply by appending .html to the address (if an automatic address, otherwise only by guessing its path). Never include code in a view that should be secured, such as logins or private API endpoints. Such code should always be placed in a controller or the values in a settings file. Only .lua files are protected from public access. You could however place all views in a named directory and then prevent access to that directory in the web server configuration.

Other files

• If present any views and controllers placed inside a subfolder named with the application.id of an application, will override its views and controllers with those having the same name in this folder.
• File and folder names starting with _ (underscore) in the site root (but not in subfolders) are ignored by Moonstalk, but may still be accessed as static files via the webserver directly. You may thus use this naming to temporarily prevent views and controllers from loading.
• File or folders whose names start with . (period) are both not loaded and are inaccessible from the web server. This allows a site folder to be defined as a version-controlled project where the versioning data is in a hidden folder (i.e. Mercurial's .hg/ folder).
• Files inside a private folder in the root of the bundle are not loaded (e.g. as views and controllers).

Table

• name
• id
• views
• controllers

Functions

Applications

A bundle whose table exists in the global environment, and having handlers that run at launch-time.

Files

• elevator.lua
  Run once for each application to initialise the elevator environment, prior to the elevator attempting to change the status of any server. May use display and output functions to provide user feedback. For applications managing server processes, should declare elevator.functions.appname as a function handling start, restart, stop and kill commands. May be used to modify the filesystem such as to write out config files for external servers prior to launch. Cannot introspect commands (See Elevator()).

Functions

Initialisation

• Elevator ()
  Run once for each application, after the elevator has completed any actionable commands with handlers defined by elevator.lua. May use display and output functions to provide user feedback. Has no effect in servers.
• Loader ()
  Run once for each application, when it is loading (but other applications, and sites are not ready). Should be used for reading files, loading data, and configuring the global moonstalk environment prior to use by other applications.
• Enabler ()
  Run once for each application, after all applications have been loaded (but not sites, as these are loaded by an Enabler). Should be used for application-dependent configuration in the current server.
• Site ()
  Run for each site, after all applications and sites have been enabled. (By scribe.Starter, thus not before all applications starter's have run.) Provides an opportunity for each application to modify the site or configure their own environment. The function is called per-application for only the applications enabled on the given site, and in the order specified but after their own dependencies.
• Starter ()
  Once the server has configured its environment, and each application's tests and Starter have finished, Moonstalk becomes ready and live. A Starter may therefore be used for connection-dependant functions and configuration, prior to handling live requests, such as in the Scribe to pre-establish a persistent database connection. If employing long-running routines such as upon data, the function should yield() in order that the server may still respond with its status. Note that Moonstalk is not live whilst Starters are running, therefore if routines are dependant upon full readiness (other applications), a Starter may return a finaliser function which will be called after all have completed and Moonstalk is live, finalisers are thus concurrent with (or blocking upon) live request handling, and may be used to bring a node online, whereas a Starter itself would be premature.
• Reader (data,format)
  Run when a view is requested for the first time (with every request during development). Allows changes to the raw file data from disk before being served. Should always return data.
• ready
  Currently application initialisation functions are run regardless of dependancies or the success of preceding functions, this key provides a boolean that allows you to inspect the state of any application's initialisation and exit when required applications are not ready.

Requests

• Curator ()
  May be invoked for each request where no matching site was found in node.sites (e.g. matched by site domains). This provides an opportunity to define sites and behaviours that may vary with every request on every site, such as ephemeral sites stored in a database (as in the Tenants application).
  Curators must be explicitly enabled by specifying node.curators={"application"}.
• Collator ()
  Invoked with each request to define or populate the page. Must return true when successful to prevent other collators from running. Will be enabled by default for all sites that enable the application (site/node.applications), should default addition not be required, the function may be named something else and must be explicitly specified with its full name in site/node.collators {"application.CollatorName"}.
• Editor ()
  Called after the response has been set and concatenated, but before it is returned. Enables modification of the raw output response value.

Behaviours

• View and controller names specified in addresses are assumed to be in the current application, unless prefixed with another's application.id and a slash e.g. appname/name. If prefixed as ~/name the view or controller is assumed to exist in the current site only.
• Bundle names define the application.id and application.name. Any part of the name before a . (period) is ignored.
• When present, a bundle subfolder named application.id/ is used to serve assets via the web server, under the path application.id/ on all domains.

Table

• global = false; set this to prevent the application being made available in the global namespace as _G[id] it is however always available as moonstalk.applications[id]; note that if the application has it's own namespace it will be populated to that

Performance

Enabling applications has no per-request performance overhead except where the following are defined by the application, where the overhead will be specific to the functionality provided. Memory overhead for applications with multiple sites is minimal (sites are populated only with references to the application).

• curators (first matches)
• collators (first matches)
  scribe.Collator:
  1. exact address (no cost)
  2. pattern-matching addresses (all must be tested until one matches)
  3. fallbacks
• editors (all)
  these may performing gsub() or less expensively, use scribe.Paste() which has negligible cost

Sites

Sites are defined by the sites() function of applications. See the Sites Folder or Tenants applications, or any other application with a sites function, for details on how sites may be defined and configured outside the Scribe environment.

Within the Scribe environment, sites are stored in the node.sites table, with references from the global domains table (used to lookup a site for each request) and ephemeral (per-request) site table.

Behaviours

• A canonical metatag using the primary domain is inserted into any HTML page served on a non-primary domain (i.e. when redirect=false). This cannot be disabled without hacking as it is considered a best practice.
• Site assets are currently served only from a site's primary domain. If you use multiple domains without redirection, you must currently use absolute URLs for all your asset references.

Table

These values may be specified in the site's settings.lua file.

• name = "My Site"
  This is displayed as the title of a site when using the generic template, if not specified request.domain is used.
• domain = "example.org"
  By default this is your primary domain and is the domain used for redirection, and canonical links, etc. Manually specifying it is not recommended.
• id = "example.org" this is used internally and not normally consumed, it is the default domain, and may be used to reference site bundles with moonstalk.sites[id].
• domains = { ['example.org']={} }
  Alternate domains that serve the same content as the primary domain (included in this table). Each of these domains is added to the domains table as a reference to the site.
• redirects = { ['www.example.org']=true, }
  Domains that redirect to the primary domain. Any domain specified here will never receive requests, unless intercepted by a curator function.
• language = "en"
  If you have specified multiple vocabularies or language-specific content, you should specify your site's default language for cases where a visitor has no preferred language, and to ensure addresses for language specific content can be indexed by search engines.
• applications = { "[application", }
  If you wish for applications to be enabled in this site, you must specify a list of their names. Order is unimportant unless more than one application attempts to define the same resource (i.e. address) in which case the first specified will be used.
• locks = { "[Name", }
  If all addresses in a site are to be locked, this setting may be specified and all addresses without their own locks specified will use this. Note that addresses with locks do not inherit these.
• collators = { "application", "application.Function", application=false }
  Any named application's Collator (or named function) will be called to find and populate the page, followed by those of any enabled applications unless specified as false, ending with the default scribe.Collator which provides a page from addresses. If no collator provides a page, the scribe falls back to the generic found page.
• token = { expires=seconds }
  This specifies a prototype token cookie uing values that scribe.Cookie accepts. By default it expires after 3 months, and is named token.

Views

Features

• Expression tags
  Markup that functions as an in-line macro for the write() function.
• Processing tags
  

Behaviours

• Except in dev mode, HTML comments are removed, although line breaks within them are preserved. Any expression or processing tags within these comments will also be removed, therefore HTML comments can be used to wrap and run debugging functions.
• If the result of an expression (the value itself or the return from a function call) is nil, an error will be generated. If the value is known to sometimes be missing you can provide an alternative default value within the expression, e.g. ?(record.field or ""), in this case the empty string default would cause the value to simply not appear if missing instead of generating an error.
• White space such as line-breaks inside expression tags is ignored (except in string values).
• The modification date of a view mapped by an address, is used for the Last-Modified HTTP header when serving the response. This enables search engines to keep track of your site's changes to reindex pages, and improve their relevancy in search results. If your view displays content that changes in a database you should specify an appropriate timestamp for page.modified in your controller or view accordingly.
• Local functions defined within a view may not contain output if also contained within a section (define the function elsewhere).
• The functions.html file, may be used to declare reusable global functions that output dynamic HTML blocks used amongst multiple views, and should be declared in an appropriate namespace. Simply wrap the appropriate block with the function declaration processing tags. Local functions should be declared within a single view that reuses them. Blocks with expansive size are better created as standalone views, and thus included with scribe.View"name".

Functions

Table

Controllers

Behaviours

• .lua files that start with #!/usr/bin/env lua (for a CLI command) or contain a line starting module (for a library) are ignored and not loaded as controllers.
• White space such as line-breaks inside expression tags is ignored, unless within a defined string.

Features

Addresses

Features

• Linked views and controllers
  Any address defining a view or controller will also have a corresponding controller or view defined, if one exists with the same name.
• Normalised path matching
  Freely use proper capitalisation and accents in your URLs—paths are both case-insensitive and transliterated for latin accented characters.

Behaviours

• Trailing slashes are ignored. I.e. mypath/ is equivalent to mypath.
• Case is ignored. I.e. MyPath is equivalent to mypath.
• Latin accented characters are ignored. I.e. minväg is equivalent to minvag.
• Unicode filenames are not currently supported with autoaddresses.
• Addresses may not contain the '.' (period) character, as this denotes static content to be served directly by the web server.
• For file names that are automatically mapped to addresses (the default) the source code of views is publicly discoverable. (Logic should be placed in controllers, rather than views; if you have to place sensitive logic in a view you may manually map their addresses to obfuscated file names and set autoaddresses=false.)

Table

Selectors

Defines how to match a request. Only one may be used.

• matches = "path" or { "path1", "path2", }
• starts = "prefix/"
• ends = "/suffix"
• contains = "value"
• pattern = "value"

Handlers

Defines what to do with the request. One or more may be specified per address. You need only specify both controller and view if they are different.

• view = "name"

• May be set to false to prevent a view from being displayed.
• controller = "name"
May be set to false to prevent a controller from running.
• template = "name"
  May be set to to prevent the default template (if any) being used. The default template is never used for complete HTML files.false
• collators = {"name"}
The named functions will be run in the specified order, and may set or change the controller and view or perform other functions prior to their execution. Names may reference functions in the current application or site, or other applications. May be set to false to prevent inheritance and modification by applications, otherwise will inherit site.collator or node.collator as the first. These functions are run after the Binder(), prior to page rendering, and may used for preferences, authentication, and content retrieval.
• redirect = "address"
Must be used exclusively (without any other handler). May be a URL or address. Optionally set redirect_preserve=true to append the path and query_string to the given URL.
• locks = { "key", }
  May specify an array of keys, if none of these matches user.keychain then the user is denied access or (if signed out) asked to signin.
• form = {"…"}
  Set default values for the form table (emulates user submitted input).
• action = "name"
  The name of an action function to run (emulates user submitted input).

Attributes

Addresses may specify any of the attributes of a page, such as template or title.

User

Represents the anonymous visitor that made the request, a profiled visitor (cookie-tracked but anonymous), or a registered user. Managed by the built-in authentication mechanism but may be extended by applications.

Features

• Locale and languages
  Provides access to
• GeoIP
  For upplements the typically unreliable browser-based locale with a network-location dervived locale. Ideally enabled only when most visitors are likely to originate from multiple countries and localised value formatting or translation is required.

Behaviours

• Table

• ip
• token
  The encrypted value of the token cookie, if signed-in or tracked.
• id
  A unique ID number for the user, if signed-in or tracked.
• languages
  An array of the user's preferred locales, derived from the request and user's settings (if any).
• locales
  An array of the user's preferred locales, derived from the request and user's settings (if any).
• keychain

Extend…

To learn how you can extend moonstalk, and make use of pre-packaged functionality, read the applications page, or to understand the Moonstalk's design jump to the architecture page.