The nity-gritty ^(develop…)

This manual is a pre-release preview. Please follow Moonstalk on Twitter or Facebook to be notified of a public release.

• Begin
• Learn
• Develop
  • Tables
    • request
    • form
    • page
    • user
    • response
  • Functions
  • Components
• Applications
• Architecture
• Credits

Tables

These data-structures are universal providing access to the Moonstalk environment, the request, the response, and in many cases also database queries and their results (see next). You may check and change the values in these tables to determine how to handle a request—and to generate a page for response.

All tables exist in either a global or ephemeral (per-request) scope. The latter are discarded after each request is completed and may be modified freely, whilst the prior exist across all requests and should not be modified. Additional reference (pointer) tables are provided in the ephemeral scope for convenient access to some global tables.

Global tables should not be changed during a request, and are intended to only be modified by application load-time functions. Failure to adhere to this will result in random results when using more than a single backend instance. Whilst reference tables change with each request, they are dynamic pointers, or aliases, to the contents of global tables and therefore should also not be changed. No tables are persisted (use a database such as the Teller).

request.

This predominantly contains details of the HTTP headers, with parsing into a more Lua-friendly format where appropriate. Use of this table is discouraged as its format and structure is dependant upon the web server API. See form and page instead.

request.get and request.post are not conditional, instead use request.method=="GET" (case sensisitive).

Ephemeral table

• time
• cputime
• method = "GET" or "POST" or "HEAD" …
• get = {attribute,key=value,…} -- conditional, onyl present if attributes are from ?attribute+attribute whilst ?key=value& and both may exist simultaneously
• post = {field=value,…}
• url
• secure
• protocol
• domain
• path = "/My/Account"
• query
• browser
  HTTP User-Agent header.
• client
  Originating IP address. Use of user.ip is advised instead.
• get = { name="value", }
  Present only if method="GET". Use of form is advised instead.
• post = { name="value", post_data="data", }
  Present only if method="POST". Use of form is advised instead.
• referrer
• cookies
• language
• languages
• locales

form.

Contains merged values from both request.get and request.post (latter has precedence), but is always present regardless of method.

Ephemeral table

• name = "value" or {name="name", size=bytes, ['content-type']="mime/type", contents="data"}
  For every input sent as either a GET or POST. If the value is a file the value is an upload table.
• uploads = { {name="name", size=bytes, ['content-type']="mime/type", content="data"}, }
If a multi-part POST this will contain an array of the parts, this is primarily useful for multi-file uploads in particular using an HTML5 file input with the multiple attribute.

site.

Reference table

• See components

page.

Defines the page to be rendered as a response.

Ephemeral table

• address = "my/account"
  This is the normalised request.path used for matching addresses. (Lowercased and transliterated for Latin accents, with leading and trailing slashes removed.)
• transliterated = true
  Indicates if tranliteration of the address occured.
• paths = { my=true, account=true }
  A keyed table for easy lookup of path components (split with forward slash).
• locks = { "name", }
  Typically inherited from an address, but may be set by a curator.
• locale = "us"
  The locale the page is using as derived from user.locales and site.locale.
• language = "en"
  The language the page is using as derived from user.languages and site.vocabulary. Defines the Language header.
• polyglot = false
  Denotes if the page uses more than one language.
• collators = { function() end, }
  Array of collator functions to be run before the controller.
• controller = "name"
• view = "name"
• template = "name"
• editors = { function() end, myapp.editor, }
  Array of editor functions to be run after the view and template.
• modified = 1290172118
  Epoch date for the Last-Modified header.
• type
  Constant for the Content-Type header.
• error
  Detail of any page-processing errors. Use scribe.Error() rather than modifying directly.
• headers = { ['Header-Name']="value", }
• status = 200

temp.

Ephemeral table

• This table should be used as a namespace-safe table for custom global values defined by a controller or server processing tag, but consumed in a view or later function.

Always declare variables as local or use the temp table.

moonstalk.

Global table

• applications

node.

The global Moonstalk environment for each server (node). Settings are configured in the data/configuration/Host.lua file, and may also be changed at launch time by the .Elevator

Settings

• servers = {"application"}
The server ids that need to be running. This defines the webserver (e.g. "openresty" for nginx or "lighttpd") and databases (e.g. "teller", "tarantool"). If not specified these are automatically detected, and will also be added from the database configuration for which "teller" is the default.

• database = {default="application"}
As above if not specified this will be detected or defaulted.

• roles = {"name"}
When running multiple nodes, allows an individual node to only handle specific roles, such as specific servers and only the correspondingly associaated database tables.

• scribe = {server="application"}
As above if not specified this will be detected or defaulted.

• curators = {"application", …}
The curators and order in which they should be run for each request.

• applications = {"application", …}
Default applications that must be enabled for all sites.

• logging = 2
  The log level.
• Using logging=4 (detailed information) generally results in a request capacity decrease of 20% compared to using logging=1 (minimal warnings).
• geoip = false
  May be used to enable lookup of a user's locale to supplement the default.

Table (global)

• All unspecified values from the Host.lua file
• geoip
• logging
• sites

scribe. ^(global)

• hits

Databases

Each database application provides its own interface and behaviours. The OpenResty application bundles drivers for MySQL, Redis, and Memcached. Some Moonstalk database applications (e.g. Databin and Tarantool) offer a standardised interface and configuration using a schema.lua file in the application as follows.

Tables

To define a new database table (as opposed extend another application's tables, e.g. users or tenants), declare it in an application's schema.

• table_name = { system="name" }

The recommended naming is a plural for a table e.g. users = {…}.

The following keys in each database table declaration are supported for each table. See the database application documentation for additional configuration such as field declarations.

• system = "tarantool"
  This declares the application that will manage the table. It should not be changed once declared and initialised by a system.

Functions

For application-specific functions see the applications page.

Content

write

• (expression)
• [[string]]

Appends the passed value to the current output section. Values may only be strings, numbers or nil (which is ignored).

A table or function value will invoke an error as they cannot be respresented as text in a response, for debugging you may however output them with tostring().

In views, do not call write from a ?() expression tag, instead call it from a <? ?> processing tag.

l.

• key

Returns the given language key's value from a translated vocabulary, in its original case. Use L.key for an initial capital.

This is a proxy table—a function with table.key syntax. (Achieved using a metatable __call overload.)

macro

• { [[text]], key="value", }

Replace defined ?(key) macros in the specified text, with their correspondingly specified values.

Whilst macros in views (expression tags) accept expressions (thus, function calls) for replacement, text macros do not, and where no corresponding key is specified a macro is not replaced.

Text macros are typically used for customising strings, such as translated language keys from vocabularies, where pre-defined grammar is customised using embedded variables.

Page

scribe.Section

• "name"

Changes the current output to a new container, as named. (The default page response output is named content.) See also Cut.

scribe.Mark

• "name"

Marks the current position in a view to later Insert a section or other value.

scribe.Insert

• "section_name"
• "mark_name"

Inserts a previously defined section into the current view/section. Takes an optional mark_name parameter to insert at the specified Mark instead of the current location. See also Paste.

scribe.Page

• "name"

Run a specified controller and/or associated view and append it's output to the current page.

scribe.View

• "name"

Run a specified view only and append it's output to the page. If the view provides versions for different cultures a matching translation/localisation will be served.

scribe.Controller

• "name"

Run a specified controller only.

scribe.Cookie

• ("name", value)
• {name="", value="", expires=, httpOnly=true, path="", domain=""}

Sets a cookie in the user's browser.

Abandonment

The following functions abandon page processing in some way, to display an alternate view or response.

The current function from which these functions are called, and any chaining from it, will continue to run but their output will be surpressed. No further functions will run except for the default template (if any). To prevent the current function from continuing, follow the function call with a return termination.

scribe.Redirect

• "address"

Abandons the page processing to instead returns a 302 redirect response to the user's browser. The address may be a path or URL.

scribe.Authorised

• { "name", }

Checks that the current user has one of the specified keys, otherwise abandons the page processing. If not signed-in the generic/signin view is displayed, or if signed-in but without a specified key, the generic/unauthorised view is displayed.

For a conditional termination use if not scribe.Authorised{"name"} then return end. For conditional content check user.keychain directly.

scribe.Error

• "synopsis"
• {title="synopsis", detail="explanation", realm="page"}

Abandons the page processing with the generic/error view using the specified details.

scribe.NotFound

• ()
• ("name")

Abandons the page processing with the generic/not-found view. Accepts an optional user-friendly name to be displayed instead of the page.address.

Utility

The following functions abandon page processing in some way, to display an alternate view or response.

Lua Modules

All standard Lua libraries are available, with the following, and some other less commonly used libraries that are documented on the credits page.

• JSON4Lua — JSON serialisation and derserialisation
• LuaCrypto — MD5 hashing
• MIME — encoding and decoding
• LFS — filesystem access

You may use load other libraries (.lua or .so) saved in the same folder as a view or controller, or within an application folder by using the full path.

• require "libraryfile"
• require "appname/libraryfile"

Moonstalk uses its own package.path and cpath, therefore if you wish to use other libraries elsewhere on your system (such as from LuaRocks), you must append those paths to Moonstalk's values. Applications may do this in the settings or functions files.

• package.path = package.path .. ";/my/lib/dir/?.lua"

Components

Web server

Moonstalk uses the data/configuration/lighttpd.conf as its base configuration, with which its own configuration is combined at launch-time, and then saved to the temporary/lighttpd.conf file to launch Lighttpd.

Behaviours

• No address may end in .lua.
• Large uploads are rejected if no token cookie is set.
• All addresses on any domain, starting application.id/ serve the corresponding static file (if any) from an application's assets folder.
• Any other address is handled by the Moonstalk pages backend unless an additional application specific configuration applies:
  • For folder sites, on the primary domain (not tenant sites) any address (except as above) containing a period but not having a query string, serves the corresponding static file (if any) directly from disk without any processing by Moonstalk. This includes the source of .html views. If a static file needs to be served from the primary domain with a query string it may be placed in an application, or the web server configuration supplemented.
  • For tenant sites, any address starting tenants/ serves the corresponding static file (if any) from data/tenants/.

Bundles

Behaviours

Views

• If the view defined in an address contains no dynamic markup, then the page.modified value (and thus Last-modified header) is automatically set using the last modification date of the corresponding view's file.false
• If a default template exists it is applied to all addresses, to prevent this explicitly set an address template to or the name of anothet template.
• If a view contains an <html> tag, page.template is not set to the default template. If you wish to run a template controler with such a view, specify the template using an address, or collator().

Other files

• If present any views and controllers placed inside a subfolder named with the application.id of an application, will override its views and controllers with those having the same name in this folder.
• File and folder names starting with _ (underscore) in the site root (but not in subfolders) are ignored by Moonstalk, but may still be accessed as static files via the webserver directly. You may thus use this naming to temporarily prevent views and controllers from loading.
• File or folders whose names start with . (period) are both not loaded and are inaccessible from the web server. This allows a site folder to be defined as a version-controlled project where the versioning data is in a hidden folder (i.e. Mercurial's .hg/ folder).

Table

• name
• id
• views
• controllers

Applications

A bundle whose table exists in the global environment, and having handlers that run at launch-time.

Files

• elevator.lua
  Run once for each application to initialise the elevator environment, prior to the elevator attempting to change the status of any server. May use display and output functions to provide user feedback. For applications managing server processes, should declare elevator.functions.appname as a function handling start, restart, stop and kill commands. May be used to modify the filesystem such as to write out config files for external servers prior to launch. Cannot introspect commands (See Elevator()).

Functions

Initialisation

• Elevator ()
  Run once for each application, after the elevator has completed any actionable commands with handlers defined by elevator.lua. May use display and output functions to provide user feedback. Has no effect in servers.
• Loader ()
  Run once for each application, when it is loading (but other applications, and sites are not ready). Should be used for reading files, loading data, and configuring the global moonstalk environment prior to use by other applications.
• Enabler ()
  Run once for each application, after all applications have been loaded (but not sites, as these are loaded by an Enabler). Should be used for application-dependent configuration in the current server.
• Site ()
  Run for each site, after all applications and sites have been enabled. (By scribe.Starter, thus not before all applications starter's have run.) Provides an opportunity for each application to modify the site or configure their own environment. The function is called per-application for only the applications enabled on the given site, and in the order specified but after their own dependencies.
• Starter ()
  Once the server has configured its environment, and each application's tests and Starter have finished, Moonstalk becomes ready and live. A Starter may therefore be used for connection-dependant functions and configuration, prior to handling live requests, such as in the Scribe to pre-establish a persistent database connection. If employing long-running routines such as upon data, the function should yield() in order that the server may still respond with its status. Note that Moonstalk is not live whilst Starters are running, therefore if routines are dependant upon full readiness (other applications), a Starter may return a finaliser function which will be called after all have completed and Moonstalk is live, finalisers are thus concurrent with (or blocking upon) live request handling, and may be used to bring a node online, whereas a Starter itself would be premature.
• Reader (data,format)
  Run when a view is requested for the first time (with every request during development). Allows changes to the raw file data from disk before being served. Should always return data.
• ready
  Currently application initialisation functions are run regardless of dependancies or the success of preceding functions, this key provides a boolean that allows you to inspect the state of any application's initialisation and exit when required applications are not ready.

Requests

• Curator ()
  Invoked for each request where no matching site was found in node.sites. This provides an opportunity to define sites and behaviours that may vary with every request, such as ephemeral sites stored in a database but that may be arbitrarily deleted (as in the Tenants application). The curators of all installed applications are called in succession until one provides a site. The Generic application provides a curator that runs last, to handle the unknown domain condition. When using multiple applications with curators (e.g. both Tenant and Sites Folder) the curator order should be specified with node.curators.
• Collator ()
  Default collator for an application's own addresses. Invoked per-request after addressing, localisation and authentication have occured, but before controllers and views. Provides an opportunity to perform content retrieval or review that is not specific to a view or controller. As the same function is used upon many addresses, it should include conditional handling as necessary.
• Editor ()
  Called after the response has been set and concatenated, but before it is returned. Enables modification of the raw output response value.

Behaviours

• View and controller names specified in addresses are assumed to be in the current application, unless prefixed with another's application.id and a slash e.g. appname/name. If prefixed as ~/name the view or controller is assumed to exist in the current site only.
• Bundle names define the application.id and application.name. Any part of the name before a . (period) is ignored.
• When present, a bundle subfolder named application.id/ is used to serve assets via the web server, under the path application.id/ on all domains.

Table

• global = false; set this to prevent the application being made available in the global namespace as _G[id] it is however always available as moonstalk.applications[id]; note that if the application has it's own namespace it will be populated to that

Performance

Enabling applications has no per-request performance overhead except where the following are defined by the application, where the overhead will be specific to the functionality provided. Memory overhead for applications with multiple sites is minimal (sites are populated only with references to the application).

• curator
• pattern-matched addresses (negligable)
• collator
• editor

Sites

Sites are defined by the sites() function of applications. See the Sites Folder or Tenants applications, or any other application with a sites function, for details on how sites may be defined and configured outside the Scribe environment.

Within the Scribe environment, sites are stored in the node.sites table, with references from the global domains table (used to lookup a site for each request) and ephemeral (per-request) site table.

Behaviours

• A canonical metatag using the primary domain is inserted into any HTML page served on a non-primary domain (i.e. when redirect=false). This cannot be disabled without hacking as it is considered a best practice.
• Site assets are currently served only from a site's primary domain. If you use multiple domains without redirection, you must currently use absolute URLs for all your asset references.

Table

• id = "example.org"
• name = "My Site"
  This is displayed as the title of a site when using the generic template, if not specified request.domain is used.
• domain = "example.org"
  By default this is your primary domain and is the domain used for redirection, and canonical links, etc. Manually specifying it is not recommended.
• domains = { ['example.org']={} }
  Alternate domains that serve the same content as the primary domain (included in this table). Each of these domains is added to the domains table as a reference to the site.
• redirects = { ['www.example.org']=true, }
  Domains that redirect to the primary domain. Any domain specified here will never receive requests, unless intercepted by a curator function.
• language = "en"
  If you have specified multiple vocabularies or language-specific content, you should specify your site's default language for cases where a visitor has no preferred language, and to ensure addresses for language specific content can be indexed by search engines.
• applications = { [application.id]=application, }
  If you wish for applications to be enabled in this site, you must specify a list of their names. Order is unimportant unless more than one application attempts to define the same resource (i.e. address) in which case the first specified will be used.

Views

Features

• Expression tags
  Markup that functions as an in-line macro for the write() function.
• Processing tags
  

Behaviours

• Except in dev mode, HTML comments are removed, although line breaks within them are preserved. Any expression or processing tags within these comments will also be removed, therefore HTML comments can be used to wrap and run debugging functions.
• If the result of an expression (the value itself or the return from a function call) is nil, an error will be generated. If the value is known to sometimes be missing you can provide an alternative default value within the expression, e.g. ?(record.field or ""), in this case the empty string default would cause the value to simply not appear if missing instead of generating an error.
• White space such as line-breaks inside expression tags is ignored (except in string values).
• The modification date of a view mapped by an address, is used for the Last-Modified HTTP header when serving the response. This enables search engines to keep track of your site's changes to reindex pages, and improve their relevancy in search results. If your view displays content that changes in a database you should specify an appropriate timestamp for page.modified in your controller or view accordingly.
• Local functions defined within a view may not contain output if also contained within a section (define the function elsewhere).
• The functions.html file, may be used to declare reusable global functions that output dynamic HTML blocks used amongst multiple views, and should be declared in an appropriate namespace. Simply wrap the appropriate block with the function declaration processing tags. Local functions should be declared within a single view that reuses them. Blocks with expansive size are better created as standalone views, and thus included with scribe.View"name".

Functions

Table

Controllers

Behaviours

• .lua files that start with #!/usr/bin/env lua (for a CLI command) or contain a line starting module (for a library) are ignored and not loaded as controllers.
• White space such as line-breaks inside expression tags is ignored, unless within a defined string.

Features

Addresses

Features

• Linked views and controllers
  Any address defining a view or controller will also have a corresponding controller or view defined, if one exists with the same name.
• Normalised path matching
  Freely use proper capitalisation and accents in your URLs—paths are both case-insensitive and transliterated for latin accented characters.

Behaviours

• Trailing slashes are ignored. I.e. mypath/ is equivalent to mypath.
• Case is ignored. I.e. MyPath is equivalent to mypath.
• Latin accented characters are ignored. I.e. minväg is equivalent to minvag.
• Unicode filenames are not currently supported with autoaddresses.
• Addresses may not contain the '.' (period) character, as this denotes static content to be served directly by the web server.
• For file names that are automatically mapped to addresses (the default) the source code of views is publicly discoverable. (Logic should be placed in controllers, rather than views; if you have to place sensitive logic in a view you may manually map their addresses to obfuscated file names and set autoaddresses=false.)

Table

Selectors

Defines how to match a request. Only one may be used.

• matches = "path" or { "path1", "path2", }
• starts = "prefix/"
• ends = "/suffix"
• contains = "value"
• pattern = "value"

Handlers

Defines what to do with the request. One or more may be specified per address. You need only specify both controller and view if they are different.

• view = "name"

• May be set to false to prevent a view from being displayed.
• controller = "name"
May be set to false to prevent a controller from running.
• template = "name"
  May be set to to prevent the default template (if any) being used. The default template is never used for complete HTML files.false
• collators = {"name"}
The named or referenced functions will be run in the specified order, and may set or change the controller and view or perform other functions prior to their execution. Names may reference functions in the current application or site, or other applications. If specified the default site or application's collator function will not be added. May be set to false or an empty table to prevent inheritance of a default collator function.
• redirect = "address"
Must be used exclusively (without any other handler). May be a URL or address.
• locks = { "key", }
  May specify an array of keys, if none of these matches user.keychain then the user is denied access or (if signed out) asked to signin.
• form = {"…"}
  Set default values for the form table (emulates user submitted input).
• action = "name"
  The name of an action function to run (emulates user submitted input).

Attributes

Addresses may specify any of the attributes of a page, such as template or title.

User

Represents the anonymous visitor that made the request, a profiled visitor (cookie-tracked but anonymous), or a registered user. Managed by the built-in authentication mechanism but may be extended by applications.

Features

• Locale and languages
  Provides access to
• GeoIP
  For upplements the typically unreliable browser-based locale with a network-location dervived locale. Ideally enabled only when most visitors are likely to originate from multiple countries and localised value formatting or translation is required.

Behaviours

• Table

• ip
• token
  The encrypted value of the token cookie, if signed-in or tracked.
• id
  A unique ID number for the user, if signed-in or tracked.
• languages
  An array of the user's preferred locales, derived from the request and user's settings (if any).
• locales
  An array of the user's preferred locales, derived from the request and user's settings (if any).
• keychain

Extend…

To learn how you can extend moonstalk, and make use of pre-packaged functionality, read the applications page, or to understand the Moonstalk's design jump to the architecture page.