This manual is a pre-release preview. Please follow Moonstalk on Twitter or Facebook to be notified of a public release. If you like running with scissors, you can pull from our public repository on Bitbucket.
Please be aware that Moonstalk is not suited to the following scenarios. See our Pivotal Tracker project for tasks and fixes in upcoming iterations.
You should be able to run Moonstalk on pretty much any Linux or UNIX-based computer.
Moonstalk is not compatible with the Windows OS, and has no roadmap for it.
For best performance when hosting busier sites with VPS hosting, choose a package that includes use of 4 or more cores (1 core is adequate for most sites).
Disable the openresty and tarantool services if installed and enabled. These are not used with moonstalk therefore you must stop the instance already started and prevent it from starting automatically.
openresty (packages Nginx with some extras) is the default web and application server. tarantool is an optional database and Lua application server (skip this if you'll be using another, or the included low-traffic datastore). lua5.1 is the language runtime that Moonstalk uses. libuv is a library for handling inter-process communications. LuaRocks is an optional package manager for Lua modules. git is required only for installing and updating Moonstalk, but you may use other systems for your own repositories (see Updating). You may manually install any of the above if you prefer.
The ./elevator command won't start anything now, but will check for required Lua modules, just hit return to install them (using LuaRocks) when it prompts you.
Lua must be available as a binary named lua5.1, create a symlink if it has any other name, or simply invoke the elevator from its binary directly lua elevator.
If you prefer you may install modules manually, providing they can be found in $LUA_PATH.
If you're setting up a production server, be sure to edit data/configuration/Host.lua before proceeding. Or simply: ./elevator hostname=server.example.com logging=1
Ensure that your moonstalk/ folder is not in a public path (e.g. an existing web or FTP server root) and that it is not readable by any non-adminstrators.
That's it, now you can start coding or use the installed applications! If you're fairly new to web development or would like a simple introduction to Moonstalk development, read on — otherwise if you're a daring adventurer leap ahead to the learn section.
Your primary tool for hands-on development with Moonstalk should be a text editor or IDE having syntax hilighting for Lua, HTML and CSS.
If you'll be using localised text it must support a 'no-BOM' encoding option for UTF-8 which most do.
If you're working with multiple colleagues, you can use the Git distributed concurrent versioning system (dCVS), as used by Moonstalk itself for installation.
…you'll also need a diff utility:
…and somewhere to store your Mercurial repository (a share point):
Moonstalk includes a simple CMS system named Manager, you'll need an administrator login to use it, simply run ./clerk scribe "manager.Operator()" and it'll give you a new password, on your development machine you can then access the web interface at localhost/Manager.
Ready for a quick hands-on tutorial?! I'm assuming you've never traded your soul for a copy of FrontPage, and know how to hand-code HTML—or at least won't be too scared reading some. If not, open the source view for this page or any other, start reading it and I'll see you back here later.
Before we do anything, you must change your working directory to your moonstalk install folder.
Sites are created as sub-folders inside the sites/ folder, each named with the site's (primary) domain.
If you don't have a domain make one up, or use example.com.
For development and testing on your own computer you must be able to access your site at its domain, but you probably shouldn't change its public DNS records yet.
127.0.0.1 which is your computer's local address, allowing only you to access it.
Remember to disable or remove this entry when you need to access the actual server again. You may need to flush your DNS cache or restart your web browser for changes to take effect.
Instead of editing /etc/hosts you could create a new DNS record like test.example.com and point this to your computer so that you can easily swap between test and production servers.
If you're using OS X I can recommend using HostsWidget to manage your /etc/hosts file.
After defining a site, you can start Moonstalk. The following command will start all the web/pages servers (and db if configured) in development mode which disables Moonstalk's page caching — allowing you to see changes as you refresh a page in your browser after changing them.
Drop any HTML files and any images or other assets e.g. logo.jpg or page.html into the folder and access them in your browser with their full filename e.g. http://example.com/logo.jpg. Static requests are served by the webserver directly, without any processing by Moonstalk.
Any request containing an extension is considered a request for static content, to be served by the webserver directly.
Let's try a simple page that just displays details of your web browser. This uses dynamic markup inside an HTML file, which we call a view. Dynamic requests are processed by Moonstalk's Scribe (pages) backend.
Any request not containing an extension is considered a request for dynamic content, to be processed by the Scribe.
<p>Moonstalk's version is: ?(moonstalk.deployment.version)</p>
?( wibble ) markup is called an expression tag (or «placeholder» / write macro). When a page containing this is requested, Moonstalk evaluates the variable or expression inside the tag (e.g. wibble) and replaces the tag with the result.
Now let's try something a little more interactive. We'll use a form to input our name and display it upon submission, using an alternative if no name has been submitted yet. This uses a Moonstalk table, surprise surprise(!) named form to access user submitted values.
<h2>Hello ?( request.form.name or "have we met?" )</h2> <p>What’s your name?</p> <form method="post"> <input type="text" name="name"> <input type="submit"> </form>
The hello.html file lacks an <html> tag to make it a complete page, because we're now going to put it in a template. This will apply a common layout to all of our test site's files when Moonstalk serves them.
<html>
<head>
<title>My Site</title>
</head>
<body>
<h1>Welcome to my site</h1>
<? scribe.Insert("content") ?>
</body>
</html>
<? ?> markup is called a server processing tag. Inside this you can run any valid Lua code.
scribe.Insert("content") is a function you can use in templates that indicates where in the template to put the content from the requested view, in this case hello.html. You can also declare other sections (than content) in views, to later insert elsewhere.
So now let's try something a little more clever. We'll save a value from a form upon its submission, and then display its saved value. We'll do this by putting code into an additional Lua file which we call a controller.
database = {"mydata",}
<h2> Saved value: ?( fetch ( mydata.isaved ) or "nothing saved" ) </h2> <form method="post"> Save a new value: <input type="text" name="myinput"> <input type="submit"> </form>
if request.form.myinput then save ( mydata.isaved, request.form.myinput ) end
If you ever lose track of what-on-earth you think should be happening during development, it's useful to see operations step-by-step. In fact you might like to keep the log open whilst developing and it will show what's happening with each request.
if request.form.myinput then save ( mydata.isaved, request.form.myinput ) log.Append( "=======UPDATE:" .. request.form.myinput ) else log.Append "=======FETCH" end
Linux
- tail -f temporary/moonstalk.log
OS X
- double-click the temporary/moonstalk.log file
If you're not too sure about any of the jargon used up til now, you might like to take a ganders at the following glossary, otherwise jump ahead to the learn page to dig into some usage guidance, and then the develop page to for reference and more advanced topics.
Although we attempt to avoid using jargon and proprietary terminology, it's useful to understand the naming conventions used to distinguish the various components and concepts.
The collective identity of one part of the Internet comprising a multitude of server computers that all perform the same function — like Email allows the exchange of messages, or FTP the exchange of files — the web allows the exchange of pages (which may be plain documents, or interactive software interfaces), collected together into zillions of separate web sites. So-named as a web because most of these pages and sites are linked together, enabling association and exploration amongst them.
A 'server' is simply something that accepts and responds to instructions from a ‘client’. These can be computers (hardware), or combinations of operating system and application programs (software) for which the term is used interchangeably (perhaps confusingly).
A collection of pages that are accessed via a web browser application (e.g. Safari, Internet Explorer, Firefox, Chrome) on your computer, either corresponding to files in a folder from a web server (a statically generated site), or a web application. [Moonstalk hosts web sites, and runs web applications.]
A 'dynamic' page (or site) may refer to either a dynamically generated page, or a page with dynamic and interactive elements (i.e. using JavaScript in the web browser), but not necessarily one that was also dynamically generated by a web application.
The protocol—akin to an envelope—used to send instructions between one place on the network, and another. HTTP is the protocol of the Web, and is thus used by a web browser (on a client computer) to send a request to a web server application (on a server computer).
The letter inside the HTTP envelope, is a request. It contains the instructions sent from the visitor's browser, asking the server for a page at a particular address, and if submitting (posting) a form, the contents of each field (which can be better considered a parcel).
All web sites and their pages have addresses. These comprise a domain name (e.g. example.com — representing a server), and an optional ‘path’ (e.g. /mystuff — representing a resource on that server, such as a folder or file). Together these form the address example.com/mystuff yet this address could be for somewhere other than the Web, so to identify it as a web address it can be further combined (prefixed) with the Web's protocol, to form a URL, becoming http://example.com/mystuff.
Although a full URL properly represents a web address, some web sites use a server with a domain name starting www. (such as www.example.com) and in this case the http:// prefix is somewhat redundant as it is clear that the domain is a ‘world wide web’ (thus HTTP) address. Further, because the web is the most popular part of the Internet, it is almost given that any address without an identifying prefix (such as example.com/mystuff or just example.com) is probably a web address and not, for example, an FTP address.
This application is the frontend of a server which accepts and responds to HTTP requests (often many simultaneously). With static websites, content is hosted (served from disk) by the web server itself, however with dynamically generated sites, the pages are generated by a web application through a backend of which a server may have one, or many for the same or different web applications. Some web servers may integrate the backend, whilst others may use separate backend application server programs or even separate server computers.
Software (a set of instructions) that dynamically generates pages, such that they appear different for every visitor, or every time they are accessed, by incorporating values from a database (a data storage application) into the page according to various criteria defined by the application developer. Such applications are built using a programming framework. [Moonstalk is used to build web applications.]
The integrated components of a programming language and libraries of useful software commands. This aids performing specific tasks, and in the development of web applications, which are run by a backend. [Moonstalk provides a framework.]
A server application that understands and interprets the language of a web application (e.g. PHP, Ruby, C#, Lua), to generate its pages on-demand when the web server forwards requests to it. [Moonstalk operates in backends.]
Another protocol, used to forward HTTP requests from the web server application, onto the backend application. In effect the request sent in a HTTP envelope between two different computers (client and server) is taken out, and put in a FastCGI envelope to be sent between the two different programs (frontend web server and backend page server—typically on the same computer).
The service and functionality of delivering something such as a web page from somewhere, such as a web server, to someone, such as a visitor—thus web hosting. Both a server or service provider offering hosting are called a host. [Moonstalk configures and provides hosting.]
This is a host, that is either a dedicated (physical), virtualised (VPS/cloud), or a shared server (shared hosting) that runs the operating system (e.g. OS X or Linux) on which a web server application is installed (a web server). The server operating system (OS) provides bottom-layer functionality such as networking and storage. A server node is managed by an operator, or service provider, and provides hosting services to multiple site owners.
This is a programming language and a runtime (application) for interpreting software code written in its language, so that it can be run (executed) on a computer. [Moonstalk and its hosted web applications use Lua.]
HTML ‘view’ (layout) files, containing database field markers or server processing tags, and optional counterpart Lua ‘controller’ (logic) files. When we refer to 'pages' we may be referring to either or both of these components.
Folders of pages and images, etc.. A site’s pages are specific to one or more domain names. Sites may be managed by individual site owners.
As with sites, but comprising additional Lua files for functions. Applications may be accessed across multiple sites. Applications are installed by the server operator and not site owners.
The FastCGI server program that receives requests from the web server, and returns pages from a site after generating them, and typically by querying an application, database or folder for their content. Multiple scribe processes ('backends') run simultaneously on a single server node, and the web server queues requests, passing them to the next available backend.
Moonstalk's database system is a server program that handles requests from scribe servers to fetch or save data. The teller uses native Lua as both the protocol and query language (not SQL), and does not use a defined schema. The teller also provides persistence (automatically saves data) and replication (synchronisation with other nodes).
This is an included web application, available by default on all your sites, that enables per-site access to a page and content management system (CMS).
This is an interactive command-line launcher program that provides the functionality to start, stop or restart the Moonstalk components.
A command-line program enabling you to directly interact with each Moonstalk environment in its application servers (on the same server), such as the web backends and database.
This application keeps an eye on Moonstalk's applications and can tell the elevator to restart them and notify someone if problems are detected.
Moonstalk relies on a variety of shared third-party software for specific functions. Applications may in turn utilise these, or their own. In Lua libraries are referred to as 'modules'.
This manual is a pre-release preview. Please follow Moonstalk on Twitter or Facebook to be notified of a public release.
This section provides an an overview for the use of Moonstalk, and the basics of adapting it to common needs. See the develop page for detailed attributes, behaviours and functionality.
If you've not learned the basics of the Lua language yet, now might be a good time. Programming In Lua is an easy to understand and straight-to-the-point guide that'll introduce you to the essentials in just a couple of hours. If you know other languages you may well be able soldier on and extrapolate much of its syntax.
Moonstalk hosts both sites and applications, and we refer to both generically as ‘bundles’. Each are defined by folders containing much the same set of files for their resources.
Creating a bundle is as simple as creating a folder, and putting some files in it. The following are files common to bundles (both sites and applications). All are optional and may define functionality indepedantly of each other, or in combination.
Any file not otherwise noted and having an extension, is considered to be static content and will be served directly by the web server. See the design page, and bundles reference section for specific details.
You may create subfolders and organise your files as you wish. However top-level sub-folders having the name of another application will extend (supplement) or override (replace) that application with its own contents.
Site bundles are created inside the Moonstalk sites folder, and each bundle (folder) defines an individual site, named with its primary domain.
The primary domain my be specified with or without the www prefix, but both are always handled, therefore ensure your DNS is setup with records for both.
See the Tenants application for details of how sites can be stored in a database (e.g. for SaaS).
domains={"example.net","www.example.net",}redirect=false in the settings.Application bundles are created inside the Moonstalk applications folder, and each bundle defines an individual application, named with its ID.
Applications provide shared or generic functionality (such as a blog system) for use on multiple sites (disparate domains). Applications must typically be enabled in a site's settings in order to use them, refer to an application's own documentation for details of its behaviour, configuration, and usage. Moonstalk includes several bundled applications.
The generic server not-found and error pages are provided by the Generic application, see its documentation to customise them.
Applications are for installation by server operators, and should be audited for security.
If your visitors primarily come from more than a single country (or state) the use of a Content Delivery Network (CDN) is highly recommend for improved performance in preference to hosting your assets from your server itself.
These are files such as images, CSS, and Javascript that are referenced in and enhance HTML pages, and also PDFs, video and other downloadable files.
Typically you would group all your assets together inside a subfolder of your site, for which we recommend one named public. When you link to them in HTML you simply use the relative src or href "public/myfile.type".
Moonstalk's primary purpose is serving content that is at least partly dynamic, however you may serve static pages (i.e. complete HTML files) in the same manner as other static assets.
Any requests for an address not containing a filename extension is considered a dynamic request, to be processed by Moonstalk's Scribe backend, which runs a sequence of functions to handle the request and provide a response.
Moonstalk provides built-in handling for dynamic pages, generated from a view file providing the primary layout and content (HTML markup), and optionally supplemented with a controller providing separate processing logic (Lua code).
Both views and controllers are transformed into functions that run with access to the same Lua environment and Moonstalk functions—the difference lies only in the file format (tags in views, code in controllers). See the develop page for details of the environment.
See the Content application for details of how static pages can be stored in a database.
Addresses for dynamic resources cannot contain periods appear after the last forward slash. If you require other behaviours you can manually configure the web server to handle them.
These are simply .html files, either complete HTML pages or fragments of formatted content for display, such as within a template. Views may utilise the following two forms of dynamic markup.
?( expression )
The expression tag is similar to the write() function and is replaced in the page by the resulting value of a Lua expression.
An expression may be a variable name, function call, concatenation operation, or a combination of these (grouped inside brackets). Code blocks such as if and while cannot be used. Expression tags may be used anywhere in your HTML file (including inside attributes, CSS, and scripts).
Expressions and variables should provide values that can be output (i.e. strings and numbers), but will ignore nil values. Some functions such as ifthen() and wrap() will ignore missing values.
<? code ?>
These may contain any valid Lua code including blocks and functions all of which do not impact the page output, except for the result of any calls to the write() function which appends output to the page in-place (ignoring nil/falsy values). Code may be split in any manner across multiple tags such as to define conditional output.
<? if mytest then ?> <b>mytest has the value ?(mytest)</b> <? end ?>
These are predominantly used to separate and protect the logic (Lua code) for data handling from the design layout in a view, thus avoiding placing complex server processing tags within a view's markup.
When paired with a view, a controller's code is placed in a .lua file of the same name. I.e. myfile.html is the view and myfile.lua is the controller. Controllers run before views and are used to process and prepare variables (such as from the request, or a database) for insertion as strings into a page, using expression or processing tags in the corresponding view, and may also to process any data submitted from it, such as to be saved back to the database (though this may be seperated out into multiple standalone controllers such as for an API).
Controllers may provide output before a view is run, using write[[<p>content</p>]] (typically using a Lua 'long-string' to avoid issues with escaping), and may thus also be used in place of a view entirely, particularly where a non-HTML response is required (e.g. a JSON or XML API).
With templates you can define the common layout or features of all pages, or groups of pages, such as providing a navigation menu, header, footer and so forth. A template wraps the content or layout (e.g. from a view) with its additional markup.
Templates are identical to dynamic pages, having both a view and an optional controller, but are run after the page controller and view. Templates may thus have both layout and logic.
In a template you must specify where to insert the page content generated by the page controller and view, using the following command.
<? scribe.Insert "content" ?>
Templates may be defined in the following manner, the latter methods take precedence over any prior.
template="name" attribute to an address.
page.template="name" from a controller or view.
By default all output is placed into a section container named content, from which the page response is generated (such as with templates). You may define your own output sections, to capture output and then later insert into the page using any arbitrary order. Use the following command to define a section.
<? section "name" ?>
All output following this will be captured, until another section is defined or the page is generated. You can reset back to the last section by not giving a name, which in most cases will restore output to the content section.
<? section() ?>
Once output is collected or moved into a section you will want to insert any custom sections into the page content, else it will simply be lost. To insert a section, use the scribe.Insert command with the name of your previously defined section.
<? scribe.Insert "name" ?>
You may include or run any view or controller in any other, by using the following functions. Bear in mind that when outputting views from a controller, output will be appended to the default content (or last declared) section. To insert a view in another, declare a named section to capture it first, else call the view from where it is to be inserted.
scribe.Page "name"scribe.View "name"All application view and controller names are prefixed with the application name, e.g. appname/viewname. The current site's are accessed with just their name.
Chaining behaviour is akin to forwarding in J2EE, and should not be confused with HTTP redirection.
To cause the user's browser to send a new request for a specified address use the following command, the redirect will occur after the current request has finished unless explicitly abandoned.
scribe.Redirect "address"
This is often used after successfully processing a submitted form to change the URL to a bookmarkable action ID, instead of simply displaying a different view at an unchanged address. The given address may be a path or an absolute URL.
To provide a response, a request's URL path must be mapped to a function (typically a controller) or content (typically a view). Moonstalk provides built-in mapping through its addressing mechanism, and ad-hoc routing through the chaining mechanism.
By default views in site bundles (but not applications) are automatically mapped to an address corresponding to their file names (without extension). E.g. the file sites/example.com/mypage.html would be accessed as http://example.com/mypage.
You may define your own mappings, by declaring an addresses array in a bundle's settings file. Each address in the array takes the following form, specifying one selector, one or more handlers, with optional attributes inherited by the page. (See addresses on the develop page for full usage.)
{selector="path", handler="name", attribute=value}
Moonstalk has built-in generic handling for user access control. You may employ this either on addresses, or within your own functions (see develop). Addresses and pages may be defined as having restricted access using locks, and users may be granted access to such locked pages using keys. Locks and keys are simply names that denote membership of certain user 'groups' as desired.
An additional application is required to define behaviours for user and key management (i.e. assignment, retraction). Moonstalk includes the people application for managing users within the Teller database. No support is provided for hard-coded lists such as from .htaccess files, but would be easily implemented yourself.
To restrict a view to a particular key holder, add a locks attribute to the page, or to desired addresses in the settings.
{ matches="private/page", view="private-page", locks={ "Private", } }
A user requiring access to that address must then be granted a key with the corresponding name in their keychain.
By default all signed-in users have a default "Guest" key, you may therefore require sign-in on pages by adding a lock of the same name.
Moonstalk includes built-in support for translated vocabularies (including handling plurals, and functions such as for ordinals), easily used via a simple key-prefix on term names. Support for locales is also built-in and may be selectively used via formatter functions. Additional built-in behaviours such as Latin transliteration, and a report for untranslated terms, facilitate localisation improving both visitor and developer experience with projects requiring ‘i18n’.
When serving localised and translated pages, a user's preferred or profile-derived locale and language is used wherever possible, followed by the site's specified default locale and language, with fallbacks to the first or only language defined (such as if translation is incomplete or only has a single language defined).
If enabled GeoIP lookups are made using an in-memory dataset to supplement an unprofiled user's locale, and completely avoiding the additional latency of third-party API calls.
Address paths are normalised using case-insensitivity, and transliteration of latin accented characters to ASCII. Pages may be addressed with or without such characters, whilst the original (i.e. with accents if your file or page names contain them) is preserved as the canonical name (and address).
You may use any desired encoding for your HTML and strings in Lua (but UTF-8 is recommended as this used by all Moonstalk's bundled applications), simply be sure to use the same encoding throughout, and ensure that the corresponding encoding is declared in your HTML.
Lua does not handle Unicode sorting, and you may need to use your own or third-party routines for this if needed.
Non-ASCII (e.g. UTF-8) encoded files must be saved without a BOM, otherwise they will fail to load. See the utilities section for some editors that won't muck this up, if yours does.
Views saved as files may have multiple versions created to provide separate content for different target cultures (translations/localisations), simply by using the following naming convention for the files (of any supported view format).
Views that are not provided in this manner are assumed to either use the site's default culture, or to be dynamically translated and/or localised using vocabularies and formatters.
Views generated from other sources may have cultures specified directly, see the Content app and view table for details.
These define the formatting of numbers, currency, dates and so forth, with country/language variations. Moonstalk will always attempt to identify and honour a user's locale and language (see vocabularies below).
Values are displayed respecting a locale if you output them using Moonstalk's format functions.
Several locales are included with Moonstalk, whilst additions or corrections may be submitted to us. You may override them in the global environment from a settings file.
These enable translations and localisation, and store translated text values (strings) for each translated language. If you don't expect to need to use translated content you may ignore this functionality and it will not affect the display of pages.
Applications, sites and pages may specify their own individual vocabularies (translations) that supplement each other.
Pages are displayed using the language matching a user's preferences or request when available, otherwise it will be substituted with a regionally acceptable language, the site or application default, or finally English, when these are available in the vocabularies.
Partial translations are undesirable, as missing translation are replaced with alternate translations according to the user's locale, which results in pages being displayed in a mix of languages, therefore it is important to ensure all applications, sites, and pages support the desired languages.
In developer mode missing translations are not substituted with alternates, but are instead displayed as ⚑name to assist in identifying missing translations, whilst the /Manager/Localisation view provides a complete report of untranslated terms.
Individual translated words or phrases are represented by keys (untranslated names) for which one is defined for each language (using its language code as defined in globals/Attributes.lua) in the bundle settings file.
en.aword = "a word" fr.aword = "un mot"
Vocabulary term names (keys) should be globally unique (amongst all applications) else may be replaced by other applications. Per-site and per-page vocabularies may be used to supplement or customise these definitions.
Translated words from the active vocabulary (the current user's language) are accessed using a variable named correspondingly but prefixed l. (for the original word) or L. (reformats the word with an initial cap).
In views the translated value's variable may be used in a placeholder as with any other variable.
Plural forms for translated words are also supported, these are defined in the same manner as words above, but with the number of plural forms corresponding to the language per the Mozilla rules (all of which are supported) e.g. 2 forms for Germanic languages (defined as an array of two strings), or 1 form for Asian languages (defined as a single string not an array).
en.items = {"item","items"}
Once defined you can access the translated plural by calling the localised vocabulary function with the identifier and a number that specifies the correct plural form to be used.
l("items", number)
Additional functions may also be localised and assigned to the vocabulary and called simply using the
l.prefix. Some are included such as the
l.Ordinal(number)function that will return an appropriate suffix for an ordinal number.
Translated blocks may contain macro placeholders for replacement with the macro function..
Each individual server (virtual or dedicated) on which Moonstalk runs is called a node, and comprises the components inside the install folder. No other location is used for configuration or temporary files, withstanding the startup script.
This shell utility is your interface for managing a Moonstalk node. Without any commands it will provide you with its status, and will always report any application or site compilation (syntax) errors. All commands are optional, accepted in any order and have several synonyms. For further usage instructions issue the help command.
The server control commands are:
These control commands may be accompanied by a server name:
To start web you must use super-user privileges (sudo or login as root).
The node configuration is stored in the data/configuration/Node.lua file and contains node-specific options for both Moonstalk and other applications. You may edit this directly, or for node attributes, you may use the elevator to change it.
For production (non-development) servers, set a unique node ID that is between 101 and 999.
For development computers, you can enable development mode.
You may also need to set a FQDN if the default hostname is not adequate.
See the develop page for additional configuration options.
Moonstalk includes a web administration application called the Manager. This is enabled on the localhost site, which is also accessible on the server's default hostname. The Manager's user database is created with a default operator user, having a unique password that is the secret from the node configuration. The elevator will notify you of these upon first use.
Moonstalk can be updated to the current release version with a single command on a non-development server (id~=100).
Your sites, applications and data will be untouched by this command, but be aware that updates to applications may require adjustments to your code or settings, therefore it is advisable to check the release notes before updating.
During the pre-release development phase please review the changelog for changes that may affect you. Subscribe by RSS, open your ~/moonstalk directory [repository] with a Mercurial GUI app after updating, or use your shell: cd ~/moonstalk; hg log | grep summary | more
You may create your own repositories using your preferred CVS for each of your application and site folders, or one for the entire sites folder—but not the entire applications folder, unless you are not using Mercurial and exclude moonstalk.* from being tracked.
If you use Mercurial for your site and application repositories, Moonstalk's update command can also pull from them for you. Simply push your changes before attempting an update, and add your repository folder names to the node configuration.
As the moonstalk directory is a version-managed Mercurial repository, if you change any of Moonstalk's managed files, the changes will be lost when Moonstalk is updated.
You may however fork/clone Moonstalk's repository, and change Moonstalk's default respoitory in .hg/hgrc to your own forked repository. Once an initial installation pull has been made from your respository on each of your deployment servers, the Moonstalk update command may be used to pull from your repoisitory thereafter.
Now that you've got a hang of the fundamentals, read the develop page to learn about data tables, configuration, functions and more that will enable you to develop your own solutions as desired.
<? ?> instead of ASP/ROR <% %> or something else?<?lua ?> syntax.This manual is a pre-release preview. Please follow Moonstalk on Twitter or Facebook to be notified of a public release.
These data-structures are universal providing access to the Moonstalk environment, the request, the response, and in many cases also database queries and their results (see next). You may check and change the values in these tables to determine how to handle a request—and to generate a page for response.
All tables exist in either a global or ephemeral (per-request) scope. The latter are discarded after each request is completed and may be modified freely, whilst the prior exist across all requests and should not be modified. Additional reference (pointer) tables are provided in the ephemeral scope for convenient access to some global tables.
Global tables should not be changed during a request, and are intended to only be modified by application load-time functions. Failure to adhere to this will result in random results when using more than a single backend instance. Whilst reference tables change with each request, they are dynamic pointers, or aliases, to the contents of global tables and therefore should also not be changed. No tables are persisted (use a database such as the Teller).
Typically if a controller has many references to, or a view consumes many instances of values in these tables, it is advisable for both cleaner code and better performance to assign them to a local at the top of these files. e.g. local data = page.data local form = request.form
This predominantly contains details of the HTTP headers, with parsing into a more Lua-friendly format where appropriate. Use of this table is discouraged as its format and structure is dependant upon the web server API. See form and page instead.
request.get and request.post are not conditional, instead use request.method=="GET" (case sensisitive).
path = "/My/Account"
browserclient = {ip="address", token=, id=, timezone=, languages=, locale=}user.ip is advised instead.
Contains attributes that lead to authentication, and some present irrespective, providing a place to find both browser and user dervived values.
Contains values encoded in the request.body such as in an HTML form; is always present regardless of method.
In OpenResty when not urlencoded post={maxsize=bytes} must be set on an address for the form to be populated, with maxsize generally only being required for files.
name = "value" or {name="name", size=bytes, ['content-type']="mime/type", contents="data"}files = { {name="name", size=bytes, ['content-type']="mime/type", content="data"}, }
If a multi-part POST this will contain an array of the parts, this is primarily useful for multi-file uploads in particular using an HTML5 file input with the multiple attribute.
redirect = false or url; default behaviour is to redirect all non-primary domains to the primary, false will disable this, or if set to a URL any domain without a redirect will then use it; the requested path will be appended to the URL (if undesired, terminate the URL with a question mark to transform the path in query string).Defines the page to be rendered as a response.
Do not set new keys in this table — use only the data key.
address = "my/account"request.path used for matching addresses. (Lowercased and transliterated for Latin accents, with leading and trailing slashes removed.)
transliterated = truepaths = { my=true, account=true }authenticator = "generic.Authenticator", }site.authenticator or node.authenticator, or false to disable.
locks = { "name", }locale = "us"user.locales and site.locale.
language = "en"user.languages and site.vocabulary. Defines the Language header.
localise = truecontroller = "name"
view = "name"
template = "name"
editors = { function() end, myapp.editor, }modified = 1290172118errorscribe.Error() rather than modifying directly.
headers = { ['Header-Name']="value", }
status = 200
page.data.foo = bar or replace with a database query page.data = db.name.field. Where any contents of the table will be referenced repetitively, create local data = page.data to use them, likewise subtables.
Always declare variables as local else use page.data.
The global Moonstalk environment for each server (node). Settings are configured in the data/configuration/Host.lua file, and may also be changed at launch time by the .Elevator
servers = {"application"}
The server ids that need to be running. This defines the webserver (e.g. "openresty" for nginx or "lighttpd") and databases (e.g. "teller", "tarantool"). If not specified these are automatically detected, and will also be added from the database configuration for which "teller" is the default.
roles = {"name"}
When running multiple nodes, allows an individual node to only handle specific roles, such as specific servers and only the correspondingly associaated database tables.
scribe = {server="application"}
As above if not specified this will be detected or defaulted.
curators = {"application", …}
The curators and order in which they should be run for each request.
applications = {"application", …}
Default applications that must be enabled for all sites.
logging = 2geoip = falseEach database application provides its own interface and behaviours. The OpenResty application bundles drivers for MySQL, Redis, and Memcached. Some Moonstalk database applications (e.g. Databin and Tarantool) offer a standardised interface and configuration using a schema.lua file in the application as follows.
schema files can also define enum.name = keyed{…} constants the same as in settings files.
To define a new database table (as opposed extend another application's tables, e.g. users or tenants), declare it in an application's schema.
The recommended naming is a plural for a table e.g. users = {…}.
The following keys in each database table declaration are supported for each table. See the database application documentation for additional configuration such as field declarations.
system = "tarantool"server = {…}role = "name"node.databases.role and providing the server parameters. If node.roles contains the role name and the system is specified in node.servers then the server is assumed to be local. See Node.
For application-specific functions see the applications page.
(expression)
[[string]]
Appends the passed value to the current output section. Values may only be strings, numbers or nil (which is ignored).
Expressions evaluated by write or ?(expression) tags may only provide string or number values, nil and false values are ignored.
A table or function value will invoke an error as they cannot be represented as text in a response, for debugging you may however output them with tostring().
In views, do not call write instead use an ?(expression) tag.
Returns the given language key's value from a translated vocabulary, in its original case. Use L.key for an initial capital.
This is a proxy table—a function with table.key syntax. (Achieved using a metatable __call overload.)
{ [[text]], key="value", }
Replace defined ?(key) macros in the specified text, with their correspondingly specified values.
Whilst macros in views (expression tags) accept expressions (thus, function calls) for replacement, text macros do not, and where no corresponding key is specified a macro is not replaced.
Text macros are typically used for customising strings, such as translated language keys from vocabularies, where pre-defined grammar is customised using embedded variables.
"name"Changes the current output to a new container, as named. (The default page response output is named content.) See also Cut.
"name"Marks the current position in a view to later Insert a section or other value.
"section_name"
"mark_name"
Inserts a previously defined section into the current view/section. Takes an optional mark_name parameter to insert at the specified Mark instead of the current location. See also Paste.
"name"Run a specified controller and/or associated view and append it's output to the current page. Name is normalised (lower-case).
"name"Run a specified view only and append it's output to the page. If the view provides versions for different cultures a matching translation/localisation will be served. Name is normalised (lower-case).
"name"Run a specified controller only. Name is normalised (lower-case).
("name", value)
{name="", value="", expires=, httpOnly=true, path="", domain=""}
Sets a cookie in the user's browser.
The following functions abandon page processing in some way, to display an alternate view or response.
The current function from which these functions are called, and any chaining from it, will continue to run but their output will be surpressed. No further functions will run except for the default template (if any). To prevent the current function from continuing, follow the function call with a return termination.
"address"
Abandons the page processing to instead returns a 302 redirect response to the user's browser. The address may be a path or URL.
{ "name", }
Checks that the current user has one of the specified keys, otherwise abandons the page processing. If not signed-in the generic/signin view is displayed, or if signed-in but without a specified key, the generic/unauthorised view is displayed.
For a conditional termination use if not scribe.Authorised{"name"} then return end. For conditional content check user.keychain directly.
"synopsis"
{title="synopsis", detail="explanation", realm="page"}
Abandons the page processing with the generic/error view using the specified details.
()
("name")
Abandons the page processing with the generic/not-found view. Accepts an optional user-friendly name to be displayed instead of the page.address.
The following functions abandon page processing in some way, to display an alternate view or response.
All standard Lua libraries are available, with the following, and some other less commonly used libraries that are documented on the credits page.
You may use load other libraries (.lua or .so) saved in the same folder as a view or controller, or within an application folder by using the full path.
require "libraryfile"
require "appname/libraryfile"
Moonstalk uses its own package.path and cpath, therefore if you wish to use other libraries elsewhere on your system (such as from LuaRocks), you must append those paths to Moonstalk's values. Applications may do this in the settings or functions files.
package.path = package.path .. ";/my/lib/dir/?.lua"
Moonstalk uses the data/configuration/lighttpd.conf as its base configuration, with which its own configuration is combined at launch-time, and then saved to the temporary/lighttpd.conf file to launch Lighttpd.
Contains named and translated terms, phrases and blocks by language. These values are used in the Scribe by referencing l.term_name (or L. to capitalise) for the language corresponding to request.client.language. Using the macro function these terms may contain placeholders for other values.
en.term_name = "Value to use in English"
Language and term IDs/names should not contain special characters nor be reserved words, if they are they must be declared as ['special-name'].
With the exception of page vocabularies (see below), all application vocabularies are merged, therefore if more than one application declares a term, the value may not be as expected. Term names should be unique and prefoxed with a suitable identifier, unless for generic use..
view-name.vocab.lua file, which is populated the same for bundle vocabularies, but is only accessible to the corresponding page (controller, view, template and editors).
view defined in an address contains no dynamic markup, then the page.modified value (and thus Last-modified header) is automatically set using the last modification date of the corresponding view's file.
template=false on page or address, or set it to the name of another template-view.
<html> tag, page.template=false. If you wish to run a template controller with such a view, specify the template using an address, or use a custom Collator.
application.id of an application, will override its views and controllers with those having the same name in this folder.
A bundle whose table exists in the global environment, and having handlers that run at launch-time.
elevator.luaElevator ()Loader ()Enabler ()Site ()Starter ()Reader (data,format)data.
readyCurator ()node.sites (e.g. matched by site domains). This provides an opportunity to define sites and behaviours that may vary with every request on every site, such as ephemeral sites stored in a database (as in the Tenants application).Collator ()Editor ()output response value.
application.id and a slash e.g. appname/name.
If prefixed as ~/name the view or controller is assumed to exist in the current site only.application.id and application.name. Any part of the name before a . (period) is ignored.
Enabling applications has no per-request performance overhead except where the following are defined by the application, where the overhead will be specific to the functionality provided. Memory overhead for applications with multiple sites is minimal (sites are populated only with references to the application).
Sites are defined by the sites() function of applications. See the Sites Folder or Tenants applications, or any other application with a sites function, for details on how sites may be defined and configured outside the Scribe environment.
Within the Scribe environment, sites are stored in the node.sites table, with references from the global domains table (used to lookup a site for each request) and ephemeral (per-request) site table.
redirect=false). This cannot be disabled without hacking as it is considered a best practice.
These values may be specified in the site's settings.lua file.
name = "My Site"request.domain is used.
domain = "example.org"id = "example.org" this is used internally and not normally consumed, it is the default domain, and may be used to reference site bundles with moonstalk.sites[id].
domains = { ['example.org']={} }domains table as a reference to the site.
redirects = { ['www.example.org']=true, }language = "en"applications = { "[application", }locks = { "[Name", }collators = { "application", "application.Function", application=false }false, ending with the default scribe.Collator which provides a page from addresses. If no collator provides a page, the scribe falls back to the generic found page.
token = { expires=seconds }?(record.field or ""), in this case the empty string default would cause the value to simply not appear if missing instead of generating an error.
Last-Modified HTTP header when serving the response. This enables search engines to keep track of your site's changes to reindex pages, and improve their relevancy in search results. If your view displays content that changes in a database you should specify an appropriate timestamp for page.modified in your controller or view accordingly.
scribe.View"name".Functions
#!/usr/bin/env lua (for a CLI command) or contain a line starting module (for a library) are ignored and not loaded as controllers.
autoaddresses.
autoaddresses=false.)
Defines how to match a request. Only one may be used.
matches = "path" or { "path1", "path2", }
starts = "prefix/"
ends = "/suffix"
contains = "value"
pattern = "value"
Defines what to do with the request. One or more may be specified per address. You need only specify both controller and view if they are different.
view = "name"
false to prevent a view from being displayed.
controller = "name"
May be set to false to prevent a controller from running.
template = "name"false
collators = {"name"}
The named functions will be run in the specified order, and may set or change the controller and view or perform other functions prior to their execution. Names may reference functions in the current application or site, or other applications. May be set to false to prevent inheritance and modification by applications, otherwise will inherit site.collator or node.collator as the first. These functions are run after the Binder(), prior to page rendering, and may used for preferences, authentication, and content retrieval.
redirect = "address"
Must be used exclusively (without any other handler). May be a URL or address. Optionally set redirect_preserve=true to append the path and query_string to the given URL.
locks = { "key", }post = {maxsize=bytes, …}mazsize permits uploads (which are otherwise disabled). Setting form=false will prevent the Scribe from preprocessing the request body so that a controller may do so instead (e.g. using moonstalk.Resume with Nginx methods).Addresses may specify any of the attributes of a page, such as template or title or type.
Represents the anonymous visitor that made the request, a profiled visitor (cookie-tracked but anonymous), or a registered user. Managed by the built-in authentication mechanism but may be extended by applications.
tokentoken cookie, if signed-in or tracked.
idTo learn how you can extend moonstalk, and make use of pre-packaged functionality, read the applications page, or to understand the Moonstalk's design jump to the architecture page.
Provides default generic behaviours, including authentication and session handling. Unlike other applications, it is always loaded for all sites.
page.error=false to prevent fallback to the error view.node.provider.name and
node.provider.domain if defined.
authenticator = "generic.Authenticator"language|locale|timezone|… to populate client; lacking these will populate client using the browser requested language, or GeoIP if enabled. Furthermore enables sign-in and sign-out on any page address using a form posted with a field named action having either the value SignIn or SignOutaction.Signin ()generic.Password ("password")user.sessionuser.session.visits=count
request.client.keychain.Usergeneric.Session (token)generic.Signout ()generic.Email { to="recipient@example.com", cc="recipient@example.com", bcc="recipient@example.com", subject="Test", body="content", server="smtp.example.com", username="login", password="secret", from="sender@example.com", headers={} }[["Name" <address>]] format. Note that this function blocks the scribe from processing other requests whilst communicating with the SMTP server, and should therefore ideally be run as a task, else response to the user will be delayed.node.localhost.applications = { "application", }Provides an abstraction for sending messages, using transports known as a 'courier', and includes a default SMTP courier capable of sending messages without a relay server.†
node.email.message.bcc to all outgoing messages for monitoring, and in dev mode node.email.message.to will capture all messages.
A single function wraps underlying behaviours, see in-code comments for further details.
email.Send{ to="recipient@example.com",
cc="recipient@example.com",
bcc="recipient@example.com",
subject="Test",
body="content",
transcribe="appname/eml-name", the message body will be the named .eml file functioning similar to a view, but providing a complete text, html or multipart message body; it should use a code block to set message.headers['Content-Type'] = 'multipart/alternative; boundary="-------abcdefghij"' as appropriate (not required for plain text), and may modify any other message attributes such as message.subject using localised values
fields={name="value",…} expression tags defer=false, in async servers this causes a wait upon the dispatch result, in sync servers this may invoke a task (refer to the server documentation)
fail="appname.Function", passed the message on failure
sent="appname.Function", passed the message on success
page="name", render a page as the body using scribe.Page, you may for example place all these views in emails/ directory and thus set the value to "emails/view-name"; may optionally specify site=id to render the view from
courier="name", e.g. "smtp"
from="sender@example.com",
headers={},
}site.from = "Display Name <mailbox@address>"node.email = "courier" e.g. "smtp" but more typically specified as a table of attributes as follownode.email = { courier="courier" }node.email.message = default attributes for all messages and couriers{to="mailbox@example.com", ‡in dev mode this will capture all outgoing messages and send them to this address instead of any specified recipients
} node.email.courier = {}
After dispatch message.status will provide the SMTP status code, and message.error where applicable.
server="domain", optional, e.g. "localhost" or "example.net"; may be set to false on messages to override a node default and use direct dispatch
sender="mailer-daemon@example.net", return address, defaults to moonstalk@hostname; should be a mailbox that can receive bounces
username="login", optional
password="secret", optional
bind_ip="127.0.0.1", optional, the outgoing IP address for use when connecting to relay or recipient servers
†When no server is specified the message will be dispatched directly to recipient mailservers. This is likely to result in the message being marked as forged, unless you provide a valid sender with reverse DNS for the dispatching server, plus configure SPF as as required.
Direct dispatch has the advantage of providing immediate delivery with confirmation of address validity, rather than being delayed through an intermediary that returns failed messages by email. A successful status with direct delivery can thus be considered as having placed the message in the recipient's mailbox, however most receiving systems use filters and may themselves relay to final mailbox servers so this cannot be assumed as immediate, but is nonetheless faster and more programmatic than using an SMTP relay server.
Loads bundles as sites from the sites folder, and optional sites folders within applications.
site.domain (the primary domain).
Any attributes of a site bundle may be specified in the site's settings.lua file, however the following are specific to the settings file, providing simplified syntax. All attributes and their values defined in the site settings file (including your own) are accessible in the site table with each request.
domains = { "example.net", "wibble.example.com", "example.org" }applications = { "manager", }autoaddresses = truefalse.
redirect = truedomains, or for the 'www' variant of your primary domain if none) — to your primary domain. I.e. if your primary domain is example.com then www.example.com will be redirected to that, or vice-versa. If you specify false, domain redirection is disabled and all domains function as aliases for the same site content. A canonical link is also added to pages on any non-primary domain, referencing your primary domain.
redirect = false, references to assets must currently use absolute URLs to a site's primary domain only. (Use absolute references, or enable the built-in CDN remapping.)This is the core application which provides handling for the page generation cycle, including outside a request-response flow.
An interesting side effect of compiling views as first-class Lua functions is that you can declare blocks of HTML as local functions themselves, essentially acting as embedded content block templates. Thus each time you call the function it will render the same layout with different values, however you reference them (arguments, upvalues, or globals). The follow example demonstrates.
table.insert(moonstalk.readers, function(view) end) -- declare a function that may conditionally parse a static view returning a modified content block e.g. for markdown or other formats, or simply to modify all content blocks, such as an alternate templating system; all readers run sequentially in an order determined only by how they're declared ><html>1
2
3
</html>
Provides helper functionality for working with HTML pages, loading and running JavaScript, and facilitating the use of third-party integrations such as Google Analytics.
Kit utilises an editor() and thus its page modifcation functions are retroactive, and may be called at any time (e.g. before any view has even provided page content to modify).
page.language (if any).
src attributes are made absolute. The root may be specified (such as to use a CDN) with node.base, or this may be set to false to disable this behaviour entirely. This feature enables you to develop views locally with relative asset references, and they will be changed automatically upon deployment.
page.javascript.varName = valuekit.Script ( script )script.Load "/assets/script.js", an assignment e.g. script.Load "myvar = 'foo'", or a function to be called after loading e.g. script.Load "myfunc()". You may delcare dependencies to be loaded simultaneously as supplementary parameters, e.g. script.Load ("/moonstalk.kit/jquery.js", "/assets/script.js", "myfunc()") or sequential dependencies as a single string using greater than as a seperator, e.g. script.Load ("/moonstalk.kit/jquery.js > /assets/script.js", "myfunc()"). Expressions should not be terminated with semi-colon.
site.services.analytics = "id"site.services.reinvigorate = "id"kit.Head ( tag )nocache = truerobots = "none"Kit includes a number of useful JavaScript libraries.
For any key in the page.flags table corresponding to a form input name created using a tag function, it's class is set to error, and if the value of the key is a string an error message is displayed in a span with the id error. The tag functions should be used in server tags not expression tags.
tag.select ( "name", list, selected, localised )form select input with the specified name, and an option corresponding to each item in the list—having integer values and display names from the list item value, either as a localised string name, or (if localised is false) the value itself. Specify selected using or syntax to provide a default e.g. request.form.name or 2.
tag.checkbox ( "name",selected )form checkbox input with the specified name, and checked as per selected, which may be specified using or syntax to provide a default e.g. request.form.name or true.
tag.input ( "name", value ) or tag.text { name="name", value=value, attribute="value", ... }form text input with the specified name and value, or the attributes and their values specified in a table parameter. Specify value using or syntax to provide a default e.g. request.form.name or "value".
If no value parameter is specified it will default to request.form[name]
page.focusfield = "name"tag.Error ("name", "message")tag functions.
format.Date ( datetime, format, toggle )format.ReferenceDate may be used to output a reftime without conversion to site localtime.
format.Time ( datetime, format )format.ReferenceTime may be used to output a reftime without conversion.
tag.time ( datetime )format.Number ( number, decimals, locale )user.locale.
format.Money ( number, decimals, locale )user.locale. If no number is provided, displays n/a.
format.TelPrefix ( number )user.locale.
captcha.Generate ()captcha.Validate ()The geo application provides geographic reference and manipulation functionality, including RAM cached (<1ms) GeoIP user location lookups, using free country and city level databases. In future this application will also provide GeoNames data.
Providing the data files are available, the application is enabled by default to provide a country for the user locale in cases where a request language is not country-specific. To disable specify geo=false in the Node settings, to force lookups for all requests, specify geo=true or geo={lookup=true}.
You may enable country-level resolution for all requests by specifying geo={countries={'cc'}}, and/or city-level resolution by specifying geo={cities={'cc'}} in the same manner, an empty table will load all available data. At the time of writing per-scribe RAM use is 10MB for country level data for ambigious languages, 40MB for UK city-level data, and 250MB RAM for the US.
geo = nilgeo = falsegeo = trueThe following unpacked third-party files must be placed in the data/library/geoip folder.
request.client.place table is defined.
geo.LocateIp ('ip.ad.dr.es')
Perform a lookup for the specified IP address string. Also defines the user.place table.
geo.Hash (latitude,longitude)
Create a geohash from the defined coordinate numbers or strings.
geo.Coordinates ('geohash')
Returns latitude, longitude numbers for the specified geohash string.
The manager application provides an adminstration interface for the Moonstalk servers, extensible by other applications. Requires the User application.
localhost site.node.secretoperator
operator key.
Enables the retrieval of localisable pages (content objects) from a database (the Teller), and extends the Manager application with functionality to manage these pages. Requires the Tenants application.
Page tables are composed of keys referred to as pieces, each having any valid value(s), including localised tables. A localised table is a table containing a key for each language identifier and a table or string value, but should also contain _localised=true. When a piece is requested corresponding to a localised table, only the value matching the user's preferred language or site language, is returned. If no language matched we also set _localised=false in the response. If the localised table does not contain a _localised flag the corresponding piece name should be specified as "name[localised]".
For site folders only, to avoid disassociation of data in the case of a folder name change, it is desirable to provide an ID that is unique and permanent. Define one to the site settings.
id = guuid
page.pieces = {"piece"}collators={content.PageCollator} to be specified.page.content_urn = "pattern"Pieces ("urn", {"piece"}){"meta", "vocabulary", "title", "body"}, however you may specify true to retrieve all (i.e. the same as the page table itself, but without any redundant languages). The following attributes are always returned: created, modified, controller, view, template, css.NewPageDelegate (page, page_id, site, site_id)
This application virtualises sites, storing their settings and content in the Teller database and extends the Manager application, where settings may be managed.
Primarily intended to facilitate developing SaaS applications where users have individual domains, it also enables support for the Content application (or any other per-site/per-user content in a database) with disk-based sites.
Tenant (virtual) sites have similar functionality as disk-based site folders. They can have per-site application settings and content (using the Content application) but they cannot define addresses (beyond those supported by the Content application), controllers, views, or databases. Nor may they use variables in content, or disable/enable applications.
All virtual sites use a foundation site (node.sites.tenants) with which a virtual site is merged, enabling a configured set of applications for every tenant site. All CoreAPI site attributes are valid within a virtual site and will replace those defined in the foundation site, and it is therefore necessary to ensure arbitrary attributes cannot be specified for virtual sites from any management interface exposed to a tenant.
Specify the following in Node.lua to customise the generic/unknown view.
provider = { domain="example.com", name="Example SaaS", }
Specify a site (loaded elsewhere, e.g. from disk using Sites Folder) to be used instead using tenant in Node.lua.
tenant = { applications = {}, template = "name", subdomain = "", }
node.curators={"sitesfolder","matchdomains","tenant"}.event.sites (site, site_id)event.domains (domain, site, site_id)request.client.keychain.
people.DisplayName (user)
Concatenates the first and lastname for output, either for the specified user, or the current user if none.people.SplitName ("name")
Returns firstname, lastname for a given user-input name string where the latter may be nil.people.New {name=name, email="address", telephone="number", }
Returns a new user object with ID. Accepts any valid keys from the user table, in addition to the above convience keys.event.user (user, user_id)
event.email (email, user, user_id)
Adds support to sites for wildcarded domains, such as to accept requests for sites on a multi-tenanted platform hosted with subdomains. To enable add the following to node/Host.lua.
node.curators = {"matchdomains",}
For a domain in the domains list of settings.lua, specify their names with the * wildcard character (asterisk) in one of the following manners. Matching is not supported for primary domains (site folder name).
Simply specify a period as the first character of the name.
domains = {"*.example.com",}domains = {"prefix.*.example.com",}With OpenResty, any domain containing a wildcard (as opposed starting with) is not declared in NGINX with its own server block, therefore cannot be used to access files, only Moonstalk pages. Its assets should be hosted by an application or another domain.
This application functions simply by parsing all site domains upon startup for a valid pattern, building an array of these domains, and providing a curator function which is utilised to return a corresponding site (if any) for each request.
The Tenants application already supports multi-tenant sub-domains for users via its database, however both applications may still be used together but node.curators should be specified such that matchdomains appears before tenant e.g. node.curators = {"matchdomains", "tenant"}.
Manages the Nginx webserver under the OpenResty framework enabling Lua functions through its lua-nginx module that provides native and asynchronous Lua execution using the LuaJIT interpreter and Lua coroutines hooked into the Nginx event architecture.
Async functions must be called with the Moonstalk coroutine handler moonstalk.Resume(ngx.req.read_body) else the request table can become invalid following the async call. Where provided, instead use Moonstalk's abstractions such as openresty.GetPost(). -- TODO: wrap all the async functions with resume by default?
Not compatible with the official NGINX package and must use the patched version provided with the OpenResty package. If you have the official version you must rename or remove it.
cd /usr/local/moonstalk; certbot certonly --webroot -w sites/example.com/ -d example.com (for as many sites as needed) and make sure to add a cron job for renewals certbot renew --deploy-hook "/usr/local/elevator restart web" so when the certificates renew there will be no downtime; note that the primary site.domain must match the first domain for the certificate.
util.Shell() is asynchronous.
post attribute on an address.?wibble&wobble=wubble as booleans e.g. request.query.wibble==true. However the Moonstalk app, also parses the first as request.query[1]=="wibble" but does not support more than one. It's worth remembering that some platforms where referring traffic append ?clid=token to URLs, therefore if the URL was originally ?wibble+wubble it would become ?wibble+wubble&clid=token
Must never use the same file with multiple instances or servers. This mechanism is for exclusive use by a single process, per file and does not provide a shared database.
Provides simple in-memory table persistence suitable for low-risk or flow-frequency updates. Uses the efficient Luabins binary serialisation library.
Database tables may be specified in an application's schema.lua and these are then available as db.name.
Not suitable for very large datasets due to save being a blocking process that will take longer the larger the dataset, thus generally unsuitable for on-demand use. With the primary save only upon shutdown there is a risk of data loss unless carried out explicitly and bearing in mind its cost.
databin.Load("file") reads the named file, returning its deserialised tabledatabin.Reoad("file") reloads data for the named database tabledatabin.Save("file", table) saves the named file with the given table serialised, or if no table is provided saves the corresponding database table; fails if no data is provideddatabin.Cache{…}; returns a cache table serialised to disk; provides a convenience wrapper for occasionally generated in-memory caches; see functions.lua comments for usagesystem = "databin"autoload = {server=true, …)autosave = "server"autoreload = {server=false, …}loaded = "bundle.function"Manages database processes for the Tarantool system, through the standard moonstalk schema.lua configuration file, with automatic support for roles and table replication across hosts having the same roles enabled.
If not running Tarantool as root, .
Always shutdown the Tarantool server after the Scribe backends to allow application writes to complete, or use a pool of servers. You will require bespoke orchestration logic with multiple servers.
msgpack.NULL for empty positions such as when deleted, instead of nil.
To enable a node as a Tarantool client you must add node.tarantool.role_name = {host="address",port=number,password="secret"} must be configured; password is optional when the server is on the same node and configured by Moonstalk, and port is not required if a unix domain socket path (absolute) which is the default on the same node.
To enable the Tarantool server you must add "tarantool" to node.servers and optionally tarantool={role="role_name"}. Its password will be the node secret with username moonstalk. The server will handle only tables having a corresponding role_name, by default "world".
To define a new database table (as opposed extend another application's tables, such as users or tenants), declare it in an application's schema.lua file.
The recommended naming is a plural for a table and the singular form for individual record models e.g. users = { record="user", …}.
table_name = {
system="tarantool",[1]="first_field", [2]="second_field", … declare the fieldnames with their Tarantool tuple positions as keysindexes = { {name="id", type="hash", parts={1,"unsigned"}}, …}, these declaration use native Tarantool values and reference the fields by their above declared tuple positionsrecord = "record_name", declaring a record name allows use of the model functionalityrole = "role_name", when used in clusters, tables will correspond to the corresponding node}Tarantool stores records as tuples (arrays) of field values, therefore the order (position) of fields cannot be changed once initialised nor can fields be removed, new fields may be added to make the tuple longer. Field names may be changed (considering that existing values may need manually removing or normalising), and must be changed everywhere referenced.
model table provides enumeration of fieldsmodel.record.field gives its tuple indexmodel.record[index] gives its field name (table key)In Tarantool you can reference fields by name e.g. tuple.field however this has a cost and in high-use scenarios it is cheaper to use tuple[model.record.field].
These are called in your pages (views, controllers) to access the database from Scribe backends.
tarantool.default("appname.FunctionName", argument, …) call a function in the Tarantool instance named default. Functions are declared in application/tarantool.luadb.record[index_key] fetch a recorddb.record[index_key] = {…} fetch value as either a Tarantool tuple query result (Lua array), or a Lua table in record form to convert between themlocal record = db.record[index_key]; record:Save()application.Procedure() native call executing the named function in the Tarantool database process; these functions are declared in the file tarantool.luamodel.record(value) converts between a Lua table and Tarantool tuple matching the model's fields; handles nil fieldsRecords are Lua tables with schema field names. Null values from Tarantool are nil. When retrieved from db.record they can be modified, and even have non-schema keys added for ephemeral use, as when saved they will be discarded.
Records
Applications may define database functions that run in the Teller (procedures), and thus have unrestricted access to the database. This provides the ability to work with records, such as to iterate over all records (e.g. for search) without the overhead of fetching [all] values into a page first. You call and define these functions as you would a standard function, but you create them in the include/tarantool.lua file.
application_name.Procedure_name() native call executing the function in the database; returns nil,err on database and connection failures, therefore ensure internal error states return false,err if differentiation is required; must return true if no other valueDatabase functions are not suitable for long-running complex queries as they block other queries unless you use the Taranool yield behaviour.
Working with hierarchical data tables (nested hashmaps and arrays) contrasts with column and row-based data tables. Consider this when designing your data structures, and optimise for retrieving tables and subtables, instead of individual values.
The core web server framework, handling the request-response flow.
Developing applications that modify the Scribe can have unintended consequences
TODO: flow diagram with hooksscribe.AddLoader(myapp, "type", function(data) return data end) myapp is the bundle (global namespace), type is the file extension ("lua" for controller, "html" for views) and function is a pointer to a function that will receive the file data, and optionally modify and return it before it is compiled, or set as contentapplication.Site(site) called with a site table when a new one is added, permitting the application to configure it or derive data from it; generally only applies with site bundlesapplication.Curator()/path/?≈=abD34Hj45ghHy343J7K12 (this also works in a mixed attribute query string), invoking an authenticator thus permitting sign-in links to function with any URL. The authenticator is responsible for identifying the type of token, determining its privileges and running corresponding mechanisms.This manual is a pre-release preview. Please follow Moonstalk on Twitter or Facebook to be notified of a public release.
Moonstalk is not ‘inspired’ by any other particular framework, instead its inception was driven by the desire for clean and flexible markup in views, with transparent database handling, and a preference to use RAM and avoid disk access as much as possible.
Its methodology does build upon the convention-over-configuration approach, extensively using tables from which values may be pulled (including the database, e.g. <title>?(db.record.title)</title>), and in which behaviours can be modified from their defaults (e.g. page.status=404).
It avoids object-oriented programming (just as Lua does out-the-box), having no classes nor :method() calls (with some minor exceptions) instead favouring function calls that pass tables with typically optional keys (e.g. Function{arg1=value,arg2=value}).
Site and application bundles (folders with standardised file naming conventions) are used to simplify file management. Applications extend the core with functionality for specific usage scenarios, attempting to avoid including functionality by default that is not optimal for any scenario.
Enable the development and programming of sites and applications to serve dynamic web content, composed of both discrete and bundled layouts (design), logic (function) and assets, with an extensible set of functions.
Domain-specific (site) and cross-domain (application) bundles, comprising functions for acting upon and generating responses, invoked and configured through addresses and hook functions. Content–logic isolation is employed with a faceted view–controller model that supports templating, wherein responses are the merger of one or more sections.
All functions and content are loaded and compiled once for caching in RAM, but may be reloaded by restarting multiple backends sequentially for zero-downtime upgrades and updates. This also means you can declare functions anywhere as tey are already available and compiled such as when views are requested. In future views and controllers will use change subscriptions to reload upon modification. As similar behaviour is available in developer mode as timestamps are checked with each request to facilitate development.
Visitor analytics (logging) is considered best handled by third-party services using JavaScript, therefore Moonstalk is typically run with minimal logging for security auditing, performance optimisation and troubleshooting purposes only. If detailed logging is required, it may be enabled in the web server.
Static and dynamic pages without database queries are typically executed in about 1 millisecond. Pages with database queries typically execute in 1–5ms.
Handle HTTP requests for dynamic and static website content, across multiple sites (domains), and shared applications.
A separate web server manages HTTP connections and serves static content directly, whilst passing requests for dynamic content amongst separate backend daemons running as individual threads (Nginx workers) or FastCGI processes. These are pre-configured with applications and sites (using the framework), and can maintain an always-open connections to a database.
Moonstalk's focus is on serving web applications. Static content is considered best served from a CDN, not from the same server as a webapp. Therefore static and dynamic content are considered separately. However Moonstalk configures the webserver to serve both, and thus can be used for low-use static assets and CDN origin-pull without unduly reducing capacity for serving web applications.
The backends preload and cache the sites (settings, views, controllers and applications) compiling functions, keeping everything in RAM, thus vastly reducing processing times as compared loading from disk with every request. Running under a LuaJIT process permits dynamic code deployments whilst offering tight integration with higher performance C libraries. In future this should be extended with framework functionality similar to Erlang-OTP.
Dynamic webpages typically comprise many static string segments (html markup) interspersed with a few dynamic variables (from database records), allocating and concatenating these with every request can be expensive. As Lua interns all strings we gain the advantage of keeping the regularly reused markup strings whilst only allocating and garbage collecting the variables.
Lua is not however efficient at concatenation thus during assembly of the response Moonstalk maintains an array of the individual strings, concatenating them upon completion. Whilst Moonstalk implements a default of content-then-template rendering, this approach permits retrospective changes to already assembled content and thus either forward or reverse rendering/chaining. (See the mark, cut and paste functions.)
With Nginx workers…
With FastCGI…
With both Scribe backends and Teller database with a typical web application, 165 requests per Gigahertz can be handled (i.e per dedicated 'CPU' second on a single CPU core). On a quad core 2.5Ghz CPU, the engines can thus serve 4,500 requests per second.
It's not yet apparent if there are scheduling (inter-process) bottlenecks that might prevent increased capacity on hardware with better performance without further optimisation.
This system is no longer being maintained and a scalable DBMS is recommended, such as Tarantool which is now supported natively.
Provides data storage (persistence) and retrieval functions to Moonstalk applications and sites. Implemented as an optional [application] module.
The database is a server daemon functioning as a shared Lua environment, operating for local clients (Scribe pages backends) over UNIX Domain Sockets. Persisted data is collated and written to an SQLite database on a dedicated thread. Additionally in a cluster configuration where replication is enabled, the database will maintain network connections with its peer nodes for synchronisation.
The database is intended for the retrieval of tables containing multiple name-value pairs in a single transaction, as opposed making multiple transactions for keys holding individual values, thus reducing overhead. Both as documents (tables) or aggregated results from Map-Reduce style function operations.
The database server keeps a dedicated connection open for each backend, executing queries sequentially on its main thread, whilst using separate threads for persistence and replication. Queries are thus not delayed by disk writes, and/or network issues resultant from the execution of preceding queries.
The persistence and replication functions batch changes together at a configurable interval to avoid resource (i.e. disk or network) saturation, and the overhead of many small transactions with commonly-changed values (e.g. counters). Typically the interval is low enough that in the case of a crash or unexpected power-off event, only a few seconds of non-critical data could ever be lost.
--TODO: Critical writes can be saved synchronously (immediately) and without delaying pending queries, through the use of a spawned thread that exits once either persistence or replication has completed. Likewise critical reads, except that they exit once a required number of nodes have responded (e.g. the first). Note however that the response to critical queries may be delayed by subsequent queries (synchronous responses are not supported).
Procedures are native Lua functions that execute on the main thread and thus block all subsequent queries. Procedures should therefore define structural actions that execute swiftly, unless they spawn a thread to complete their task (in which case no return value is possible), or utilise the functionality provided by the Tasks application to provide a result via polling.
Indexes allow for faster alternate database table lookups but introduce additional complexity in the maintenance of table structures and ownership by applications. Indexes are not saved to reduce persistence tasks, but [re]built on startup, and should be maintained on a per-query basis using delegated event handlers. See the Teller section for details.
Saved data accessibility is provided by using the open SQLite database file format. Individual values are saved in this under their unique combined table key namespace, and using flags to preserve data types. This data is read into the database's Lua environment upon starting.
Transaction times depend upon 3 factors—the size of the response, the complexity of the query (negligible for most fetch lookups), and the queue of pending queries.
This does not consider outstanding requests being queued for distribution amongst page backends by the web server, when under high traffic.
For a single fetch transaction, inclusive of overheads, a nil or simple chunk of text/HTML typically returns in 150 microseconds (0.15 milliseconds), whilst a table containing a few dozen nested keys returns in 200–300 microseconds. Lookups for non-numeric keys, within nested tables, and within very large tables introduce a little additional overhead.
This results in peak performance of 15,000 operations per second per Gigahertz (i.e per dedicated 'CPU' second on a single CPU core). Pages backends carrying out this many operations would probably be using at least the same amount of CPU time, therefore you would get half this performance per core, unless the database has a dedicated core. Realistically a page request would never make more than a few database requests, and currently 10,000+ requests per second is perfectly feasible. Performance for complex queries (functions) obviously varies according to the scope of the data. For higher traffic sites a multi-core 2Ghz+ processor is essential.
RAM requirements are approximately double that of disk storage requirements, i.e. for 5 MB of on-disk data (about 40,000 records of 200 characters, or 6,000 blog posts), the database will require 10 MB of RAM. Bear in mind that the non-persistent page cache functionality will require additional memory for every non-authenticated URL's HTML content.
Moonstalk's development is supported by The Moon Mill, and is released as open-source under the terms of the Artistic Licence 2.0. (Note that the licence will be changed to GPL once a roadmap has been established.)
Moonstalk was conceived in mid-2010 during the development of a SaaS e-commerce platform, that had been designed to take advantage of the Lua environment to provide very low-latency database-driven page generation. It's architecture and (potentially dubious!) programming has been undertaken by Jacob Jay (@jacobjay) almost continuously since then, in both India and France.
LuaRocks is recommended for the management and installation of dependancies, and the following modules are shown with their LuaRocks names.
Moonstalk builds its framework upon core libraries developed by the Kepler Project.